docs.md

DESI Spectra Zoo – Technical Documentation

Architecture

DESI Spectra Zoo is an expert annotation tool built with Dash/Flask. Users see one spectrum at a time and classify it using configurable category buttons (Galaxy, Star, QSO, Interesting, Bad Data). Annotations persist to HuggingFace Hub.

Data Flow

1. Pool initialization (spectrum_data.sample_pool_streaming): selects spectra from the DESI dataset using a fixed seed, optionally filtered by redshift/S\N
2. Pre-fetch: fetches 4 spectra synchronously via HF datasets streaming, then starts a background thread for the rest
3. Plot generation (spectrum_data._generate_plot): dark-themed matplotlib plots with spectral line annotations (Hα, Hβ, [OIII], etc.), cached as PNG on disk
4. Spectrum selection (annotations.select_next_spectrum): picks a random un-annotated spectrum, prefers cached plots
5. Annotation: click triggers annotations.record_annotation → saves state → logs event via CommitScheduler

Configuration

All annotation categories and spectrum filters are defined in dataset_config.yaml.

Plot Generation

• Backend: matplotlib Agg
• Background: #0a0a1e
• Spectrum line: #4fc3f7 (cyan)
• Spectral lines annotated at observed wavelengths (rest × (1+z))
• S/N ratio displayed in corner
• Smoothing: configurable via PLOT_SMOOTH (default: 3-pixel boxcar)
• Output: 18×6 figure at 200 DPI, PNG

Annotation State

• State persisted to state/annotations.json locally, synced to HF Hub via CommitScheduler
• Each annotation records: spectrum_index, category, session_id, timestamp