docs.md

# DESI Spectra Zoo – Technical Documentation

## Architecture

DESI Spectra Zoo is an expert annotation tool built with Dash/Flask. Users see one spectrum at a time and classify it using configurable category buttons (Galaxy, Star, QSO, Interesting, Bad Data). Annotations persist to HuggingFace Hub.

## Data Flow

1. **Pool initialization** (`spectrum_data.sample_pool_streaming`): selects spectra from the DESI dataset using a fixed seed, optionally filtered by redshift/S\N
2. **Pre-fetch**: fetches 4 spectra synchronously via HF datasets streaming, then starts a background thread for the rest
3. **Plot generation** (`spectrum_data._generate_plot`): dark-themed matplotlib plots with spectral line annotations (Hα, Hβ, [OIII], etc.), cached as PNG on disk
4. **Spectrum selection** (`annotations.select_next_spectrum`): picks a random un-annotated spectrum, prefers cached plots
5. **Annotation**: click triggers `annotations.record_annotation` → saves state → logs event via `CommitScheduler`

## Configuration

All annotation categories and spectrum filters are defined in `dataset_config.yaml`.

## Plot Generation

- Backend: matplotlib Agg
- Background: `#0a0a1e`
- Spectrum line: `#4fc3f7` (cyan)
- Spectral lines annotated at observed wavelengths (rest × (1+z))
- S/N ratio displayed in corner
- Smoothing: configurable via `PLOT_SMOOTH` (default: 3-pixel boxcar)
- Output: 18×6 figure at 200 DPI, PNG

## Annotation State

- State persisted to `state/annotations.json` locally, synced to HF Hub via `CommitScheduler`
- Each annotation records: spectrum_index, category, session_id, timestamp