---
title: DESI Spectra Zoo
emoji: 🧬
colorFrom: yellow
colorTo: red
sdk: docker
app_file: app.py
pinned: false
short_description: Expert annotation of DESI spectra
---

# DESI Spectra Zoo

Classify DESI spectra one-by-one to build a Gold dataset of labelled astronomical objects (Galaxy, Star, QSO, Interesting, Bad Data).

Built on [MultimodalUniverse/desi](https://huggingface.co/datasets/MultimodalUniverse/desi) (~100K spectra).

## Environment variables

### Local development

Create a `.env` file in the project root (never commit this):

```
# HuggingFace — required for persistent annotation state across restarts
HF_TOKEN=hf_...
HF_LOG_REPO_ID=your-username/desi-spectra-zoo-logs

# Optional
HF_LOG_EVERY_MINUTES=10

# Dataset
DATASET_ID=MultimodalUniverse/desi
DATASET_CONFIG=default
DATASET_SPLIT=train
ID_COLUMN=object_id
SPECTRUM_COLUMN=spectrum
POOL_SIZE=500
POOL_SEED=42

# Plot cache
PLOT_CACHE_DIR=cache/plots
PLOT_SMOOTH=3
```

Then run:

```bash
python app.py
```

The app serves at `http://localhost:7860`.

### HuggingFace Spaces

Set variables in **Settings → Variables and secrets** for your Space:

| Variable | Type | Description |
|---|---|---|
| `HF_TOKEN` | Secret | HuggingFace token with read/write access to `HF_LOG_REPO_ID` |
| `HF_LOG_REPO_ID` | Secret | Dataset repo for persisting annotation state, e.g. `your-username/desi-spectra-zoo-logs` |
| `HF_LOG_EVERY_MINUTES` | Variable | How often to sync state to HF Hub (default: `10`) |
| `DATASET_ID` | Variable | HF dataset to load spectra from (default: `MultimodalUniverse/desi`) |
| `DATASET_CONFIG` | Variable | Dataset config name (default: `default`) |
| `DATASET_SPLIT` | Variable | Dataset split (default: `train`) |
| `ID_COLUMN` | Variable | Name of the ID column (default: `object_id`) |
| `SPECTRUM_COLUMN` | Variable | Name of the spectrum column (default: `spectrum`) |
| `POOL_SIZE` | Variable | Number of spectra to pre-load (default: `500`) |
| `POOL_SEED` | Variable | Fixed seed so all annotators see the same pool (default: `42`) |
| `PLOT_SMOOTH` | Variable | Smoothing kernel size for spectrum plots (default: `3`) |

> **Note**: `HF_TOKEN` must be added as a **Secret** (not a variable) to keep it private.

### Persistent annotation state

Annotations are written to `state/annotations.json` on every classification and periodically synced to HF Hub via `CommitScheduler`. Annotation events are also logged as JSONL in `data/`.

Without `HF_LOG_REPO_ID` set, state is only saved locally and will be lost on container restart.

## How it works

1. On startup, metadata for the first 500 spectra is streamed (fast, no spectrum data).
2. The first spectrum plot is generated synchronously — app becomes available.
3. In the background: the full 100K metadata index loads, then remaining pool plots are generated.
4. Filters (redshift, E(B-V), ZWARN) operate on the full index once loaded.
5. When filters select spectra outside the initial pool, their data is fetched on-demand.

## License

MIT