Spaces:
Sleeping
Sleeping
File size: 3,011 Bytes
00d1277 009ddf2 00d1277 942f8ca 00d1277 942f8ca | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 | ---
title: DESI Spectra Zoo
emoji: 🧬
colorFrom: yellow
colorTo: red
sdk: docker
app_file: app.py
pinned: false
short_description: Expert annotation of DESI spectra
---
# DESI Spectra Zoo
Classify DESI spectra one-by-one to build a Gold dataset of labelled astronomical objects (Galaxy, Star, QSO, Interesting, Bad Data).
Built on [MultimodalUniverse/desi](https://huggingface.co/datasets/MultimodalUniverse/desi) (~100K spectra).
## Environment variables
### Local development
Create a `.env` file in the project root (never commit this):
```
# HuggingFace — required for persistent annotation state across restarts
HF_TOKEN=hf_...
HF_LOG_REPO_ID=your-username/desi-spectra-zoo-logs
# Optional
HF_LOG_EVERY_MINUTES=10
# Dataset
DATASET_ID=MultimodalUniverse/desi
DATASET_CONFIG=default
DATASET_SPLIT=train
ID_COLUMN=object_id
SPECTRUM_COLUMN=spectrum
POOL_SIZE=500
POOL_SEED=42
# Plot cache
PLOT_CACHE_DIR=cache/plots
PLOT_SMOOTH=3
```
Then run:
```bash
python app.py
```
The app serves at `http://localhost:7860`.
### HuggingFace Spaces
Set variables in **Settings → Variables and secrets** for your Space:
| Variable | Type | Description |
|---|---|---|
| `HF_TOKEN` | Secret | HuggingFace token with read/write access to `HF_LOG_REPO_ID` |
| `HF_LOG_REPO_ID` | Secret | Dataset repo for persisting annotation state, e.g. `your-username/desi-spectra-zoo-logs` |
| `HF_LOG_EVERY_MINUTES` | Variable | How often to sync state to HF Hub (default: `10`) |
| `DATASET_ID` | Variable | HF dataset to load spectra from (default: `MultimodalUniverse/desi`) |
| `DATASET_CONFIG` | Variable | Dataset config name (default: `default`) |
| `DATASET_SPLIT` | Variable | Dataset split (default: `train`) |
| `ID_COLUMN` | Variable | Name of the ID column (default: `object_id`) |
| `SPECTRUM_COLUMN` | Variable | Name of the spectrum column (default: `spectrum`) |
| `POOL_SIZE` | Variable | Number of spectra to pre-load (default: `500`) |
| `POOL_SEED` | Variable | Fixed seed so all annotators see the same pool (default: `42`) |
| `PLOT_SMOOTH` | Variable | Smoothing kernel size for spectrum plots (default: `3`) |
> **Note**: `HF_TOKEN` must be added as a **Secret** (not a variable) to keep it private.
### Persistent annotation state
Annotations are written to `state/annotations.json` on every classification and periodically synced to HF Hub via `CommitScheduler`. Annotation events are also logged as JSONL in `data/`.
Without `HF_LOG_REPO_ID` set, state is only saved locally and will be lost on container restart.
## How it works
1. On startup, metadata for the first 500 spectra is streamed (fast, no spectrum data).
2. The first spectrum plot is generated synchronously — app becomes available.
3. In the background: the full 100K metadata index loads, then remaining pool plots are generated.
4. Filters (redshift, E(B-V), ZWARN) operate on the full index once loaded.
5. When filters select spectra outside the initial pool, their data is fetched on-demand.
## License
MIT
|