DEPLOY.md

Deploying MLX Model Explorer

The Space and dataset are created private, verified end to end, and only then made public.

Prerequisites: hf auth login as a member of mlx-community with write access, plus the project venv (uv pip install -r requirements.txt).

1. Verify locally

.venv/bin/python -m pytest
EXPLORER_SINK=local EXPLORER_FLUSH_SECONDS=20 .venv/bin/uvicorn app.main:app --port 7860
# walk through the UI; events land in .runtime/local_dataset/data/events/
docker build --platform linux/amd64 -t mlx-model-explorer . \
  && docker run --rm -p 7860:7860 -e EXPLORER_SINK=local mlx-model-explorer

2. Create private repos

.venv/bin/python scripts/deploy.py status
.venv/bin/python scripts/deploy.py create      # refuses if the Space exists; add --allow-existing-dataset if the dataset was created earlier

This creates mlx-community/mlx-model-explorer-data (private dataset, card uploaded) and codelion/mlx-model-explorer (private Docker Space with EXPLORER_SINK=hub, DATASET_REPO, EXPLORER_FLUSH_SECONDS=600).

Why the Space isn't in mlx-community: Hugging Face now requires a Team or Enterprise plan for an organization to run Docker or Gradio Spaces on free CPU (402 Payment Required). The dataset can live in the org. Once an org admin has a plan or a hardware grant, deploy there with SPACE_REPO=mlx-community/mlx-model-explorer scripts/deploy.py create --allow-existing-dataset followed by upload.

3. Dataset write token

Create a fine-grained token at https://huggingface.co/settings/tokens with write permission on mlx-community/mlx-model-explorer-data only. Don't use a personal all-access token: anything in a Space secret is available to the running app.

HF_DATASET_WRITE_TOKEN=hf_xxx .venv/bin/python scripts/deploy.py secret

4. Test the ingestion path against the private dataset from this machine

EXPLORER_SINK=hub EXPLORER_DATA_PREFIX=data/_test EXPLORER_FLUSH_SECONDS=30 \
  HF_TOKEN=$(cat ~/.cache/huggingface/token) .venv/bin/uvicorn app.main:app --port 7860

5. Upload and test the private Space

.venv/bin/python scripts/deploy.py variable EXPLORER_DATA_PREFIX data/_test   # keep test events separate
.venv/bin/python scripts/deploy.py upload
.venv/bin/python scripts/deploy.py status                                     # wait for RUNNING

Open https://huggingface.co/spaces/codelion/mlx-model-explorer while logged in. Run the full flow, then confirm a shard appears under data/_test/ in the dataset (after the flush interval, or restart the Space to force a flush).

To submit a benchmark to the private Space, use its direct URL and a token:

python bench/mlx_explorer_bench.py --model mlx-community/Qwen3-0.6B-4bit \
  --submit https://codelion-mlx-model-explorer.hf.space --hf-token $(cat ~/.cache/huggingface/token)

6. Go public (only after review)

.venv/bin/python scripts/deploy.py variable EXPLORER_DATA_PREFIX data/events
.venv/bin/python scripts/deploy.py clean-test-data
.venv/bin/python scripts/deploy.py publish --yes-make-public

publish refuses while data/_test/ still exists.

Updating

Change code, run the tests, docker build locally, then scripts/deploy.py upload -m "what changed". Events are append-only, so redeploys never touch existing data. On shutdown the app flushes buffered events, and anything that fails to upload is retried from the on-disk spool.

Kill switch

scripts/deploy.py variable EXPLORER_SINK off stops all collection. The UI keeps working and the event endpoint accepts nothing.