scythe410 commited on
Commit
7c265b7
Β·
1 Parent(s): 7b46e60

chore(reducer): dockerize for hugging face spaces deploy

Browse files

Vercel functions can't reach the local FastAPI reducer, so package
services/reducer/ as a Docker Space image: uid-1000 user, port 7860,
pre-warmed MiniLM model baked into the image so cold starts return in
seconds. README carries the Space frontmatter and the env-var manifest
operators must set as Space secrets.

Files changed (3) hide show
  1. .dockerignore +12 -0
  2. Dockerfile +54 -0
  3. README.md +45 -0
.dockerignore ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ __pycache__/
2
+ *.pyc
3
+ *.pyo
4
+ .pytest_cache/
5
+ .ruff_cache/
6
+ .venv/
7
+ tests/
8
+ samples/
9
+ *.md
10
+ !README.md
11
+ .env
12
+ .env.*
Dockerfile ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # VectorScape reducer β€” Hugging Face Spaces (Docker SDK) image.
2
+ #
3
+ # HF Spaces requirements baked in:
4
+ # - Runs as uid 1000 ("user"); $HOME must be writable.
5
+ # - App must bind to 0.0.0.0:7860.
6
+ # - sentence-transformers / huggingface caches live under $HOME and are
7
+ # populated at build time so the first request doesn't pay the model
8
+ # download cost (~90 MB for all-MiniLM-L6-v2).
9
+ FROM python:3.11-slim
10
+
11
+ # build-essential is kept because pacmap / hdbscan can fall back to a Cython
12
+ # build when a manylinux wheel isn't published for the resolved version. git
13
+ # is occasionally needed for pip VCS deps. Both can be dropped later if the
14
+ # resolved lock turns out to be pure-wheel.
15
+ RUN apt-get update && apt-get install -y --no-install-recommends \
16
+ build-essential \
17
+ git \
18
+ && rm -rf /var/lib/apt/lists/*
19
+
20
+ # uv from the official static image β€” matches local toolchain.
21
+ COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
22
+
23
+ RUN useradd -m -u 1000 user
24
+ USER user
25
+
26
+ ENV HOME=/home/user \
27
+ PATH=/home/user/.local/bin:$PATH \
28
+ HF_HOME=/home/user/.cache/huggingface \
29
+ TRANSFORMERS_CACHE=/home/user/.cache/huggingface \
30
+ SENTENCE_TRANSFORMERS_HOME=/home/user/.cache/huggingface \
31
+ REDUCER_CACHE_DIR=/home/user/.cache/vectorscape/embeddings \
32
+ UV_LINK_MODE=copy \
33
+ UV_PROJECT_ENVIRONMENT=/home/user/.venv \
34
+ PYTHONUNBUFFERED=1
35
+
36
+ WORKDIR /home/user/app
37
+
38
+ # Layer 1: install deps only (no project), so source edits don't bust this cache.
39
+ COPY --chown=user:user pyproject.toml uv.lock ./
40
+ RUN uv sync --frozen --no-dev --no-install-project
41
+
42
+ # Layer 2: pre-warm the local embedding model into the HF cache. First call to
43
+ # /embed-reduce in production would otherwise pay a one-time ~90 MB download
44
+ # AND torch warmup β€” both happen here instead, in the build.
45
+ RUN uv run python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('all-MiniLM-L6-v2')"
46
+
47
+ # Layer 3: source + install the project itself.
48
+ COPY --chown=user:user app ./app
49
+ COPY --chown=user:user reducer ./reducer
50
+ RUN uv sync --frozen --no-dev
51
+
52
+ EXPOSE 7860
53
+
54
+ CMD ["uv", "run", "uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "7860"]
README.md ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: VectorScape Reducer
3
+ emoji: 🌌
4
+ colorFrom: indigo
5
+ colorTo: purple
6
+ sdk: docker
7
+ app_port: 7860
8
+ pinned: false
9
+ short_description: Embeddings + dim reduction + clustering for VectorScape.
10
+ ---
11
+
12
+ # VectorScape Reducer
13
+
14
+ FastAPI service that turns text rows into 3D coordinates + clusters for the
15
+ VectorScape web app. The web app calls this service over HTTPS with a shared
16
+ secret in the `X-Reducer-Secret` header.
17
+
18
+ ## Endpoints
19
+
20
+ - `GET /health` β€” liveness probe (unauthenticated).
21
+ - `POST /embed-reduce` β€” embed β†’ optional PCA β†’ PaCMAP/UMAP β†’ HDBSCAN, then
22
+ write `points` + `clusters` rows to Supabase. Requires `X-Reducer-Secret`.
23
+ - `GET /status/{project_id}` β€” poll project status. Requires `X-Reducer-Secret`.
24
+ - `POST /bridge` β€” explain the gap between two clusters via LLM. Requires
25
+ `X-Reducer-Secret`.
26
+
27
+ ## Required environment
28
+
29
+ Set these as **secrets** in the Space settings (Settings β†’ Variables and secrets):
30
+
31
+ | Name | Purpose |
32
+ | --- | --- |
33
+ | `REDUCER_SHARED_SECRET` | Must match the value set on the web app. Auth fails closed if unset. |
34
+ | `DATABASE_URL` | Supabase Postgres connection string (service-role; bypasses RLS). |
35
+ | `OPENAI_API_KEY` | Optional. Only used when callers request `embed_model=openai` or for `/bridge`. |
36
+ | `REDUCER_ASYNC_THRESHOLD` | Set to `999999` here β€” no Redis on this Space, so all jobs must run inline. |
37
+
38
+ ## Notes
39
+
40
+ - The `all-MiniLM-L6-v2` model is pre-downloaded at image build time, so the
41
+ first request after a cold start doesn't pay the model download cost.
42
+ - Inline-only on this Space (no arq worker, no Redis). Datasets above the
43
+ threshold will time out the HTTP request β€” small/demo CSVs only.
44
+ - Source of truth: this directory is mirrored from the
45
+ [`vector-scape`](https://github.com/) monorepo's `services/reducer/`.