Spaces:
Sleeping
Sleeping
chore(reducer): dockerize for hugging face spaces deploy
Browse filesVercel functions can't reach the local FastAPI reducer, so package
services/reducer/ as a Docker Space image: uid-1000 user, port 7860,
pre-warmed MiniLM model baked into the image so cold starts return in
seconds. README carries the Space frontmatter and the env-var manifest
operators must set as Space secrets.
- .dockerignore +12 -0
- Dockerfile +54 -0
- README.md +45 -0
.dockerignore
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
__pycache__/
|
| 2 |
+
*.pyc
|
| 3 |
+
*.pyo
|
| 4 |
+
.pytest_cache/
|
| 5 |
+
.ruff_cache/
|
| 6 |
+
.venv/
|
| 7 |
+
tests/
|
| 8 |
+
samples/
|
| 9 |
+
*.md
|
| 10 |
+
!README.md
|
| 11 |
+
.env
|
| 12 |
+
.env.*
|
Dockerfile
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# VectorScape reducer β Hugging Face Spaces (Docker SDK) image.
|
| 2 |
+
#
|
| 3 |
+
# HF Spaces requirements baked in:
|
| 4 |
+
# - Runs as uid 1000 ("user"); $HOME must be writable.
|
| 5 |
+
# - App must bind to 0.0.0.0:7860.
|
| 6 |
+
# - sentence-transformers / huggingface caches live under $HOME and are
|
| 7 |
+
# populated at build time so the first request doesn't pay the model
|
| 8 |
+
# download cost (~90 MB for all-MiniLM-L6-v2).
|
| 9 |
+
FROM python:3.11-slim
|
| 10 |
+
|
| 11 |
+
# build-essential is kept because pacmap / hdbscan can fall back to a Cython
|
| 12 |
+
# build when a manylinux wheel isn't published for the resolved version. git
|
| 13 |
+
# is occasionally needed for pip VCS deps. Both can be dropped later if the
|
| 14 |
+
# resolved lock turns out to be pure-wheel.
|
| 15 |
+
RUN apt-get update && apt-get install -y --no-install-recommends \
|
| 16 |
+
build-essential \
|
| 17 |
+
git \
|
| 18 |
+
&& rm -rf /var/lib/apt/lists/*
|
| 19 |
+
|
| 20 |
+
# uv from the official static image β matches local toolchain.
|
| 21 |
+
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
|
| 22 |
+
|
| 23 |
+
RUN useradd -m -u 1000 user
|
| 24 |
+
USER user
|
| 25 |
+
|
| 26 |
+
ENV HOME=/home/user \
|
| 27 |
+
PATH=/home/user/.local/bin:$PATH \
|
| 28 |
+
HF_HOME=/home/user/.cache/huggingface \
|
| 29 |
+
TRANSFORMERS_CACHE=/home/user/.cache/huggingface \
|
| 30 |
+
SENTENCE_TRANSFORMERS_HOME=/home/user/.cache/huggingface \
|
| 31 |
+
REDUCER_CACHE_DIR=/home/user/.cache/vectorscape/embeddings \
|
| 32 |
+
UV_LINK_MODE=copy \
|
| 33 |
+
UV_PROJECT_ENVIRONMENT=/home/user/.venv \
|
| 34 |
+
PYTHONUNBUFFERED=1
|
| 35 |
+
|
| 36 |
+
WORKDIR /home/user/app
|
| 37 |
+
|
| 38 |
+
# Layer 1: install deps only (no project), so source edits don't bust this cache.
|
| 39 |
+
COPY --chown=user:user pyproject.toml uv.lock ./
|
| 40 |
+
RUN uv sync --frozen --no-dev --no-install-project
|
| 41 |
+
|
| 42 |
+
# Layer 2: pre-warm the local embedding model into the HF cache. First call to
|
| 43 |
+
# /embed-reduce in production would otherwise pay a one-time ~90 MB download
|
| 44 |
+
# AND torch warmup β both happen here instead, in the build.
|
| 45 |
+
RUN uv run python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('all-MiniLM-L6-v2')"
|
| 46 |
+
|
| 47 |
+
# Layer 3: source + install the project itself.
|
| 48 |
+
COPY --chown=user:user app ./app
|
| 49 |
+
COPY --chown=user:user reducer ./reducer
|
| 50 |
+
RUN uv sync --frozen --no-dev
|
| 51 |
+
|
| 52 |
+
EXPOSE 7860
|
| 53 |
+
|
| 54 |
+
CMD ["uv", "run", "uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "7860"]
|
README.md
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: VectorScape Reducer
|
| 3 |
+
emoji: π
|
| 4 |
+
colorFrom: indigo
|
| 5 |
+
colorTo: purple
|
| 6 |
+
sdk: docker
|
| 7 |
+
app_port: 7860
|
| 8 |
+
pinned: false
|
| 9 |
+
short_description: Embeddings + dim reduction + clustering for VectorScape.
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
# VectorScape Reducer
|
| 13 |
+
|
| 14 |
+
FastAPI service that turns text rows into 3D coordinates + clusters for the
|
| 15 |
+
VectorScape web app. The web app calls this service over HTTPS with a shared
|
| 16 |
+
secret in the `X-Reducer-Secret` header.
|
| 17 |
+
|
| 18 |
+
## Endpoints
|
| 19 |
+
|
| 20 |
+
- `GET /health` β liveness probe (unauthenticated).
|
| 21 |
+
- `POST /embed-reduce` β embed β optional PCA β PaCMAP/UMAP β HDBSCAN, then
|
| 22 |
+
write `points` + `clusters` rows to Supabase. Requires `X-Reducer-Secret`.
|
| 23 |
+
- `GET /status/{project_id}` β poll project status. Requires `X-Reducer-Secret`.
|
| 24 |
+
- `POST /bridge` β explain the gap between two clusters via LLM. Requires
|
| 25 |
+
`X-Reducer-Secret`.
|
| 26 |
+
|
| 27 |
+
## Required environment
|
| 28 |
+
|
| 29 |
+
Set these as **secrets** in the Space settings (Settings β Variables and secrets):
|
| 30 |
+
|
| 31 |
+
| Name | Purpose |
|
| 32 |
+
| --- | --- |
|
| 33 |
+
| `REDUCER_SHARED_SECRET` | Must match the value set on the web app. Auth fails closed if unset. |
|
| 34 |
+
| `DATABASE_URL` | Supabase Postgres connection string (service-role; bypasses RLS). |
|
| 35 |
+
| `OPENAI_API_KEY` | Optional. Only used when callers request `embed_model=openai` or for `/bridge`. |
|
| 36 |
+
| `REDUCER_ASYNC_THRESHOLD` | Set to `999999` here β no Redis on this Space, so all jobs must run inline. |
|
| 37 |
+
|
| 38 |
+
## Notes
|
| 39 |
+
|
| 40 |
+
- The `all-MiniLM-L6-v2` model is pre-downloaded at image build time, so the
|
| 41 |
+
first request after a cold start doesn't pay the model download cost.
|
| 42 |
+
- Inline-only on this Space (no arq worker, no Redis). Datasets above the
|
| 43 |
+
threshold will time out the HTTP request β small/demo CSVs only.
|
| 44 |
+
- Source of truth: this directory is mirrored from the
|
| 45 |
+
[`vector-scape`](https://github.com/) monorepo's `services/reducer/`.
|