Spaces:
Running on Zero
Running on Zero
docs: create roadmap (6 phases)
Browse files- .planning/REQUIREMENTS.md +37 -5
- .planning/ROADMAP.md +134 -0
- .planning/STATE.md +67 -0
- CLAUDE.md +220 -0
.planning/REQUIREMENTS.md
CHANGED
|
@@ -99,13 +99,45 @@ Which phases cover which requirements. Updated during roadmap creation.
|
|
| 99 |
|
| 100 |
| Requirement | Phase | Status |
|
| 101 |
|-------------|-------|--------|
|
| 102 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 103 |
|
| 104 |
**Coverage:**
|
| 105 |
-
- v1 requirements: 31 total
|
| 106 |
-
- Mapped to phases:
|
| 107 |
-
- Unmapped:
|
| 108 |
|
| 109 |
---
|
| 110 |
*Requirements defined: 2026-08-08*
|
| 111 |
-
*Last updated: 2026-08-08 after
|
|
|
|
| 99 |
|
| 100 |
| Requirement | Phase | Status |
|
| 101 |
|-------------|-------|--------|
|
| 102 |
+
| AVTR-01 | Phase 1 | Pending |
|
| 103 |
+
| AVTR-02 | Phase 1 | Pending |
|
| 104 |
+
| AVTR-03 | Phase 3 | Pending |
|
| 105 |
+
| VOIC-01 | Phase 1 | Pending |
|
| 106 |
+
| VOIC-02 | Phase 1 | Pending |
|
| 107 |
+
| VOIC-03 | Phase 1 | Pending |
|
| 108 |
+
| VOIC-04 | Phase 1 | Pending |
|
| 109 |
+
| VOIC-05 | Phase 1 | Pending |
|
| 110 |
+
| JPN-01 | Phase 2 | Pending |
|
| 111 |
+
| JPN-02 | Phase 2 | Pending |
|
| 112 |
+
| JPN-03 | Phase 2 | Pending |
|
| 113 |
+
| JPN-04 | Phase 2 | Pending |
|
| 114 |
+
| JPN-05 | Phase 3 | Pending |
|
| 115 |
+
| TUTR-01 | Phase 3 | Pending |
|
| 116 |
+
| TUTR-02 | Phase 3 | Pending |
|
| 117 |
+
| TUTR-03 | Phase 3 | Pending |
|
| 118 |
+
| TUTR-04 | Phase 5 | Pending |
|
| 119 |
+
| TUTR-05 | Phase 5 | Pending |
|
| 120 |
+
| TUTR-06 | Phase 5 | Pending |
|
| 121 |
+
| DRIL-01 | Phase 5 | Pending |
|
| 122 |
+
| DRIL-02 | Phase 5 | Pending |
|
| 123 |
+
| DRIL-03 | Phase 5 | Pending |
|
| 124 |
+
| LLMB-01 | Phase 3 | Pending |
|
| 125 |
+
| LLMB-02 | Phase 4 | Pending |
|
| 126 |
+
| LLMB-03 | Phase 3 | Pending |
|
| 127 |
+
| ACCT-01 | Phase 4 | Pending |
|
| 128 |
+
| ACCT-02 | Phase 4 | Pending |
|
| 129 |
+
| ACCT-03 | Phase 4 | Pending |
|
| 130 |
+
| ACCT-04 | Phase 4 | Pending |
|
| 131 |
+
| DPLY-01 | Phase 1 | Pending |
|
| 132 |
+
| DPLY-02 | Phase 6 | Pending |
|
| 133 |
+
| DPLY-03 | Phase 6 | Pending |
|
| 134 |
+
| DPLY-04 | Phase 1 | Pending |
|
| 135 |
|
| 136 |
**Coverage:**
|
| 137 |
+
- v1 requirements: 33 total (corrected from 31 β original count miscounted; enumerated IDs total 33)
|
| 138 |
+
- Mapped to phases: 33 β
|
| 139 |
+
- Unmapped: 0
|
| 140 |
|
| 141 |
---
|
| 142 |
*Requirements defined: 2026-08-08*
|
| 143 |
+
*Last updated: 2026-08-08 after roadmap creation (traceability populated, v1 count corrected to 33)*
|
.planning/ROADMAP.md
ADDED
|
@@ -0,0 +1,134 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Roadmap: Japanese Learning Avatar
|
| 2 |
+
|
| 3 |
+
## Overview
|
| 4 |
+
|
| 5 |
+
The journey runs risk-first, not feature-first. Phase 1 proves the thing nobody has built in a Gradio Space β a VRM avatar with mora-timed lip-sync and a real voice round-trip β deployed publicly with zero AI, so its latency and hosting reality are measured facts before any pedagogy is written. Phase 2 lays the invisible foundation (SudachiPy tokenization) that furigana, word lookup, level gating, and vocabulary tracking all depend on. Phase 3 turns the avatar into a tutor with deterministic guardrails: one LLM gateway, a level guard that tokenizes every reply, and a curated grammar database the model may quote but never author. Phase 4 makes progress real β HF OAuth over a fully-working guest path, an append-only event log in Neon Postgres, and per-session isolation strict enough to hold a visitor's BYOK key safely. Phase 5 builds the curriculum on top of that schema: conversational level assessment, two-directional N5βN2 adaptation, roleplay scenarios, drills, and an FSRS review queue fed by real mistakes. Phase 6 is the stranger test β cold start, phone screen, exhausted GPU quota, verified licenses, and a transparency panel that shows a hiring manager the engineering underneath.
|
| 6 |
+
|
| 7 |
+
## Phases
|
| 8 |
+
|
| 9 |
+
**Phase Numbering:**
|
| 10 |
+
- Integer phases (1, 2, 3): Planned milestone work
|
| 11 |
+
- Decimal phases (2.1, 2.2): Urgent insertions (marked with INSERTED)
|
| 12 |
+
|
| 13 |
+
Decimal phases appear between their surrounding integers in numeric order.
|
| 14 |
+
|
| 15 |
+
- [ ] **Phase 1: Voice + Avatar Loop Skeleton** - Deployed Space where a lip-synced VRM avatar speaks Japanese and hears you back, with measured real-world latency and zero AI
|
| 16 |
+
- [ ] **Phase 2: Japanese Language Core** - Every Japanese string becomes machine-understood tokens: readings, furigana, lookup, translation
|
| 17 |
+
- [ ] **Phase 3: Tutoring Brain** - The avatar becomes a tutor: one LLM gateway, deterministic level gating, grammar-DB-grounded corrections
|
| 18 |
+
- [ ] **Phase 4: Accounts & Persistence** - Guest-first sign-in with HF OAuth and server-side progress that survives sessions, devices, and concurrent users
|
| 19 |
+
- [ ] **Phase 5: Adaptive Level, Lessons & SRS** - Conversational level assessment, two-directional N5βN2 adaptation, roleplay scenarios, drills, and FSRS review
|
| 20 |
+
- [ ] **Phase 6: Deploy & Portfolio Hardening** - Cold-start, mobile, quota-exhausted survival plus the engineering transparency panel
|
| 21 |
+
|
| 22 |
+
## Phase Details
|
| 23 |
+
|
| 24 |
+
### Phase 1: Voice + Avatar Loop Skeleton
|
| 25 |
+
**Goal**: A visitor at a public Space URL can talk to a living, lip-synced VRM avatar that talks back β no AI tutoring yet, but the entire audio round-trip works on real hosting and its actual latency is a known number.
|
| 26 |
+
**Depends on**: Nothing (first phase)
|
| 27 |
+
**Requirements**: AVTR-01, AVTR-02, VOIC-01, VOIC-02, VOIC-03, VOIC-04, VOIC-05, DPLY-01, DPLY-04
|
| 28 |
+
**Success Criteria** (what must be TRUE):
|
| 29 |
+
1. Anyone opening the public Space on the WolfDavid profile sees a VRM avatar with idle life β blinking, breathing, subtle motion β rendering smoothly in desktop Chrome and at least one mobile browser.
|
| 30 |
+
2. The avatar speaks a Japanese utterance aloud and its mouth shapes track the per-mora vowel timings from the TTS engine (five VRM vowel visemes), visibly not amplitude flapping.
|
| 31 |
+
3. A visitor can push-to-talk in Japanese and see their transcript appear, type instead of speaking and get the same round-trip, and replay the avatar's last utterance β including a slower re-read.
|
| 32 |
+
4. p50/p95 turn latency is measured and recorded on the deployed Space (cold and warm), and the full loop still completes with the GPU path forcibly disabled.
|
| 33 |
+
5. LICENSES.md documents verified licenses for the exact VRM character and TTS voice that ship, including any per-character credit requirement.
|
| 34 |
+
**Plans**: TBD
|
| 35 |
+
|
| 36 |
+
### Phase 2: Japanese Language Core
|
| 37 |
+
**Goal**: Every piece of Japanese text in the app is machine-understood β surface, reading, lemma, POS, JLPT level β so learners can decode it on screen and every later system can reason about it.
|
| 38 |
+
**Depends on**: Phase 1
|
| 39 |
+
**Requirements**: JPN-01, JPN-02, JPN-03, JPN-04
|
| 40 |
+
**Success Criteria** (what must be TRUE):
|
| 41 |
+
1. Any Japanese string the app displays resolves to per-token `{surface, reading, lemma, pos, jlpt_level}`, demonstrated against a fixed sample sentence set with no LLM involved.
|
| 42 |
+
2. A learner can toggle furigana on and off for all Japanese text, and the ruby readings render correctly on the deployed Space (surviving Gradio's Markdown sanitization), not just locally.
|
| 43 |
+
3. A learner can click any word in any Japanese text and immediately see its reading, meaning, and JLPT level.
|
| 44 |
+
4. A learner can request a full-sentence translation on demand for any Japanese line shown.
|
| 45 |
+
**Plans**: TBD
|
| 46 |
+
|
| 47 |
+
### Phase 3: Tutoring Brain
|
| 48 |
+
**Goal**: The avatar stops echoing and starts teaching β holding real conversation, never speaking above the learner's level, correcting from a curated grammar source rather than its own imagination.
|
| 49 |
+
**Depends on**: Phase 2
|
| 50 |
+
**Requirements**: AVTR-03, JPN-05, TUTR-01, TUTR-02, TUTR-03, LLMB-01, LLMB-03
|
| 51 |
+
**Success Criteria** (what must be TRUE):
|
| 52 |
+
1. A learner can hold a multi-turn free-form Japanese conversation with level-appropriate replies, and the conversation works for an anonymous visitor on the free open-model tier even with the GPU path disabled or quota exhausted (degraded, still functional).
|
| 53 |
+
2. Every tutor reply is tokenized and rejected/regenerated when it exceeds the learner's JLPT level β demonstrated on an expert-answered regression set across both the free and frontier model tiers, with drift not recovering by prompt alone.
|
| 54 |
+
3. Corrections and grammar explanations trace to entries in the curated JLPT-tagged grammar database; the tutor selects and phrases entries but never authors a rule.
|
| 55 |
+
4. A learner can move a casual β polite β formal register dial and hear the tutor's speech change accordingly.
|
| 56 |
+
5. The avatar's expression reflects what the tutor is doing β listening, thinking, encouraging after a mistake, celebrating progress β rather than the sentiment of its text.
|
| 57 |
+
**Plans**: TBD
|
| 58 |
+
|
| 59 |
+
### Phase 4: Accounts & Persistence
|
| 60 |
+
**Goal**: Progress becomes real and portable β a stranger starts learning with zero setup, signs in when they choose, and finds their history waiting on any device, with nothing leaking between concurrent users.
|
| 61 |
+
**Depends on**: Phase 3
|
| 62 |
+
**Requirements**: ACCT-01, ACCT-02, ACCT-03, ACCT-04, LLMB-02
|
| 63 |
+
**Success Criteria** (what must be TRUE):
|
| 64 |
+
1. An anonymous visitor reaches a talking avatar and completes a full conversation with no signup gate anywhere in the path.
|
| 65 |
+
2. A learner can sign in with Hugging Face OAuth from the live Space and see their signed-in identity reflected in the app.
|
| 66 |
+
3. A signed-in learner's level estimate, vocabulary exposure, mistakes, and review state survive closing the session entirely and reappear in a different browser or device.
|
| 67 |
+
4. Progress accumulated as a guest is adopted into the account on sign-in rather than discarded.
|
| 68 |
+
5. A learner can supply their own Claude or OpenAI key for the session and it is never persisted or logged; two browsers driving the app simultaneously never see each other's key, level, or history.
|
| 69 |
+
**Plans**: TBD
|
| 70 |
+
|
| 71 |
+
### Phase 5: Adaptive Level, Lessons & SRS
|
| 72 |
+
**Goal**: The tutor teaches a curriculum rather than making conversation β it works out where the learner is, moves that estimate in both directions, drills what they got wrong, and schedules the review.
|
| 73 |
+
**Depends on**: Phase 4
|
| 74 |
+
**Requirements**: TUTR-04, TUTR-05, TUTR-06, DRIL-01, DRIL-02, DRIL-03
|
| 75 |
+
**Success Criteria** (what must be TRUE):
|
| 76 |
+
1. A new learner's JLPT level is established through ordinary conversation β no quiz screen β and the resulting estimate is visible to them and stored.
|
| 77 |
+
2. The level estimate moves in both directions across N5βN2 in response to demonstrated performance, verified by replaying a scripted synthetic learner who improves and then degrades.
|
| 78 |
+
3. A learner can play 3-5 roleplay scenarios (ordering food, asking directions, etc.) where the avatar holds the counterpart role and the scene actually ends when the scenario's exit condition is met.
|
| 79 |
+
4. A learner can run structured kana, vocabulary, and grammar drills with visible progression through the material.
|
| 80 |
+
5. Vocabulary encountered in conversation and drills enters an FSRS review queue with due dates that behave correctly across successive reviews, and a mistake made in conversation reappears later as a targeted review drill.
|
| 81 |
+
**Plans**: TBD
|
| 82 |
+
|
| 83 |
+
### Phase 6: Deploy & Portfolio Hardening
|
| 84 |
+
**Goal**: The Space survives a stranger arriving cold, on a phone, with no GPU quota β and visibly shows the engineering underneath to the audience this portfolio piece targets.
|
| 85 |
+
**Depends on**: Phase 5
|
| 86 |
+
**Requirements**: DPLY-02, DPLY-03
|
| 87 |
+
**Success Criteria** (what must be TRUE):
|
| 88 |
+
1. An engineering transparency panel shows the live per-stage latency breakdown and the active model/provider for the turn just taken.
|
| 89 |
+
2. The full experience β avatar, transcript, controls, drills β is usable on a phone-sized viewport with no horizontal scroll or unreachable controls.
|
| 90 |
+
3. A cold start from a factory rebuild reaches a talking avatar within a measured, recorded time budget, verified on the deployed Space rather than locally.
|
| 91 |
+
4. With GPU quota exhausted or forcibly disabled, a first-time anonymous visitor still completes a full conversation turn end to end.
|
| 92 |
+
5. Final audit passes: LICENSES.md covers every shipped third-party asset, and multi-user isolation is re-verified concurrently on the live Space.
|
| 93 |
+
**Plans**: TBD
|
| 94 |
+
|
| 95 |
+
## Progress
|
| 96 |
+
|
| 97 |
+
**Execution Order:**
|
| 98 |
+
Phases execute in numeric order: 1 β 2 β 3 β 4 β 5 β 6
|
| 99 |
+
|
| 100 |
+
| Phase | Plans Complete | Status | Completed |
|
| 101 |
+
|-------|----------------|--------|-----------|
|
| 102 |
+
| 1. Voice + Avatar Loop Skeleton | 0/TBD | Not started | - |
|
| 103 |
+
| 2. Japanese Language Core | 0/TBD | Not started | - |
|
| 104 |
+
| 3. Tutoring Brain | 0/TBD | Not started | - |
|
| 105 |
+
| 4. Accounts & Persistence | 0/TBD | Not started | - |
|
| 106 |
+
| 5. Adaptive Level, Lessons & SRS | 0/TBD | Not started | - |
|
| 107 |
+
| 6. Deploy & Portfolio Hardening | 0/TBD | Not started | - |
|
| 108 |
+
|
| 109 |
+
## Requirement Coverage
|
| 110 |
+
|
| 111 |
+
| Phase | Requirements | Count |
|
| 112 |
+
|-------|--------------|-------|
|
| 113 |
+
| 1 | AVTR-01, AVTR-02, VOIC-01, VOIC-02, VOIC-03, VOIC-04, VOIC-05, DPLY-01, DPLY-04 | 9 |
|
| 114 |
+
| 2 | JPN-01, JPN-02, JPN-03, JPN-04 | 4 |
|
| 115 |
+
| 3 | AVTR-03, JPN-05, TUTR-01, TUTR-02, TUTR-03, LLMB-01, LLMB-03 | 7 |
|
| 116 |
+
| 4 | ACCT-01, ACCT-02, ACCT-03, ACCT-04, LLMB-02 | 5 |
|
| 117 |
+
| 5 | TUTR-04, TUTR-05, TUTR-06, DRIL-01, DRIL-02, DRIL-03 | 6 |
|
| 118 |
+
| 6 | DPLY-02, DPLY-03 | 2 |
|
| 119 |
+
|
| 120 |
+
**Total: 33/33 v1 requirements mapped. No orphans, no duplicates.**
|
| 121 |
+
|
| 122 |
+
## Research Flags
|
| 123 |
+
|
| 124 |
+
Phases where `/gsd:plan-phase` should commission deeper research (from research/SUMMARY.md):
|
| 125 |
+
|
| 126 |
+
- **Phase 1**: Gradio 6 `gr.HTML` custom-component behavior under real deployment vs. local dev; three-vrm CDN importmap loading; TTS-engine-with-mora-timings selection and its licensing footprint (VOICEVOX character terms gate a design branch, not just a checkbox); confirm the WolfDavid account's actual ZeroGPU eligibility and quota arithmetic before committing the phase design.
|
| 127 |
+
- **Phase 2**: Whether furigana/ruby survives Gradio's Markdown sanitization on the deployed Space. (Pronunciation-assessment mechanism research is deferred with PRON-01 to v2 β Whisper-as-scorer is disqualified.)
|
| 128 |
+
- **Phase 3**: Grammar-grounding design (retrieval vs. generation split) and the level-validation guardrail implementation β flagged independently by both ARCHITECTURE.md and PITFALLS.md as needing a dedicated spike.
|
| 129 |
+
- **Phases 4, 5, 6**: Standard, well-documented patterns (HF OAuth + Neon Postgres, py-fsrs, deploy checklist). Skip dedicated research; the work is discipline, not discovery.
|
| 130 |
+
|
| 131 |
+
**Cross-phase constraint:** every phase's exit criteria include "still works with the GPU path forcibly disabled / quota exhausted."
|
| 132 |
+
|
| 133 |
+
---
|
| 134 |
+
*Roadmap created: 2026-08-08*
|
.planning/STATE.md
ADDED
|
@@ -0,0 +1,67 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Project State
|
| 2 |
+
|
| 3 |
+
## Project Reference
|
| 4 |
+
|
| 5 |
+
See: .planning/PROJECT.md (updated 2026-08-08)
|
| 6 |
+
|
| 7 |
+
**Core value:** A learner can hold a real, level-appropriate spoken Japanese conversation with an animated avatar that talks back β and measurably improve over time because the avatar remembers them.
|
| 8 |
+
**Current focus:** Phase 1 β Voice + Avatar Loop Skeleton
|
| 9 |
+
|
| 10 |
+
## Current Position
|
| 11 |
+
|
| 12 |
+
Phase: 1 of 6 (Voice + Avatar Loop Skeleton)
|
| 13 |
+
Plan: 0 of TBD in current phase
|
| 14 |
+
Status: Ready to plan
|
| 15 |
+
Last activity: 2026-08-08 β Roadmap created (6 phases, 33/33 v1 requirements mapped)
|
| 16 |
+
|
| 17 |
+
Progress: [ββββββββββ] 0%
|
| 18 |
+
|
| 19 |
+
## Performance Metrics
|
| 20 |
+
|
| 21 |
+
**Velocity:**
|
| 22 |
+
- Total plans completed: 0
|
| 23 |
+
- Average duration: β
|
| 24 |
+
- Total execution time: 0.0 hours
|
| 25 |
+
|
| 26 |
+
**By Phase:**
|
| 27 |
+
|
| 28 |
+
| Phase | Plans | Total | Avg/Plan |
|
| 29 |
+
|-------|-------|-------|----------|
|
| 30 |
+
| - | - | - | - |
|
| 31 |
+
|
| 32 |
+
**Recent Trend:**
|
| 33 |
+
- Last 5 plans: β
|
| 34 |
+
- Trend: β
|
| 35 |
+
|
| 36 |
+
*Updated after each plan completion*
|
| 37 |
+
|
| 38 |
+
## Accumulated Context
|
| 39 |
+
|
| 40 |
+
### Decisions
|
| 41 |
+
|
| 42 |
+
Decisions are logged in PROJECT.md Key Decisions table.
|
| 43 |
+
Recent decisions affecting current work:
|
| 44 |
+
|
| 45 |
+
- [Roadmap]: Risk-retirement ordering β the avatar/voice transport is proven first with zero AI, because it has the least prior art in a Gradio context and cannot be worked around if it fails.
|
| 46 |
+
- [Roadmap]: SudachiPy tokenization (Phase 2) lands before any level gating or grammar grounding β furigana, lookup, level gating, and vocab tracking all collapse without it.
|
| 47 |
+
- [Roadmap]: Persistence (Phase 4) is deliberately after the tutor core so the event-sourced schema reflects what the assessment engine actually emits, avoiding a Phase 5 migration.
|
| 48 |
+
- [Roadmap]: BYOK (LLMB-02) is owned by Phase 4, not Phase 3 β its session-only, never-leaked guarantee is verified by the same concurrent-session isolation gate as account data.
|
| 49 |
+
|
| 50 |
+
### Pending Todos
|
| 51 |
+
|
| 52 |
+
[From .planning/todos/pending/ β ideas captured during sessions]
|
| 53 |
+
|
| 54 |
+
None yet.
|
| 55 |
+
|
| 56 |
+
### Blockers/Concerns
|
| 57 |
+
|
| 58 |
+
- **Hosting model unresolved (pre-Phase-1 decision).** STACK research found free Gradio Spaces on personal accounts now require PRO; the free path is ZeroGPU with per-visitor quota (~2 min/day anonymous). Confirm WolfDavid account eligibility (verified email, >30 days old, <2 existing ZeroGPU Spaces) during Phase 1 planning; if ineligible, PRO ($9/mo) becomes a decision before any build.
|
| 59 |
+
- **Asset licensing gates a design branch.** VOICEVOX character terms need a human read (third-party web synthesis, not just generated audio) before TTS is built around it β the fallback loses free mora timings and changes the lip-sync design. VRM model licenses vary per author; self-authored VRoid Studio character removes the question.
|
| 60 |
+
- **Per-turn GPU-second cost is unmeasured.** The entire free-tier UX budget depends on this number β measure in Phase 1/3, do not assume.
|
| 61 |
+
- **REQUIREMENTS.md originally miscounted v1 as 31.** Actual count is 33; traceability and coverage corrected during roadmap creation.
|
| 62 |
+
|
| 63 |
+
## Session Continuity
|
| 64 |
+
|
| 65 |
+
Last session: 2026-08-08
|
| 66 |
+
Stopped at: ROADMAP.md and STATE.md written; REQUIREMENTS.md traceability populated.
|
| 67 |
+
Resume file: None
|
CLAUDE.md
ADDED
|
@@ -0,0 +1,220 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<!-- GSD:project-start source:PROJECT.md -->
|
| 2 |
+
## Project
|
| 3 |
+
|
| 4 |
+
**Japanese Learning Avatar**
|
| 5 |
+
|
| 6 |
+
An animated 3D avatar tutor that teaches Japanese through spoken conversation, structured lessons, and immersion role-play β hosted as a Hugging Face Space. A VRM anime-style character listens to the learner speak Japanese, responds aloud with lip-synced speech and expressions, drills vocab and grammar, and adapts to the learner's level (N5 through N2). Built both as a daily-use learning tool for the author and as the flagship AI/ML engineering showcase on the WolfDavid Hugging Face profile.
|
| 7 |
+
|
| 8 |
+
**Core Value:** A learner can hold a real, level-appropriate spoken Japanese conversation with an animated avatar that talks back β and measurably improve over time because the avatar remembers them.
|
| 9 |
+
|
| 10 |
+
### Constraints
|
| 11 |
+
|
| 12 |
+
- **Platform**: Hugging Face Space (Gradio) β portfolio strategy requires it; free-tier visitors must get a working experience
|
| 13 |
+
- **Cost**: Free baseline for visitors β small open model default; frontier quality only via visitor's own API key (BYOK)
|
| 14 |
+
- **Tech**: In-browser avatar rendering (three.js/VRM) β no server-side video generation
|
| 15 |
+
- **Quality bar**: Flagship portfolio piece β "would a Google AI/ML hiring manager be impressed?" drives polish decisions
|
| 16 |
+
<!-- GSD:project-end -->
|
| 17 |
+
|
| 18 |
+
<!-- GSD:stack-start source:research/STACK.md -->
|
| 19 |
+
## Technology Stack
|
| 20 |
+
|
| 21 |
+
## β οΈ Read This First: Two Findings That Reshape the Project
|
| 22 |
+
### 1. "Free CPU tier" is no longer the free path for a Gradio Space
|
| 23 |
+
### 2. ZeroGPU quota is consumed by the *visitor*, not the owner
|
| 24 |
+
| Visitor account type | Included daily GPU quota | Queue priority |
|
| 25 |
+
|---|---|---|
|
| 26 |
+
| Unauthenticated | **2 minutes/day** | Low |
|
| 27 |
+
| Free HF account | **5 minutes/day** | Medium |
|
| 28 |
+
| PRO account | 40 minutes/day (extensible) | Highest |
|
| 29 |
+
## Recommended Stack
|
| 30 |
+
### Core Technologies
|
| 31 |
+
| Technology | Version | Purpose | Why Recommended |
|
| 32 |
+
|---|---|---|---|
|
| 33 |
+
| **Gradio** | `6.22.0` (pin exactly) | App framework + Space SDK | Gradio 6 is the only maintained major line; the team has stated only v6 gets future releases. Crucially, **ZeroGPU is exclusively compatible with the Gradio SDK** β no Docker, no Static. Gradio 6 also ships the `gr.HTML` templating system (below) that makes the three.js avatar viable without a build step. |
|
| 34 |
+
| **Python** | `3.12.12` (declare in README) | Runtime | ZeroGPU provides **only** Python `3.10.13` and `3.12.12`. Pick 3.12.12. **Do not use 3.13** β this is the recurring "Python 3.13 / Gradio pin" gotcha from prior Spaces work, and here it's a hard platform limit, not a soft one (`spaces` requires `>=3.10,<3.15`, but ZeroGPU only builds those two images). |
|
| 35 |
+
| **`spaces`** | `0.51.1` | ZeroGPU allocation | Provides `@spaces.GPU(duration=...)`. Effect-free off-ZeroGPU, so local dev is unaffected. Use **dynamic durations** (`duration=callable`) β shorter declared durations improve queue priority for your visitors. |
|
| 36 |
+
| **three.js** | `0.185.1` (MIT) | 3D renderer | The only realistic in-browser 3D engine with first-class VRM support. Pin the exact version β three.js makes breaking renderer changes at a ~6-week cadence and `three-vrm` tracks it loosely. |
|
| 37 |
+
| **@pixiv/three-vrm** | `3.5.5` (MIT) | VRM avatar loading, humanoid bones, expressions, spring-bone physics | The reference implementation from the company that runs VRoid Hub. Supports both VRM 0.0 and VRM 1.0, so community VRoid models load without conversion. Peer dep is `three >= 0.137`, so 0.185.1 is safe. |
|
| 38 |
+
| **VOICEVOX CORE** | `0.16.4` (`+cpu` abi3 wheel) | Japanese TTS **with per-mora phoneme timing** | See "The lip-sync decision" below. This is the highest-leverage choice in the whole stack. |
|
| 39 |
+
| **whisper-large-v3-turbo** | via `transformers` 5.x on ZeroGPU | Japanese ASR | Best measured accuracy/speed balance for Japanese conversational audio in the Feb-2026 benchmark: CER 0.184, **RTF 0.013**. MIT-licensed. |
|
| 40 |
+
| **Qwen3.5-4B** | Apache-2.0, released 2026-03-02 | Free-default tutor LLM | 4B dense, **201 languages incl. Japanese**, 256K context, hybrid thinking (disable thinking for latency). Apache-2.0 means no license story to explain to a hiring manager. Fits easily in 48 GB with room for KV cache. |
|
| 41 |
+
| **`transformers`** | `5.14.1` | Model loading | v5 is current. Note: v5 was a breaking release β do not copy v4-era snippets from blogs. |
|
| 42 |
+
| **`huggingface_hub`** | `1.27.0` | Hub + Inference Providers client | v1.x. Provides `InferenceClient` for the BYOK/routed paths. |
|
| 43 |
+
| **Neon Serverless Postgres** | Free plan | User accounts + progress persistence | See "The database decision" below. |
|
| 44 |
+
### The lip-sync decision (do not skip this)
|
| 45 |
+
- **No audio-amplitude analysis.** ChatVRM and most AITuber kits drive the `aa` blendshape from a Web Audio `AnalyserNode` RMS β the mouth just flaps open and shut. Mora-timed visemes are visibly better and are the thing that makes the demo look expensive.
|
| 46 |
+
- **No forced alignment, no viseme ML model, no GPU.**
|
| 47 |
+
- **Zero ZeroGPU quota consumed for speech output**, which is what makes the free tier survivable.
|
| 48 |
+
- The same mora timeline doubles as **pedagogical data**: mora-by-mora pitch accent is already in the payload, so "here is where your pitch accent was wrong" is nearly free.
|
| 49 |
+
- `voicevox_core` is dual-licensed **LGPL v3** + a commercial no-source-disclosure license. Dynamic linking via the Python wheel keeps you clear of copyleft on your own code.
|
| 50 |
+
- **Each character voice has its own terms.** The general rule is *free for commercial and non-commercial use provided you display credit* in the form `VOICEVOX:γγ£γ©γ―γΏγΌε`. Without credit, per-character paid licensing is reportedly Β₯400,000/character.
|
| 51 |
+
- **Read the specific character's terms before shipping**, and render the credit string persistently in the UI (a footer line next to the avatar). Prefer a character with permissive, well-documented terms (γγγ γγ / VOICEVOX Nemo voices are the usual safe picks).
|
| 52 |
+
- Also verify the terms permit *offering synthesis to third parties via a web app*, not merely using generated audio β this is the one clause a portfolio Space most plausibly trips.
|
| 53 |
+
### The avatar-embedding decision
|
| 54 |
+
| Approach | Verdict |
|
| 55 |
+
|---|---|
|
| 56 |
+
| `gr.HTML` with `html_template`/`js_on_load` | β
**Recommended.** No build step, no npm publish, hot-reloads with the app, `server_functions` gives direct JSβPython calls, `watch('value')` gives PythonβJS push. Ships as one `app.py`. |
|
| 57 |
+
| Full Gradio custom component (`gradio cc`) | β Requires Svelte, Node 18+, npm 9+, a build/publish cycle, and a separate package on PyPI. Justified only if you intend to publish the VRM component for others β a nice-to-have, not a v1 need. |
|
| 58 |
+
| Raw `<iframe>` + `postMessage` | β You hand-roll the entire bridge, fight Space iframe sandboxing and cookie issues, and lose Gradio's event system. |
|
| 59 |
+
| Server-rendered talking-head video | β Already correctly in `PROJECT.md` Out of Scope. Would also incinerate ZeroGPU quota. |
|
| 60 |
+
### The database decision
|
| 61 |
+
| Option | Verdict |
|
| 62 |
+
|---|---|
|
| 63 |
+
| **Neon serverless Postgres (free plan)** | β
**Recommended.** 0.5 GB storage + 100 compute-hours/project/month, up to 100 projects, no credit card. Compute **scales to zero after 5 min idle and auto-resumes in a few hundred milliseconds on the next query** β no manual intervention, ever. |
|
| 64 |
+
| Supabase (free plan) | β οΈ Only if you need its Auth/Storage too. **Free projects are auto-paused after 7 days of inactivity and require a manual dashboard unpause.** For a portfolio piece whose entire value is "a hiring manager clicks the link and it works," a silent 7-day dead-man's-switch on your user database is a project-killing failure mode. |
|
| 65 |
+
| SQLite on Space disk | β Space disk is **ephemeral** β wiped on every restart/rebuild, and Spaces rebuild on every git push. You would lose user progress every deploy. |
|
| 66 |
+
| HF Storage Buckets (S3-like, Xet-backed) | β οΈ Great for **assets** (VRM files, cached TTS audio, lesson media) β mountable as a Space volume, mutable, free allowance. **Not** a transactional database; SQLite over a network-mounted bucket invites corruption under concurrency. |
|
| 67 |
+
| HF Dataset repo + `CommitScheduler` | β οΈ The classic HF persistence pattern, fine for **append-only analytics/telemetry**. Wrong shape for per-user read-modify-write of progress records (git commits, no transactions, no concurrent-write story). |
|
| 68 |
+
### Supporting Libraries
|
| 69 |
+
| Library | Version | Purpose | When to Use |
|
| 70 |
+
|---|---|---|---|
|
| 71 |
+
| **SudachiPy** | `0.6.11` (Apache-2.0) | Japanese tokenization + `reading_form()` for furigana + lemmas | Primary Japanese NLP engine. Chosen over MeCab/fugashi for the clean Apache-2.0 story and pure-wheel install (no MeCab system dependency to fight in a Space build). Three splitting modes (A/B/C) map neatly onto "vocab item" vs "phrase" granularity for a tutor. |
|
| 72 |
+
| **SudachiDict-core** | `20260723` (Apache-2.0) | Dictionary for SudachiPy | Required. `-core` is the right size/coverage tradeoff; `-full` bloats the image. |
|
| 73 |
+
| **jaconv** | `0.5.0` (MIT) | hiragana β katakana β half-width conversion | Normalizing learner input, generating kana drills, feeding the viseme mapper. Tiny, zero-dependency, MIT. |
|
| 74 |
+
| **cutlet** | `0.5.2` | Romaji generation | N5 learners need romaji scaffolding. Built on fugashi; use only if you also pull fugashi, otherwise derive romaji from SudachiPy readings + `jaconv`. |
|
| 75 |
+
| **`@huggingface/transformers`** (transformers.js) | `4.2.0` (Apache-2.0) | **Browser-side** Whisper ASR fallback | v4 (Feb 2026) shipped a C++-rewritten WebGPU runtime (~4Γ faster, 53% smaller bundles). Running ASR here costs **zero ZeroGPU quota** β this is how anonymous visitors get more than four conversational turns. |
|
| 76 |
+
| **`anthropic`** | `0.121.0` | BYOK frontier path (Claude) | Visitor pastes their key; nothing is stored server-side. |
|
| 77 |
+
| **`openai`** | `2.53.0` | BYOK frontier path (OpenAI) β and reused as the client for HF's OpenAI-compatible router (`https://router.huggingface.co/v1`) | One client library covers two providers. |
|
| 78 |
+
| **`psycopg[binary]`** | 3.x | Postgres driver | Neon over TLS. Use a pooled connection string; a Space restart should not leak connections. |
|
| 79 |
+
| **SQLAlchemy** | 2.x | ORM / schema | Optional but recommended once progress modelling gets past ~4 tables (level, vocab-seen SRS state, mistakes, sessions). |
|
| 80 |
+
| **`onnxruntime`** | pulled by voicevox_core downloader | VOICEVOX inference backend | CPU build. Managed by the VOICEVOX downloader, do not hand-install. |
|
| 81 |
+
| **`fastrtc`** | `0.0.34` | *Deferred.* True barge-in / VAD / turn-taking over WebRTC | HF's real-time library, Gradio-team maintained, with built-in VAD and turn detection + Cloudflare TURN. **Still `0.0.x`** β pre-1.0 API churn is not what you want under a flagship portfolio piece in v1. Revisit for a "real-time conversation" milestone once turn-based voice is solid. |
|
| 82 |
+
### Development Tools
|
| 83 |
+
| Tool | Purpose | Notes |
|
| 84 |
+
|---|---|---|
|
| 85 |
+
| **uv** | Dependency resolution + venv | Fast, and its lockfile makes the "works locally, breaks on Space rebuild" class of bug reproducible. Export to `requirements.txt` for the Space (Spaces do not read `uv.lock`). |
|
| 86 |
+
| **ruff** | Lint + format | Pre-commit hook. Prior GSD work in this account has been blocked by accumulated ruff debt β enforce from commit #1 here. |
|
| 87 |
+
| **pytest** | Unit tests | Focus on the viseme-timeline builder and the JLPT-level content selector β both are pure functions with high bug density. |
|
| 88 |
+
| **Playwright** | E2E | The avatar is a canvas; assert on `trigger()` events and audio playback state, not pixels. Screenshot-diff the VRM render at your own risk. |
|
| 89 |
+
| **`hf` CLI** | Space + bucket management | `hf buckets sync` for asset deploys. |
|
| 90 |
+
| **Space README front-matter** | Config | Must pin `sdk: gradio`, `sdk_version: 6.22.0`, `python_version: 3.12.12`, `hf_oauth: true`. |
|
| 91 |
+
## Authentication
|
| 92 |
+
# README.md front-matter
|
| 93 |
+
# no extra scopes needed for progress tracking β "openid profile" is always included
|
| 94 |
+
- Request **only** `openid profile` (always included) since progress lives in your own DB. Every extra scope is a consent-screen deterrent.
|
| 95 |
+
- Add the `inference-api` scope **only** if you implement the optional "spend my own HF credits" path (lets you call Inference Providers on the visitor's behalf β free users get ~$0.10/month of credits, PRO $2.00, so treat it as a nicety, not a tier).
|
| 96 |
+
- `hf_oauth_expiration_minutes` maxes at 43200 (30 days). Use it β re-authing daily kills a daily-habit app.
|
| 97 |
+
- β οΈ **`gr.LogoutButton` was removed in Gradio 6.** Build your own sign-out. (`gr.LoginButton` / `gr.OAuthProfile` / `gr.OAuthToken` are believed intact β **verify on first spike**, MEDIUM confidence.)
|
| 98 |
+
- β οΈ Use `target="_blank"` on the sign-in button, per HF docs, or third-party-cookie policies break the flow inside the Space iframe.
|
| 99 |
+
## Installation
|
| 100 |
+
# requirements.txt (Hugging Face Space)
|
| 101 |
+
# --- inference ---
|
| 102 |
+
# --- Japanese TTS (CPU, zero GPU quota) ---
|
| 103 |
+
# --- Japanese NLP ---
|
| 104 |
+
# --- persistence ---
|
| 105 |
+
# --- BYOK ---
|
| 106 |
+
# One-time, in the Space build (voicevox_core needs runtime + dict + voice models)
|
| 107 |
+
# The official downloader fetches onnxruntime, the Open JTalk dictionary, and .vvm voice models.
|
| 108 |
+
## Alternatives Considered
|
| 109 |
+
| Recommended | Alternative | When to Use Alternative |
|
| 110 |
+
|---|---|---|
|
| 111 |
+
| **VOICEVOX** (TTS) | **Qwen3-TTS-12Hz-0.6B / 1.7B** (Apache-2.0, Jan 2026, JA among 10 langs, voice cloning from 3s) | If VOICEVOX character terms turn out to prohibit third-party web synthesis, or you want a custom/cloned tutor voice. Cost: runs on GPU (burns visitor quota), and **gives you no phoneme timings** β you fall back to amplitude-driven lip-sync or add a forced aligner. |
|
| 112 |
+
| **VOICEVOX** (TTS) | **Style-Bert-VITS2** (`2.5.0`) | If you want maximum expressive/emotional Japanese prosody. Cost: **AGPL-3.0**, which would infect your whole app unless isolated behind a separate network service. For a public portfolio repo, that is a conversation you do not want to have in an interview. |
|
| 113 |
+
| **VOICEVOX** (TTS) | **Kokoro-82M** (Apache-2.0, `kokoro-js` 1.2.1 runs in-browser) | English-first fallback, or if you need TTS with literally zero server cost. **Rejected for Japanese**: its own VOICES.md grades the five JA voices **C- to C+** on <10 hours of training data and warns "support for non-English languages may be absent or thin due to weak G2P." A pronunciation-teaching app cannot ship C-grade Japanese. |
|
| 114 |
+
| **whisper-large-v3-turbo** (ASR) | **Qwen3-ASR-1.7B** | Best measured Japanese accuracy (CER 0.140 vs 0.184) β use if transcription errors are visibly hurting the correction/scoring feature. Cost: RTF 0.036 vs 0.013 (~3Γ slower), which matters when GPU-seconds are the visitor's scarcest resource. |
|
| 115 |
+
| **whisper-large-v3-turbo** (ASR) | **transformers.js Whisper in-browser (WebGPU)** | **Recommend shipping both.** Browser-side = zero quota, works while the Space is queued, and is a genuinely impressive "runs on your GPU" demo. Server-side = the accuracy fallback for weak devices / no-WebGPU browsers. |
|
| 116 |
+
| **whisper-large-v3-turbo** (ASR) | **nvidia/parakeet-tdt-0.6b-v3** | Only if latency is the binding constraint (RTF 0.003) and you can tolerate CER 0.321 β too error-prone to grade a learner's pronunciation against. |
|
| 117 |
+
| **Qwen3.5-4B** (LLM) | **Nemotron Nano 9B JP** (NVIDIA) | Ranked **#1 in the sub-10B category on Nejumi Leaderboard 4** (the Japanese LLM leaderboard). Swap in if Qwen3.5-4B's Japanese pedagogy/naturalness disappoints in evaluation. Cost: 2Γ the size (more GPU-seconds per turn) and the NVIDIA Open Model License rather than Apache-2.0. |
|
| 118 |
+
| **Qwen3.5-4B** (LLM) | **Qwen3.5-2B / 0.8B** | If per-turn GPU-seconds prove to be the binding UX constraint. Cheapest lever for stretching an anonymous visitor's 2-minute daily quota. |
|
| 119 |
+
| **Qwen3.5-4B** (LLM) | **LLM-jp-4** (32B MoE, 3.8B active, Apache-2.0, NII) | The Japanese-sovereign, research-credible choice β reportedly MT-Bench JA 7.82 vs GPT-4o's 7.29. MoE with 3.8B active params is surprisingly viable on a 48 GB Blackwell. Strong "I evaluated Japanese-specific models" story for a Google interview. Cost: 32B of weights to load = slow cold starts. |
|
| 120 |
+
| **Neon** (DB) | **Supabase** | If you later want its Auth, Realtime, or Storage as a bundle and will pay for Pro (which removes the 7-day pause). |
|
| 121 |
+
| **Gradio `gr.HTML`** | **Gradio custom component (`gradio cc`)** | If you decide to publish `gradio-vrm-avatar` as a reusable package β genuinely good portfolio surface area, but a Phase-N+1 concern. |
|
| 122 |
+
## What NOT to Use
|
| 123 |
+
| Avoid | Why | Use Instead |
|
| 124 |
+
|---|---|---|
|
| 125 |
+
| **CPU Basic as the free hosting plan** | Gradio Spaces on compute now require PRO for personal accounts. CPU Basic is not the free path, and its 2 vCPU could not run a 4B LLM anyway. | ZeroGPU Gradio Space (free for up to 2 Spaces on an account in good standing). |
|
| 126 |
+
| **Python 3.13** | ZeroGPU images provide **only** 3.10.13 and 3.12.12. This is the known "Python 3.13 / gradio pin" gotcha, and on ZeroGPU it is a hard wall, not a warning. | `python_version: 3.12.12` in README front-matter. |
|
| 127 |
+
| **Unpinned `sdk_version`** | Spaces silently rebuild on push. An unpinned Gradio floats you into a breaking release while you sleep β and Gradio 6 removed/renamed a lot (`show_api`β`api_visibility`, tuple chatbot messages, `gr.LogoutButton`, `mirror_webcam`, `cache_examples="lazy"`). | Pin `sdk_version: 6.22.0` and bump deliberately. |
|
| 128 |
+
| **Gradio 5-era tutorials and code** | Gradio 6 moved `theme`/`css`/`js`/`head` from the `Blocks()` constructor to `Blocks.launch()`, consolidated all `show_*_button` params into `buttons`, and **removed tuple-format chatbot messages entirely**. Copy-pasted v5 snippets will fail in non-obvious ways. | The [Gradio 6 migration guide](https://gradio.app/main/guides/gradio-6-migration-guide) as the primary reference. |
|
| 129 |
+
| **`torch.compile` on ZeroGPU** | Explicitly unsupported. | PyTorch **ahead-of-time** compilation (`torch >= 2.8`), documented in HF's zerogpu-aoti guide. Meaningful latency win = meaningful quota win. |
|
| 130 |
+
| **Lazy `.to('cuda')` inside `@spaces.GPU`** | Docs are explicit that CUDA transfers are optimized for startup placement; lazy loading is "significantly less efficient." Every wasted second is billed to your visitor's daily quota. | Load models to `cuda` at **module level** (a CUDA emulation layer makes this work outside GPU context). |
|
| 131 |
+
| **`pykakasi`** | **GPL-3.0-or-later.** For kana/romaji conversion β a trivially replaceable function β it would impose GPL on a flagship public portfolio repo. | `SudachiPy` (Apache-2.0) readings + `jaconv` (MIT) + `cutlet` for romaji. |
|
| 132 |
+
| **Style-Bert-VITS2 embedded in-process** | **AGPL-3.0.** Β§13 means network use triggers source disclosure for the embedding server. | VOICEVOX (LGPL + credit) or Qwen3-TTS (Apache-2.0). |
|
| 133 |
+
| **`edge-tts` / unofficial Microsoft endpoints** | Excellent Japanese voices, but it's an undocumented consumer endpoint used against its ToS, and it breaks without notice. A hiring manager finding a ToS-violating dependency in a flagship repo is a pure downside. | VOICEVOX. |
|
| 134 |
+
| **Web Speech API (`SpeechRecognition`) as primary ASR** | Chromium-only in practice, silently ships the learner's audio to Google, and returns no usable confidence/timing data β which is exactly the data pronunciation scoring needs. | transformers.js Whisper (browser) + whisper-large-v3-turbo (ZeroGPU). Web Speech is acceptable as a last-resort tertiary fallback only. |
|
| 135 |
+
| **kotoba-whisper (v1/v2)** | Fast (6.3Γ large-v3) and great in-domain, but the Feb-2026 benchmark flags it as struggling with **unscripted/natural conversation** β which is 100% of this app's input. | whisper-large-v3-turbo. |
|
| 136 |
+
| **SQLite on Space disk** | Space disk is ephemeral; every `git push` rebuilds and wipes it. | Neon Postgres. |
|
| 137 |
+
| **Supabase free tier for the user DB** | Auto-pauses after 7 days idle, requires manual dashboard unpause. Guaranteed to be paused exactly when someone finally clicks your portfolio link. | Neon (scale-to-zero, auto-resume in ~hundreds of ms). |
|
| 138 |
+
| **`fastrtc` in v1** | `0.0.34` β pre-1.0, active API churn, and it wraps the Gradio version you must pin. | Turn-based `gr.Audio(sources=["microphone"])`; adopt fastrtc in a later real-time milestone. |
|
| 139 |
+
| **AI talking-head video generation** | Already Out of Scope in PROJECT.md β and it would consume a visitor's entire daily GPU quota in a single utterance. | VRM + mora-timed visemes. |
|
| 140 |
+
## Stack Patterns by Variant
|
| 141 |
+
- Hardware: **ZeroGPU**, max 2 such Spaces on the account β budget that slot deliberately against the other 4 HF-profile projects.
|
| 142 |
+
- Design every turn to fit inside ~3β6 GPU-seconds so an anonymous visitor gets 20+ turns from 2 minutes.
|
| 143 |
+
- Push ASR to the browser by default; use ZeroGPU ASR only as fallback.
|
| 144 |
+
- Free Spaces sleep after inactivity β **expect a cold start on the recruiter's first click.** Make the loading state part of the show: render the VRM avatar immediately (it's client-side and needs no backend) with a "waking up" idle animation while the Python backend boots.
|
| 145 |
+
- Up to 10 ZeroGPU Spaces; the owner's own testing gets 40 min/day at highest queue priority.
|
| 146 |
+
- $2.00/month Inference Provider credits make an **Inference Providersβrouted default LLM** viable, removing cold-start and quota concerns entirely for light traffic.
|
| 147 |
+
- Visitor quotas are unchanged β PRO does not fix the anonymous visitor's 2 minutes. Design for free-tier visitors regardless.
|
| 148 |
+
- Swap to **Qwen3-TTS-12Hz-0.6B** (Apache-2.0) on ZeroGPU.
|
| 149 |
+
- You lose free phoneme timings β recover lip-sync by deriving the mora sequence from **SudachiPy readings β katakana β vowel sequence** and distributing it proportionally across the audio duration, refined by a Web Audio RMS envelope. Noticeably worse than VOICEVOX timings, still far better than pure amplitude flapping.
|
| 150 |
+
- Move the default LLM to **HF Inference Providers** routed with the *visitor's* OAuth token (`inference-api` scope) β their credits, their latency, no queue.
|
| 151 |
+
- Keep BYOK (Anthropic/OpenAI) as the premium path. This is the honest three-tier story: free/queued β your-HF-credits β your-frontier-key.
|
| 152 |
+
## Version Compatibility
|
| 153 |
+
| Package A | Compatible With | Notes |
|
| 154 |
+
|---|---|---|
|
| 155 |
+
| `gradio==6.22.0` | Python `>=3.10` (3.10β3.13 classifiers) | But **ZeroGPU narrows this to 3.10.13 / 3.12.12**. The platform is stricter than the package. |
|
| 156 |
+
| `spaces==0.51.1` | Python `>=3.10,<3.15` | Consistent with the above. |
|
| 157 |
+
| ZeroGPU | `torch` **2.8.0 β 2.11.0**, Gradio **4+** | Pin torch explicitly; a transitive bump outside this range breaks GPU allocation. |
|
| 158 |
+
| ZeroGPU | **Gradio SDK only** | Not Docker, not Static. This makes Gradio non-negotiable, which conveniently matches the portfolio requirement. |
|
| 159 |
+
| `@pixiv/three-vrm@3.5.5` | `three >= 0.137` (peer) | Verified against `three@0.185.1`. Load both from the *same* CDN origin via importmap β mismatched three.js instances cause the classic "multiple instances of three.js imported" breakage. |
|
| 160 |
+
| `@pixiv/three-vrm-animation@3.5.5` | same peer range | Only needed if you use `.vrma` animation clips for idle/gesture motion. Recommended for idle breathing/blinking. |
|
| 161 |
+
| `transformers==5.14.1` | `huggingface_hub>=1.x` | Both are v-major releases from the last cycle; **v4-era `transformers` snippets will not run**. |
|
| 162 |
+
| `voicevox_core` 0.16.4 `cp310-abi3` wheel | Python `>=3.10` incl. 3.12 | `abi3` wheels are forward-compatible across minor versions β the `cp310` tag is not a 3.10-only restriction. Linux wheel: `manylinux_2_34_x86_64`. |
|
| 163 |
+
| `SudachiPy 0.6.11` | `SudachiDict-core 20260723` | Dict packages are date-versioned and must match the SudachiPy major line; pin both together. |
|
| 164 |
+
| Neon Postgres | `psycopg[binary]` 3.x | Requires TLS + SNI. Use the pooled endpoint; Spaces restart often and unpooled connections leak. |
|
| 165 |
+
## Open Questions for the Roadmap
|
| 166 |
+
## Sources
|
| 167 |
+
- [HF Spaces Overview](https://huggingface.co/docs/hub/spaces-overview) β hardware tiers, PRO requirement for Gradio/Docker Spaces, ephemeral disk, sleep behavior, built-in env vars
|
| 168 |
+
- [HF Spaces ZeroGPU](https://huggingface.co/docs/hub/spaces-zerogpu) β RTX Pro 6000 Blackwell, Gradio-only compatibility, torch 2.8β2.11, Python 3.10.13/3.12.12, per-visitor daily quotas, `@spaces.GPU`, no `torch.compile`
|
| 169 |
+
- [HF Spaces OAuth](https://huggingface.co/docs/hub/spaces-oauth) β `hf_oauth` metadata, scopes, expiration limits, `target=_blank` caveat
|
| 170 |
+
- [HF Spaces disk usage](https://huggingface.co/docs/hub/spaces-storage) + [Storage Buckets](https://huggingface.co/docs/hub/storage-buckets) β ephemerality, bucket volumes
|
| 171 |
+
- [HF Inference Providers pricing](https://huggingface.co/docs/inference-providers/pricing) β $0.10 free / $2.00 PRO monthly credits, OpenAI-compatible router
|
| 172 |
+
- [HF pricing](https://huggingface.co/pricing) β PRO $9/mo
|
| 173 |
+
- [Gradio 6 migration guide](https://gradio.app/main/guides/gradio-6-migration-guide) β breaking changes, removals
|
| 174 |
+
- [Gradio custom HTML components](https://gradio.app/main/guides/custom-HTML-components) + [HF blog: gr.HTML one-shot apps](https://huggingface.co/blog/gradio-html-one-shot-apps) β `html_template`, `head`, `js_on_load`, `server_functions`, three.js precedent
|
| 175 |
+
- [VRM 1.0 expressions spec](https://github.com/vrm-c/vrm-specification/blob/master/specification/VRMC_vrm-1.0/expressions.md) β `aa`/`ih`/`ou`/`ee`/`oh` presets
|
| 176 |
+
- [VOICEVOX core releases](https://github.com/VOICEVOX/voicevox_core/releases) β 0.16.4, manylinux abi3 CPU wheel
|
| 177 |
+
- [Kokoro-82M VOICES.md](https://huggingface.co/hexgrad/Kokoro-82M/blob/main/VOICES.md) β Japanese voice grades C- to C+
|
| 178 |
+
- [Qwen3.5-4B model card](https://huggingface.co/Qwen/Qwen3.5-4B) β Apache-2.0, 201 languages, 256K context, thinking mode
|
| 179 |
+
- PyPI/npm registry APIs queried directly for every pinned version in this document
|
| 180 |
+
- [Japanese ASR benchmark, Feb 2026 (Neosophie)](https://neosophie.com/en/blog/20260226-japanese-asr-benchmark) β CER/WER/RTF table; single benchmark, not independently replicated
|
| 181 |
+
- [Japanese LLMs compared, Apr 2026 (lilting.ch)](https://lilting.ch/en/articles/japanese-llm-options-compared) β LLM-jp-4, Nemotron Nano 9B JP, PLaMo, Swallow, Namazu; Nejumi 4 rankings quoted secondhand
|
| 182 |
+
- [VOICEVOX engine API reference (DeepWiki)](https://deepwiki.com/VOICEVOX/voicevox_engine/4.1-tts-pipeline-api) β AudioQuery/mora structure
|
| 183 |
+
- VOICEVOX commercial-use and credit requirements β Japanese-language secondary sources; **must be confirmed against the official ε©η¨θ¦η΄ per character**
|
| 184 |
+
- [Transformers.js v4 release](https://huggingface.co/blog/transformersjs-v4) β WebGPU runtime rewrite
|
| 185 |
+
- Neon vs Supabase free-tier behavior β multiple 2026 sources agree, vendor docs corroborate scale-to-zero
|
| 186 |
+
<!-- GSD:stack-end -->
|
| 187 |
+
|
| 188 |
+
<!-- GSD:conventions-start source:CONVENTIONS.md -->
|
| 189 |
+
## Conventions
|
| 190 |
+
|
| 191 |
+
Conventions not yet established. Will populate as patterns emerge during development.
|
| 192 |
+
<!-- GSD:conventions-end -->
|
| 193 |
+
|
| 194 |
+
<!-- GSD:architecture-start source:ARCHITECTURE.md -->
|
| 195 |
+
## Architecture
|
| 196 |
+
|
| 197 |
+
Architecture not yet mapped. Follow existing patterns found in the codebase.
|
| 198 |
+
<!-- GSD:architecture-end -->
|
| 199 |
+
|
| 200 |
+
<!-- GSD:workflow-start source:GSD defaults -->
|
| 201 |
+
## GSD Workflow Enforcement
|
| 202 |
+
|
| 203 |
+
Before using Edit, Write, or other file-changing tools, start work through a GSD command so planning artifacts and execution context stay in sync.
|
| 204 |
+
|
| 205 |
+
Use these entry points:
|
| 206 |
+
- `/gsd:quick` for small fixes, doc updates, and ad-hoc tasks
|
| 207 |
+
- `/gsd:debug` for investigation and bug fixing
|
| 208 |
+
- `/gsd:execute-phase` for planned phase work
|
| 209 |
+
|
| 210 |
+
Do not make direct repo edits outside a GSD workflow unless the user explicitly asks to bypass it.
|
| 211 |
+
<!-- GSD:workflow-end -->
|
| 212 |
+
|
| 213 |
+
|
| 214 |
+
|
| 215 |
+
<!-- GSD:profile-start -->
|
| 216 |
+
## Developer Profile
|
| 217 |
+
|
| 218 |
+
> Profile not yet configured. Run `/gsd:profile-user` to generate your developer profile.
|
| 219 |
+
> This section is managed by `generate-claude-profile` -- do not edit manually.
|
| 220 |
+
<!-- GSD:profile-end -->
|