## Project **Japanese Learning Avatar** An animated 3D avatar tutor that teaches Japanese through spoken conversation, structured lessons, and immersion role-play — hosted as a Hugging Face Space. A VRM anime-style character listens to the learner speak Japanese, responds aloud with lip-synced speech and expressions, drills vocab and grammar, and adapts to the learner's level (N5 through N2). Built both as a daily-use learning tool for the author and as the flagship AI/ML engineering showcase on the WolfDavid Hugging Face profile. **Core Value:** A learner can hold a real, level-appropriate spoken Japanese conversation with an animated avatar that talks back — and measurably improve over time because the avatar remembers them. ### Constraints - **Platform**: Hugging Face Space (Gradio) — portfolio strategy requires it; free-tier visitors must get a working experience - **Cost**: Free baseline for visitors — small open model default; frontier quality only via visitor's own API key (BYOK) - **Tech**: In-browser avatar rendering (three.js/VRM) — no server-side video generation - **Quality bar**: Flagship portfolio piece — "would a Google AI/ML hiring manager be impressed?" drives polish decisions ## Technology Stack ## ⚠️ Read This First: Two Findings That Reshape the Project ### 1. "Free CPU tier" is no longer the free path for a Gradio Space ### 2. ZeroGPU quota is consumed by the *visitor*, not the owner | Visitor account type | Included daily GPU quota | Queue priority | |---|---|---| | Unauthenticated | **2 minutes/day** | Low | | Free HF account | **5 minutes/day** | Medium | | PRO account | 40 minutes/day (extensible) | Highest | ## Recommended Stack ### Core Technologies | Technology | Version | Purpose | Why Recommended | |---|---|---|---| | **Gradio** | `6.22.0` (pin exactly) | App framework + Space SDK | Gradio 6 is the only maintained major line; the team has stated only v6 gets future releases. Crucially, **ZeroGPU is exclusively compatible with the Gradio SDK** — no Docker, no Static. Gradio 6 also ships the `gr.HTML` templating system (below) that makes the three.js avatar viable without a build step. | | **Python** | `3.12.12` (declare in README) | Runtime | ZeroGPU provides **only** Python `3.10.13` and `3.12.12`. Pick 3.12.12. **Do not use 3.13** — this is the recurring "Python 3.13 / Gradio pin" gotcha from prior Spaces work, and here it's a hard platform limit, not a soft one (`spaces` requires `>=3.10,<3.15`, but ZeroGPU only builds those two images). | | **`spaces`** | `0.51.1` | ZeroGPU allocation | Provides `@spaces.GPU(duration=...)`. Effect-free off-ZeroGPU, so local dev is unaffected. Use **dynamic durations** (`duration=callable`) — shorter declared durations improve queue priority for your visitors. | | **three.js** | `0.185.1` (MIT) | 3D renderer | The only realistic in-browser 3D engine with first-class VRM support. Pin the exact version — three.js makes breaking renderer changes at a ~6-week cadence and `three-vrm` tracks it loosely. | | **@pixiv/three-vrm** | `3.5.5` (MIT) | VRM avatar loading, humanoid bones, expressions, spring-bone physics | The reference implementation from the company that runs VRoid Hub. Supports both VRM 0.0 and VRM 1.0, so community VRoid models load without conversion. Peer dep is `three >= 0.137`, so 0.185.1 is safe. | | **VOICEVOX CORE** | `0.16.4` (`+cpu` abi3 wheel) | Japanese TTS **with per-mora phoneme timing** | See "The lip-sync decision" below. This is the highest-leverage choice in the whole stack. | | **whisper-large-v3-turbo** | via `transformers` 5.x on ZeroGPU | Japanese ASR | Best measured accuracy/speed balance for Japanese conversational audio in the Feb-2026 benchmark: CER 0.184, **RTF 0.013**. MIT-licensed. | | **Qwen3.5-4B** | Apache-2.0, released 2026-03-02 | Free-default tutor LLM | 4B dense, **201 languages incl. Japanese**, 256K context, hybrid thinking (disable thinking for latency). Apache-2.0 means no license story to explain to a hiring manager. Fits easily in 48 GB with room for KV cache. | | **`transformers`** | `5.14.1` | Model loading | v5 is current. Note: v5 was a breaking release — do not copy v4-era snippets from blogs. | | **`huggingface_hub`** | `1.27.0` | Hub + Inference Providers client | v1.x. Provides `InferenceClient` for the BYOK/routed paths. | | **Neon Serverless Postgres** | Free plan | User accounts + progress persistence | See "The database decision" below. | ### The lip-sync decision (do not skip this) - **No audio-amplitude analysis.** ChatVRM and most AITuber kits drive the `aa` blendshape from a Web Audio `AnalyserNode` RMS — the mouth just flaps open and shut. Mora-timed visemes are visibly better and are the thing that makes the demo look expensive. - **No forced alignment, no viseme ML model, no GPU.** - **Zero ZeroGPU quota consumed for speech output**, which is what makes the free tier survivable. - The same mora timeline doubles as **pedagogical data**: mora-by-mora pitch accent is already in the payload, so "here is where your pitch accent was wrong" is nearly free. - `voicevox_core` is dual-licensed **LGPL v3** + a commercial no-source-disclosure license. Dynamic linking via the Python wheel keeps you clear of copyleft on your own code. - **Each character voice has its own terms.** The general rule is *free for commercial and non-commercial use provided you display credit* in the form `VOICEVOX:キャラクター名`. Without credit, per-character paid licensing is reportedly ¥400,000/character. - **Read the specific character's terms before shipping**, and render the credit string persistently in the UI (a footer line next to the avatar). Prefer a character with permissive, well-documented terms (ずんだもん / VOICEVOX Nemo voices are the usual safe picks). - Also verify the terms permit *offering synthesis to third parties via a web app*, not merely using generated audio — this is the one clause a portfolio Space most plausibly trips. ### The avatar-embedding decision | Approach | Verdict | |---|---| | `gr.HTML` with `html_template`/`js_on_load` | ✅ **Recommended.** No build step, no npm publish, hot-reloads with the app, `server_functions` gives direct JS→Python calls, `watch('value')` gives Python→JS push. Ships as one `app.py`. | | Full Gradio custom component (`gradio cc`) | ❌ Requires Svelte, Node 18+, npm 9+, a build/publish cycle, and a separate package on PyPI. Justified only if you intend to publish the VRM component for others — a nice-to-have, not a v1 need. | | Raw `