.planning/PROJECT.md

Japanese Learning Avatar

What This Is

An animated 3D avatar tutor that teaches Japanese through spoken conversation, structured lessons, and immersion role-play — hosted as a Hugging Face Space. A VRM anime-style character listens to the learner speak Japanese, responds aloud with lip-synced speech and expressions, drills vocab and grammar, and adapts to the learner's level (N5 through N2). Built both as a daily-use learning tool for the author and as the flagship AI/ML engineering showcase on the WolfDavid Hugging Face profile.

Core Value

A learner can hold a real, level-appropriate spoken Japanese conversation with an animated avatar that talks back — and measurably improve over time because the avatar remembers them.

Requirements

Validated

(None yet — ship to validate)

Active

• Animated VRM 3D avatar rendered in-browser with lip-sync and expressions
• Avatar speaks Japanese aloud (TTS) synchronized to mouth animation
• Learner can speak Japanese (speech recognition transcribes and scores)
• Free-form conversation practice with corrections and level-appropriate replies
• Structured lessons/drills: kana, vocab, grammar with progression
• Immersion role-play scenarios (ordering food, directions, etc.) with the avatar playing the counterpart
• Adaptive difficulty spanning JLPT N5 → N2 (assesses and adjusts)
• Hybrid AI backend: small open model as free default + optional BYOK frontier model (Claude/OpenAI)
• Real user accounts with server-persistent progress (level, vocab seen, mistakes)
• Deployed as a public Hugging Face Space on the WolfDavid profile

Out of Scope

• Mobile native app — web-first on HF Spaces; mobile-responsive is enough
• Languages other than Japanese — single-language focus keeps content quality high
• AI-generated talking-head video — GPU-heavy and slow vs. real-time VRM rendering
• Community/social features (leaderboards, friends) — not core to tutoring value

Context

• Lives at hugginface_profile/japanese-learning-avatar as its own git repo, part of the 5-project HF profile strategy targeting Google AI/ML roles (this is the Gradio/Spaces demo slot, and can showcase fine-tuning/MLOps skills via the hybrid backend).
• Dual purpose: the author genuinely wants to learn Japanese with it (both-equally answer), so real pedagogical quality matters, not just demo polish.
• HF Spaces constraints matter: free CPU tier for baseline experience, Gradio supports "Sign in with Hugging Face" OAuth natively — the natural auth path for accounts. Server-side progress needs a hosted DB (e.g. Supabase/Postgres or Space persistent storage); research phase to decide.
• VRM/VRoid characters are freely available and render in-browser via three.js + @pixiv/three-vrm — real lip-sync and expression control without GPU inference.
• Full-duplex voice: Japanese TTS out (with viseme/lip-sync data) and speech recognition in (e.g. Whisper-class model) — model choices to be researched.
• Known HF profile gotchas from prior Spaces: Python 3.13 / Gradio version-pin issues (see prior audit work).

Constraints

• Platform: Hugging Face Space (Gradio) — portfolio strategy requires it; free-tier visitors must get a working experience
• Cost: Free baseline for visitors — small open model default; frontier quality only via visitor's own API key (BYOK)
• Tech: In-browser avatar rendering (three.js/VRM) — no server-side video generation
• Quality bar: Flagship portfolio piece — "would a Google AI/ML hiring manager be impressed?" drives polish decisions

Key Decisions

Decision	Rationale	Outcome
VRM 3D avatar over 2D sprites or talking-head video	Real lip-sync + expressions in-browser, impressive but well-trodden tech	— Pending
Hybrid AI backend (small open model + BYOK)	Free for everyone, showcases both fine-tuning/deployment and frontier integration	— Pending
Full-duplex voice in v1	Speaking practice is the core of language learning; differentiates from chat-only tutors	— Pending
Adaptive N5→N2 rather than beginner-only	User wants the real product; content system must be level-parameterized from day one	— Pending
Real accounts + server DB over browser-local storage	Persistent progress is core value ("measurably improve over time"); HF OAuth keeps auth light	— Pending

Evolution

This document evolves at phase transitions and milestone boundaries.

After each phase transition (via /gsd:transition):

1. Requirements invalidated? → Move to Out of Scope with reason
2. Requirements validated? → Move to Validated with phase reference
3. New requirements emerged? → Add to Active
4. Decisions to log? → Add to Key Decisions
5. "What This Is" still accurate? → Update if drifted

After each milestone (via /gsd:complete-milestone):

1. Full review of all sections
2. Core Value check — still the right priority?
3. Audit Out of Scope — reasons still valid?
4. Update Context with current state

---

Last updated: 2026-08-08 after initialization