.planning/PROJECT.md

# Japanese Learning Avatar

## What This Is

An animated 3D avatar tutor that teaches Japanese through spoken conversation, structured lessons, and immersion role-play — hosted as a Hugging Face Space. A VRM anime-style character listens to the learner speak Japanese, responds aloud with lip-synced speech and expressions, drills vocab and grammar, and adapts to the learner's level (N5 through N2). Built both as a daily-use learning tool for the author and as the flagship AI/ML engineering showcase on the WolfDavid Hugging Face profile.

## Core Value

A learner can hold a real, level-appropriate spoken Japanese conversation with an animated avatar that talks back — and measurably improve over time because the avatar remembers them.

## Requirements

### Validated

(None yet — ship to validate)

### Active

- [ ] Animated VRM 3D avatar rendered in-browser with lip-sync and expressions
- [ ] Avatar speaks Japanese aloud (TTS) synchronized to mouth animation
- [ ] Learner can speak Japanese (speech recognition transcribes and scores)
- [ ] Free-form conversation practice with corrections and level-appropriate replies
- [ ] Structured lessons/drills: kana, vocab, grammar with progression
- [ ] Immersion role-play scenarios (ordering food, directions, etc.) with the avatar playing the counterpart
- [ ] Adaptive difficulty spanning JLPT N5 → N2 (assesses and adjusts)
- [ ] Hybrid AI backend: small open model as free default + optional BYOK frontier model (Claude/OpenAI)
- [ ] Real user accounts with server-persistent progress (level, vocab seen, mistakes)
- [ ] Deployed as a public Hugging Face Space on the WolfDavid profile

### Out of Scope

- Mobile native app — web-first on HF Spaces; mobile-responsive is enough
- Languages other than Japanese — single-language focus keeps content quality high
- AI-generated talking-head video — GPU-heavy and slow vs. real-time VRM rendering
- Community/social features (leaderboards, friends) — not core to tutoring value

## Context

- Lives at `hugginface_profile/japanese-learning-avatar` as its own git repo, part of the 5-project HF profile strategy targeting Google AI/ML roles (this is the Gradio/Spaces demo slot, and can showcase fine-tuning/MLOps skills via the hybrid backend).
- Dual purpose: the author genuinely wants to learn Japanese with it (both-equally answer), so real pedagogical quality matters, not just demo polish.
- HF Spaces constraints matter: free CPU tier for baseline experience, Gradio supports "Sign in with Hugging Face" OAuth natively — the natural auth path for accounts. Server-side progress needs a hosted DB (e.g. Supabase/Postgres or Space persistent storage); research phase to decide.
- VRM/VRoid characters are freely available and render in-browser via three.js + @pixiv/three-vrm — real lip-sync and expression control without GPU inference.
- Full-duplex voice: Japanese TTS out (with viseme/lip-sync data) and speech recognition in (e.g. Whisper-class model) — model choices to be researched.
- Known HF profile gotchas from prior Spaces: Python 3.13 / Gradio version-pin issues (see prior audit work).

## Constraints

- **Platform**: Hugging Face Space (Gradio) — portfolio strategy requires it; free-tier visitors must get a working experience
- **Cost**: Free baseline for visitors — small open model default; frontier quality only via visitor's own API key (BYOK)
- **Tech**: In-browser avatar rendering (three.js/VRM) — no server-side video generation
- **Quality bar**: Flagship portfolio piece — "would a Google AI/ML hiring manager be impressed?" drives polish decisions

## Key Decisions

| Decision | Rationale | Outcome |
|----------|-----------|---------|
| VRM 3D avatar over 2D sprites or talking-head video | Real lip-sync + expressions in-browser, impressive but well-trodden tech | — Pending |
| Hybrid AI backend (small open model + BYOK) | Free for everyone, showcases both fine-tuning/deployment and frontier integration | — Pending |
| Full-duplex voice in v1 | Speaking practice is the core of language learning; differentiates from chat-only tutors | — Pending |
| Adaptive N5→N2 rather than beginner-only | User wants the real product; content system must be level-parameterized from day one | — Pending |
| Real accounts + server DB over browser-local storage | Persistent progress is core value ("measurably improve over time"); HF OAuth keeps auth light | — Pending |

## Evolution

This document evolves at phase transitions and milestone boundaries.

**After each phase transition** (via `/gsd:transition`):
1. Requirements invalidated? → Move to Out of Scope with reason
2. Requirements validated? → Move to Validated with phase reference
3. New requirements emerged? → Add to Active
4. Decisions to log? → Add to Key Decisions
5. "What This Is" still accurate? → Update if drifted

**After each milestone** (via `/gsd:complete-milestone`):
1. Full review of all sections
2. Core Value check — still the right priority?
3. Audit Out of Scope — reasons still valid?
4. Update Context with current state

---
*Last updated: 2026-08-08 after initialization*