README.md

metadata

title: Japanese Learning Avatar
emoji: 🗾
colorFrom: pink
colorTo: indigo
sdk: gradio
sdk_version: 6.22.0
python_version: 3.12.12
app_file: app.py
pinned: false
license: mit
short_description: Talk to a lip-synced VRM avatar in Japanese

Japanese Learning Avatar

An animated 3D avatar tutor that teaches Japanese through spoken conversation. A VRM anime-style character listens to you speak Japanese and answers aloud, with lip-sync driven by real per-mora phoneme timings rather than by microphone-style mouth flapping.

The avatar is rendered in your browser with three.js and @pixiv/three-vrm. Nothing about the character is generated server-side, so it paints and starts breathing while the Python backend is still waking up.

Scope of this phase

Phase 1 has no AI tutoring in it yet. This phase exists to prove the transport — that a VRM avatar can live inside a Gradio gr.HTML component on a real Hugging Face Space, that speech can be synthesised with per-mora timings, and that the microphone round trip works without spending any GPU quota. The tutoring brain, level gating and progress tracking come in later phases.

What is live here:

• A VRM avatar with idle life — blinking, breathing, a slight sway.
• Japanese speech synthesis with mora-accurate visemes (VOICEVOX, CPU only).
• Push-to-talk capture with in-browser speech recognition (Whisper via transformers.js), which costs zero ZeroGPU quota and therefore does not eat into a visitor's daily allowance.

Run it locally

uv sync --extra dev
uv run python app.py

Requires Python 3.12.12 (see .python-version). The Space itself is pinned to Gradio 6.22.0 and Python 3.12.12 — ZeroGPU only provides 3.10.13 and 3.12.12, so those pins are a platform constraint, not a preference.

Tests:

uv run pytest tests/ -q --ignore=tests/e2e   # quick loop
uv run pytest tests/e2e/ -q                  # browser suite (run it whole, never per-file)

Credits

Credits and third-party licences are recorded in docs/ASSETS.md and docs/VOICEVOX-SETUP.md, and are consolidated into LICENSES.md before the phase ships.

Licence

MIT for this repository's own code. Third-party assets and models carry their own terms — see above.