docs/LATENCY.md

Turn latency (VOIC-05)

Measured: 2026-09-06T04:32:30Z
Space: WolfDavid/japanese-learning-avatar Revision: c911d74b378981e6ba0d0111eb570ed3b20927db URL measured: https://wolfdavid-japanese-learning-avatar.hf.space
Hardware: zero-a10g (stage RUNNING) DISABLE_GPU: 1
Browser: Chromium 151.0.7922.34 (Playwright, headless) Client: Windows 11, public internet, one visitor
Method: N=30 warm text turns after 1 throwaway warm-up (こんにちは), 5 replay turns, five sentences of 5–36 moras in a fixed cycle (tests/e2e/test_latency_harness.py). Measured from the page's own performance.mark entries; server stages from __debug.lastStageTimings.
p95 computed by nearest-rank (the ceil(0.95·N)-th smallest observed value; p50 likewise), so no percentile is an interpolated value that was never observed.

Generated by the harness; the Cold start, Mobile and Lip-sync sections are the owner's and are carried forward verbatim when the harness re-runs.

Warm turns (N=30)

Stage	p50 (ms)	p95 (ms)	min	max
dispatch -> speech-start (THE NUMBER)	7107	10713	4589	10858
dispatch -> response	6829	10443	4326	10592
server: audio_query	1	2	1	2
server: synthesis	6422	10118	4134	10120
server: timeline	0	1	0	1
server: encode	0	1	0	1
server: total	6424	10121	4136	10124

Per sentence (dispatch -> speech-start, ms; each sentence was spoken 6 times):

#	Sentence	Moras	p50	min	max	synthesis p50
1	こんにちは	5	6417	4589	6706	5860
2	はじめまして、よろしくお願いします。	17 (hand)	8142	5572	10677	7510
3	今日はいい天気ですから、公園を散歩してから、買い物に行きました。	36	10207	9634	10858	9653
4	駅はどこですか。	8 (hand)	6828	6674	7113	6224
5	日本語を勉強しています。	14 (hand)	6365	4872	8437	5835

Replay (N=5, client-side, zero network)

Stage	p50 (ms)	p95 (ms)	min	max
dispatch -> speech-start	0	0	0	0

Replay p50 0 ms vs warm-turn p50 7107 ms: the cache is free, the round trip is the cost.

Cold start

Measurement	Value	How obtained
Space PAUSED -> first HTTP 200 serving the Gradio app	18.0 s after restart_space — stage BUILDING at +0.2 s, APP_STARTING at +2.2 s, RUNNING at +18.6 s	Hub API: HfApi.pause_space() (stage PAUSED 0.1 s later; the URL answered 503, 51 bytes, while paused), then HfApi.restart_space(factory_reboot=False); GET / polled every 2 s until a 200 whose body names gradio. Method: pause + restart via the API, not a factory rebuild
Page load -> avatar ready -> first rendered frame, first visit after the restart	ready +38.1 s, first rendered frame +39.6 s from the restart (19.4 s / 20.8 s after navigation, which began at +18.7 s)	headless Chromium 151 navigated the moment the first 200 arrived; AVATAR_READY, then breathValue !== 0; threeInstanceCount 1, armDown -0.932 / -0.932, mountCount 1, transport inline
First turn after boot (typed こんにちは -> speech-start)	7.2 s after Enter (+46.9 s from the restart); lastTurnMs 6600, synthesis_ms 6076; the second turn in the same session: 6368 / 5796	the first typed turn on the freshly restarted container. The synthesiser is loaded at app start, so the first turn pays no model load — the plan's "synthesiser load dominates" prediction did not materialise; the first turn is the same CPU synthesis as every warm turn
Page load -> avatar ready (client-side, no backend), Space already RUNNING	harness run: ready 4.2 s, first rendered frame 5.7 s; lip-sync run (headed, GPU): ready 6.7 s; mobile-emulation runs 9.9–12.4 s (SwiftShader at 2.6–3x DPR)	wait_for_avatar_ready; plan 01-05 recorded 4.1–8.3 s warm with tutor.vrm 3.0–6.8 s of it, 01-09 4.0–16.8 s; Space rebuild-to-RUNNING 106 s (01-05); warm wake 0.2–2.7 s

Revision live during the cold-start test: b4d182ba9d4ccf77c9910375c9842ddf29496a22 (read from the Hub before the pause, during the restart and after: unchanged). Measured 2026-09-06T12:09:04Z; performed by the execution agent through the Hub API on the owner's instruction ("yes to all five"), not from the Space Settings page; the Space was confirmed RUNNING afterwards and left that way. Raw record with every stage transition and HTTP poll: docs/evidence/2026-09-06-cold-start.json.

What this is and is not: a pause/restart reuses the built image, which is why it reaches RUNNING in 18 s where the plan 01-05 rebuild took 106 s. The 48-hour SLEEPING -> wake path (what a first visitor gets after two idle days) was not forced and remains unmeasured as such; per 01-RESEARCH.md "cold = SLEEPING or manually paused/restarted", the restart is the sanctioned proxy. The recruiter-facing sum on this path is ~40 s to a moving avatar and ~47 s to the first spoken reply from a paused container, of which the browser's own VRM load (19–21 s here over the public internet, incl. the 10.3 MiB tutor.vrm) is the larger half.

Mobile

EMULATED — real-device verification still open. No real phone was available; these rows are Playwright device descriptors driven by the execution agent on the owner's instruction, and they say nothing about a real mobile GPU, thermals or the mobile microphone-permission flow. AVTR-01's manual row therefore stays open and the requirement is not marked on this evidence. Space revision b4d182ba9d4ccf77c9910375c9842ddf29496a22, 2026-09-06T12:32:52Z; raw record docs/evidence/2026-09-06-mobile-emulation.json; screenshots docs/evidence/2026-09-06-mobile-emulated-*.png.

Device (EMULATED)	OS	Browser	URL	Renders?	Observed smoothness	Push-to-talk	Turn latency	Page -> ready / first frame
Playwright Pixel 7 descriptor (412x839 CSS px, DPR 2.625, touch, mobile UA)	Android 14 UA string	Chromium 151.0.7922.34 headless, WebGL 2 via SwiftShader (no GPU)	direct hf.space	yes — ready, first frame, threeInstanceCount 1, mountCount 1, armDown -0.932 / -0.932, canvas 324x480 CSS (648x960 buffer)	7.7 fps over 5 s (rAF = getDebug samples) — a slideshow, but that is the software rasteriser on this laptop, not a phone GPU	not tested (mobile mic permission needs a device)	typed こんにちは -> speech-start 8.0 s after Enter; lastTurnMs 6612, synthesis_ms 5694; mouth peaks aa/ih/oh 1.0; audio 1.056 s	4.5 s / 6.0 s
same Pixel 7 descriptor	Android 14 UA	same Chromium	huggingface.co Space page (inside the Hub's iframe.space-iframe, ?__theme=system)	yes — same numbers read inside the iframe: 1 instance, mount 1, arms posed, "Running on ZERO" badge above the avatar	7.3 fps (SwiftShader)	not tested	speech-start 8.8 s after Enter; lastTurnMs 7354, synthesis_ms 6012; turn completed	5.7 s / 8.0 s
Playwright iPhone 14 descriptor (390x664, DPR 3, touch, iOS UA)	iOS UA string	WebKit 26.5, Playwright's Windows build — WebGL 2 present ("Apple GPU"), no Web Audio API (AudioContext and webkitAudioContext both undefined)	direct and Hub page	NO — not a device result. Boot aborts before the stage mounts: avatar boot failed: TypeError: undefined is not a constructor (evaluating 'new AudioCtor()'), status "the avatar failed to load - see the browser console", blank canvas (…iphone14-webkit-bootfail.png)	—	—	—	—

What the emulation does and does not establish:

• Established on the Android/Chromium profile: the responsive layout stacks the canvas above the controls at a phone width; the VRM, the idle life and a full typed turn (server round trip, decode, lip-synced playback) work at mobile viewport, DPR and user agent, both on the direct URL and embedded in the Hub page. The Hub page swaps its iframe once while resolving the Space (a frame found too early detaches), and the Hub's own script logs one getBoundingClientRect error at that moment — neither is the app's.
• Not established: frame rate on a phone (SwiftShader's 7–8 fps is this laptop's CPU rasterising a 648x960 buffer; a real Adreno/Mali/Apple GPU is a different measurement), warmth, battery, the mobile mic-permission and push-to-talk flow, and anything about Safari: the iPhone rows are a host limitation, not an iOS result. Real iOS Safari 14.5+ ships AudioContext, so the boot is expected to succeed there — but it is expected, not observed.
• Finding for the product (deferred, deferred-items.md): the boot constructs the AudioContext before the stage mounts (avatar/avatar.js:58-59), so a browser without Web Audio gets a blank canvas even though rendering needs no audio. Constructing it lazily on the first turn or mic press would let the avatar render everywhere and only degrade sound.

To close the row: open https://huggingface.co/spaces/WolfDavid/japanese-learning-avatar on a real phone; record device, OS version, browser version, renders / smooth-or-slideshow, warmth, whether the mic permission and push-to-talk worked, and a rough turn time; then mark AVTR-01.

Audio on the phone (01-HUMAN-UAT gap 1, fixed in plan 01-11 — retest after the redeploy). The owner's first phone visit (2026-09-06, revision b4d182b) rendered but was silent: the only AudioContext.resume() on that build ran inside playBuffer, seconds after the tap, and a gesture-gated browser refuses it. Reproduced in Chromium with the Space inside a cross-origin frame under --autoplay-policy=user-gesture-required (Chrome-on-Android's policy, and the Hub page's embed): the resume was refused 8.5 s after the tap and the page stayed on "thinking…" (docs/evidence/2026-09-06-autoplay-embed-b4d182b.txt). Plan 01-11 resumes the context inside every tap. After the push:

1. On the phone, open the Hub page, make sure the ring/silent switch is OFF (on iOS it mutes Web Audio entirely and looks identical to this bug) and the media volume is up.
2. Tap Say hello once, before anything else. Expect the greeting to be audible within ~10 s and the status line to go "thinking…" → "speaking…" → "ready".
3. Then a typed turn and a Replay. Both should be audible without another unlock.
4. If it stays on "thinking…", or the status reads "audio is blocked by the browser - tap Say hello or Send once, then try again", record device / OS / browser and the status text here. From a laptop, python scripts/embed_autoplay_probe.py re-runs the cross-origin embed scenario against the live Space and prints whether speech-end arrived.

Lip-sync verification (AVTR-02)

Reviewed from captured frames and viseme telemetry by the execution agent, not by the owner (the owner asked for the manual rows to be performed on their behalf; this section is that review and says so). Space revision b4d182ba9d4ccf77c9910375c9842ddf29496a22, 2026-09-06T12:16:28Z, headed Chromium 151 (real GPU, so the render rate is a visitor's, not SwiftShader's), viewport 800x600 at device-scale 2.

Sentence (113 chars, 211 timeline events, decoded audio 20.565 s): こんにちは。今日はいい天気ですね。昨日、駅の近くの喫茶店で、友達とコーヒーを飲みました。ちょっと高かったですが、とても美味しかったです。明日は、きっと図書館で日本語を勉強します。それから、うちの近くの公園を、ゆっくり歩きます。 — chosen to carry ん (こんにちは, 天気, 飲み, 勉強, 公園), っ (ちょっと, きっと, ゆっくり), 、and 。pauses, です/ました/ます endings (devoiced vowels) and several voiced う (うち, ゆっくり, 歩きます). Submit -> speech-start 22.06 s (synthesis_ms 20,700 for 20.6 s of audio: the Space's CPU synthesises long text at roughly 1.0x real time). Speech-start -> speech-end 20.66 s wall for a 20.565 s buffer.

Recording: docs/evidence/2026-09-06-lipsync-20s.webm (23 s, the utterance window of the page recording, VP8 800x600, 1.98 MB; the 57 s original was mostly the avatar idling while the server synthesised). Face crops during the utterance: docs/evidence/2026-09-06-lipsync-frames-01..10.png, each targeted at a long timeline event and labelled with the mouth read immediately before and after the capture (docs/evidence/2026-09-06-lipsync-telemetry.json § frames). Reference sheet of the five presets held at weight 1.0 on the same VRM by the same vrm-stage.js (local static server, same canvas size): docs/evidence/2026-09-06-viseme-ref-{closed,aa,ih,ou,ee,oh}.png. Telemetry: 2,063 samples of currentVisemes + the player's audio clock at ~100 samples/s (mean interval 10.0 ms) from speech-start to speech-end, plus the server's timeline for the same utterance — all in the telemetry JSON.

Frame	Target (timeline)	Mouth read after the capture	Reads as
01	い @0.50 s (96 ms)	ih 0.20 (fading in)	mostly closed, opening
02	あ @0.85 s (160 ms)	aa 0.69	open oval — あ
03	い @1.79 s (96 ms)	ih 0.88 (aa 0.11 residual)	open, on the あ->い cross-fade
04	え @2.69 s (171 ms)	ee 0.92	flat, wide, teeth visible — え
05	お @3.50 s (160 ms)	oh 1.00	tall rounded — お
06	あ @8.09 s (160 ms)	aa 1.00 (captured 0.9 s late, at 9.09 s — a later あ; the screenshot call stalled)	open oval — あ
07	う @17.10 s (117 ms)	ou 0.60	small, pursed — う
08	pause @18.79 s (459 ms, before それから)	all 0 (oh 0.19 decaying at the read)	shut
09	う @19.32 s (107 ms)	ou 0.87	small, pursed — う
10	devoiced う @20.42 s (ます, weight 0.5)	ou 0.21	nearly shut, slight purse

The four judgements

1. Do あ / い / う visibly differ? — YES. Each preset is driven independently to full weight: stage peak-hold over the utterance aa 1.00, ih 1.00, ou 0.92, ee 1.00, oh 1.00 (ou tops out at 0.92 on a 117 ms う and at 0.66–0.69 on 53 ms ones — the 50 ms cross-fade in lipsync.js is the limit, not the timeline). On this VRM あ is a tall open oval (ref-aa, frames 02/06), い a flat wide opening with teeth (ref-ih), う a small pursed mouth (ref-ou, frames 07/09) — three shapes no one would confuse, so this is not amplitude flapping. Two honest caveats. (a) On this model ih≈ee and aa≈oh are visually near-identical pairs (reference sheet): the five presets read as three groups. That is a property of the sample character, recorded in docs/ASSETS.md, not of the pipeline. (b) Both deployed い captures (01, 03) landed on a cross-fade — い moras in this sentence are 96 ms, shorter than the ~110 ms a screenshot round trip takes — so the clean い is the reference frame; the telemetry shows ih at 1.00 with aa 0 on the 21 full-weight い/し/ち moras.
2. Does the mouth close on ん, っ and pauses? — YES. The timeline has 113 closed events (N, cl, pau, and consonant onsets). Every pause ≥ 139 ms (the 、/。 rests, 245–459 ms; 12 events) reached a sampled mouth sum of exactly 0. The mora-length closures — ん and っ, 85–107 ms, 17 events — reached a median 0.02 of open (min 0, max 0.44): shut to the eye, the residual being the 50 ms cross-fade's tail on the shortest ones. The 32–75 ms consonant onsets inside moras dip to a median 0.20 and reopen, which is how speech looks. Frame 08 is the 459 ms pause: shut.
3. Is the END of the sentence as well synced as the start? — YES, no drift. The timeline ends at 20.565333 s and the decoded AudioBuffer lasts 20.565333 s — a 0.000 ms difference over 211 events. Per-event agreement (the dominant sampled viseme at 60 % through every vowel event ≥ 90 ms) is 19/19 in the first third, 19/19 in the second, 22/22 in the last — zero mismatches. The last vowel (the devoiced う of ます, 20.363–20.469 s) released on time and the last sample with the mouth open is at clock 20.550 s, 15 ms before speech-end; the first vowel event is at 0.192 s and the first open sample at 0.198 s. Clock discipline held: the player reads audioCtx.currentTime - scheduledStart, and the last third looks exactly like the first.
4. Does the mouth freeze on です / した (devoiced vowels)? — NO, it moves at half weight as designed. Eight devoiced moras (weight 0.5 in the timeline: す ×5 incl. です/します/ます, し/ち ×3 in ました/美味しかった/ちょっと) opened to sampled peaks of 0.35–0.45 against the 0.5 target (attack-limited on 53–107 ms moras) and were never held open (0 past 0.75) and never frozen shut (0 at 0.00 while a vowel was due). Frame 10 is the final devoiced ます: nearly shut with a slight purse, which is the intended look. The uppercase-vowel path is intact.

Verdict of this review: all four criteria hold on the deployed Space; no drift, no frozen mouth, no amplitude flapping. Escalation per the plan is therefore not triggered. The judgement is the execution agent's from the evidence above; the owner has the recording and the frames to overrule it.

Findings recorded, not fixed (Phase 1 ships as is): (a) under headless SwiftShader at ~15 fps — a first, discarded run — a lone 53 ms う was skipped entirely (peak-hold ou 0.5, from a devoiced mora only): a mora shorter than one render frame can be dropped when the browser renders below 19 fps. Real GPUs render at 60+; the emulated-mobile rows above (6.7 fps under SwiftShader) say what a genuinely slow device would look like, which is one more reason the real-device row stays open. (b) The 50 ms cross-fade caps how far a 53 ms mora opens (0.66) — a tunable, and a design choice this review did not second-guess.

Interpretation

• p50 dispatch -> speech-start is 7107 ms (over the ~1500 ms VOIC-05 perceived-response target; OVER the 2500 ms re-plan trigger).

RE-PLAN TRIGGER FIRED. p50 7107 ms exceeds 2500 ms. Per 01-RESEARCH.md Open Question 6 - "treat p50 > ~2.5 s as a re-plan trigger, not a phase failure" - this is a decision for the roadmap, not a failing test: the criterion for Phase 1 is "p50/p95 measured and recorded", which this file is.

• What dominates: VOICEVOX synthesis on the Space's CPU - server synthesis_ms p50 6422 ms is 90 % of the p50 turn; audio_query (1.4 ms), timeline (0.36 ms) and encode (0.17 ms) are noise. Network + Gradio round trip above the server total: 405 ms at p50; browser decode + schedule after the response: 278 ms at p50.
• Cheapest improvement (recorded, NOT implemented in Phase 1): the same sentence synthesises 3–6x faster on a developer laptop than on the Space (plan 01-09), so the container's CPU share is the lever - check cpu_num_threads on the Space's Synthesizer and whether the ZeroGPU container's CPU allocation is the ceiling; beyond that, VOICEVOX CORE 0.17's streaming synthesis would let speech start before the whole utterance is rendered, and per-sentence synthesis of long replies would bound the first-sample latency by the first sentence rather than the whole reply.

Notes

• Phase 1 has no LLM. These numbers are NOT predictive of Phase 3, which adds one.
• Space sleeps after 48 h (gcTimeout 172800), so a cold start is the default first-visit experience.
• Numbers live in git because Space disk is ephemeral. Continuous telemetry is deliberately deferred to Phase 4/6, which owns the privacy story.
• Headless Chromium renders through SwiftShader on the client; the client-side decode + schedule share above is therefore an upper bound on what a GPU-rendered visitor sees.

Raw samples (dispatch -> speech-start, ms, in run order)

Turn	Sentence #	dispatch->speech	dispatch->response	synthesis	lastTurnMs
1	1	6538	6262	5997	6538
2	2	9639	9361	8902	9639
3	3	9661	9390	9165	9661
4	4	6674	6401	6117	6674
5	5	7722	7441	7100	7722
6	1	4589	4326	4134	4589
7	2	10677	10406	10099	10677
8	3	10713	10443	10120	10713
9	4	7113	6836	6503	7113
10	5	6365	6086	5787	6365
11	1	6417	6144	5860	6417
12	2	8740	8460	8201	8740
13	3	10858	10592	10118	10858
14	4	7107	6829	6422	7107
15	5	8437	8157	7936	8437
16	1	6662	6390	6097	6662
17	2	7552	7282	6998	7553
18	3	9634	9360	9083	9634
19	4	7113	6847	6646	7113
20	5	6635	6364	6089	6635
21	1	6706	6434	6081	6706
22	2	8142	7868	7510	8142
23	3	10552	10277	9963	10552
24	4	6828	6552	6127	6828
25	5	4872	4601	4284	4872
26	1	6188	5912	5713	6188
27	2	5572	5297	4900	5572
28	3	10207	9924	9653	10207
29	4	6775	6505	6224	6775
30	5	6337	6068	5835	6337

Replays (dispatch -> speech-start, ms): 0, 0, 0, 0, 0