test(02-07): rehearsal script; furigana at three layers, and a per-thread Sudachi tokenizer

- scripts/rehearse_deployed.py: free port, DISABLE_GPU=1 app.py, poll /config,
run the given pytest selection with --space-url, stop the app, exit with
pytest's code (3 with the app's last log lines when it never came up)
- test_facade_parity.py::test_ruby_under_both_transports[inline|iframe]: a real
typed turn puts ruby on the learner's line and the avatar's under both
transports, gated against the Python oracle over the page's own tokens
- test_avatar_loop.py: deployed rows test_furigana_rendered,
test_language_assets_loaded, test_furigana_modes (both preferences across a
reload), plus transcript_dump/wait_for_ruby so a missing rt reports why
- nlp/tokenizer.py: the Tokenizer is now per THREAD over one shared Dictionary.
A shared sudachipy.Tokenizer raises RuntimeError: Already borrowed under
concurrency (5 of 8 threads measured), which this plan made reachable - the
host analyses the learner's line while the turn's analyse stage runs, and the
lost analysis shipped a directive with tokens: [] and an unannotated line
- tests/test_analyzer.py::test_analyze_is_thread_safe: 32 concurrent analyses
all succeed and agree; mutation-tested against the shared-tokenizer revision

scripts/rehearse_deployed.py

@@ -0,0 +1,191 @@
+"""Rehearse the deployed suite against a LOCAL app.py, on a free port, with one command.
+
+    .venv/Scripts/python.exe scripts/rehearse_deployed.py tests/e2e/test_avatar_loop.py -q
+    .venv/Scripts/python.exe scripts/rehearse_deployed.py tests/e2e/test_avatar_loop.py \
+        -q -k "furigana or language_assets"
+
+The deployed rows (``pytestmark = pytest.mark.deployed``) skip unless pytest is given
+``--space-url``. Until the owner pushes, the only honest way to run them is against a real
+app.py serving the working tree - and "start the app by hand on 7860, remember to export
+DISABLE_GPU, remember to kill it" is exactly the ritual that produces a stale process, a
+port clash with the parity suite's own apps, or a rehearsal that silently ran against
+yesterday's build. So it is a script, and every later plan in this phase reuses it.
+
+What it guarantees:
+
+* a FREE port, taken by binding ``('127.0.0.1', 0)`` and reading the number back, so two
+  rehearsals (or a rehearsal and the parity suite) never collide. Nothing is hard-coded.
+* ``DISABLE_GPU=1`` - the Phase 1 kill switch. Locally there is no ZeroGPU allocator, and
+  a rehearsal must never reach ``@spaces.GPU``.
+* the app is up before pytest starts: ``/config`` is polled until it answers 200, which on
+  a cold working tree means the Sudachi dictionary, the compact JMdict and the CTranslate2
+  translator have all been read (``Blocks.load`` warms them) - a wait measured in tens of
+  seconds, not a sleep.
+* the app is stopped afterwards, whatever pytest did, and its tree is killed if it ignores
+  the polite request. A rehearsal leaves no listener behind.
+
+Exit code: pytest's own, so this is drop-in for CI - except 3, which means the app never
+came up; its last log lines are printed so the reason is on screen rather than in a pipe.
+
+The URL handed to pytest is a loopback one, which keeps ``docs/LATENCY.md``'s rule in
+force: the latency harness writes to tmp_path for a non-Space URL, so a rehearsal can
+never overwrite the Space's recorded numbers.
+"""
+
+from __future__ import annotations
+
+import collections
+import os
+import socket
+import subprocess
+import sys
+import threading
+import time
+import urllib.error
+import urllib.request
+from pathlib import Path
+
+REPO_ROOT = Path(__file__).resolve().parent.parent
+
+# The app has to import Gradio, build the Blocks, and warm the tokenizer, the compact
+# JMdict and the translator before /config answers. Measured ~30-60 s on this fleet from a
+# warm page cache; 180 s is the headroom a cold one needs.
+BOOT_TIMEOUT_S = 180
+POLL_S = 0.5
+STOP_GRACE_S = 15
+LOG_TAIL_LINES = 40
+APP_NEVER_CAME_UP = 3
+
+
+def free_port() -> int:
+    """A port the OS says is free right now. Bind to 0, read it back, release it."""
+    with socket.socket() as probe:
+        probe.bind(("127.0.0.1", 0))
+        return int(probe.getsockname()[1])
+
+
+def _drain(stream, sink: collections.deque) -> None:
+    """Read the child's merged output forever, keeping only the tail.
+
+    A pipe nobody reads fills and blocks the writer: Gradio logs every request, so an app
+    left unread would deadlock partway through a long suite. The deque bounds the memory.
+    """
+    for line in iter(stream.readline, ""):
+        sink.append(line.rstrip("\n"))
+    stream.close()
+
+
+def start_app(port: int) -> tuple[subprocess.Popen, collections.deque]:
+    env = {
+        **os.environ,
+        "DISABLE_GPU": "1",
+        "GRADIO_SERVER_NAME": "127.0.0.1",
+        "GRADIO_SERVER_PORT": str(port),
+        "GRADIO_ANALYTICS_ENABLED": "False",
+        "PYTHONIOENCODING": "utf-8",
+    }
+    proc = subprocess.Popen(  # noqa: S603 - sys.executable and this repo's own app.py
+        [sys.executable, "app.py"],
+        cwd=str(REPO_ROOT),
+        env=env,
+        stdout=subprocess.PIPE,
+        stderr=subprocess.STDOUT,
+        text=True,
+        encoding="utf-8",
+        errors="replace",
+        bufsize=1,
+    )
+    log: collections.deque = collections.deque(maxlen=400)
+    threading.Thread(target=_drain, args=(proc.stdout, log), daemon=True).start()
+    return proc, log
+
+
+def wait_for_config(port: int, proc: subprocess.Popen) -> float:
+    """Block until GET /config answers 200. Returns the seconds it took.
+
+    ``/config`` rather than ``/``: Gradio 6 serves an SSR shell for the root before the app
+    is fully assembled, while /config is the app's own description.
+    """
+    url = f"http://127.0.0.1:{port}/config"
+    deadline = time.monotonic() + BOOT_TIMEOUT_S
+    t0 = time.monotonic()
+    while time.monotonic() < deadline:
+        if proc.poll() is not None:
+            raise RuntimeError(f"app.py exited with {proc.returncode} before serving {url}")
+        try:
+            with urllib.request.urlopen(url, timeout=5) as response:  # noqa: S310 - loopback
+                if response.status == 200:
+                    return time.monotonic() - t0
+        except (urllib.error.URLError, TimeoutError, ConnectionError, OSError):
+            time.sleep(POLL_S)
+    raise RuntimeError(f"app.py did not answer {url} within {BOOT_TIMEOUT_S}s")
+
+
+def stop_app(proc: subprocess.Popen) -> None:
+    """Terminate, then kill the whole tree if it lingers. Gradio's server thread has
+    ignored a terminate here before, and a stray listener would poison the next run."""
+    if proc.poll() is not None:
+        return
+    proc.terminate()
+    try:
+        proc.wait(timeout=STOP_GRACE_S)
+        return
+    except subprocess.TimeoutExpired:
+        pass
+    if os.name == "nt":
+        subprocess.run(  # noqa: S603, S607 - fixed argv, no shell
+            ["taskkill", "/F", "/T", "/PID", str(proc.pid)],
+            capture_output=True,
+            check=False,
+        )
+    else:
+        proc.kill()
+    try:
+        proc.wait(timeout=STOP_GRACE_S)
+    except subprocess.TimeoutExpired:
+        print("[rehearse] WARNING: app.py did not die; check for a stray process", flush=True)
+
+
+def main(argv: list[str]) -> int:
+    if not argv:
+        print(__doc__)
+        return 2
+    port = free_port()
+    space_url = f"http://127.0.0.1:{port}"
+    print(f"[rehearse] starting DISABLE_GPU=1 app.py on {space_url}", flush=True)
+    proc, log = start_app(port)
+    try:
+        try:
+            boot_s = wait_for_config(port, proc)
+        except RuntimeError as err:
+            print(f"[rehearse] {err}", flush=True)
+            print(f"[rehearse] last {LOG_TAIL_LINES} log lines from app.py:", flush=True)
+            for line in list(log)[-LOG_TAIL_LINES:]:
+                print(f"    {line}", flush=True)
+            return APP_NEVER_CAME_UP
+        print(f"[rehearse] app answered /config after {boot_s:.1f}s", flush=True)
+        command = [
+            sys.executable,
+            "-m",
+            "pytest",
+            *argv,
+            "--space-url",
+            space_url,
+            "-p",
+            "no:cacheprovider",
+        ]
+        print(f"[rehearse] {' '.join(command)}", flush=True)
+        result = subprocess.run(  # noqa: S603 - argv built here, no shell
+            command,
+            cwd=str(REPO_ROOT),
+            env={**os.environ, "PYTHONIOENCODING": "utf-8"},
+            check=False,
+        )
+        return result.returncode
+    finally:
+        stop_app(proc)
+        print("[rehearse] app stopped", flush=True)
+
+
+if __name__ == "__main__":
+    raise SystemExit(main(sys.argv[1:]))

src/japanese_avatar/nlp/tokenizer.py

@@ -2,9 +2,21 @@
 
 Same shape as ``voice.tts``: nothing is built at import time, ``sudachipy`` is imported inside
 the functions, and the dictionary (``sudachidict_core``, memory-mapped, ~0.1-0.2 s to open) is
-created once by :func:`get_tokenizer` and shared read-only across sessions. :func:`warmup` lets
+created once by :func:`get_dictionary` and shared read-only across sessions. :func:`warmup` lets
 app startup pay that cost before the first visitor does.
 
+**The Tokenizer is per THREAD, the Dictionary is per process.** A ``sudachipy.Tokenizer`` is a
+PyO3 object holding a mutable borrow of its internals for the duration of a ``tokenize`` call:
+a second thread entering the same instance gets ``RuntimeError: Already borrowed``, not a wrong
+answer. Measured here (8 threads, one shared tokenizer): 5 of 8 calls raised. Plan 02-07 made
+that reachable in production - the host page now analyses the learner's line while the turn's
+own analyse stage runs, so two requests hit the analyzer concurrently on every typed turn, and
+a lost analysis showed up as a directive carrying ``tokens: []`` and a line with no furigana.
+
+The Dictionary is the expensive part (the memory-mapped ~110 MB ``system.dic``) and stays
+shared; ``create()`` is cheap, is called at most once per thread, and is serialised by
+``_BUILD_LOCK`` because it borrows the Dictionary too. Threads that never tokenise pay nothing.
+
 :func:`morphemes` is the only place a Sudachi object is touched. It returns plain dicts so that
 ``units.build_units`` and everything downstream is testable without the engine and no Sudachi
 type ever crosses a function boundary.
@@ -13,6 +25,7 @@ type ever crosses a function boundary.
 from __future__ import annotations
 
 import functools
+import threading
 import time
 from typing import Any
 
@@ -20,17 +33,37 @@ from typing import Any
 #: which ``units.build_units`` joins back together.
 _SPLIT_MODE = "C"
 
+_LOCAL = threading.local()
+_BUILD_LOCK = threading.Lock()
+
 
 @functools.lru_cache(maxsize=1)
-def get_tokenizer() -> Any:
-    """The process-wide Sudachi tokenizer over the core dictionary. Read-only; safe to share."""
+def get_dictionary() -> Any:
+    """The process-wide memory-mapped core dictionary. Opened once; shared read-only."""
     from sudachipy import Dictionary
 
-    return Dictionary(dict="core").create()
+    return Dictionary(dict="core")
+
+
+def get_tokenizer() -> Any:
+    """This thread's tokenizer over the shared dictionary. Never share the return value."""
+    tokenizer = getattr(_LOCAL, "tokenizer", None)
+    if tokenizer is None:
+        # create() borrows the Dictionary, so two threads building at once would raise the
+        # very error this function exists to prevent. Once per thread; contention is nil.
+        with _BUILD_LOCK:
+            tokenizer = get_dictionary().create()
+        _LOCAL.tokenizer = tokenizer
+    return tokenizer
 
 
 def warmup() -> float:
-    """Build the tokenizer now and return the seconds it took (0.0 once it is already built)."""
+    """Open the dictionary now and return the seconds it took (0.0 once it is already open).
+
+    Pays the memory-mapping cost, which is what the visitor would otherwise wait for; the
+    per-thread ``create()`` that follows is microseconds and cannot be pre-paid for a
+    request thread that does not exist yet.
+    """
     t0 = time.perf_counter()
     get_tokenizer()
     return time.perf_counter() - t0

tests/e2e/test_avatar_loop.py

@@ -26,6 +26,7 @@ from urllib.parse import urlparse
 
 import pytest
 import requests
+from playwright.sync_api import TimeoutError as PlaywrightTimeoutError
 
 from japanese_avatar.nlp.levels import LEVEL_RANK, kanji_levels
 from japanese_avatar.nlp.ruby import is_kanji
@@ -1202,3 +1203,276 @@ def expected_rt(tokens: list[dict], mode: str, level: str) -> int:
                 shown += 1
                 break
     return shown
+
+
+STUDY_TEXT = "日本語を勉強しています。"
+STUDY_READING_FIRST = "にほんご"
+
+# host.js binds after boot - the shared template imports host.js and transcript.js once the
+# facade exists - so `ready` is true for a window during which getDebug() carries no
+# furigana slice at all. Measured on the local rehearsal: at `ready` it reads null. Every
+# furigana read below waits for this first, so none of these rows can race the bind.
+HOST_BOUND = "() => !!(window.Avatar && window.Avatar.__debug && window.Avatar.__debug.furigana)"
+AVATAR_READY = (
+    "() => !!window.Avatar && window.Avatar.__debug && window.Avatar.__debug.ready === true"
+)
+
+FURIGANA = """
+async () => {
+  const d = await window.Avatar.getDebug();
+  return d.furigana ? JSON.parse(JSON.stringify(d.furigana)) : null;
+}
+"""
+LAST_TOKENS = "async () => (await window.Avatar.getDebug()).lastTokens"
+RT_COUNTS = """
+() => ({
+  you: document.querySelectorAll('#transcript-text .turn-you rt').length,
+  avatar: document.querySelectorAll('#transcript-text .turn-avatar rt').length,
+  total: document.querySelectorAll('#transcript-text rt').length,
+})
+"""
+LEARNER_RUBY = "() => document.querySelectorAll('#transcript-text .turn-you rt').length > 0"
+AVATAR_RUBY = "() => document.querySelectorAll('#transcript-text .turn-avatar rt').length > 0"
+
+# The learner's line is upgraded to ruby by a second round trip (analyze), issued the
+# instant the plain echo lands and then racing the turn's own synthesis through one Gradio
+# queue. The server-side analysis is sub-millisecond (lastAnalyzeMs is printed beside the
+# wall time, so the difference is visibly queueing); on the public Space the queue is also
+# where a cold container's warm-up sits. Generous, and RECORDED rather than asserted.
+LEARNER_RUBY_TIMEOUT_MS = 120_000
+
+
+def _furigana(page) -> dict:
+    return page.evaluate(FURIGANA)
+
+
+def _set_reading(page, select_id: str, value: str) -> dict:
+    page.select_option(f"#{select_id}", value)
+    page.wait_for_timeout(150)
+    return _furigana(page)
+
+
+def transcript_dump(page) -> str:
+    """Everything needed to tell WHY a line has no ruby, gathered on failure only.
+
+    A bare ``wait_for_function`` timeout says "no rt appeared" and nothing else, and the
+    three causes look identical from Python: the line was never added (the turn carried no
+    subtitle), it was added with an empty token list (``turn`` returns ``[]`` rather than
+    losing the utterance when the analyse stage fails), or the rt is there and the selector
+    is wrong. The markup plus the turn event's own token surfaces separate them in one shot.
+
+    Shared with the both-transports layer, which imports it, so a failure at either layer
+    prints the same evidence.
+    """
+    markup = page.evaluate(
+        "() => { const el = document.querySelector('#transcript-text');"
+        " return el ? el.innerHTML : '(no #transcript-text)'; }"
+    )
+    turns = page.evaluate(
+        "() => (window.__events || []).filter((e) => e.event === 'turn').map((e) => ({"
+        " subtitle: e.data && e.data.subtitle,"
+        " tokens: e.data && e.data.tokens ? e.data.tokens.map((t) => t.surface) : null,"
+        " speed: e.data && e.data.speed, timings: e.data && e.data.timings }))"
+    )
+    errors = page.evaluate(
+        "() => (window.__events || []).filter((e) => e.event === 'error').map((e) => e.data)"
+    )
+    return "\n".join(
+        [f"transcript={markup!r}", f"turn events={turns!r}", f"error events={errors!r}"]
+    )
+
+
+def wait_for_ruby(page, predicate: str, timeout_ms: int, what: str) -> None:
+    """wait_for_function, but a timeout carries the page's own explanation."""
+    try:
+        page.wait_for_function(predicate, timeout=timeout_ms)
+    except PlaywrightTimeoutError:
+        dump = transcript_dump(page)
+        raise AssertionError(f"{what} within {timeout_ms} ms.\n{dump}") from None
+
+
+def _study_turn(page, events) -> dict:
+    """One typed turn of the study sentence through the host's own path, timed.
+
+    Not ``_text_turn``: the learner's ruby lands on its own schedule, before speech-end,
+    and the point of this row is that BOTH lines get it - so the two waits are separate
+    and both numbers are reported.
+    """
+    before = len(events.named(page, "speech-end"))
+    submitted = time.monotonic()
+    _submit_text(page, STUDY_TEXT)
+    wait_for_ruby(page, LEARNER_RUBY, LEARNER_RUBY_TIMEOUT_MS, "no ruby on the learner's line")
+    learner_ms = round((time.monotonic() - submitted) * 1000)
+    ends = events.wait_for(
+        page, "speech-end", timeout_ms=SPEECH_END_TIMEOUT_MS, at_least=before + 1
+    )
+    wait_for_ruby(page, AVATAR_RUBY, SPEECH_END_TIMEOUT_MS, "no ruby on the avatar's line")
+    return {
+        "learner_ruby_ms": learner_ms,
+        "turn_ms": round((time.monotonic() - submitted) * 1000),
+        "duration": ends[-1]["audioDuration"],
+        "analyze_ms": page.evaluate("async () => (await window.Avatar.getDebug()).lastAnalyzeMs"),
+    }
+
+
+def _reload_ready(page) -> None:
+    """Reload in place (not a fresh goto) and wait until the host has bound again."""
+    page.reload(timeout=STAGE_ATTACHED_TIMEOUT_MS + 30_000)
+    page.wait_for_selector("#vrm-stage", state="attached", timeout=STAGE_ATTACHED_TIMEOUT_MS)
+    page.wait_for_function(AVATAR_READY, timeout=READY_TIMEOUT_MS)
+    page.wait_for_function(HOST_BOUND, timeout=READY_TIMEOUT_MS)
+
+
+@pytest.mark.deployed
+def test_furigana_rendered(page, space_url, warm_space, speech_events, wait_for_avatar_ready):
+    """JPN-02 / D-01 / D-03: on a first visit, with no control touched, every kanji on the
+    avatar's line AND the learner's own line carries its reading as real ``<ruby><rt>``.
+
+    Three layers of number, because a count alone cannot distinguish rendered ruby from
+    ruby a stylesheet has hidden (research Pitfall 4, the T-pose lesson): the rt COUNT from
+    both the DOM and getDebug(), the rendered rt HEIGHT, and the line HEIGHT with the
+    annotation versus without it. If this fails while
+    tests/e2e/test_transcript_standalone.py passes, the fault is provably the host page -
+    the stylesheet did not load, or the boot template did not hand the module over.
+    """
+    speech_events.install(page)
+    timings = wait_for_avatar_ready(page, space_url)
+    page.wait_for_function(HOST_BOUND, timeout=READY_TIMEOUT_MS)
+
+    # D-01: untouched defaults, server-rendered on the first paint.
+    assert page.input_value("#furigana-mode") == "always"
+    assert page.input_value("#level-select") == "N5"
+    seeded = _furigana(page)
+    print(
+        f"\n[deployed] ready {timings['ready_seconds']:.1f}s, first frame "
+        f"{timings['first_frame_seconds']:.1f}s; furigana at bind: {seeded}"
+    )
+    assert seeded["mode"] == "always" and seeded["level"] == "N5"
+    assert seeded["storage"] == "ok", "the Space's page could not use localStorage"
+
+    turn = _study_turn(page, speech_events)
+    counts = page.evaluate(RT_COUNTS)
+    always = _furigana(page)
+    tokens = page.evaluate(LAST_TOKENS)
+    print(
+        f"[deployed] {STUDY_TEXT}: rt {counts}; learner ruby {turn['learner_ruby_ms']} ms after "
+        f"Enter (server analyze {turn['analyze_ms']} ms), turn {turn['turn_ms']} ms, audio "
+        f"{turn['duration']}s; furigana {always}"
+    )
+    assert counts["you"] == 2, "D-03: the learner's own line must carry the readings too"
+    assert counts["avatar"] == 2
+    assert always["lastLineRt"] == 2
+    assert always["lastLineKanjiRuns"] == 2
+    assert always["lastLineRtHeightPx"] > 0, "rt exists but rendered with no height"
+    assert always["rtTotal"] == counts["total"] == 4
+    assert expected_rt(tokens, "always", "N5") == 2, tokens
+    first_rt = page.locator("#transcript-text .turn-avatar rt").first.text_content()
+    assert first_rt == STUDY_READING_FIRST, first_rt
+
+    never = _set_reading(page, "furigana-mode", "never")
+    print(
+        f"[deployed] rendered height: always rt {always['lastLineRtHeightPx']} px on a line of "
+        f"{always['lastLineHeightPx']} px; never {never['lastLineHeightPx']} px"
+    )
+    assert never["rtTotal"] == 0
+    assert always["lastLineHeightPx"] > never["lastLineHeightPx"], (
+        "the annotated line is not taller than the bare one; transcript.css did not reach "
+        f"the Space's page (always {always['lastLineHeightPx']} px vs never "
+        f"{never['lastLineHeightPx']} px)"
+    )
+    _set_reading(page, "furigana-mode", "always")
+
+
+@pytest.mark.deployed
+def test_language_assets_loaded(page, space_url, warm_space, wait_for_avatar_ready):
+    """JPN-01 on the Space: the language assets are REAL FILES in the container, not LFS
+    pointers (research Pitfall 7), and the load warm-up ran to completion.
+
+    ``languageInfo()`` blocks on the same lock ``Blocks.load`` warms behind (plan 02-06), so
+    this row cannot race the warm-up on a slow cold Space; it waits for it. A 133-byte LFS
+    pointer where model.bin should be fails HERE, with a byte count, rather than as a
+    mystery at some visitor's first tap on an English gloss.
+    """
+    wait_for_avatar_ready(page, space_url)
+    info = page.evaluate("() => window.Avatar.languageInfo()")
+    assert "error" not in info, info
+    warm = info["warm"]
+    print(
+        f"\n[deployed] languageInfo: jmdict_entries={info['jmdict_entries']} "
+        f"jmdict_bytes={info['jmdict_bytes']} "
+        f"tokenizer_dict_bytes={info['tokenizer_dict_bytes']} "
+        f"mt_model_bytes={info['mt_model_bytes']} rss_mb={info['rss_mb']} "
+        f"cpu_cores={info['cpu_cores']} memory={info['memory']}"
+    )
+    print(
+        f"[deployed] warm-up: total {warm['total_s']}s, rss_delta {warm['rss_delta_mb']} MB vs "
+        f"expected {warm['expected_rss_delta_mb']} MB, errors {warm['errors']}; "
+        f"pins {info['pins']}"
+    )
+    assert info["jmdict_entries"] == 218672
+    assert info["mt_model_bytes"] > 70_000_000, "model.bin is an LFS pointer, not the model"
+    assert info["tokenizer_dict_bytes"] > 100_000_000, "SudachiDict-core is not installed whole"
+    assert isinstance(warm["total_s"], int | float)
+    assert warm["errors"] == []
+    assert info["pins"]["sudachidict_core"] == "20260723"
+
+
+@pytest.mark.deployed
+def test_furigana_modes(page, space_url, warm_space, speech_events, wait_for_avatar_ready):
+    """JPN-02 / D-02 / D-13 / D-14: the three-way control gates on the KANJI axis at the
+    learner's level, and BOTH preferences survive a reload.
+
+    The gated counts come from :func:`expected_rt` over the page's OWN tokens and the pinned
+    kanji list, evaluated in Python: 勉 and 強's levels are the data's to say, and hard-coding
+    them would let a wrong record and a wrong gate agree. The reload is the D-14 proof, and
+    it reads the SELECTS as well as the debug slice - a mode that survives in memory but not
+    on the control is a visitor confusion, not a persisted preference.
+    """
+    speech_events.install(page)
+    wait_for_avatar_ready(page, space_url)
+    page.wait_for_function(HOST_BOUND, timeout=READY_TIMEOUT_MS)
+    _study_turn(page, speech_events)
+    tokens = page.evaluate(LAST_TOKENS)
+
+    never = _set_reading(page, "furigana-mode", "never")
+    assert never["rtTotal"] == 0 and never["lastLineRt"] == 0
+    assert page.locator("#transcript-text rt").count() == 0
+
+    above_n5 = _set_reading(page, "furigana-mode", "above")
+    want_n5 = expected_rt(tokens, "above", "N5")
+    assert above_n5["mode"] == "above" and above_n5["level"] == "N5"
+    assert above_n5["lastLineRt"] == want_n5, (above_n5, want_n5)
+
+    above_n2 = _set_reading(page, "level-select", "N2")
+    want_n2 = expected_rt(tokens, "above", "N2")
+    assert above_n2["level"] == "N2"
+    assert above_n2["lastLineRt"] == want_n2, (above_n2, want_n2)
+    assert want_n2 <= want_n5, "a higher level cannot annotate MORE"
+
+    _set_reading(page, "level-select", "N5")
+    back = _set_reading(page, "furigana-mode", "always")
+    print(
+        f"\n[deployed] modes: always rt={back['lastLineRt']}; never rt=0; "
+        f"above@N5 rt={above_n5['lastLineRt']} (oracle {want_n5}); "
+        f"above@N2 rt={above_n2['lastLineRt']} (oracle {want_n2})"
+    )
+    assert back["lastLineRt"] == 2
+
+    # D-14: both preferences, across a reload, on the controls and in the renderer.
+    _set_reading(page, "furigana-mode", "never")
+    _set_reading(page, "level-select", "N3")
+    _reload_ready(page)
+    restored = _furigana(page)
+    selects = {
+        "mode": page.input_value("#furigana-mode"),
+        "level": page.input_value("#level-select"),
+    }
+    print(f"[deployed] after reload: selects {selects}, furigana {restored}")
+    assert selects == {"mode": "never", "level": "N3"}
+    assert restored["mode"] == "never" and restored["level"] == "N3"
+    assert restored["storage"] == "ok"
+
+    # Leave the Space as a first visitor should find it.
+    _set_reading(page, "furigana-mode", "always")
+    _set_reading(page, "level-select", "N5")
+    assert _furigana(page)["mode"] == "always"

tests/e2e/test_facade_parity.py

@@ -16,9 +16,12 @@ Marked slow but NOT deployed: they run entirely locally.
 
 from __future__ import annotations
 
+import time
+
 import pytest
 
 from tests.e2e.conftest import AVATAR_FIRST_FRAME_EXPR, AVATAR_READY_EXPR
+from tests.e2e.test_avatar_loop import expected_rt, wait_for_ruby
 from tests.e2e.test_stage_standalone import (
     ARM_DOWN_MAX,
     FIRST_FRAME_TIMEOUT_MS,
@@ -649,3 +652,140 @@ def test_language_bridge_under_both_transports(transport, page, gradio_apps, spe
     after = page.evaluate(LANGUAGE_DEBUG)
     assert after["translateCount"] == 1, "a refused translate is not a translation"
     assert after["lastTranslateError"] is not None and "characters" in after["lastTranslateError"]
+
+
+# --------------------------------------------------------------- the rendered ruby (02-07)
+#
+# The standalone layer (tests/e2e/test_transcript_standalone.py) already proved the renderer
+# in a browser with no server. This row proves the HOST: that the boot template hands
+# transcript.js to bindHost under BOTH transports, that the stylesheet the app links is the
+# one being applied, and that a real typed turn - the learner's own path through
+# #text-input, not dispatchTurn - puts ruby on the learner's line as well as the avatar's.
+
+STUDY_TEXT = "日本語を勉強しています。"
+STUDY_READING_FIRST = "にほんご"
+
+# host.js binds after boot (the template imports host.js and transcript.js once the facade
+# exists), so `ready` is TRUE for a short window during which getDebug() carries no
+# furigana slice. Every furigana read waits for this first - measured, not assumed: at
+# `ready` the slice reads null on this fleet.
+HOST_BOUND = "() => !!(window.Avatar && window.Avatar.__debug && window.Avatar.__debug.furigana)"
+
+FURIGANA = """
+async () => {
+  const d = await window.Avatar.getDebug();
+  return d.furigana ? JSON.parse(JSON.stringify(d.furigana)) : null;
+}
+"""
+LAST_TOKENS = "async () => (await window.Avatar.getDebug()).lastTokens"
+RT_COUNTS = """
+() => ({
+  you: document.querySelectorAll('#transcript-text .turn-you rt').length,
+  avatar: document.querySelectorAll('#transcript-text .turn-avatar rt').length,
+  total: document.querySelectorAll('#transcript-text rt').length,
+})
+"""
+LEARNER_RUBY = "() => document.querySelectorAll('#transcript-text .turn-you rt').length > 0"
+AVATAR_RUBY = "() => document.querySelectorAll('#transcript-text .turn-avatar rt').length > 0"
+
+# The learner's line is upgraded to ruby by a SECOND round trip (analyze), issued the
+# moment the echo lands and racing the turn's own synthesis through one Gradio queue on a
+# machine already running two apps under SwiftShader. The plan budgeted 5 s; the measured
+# first-turn wall time here is 15-20 s (the server-side analysis itself is sub-millisecond
+# - lastAnalyzeMs is printed beside it, so the difference is visibly queueing, not work).
+# The wait is therefore generous and the number is RECORDED rather than asserted; a
+# threshold would be measuring the fleet, not the feature.
+LEARNER_RUBY_TIMEOUT_MS = 90_000
+
+
+def _select(page, select_id: str, value: str) -> dict:
+    page.select_option(f"#{select_id}", value)
+    page.wait_for_timeout(150)
+    return page.evaluate(FURIGANA)
+
+
+@pytest.mark.parametrize("transport", TRANSPORTS)
+def test_ruby_under_both_transports(transport, page, gradio_apps, speech_events):
+    """JPN-02 / D-01 / D-02 / D-03 / D-13: the same ruby, from the same numbers, inline and
+    in an iframe.
+
+    Every claim is a number read from ``getDebug().furigana`` AND cross-checked against the
+    DOM, because a count alone cannot tell a rendered annotation from a hidden one: the rt
+    height and the always-versus-never line height are what make "invisible ruby" fail
+    (research Pitfall 4). The gated counts come from :func:`expected_rt` over the page's own
+    ``lastTokens`` and the pinned kanji list - 勉 and 強's levels are the data's to say.
+    """
+    url = gradio_apps(transport)
+    speech_events.install(page)
+    page.goto(url)
+    page.wait_for_function(AVATAR_READY, timeout=BOOT_TIMEOUT_MS)
+    page.wait_for_function(FIRST_FRAME, timeout=FIRST_FRAME_TIMEOUT_MS)
+    page.wait_for_function(HOST_BOUND, timeout=BOOT_TIMEOUT_MS)
+    assert page.evaluate("() => window.Avatar.__debug.transport") == transport
+
+    # D-01: the first visit is annotated, with nothing touched.
+    assert page.input_value("#furigana-mode") == "always"
+    assert page.input_value("#level-select") == "N5"
+    seeded = page.evaluate(FURIGANA)
+    print(f"[{transport}] furigana at bind: {seeded}")
+    assert seeded["mode"] == "always" and seeded["level"] == "N5"
+    assert seeded["storage"] == "ok", "localStorage was refused on a plain loopback page"
+
+    # A real typed turn through the host's own path, so the learner line is exercised.
+    ends_before = len(speech_events.named(page, "speech-end"))
+    submitted = time.monotonic()
+    page.fill("#text-input input", STUDY_TEXT)
+    page.press("#text-input input", "Enter")
+    wait_for_ruby(page, LEARNER_RUBY, LEARNER_RUBY_TIMEOUT_MS, "no ruby on the learner's line")
+    learner_ms = round((time.monotonic() - submitted) * 1000)
+    speech_events.wait_for(page, "speech-end", timeout_ms=TURN_TIMEOUT_MS, at_least=ends_before + 1)
+    wait_for_ruby(page, AVATAR_RUBY, TURN_TIMEOUT_MS, "the avatar's line never got ruby")
+
+    counts = page.evaluate(RT_COUNTS)
+    always = page.evaluate(FURIGANA)
+    tokens = page.evaluate(LAST_TOKENS)
+    analyze_ms = page.evaluate("async () => (await window.Avatar.getDebug()).lastAnalyzeMs")
+    print(
+        f"[{transport}] {STUDY_TEXT}: rt {counts}; learner ruby {learner_ms} ms after Enter "
+        f"(server analyze {analyze_ms} ms); furigana {always}"
+    )
+    assert counts["you"] == 2, "D-03: the learner's own line carries the same readings"
+    assert counts["avatar"] == 2
+    assert always["lastLineRt"] == 2
+    assert always["lastLineRtHeightPx"] > 0, "rt exists but rendered with no height"
+    assert always["mode"] == "always" and always["level"] == "N5" and always["storage"] == "ok"
+    assert always["rtTotal"] == counts["total"] == 4
+    assert (
+        page.locator("#transcript-text .turn-avatar rt").first.text_content() == STUDY_READING_FIRST
+    )
+    assert expected_rt(tokens, "always", "N5") == 2, tokens
+
+    never = _select(page, "furigana-mode", "never")
+    assert never["rtTotal"] == 0 and never["lastLineRt"] == 0
+    assert page.locator("#transcript-text rt").count() == 0
+
+    above_n5 = _select(page, "furigana-mode", "above")
+    want_n5 = expected_rt(tokens, "above", "N5")
+    assert above_n5["lastLineRt"] == want_n5, (above_n5, want_n5)
+
+    above_n2 = _select(page, "level-select", "N2")
+    want_n2 = expected_rt(tokens, "above", "N2")
+    assert above_n2["level"] == "N2"
+    assert above_n2["lastLineRt"] == want_n2, (above_n2, want_n2)
+    assert want_n2 <= want_n5
+
+    _select(page, "level-select", "N5")
+    back = _select(page, "furigana-mode", "always")
+    print(
+        f"[{transport}] modes: always rt={always['lastLineRt']} h={always['lastLineHeightPx']} "
+        f"rtH={always['lastLineRtHeightPx']}; never rt=0 h={never['lastLineHeightPx']}; "
+        f"above@N5 rt={above_n5['lastLineRt']} (oracle {want_n5}); above@N2 "
+        f"rt={above_n2['lastLineRt']} (oracle {want_n2}); back to always "
+        f"h={back['lastLineHeightPx']}"
+    )
+    assert back["lastLineRt"] == 2 and back["rtTotal"] == 4
+    assert back["lastLineHeightPx"] > never["lastLineHeightPx"], (
+        "the ruby line is not taller than the bare line: transcript.css did not reach the "
+        f"Gradio page (always {back['lastLineHeightPx']} px vs never "
+        f"{never['lastLineHeightPx']} px)"
+    )

tests/test_analyzer.py

@@ -18,6 +18,7 @@ and one compact-JMdict load per session.
 
 from __future__ import annotations
 
+import concurrent.futures
 import functools
 import importlib.metadata
 import json
@@ -214,6 +215,34 @@ def test_analyze_is_fast(analyzer):
     assert mean_ms < 50, f"{mean_ms:.1f} ms per analysis (budget 50 ms; expected < 5 ms)"
 
 
+def test_analyze_is_thread_safe(analyzer):
+    """Plan 02-07: concurrent analyses all succeed and all agree. The Space serves them.
+
+    ``sudachipy.Tokenizer`` is a PyO3 object that takes a mutable borrow for the duration of
+    a ``tokenize`` call, so ONE shared instance answers the first caller and raises
+    ``RuntimeError: Already borrowed`` at every thread that arrives while it is busy - a lost
+    analysis, not a wrong one. Before ``nlp.tokenizer`` gave each thread its own tokenizer,
+    8 threads on this sentence lost 5 of 8 calls; the deployed symptom was a directive
+    carrying ``tokens: []`` and an avatar line rendered with no furigana at all.
+
+    It became reachable in 02-07 because the host page now analyses the learner's line while
+    the turn's own analyse stage runs, so a single typed turn issues two overlapping
+    requests - and a busy Space multiplies that by its visitors.
+
+    A real regression test: reverting tokenizer.get_tokenizer to one shared instance fails
+    this, and the failure is the exception, not a flaky count.
+    """
+    workers = 8
+    rounds = 4
+    expected = analyzer(LONG_36_MORA)
+    with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as pool:
+        futures = [pool.submit(analyzer, LONG_36_MORA) for _ in range(workers * rounds)]
+        results = [f.result() for f in futures]
+    assert len(results) == workers * rounds
+    for i, units in enumerate(results):
+        assert units == expected, f"thread {i} disagreed with the single-threaded analysis"
+
+
 def test_empty_text(analyzer):
     """Nothing in, nothing out; whitespace is a non-tappable unit with no level."""
     assert analyzer("") == []