gavinlaw/moss-tts-realtime-infinisst-en-zh-v7-traj

MOSS-TTS-Realtime · InfiniSST en→zh cascade finetune (v7)

Turn-by-turn streaming TTS checkpoint for the cascade InfiniSST (en→zh simultaneous S2T) → MOSS-TTS-Realtime: the model speaks incremental Chinese translation deltas as they are emitted, conditioned on a fixed voice prompt plus the running (text, audio) history.

• Base model: OpenMOSS-Team/MOSS-TTS-Realtime (Qwen3-1.7B backbone, 16 RVQ codebooks @ 12.5 Hz, text/audio interleave with 12-token delay).
• Finetune data (train_v7, 49,047 rows): 36,529 rows of the v6 lineage (long-passage synthesis, ~1.92 s increment granularity, incl. mid-start and self-history variants) + 12,518 rows derived from a FLORAS/GigaSpeech trajectory-aligned corpus (6,385 full-passage rows + 6,133 mid-start copies; 0.96 s increment granularity, median turn 4–5 chars). Targets synthesized whole-passage, then sliced back to turns by wav2vec forced alignment.
• Training: 1 epoch, global batch 15, patched upstream sft.py (LR-scheduler process-count fix). Loss 3.4647.

Usage

Inference/eval scripts live in the public repo luojiaxuan/S2S_omni — scripts/moss_multiturn_infer.py. Recommended configuration (as evaluated): soft-reset context window (--sliding-window 11 --soft-reset-keep 3) with the fixed inference engine (repo commit 646f995+): rebuilt-history assistant header fix, true KV-cache append during window growth, first-turn buffering.

Evaluation (ACL6060 en→zh, self-hosted Qwen3-ASR → SEGALE sentence BLEU /

XCOMET-XL-ref / fixed-reference under-translation)

config	1×	1.5×
soft reset keep=3 (fixed engine)	29.98 / 0.613 / 6.6%	31.14 / 0.655 / 4.5%
constant sliding w=11	31.03 / 0.652 / 1.3%	30.73 / 0.589 / 7.5%

Boundary-controlled rescoring (turn boundaries fixed before ASR) shows the soft-reset "under-translation gap" at 1× is an instrument artifact; soft reset wins BLEU at both speeds under boundary control (34.11/36.50 vs 33.30/35.45). Audio samples & per-turn browser: https://luojiaxuan.github.io/S2S_omni/v7_audio_browser.html. Full experiment ledger: docs/experiment_ledger_moss_tts_cascade_20260808.md in the GitHub repo.