devendradhakad/autodroid-litert-community-Qwen3-TTS-12Hz-0.6B-Base

Qwen3-TTS-12Hz-0.6B-Base — LiteRT

Qwen3-TTS-12Hz-0.6B-Base (Apache-2.0) converted to LiteRT (.tflite) for fully on-device text-to-speech with 3-second voice cloning, in 10 languages at 24 kHz.

Qwen3-TTS is a speech LM: a Qwen3-style talker predicts 12.5 Hz frames of 16 codec tokens (first codebook by the talker, 15 residual codebooks by an inner "MTP" transformer), and a neural codec decoder renders PCM. LiteRT-LM's Engine decode loop does not support this generation structure yet, so the model runs as three LiteRT graphs driven by a host-side loop (LiteRT Compiled Model pattern). A complete Python reference pipeline and all conversion scripts live in the litert-samples sample: compiled_model_api/text_to_speech_lm.

Quick start (Python, desktop)

git clone -b qwen3-tts-sample https://github.com/john-rocky/litert-samples.git
cd litert-samples/compiled_model_api/text_to_speech_lm/python
pip install -r requirements.txt
python synthesize.py --text "Hello from LiteRT running fully on device." --output hello.wav

The script downloads this repository automatically (~1.4 GB for the default int4 configuration) and speaks in the bundled demo voice. Enroll your own voice from ~3 s of audio with the sample's conversion/extract_speaker_embedding.py, then pass --speaker my_voice.npy.

Android app

The same sample ships an Android app (Kotlin, Compiled Model API, CPU) under compiled_model_api/text_to_speech_lm/kotlin_cpu/android/: build with Android Studio or ./gradlew :app:installDebug, then run ./install_to_device.sh to download the model files from this repository and push them to the device. Device-verified on Pixel 8a. With the reference mtp_fp32 + codec_decoder_fp32 graphs: RTF ≈ 6.7. The app auto-selects the fast graphs when present: mtp_folded_int8 drops the MTP from ≈333 to ≈68 ms/frame (5×), and the split codec_partA/codec_partB drops the codec from ≈114 to ≈40 ms/frame (2.5×). Together the end-to-end RTF falls to ≈2.06 (~3.2× vs the reference graphs), ASR-lossless.

Files

File	Size	Role
talker_int4.tflite	256 MB	Talker LM (28-layer Qwen3, prefill_32/prefill_128/decode signatures, KV 1024), blockwise-32 OCTAV int4 weights
talker_fp32.tflite	1.8 GB	fp32 talker; under greedy decoding it reproduces the PyTorch reference token-for-token
mtp_fp32.tflite	440 MB	MTP decode step (5-layer transformer, 17-slot KV cache, 15 lm_heads), invoked 17× per frame — the exact reference graph
mtp_folded_int8.tflite	218 MB	Fast MTP: all 16 inner steps × 5 layers folded into one graph (in-graph argmax + embedding gather, KV internal), GPTQ dynamic-int8 weights. One invoke per frame; ~5× faster on device. Drop-in replacement for mtp_fp32.tflite
codec_decoder_fp32.tflite	457 MB	Codec decoder (RVQ + 8-layer transformer + causal ConvNet, 64-frame chunks → 24 kHz PCM)
codec_partA.tflite / codec_partB.tflite	163 + 294 MB	Fast codec: the decoder split at the transformer/convnet boundary. Part A (transformer) runs fp32; Part B (the conv upsampler, ~all the FLOPs) runs with XNNPACK FORCE_FP16 → ~2.5× on device, ASR-identical. Drop-in replacement for codec_decoder_fp32.tflite
tokenizer.json	11 MB	Qwen2 BPE tokenizer (Python sample / tokenizers)
vocab.json, merges.txt	4.5 MB	Same vocabulary in raw form (used by the Android app's Kotlin tokenizer)
tables/*	723 MB	Host-side embedding tables: codec embedding (fp32), 15 MTP embeddings (fp16), text embedding (fp16), text projection MLP (fp32)
voices/demo_speaker.npy	4 KB	Demo voice x-vector (enrolled from the official Qwen3-TTS demo clip)

Accuracy

• Each graph is numerically verified against the PyTorch reference: talker bit-exact at torch level and correlation 1.0 / top-1 100% as .tflite; MTP 15/15 greedy tokens; codec decoder correlation 1.0 (max abs diff 1.8e-5).
• End to end with talker_fp32 + greedy: token-for-token identical codes to the reference implementation, waveform correlation 1.000000, ASR round-trip returns the input sentence.
• talker_int4 (data-free blockwise-32 OCTAV) produces a different but valid sampling trajectory; outputs transcribe identically under ASR round-trip. Channelwise int8/int4 quantization (the tooling default) degenerates on this model family — use blockwise granularity.

Performance (Apple M4 Max, CPU/XNNPACK)

Stage	per 80 ms audio frame
Talker decode (8 threads)	45–50 ms
MTP inner loop (17 invokes, 1 thread)	~148 ms
Codec decoder (amortized)	~10 ms
Total	~205 ms → RTF ≈ 2.5

The MTP inner loop dominates (a 78M-parameter transformer streams its weights 17 times per frame). mtp_folded_int8.tflite folds those 17 invokes into one graph and quantizes it: on the M4 Max the MTP drops to 41 ms/frame, and on a Pixel 8a from ~333 to ~68 ms/frame. The fold is token-identical to the reference; the dynamic-int8 weights give a different-but-intelligible trajectory (ASR round-trip exact). The codec then dominates, and codec_partA/codec_partB split it so the conv-heavy back half runs in fp16 (2.5× on device). Together the end-to-end RTF drops from ≈6.7 to ≈2.06 on a Pixel 8a (≈1.44 on M4 Max), ASR-lossless. Conversion scripts: export_mtp_folded.py / gptq_mtp_folded.py / export_codec_split.py. Remaining lever: the talker (now ~52 ms/frame).

Android (Pixel 8a)

Android figures use the standard TFLite benchmark_model on a Pixel 8a (Tensor G3, Android 16) — 5 warm-up runs then 20 timed runs, CPU at 4 threads.

Graph	GPU (OpenCL)	CPU (XNNPACK, 4 threads)
mtp_folded_int8.tflite	225 ms	153 ms
mtp_fp32.tflite	113 ms	27 ms

Nothing here is faster on the GPU; run this pipeline on the CPU on Android.

Limitations

• Voice cloning is x-vector mode only (speaker embedding). ICL-mode cloning (reference transcript + codec encoding of the reference audio) additionally needs the codec encoder, which is kept off-device (enrollment-time PyTorch).
• The prompt prefill is capped at 32 positions (the x-vector prompt is always 10); the KV cache is 1024 (80 s of audio), generation is capped at 512 frames (41 s) in the sample.
• Streaming synthesis (the model's dual-track design supports it) is not implemented in the sample loop yet.

License

Apache-2.0, inherited from the base model by the Qwen team, Alibaba Group.