Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,89 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: Qwen/Qwen3-TTS-12Hz-1.7B-Base
|
| 4 |
+
language:
|
| 5 |
+
- zh
|
| 6 |
+
tags:
|
| 7 |
+
- text-to-speech
|
| 8 |
+
- tts
|
| 9 |
+
- qwen3-tts
|
| 10 |
+
- live-streaming
|
| 11 |
+
- voice-clone
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# Qwen3-TTS-TBLive-Base
|
| 15 |
+
|
| 16 |
+
A Chinese live-streaming domain TTS checkpoint fine-tuned from
|
| 17 |
+
`Qwen3-TTS-12Hz-1.7B-Base` via continual pre-training (CPT) and GRPO
|
| 18 |
+
post-training on live-streaming speech data.
|
| 19 |
+
|
| 20 |
+
**Recommended starting point for downstream fine-tuning.** If you need
|
| 21 |
+
production-ready built-in live-streaming voices, use the sibling checkpoint
|
| 22 |
+
[TaoLiveAIGC/Qwen3-TTS-TBLive-CustomVoice](https://huggingface.co/TaoLiveAIGC/Qwen3-TTS-TBLive-CustomVoice).
|
| 23 |
+
|
| 24 |
+
## Benchmark
|
| 25 |
+
|
| 26 |
+
Zero-shot voice-cloning results on the public
|
| 27 |
+
[seed-tts-eval](https://github.com/BytedanceSpeech/seed-tts-eval) test sets,
|
| 28 |
+
compared against open-source baselines. `/` marks numbers not reported.
|
| 29 |
+
|
| 30 |
+
| Model | test-en SIM-o โ | test-en WER โ | test-en UTMOS โ | test-zh SIM-o โ | test-zh WER โ | test-zh UTMOS โ |
|
| 31 |
+
|---|---|---|---|---|---|---|
|
| 32 |
+
| Ground-truth | 0.734 | 2.14 | 3.52 | 0.755 | 1.25 | 2.78 |
|
| 33 |
+
| IndexTTS2 | 0.706 | 2.33 | 3.65 | 0.764 | 1.05 | 3.00 |
|
| 34 |
+
| CosyVoice3 | 0.696 | 2.17 | 3.96 | 0.778 | 1.14 | 3.32 |
|
| 35 |
+
| VoxCPM | 0.731 | 1.92 | 3.77 | 0.772 | 0.99 | 2.94 |
|
| 36 |
+
| MossTTS Local | 0.732 | 1.93 | / | 0.796 | 1.44 | / |
|
| 37 |
+
| Qwen3-TTS | 0.708 | 1.54 | 4.16 | 0.766 | 1.15 | 3.46 |
|
| 38 |
+
| **Qwen3-TTS-TBLive-Base** | **0.732** | **1.63** | **4.11** | **0.782** | **1.28** | **3.38** |
|
| 39 |
+
|
| 40 |
+
## Quickstart
|
| 41 |
+
|
| 42 |
+
```bash
|
| 43 |
+
pip install -U qwen-tts
|
| 44 |
+
```
|
| 45 |
+
|
| 46 |
+
```python
|
| 47 |
+
import torch
|
| 48 |
+
from qwen_tts import Qwen3TTSModel
|
| 49 |
+
|
| 50 |
+
model = Qwen3TTSModel.from_pretrained(
|
| 51 |
+
"TaoLiveAIGC/Qwen3-TTS-TBLive-Base",
|
| 52 |
+
device_map="cuda:0",
|
| 53 |
+
dtype=torch.bfloat16,
|
| 54 |
+
)
|
| 55 |
+
|
| 56 |
+
# Zero-shot voice cloning with a short reference audio
|
| 57 |
+
wavs, sr = model.generate_voice_clone(
|
| 58 |
+
text="ๅฎถไบบไปฌ๏ผไปๅคฉ็ดๆญ้ด็็ฆๅฉ็็ๆๆปกไบ๏ผ",
|
| 59 |
+
language="Chinese",
|
| 60 |
+
ref_audio="path/to/reference.wav",
|
| 61 |
+
ref_text="ๅ่้ณ้ขๅฏนๅบ็ๆๆฌ",
|
| 62 |
+
)
|
| 63 |
+
```
|
| 64 |
+
|
| 65 |
+
For training pipelines (SFT / GRPO), inference post-selection, and advanced
|
| 66 |
+
usage, see the companion training repository. For upstream features (vLLM
|
| 67 |
+
serving, DashScope API, deployment), refer to the
|
| 68 |
+
[upstream Qwen3-TTS repository](https://github.com/QwenLM/Qwen3-TTS#readme).
|
| 69 |
+
|
| 70 |
+
## Checkpoint contents
|
| 71 |
+
|
| 72 |
+
```
|
| 73 |
+
โโโ model.safetensors # main checkpoint (~3.6 GB)
|
| 74 |
+
โโโ config.json / configuration.json / generation_config.json
|
| 75 |
+
โโโ preprocessor_config.json
|
| 76 |
+
โโโ tokenizer_config.json / vocab.json / merges.txt
|
| 77 |
+
โโโ speech_tokenizer/ # 12 Hz speech tokenizer (~651 MB, same as upstream)
|
| 78 |
+
```
|
| 79 |
+
|
| 80 |
+
## License
|
| 81 |
+
|
| 82 |
+
Released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0),
|
| 83 |
+
consistent with upstream Qwen3-TTS.
|
| 84 |
+
|
| 85 |
+
## Acknowledgements
|
| 86 |
+
|
| 87 |
+
Our sincere thanks to the Qwen team for open-sourcing
|
| 88 |
+
[Qwen3-TTS](https://github.com/QwenLM/Qwen3-TTS) โ the base model, tokenizer,
|
| 89 |
+
and reference training / inference code that make this work possible.
|