Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,75 @@
+---
+license: cc-by-nc-4.0
+base_model: k2-fsa/OmniVoice
+datasets:
+  - XXH333/WordVoice-5A
+language:
+  - en
+pipeline_tag: text-to-speech
+tags:
+  - tts
+  - voice-cloning
+  - controllable-tts
+  - word-level-control
+  - omnivoice
+---
+
+# OmniVoice Word-Control
+
+[OmniVoice](https://huggingface.co/k2-fsa/OmniVoice) (0.6B masked-diffusion LM TTS on
+Qwen3-0.6B) fine-tuned for **five-dimensional word-level acoustic control**, following the
+task of [WordVoice (arXiv:2607.06461)](https://huggingface.co/papers/2607.06461) and its
+dataset [WordVoice-5A](https://huggingface.co/datasets/XXH333/WordVoice-5A).
+
+Control is injected as **inline special tokens** placed before any word (any subset works;
+untagged words are planned freely by the model):
+
+```
+<|dur_9|><|bnd_0|><|eng_15|><|pit_17|><|ton_rise|>word
+```
+
+| tokens | attribute | discretization |
+|---|---|---|
+| `<\|dur_0..63\|>` | word duration | 40 ms frames (40 ms – 2.56 s) |
+| `<\|bnd_0..4\|>` | pause after word | b0 (none) … b4 (long, >0.4 s) |
+| `<\|eng_0..19\|>` | energy | 20 uniform bins over [0, 1] |
+| `<\|pit_0..19\|>` | pitch (core F0) | 20 uniform bins over [−1, 1] |
+| `<\|ton_flat\|rise\|rrise\|fall\|ffall\|peak\|valley\|>` | pitch contour | 7 morphologies |
+
+## Usage
+
+```python
+from omnivoice import OmniVoice  # code: github.com/k2-fsa/OmniVoice
+
+model = OmniVoice.from_pretrained("multimodalart/omnivoice-word-control", device_map="cuda")
+audio = model.generate(
+    text="I will <|pit_3|><|ton_ffall|>never agree to <|bnd_4|>this!",
+    language="en",
+    ref_audio="ref.wav",
+    ref_text="Transcript of the reference clip.",
+)
+```
+
+The tokenizer in this repo already contains the 116 control tokens. For strict overall
+timing, also pass `duration=` (the diffusion canvas is fixed before generation, so word
+duration tokens redistribute time within it). Try it in the
+[demo Space](https://huggingface.co/spaces/multimodalart/omnivoice-word-control).
+
+## Training
+
+- Init from `k2-fsa/OmniVoice`; all parameters trained; masked-diffusion objective unchanged
+  (control tokens are conditioning only — no loss on text).
+- Data: WordVoice-5A **English**, all 4 train shards (~2,138 h, 1.18 M utterances); on-the-fly
+  tag construction with dropout (15% utterances untagged / 20% words / attributes kept @ 0.7)
+  to preserve plain TTS and enable partial control.
+- 40k steps × 8,192 effective tokens (~1 epoch), lr 5e-5 cosine, bf16, single A100-40GB, ≈6 h.
+- Final eval loss 4.109 (best of run). Directional probes (hi/lo tag ratio, seed-matched):
+  pitch 3.61, energy 2.52, duration 2.70.
+
+Fine-tuning recipe: the `word-control` patch set on the OmniVoice trainer
+(`omnivoice/data/word_control.py` — tag vocabulary, binning, alignment, dropout).
+
+## License & lineage
+
+Weights are **CC-BY-NC-4.0** (derivative of the CC-BY-NC OmniVoice release).
+Data: WordVoice-5A (CC-BY-4.0). Task formulation: WordVoice, arXiv:2607.06461.