Instructions to use build-small-hackathon/baby-mynah-welsh with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- VoxCPM
How to use build-small-hackathon/baby-mynah-welsh with VoxCPM:
import soundfile as sf from voxcpm import VoxCPM model = VoxCPM.from_pretrained("build-small-hackathon/baby-mynah-welsh") wav = model.generate( text="VoxCPM is an innovative end-to-end TTS model from ModelBest, designed to generate highly expressive speech.", prompt_wav_path=None, # optional: path to a prompt speech for voice cloning prompt_text=None, # optional: reference text cfg_value=2.0, # LM guidance on LocDiT, higher for better adherence to the prompt, but maybe worse inference_timesteps=10, # LocDiT inference timesteps, higher for better result, lower for fast speed normalize=True, # enable external TN tool denoise=True, # enable external Denoise tool retry_badcase=True, # enable retrying mode for some bad cases (unstoppable) retry_badcase_max_times=3, # maximum retrying times retry_badcase_ratio_threshold=6.0, # maximum length restriction for bad case detection (simple but effective), it could be adjusted for slow pace speech ) sf.write("output.wav", wav, 16000) print("saved: output.wav") - Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,73 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-4.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
base_model: openbmb/VoxCPM-0.5B
|
| 6 |
+
base_model_relation: adapter
|
| 7 |
+
pipeline_tag: text-to-speech
|
| 8 |
+
tags:
|
| 9 |
+
- text-to-speech
|
| 10 |
+
- voxcpm
|
| 11 |
+
- lora
|
| 12 |
+
- accent
|
| 13 |
+
- voice
|
| 14 |
+
- vctk
|
| 15 |
+
- baby-mynah
|
| 16 |
+
---
|
| 17 |
+
|
| 18 |
+
# baby mynah — Welsh accent (LoRA for VoxCPM-0.5B)
|
| 19 |
+
|
| 20 |
+
A LoRA adapter that voices [VoxCPM-0.5B](https://huggingface.co/openbmb/VoxCPM-0.5B)
|
| 21 |
+
in the **Welsh** accent of a single VCTK speaker. One of six accent experts
|
| 22 |
+
behind the [**baby mynah** Space](https://huggingface.co/spaces/build-small-hackathon/baby-mynah), where
|
| 23 |
+
you can slide between any two of them in weight space.
|
| 24 |
+
|
| 25 |
+
## What it is
|
| 26 |
+
|
| 27 |
+
- **Base model:** [`openbmb/VoxCPM-0.5B`](https://huggingface.co/openbmb/VoxCPM-0.5B) (OpenBMB),
|
| 28 |
+
a tokenizer-free TTS model. This repo holds **only the LoRA delta** (~28 MB) —
|
| 29 |
+
you also need the base model.
|
| 30 |
+
- **Accent:** Welsh (Cardiff), learned from a single
|
| 31 |
+
[VCTK](https://datashare.ed.ac.uk/handle/10283/3443) speaker (**p253**,
|
| 32 |
+
age 22, female) — ~8 minutes of read speech.
|
| 33 |
+
- **Not PEFT.** VoxCPM uses a custom `LoRALinear`, so this loads through the
|
| 34 |
+
baby-mynah loader, not `peft`. The weights file stores, per target Linear,
|
| 35 |
+
`<prefix>.lora_A` (r, in) and `<prefix>.lora_B` (out, r); the layer applies
|
| 36 |
+
`out += (alpha/r) * (x @ A^T @ B^T)`.
|
| 37 |
+
|
| 38 |
+
## Training
|
| 39 |
+
|
| 40 |
+
| | |
|
| 41 |
+
|---|---|
|
| 42 |
+
| Method | LoRA (rank 32, alpha 32, dropout 0) |
|
| 43 |
+
| Target modules | `q/k/v/o_proj` on both the LM and the DiT |
|
| 44 |
+
| Steps | 800 (effective batch 4 = batch 1 x grad-accum 4) |
|
| 45 |
+
| LR | 1e-4, 50 warmup, weight decay 0.01 |
|
| 46 |
+
| Audio | ~8 min, 16 kHz mono |
|
| 47 |
+
|
| 48 |
+
## Usage
|
| 49 |
+
|
| 50 |
+
These adapters are meant to be *blended*, which is what the Space does. To load
|
| 51 |
+
one on its own, use the project's `load_adapter` / `blend_into_model` helpers
|
| 52 |
+
(see [`scripts/blend.py`](https://huggingface.co/spaces/build-small-hackathon/baby-mynah/blob/main/scripts/blend.py)):
|
| 53 |
+
|
| 54 |
+
```python
|
| 55 |
+
from blend import load_adapter, blend_into_model, restore_weights, scaling_from_config
|
| 56 |
+
from voxcpm import VoxCPM
|
| 57 |
+
|
| 58 |
+
model = VoxCPM.from_pretrained(hf_model_id="openbmb/VoxCPM-0.5B", optimize=False, device="cuda")
|
| 59 |
+
adapter = load_adapter("path/to/this/adapter") # {prefix: (A, B)}
|
| 60 |
+
scaling = scaling_from_config("path/to/this/adapter") # alpha / r
|
| 61 |
+
|
| 62 |
+
# alpha=1.0 applies this expert fully; restore afterwards (surgery is in-place).
|
| 63 |
+
snap = blend_into_model(model.tts_model, adapter, adapter, alpha=1.0, scaling=scaling)
|
| 64 |
+
wav = model.generate(text="Hello! I am a very small bird.", normalize=True)
|
| 65 |
+
restore_weights(model.tts_model, snap)
|
| 66 |
+
```
|
| 67 |
+
|
| 68 |
+
## License & attribution
|
| 69 |
+
|
| 70 |
+
- LoRA weights released under **CC BY 4.0**, following the
|
| 71 |
+
[VCTK corpus](https://datashare.ed.ac.uk/handle/10283/3443) (CSTR, University
|
| 72 |
+
of Edinburgh) they were trained on.
|
| 73 |
+
- Base model **VoxCPM-0.5B** by **OpenBMB**, Apache-2.0.
|