tan-en-yao commited on
Commit
8155364
·
verified ·
1 Parent(s): 9ca7373

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +73 -0
README.md ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-4.0
3
+ language:
4
+ - en
5
+ base_model: openbmb/VoxCPM-0.5B
6
+ base_model_relation: adapter
7
+ pipeline_tag: text-to-speech
8
+ tags:
9
+ - text-to-speech
10
+ - voxcpm
11
+ - lora
12
+ - accent
13
+ - voice
14
+ - vctk
15
+ - baby-mynah
16
+ ---
17
+
18
+ # baby mynah — Welsh accent (LoRA for VoxCPM-0.5B)
19
+
20
+ A LoRA adapter that voices [VoxCPM-0.5B](https://huggingface.co/openbmb/VoxCPM-0.5B)
21
+ in the **Welsh** accent of a single VCTK speaker. One of six accent experts
22
+ behind the [**baby mynah** Space](https://huggingface.co/spaces/build-small-hackathon/baby-mynah), where
23
+ you can slide between any two of them in weight space.
24
+
25
+ ## What it is
26
+
27
+ - **Base model:** [`openbmb/VoxCPM-0.5B`](https://huggingface.co/openbmb/VoxCPM-0.5B) (OpenBMB),
28
+ a tokenizer-free TTS model. This repo holds **only the LoRA delta** (~28 MB) —
29
+ you also need the base model.
30
+ - **Accent:** Welsh (Cardiff), learned from a single
31
+ [VCTK](https://datashare.ed.ac.uk/handle/10283/3443) speaker (**p253**,
32
+ age 22, female) — ~8 minutes of read speech.
33
+ - **Not PEFT.** VoxCPM uses a custom `LoRALinear`, so this loads through the
34
+ baby-mynah loader, not `peft`. The weights file stores, per target Linear,
35
+ `<prefix>.lora_A` (r, in) and `<prefix>.lora_B` (out, r); the layer applies
36
+ `out += (alpha/r) * (x @ A^T @ B^T)`.
37
+
38
+ ## Training
39
+
40
+ | | |
41
+ |---|---|
42
+ | Method | LoRA (rank 32, alpha 32, dropout 0) |
43
+ | Target modules | `q/k/v/o_proj` on both the LM and the DiT |
44
+ | Steps | 800 (effective batch 4 = batch 1 x grad-accum 4) |
45
+ | LR | 1e-4, 50 warmup, weight decay 0.01 |
46
+ | Audio | ~8 min, 16 kHz mono |
47
+
48
+ ## Usage
49
+
50
+ These adapters are meant to be *blended*, which is what the Space does. To load
51
+ one on its own, use the project's `load_adapter` / `blend_into_model` helpers
52
+ (see [`scripts/blend.py`](https://huggingface.co/spaces/build-small-hackathon/baby-mynah/blob/main/scripts/blend.py)):
53
+
54
+ ```python
55
+ from blend import load_adapter, blend_into_model, restore_weights, scaling_from_config
56
+ from voxcpm import VoxCPM
57
+
58
+ model = VoxCPM.from_pretrained(hf_model_id="openbmb/VoxCPM-0.5B", optimize=False, device="cuda")
59
+ adapter = load_adapter("path/to/this/adapter") # {prefix: (A, B)}
60
+ scaling = scaling_from_config("path/to/this/adapter") # alpha / r
61
+
62
+ # alpha=1.0 applies this expert fully; restore afterwards (surgery is in-place).
63
+ snap = blend_into_model(model.tts_model, adapter, adapter, alpha=1.0, scaling=scaling)
64
+ wav = model.generate(text="Hello! I am a very small bird.", normalize=True)
65
+ restore_weights(model.tts_model, snap)
66
+ ```
67
+
68
+ ## License & attribution
69
+
70
+ - LoRA weights released under **CC BY 4.0**, following the
71
+ [VCTK corpus](https://datashare.ed.ac.uk/handle/10283/3443) (CSTR, University
72
+ of Edinburgh) they were trained on.
73
+ - Base model **VoxCPM-0.5B** by **OpenBMB**, Apache-2.0.