rename weights to v4
Browse files- README.md +6 -6
- nar_lora_joint_v3.pt β nar_lora_joint_v4.pt +0 -0
- tokenizer_head_joint_v4.pt +3 -0
README.md
CHANGED
|
@@ -19,8 +19,8 @@ an artist, and generate new songs or covers.
|
|
| 19 |
## Files
|
| 20 |
| file | what |
|
| 21 |
|---|---|
|
| 22 |
-
| `
|
| 23 |
-
| `
|
| 24 |
| `scripts/` | the training loop and inference scripts (below). |
|
| 25 |
|
| 26 |
Trained on 4,765 YuE2 self-generated songs, then adapted to real audio. If your material sounds off, rerun `joint.py` on your own audio (step 3 below).
|
|
@@ -45,21 +45,21 @@ and `<name>.txt` = a style caption starting with your trigger phrase (e.g. `xyzq
|
|
| 45 |
|
| 46 |
1. `python prep_real.py` β MERT features, VAE latents and the prompt prefix per song.
|
| 47 |
2. `python cursor_prep.py` β Demucs vocal stem β MMS forced alignment of the lyrics β lyric-cursor targets (automatic).
|
| 48 |
-
3. *(optional, recommended for a new artist/era)* `HOLD_TRACK=<one song name> python joint.py joint_mine 3000 1 1
|
| 49 |
β adapts head + NAR to your audio. Otherwise use the v3 files as-is.
|
| 50 |
4. `python ar_prep.py <head.pt>` β tokenizes your songs and merges the regularizer pack into `dataset.pt` (`ar_prep.py` expects the pack's records;
|
| 51 |
point it at the downloaded file).
|
| 52 |
5. `SCHED_STEPS=3000 CK_FROM=600 CK_EVERY=200 python ar_lora_cursor.py my_lora 1600 64 0.5 none 1e-4 0.08`
|
| 53 |
β rank-64 AR LoRA, 50/50 artist vs minted, lyric-cursor weight 0.08, checkpoints at 600/800/1000/1200/1400/1600. **Do not train longer**: past
|
| 54 |
~1,500 steps the model memorises the songs.
|
| 55 |
-
6. `LADDER_STYLE_TRACK=<song> LADDER_LYRICS=<lyrics.txt> bash ladder.sh my_lora
|
| 56 |
|
| 57 |
## Inference
|
| 58 |
```bash
|
| 59 |
# new song: style caption with your trigger + lyrics, score-free
|
| 60 |
-
python ar_generate.py my_lora/step-800.pt
|
| 61 |
# cover: transcribe any recording with SheetSage2 (--melody-only), then
|
| 62 |
-
ABC_FILE=score.abc COT=melody STRIP_TEMPO_KEY=1 python ar_generate.py my_lora/step-800.pt
|
| 63 |
# LoRA strength: AR_SCALE=0.77 ... stock model control: pass `none` for either LoRA
|
| 64 |
```
|
| 65 |
`ar_generate.py` folds both LoRAs into the base weights and runs YuE2's own pipeline, so the stock sampler, CFG and VAE apply unchanged.
|
|
|
|
| 19 |
## Files
|
| 20 |
| file | what |
|
| 21 |
|---|---|
|
| 22 |
+
| `tokenizer_head_joint_v4.pt` | MERT-v2-FullSong layer-20 features (per-track instance-normalised, 25 Hz) β 32,768 YuE2 semantic codes. 8-layer transformer, d=512, 512-frame windows. Held-out exact match on YuE2's own songs: 16.1% top-1 (near-miss codes render almost identically; ear tests of NAR round-trips sit around 95%). |
|
| 23 |
+
| `nar_lora_joint_v4.pt` | rank-32 LoRA on `nar_self_attn.{q,k,v,o}_proj` + `nar_mlp.{gate,up,down}_proj` (28 layers) + full `vae2llm`/`llm2vae`. Trained jointly with the head on real audio. |
|
| 24 |
| `scripts/` | the training loop and inference scripts (below). |
|
| 25 |
|
| 26 |
Trained on 4,765 YuE2 self-generated songs, then adapted to real audio. If your material sounds off, rerun `joint.py` on your own audio (step 3 below).
|
|
|
|
| 45 |
|
| 46 |
1. `python prep_real.py` β MERT features, VAE latents and the prompt prefix per song.
|
| 47 |
2. `python cursor_prep.py` β Demucs vocal stem β MMS forced alignment of the lyrics β lyric-cursor targets (automatic).
|
| 48 |
+
3. *(optional, recommended for a new artist/era)* `HOLD_TRACK=<one song name> python joint.py joint_mine 3000 1 1 tokenizer_head_joint_v4.pt nar_lora_joint_v4.pt`
|
| 49 |
β adapts head + NAR to your audio. Otherwise use the v3 files as-is.
|
| 50 |
4. `python ar_prep.py <head.pt>` β tokenizes your songs and merges the regularizer pack into `dataset.pt` (`ar_prep.py` expects the pack's records;
|
| 51 |
point it at the downloaded file).
|
| 52 |
5. `SCHED_STEPS=3000 CK_FROM=600 CK_EVERY=200 python ar_lora_cursor.py my_lora 1600 64 0.5 none 1e-4 0.08`
|
| 53 |
β rank-64 AR LoRA, 50/50 artist vs minted, lyric-cursor weight 0.08, checkpoints at 600/800/1000/1200/1400/1600. **Do not train longer**: past
|
| 54 |
~1,500 steps the model memorises the songs.
|
| 55 |
+
6. `LADDER_STYLE_TRACK=<song> LADDER_LYRICS=<lyrics.txt> bash ladder.sh my_lora nar_lora_joint_v4.pt` β renders one fixed prompt from every checkpoint (optional `FINALS=<file>` with lines `tag style_track lyrics seed`). Pick by ear (ours: step 800).
|
| 56 |
|
| 57 |
## Inference
|
| 58 |
```bash
|
| 59 |
# new song: style caption with your trigger + lyrics, score-free
|
| 60 |
+
python ar_generate.py my_lora/step-800.pt nar_lora_joint_v4.pt out_tag <style_track> lyrics.txt 12
|
| 61 |
# cover: transcribe any recording with SheetSage2 (--melody-only), then
|
| 62 |
+
ABC_FILE=score.abc COT=melody STRIP_TEMPO_KEY=1 python ar_generate.py my_lora/step-800.pt nar_lora_joint_v4.pt cover_tag <style_track> lyrics.txt 21
|
| 63 |
# LoRA strength: AR_SCALE=0.77 ... stock model control: pass `none` for either LoRA
|
| 64 |
```
|
| 65 |
`ar_generate.py` folds both LoRAs into the base weights and runs YuE2's own pipeline, so the stock sampler, CFG and VAE apply unchanged.
|
nar_lora_joint_v3.pt β nar_lora_joint_v4.pt
RENAMED
|
File without changes
|
tokenizer_head_joint_v4.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d23c4f757a05f031134b8471ec84245ec2338966516e1a9e26a17ff300a5f87e
|
| 3 |
+
size 171305291
|