model card revision
Browse files
README.md
CHANGED
|
@@ -15,7 +15,7 @@ language:
|
|
| 15 |
- zh
|
| 16 |
---
|
| 17 |
|
| 18 |
-
# YuE2-3B
|
| 19 |
|
| 20 |
This is an NVFP4 requantisation of the autoregressive (AR) half of
|
| 21 |
[m-a-p/YuE2-3B](https://huggingface.co/m-a-p/YuE2-3B). It is packaged as the
|
|
@@ -85,7 +85,7 @@ share of sampling probability that actually moved, and is the number to read.
|
|
| 85 |
|---|---|---|---|---|---|
|
| 86 |
| bf16 vs bf16, two engine builds | 0.999 | 0.141 / 0.141 | 0.982 | 3.414 / 3.414 | 1.2% |
|
| 87 |
| this checkpoint, NVFP4 W4A16 GPTQ | 0.996 | 0.141 / 0.142 | 0.905 | 3.414 / 3.424 (+0.3%) | 6.3% |
|
| 88 |
-
| FP8 W8A8, sibling repo DKmode22/YuE2-3B-
|
| 89 |
| NVFP4 W4A16, AWQ | 0.993 | 0.141 / 0.143 | 0.896 | 3.414 / 3.433 (+0.6%) | 7.4% |
|
| 90 |
| NVFP4 W4A4, GPTQ (not published) | 0.992 | 0.141 / 0.144 | 0.854 | 3.414 / 3.446 (+0.9%) | 10.1% |
|
| 91 |
|
|
@@ -112,8 +112,8 @@ else.
|
|
| 112 |
```bash
|
| 113 |
pip install "yue2_infer[fast]" # the wheel from m-a-p/YuE2-3B
|
| 114 |
python patch_fast.py "$(python -c 'import yue2.fast,os;print(os.path.abspath(yue2.fast.__file__))')"
|
| 115 |
-
hf download DKmode22/YuE2-3B-
|
| 116 |
-
export YUE2_AR_CHECKPOINT=$PWD/yue2-
|
| 117 |
python - <<'EOF'
|
| 118 |
from yue2 import YuE2Pipeline
|
| 119 |
pipe = YuE2Pipeline.from_pretrained("m-a-p/YuE2-3B", backend="vllm", memory_budget_gib=16)
|
|
|
|
| 15 |
- zh
|
| 16 |
---
|
| 17 |
|
| 18 |
+
# YuE2-3B-NVFP4 (autoregressive checkpoint, W4A16 GPTQ)
|
| 19 |
|
| 20 |
This is an NVFP4 requantisation of the autoregressive (AR) half of
|
| 21 |
[m-a-p/YuE2-3B](https://huggingface.co/m-a-p/YuE2-3B). It is packaged as the
|
|
|
|
| 85 |
|---|---|---|---|---|---|
|
| 86 |
| bf16 vs bf16, two engine builds | 0.999 | 0.141 / 0.141 | 0.982 | 3.414 / 3.414 | 1.2% |
|
| 87 |
| this checkpoint, NVFP4 W4A16 GPTQ | 0.996 | 0.141 / 0.142 | 0.905 | 3.414 / 3.424 (+0.3%) | 6.3% |
|
| 88 |
+
| FP8 W8A8, sibling repo DKmode22/YuE2-3B-FP8 | 0.997 | 0.141 / 0.141 | 0.942 | 3.414 / 3.418 (+0.1%) | 3.8% |
|
| 89 |
| NVFP4 W4A16, AWQ | 0.993 | 0.141 / 0.143 | 0.896 | 3.414 / 3.433 (+0.6%) | 7.4% |
|
| 90 |
| NVFP4 W4A4, GPTQ (not published) | 0.992 | 0.141 / 0.144 | 0.854 | 3.414 / 3.446 (+0.9%) | 10.1% |
|
| 91 |
|
|
|
|
| 112 |
```bash
|
| 113 |
pip install "yue2_infer[fast]" # the wheel from m-a-p/YuE2-3B
|
| 114 |
python patch_fast.py "$(python -c 'import yue2.fast,os;print(os.path.abspath(yue2.fast.__file__))')"
|
| 115 |
+
hf download DKmode22/YuE2-3B-NVFP4 --local-dir ./yue2-3b-nvfp4
|
| 116 |
+
export YUE2_AR_CHECKPOINT=$PWD/yue2-3b-nvfp4
|
| 117 |
python - <<'EOF'
|
| 118 |
from yue2 import YuE2Pipeline
|
| 119 |
pipe = YuE2Pipeline.from_pretrained("m-a-p/YuE2-3B", backend="vllm", memory_budget_gib=16)
|