DKmode22 commited on
Commit
90a70bc
·
verified ·
1 Parent(s): 1327dfc

model card revision

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -15,7 +15,7 @@ language:
15
  - zh
16
  ---
17
 
18
- # YuE2-3B AR checkpoint, NVFP4 W4A16 (GPTQ)
19
 
20
  This is an NVFP4 requantisation of the autoregressive (AR) half of
21
  [m-a-p/YuE2-3B](https://huggingface.co/m-a-p/YuE2-3B). It is packaged as the
@@ -85,7 +85,7 @@ share of sampling probability that actually moved, and is the number to read.
85
  |---|---|---|---|---|---|
86
  | bf16 vs bf16, two engine builds | 0.999 | 0.141 / 0.141 | 0.982 | 3.414 / 3.414 | 1.2% |
87
  | this checkpoint, NVFP4 W4A16 GPTQ | 0.996 | 0.141 / 0.142 | 0.905 | 3.414 / 3.424 (+0.3%) | 6.3% |
88
- | FP8 W8A8, sibling repo DKmode22/YuE2-3B-AR-FP8 | 0.997 | 0.141 / 0.141 | 0.942 | 3.414 / 3.418 (+0.1%) | 3.8% |
89
  | NVFP4 W4A16, AWQ | 0.993 | 0.141 / 0.143 | 0.896 | 3.414 / 3.433 (+0.6%) | 7.4% |
90
  | NVFP4 W4A4, GPTQ (not published) | 0.992 | 0.141 / 0.144 | 0.854 | 3.414 / 3.446 (+0.9%) | 10.1% |
91
 
@@ -112,8 +112,8 @@ else.
112
  ```bash
113
  pip install "yue2_infer[fast]" # the wheel from m-a-p/YuE2-3B
114
  python patch_fast.py "$(python -c 'import yue2.fast,os;print(os.path.abspath(yue2.fast.__file__))')"
115
- hf download DKmode22/YuE2-3B-AR-NVFP4 --local-dir ./yue2-ar-nvfp4
116
- export YUE2_AR_CHECKPOINT=$PWD/yue2-ar-nvfp4
117
  python - <<'EOF'
118
  from yue2 import YuE2Pipeline
119
  pipe = YuE2Pipeline.from_pretrained("m-a-p/YuE2-3B", backend="vllm", memory_budget_gib=16)
 
15
  - zh
16
  ---
17
 
18
+ # YuE2-3B-NVFP4 (autoregressive checkpoint, W4A16 GPTQ)
19
 
20
  This is an NVFP4 requantisation of the autoregressive (AR) half of
21
  [m-a-p/YuE2-3B](https://huggingface.co/m-a-p/YuE2-3B). It is packaged as the
 
85
  |---|---|---|---|---|---|
86
  | bf16 vs bf16, two engine builds | 0.999 | 0.141 / 0.141 | 0.982 | 3.414 / 3.414 | 1.2% |
87
  | this checkpoint, NVFP4 W4A16 GPTQ | 0.996 | 0.141 / 0.142 | 0.905 | 3.414 / 3.424 (+0.3%) | 6.3% |
88
+ | FP8 W8A8, sibling repo DKmode22/YuE2-3B-FP8 | 0.997 | 0.141 / 0.141 | 0.942 | 3.414 / 3.418 (+0.1%) | 3.8% |
89
  | NVFP4 W4A16, AWQ | 0.993 | 0.141 / 0.143 | 0.896 | 3.414 / 3.433 (+0.6%) | 7.4% |
90
  | NVFP4 W4A4, GPTQ (not published) | 0.992 | 0.141 / 0.144 | 0.854 | 3.414 / 3.446 (+0.9%) | 10.1% |
91
 
 
112
  ```bash
113
  pip install "yue2_infer[fast]" # the wheel from m-a-p/YuE2-3B
114
  python patch_fast.py "$(python -c 'import yue2.fast,os;print(os.path.abspath(yue2.fast.__file__))')"
115
+ hf download DKmode22/YuE2-3B-NVFP4 --local-dir ./yue2-3b-nvfp4
116
+ export YUE2_AR_CHECKPOINT=$PWD/yue2-3b-nvfp4
117
  python - <<'EOF'
118
  from yue2 import YuE2Pipeline
119
  pipe = YuE2Pipeline.from_pretrained("m-a-p/YuE2-3B", backend="vllm", memory_budget_gib=16)