Safetensors
qwen2
4-bit precision
gptq
cicdatopea commited on
Commit
188fb5f
·
verified ·
1 Parent(s): e074213

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -1
README.md CHANGED
@@ -163,7 +163,6 @@ lm-eval --model hf --model_args pretrained=OPEA/DeepSeek-R1-Distill-Qwen-32B-int
163
  | Metric | BF16 | INT4 |
164
  | :------------------------: | :----------------------: | :---------------: |
165
  | avg | 0.6647 | 0.6639|
166
- |----------------------|--------|--------|
167
  | leaderboard_mmlu_pro | - | - |
168
  | mmlu | 0.7964 | 0.7928 |
169
  | lambada_openai | 0.6649 | 0.6718 |
@@ -183,7 +182,17 @@ lm-eval --model hf --model_args pretrained=OPEA/DeepSeek-R1-Distill-Qwen-32B-int
183
  ### Generate the model
184
 
185
  ~~~bash
 
186
 
 
 
 
 
 
 
 
 
 
187
 
188
  ~~~
189
 
 
163
  | Metric | BF16 | INT4 |
164
  | :------------------------: | :----------------------: | :---------------: |
165
  | avg | 0.6647 | 0.6639|
 
166
  | leaderboard_mmlu_pro | - | - |
167
  | mmlu | 0.7964 | 0.7928 |
168
  | lambada_openai | 0.6649 | 0.6718 |
 
182
  ### Generate the model
183
 
184
  ~~~bash
185
+ Here is the sample command to generate the model.
186
 
187
+ ```bash
188
+ auto-round \
189
+ --model \
190
+ --device 0 \
191
+ --bits 4 \
192
+ --iter 200 \
193
+ --disable_eval \
194
+ --format 'auto_gptq,auto_round' \
195
+ --output_dir "./tmp_autoround"
196
 
197
  ~~~
198