Model card: the command and the recipe
Browse files
README.md
CHANGED
|
@@ -67,33 +67,15 @@ structured output, and `image_url` / video parts in chat messages.
|
|
| 67 |
|
| 68 |
## How this file was made
|
| 69 |
|
| 70 |
-
The exact command, run on 2026-10-03 from a radiance build directory of the commit series merged
|
| 71 |
-
to radiance's main branch as 2531f14 (`radiance_home` is that build's plugin directory):
|
| 72 |
-
|
| 73 |
```sh
|
| 74 |
-
CALIB=
|
| 75 |
-
|
| 76 |
-
--
|
| 77 |
-
--reuse ~/models/rad/w4nl-i8lm.rad -o ~/models/rad/w4nl64-i8lm.rad -v
|
| 78 |
```
|
| 79 |
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
quantised by this run. Without `--reuse` the same command quantises those weights from the
|
| 84 |
-
checkpoint by the same rules.
|
| 85 |
-
- `CALIB` is the calibration directory the GPTQ rule reads: each layer's routed-expert input Gram
|
| 86 |
-
from ten million tokens of the bf16 model, and each expert's own down-projection Gram from one
|
| 87 |
-
million, recorded with the engine's calibration mode on a mixed chat, tool-use and code corpus.
|
| 88 |
-
It is not published.
|
| 89 |
-
- The published file is that output under this repository's name, with the local directory
|
| 90 |
-
removed from the `created by`, recipe and calibration-path strings in its header; the weights
|
| 91 |
-
are the converter's bytes.
|
| 92 |
-
|
| 93 |
-
The recipe file the command names is in this repository as `qwen4exp-w4nl64-i8-hc8m.recipe`, with
|
| 94 |
-
its comments; its `$CALIB` is the calibration directory. Its rules, as
|
| 95 |
-
`rad-info --recipe qwen3.8-next-flash-fp8-iq4r-moe.rad` prints them; the first rule that matches a
|
| 96 |
-
weight decides it, and everything no rule names is the checkpoint's bf16:
|
| 97 |
|
| 98 |
```
|
| 99 |
blk.34.ffn_*_exps.407.weight cast dtype=bf16
|
|
|
|
| 67 |
|
| 68 |
## How this file was made
|
| 69 |
|
|
|
|
|
|
|
|
|
|
| 70 |
```sh
|
| 71 |
+
CALIB=calib/w4nl-calib rad-convert Qwen/Qwen3.8-Flash-Next \
|
| 72 |
+
--tokenizer Qwen/Qwen3.8-Flash-Next/tokenizer.json \
|
| 73 |
+
--recipe qwen4exp-w4nl64-i8-hc8m.recipe -o qwen3.8-next-flash-fp8-iq4r-moe.rad
|
|
|
|
| 74 |
```
|
| 75 |
|
| 76 |
+
The recipe (`qwen4exp-w4nl64-i8-hc8m.recipe` in this repository; `$CALIB` is its calibration data,
|
| 77 |
+
which is not published). The first rule that matches a weight decides it, and everything no rule
|
| 78 |
+
names is the checkpoint's bf16:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 79 |
|
| 80 |
```
|
| 81 |
blk.34.ffn_*_exps.407.weight cast dtype=bf16
|