license: apache-2.0
tags:
- quantization
- w4a4
- gptq
- rotation
- research-artifact
NAR W4A4KV4 GPTQ checkpoints (qwen3_0.6b_base, qwen3_1.7b_base, qwen3_4b_base, qwen3_8b_base)
Private research artifact from nar-offline-validation, report section "E22 — the Qwen3 Base family under one protocol".
Base model(s): Qwen/Qwen3-0.6B-Base, Qwen/Qwen3-1.7B-Base, Qwen/Qwen3-4B-Base, Qwen/Qwen3-8B-Base.
What the files are
checkpoints/<model>/<checkpoint>/layer_XX.pt holds one decoder layer's weights after the rotation
(Hadamard or NAR) has been folded into the base model and GPTQ has quantized them. The values are the
dequantized INT4 weights stored in floating point (fake quantization), not a packed INT4 format, so
transformers cannot load them on its own. DONE.json records the fold audit, GPTQ settings and
provenance (hostnames and GPU ids redacted). gptq_audit.csv has per-module GPTQ statistics.
At inference the code also quantizes activations per group of 128 and the KV cache (KIVI), and applies
the online rotations (R2 on V, R4 at the down-projection input). Those need the calibrated factors in
rotations/<model>/.
Loading
Clone the code repository, download this repo so that checkpoints/<model>/ sits at
<workdir>/artifacts/e22/<model>/ and rotations/<model>/ at
<workdir>/activations/<model>/, then use nar/e14_w4a4kv4.py::load_quantized_model (as the
experiment's evaluate command does).
Checkpoints
| model | checkpoint | rotation | seed | weight protocol | size |
|---|---|---|---|---|---|
qwen3_0.6b_base |
gptq_hadamard_seed0_g128_asym |
hadamard | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 1.8 GB |
qwen3_0.6b_base |
gptq_nar_k8_seed0_g128_asym |
nar_k8 | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 1.8 GB |
qwen3_0.6b_base |
gptq_nar_kmax_seed0_g128_asym |
nar_kmax | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 1.8 GB |
qwen3_1.7b_base |
gptq_hadamard_seed0_g128_asym |
hadamard | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 5.6 GB |
qwen3_1.7b_base |
gptq_nar_k8_seed0_g128_asym |
nar_k8 | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 5.6 GB |
qwen3_1.7b_base |
gptq_nar_kmax_seed0_g128_asym |
nar_kmax | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 5.6 GB |
qwen3_4b_base |
gptq_hadamard_seed0_g128_asym |
hadamard | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 14.5 GB |
qwen3_4b_base |
gptq_nar_k8_seed0_g128_asym |
nar_k8 | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 14.5 GB |
qwen3_4b_base |
gptq_nar_kmax_seed0_g128_asym |
nar_kmax | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 14.5 GB |
qwen3_8b_base |
gptq_hadamard_seed0_g128_asym |
hadamard | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 27.8 GB |
qwen3_8b_base |
gptq_nar_k8_seed0_g128_asym |
nar_k8 | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 27.8 GB |
qwen3_8b_base |
gptq_nar_kmax_seed0_g128_asym |
nar_kmax | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 27.8 GB |