---
license: apache-2.0
tags: [quantization, w4a4, gptq, rotation, research-artifact]
---

# NAR W4A4KV4 GPTQ checkpoints (qwen3_0.6b_base, qwen3_1.7b_base, qwen3_4b_base, qwen3_8b_base)

Private research artifact from [nar-offline-validation](https://github.com/ForeverBlue816/nar-offline-validation), report section "E22 — the Qwen3 Base family under one protocol".
Base model(s): [`Qwen/Qwen3-0.6B-Base`](https://huggingface.co/Qwen/Qwen3-0.6B-Base), [`Qwen/Qwen3-1.7B-Base`](https://huggingface.co/Qwen/Qwen3-1.7B-Base), [`Qwen/Qwen3-4B-Base`](https://huggingface.co/Qwen/Qwen3-4B-Base), [`Qwen/Qwen3-8B-Base`](https://huggingface.co/Qwen/Qwen3-8B-Base).

## What the files are

`checkpoints/<model>/<checkpoint>/layer_XX.pt` holds one decoder layer's weights after the rotation
(Hadamard or NAR) has been folded into the base model and GPTQ has quantized them. The values are the
**dequantized** INT4 weights stored in floating point (fake quantization), not a packed INT4 format, so
`transformers` cannot load them on its own. `DONE.json` records the fold audit, GPTQ settings and
provenance (hostnames and GPU ids redacted). `gptq_audit.csv` has per-module GPTQ statistics.

At inference the code also quantizes activations per group of 128 and the KV cache (KIVI), and applies
the online rotations (R2 on V, R4 at the down-projection input). Those need the calibrated factors in
`rotations/<model>/`.

## Loading

Clone the code repository, download this repo so that `checkpoints/<model>/` sits at
`<workdir>/artifacts/e22/<model>/` and `rotations/<model>/` at
`<workdir>/activations/<model>/`, then use `nar/e14_w4a4kv4.py::load_quantized_model` (as the
experiment's `evaluate` command does).

## Checkpoints

| model | checkpoint | rotation | seed | weight protocol | size |
|---|---|---|---:|---|---:|
| `qwen3_0.6b_base` | `gptq_hadamard_seed0_g128_asym` | hadamard | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 1.8 GB |
| `qwen3_0.6b_base` | `gptq_nar_k8_seed0_g128_asym` | nar_k8 | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 1.8 GB |
| `qwen3_0.6b_base` | `gptq_nar_kmax_seed0_g128_asym` | nar_kmax | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 1.8 GB |
| `qwen3_1.7b_base` | `gptq_hadamard_seed0_g128_asym` | hadamard | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 5.6 GB |
| `qwen3_1.7b_base` | `gptq_nar_k8_seed0_g128_asym` | nar_k8 | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 5.6 GB |
| `qwen3_1.7b_base` | `gptq_nar_kmax_seed0_g128_asym` | nar_kmax | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 5.6 GB |
| `qwen3_4b_base` | `gptq_hadamard_seed0_g128_asym` | hadamard | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 14.5 GB |
| `qwen3_4b_base` | `gptq_nar_k8_seed0_g128_asym` | nar_k8 | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 14.5 GB |
| `qwen3_4b_base` | `gptq_nar_kmax_seed0_g128_asym` | nar_kmax | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 14.5 GB |
| `qwen3_8b_base` | `gptq_hadamard_seed0_g128_asym` | hadamard | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 27.8 GB |
| `qwen3_8b_base` | `gptq_nar_k8_seed0_g128_asym` | nar_k8 | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 27.8 GB |
| `qwen3_8b_base` | `gptq_nar_kmax_seed0_g128_asym` | nar_kmax | 0 | GPTQ group-128 asymmetric INT4 weights (4.156 bits) | 27.8 GB |