---
base_model:
- Qwen/Qwen3-0.6B-Base
- Qwen/Qwen3-1.7B-Base
- Qwen/Qwen3-4B-Base
- Qwen/Qwen3-8B-Base
license: apache-2.0
pipeline_tag: text-generation
tags:
- prismquant
- quantization
- gptq
- w4a4kv4
- custom-code
---

# PrismQuant-Qwen3-Base

[Paper](https://arxiv.org/abs/2609.32429) · [Code](https://github.com/ForeverBlue816/PrismQuant) · [Loading guide](https://github.com/ForeverBlue816/PrismQuant/blob/main/docs/models.md)

Official checkpoints for **PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers**. PrismQuant aligns dominant activation directions with the constant directions of asymmetric quantization groups.

## Models

| Model key | Base model | Quantization |
| --- | --- | --- |
| `qwen3_0.6b_base` | [Qwen/Qwen3-0.6B-Base](https://huggingface.co/Qwen/Qwen3-0.6B-Base) | W4A4KV4 |
| `qwen3_1.7b_base` | [Qwen/Qwen3-1.7B-Base](https://huggingface.co/Qwen/Qwen3-1.7B-Base) | W4A4KV4 |
| `qwen3_4b_base` | [Qwen/Qwen3-4B-Base](https://huggingface.co/Qwen/Qwen3-4B-Base) | W4A4KV4 |
| `qwen3_8b_base` | [Qwen/Qwen3-8B-Base](https://huggingface.co/Qwen/Qwen3-8B-Base) | W4A4KV4 |

## Quick start

```bash
git clone --depth 1 https://github.com/ForeverBlue816/PrismQuant.git
cd PrismQuant
pip install -e .
```

```python
from prismquant import load_model

loaded = load_model("qwen3_0.6b_base")
print(loaded.generate("The key idea behind quantization is", max_new_tokens=64))
```

The loader selects the default PrismQuant checkpoint, downloads its required files at a pinned revision, and obtains the matching base model and tokenizer. These are base models for text completion.

## Checkpoint format

The GPTQ INT4 values are stored as **dequantized floating-point tensors**. The reference runtime applies activation and KV quantization while retaining floating-point storage. Use the PrismQuant loader; this repository is not a standalone Transformers checkpoint or a packed INT4 serving model.

Reference compute: **float32**. Budget for the base-model state, activations and KV cache as well as the downloaded tensors.

`config.json` declares the PrismQuant artifact format and default models. `checkpoints/` contains decoder weights; `rotations/` contains the factors required by the loader.

## License

Qwen-derived weights retain [Apache-2.0](LICENSE).