---
license: apache-2.0
base_model: Qwen/Qwen3.6-27B
base_model_relation: quantized
library_name: transformers
tags:
- dashq
- quantized
- post-training-quantization
- int3
---
![DASH-Q](https://raw.githubusercontent.com/JaeminK/dashq/main/assets/dashq_banner.png)

# Qwen3.6-27B-DASHQ-INT3-g64

> **DASH-Q** — Diagonal-Aware Shrinkage for Robust PTQ.
> `INT3` · group size 64 · **17.2748 GB** (from 55.5630 GB — **3.2x smaller**)

This checkpoint runs directly with Transformers: the packed quantized layers and a Triton decode kernel are bundled in the repository, so no additional package is needed.

## Usage

```python
from transformers import AutoModelForImageTextToText, AutoTokenizer

model = AutoModelForImageTextToText.from_pretrained(
    "jkim96/Qwen3.6-27B-DASHQ-INT3-g64", trust_remote_code=True, device_map="cuda", dtype="auto"
)
tokenizer = AutoTokenizer.from_pretrained("jkim96/Qwen3.6-27B-DASHQ-INT3-g64")

messages = [{"role": "user", "content": "Explain 2-bit quantization in one sentence."}]
text = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
print(tokenizer.decode(model.generate(**inputs, max_new_tokens=256)[0]))
```

No extra package is required: this repository carries its own inference code
(`modeling_dashq.py`) and a Triton weight-only GEMV decode kernel
(`dashq_kernel.py`). Quantized layers are rebuilt from `dashq_config.json` at load
time and converted to the kernel automatically; on CPU or without Triton the model
falls back to a PyTorch dequantize-and-matmul path.

Requirements: `transformers`, `torch`, and `triton` (bundled with CUDA builds of
PyTorch). The [DASH-Q repository](https://github.com/JaeminK/dashq) is only needed
to quantize your own models.

### Runtime format

| Field | Value |
| --- | --- |
| Weights | group-wise asymmetric integers (3-bit, group size 64) packed into int32 words |
| Decode kernel | Triton K-major GEMV (supports 2/3/4/8-bit at any group size) |
| Prefill | unpack-and-matmul on the same buffers |

## Quantization

| Field | Value |
| --- | --- |
| Base model | `Qwen/Qwen3.6-27B` |
| Precision | INT3, group size 64 |
| Scale / zero dtype | float16 |
| Calibration | wikitext2, 128 samples x 2048 |
| Size | 17.2748 GB  ·  original 55.5630 GB  ·  3.2x compression |

## Benchmarks

Full zero-shot / few-shot results for every DASH-Q checkpoint:
**[github.com/JaeminK/dashq#benchmarks](https://github.com/JaeminK/dashq#benchmarks)**