| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3.5-9B |
| pipeline_tag: text-generation |
| tags: |
| - nvfp4 |
| - gptq |
| - quantization |
| - compressed-tensors |
| - llm-compressor |
| - vllm |
| --- |
| |
| # Qwen3.5-9B-NVFP4-GPTQ |
|
|
| GPTQ (Hessian-corrected PTQ) NVFP4 quantization of Qwen/Qwen3.5-9B in vLLM |
| compressed-tensors `nvfp4-pack-quantized` format, **with calibrated input |
| global scales** (W4A4-servable, default load, SM100+). Strong-PTQ baseline of |
| the Qwen3.5-9B standardized campaign; 9B companion of |
| [Qwen3.5-27B-NVFP4-GPTQ](https://huggingface.co/weili-0234/Qwen3.5-27B-NVFP4-GPTQ), |
| produced by the identical recipe. |
|
|
| ## How this checkpoint was produced — exact reproduction |
|
|
| | | | |
| |---|---| |
| | Tooling | [llm-compressor](https://github.com/vllm-project/llm-compressor) **0.12.0** (`GPTQModifier`), transformers 5.9.0 | |
| | Base model | [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) (dense `qwen3_5`, bf16) | |
| | Recipe | `GPTQModifier(targets="Linear", scheme="NVFP4", ignore=["lm_head", "re:.*visual.*", "re:.*mtp.*", "re:.*embed_tokens.*"])`; NVFP4 scheme also calibrates static per-tensor input global scales from the same pass | |
| | Calibration | Fixed 512-conversation set `calib_512.jsonl` (md5 `d1ff8cce1d785f8e51b71eb237fb7a71`): `random.Random(42).sample(range(n_rows), 512)`, sorted, from the study's OpenPerfectBlend-derived ChatML train corpus (md5 `0406bb3a7a482352360716a1bc5e9e04`); stock chat template, `max_seq_length=8192`, `num_calibration_samples=512` | |
| | Script | `gptq_9b.py` = the 27B script (`experiments/w4a4-qwen35-27b/jbom/gptq_27b.py` in the study workspace) with MODEL/OUT set to 9B | |
| | Hardware | 1xB200 (jbom-03), 2026-07-27 | |
| | Role | Strong-PTQ baseline row; results filed in [qatfactory-experiments](https://github.com/Weili-0234/qatfactory-experiments) `qwen3.5-9b/` | |
|
|
| ## Inference |
|
|
| ```bash |
| vllm serve weili-0234/Qwen3.5-9B-NVFP4-GPTQ --max-model-len 33024 |
| ``` |
|
|
