metadata
license: apache-2.0
base_model: Qwen/Qwen3.5-9B
pipeline_tag: text-generation
tags:
- nvfp4
- gptq
- quantization
- compressed-tensors
- llm-compressor
- vllm
Qwen3.5-9B-NVFP4-GPTQ
GPTQ (Hessian-corrected PTQ) NVFP4 quantization of Qwen/Qwen3.5-9B in vLLM
compressed-tensors nvfp4-pack-quantized format, with calibrated input
global scales (W4A4-servable, default load, SM100+). Strong-PTQ baseline of
the Qwen3.5-9B standardized campaign; 9B companion of
Qwen3.5-27B-NVFP4-GPTQ,
produced by the identical recipe.
How this checkpoint was produced — exact reproduction
| Tooling | llm-compressor 0.12.0 (GPTQModifier), transformers 5.9.0 |
| Base model | Qwen/Qwen3.5-9B (dense qwen3_5, bf16) |
| Recipe | GPTQModifier(targets="Linear", scheme="NVFP4", ignore=["lm_head", "re:.*visual.*", "re:.*mtp.*", "re:.*embed_tokens.*"]); NVFP4 scheme also calibrates static per-tensor input global scales from the same pass |
| Calibration | Fixed 512-conversation set calib_512.jsonl (md5 d1ff8cce1d785f8e51b71eb237fb7a71): random.Random(42).sample(range(n_rows), 512), sorted, from the study's OpenPerfectBlend-derived ChatML train corpus (md5 0406bb3a7a482352360716a1bc5e9e04); stock chat template, max_seq_length=8192, num_calibration_samples=512 |
| Script | gptq_9b.py = the 27B script (experiments/w4a4-qwen35-27b/jbom/gptq_27b.py in the study workspace) with MODEL/OUT set to 9B |
| Hardware | 1xB200 (jbom-03), 2026-07-27 |
| Role | Strong-PTQ baseline row; results filed in qatfactory-experiments qwen3.5-9b/ |
Inference
vllm serve weili-0234/Qwen3.5-9B-NVFP4-GPTQ --max-model-len 33024