---
license: apache-2.0
library_name: transformers
pipeline_tag: image-text-to-text
base_model: Qwen/Qwen3.5-27B
base_model_relation: quantized
tags:
  - transformers
  - safetensors
  - qwen3_5
  - quantized
  - gptq
  - int4
  - w4a16
  - 4-bit
  - vllm
  - llm-compressor
  - image-text-to-text
  - conversational
datasets:
  - HuggingFaceH4/ultrachat_200k
---

# Qwen3.5-27B-quantized.w4a16

This is a quantized version of [Qwen/Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B). This model accepts text and images as inputs and generates text as outputs. The weights were quantized to INT4 using GPTQ via [llm-compressor](https://github.com/vllm-project/llm-compressor) with 512 calibration samples from [HuggingFaceH4/ultrachat_200k](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k), reducing the model size from 51.8 GB to 17.3 GB (~3.0x reduction) while maintaining 100.3% average accuracy recovery.

---

## Inference

As of 2/27/2026, this model is supported in vLLM nightly. To serve the model:

```bash
vllm serve Kbenkhaled/Qwen3.5-27B-quantized.w4a16 \
    --reasoning-parser qwen3 \
    --enable-prefix-caching
```

---

## Evaluation

Evaluated with [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness), 0-shot, thinking mode ON.

| Benchmark | Qwen3.5-27B | Qwen3.5-27B-quantized.w4a16 (this model) | Recovery |
|---|---|---|---|
| GPQA Diamond | 80.30% | 80.81% | 100.6% |
| IFEval | 95.08% | 95.20% | 100.1% |
| MMLU-Redux | 93.90% | 94.13% | 100.2% |
| **Average** | **89.76%** | **90.05%** | **100.3%** |