---
library_name: gguf
pipeline_tag: text-generation
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3.5-4B/blob/main/LICENSE
base_model:
- Qwen/Qwen3.5-4B
base_model_relation: quantized
language:
- zh
- en
tags:
- qwen3.5
- gguf
- q4_k_m
- imatrix
- llama.cpp
- lynn
quantized_by: Lynn (nerkyor)
---

# Qwen3.5-4B Q4_K_M (imatrix) - Lynn calibration

This repository ships an **imatrix-calibrated Q4_K_M GGUF** of the official
[Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) BF16 weights, built on
DGX Spark (GB10, sm_121) by the Lynn team.

This is a pure quantization of the upstream model. It is not a distillation and
not a Lynn-native NVFP4/W4A8 checkpoint.

## Files

| File | Size | SHA256 | Role |
|---|---:|---|---|
| `Qwen3.5-4B-Q4_K_M-imatrix.gguf` | 2.6 GB | `7abaf02bbe25c608deb308db526766f761ad4fb85c512a69ff36520c4b304b23` | llama.cpp / Ollama / LM Studio GGUF weights |
| `Qwen3.5-4B.imatrix` | 3.5 MB | `863a93c58a14925b58303a369d9bb155411b40d52fde121f195eaf7a6691f07c` | imatrix calibration data used for quantization |

## Build Details

- Source weights: `Qwen/Qwen3.5-4B` official BF16
- Converter: `llama.cpp convert_hf_to_gguf.py --outtype f16`
- Calibration: Lynn Chinese + English + code mix, 256 chunks, 512 ctx
- Quantizer: `llama-quantize --imatrix Qwen3.5-4B.imatrix ... Q4_K_M`
- Built on: DGX Spark (GB10, sm_121), 2026-05-24

## Evaluation

V8/V9, MMLU500, and GPQA Diamond thinking-on evaluations are running on Spark.
Those artifacts and scores will be added in a follow-up update.

## Run

```bash
llama-server \
  -m Qwen3.5-4B-Q4_K_M-imatrix.gguf \
  --host 0.0.0.0 --port 18099 \
  --ctx-size 32768 --n-gpu-layers 999 \
  --jinja --reasoning on
```

The GGUF embeds the upstream Qwen3.5 chat template, including thinking-mode
support via `chat_template_kwargs.enable_thinking`.

## License

Apache 2.0, inherited from [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B).