---
license: apache-2.0
base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct
language:
  - en
  - multilingual
tags:
  - gguf
  - qwen
  - qwen2.5-coder
  - code
  - qwen2
  - llama.cpp
  - q4_k_m
  - quantization
pipeline_tag: text-generation
---

# Qwen2.5-Coder-1.5B-Instruct (Q4_K_M GGUF)

A compact **GGUF** quantization of
[Qwen/Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct) —
a code-focused 1.5B instruction model.

This card covers the single file:

- [`Qwen2.5-Coder-1.5B-Instruct-f16-q4_k_m.gguf`](Qwen2.5-Coder-1.5B-Instruct-f16-q4_k_m.gguf)

| Property | Value |
|---|---|
| File | `Qwen2.5-Coder-1.5B-Instruct-f16-q4_k_m.gguf` |
| Size | 940.37 MiB (986.05 MB) |
| Quantization | **Q4_K_M** (≈ 4.85 bits/weight) |
| Parameters | 1.54B |
| Architecture | Qwen2 |
| Context length | 32768 tokens |
| GGUF version | 3 (latest) |

## Why this quantization

`Q4_K_M` is the sweet spot for this model on **low-VRAM hardware**: ~0.94 GB
of weights fit any 4 GB-class GPU (e.g. GTX 1650) with plenty of headroom for
the KV cache. It keeps the model fluent for code generation, debugging and
algorithm explanations while being a fraction of the F16 size (2.9 GB).

Unlike reasoning (R1-style) models, Qwen2.5-Coder answers directly — no
mandatory chain-of-thought — so it is fast and responsive in interactive
agents and never falls into a "thinking loop".

It was produced from the F16 source with **llama.cpp**, and the advertised
context was capped at 32768 tokens so the KV cache fits a 4 GB GPU when fully
offloaded (`-ngl 999`).

## Usage

### llama.cpp (CLI)

```bash
llama-cli \
  -m Qwen2.5-Coder-1.5B-Instruct-f16-q4_k_m.gguf \
  -p "Write a Python function to reverse a linked list." \
  -n 256 \
  -ngl 999 \
  -c 32768
```

### llama-server (OpenAI-compatible API)

```bash
llama-server \
  -m Qwen2.5-Coder-1.5B-Instruct-f16-q4_k_m.gguf \
  -ngl 999 \
  -c 32768 \
  --port 8080
# curl http://localhost:8080/v1/chat/completions ...
```

### Docker Model Runner (`docker model`)

```bash
docker model package \
  --gguf Qwen2.5-Coder-1.5B-Instruct-f16-q4_k_m.gguf \
  --context-size 32768 \
  qwen-coder
docker model run qwen-coder "Write a Python function to reverse a linked list."
```

### Ollama

```bash
ollama create qwen-coder -f Modelfile   # FROM ./Qwen2.5-Coder-1.5B-Instruct-f16-q4_k_m.gguf
ollama run qwen-coder
```

## Quality expectations

Q4_K_M on a 1.5B coder model handles common programming tasks well: function
writing, debugging, algorithm explanations, and simple refactors. Larger or
highly nuanced codebases may exceed its capacity (1.5B is a small model).
Quality is noticeably higher than IQ-type quantizations of the same size; for
the same VRAM footprint prefer `Q4_K_M` over `Q4_0`/`IQ3_XXS`.

## Credits & license

- Base model: [Qwen/Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct) (Apache 2.0)
- Source GGUF: [bartowski/Qwen2.5-Coder-1.5B-Instruct-GGUF](https://huggingface.co/bartowski/Qwen2.5-Coder-1.5B-Instruct-GGUF) (MIT)
- Quantized with [llama.cpp](https://github.com/ggml-org/llama.cpp)
- This file is released under the **Apache 2.0** license.