---
license: apache-2.0
language:
  - en
  - zh
base_model: openbmb/MiniCPM5-2B
base_model_relation: quantized
pipeline_tag: text-generation
library_name: gguf
tags:
  - gguf
  - llama.cpp
  - minicpm5
  - long-context
  - 131k-context
  - dense
  - tool-calling
---

# MiniCPM5-2B GGUF

GGUF quantizations of [OpenBMB/MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B), a 2B dense Llama-based model for local deployment, coding, reasoning, long-context work, and tool use. The source checkpoint supports a native context length of **131,072 tokens (128K)**.

## Benchmarks

![MiniCPM5-2B evaluation results](assets/minicpm5-benchmark.png)

*Benchmark results reported by OpenBMB for the original MiniCPM5-2B checkpoint.*

![MiniCPM5-2B capability radar](assets/minicpm5-benchmark2.png)

*Capability comparison reported by OpenBMB for the original MiniCPM5-2B checkpoint.*

## GGUF files

| Quantization | File | Size |
| --- | --- | ---: |
| Q2_K | [MiniCPM5-2B-Q2_K.gguf](MiniCPM5-2B-Q2_K.gguf) | 1.04 GB |
| Q3_K_M | [MiniCPM5-2B-Q3_K_M.gguf](MiniCPM5-2B-Q3_K_M.gguf) | 1.29 GB |
| Q4_0 | [MiniCPM5-2B-Q4_0.gguf](MiniCPM5-2B-Q4_0.gguf) | 1.49 GB |
| Q4_K_S | [MiniCPM5-2B-Q4_K_S.gguf](MiniCPM5-2B-Q4_K_S.gguf) | 1.50 GB |
| Q4_K_M | [MiniCPM5-2B-Q4_K_M.gguf](MiniCPM5-2B-Q4_K_M.gguf) | 1.56 GB |
| Q5_K_M | [MiniCPM5-2B-Q5_K_M.gguf](MiniCPM5-2B-Q5_K_M.gguf) | 1.81 GB |
| Q6_K | [MiniCPM5-2B-Q6_K.gguf](MiniCPM5-2B-Q6_K.gguf) | 2.07 GB |
| Q8_0 | [MiniCPM5-2B-Q8_0.gguf](MiniCPM5-2B-Q8_0.gguf) | 2.68 GB |
| IQ2_M | [MiniCPM5-2B-IQ2_M.gguf](MiniCPM5-2B-IQ2_M.gguf) | 0.97 GB |
| IQ3_M | [MiniCPM5-2B-IQ3_M.gguf](MiniCPM5-2B-IQ3_M.gguf) | 1.23 GB |
| IQ4_XS | [MiniCPM5-2B-IQ4_XS.gguf](MiniCPM5-2B-IQ4_XS.gguf) | 1.42 GB |

The model is text-only. No vision projector or MTP files are included. The IQ files use an importance matrix generated from WikiText-2 and are intended for recent llama.cpp builds. SHA-256 checksums are provided in [`SHA256SUMS.txt`](SHA256SUMS.txt).

## Chat template

The GGUF files embed the upstream chat template. [`chat_template.jinja`](chat_template.jinja) is provided as an external copy for runtimes that require a separate template file.

## Usage

Use a current [llama.cpp](https://github.com/ggml-org/llama.cpp) build with MiniCPM5 support. The example below uses an 8K context; increase `-c` up to `131072` when sufficient memory is available.

```bash
llama-cli \
  -m MiniCPM5-2B-Q4_K_M.gguf \
  -c 8192 --jinja \
  --temp 1.0 --top-p 0.95 \
  -p "Explain why reproducible builds matter."
```

For an OpenAI-compatible server:

```bash
llama-server \
  -m MiniCPM5-2B-Q4_K_M.gguf \
  -c 8192 --jinja --host 0.0.0.0 --port 8080
```

Tool-call behavior depends on the serving runtime's parser and API integration; use the embedded template and verify tool calls in the target application.

## Source

- Model: [OpenBMB/MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B)
- Source revision: [`3497c460c89e00520c3cfa2e73f49ab7647f1177`](https://huggingface.co/openbmb/MiniCPM5-2B/tree/3497c460c89e00520c3cfa2e73f49ab7647f1177)
- Conversion: the original Q4_0–Q8_0 files use upstream llama.cpp commit `f114f91f9ed6792cf402437e3874adad98902744`; the additional Q2_K, Q3_K_M, Q4_K_S, IQ2_M, IQ3_M, and IQ4_XS files use upstream commit `67672dc5b76f8bc17785a19d3dc6d1463fc2902c`
- License: [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)
- Checksums: [SHA256SUMS.txt](SHA256SUMS.txt)