README.md

---
base_model: openbmb/MiniCPM5-2B
base_model_relation: quantized
library_name: mlx
license: apache-2.0
language:
  - en
  - zh
pipeline_tag: text-generation
tags:
  - mlx
  - mlx-lm
  - safetensors
  - minicpm
  - minicpm5
  - llama
  - text-generation
  - conversational
  - quantized
  - quantization
  - mixed-precision
  - apple-silicon
  - macos
  - local-llm
  - local-inference
  - on-device
  - edge-ai
  - long-context
  - 131k-context
  - tool-calling
  - function-calling
  - agent
  - agentic
  - coding
  - reasoning
  - omlx
  - oq
  - oqe
  - uraion-labs
datasets:
  - openbmb/Ultra-FineWeb
  - openbmb/UltraX-Preview
  - openbmb/Ultra-FineWeb-L3
  - openbmb/UltraData-Math
  - openbmb/UltraData-Code
  - openbmb/UltraData-SFT-2605
  - openbmb/UltraData-SFT-Agent-2609
  - openbmb/UltraData-RL-2609
---

<p align="center">
  <img src="https://uraionlabs.com/public/icons/icon-192.png" alt="Uraion Labs" width="56">
</p>

<p align="center">
  <strong>Uraion Labs</strong><br>
  <sub>Foundational systems research.</sub>
</p>

# MiniCPM5-2B — oQ2e

> An MLX mixed-precision quantization of OpenBMB/MiniCPM5-2B for local inference on Apple Silicon, published by Uraion Labs.

**MiniCPM5-2B-oQ2e** is an MLX mixed-precision quantization of [OpenBMB/MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B) optimized for local inference on Apple Silicon Macs. It was quantized and published by Uraion Labs using the oMLX oQe quantization workflow with importance-matrix guided sensitivity allocation. The checkpoint uses the standard MLX format and is directly compatible with both [oMLX](https://github.com/jundot/omlx) and [mlx-lm](https://github.com/ml-explore/mlx-examples/tree/main/llms/mlx_lm).

## About This Quantization

- **Quantization Level**: `oQ2e`
- **Quantization Workflow**: oMLX oQe mixed-precision quantization
- **Base Weight Precision**: 2-bit
- **Mixed-Precision Profile**: Mixed 2/5/6-bit precision (10 @ 5b, 6 @ 6b, lm_head @ 6b)
- **Layer Overrides**: 16 overrides: 10 layers boosted to 5-bit, 6 layers boosted to 6-bit, and lm_head preserved at 6-bit
- **Output Head (`lm_head`) Precision**: 6-bit
- **Group Size**: 64
- **Quantization Mode**: Affine (scale and bias per group)
- **Non-Quantized Tensor Precision**: BF16 (LayerNorm weights, embedding scales and biases)
- **Calibration Dataset**: `oqe_code_multilingual` (294 calibration samples)
- **Model File Size**: 0.88 GB (900.44 MB)
- **Effective Bits Per Weight**: Effective ~2.8 bits per weight maximum compression variant under 900 MB.

## Model Information

| Property | Value |
| :--- | :--- |
| Original Model | [OpenBMB/MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B) |
| Quantization Variant | oQ2e |
| Quantized By | Uraion Labs |
| Architecture | `LlamaForCausalLM` (`llama`) |
| Base Precision | 2-bit |
| Mixed-Precision Allocation | Mixed 2/5/6-bit precision (10 @ 5b, 6 @ 6b, lm_head @ 6b) |
| Output Head (`lm_head`) | 6-bit |
| Quantization Mode | Affine (group size 64) |
| Non-Quantized Precision | BF16 |
| Total Parameters | 2,516,756,480 (~2.52B) |
| Non-Embedding Parameters | 1,981,982,720 (~1.98B) |
| Layers | 42 |
| Attention Configuration | GQA — 16 Q heads / 2 KV heads (head dim 128) |
| Context Length | 131,072 tokens |
| Storage Format | MLX safetensors |
| Target Runtime | Apple Silicon macOS (`omlx`, `mlx-lm`) |
| Model Weight Size | 0.88 GB (900.44 MB) |
| License | Apache-2.0 |

## Upstream MiniCPM5-2B Highlights

MiniCPM5-2B is developed by OpenBMB as the 2B-scale model in the MiniCPM5 series:

- **2B-Class Open-Source State of the Art**: Reaches top-tier performance among models in its size category (averaging 53.9 across OpenBMB's evaluation set) while remaining competitive with 3B and 4B class models.
- **Native 131k Context**: Supports up to 131,072 tokens context window out of the box for document synthesis, repo-wide code reasoning, and long conversation threads.
- **Standard LLaMA Architecture**: Uses standard `LlamaForCausalLM` with Grouped-Query Attention (GQA, 16 Q heads, 2 KV heads), enabling out-of-the-box execution across standard inference runtimes.
- **High-Quality Training Datasets**: Trained using OpenBMB's UltraData data curriculum, including Ultra-FineWeb, UltraX, UltraData-Code, UltraData-SFT, and UltraData-RL.
- **Agentic and Tool-Use Capabilities**: Engineered for high accuracy on function calling, structured output generation, and coding assistant workflows.

## Quantization Family Navigation

| Quant | Base Bits | Mixed-Precision Profile | lm_head | Model Size | Repository |
| :--- | :--- | :--- | :--- | :--- | :--- |
| **oQ8e** | 8-bit | Uniform 8-bit | 8-bit | 2.49 GB | [UraionLabs/MiniCPM5-2B-oQ8e](https://huggingface.co/UraionLabs/MiniCPM5-2B-oQ8e) |
| **oQ6e** | 6-bit | Mixed 6/8-bit (28 layers @ 8-bit) | 6-bit | 1.95 GB | [UraionLabs/MiniCPM5-2B-oQ6e](https://huggingface.co/UraionLabs/MiniCPM5-2B-oQ6e) |
| **oQ5e** | 5-bit | Mixed 5/6/8-bit (19 @ 6b, 6 @ 8b) | 6-bit | 1.67 GB | [UraionLabs/MiniCPM5-2B-oQ5e](https://huggingface.co/UraionLabs/MiniCPM5-2B-oQ5e) |
| **oQ4e** | 4-bit | Mixed 4/5/6-bit (58 @ 5b, 9 @ 6b) | 4-bit | 1.38 GB | [UraionLabs/MiniCPM5-2B-oQ4e](https://huggingface.co/UraionLabs/MiniCPM5-2B-oQ4e) |
| **oQ3.5e** | 3-bit | Mixed 3/5/6-bit (29 @ 5b, 7 @ 6b) | 6-bit | 1.17 GB | [UraionLabs/MiniCPM5-2B-oQ3.5e](https://huggingface.co/UraionLabs/MiniCPM5-2B-oQ3.5e) |
| **oQ3e** | 3-bit | Mixed 3/5/6-bit (31 @ 5b, 7 @ 6b) | 3-bit | 1.08 GB | [UraionLabs/MiniCPM5-2B-oQ3e](https://huggingface.co/UraionLabs/MiniCPM5-2B-oQ3e) |
| **oQ2.7e** | 2-bit | Mixed 2/5/6/8-bit (23 @ 5b, 8 @ 6b, lm_head @ 8b) | 8-bit | 0.98 GB | [UraionLabs/MiniCPM5-2B-oQ2.7e](https://huggingface.co/UraionLabs/MiniCPM5-2B-oQ2.7e) |
| **oQ2e** | 2-bit | Mixed 2/5/6-bit (10 @ 5b, 6 @ 6b) | 6-bit | 0.88 GB | [UraionLabs/MiniCPM5-2B-oQ2e](https://huggingface.co/UraionLabs/MiniCPM5-2B-oQ2e) |

## Quickstart

### Using oMLX

[oMLX](https://github.com/jundot/omlx) provides a high-throughput runtime for MLX models on Apple Silicon:

```bash
# Install oMLX
pip install omlx

# Run inference
omlx run UraionLabs/MiniCPM5-2B-oQ2e --prompt "Explain quantum entanglement in simple terms."
```

### Using mlx-lm

This quantization is fully compatible with Apple's standard [mlx-lm](https://github.com/ml-explore/mlx-examples/tree/main/llms/mlx_lm) library:

```bash
# Install mlx-lm
pip install mlx-lm
```

#### Python Example

```python
from mlx_lm import load, generate

model, tokenizer = load("UraionLabs/MiniCPM5-2B-oQ2e")

prompt = "Explain quantum entanglement in simple terms."
messages = [{"role": "user", "content": prompt}]
prompt_formatted = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)

response = generate(
    model,
    tokenizer,
    prompt=prompt_formatted,
    max_tokens=512,
    temp=1.0,
    top_p=0.95,
    verbose=True,
)
```

#### CLI Example

```bash
python -m mlx_lm.generate \
  --model UraionLabs/MiniCPM5-2B-oQ2e \
  --prompt "Explain quantum entanglement in simple terms." \
  --temp 1.0 \
  --top-p 0.95 \
  --max-tokens 512
```

### Hugging Face Download

To download the repository files locally:

```bash
hf download UraionLabs/MiniCPM5-2B-oQ2e
```

## Recommended Generation Settings

The following sampling parameters are officially recommended by OpenBMB for MiniCPM5-2B:

| Parameter | Recommended Value | Note |
| :--- | :--- | :--- |
| `temperature` | `1.0` | Balanced diversity and reasoning coherence |
| `top_p` | `0.95` | Nucleus sampling threshold |
| `do_sample` | `true` | Stochastic sampling enabled |

When performing deterministic evaluation or greedy decoding, set `temperature=0.0` or `do_sample=false`.

## Evaluation Results and Benchmark Context

> Benchmark results below are reported by OpenBMB for the original MiniCPM5-2B release. This Uraion Labs quantization has not been independently benchmarked unless explicitly stated otherwise.

| Capability / Benchmark | MiniCPM5-2B (Upstream) | Qwen3.5-4B | granite-4.2-3B | LFM2.5-2.6B |
| :--- | :--- | :--- | :--- | :--- |
| **Average Score** | **53.9** | 51.1 | 42.7 | 33.2 |
| **LiveCodeBench v6** | **69.1** | 56.4 | 58.9 | 42.1 |
| **LCB-Pro 25Q2 (Easy)** | **68.0** | 58.3 | 54.6 | 30.9 |
| **AIME 2025** | **86.5** | 78.8 | 79.4 | 41.9 |
| **AIME 2026** | **86.5** | 82.7 | 83.5 | 45.2 |
| **MATH-500** | 94.6 | **99.0** | 97.0 | 89.6 |
| **MMLU-Pro** | 70.8 | **78.0** | 65.8 | 65.2 |
| **MMLU-Redux** | 84.7 | **88.7** | 78.9 | 80.0 |
| **IFBench** | 66.3 | 59.0 | **73.0** | 59.0 |
| **AA-LCR (Long Context)** | **59.0** | 46.1 | 35.8 | 5.3 |

## Limitations and Disclaimer

This model generates text based on statistical language modeling patterns and may produce inaccurate, biased, or hallucinated statements. Responses regarding sensitive or specialized domains (such as law, medical treatment, or financial investment) must not be treated as professional advice.

This model is provided "AS IS", without warranty of any kind, express or implied. Users are responsible for verifying outputs, implementing safety guardrails, and complying with all applicable local regulations and acceptable use policies.

## Original Model and Attribution

This repository contains a quantized derivative of [OpenBMB/MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B).

MiniCPM5-2B was developed and trained by OpenBMB. Model architecture, pre-training, instruction tuning, benchmarks, and capabilities described on this page originate from the upstream MiniCPM5-2B release.

Training-data metadata is inherited from the upstream OpenBMB/MiniCPM5-2B release. Uraion Labs performed quantization only and did not retrain this checkpoint.

### Upstream Resources
- **Model Card**: [openbmb/MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B)
- **GitHub Repository**: [OpenBMB/MiniCPM](https://github.com/OpenBMB/MiniCPM)
- **MiniCPM Technical Report**: [arXiv:2506.07900](https://arxiv.org/abs/2506.07900)
- **UltraData Paper**: [arXiv:2602.09003](https://arxiv.org/abs/2602.09003)
- **Online Demo**: [MiniCPM5-2B-Demo](https://huggingface.co/spaces/openbmb/MiniCPM5-2B-Demo)
- **OpenBMB MLX Deployment Guide**: [mlx.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/mlx.md)

## License

This repository and the underlying MiniCPM model weights are released under the [Apache-2.0](https://github.com/OpenBMB/MiniCPM/blob/main/LICENSE) License.

## Citation

Please cite the original MiniCPM research when referencing this work:

```bibtex
@article{minicpm4,
  title={Minicpm4: Ultra-efficient llms on end devices},
  author={MiniCPM, Team},
  journal={arXiv preprint arXiv:2506.07900},
  year={2025}
}
```

<p align="center">
  <sub>Quantized and published by <a href="https://uraionlabs.com">Uraion Labs</a>. Foundational systems research.</sub>
</p>