Jeff98/MiniCPM5-2B-openvino

This model was converted to OpenVINO from openbmb/MiniCPM5-2B using optimum-intel via the export space.

First make sure you have optimum-intel installed:

pip install optimum-intel

To load your model you can do as follows:

from optimum.intel import OVModelForCausalLM

model_id = "Jeff98/MiniCPM5-2B-openvino"
model = OVModelForCausalLM.from_pretrained(model_id)

MiniCPM5-2B OpenVINO

This repository contains an OpenVINO-converted version of openbmb/MiniCPM5-2B for efficient local inference and deployment with OpenVINO and OpenVINO Model Server (OVMS).

The original model architecture, tokenizer, chat template, and model capabilities are inherited from openbmb/MiniCPM5-2B. This repository only provides an OpenVINO-compatible representation for inference and serving.

Model Details

• Base model: openbmb/MiniCPM5-2B
• Model family: MiniCPM5
• Parameters: approximately 2B
• Framework: OpenVINO
• Primary task: Text generation / chat
• Supported languages: Chinese, English, and other languages supported by the original model
• Target devices: CPU / Intel GPU
• Serving: OpenVINO Model Server (OVMS)

This repository is intended primarily for inference and deployment. It does not modify the model architecture or provide additional fine-tuning.

Repository Contents

The repository contains the OpenVINO model and the files required for LLM serving, including:

openvino_model.xml
openvino_model.bin

openvino_tokenizer.xml
openvino_tokenizer.bin

openvino_detokenizer.xml
openvino_detokenizer.bin

tokenizer_config.json
chat_template.jinja

The OpenVINO tokenizer and detokenizer are provided so that the model can be used directly with OpenVINO GenAI and OpenVINO Model Server.

The chat_template.jinja file is inherited from the original MiniCPM5 model and is required by OVMS when using the OpenAI-compatible Chat Completions API.

Usage with OpenVINO Model Server

The model can be served using OVMS.

Example:

ovms \
  --model_name MiniCPM5-2B \
  --model_path /path/to/MiniCPM5-2B-openvino \
  --rest_port 8000 \
  --cache_size 0 \
  --target_device GPU

On Windows PowerShell:

ovms `
  --model_name MiniCPM5-2B `
  --model_path "C:\path\to\MiniCPM5-2B-openvino" `
  --rest_port 8000 `
  --cache_size 0 `
  --target_device GPU

For CPU inference, use:

--target_device CPU

The available OpenVINO devices depend on your local hardware and OpenVINO installation.

OpenAI-Compatible API

Once OVMS is running, the model can be accessed through its OpenAI-compatible REST API.

Chat Completions

curl http://localhost:8000/v3/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniCPM5-2B",
    "messages": [
      {
        "role": "user",
        "content": "Explain what OpenVINO is in simple terms."
      }
    ],
    "max_tokens": 256
  }'

The chat endpoint uses the original MiniCPM5 chat template included in this repository.

Python Example

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v3",
    api_key="unused"
)

response = client.chat.completions.create(
    model="MiniCPM5-2B",
    messages=[
        {
            "role": "user",
            "content": "Write a Python function to compute Fibonacci numbers."
        }
    ],
    max_tokens=256,
)

print(response.choices[0].message.content)

OpenVINO Conversion

This model was converted from:

openbmb/MiniCPM5-2B

to OpenVINO IR format.

The conversion preserves the original model behavior as closely as possible while enabling inference through the OpenVINO runtime.

The tokenizer and detokenizer are additionally converted to OpenVINO IR so that the model can be served directly through OVMS without relying on a separate Python tokenizer during inference.

Conceptually, the deployment package consists of:

Original MiniCPM5-2B
        │
        ├── Language Model
        │        ↓
        │  OpenVINO Model IR
        │
        ├── Tokenizer
        │        ↓
        │  OpenVINO Tokenizer IR
        │
        ├── Detokenizer
        │        ↓
        │  OpenVINO Detokenizer IR
        │
        └── Chat Template
                 ↓
          chat_template.jinja

Intended Use

This repository is suitable for:

• Local LLM inference
• OpenVINO-based deployment
• Intel CPU/GPU/NPU inference experiments
• OpenVINO Model Server deployment
• OpenAI-compatible local API serving
• Edge and workstation LLM experiments
• Development of local assistants and coding tools

For model capabilities, supported tasks, training details, and evaluation results, please refer to the original model:

openbmb/MiniCPM5-2B

Limitations

This repository does not introduce any additional model training or alignment beyond the original MiniCPM5-2B model.

Therefore, the model inherits the limitations of the original model, including possible:

• factual errors,
• hallucinations,
• incorrect code generation,
• sensitivity to prompt formulation,
• language-dependent performance differences.

The OpenVINO conversion may also introduce small numerical differences compared with inference using the original framework.

Performance and memory consumption depend strongly on:

• OpenVINO version,
• target device,
• model precision,
• driver version,
• context length,
• serving configuration.

This repository is intended for experimentation and inference rather than as a guarantee of identical numerical output across all backends.

Coding and Autocomplete

MiniCPM5-2B can generate and complete source code. However, this repository is not specifically fine-tuned as an IDE Fill-in-the-Middle (FIM) autocomplete model.

For latency-sensitive IDE autocomplete workloads, dedicated code-completion models may provide better completion quality.

MiniCPM5-2B may still be useful for:

• code generation,
• code explanation,
• code modification,
• lightweight local coding assistants,
• general-purpose developer chat.

Acknowledgements

All model architecture, training, tokenizer design, and model capabilities originate from the MiniCPM5 project by OpenBMB.

Original model:

• openbmb/MiniCPM5-2B

This repository only provides the OpenVINO conversion and deployment artifacts.

Please follow the license and usage requirements of the original model.