Text Generation
Transformers
Safetensors
English
Chinese
llama
minicpm
minicpm5
long-context
tool-calling
on-device
edge-ai
conversational
text-generation-inference
Instructions to use openbmb/MiniCPM5-2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use openbmb/MiniCPM5-2B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="openbmb/MiniCPM5-2B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("openbmb/MiniCPM5-2B") model = AutoModelForCausalLM.from_pretrained("openbmb/MiniCPM5-2B", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use openbmb/MiniCPM5-2B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "openbmb/MiniCPM5-2B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openbmb/MiniCPM5-2B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/openbmb/MiniCPM5-2B
- SGLang
How to use openbmb/MiniCPM5-2B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "openbmb/MiniCPM5-2B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openbmb/MiniCPM5-2B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "openbmb/MiniCPM5-2B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openbmb/MiniCPM5-2B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use openbmb/MiniCPM5-2B with Docker Model Runner:
docker model run hf.co/openbmb/MiniCPM5-2B
update README
Browse files- README-cn.md +0 -46
- README.md +0 -1
README-cn.md
CHANGED
|
@@ -147,7 +147,6 @@ datasets:
|
|
| 147 |
|
| 148 |
📂 **开放高质量数据**:与模型一同开源其背后的高质量训练数据,均属于 [UltraData](https://ultradata.openbmb.cn/) 数据体系:[UltraX](https://huggingface.co/datasets/openbmb/UltraX-Preview),高质量网页预训练数据集;[UltraData-Code](https://huggingface.co/datasets/openbmb/UltraData-Code),L0–L3 分级代码治理,推动代码能力显著跃升;[UltraData-SFT-Agent-2609](https://huggingface.co/datasets/openbmb/UltraData-SFT-Agent-2609),50 万 Agent 训练样本,赋能端侧 Agent 综合能力提升;[UltraData-RL-2609](https://huggingface.co/datasets/openbmb/UltraData-RL-2609),超 8 万条高质量 RL 训练样本,覆盖数学、代码、通用知识与长文本推理。
|
| 149 |
|
| 150 |
-
|
| 151 |
## 模型列表
|
| 152 |
|
| 153 |
你可以按运行环境选择对应模型格式:
|
|
@@ -183,8 +182,6 @@ MiniCPM5-2B 具有以下特性:
|
|
| 183 |
- **注意力头(GQA)**:16 个 Q heads / 2 个 KV heads
|
| 184 |
- **上下文长度**:131,072
|
| 185 |
|
| 186 |
-
|
| 187 |
-
|
| 188 |
## 简介
|
| 189 |
|
| 190 |
MiniCPM5-2B 是 MiniCPM5 系列的第二个模型,面向本地助手、coding agent、工具调用流程以及需要紧凑模型的推理场景。它在较小部署成本下提供原生长上下文能力。
|
|
@@ -195,7 +192,6 @@ MiniCPM5-2B 是 MiniCPM5 系列的第二个模型,面向本地助手、coding
|
|
| 195 |
|
| 196 |
在这组对比中,MiniCPM5-2B 达到同尺寸开源模型 SOTA 水平(平均分 53.9 ),也超过了参与对比的全部更大规模模型(最高 51.1)。其优势主要体现在代码推理、数学推理、长文本、工具调用与多个智能体任务上。
|
| 197 |
|
| 198 |
-
|
| 199 |
<div style="width:100%;max-width:1080px;margin:0 auto;padding:16px 0;background:#fff;
|
| 200 |
font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,'PingFang SC',
|
| 201 |
'Hiragino Sans GB','Microsoft YaHei',sans-serif;color:#171717">
|
|
@@ -266,18 +262,14 @@ MiniCPM5-2B 的训练过程是 **[UltraData 分级数据管理体系](https://ar
|
|
| 266 |
|
| 267 |
**RL + OPD** 是 MiniCPM5-2B 后训练中的关键环节。**RL** 阶段,使用了 [JustRL II](https://app.notion.com/p/panhaoxuan/JustRL-II-Scaling-Small-LLMs-to-128K-Reasoning-with-a-Critic-3c77e972297c80adb8b5f4b05d267012#5f3eb56b29f048ebab5f71138f12e36f) 阐述的 critic-based 算法,大幅提升训练稳定性,并在多个领域取得了显著的收益。在下面列出的基准中,RL + OPD 在推理与通用能力上平均提升 **↑ 10.96 分**,Agent 能力平均提升 **↑ 6.96 分**。
|
| 268 |
|
| 269 |
-
|
| 270 |
**OPD** 阶段对 16 个 RL 训练所得到的专家模型(含 5 个 agentic 专家模型)实现了能力合并。训练方式上,我们在 response 序列的每个位置分别对学生模型和教师模型 logits 计算全词表的反向 KL 散度作为优势估计值,替代原有的 verification-based advantage;训练数据上,我们的 OPD 直接复用各 RL teacher 训练时 prompt 作为蒸馏数据,无需额外构造语料。
|
| 271 |
|
| 272 |

|
| 273 |
|
| 274 |

|
| 275 |
|
| 276 |
-
|
| 277 |
## 快速上手
|
| 278 |
|
| 279 |
-
|
| 280 |
-
|
| 281 |
### vLLM
|
| 282 |
|
| 283 |
```bash
|
|
@@ -296,8 +288,6 @@ curl http://localhost:8000/v1/chat/completions \
|
|
| 296 |
}'
|
| 297 |
```
|
| 298 |
|
| 299 |
-
|
| 300 |
-
|
| 301 |
### SGLang
|
| 302 |
|
| 303 |
```bash
|
|
@@ -328,8 +318,6 @@ python -m sglang.launch_server \
|
|
| 328 |
--port 30000
|
| 329 |
```
|
| 330 |
|
| 331 |
-
|
| 332 |
-
|
| 333 |
### Transformers
|
| 334 |
|
| 335 |
```bash
|
|
@@ -369,15 +357,12 @@ python -m sglang.launch_server --model-path openbmb/MiniCPM5-2B --port 30000 \
|
|
| 369 |
--tool-call-parser minicpm5 # 或:--tool-call-parser auto
|
| 370 |
```
|
| 371 |
|
| 372 |
-
|
| 373 |
-
|
| 374 |
## GitHub Cookbooks 与 Agent Skills
|
| 375 |
|
| 376 |
MiniCPM5-2B 使用**标准** `LlamaForCausalLM` **架构**,主流推理引擎可直接加载,**无需自定义算子,也无模型代码 fork**。逐步部署和微调说明请参考下方 GitHub cookbooks;Agent Skills 作为 GitHub 资源提供给使用 Cursor / Claude Code 类 coding agent 的用户。
|
| 377 |
|
| 378 |
### 部署
|
| 379 |
|
| 380 |
-
|
| 381 |
| 后端 | 模型格式 / 适用场景 | Cookbook | Agent Skill |
|
| 382 |
| ------------ | ------------------------------------------- | ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
|
| 383 |
| Transformers | BF16 / FP16,本地 Python 推理,GPU + CPU | [transformers.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/transformers.md) | [minicpm5-deploy-transformers](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-transformers/SKILL.md) |
|
|
@@ -390,12 +375,8 @@ MiniCPM5-2B 使用**标准** `LlamaForCausalLM` **架构**,主流推理引擎
|
|
| 390 |
| ArcLight | GGUF 本地端侧 / CPU / 桌面 / 服务器 | [arclight.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/arclight.md) | [minicpm5-deploy-arclight](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-arclight/SKILL.md) |
|
| 391 |
| vLLM Ascend | BF16 / FP16 OpenAI server | [vllm_ascend.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/vllm_ascend.md) | [minicpm5-deploy-vllm-ascend](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-vllm-ascend/SKILL.md) |
|
| 392 |
|
| 393 |
-
|
| 394 |
-
|
| 395 |
-
|
| 396 |
### 微调
|
| 397 |
|
| 398 |
-
|
| 399 |
| 框架 | 适用场景 | Cookbook | Agent Skill |
|
| 400 |
| ------------- | ------------- | --------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
|
| 401 |
| TRL + PEFT | LoRA / SFT 微调 | [trl.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/finetune/trl.md) | [minicpm5-finetune-trl](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-finetune-trl/SKILL.md) |
|
|
@@ -403,9 +384,6 @@ MiniCPM5-2B 使用**标准** `LlamaForCausalLM` **架构**,主流推理引擎
|
|
| 403 |
| ms-swift | 微调 | [ms_swift.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/finetune/ms_swift.md) | [minicpm5-finetune-ms-swift](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-finetune-ms-swift/SKILL.md) |
|
| 404 |
| unsloth | 微调 | [unsloth.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/finetune/unsloth.md) | [minicpm5-finetune-unsloth](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-finetune-unsloth/SKILL.md) |
|
| 405 |
|
| 406 |
-
|
| 407 |
-
|
| 408 |
-
|
| 409 |
### 其他支持的框架
|
| 410 |
|
| 411 |
除上文列出的部署与微调框架外,MiniCPM5-2B 也支持通过 FlagOS 进行多芯片部署。
|
|
@@ -418,7 +396,6 @@ FlagOS 社区致力于打造面向多种 AI 芯片的统一、开源的系统软
|
|
| 418 |
|
| 419 |
官网速递:[https://flagos.io](https://flagos.io/)
|
| 420 |
|
| 421 |
-
|
| 422 |
<details>
|
| 423 |
<summary>FlagOS 多 AI 芯片支持与使用方式</summary>
|
| 424 |
|
|
@@ -426,7 +403,6 @@ FlagOS 社区致力于打造面向多种 AI 芯片的统一、开源的系统软
|
|
| 426 |
|
| 427 |
基于 FlagOS 极短时间内适配 MiniCPM5-2B 到 9 种不同的 AI 芯片,得益于众智 FlagOS 的多芯片统一 AI 系统软件栈的能力。目前,在 FlagOS 团队构建的面向多架构人工智能芯片的大模型自动迁移、适配与发布平台 FlagRelease 上,已发布 MiniCPM5-2B 的多芯片版本。细节如下:
|
| 428 |
|
| 429 |
-
|
| 430 |
| Vendor | ModelScope | Huggingface |
|
| 431 |
| --------- | ------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
|
| 432 |
| Nvidia | [MiniCPM5-2B-nvidia-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-nvidia-FlagOS) | [MiniCPM5-2B-nvidia-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-nvidia-FlagOS) |
|
|
@@ -439,28 +415,18 @@ FlagOS 社区致力于打造面向多种 AI 芯片的统一、开源的系统软
|
|
| 439 |
| Ascend | [MiniCPM5-2B-ascend-FlagOS](https://modelscope.cn/models/FlagRelease/MiniCPM5-2B-ascend-FlagOS) | [MiniCPM5-2B-ascend-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-ascend-FlagOS) |
|
| 440 |
| ARM-v9 | [MiniCPM5-2B-Armv9-FlagOS](https://modelscope.cn/models/FlagRelease/MiniCPM5-2B-Armv9-FlagOS) | [MiniCPM5-2B-Armv9-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-Armv9-FlagOS) |
|
| 441 |
|
| 442 |
-
|
| 443 |
-
|
| 444 |
-
|
| 445 |
#### FlagOS 使用方式
|
| 446 |
|
| 447 |
-
|
| 448 |
-
|
| 449 |
##### 使用 FlagOS 在 Nvidia 体验性能加速
|
| 450 |
|
| 451 |
-
|
| 452 |
-
|
| 453 |
###### From FlagRelease(**推荐**)
|
| 454 |
|
| 455 |
FlagRelease是FlagOS团队构建的一套面向多架构人工智能芯片的大模型自动迁移、适配与发布平台,已发布MiniCPM5-2B的多芯片版本。FlagRelease 已内置相关软件包,无需用户安装。
|
| 456 |
|
| 457 |
###### FlagRelease 镜像关键版本信息
|
| 458 |
|
| 459 |
-
|
| 460 |
-
|
| 461 |
###### FlagRelease 使用速递
|
| 462 |
|
| 463 |
-
|
| 464 |
| Vendor | ModelScope | Huggingface |
|
| 465 |
| --------- | ------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
|
| 466 |
| Nvidia | [MiniCPM5-2B-nvidia-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-nvidia-FlagOS) | [MiniCPM5-2B-nvidia-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-nvidia-FlagOS) |
|
|
@@ -473,19 +439,12 @@ FlagRelease是FlagOS团队构建的一套面向多架构人工智能芯片的大
|
|
| 473 |
| Ascend | [MiniCPM5-2B-ascend-FlagOS](https://modelscope.cn/models/FlagRelease/MiniCPM5-2B-ascend-FlagOS) | [MiniCPM5-2B-ascend-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-ascend-FlagOS) |
|
| 474 |
| ARM-v9 | [MiniCPM5-2B-Armv9-FlagOS](https://modelscope.cn/models/FlagRelease/MiniCPM5-2B-Armv9-FlagOS) | [MiniCPM5-2B-Armv9-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-Armv9-FlagOS) |
|
| 475 |
|
| 476 |
-
|
| 477 |
-
|
| 478 |
-
|
| 479 |
###### 从零开始
|
| 480 |
|
| 481 |
- 依赖Python3.12, GLIBC_2.39, GLIBCXX_3.4.33, CXXABI_1.3.15 环境
|
| 482 |
|
| 483 |
-
|
| 484 |
-
|
| 485 |
###### Vllm 版本
|
| 486 |
|
| 487 |
-
|
| 488 |
-
|
| 489 |
###### 安装 FlagOS 算子库
|
| 490 |
|
| 491 |
官方仓库:[https://github.com/flagos-ai/FlagGems](https://github.com/flagos-ai/FlagGems)
|
|
@@ -495,8 +454,6 @@ pip install flag-gems==4.2.1rc0
|
|
| 495 |
pip install triton==3.5.1
|
| 496 |
```
|
| 497 |
|
| 498 |
-
|
| 499 |
-
|
| 500 |
###### 开启加速
|
| 501 |
|
| 502 |
通过在vllm执行推理的源码中增加flagGems的导入即可开启flagGems加速
|
|
@@ -516,15 +473,12 @@ vllm serve ${model_path} \
|
|
| 516 |
--gpu-memory-utilization 0.85
|
| 517 |
```
|
| 518 |
|
| 519 |
-
|
| 520 |
-
|
| 521 |
##### 使用 FlagOS 统一多芯片后端插件
|
| 522 |
|
| 523 |
**[vllm-plugin-FL](https://github.com/flagos-ai/vllm-plugin-FL)** 是一个为 **vLLM** 推理/服务框架构建的插件,它基于 **FlagOS 的统一多芯片后端**开发,旨在扩展 vLLM 在多种硬件环境下的功能和性能表现。
|
| 524 |
|
| 525 |
###### vllm-plugin-FL 使用
|
| 526 |
|
| 527 |
-
|
| 528 |
| 厂商 | 从零开始 | 从 FlagRelease 开始 | |
|
| 529 |
| --- | ---------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
|
| 530 |
| 英伟达 | [vllm-plugin-FL/MiniCPM5-2B](https://github.com/flagos-ai/vllm-plugin-FL/blob/main/examples/minicpm/README.md) | [MiniCPM5-2B-ModelScope](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-nvidia-FlagOS) | [MiniCPM5-2B-nvidia-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-nvidia-FlagOS) |
|
|
|
|
| 147 |
|
| 148 |
📂 **开放高质量数据**:与模型一同开源其背后的高质量训练数据,均属于 [UltraData](https://ultradata.openbmb.cn/) 数据体系:[UltraX](https://huggingface.co/datasets/openbmb/UltraX-Preview),高质量网页预训练数据集;[UltraData-Code](https://huggingface.co/datasets/openbmb/UltraData-Code),L0–L3 分级代码治理,推动代码能力显著跃升;[UltraData-SFT-Agent-2609](https://huggingface.co/datasets/openbmb/UltraData-SFT-Agent-2609),50 万 Agent 训练样本,赋能端侧 Agent 综合能力提升;[UltraData-RL-2609](https://huggingface.co/datasets/openbmb/UltraData-RL-2609),超 8 万条高质量 RL 训练样本,覆盖数学、代码、通用知识与长文本推理。
|
| 149 |
|
|
|
|
| 150 |
## 模型列表
|
| 151 |
|
| 152 |
你可以按运行环境选择对应模型格式:
|
|
|
|
| 182 |
- **注意力头(GQA)**:16 个 Q heads / 2 个 KV heads
|
| 183 |
- **上下文长度**:131,072
|
| 184 |
|
|
|
|
|
|
|
| 185 |
## 简介
|
| 186 |
|
| 187 |
MiniCPM5-2B 是 MiniCPM5 系列的第二个模型,面向本地助手、coding agent、工具调用流程以及需要紧凑模型的推理场景。它在较小部署成本下提供原生长上下文能力。
|
|
|
|
| 192 |
|
| 193 |
在这组对比中,MiniCPM5-2B 达到同尺寸开源模型 SOTA 水平(平均分 53.9 ),也超过了参与对比的全部更大规模模型(最高 51.1)。其优势主要体现在代码推理、数学推理、长文本、工具调用与多个智能体任务上。
|
| 194 |
|
|
|
|
| 195 |
<div style="width:100%;max-width:1080px;margin:0 auto;padding:16px 0;background:#fff;
|
| 196 |
font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,'PingFang SC',
|
| 197 |
'Hiragino Sans GB','Microsoft YaHei',sans-serif;color:#171717">
|
|
|
|
| 262 |
|
| 263 |
**RL + OPD** 是 MiniCPM5-2B 后训练中的关键环节。**RL** 阶段,使用了 [JustRL II](https://app.notion.com/p/panhaoxuan/JustRL-II-Scaling-Small-LLMs-to-128K-Reasoning-with-a-Critic-3c77e972297c80adb8b5f4b05d267012#5f3eb56b29f048ebab5f71138f12e36f) 阐述的 critic-based 算法,大幅提升训练稳定性,并在多个领域取得了显著的收益。在下面列出的基准中,RL + OPD 在推理与通用能力上平均提升 **↑ 10.96 分**,Agent 能力平均提升 **↑ 6.96 分**。
|
| 264 |
|
|
|
|
| 265 |
**OPD** 阶段对 16 个 RL 训练所得到的专家模型(含 5 个 agentic 专家模型)实现了能力合并。训练方式上,我们在 response 序列的每个位置分别对学生模型和教师模型 logits 计算全词表的反向 KL 散度作为优势估计值,替代原有的 verification-based advantage;训练数据上,我们的 OPD 直接复用各 RL teacher 训练时 prompt 作为蒸馏数据,无需额外构造语料。
|
| 266 |
|
| 267 |

|
| 268 |
|
| 269 |

|
| 270 |
|
|
|
|
| 271 |
## 快速上手
|
| 272 |
|
|
|
|
|
|
|
| 273 |
### vLLM
|
| 274 |
|
| 275 |
```bash
|
|
|
|
| 288 |
}'
|
| 289 |
```
|
| 290 |
|
|
|
|
|
|
|
| 291 |
### SGLang
|
| 292 |
|
| 293 |
```bash
|
|
|
|
| 318 |
--port 30000
|
| 319 |
```
|
| 320 |
|
|
|
|
|
|
|
| 321 |
### Transformers
|
| 322 |
|
| 323 |
```bash
|
|
|
|
| 357 |
--tool-call-parser minicpm5 # 或:--tool-call-parser auto
|
| 358 |
```
|
| 359 |
|
|
|
|
|
|
|
| 360 |
## GitHub Cookbooks 与 Agent Skills
|
| 361 |
|
| 362 |
MiniCPM5-2B 使用**标准** `LlamaForCausalLM` **架构**,主流推理引擎可直接加载,**无需自定义算子,也无模型代码 fork**。逐步部署和微调说明请参考下方 GitHub cookbooks;Agent Skills 作为 GitHub 资源提供给使用 Cursor / Claude Code 类 coding agent 的用户。
|
| 363 |
|
| 364 |
### 部署
|
| 365 |
|
|
|
|
| 366 |
| 后端 | 模型格式 / 适用场景 | Cookbook | Agent Skill |
|
| 367 |
| ------------ | ------------------------------------------- | ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
|
| 368 |
| Transformers | BF16 / FP16,本地 Python 推理,GPU + CPU | [transformers.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/transformers.md) | [minicpm5-deploy-transformers](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-transformers/SKILL.md) |
|
|
|
|
| 375 |
| ArcLight | GGUF 本地端侧 / CPU / 桌面 / 服务器 | [arclight.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/arclight.md) | [minicpm5-deploy-arclight](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-arclight/SKILL.md) |
|
| 376 |
| vLLM Ascend | BF16 / FP16 OpenAI server | [vllm_ascend.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/vllm_ascend.md) | [minicpm5-deploy-vllm-ascend](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-vllm-ascend/SKILL.md) |
|
| 377 |
|
|
|
|
|
|
|
|
|
|
| 378 |
### 微调
|
| 379 |
|
|
|
|
| 380 |
| 框架 | 适用场景 | Cookbook | Agent Skill |
|
| 381 |
| ------------- | ------------- | --------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
|
| 382 |
| TRL + PEFT | LoRA / SFT 微调 | [trl.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/finetune/trl.md) | [minicpm5-finetune-trl](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-finetune-trl/SKILL.md) |
|
|
|
|
| 384 |
| ms-swift | 微调 | [ms_swift.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/finetune/ms_swift.md) | [minicpm5-finetune-ms-swift](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-finetune-ms-swift/SKILL.md) |
|
| 385 |
| unsloth | 微调 | [unsloth.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/finetune/unsloth.md) | [minicpm5-finetune-unsloth](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-finetune-unsloth/SKILL.md) |
|
| 386 |
|
|
|
|
|
|
|
|
|
|
| 387 |
### 其他支持的框架
|
| 388 |
|
| 389 |
除上文列出的部署与微调框架外,MiniCPM5-2B 也支持通过 FlagOS 进行多芯片部署。
|
|
|
|
| 396 |
|
| 397 |
官网速递:[https://flagos.io](https://flagos.io/)
|
| 398 |
|
|
|
|
| 399 |
<details>
|
| 400 |
<summary>FlagOS 多 AI 芯片支持与使用方式</summary>
|
| 401 |
|
|
|
|
| 403 |
|
| 404 |
基于 FlagOS 极短时间内适配 MiniCPM5-2B 到 9 种不同的 AI 芯片,得益于众智 FlagOS 的多芯片统一 AI 系统软件栈的能力。目前,在 FlagOS 团队构建的面向多架构人工智能芯片的大模型自动迁移、适配与发布平台 FlagRelease 上,已发布 MiniCPM5-2B 的多芯片版本。细节如下:
|
| 405 |
|
|
|
|
| 406 |
| Vendor | ModelScope | Huggingface |
|
| 407 |
| --------- | ------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
|
| 408 |
| Nvidia | [MiniCPM5-2B-nvidia-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-nvidia-FlagOS) | [MiniCPM5-2B-nvidia-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-nvidia-FlagOS) |
|
|
|
|
| 415 |
| Ascend | [MiniCPM5-2B-ascend-FlagOS](https://modelscope.cn/models/FlagRelease/MiniCPM5-2B-ascend-FlagOS) | [MiniCPM5-2B-ascend-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-ascend-FlagOS) |
|
| 416 |
| ARM-v9 | [MiniCPM5-2B-Armv9-FlagOS](https://modelscope.cn/models/FlagRelease/MiniCPM5-2B-Armv9-FlagOS) | [MiniCPM5-2B-Armv9-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-Armv9-FlagOS) |
|
| 417 |
|
|
|
|
|
|
|
|
|
|
| 418 |
#### FlagOS 使用方式
|
| 419 |
|
|
|
|
|
|
|
| 420 |
##### 使用 FlagOS 在 Nvidia 体验性能加速
|
| 421 |
|
|
|
|
|
|
|
| 422 |
###### From FlagRelease(**推荐**)
|
| 423 |
|
| 424 |
FlagRelease是FlagOS团队构建的一套面向多架构人工智能芯片的大模型自动迁移、适配与发布平台,已发布MiniCPM5-2B的多芯片版本。FlagRelease 已内置相关软件包,无需用户安装。
|
| 425 |
|
| 426 |
###### FlagRelease 镜像关键版本信息
|
| 427 |
|
|
|
|
|
|
|
| 428 |
###### FlagRelease 使用速递
|
| 429 |
|
|
|
|
| 430 |
| Vendor | ModelScope | Huggingface |
|
| 431 |
| --------- | ------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
|
| 432 |
| Nvidia | [MiniCPM5-2B-nvidia-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-nvidia-FlagOS) | [MiniCPM5-2B-nvidia-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-nvidia-FlagOS) |
|
|
|
|
| 439 |
| Ascend | [MiniCPM5-2B-ascend-FlagOS](https://modelscope.cn/models/FlagRelease/MiniCPM5-2B-ascend-FlagOS) | [MiniCPM5-2B-ascend-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-ascend-FlagOS) |
|
| 440 |
| ARM-v9 | [MiniCPM5-2B-Armv9-FlagOS](https://modelscope.cn/models/FlagRelease/MiniCPM5-2B-Armv9-FlagOS) | [MiniCPM5-2B-Armv9-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-Armv9-FlagOS) |
|
| 441 |
|
|
|
|
|
|
|
|
|
|
| 442 |
###### 从零开始
|
| 443 |
|
| 444 |
- 依赖Python3.12, GLIBC_2.39, GLIBCXX_3.4.33, CXXABI_1.3.15 环境
|
| 445 |
|
|
|
|
|
|
|
| 446 |
###### Vllm 版本
|
| 447 |
|
|
|
|
|
|
|
| 448 |
###### 安装 FlagOS 算子库
|
| 449 |
|
| 450 |
官方仓库:[https://github.com/flagos-ai/FlagGems](https://github.com/flagos-ai/FlagGems)
|
|
|
|
| 454 |
pip install triton==3.5.1
|
| 455 |
```
|
| 456 |
|
|
|
|
|
|
|
| 457 |
###### 开启加速
|
| 458 |
|
| 459 |
通过在vllm执行推理的源码中增加flagGems的导入即可开启flagGems加速
|
|
|
|
| 473 |
--gpu-memory-utilization 0.85
|
| 474 |
```
|
| 475 |
|
|
|
|
|
|
|
| 476 |
##### 使用 FlagOS 统一多芯片后端插件
|
| 477 |
|
| 478 |
**[vllm-plugin-FL](https://github.com/flagos-ai/vllm-plugin-FL)** 是一个为 **vLLM** 推理/服务框架构建的插件,它基于 **FlagOS 的统一多芯片后端**开发,旨在扩展 vLLM 在多种硬件环境下的功能和性能表现。
|
| 479 |
|
| 480 |
###### vllm-plugin-FL 使用
|
| 481 |
|
|
|
|
| 482 |
| 厂商 | 从零开始 | 从 FlagRelease 开始 | |
|
| 483 |
| --- | ---------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
|
| 484 |
| 英伟达 | [vllm-plugin-FL/MiniCPM5-2B](https://github.com/flagos-ai/vllm-plugin-FL/blob/main/examples/minicpm/README.md) | [MiniCPM5-2B-ModelScope](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-nvidia-FlagOS) | [MiniCPM5-2B-nvidia-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-nvidia-FlagOS) |
|
README.md
CHANGED
|
@@ -157,7 +157,6 @@ We are releasing **MiniCPM5-2B**, the second model in the **MiniCPM5** series, f
|
|
| 157 |
|
| 158 |
📂 **Open High-Quality Data**: Alongside the model, we are releasing the high-quality training datasets behind it as part of the [UltraData](https://ultradata.openbmb.cn/) family: [UltraX](https://huggingface.co/datasets/openbmb/UltraX-Preview), a high-quality web pre-training dataset; [UltraData-Code](https://huggingface.co/datasets/openbmb/UltraData-Code), featuring L0–L3 tiered code data management to drive a significant leap in coding capabilities; [UltraData-SFT-Agent-2609](https://huggingface.co/datasets/openbmb/UltraData-SFT-Agent-2609), comprising 500K agent training samples to enhance comprehensive on-device agent capabilities; and [UltraData-RL-2609](https://huggingface.co/datasets/openbmb/UltraData-RL-2609), with 80K+ high-quality RL training samples covering mathematics, code, general knowledge, and long-context reasoning.
|
| 159 |
|
| 160 |
-
|
| 161 |
## Model List
|
| 162 |
|
| 163 |
Use this directory to choose the model format that matches your runtime:
|
|
|
|
| 157 |
|
| 158 |
📂 **Open High-Quality Data**: Alongside the model, we are releasing the high-quality training datasets behind it as part of the [UltraData](https://ultradata.openbmb.cn/) family: [UltraX](https://huggingface.co/datasets/openbmb/UltraX-Preview), a high-quality web pre-training dataset; [UltraData-Code](https://huggingface.co/datasets/openbmb/UltraData-Code), featuring L0–L3 tiered code data management to drive a significant leap in coding capabilities; [UltraData-SFT-Agent-2609](https://huggingface.co/datasets/openbmb/UltraData-SFT-Agent-2609), comprising 500K agent training samples to enhance comprehensive on-device agent capabilities; and [UltraData-RL-2609](https://huggingface.co/datasets/openbmb/UltraData-RL-2609), with 80K+ high-quality RL training samples covering mathematics, code, general knowledge, and long-context reasoning.
|
| 159 |
|
|
|
|
| 160 |
## Model List
|
| 161 |
|
| 162 |
Use this directory to choose the model format that matches your runtime:
|