Runtime: Osaurus 0.25.21 or newer (osaurus.json: required_osaurus_version).

OsaurusAI/MiniCPM-V-4.7-35B-A3B-JANG_8M

A JANG bundle of openbmb/MiniCPM-V-4.7-35B-A3B: a 35B mixture-of-experts model with 3B active parameters (40 layers, GatedDeltaNet linear attention and full attention 3:1, 256 routed experts with 8 active plus a shared expert), with image, multi-image and video understanding, thinking on by default, and Qwen-style XML tool calls. 65.7 GiB of bf16 weights in 35.3 GiB.

What is quantized how

part	format
routed experts (256 × 40 layers)	affine 8-bit, group 64, GPTQ per expert on that expert's own input statistics
shared expert, attention, GatedDeltaNet projections	affine 8-bit, GPTQ
embeddings, lm_head	affine 8-bit
routers	bf16
vision tower + projector	bf16
norms, conv, A_log, dt_bias	source values

Every quantized matrix with an input Hessian was rounded with GPTQ. The Hessians come from the model's own outputs: the bf16 model wrote reasoning traces (thinking on and off), tool calls, code, long-document answers and multi-turn agent turns across 11 prompt families at its native sampling settings, and those sequences were the calibration data. Affine quantization is MLX's own grid (bf16 scales and biases), so the bundle loads at full speed on the standard kernels.

Fidelity vs the bf16 model

150,892 teacher-forced positions on 164 held-out sequences the bf16 model generated itself (never seen in calibration), scored against the bf16 model: top-128 KL, top-1 agreement, and the change in log-likelihood of the real next token (dNLL).

	size	KL median	KL mean	top-1	dNLL
35B-A3B-JANG_8M	35.3 GiB	6.7e-08	0.0022	99.07%	+0.0002

By prompt family (KL median, top-1):

family	KL median	top-1
agentic / tool use	0.0000	99.5%
coding	0.0000	99.2%
creative writing	0.0008	97.2%
cybersecurity	0.0000	98.4%
long documents	0.0000	99.0%
math	0.0000	99.7%
networking	0.0000	99.4%
office tasks	0.0000	99.4%
science	0.0000	99.5%
terminal agent	0.0000	99.0%
turn after a tool result	0.0000	99.3%

Decisions: the quantized model picks the same token as bf16 at 80 / 80 </think> closes and at 23 / 25 tool-call starts (bf16 vs itself: 23 / 25).

Speculative decoding

dspark/ holds openbmb's DSpark draft model for this size (openbmb/MiniCPM-V-4.7-35B-A3B-DSpark), quantized to affine 8-bit (its Markov and confidence heads stay bf16). It is stamped on by default (jang_config.json → speculative). Draft acceptance has not been measured on the 35B; on the 1B drafter the 8-bit quantization did not change acceptance (2.74 vs 2.74 tokens per round, greedy).

Model notes

• Modalities: text, images (multi-image, high-resolution slicing), video. No audio. downsample_mode 16x (default) or 4x selects the visual token budget.
• Reasoning: on by default; the chat template opens <think>. enable_thinking: false turns it off. There are no reasoning-effort levels.
• Tools: Qwen3-Coder-style XML (<tool_call><function=…><parameter=…>).
• Sampling (generation_config.json, mirrored in jang_config.json): temperature 1.0, top-p 0.95, top-k 20, EOS <|im_end|> / <|endoftext|>.
• The source config lists one MTP layer, but the checkpoint contains no MTP weights; none is included.

Source

• openbmb/MiniCPM-V-4.7-35B-A3B revision 4c7623a9326e, Apache-2.0.
• Quantization and bundle by Jinho Jang (eric@osaurus.ai), 2026-10-12.