MiMo-V2.6-Flash-RL-JANG_2L

Calibrated mixed 2-bit-class JANG · approximately 2.73 effective bits/parameter in the main shards · 95.87 GiB text weights · approximately 103.19 GiB complete download.

Iteration 2 of XiaomiMiMo/MiMo-V2.6-Flash-RL, calibrated with imatrix + AWQ + per-expert GPTQ on 258,507 tokens. It protects ten early gates in source-native MXFP4 and 22 later gates at 3-bit. The native tokenizer/chat template and image, video and audio-input weights are preserved.

Requires Osaurus 0.25.13 or newer. Version 0.25.12 does not support this bundle. osaurus.json records the minimum runtime and this repository's model version (1; the weights are quantization iteration 2). The runtime is prepared for the upcoming release; do not use an older MiMo-V2.5 or generic uniform-bit loader.

Osaurus qualification: the exact weight bundle produced coherent text/tool conversations, image and video responses, and SSD-restored continuations in the integrated development build. Audio transcription failed in the current Osaurus test; audio weights are preserved, but reliable audio understanding is not qualified. The Python runtime measurements below are labeled separately and are not Osaurus performance claims.

OsaurusAI · osaurus.ai · JANG source · Source model

Measured quality

KL is KL(source || quantized) in nats, against the exact source checkpoint decoded in its original FP8/MXFP4 formats. Lower is better. Top-1 agreement measures matching next-token predictions, not task accuracy. Calibration, validation and the final sealed set are separate. Results on different sets must not be compared as if paired.

Validation metric	Previous GPTQ allocation	This JANG_2L
Mean KL	0.942919	0.914656
Median KL	0.061255	0.055000
90th percentile	2.703788	2.616918
99th percentile	13.062842	12.778649
Top-1 agreement	76.5213%	77.0381%
Positions	32,506	32,506

Final sealed metric	Previous GPTQ allocation	This JANG_2L
Mean KL	0.905232	0.866514
Median KL	0.072926	0.063002
90th percentile	2.584622	2.413376
99th percentile	12.561462	12.266034
Worst 1% mean	15.539492	15.194221
Maximum KL	26.005333	29.807011
Top-1 agreement	75.7578%	76.6198%
Positions	32,365	32,365

The final sealed set was scored once after selecting this recipe; it was not used to tune allocation. Machine-readable reports are in evaluation/. The mean remains much higher than the median, reflecting substantial distortion on sensitive positions. This is an aggressive memory-budget quant, not a claim of source benchmark parity or globally optimal quality. No matched whole-model stock-MLX RTN score was measured.

A small frozen public subset using native thinking On, temperature 1 and top-p 0.95 passed 5/6 BFCL tool tasks, versus 4/6 previously. Both allocations passed 0/3 adapted Aider/Exercism JavaScript tasks under the same bounded budgets; reasoning consumed the output budget on several attempts. A separate Off branch preserved the failed trajectory and repaired linked-list tests 20/23 → 23/23; grade-school remained 9/10 and REST 6/9. These are subset diagnostics, not official leaderboard scores or a replacement for the On result.

The source's GRPO, groupwise grading, self-correction and multi-prefix distillation describe training methods. They are not inference switches or online learning from SSD cache hits. The observed correction example does not establish that every source RL capability survived quantization.

Precision and layout

Component	Stored format
109 routed expert projections	2-bit affine, group 128
22 protected later gates	3-bit affine, group 128
Ten protected early gates	Source-native MXFP4, group 32
Attention, embedding, output head, dense and MTP matrices	8-bit affine, group 64
Routers	FP32
Vision/audio components	Native BF16 where supplied
Affine scales and biases	BF16

Calibration provenance is recorded in jang_config.json under quantization.awq, quantization.imatrix and quantization.gptq, each with applied: true. AWQ transformations are already folded into the saved weights; loaders must not apply them a second time.

Honor every per-module quantization override. The top-level 8-bit default is not the expert precision. JANG_2L describes a mixed-precision tier; effective bits include scales, biases and protected components.

The root contains 24 naturally aligned model shards, one index and 1,477 tensors. The untied head is bare lm_head.weight, measured from packed bytes as q8/g64 affine. All 72 MTP tensors stay in the main shards. Audio-tokenizer and DFlash sidecars remain in subdirectories. All tensor files, including sidecars, pass dtype-alignment checks. Main-shard tensor payload is 98.69 GiB; loaded text parameters are 95.868648 GiB. Activations, media towers and active KV need additional memory; tests used an M5 Max with 128 GiB unified memory.

Reasoning, tools and generation

• Reasoning defaults On. Disable with chat_template_kwargs={"enable_thinking": false}; the native template adds a closed empty thinking block.
• Use reasoning parser think_xml and tool parser xml_function, with automatic tool choice enabled. Preserve reasoning_content, tool-call IDs, parsed JSON arguments and complete tool-result batches in history.
• Low/high reasoning effort do not select distinct native levels. reasoning_effort="none" selects Off. Output-token caps include reasoning.
• generation_config.json uses source-recommended temperature 1.0, top-p 0.95, do_sample=true. EOS IDs: 151643, 151645, 151672. Tokenizer: Qwen2Tokenizer; embedded and standalone native templates agree.
• Tool syntax is <tool_call><function=NAME><parameter=ARG>VALUE</parameter></function></tool_call>. Partial or malformed generated arguments must not be executed as complete calls.

Python runtime measurements and multimodal limits

Python vMLX exercises image, video and audio-input preprocessing, reasoning/tool parsing, immediate tool continuations, multimodal cache reuse and process restart. Functional quality is mixed: a horizontal-motion video is answered incorrectly as downward in Off mode, including cache bypass; On answers right in the bounded control. Some combined image/audio prompts omit or misidentify requested details. These are model behavior failures, not evidence that a cache hit fixes perception. Audio output and synchronized video/audio understanding are not qualified. MTP and DFlash execution remain disabled.

The qualified SSD format stores native BF16 KV losslessly as U16 payloads with typed metadata, not FP32 expansion: nine full-attention slots plus 39 rotating sliding-window slots. Exact reconstruction/logits were verified for the native format; optional q4/q8 cache experiments were not selected. Active request KV and OS page cache still consume RAM. Reusable application cache tiers retain zero bytes under the tested SSD-only policy.

Immediate tool-result continuations wait for the server's terminal SSD persistence barrier; no client sleeps were needed. The paired candidate public run persisted all 52 responses and restored all 31 tool continuations. Stored coverage is the prompt checkpoint through N−1; later assistant/tool suffixes are prefetched on the next request.

At the stock GPU limit, a 32,036-token prompt took 54.916 s cold, 1.045 s on SSD reuse (32,035 tokens restored), and 55.846 s bypassed, with matching answers. Peak allocation was about 104.47 GiB with media towers loaded. A short visible UI correction measured about 46.7–46.8 tokens/s; neither timing is a steady-state benchmark. The tested runtime window is 32K. The source declares 1M context; 1M is not qualified for this bundle. Set prompt/window and SSD capacity controls to fit the machine.

한국어 안내

258,507개 토큰으로 imatrix/AWQ 보정과 전문가별 GPTQ를 적용한 JANG_2L 혼합 정밀도 모델입니다. 대부분의 전문가 행렬은 2비트이며, 초기 게이트 10개는 원본 MXFP4, 이후 게이트 22개는 3비트로 보호합니다. 텍스트 가중치는 약 95.87 GiB입니다.

검증 세트 평균 KL은 0.914656 nats, 다음 토큰 일치율은 **77.0381%**입니다. 별도 최종 평가의 평균 KL은 0.866514 nats, 일치율은 **76.6198%**입니다. 원본 성능과 동일하다는 의미는 아닙니다. 추론은 기본 활성화되며 enable_thinking=false로 끌 수 있습니다. 추론 파서는 think_xml, 도구 파서는 xml_function입니다.

새 MiMo-V2.6 Python 런타임이 필요합니다. 일부 영상 방향 판단과 복합 멀티모달 지시는 실패하며, 1M 문맥·오디오 출력·MTP/DFlash 실행은 검증되지 않았습니다. Osaurus는 0.25.13 이상이 필요하며 0.25.12는 지원하지 않습니다. Osaurus 오디오 전사 테스트는 실패하여 오디오 품질을 보장하지 않습니다. 기본 SSD 캐시는 네이티브 BF16 데이터를 손실 없이 저장합니다.

License

The source checkpoint declares MIT. Source revision: 5711b268169967567844e1e560e8a3966da959b1. Quantization does not change the source license.