Qwen3.6-35B-A3B-Claude-4.7-Opus-oQ4e-DWQ-MC-MTP-Vision-MLX

⭐ v2 — the best build of this model we ship

Two-stage imatrix → DWQ (reasoning, then math/code). Materially better on code than the v1 repos: HumanEval 90.2 % (v1-stage: 84.8 %) · LiveCodeBench 39.9 % (32 %) · MBPP 75.4 %. If you used any of the four …-DWQ4-… / …-oQ4-… v1 repos, use this instead.

A 4-bit MLX quantization of lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled — vision + text, with an embedded MTP head for speculative decoding.

⚠️ These are quantized weights. All capability comes from the base model — please star/cite it first. This repo's contribution is the quantization recipe + MTP/vision packaging and its fidelity/benchmark validation.

Model lineage

Qwen/Qwen3.6-35B-A3B                                  (Apache-2.0 base · 35B MoE · 256 experts · ~3B active · 262k ctx)
  └─ lordx64/…-Claude-4.7-Opus-Reasoning-Distilled     (SFT distill of Claude Opus 4.7 reasoning)
       └─ THIS REPO: oQ4e (imatrix) → DWQ¹ reasoning → DWQ² math/code  + MTP + vision

• Architecture: Qwen3.6-35B-A3B — MoE, 256 experts (8 routed + 1 shared), ~3B active params/token, up to 262,144-token context.
• Base distillation (by lordx64): SFT on ~7,800 chain-of-thought traces from Claude Opus 4.7, teaching explicit <think>…</think> reasoning.

Quantization: oQ4e (imatrix) → two-stage DWQ

Not a plain round-to-nearest 4-bit. Three steps:

1. oQ4e — imatrix mixed precision. oMLX's enhanced quantizer builds an importance matrix from calibration activations (sized for MoE expert coverage) and allocates bits by sensitivity. Base 4-bit affine, group size 64, with sensitive tensors promoted: 196 → 8-bit, 114 → 5-bit, 1 → 6-bit.
2. DWQ¹ — reasoning. The sub-8-bit affine scales/biases are gradient-distilled toward an oQ8e (imatrix 8-bit) teacher on reasoning calibration (KL over the teacher's top-1024 logits, temperature 2.0, Adam, cosine LR, validation early-stopping, finalized from the best checkpoint).
3. DWQ² — math/code. A second distillation pass over a code-heavy mix, cascading from the stage-1 model. This is what lifts HumanEval 84.8 % → 90.2 % and LiveCodeBench 32 % → 39.9 %.

Scheme	affine 4-bit base (g64) · imatrix-promoted 5/6/8-bit · two DWQ passes
Effective size	~4.5 bits/weight → 21.6 GB on disk (incl. bf16 vision tower + MTP head)
Distillation teacher	oQ8e — imatrix 8-bit quant of the same base
Calibration	leak-free, document-level splits, head/middle/tail ≤512-token windows
— stage 1 (reasoning-v2)	2,048 train / 128 valid — Opus-4.7 reasoning traces
— stage 2 (benchmark-gap-math-code-swe-v4)	2,048 train / 128 valid — verified math, competitive programming, concise-function code, resolved SWE trajectories + a small reasoning replay partition to limit forgetting
Tooling	oMLX oq (enhanced) + mlx_lm.quant.dwq

Why two stages: stage 1 recovers general fidelity to the 8-bit teacher; stage 2 re-tunes the 4-bit grid toward code/math activations where the v1 builds were weakest. The reasoning replay partition in stage 2 keeps the reasoning behaviour from drifting.

Evaluation

Measured on this quant (served via oMLX)

Benchmark	v1 (reasoning-DWQ stage)	v2 — this repo
HumanEval (pass@1)	84.8 %	90.2 %
MBPP	74.8 %	75.4 %
LiveCodeBench	32 %	39.9 %

Run by the uploader on this 4-bit build via oMLX. Reported to show that the recipe preserves and re-tunes base capability, not as official base-model results — confirm harness/shot settings against your own eval before citing.

Inherited base-model benchmarks (bf16, from the base card)

Reported by the base model via lm-evaluation-harness; a capability reference for the family, not re-measured on this quant.

Benchmark	Base (bf16)
GSM8K (8-shot CoT)	84.3 %
MMLU-Pro (5-shot)	74.9 %
MMLU-Pro · Math	83.6 %
MMLU-Pro · Physics	81.0 %
MMLU-Pro · Computer Science	79.0 %

Note on long-form math. This is a reasoning model that routinely emits 8,000+ tokens of <think> on hard problems. Fixed-budget pass/fail there is token-budget-limited, not quantization-limited. Give it a large max_tokens (32k) for hard problems.

Repos in this family

Repo	Stage	Vision	Size
…-oQ4e-DWQ-MC-MTP-Vision-MLX (this)	v2 · imatrix→DWQ×2	✓	~21.6 GB	best · multimodal
…-oQ4e-DWQ-MC-MTP-MLX	v2 · imatrix→DWQ×2	–	~20.7 GB	best · text-only
…-DWQ4-MTP-Vision-MLX, …-DWQ4-MTP-MLX, …-oQ4-MTP-Vision-MLX, …-oQ4-MTP-MLX	v1	✓/–	~19–22 GB	superseded — kept for reproducibility

Both v2 repos share the same distilled backbone + MTP head (byte-identical); they differ only by the vision tower — identical task quality.

How to run

MLX weights (Apple Silicon). Tested serving path is oMLX ≥ 0.5.0, which supports this model's native MTP speculative decoding and vision out of the box. This repo includes the vision tower (vision_tower.*, bf16) — it accepts images as well as text.

# 1. place the folder in your oMLX models directory
mv Qwen3.6-35B-A3B-Claude-4.7-Opus-oQ4e-DWQ-MC-MTP-Vision-MLX ~/.omlx/models/

# 2. enable MTP once, then call the OpenAI-compatible API
curl -X PUT http://127.0.0.1:8003/admin/api/models/Qwen3.6-35B-A3B-Claude-4.7-Opus-oQ4e-DWQ-MC-MTP-Vision-MLX/settings \
  -H "Authorization: Bearer sk-local" -H "Content-Type: application/json" \
  -d '{"mtp_enabled": true}'

curl -X POST http://127.0.0.1:8003/v1/chat/completions \
  -H "Authorization: Bearer sk-local" -H "Content-Type: application/json" \
  -d '{"model": "Qwen3.6-35B-A3B-Claude-4.7-Opus-oQ4e-DWQ-MC-MTP-Vision-MLX",
       "messages": [{"role": "user", "content": "Write a Python LRU cache with tests."}],
       "max_tokens": 8000, "temperature": 0.6}'

The backbone also loads directly in stock mlx-lm as a qwen3_5_moe text model. MTP speculative decoding needs an MTP-aware runtime — oMLX (tested). Note the oMLX model id is case-sensitive (it matches the folder name).

Recommended sampling

temperature 0.6, top_p 0.95, top_k 20, min_p 0. For hard reasoning set max_tokens ≥ 32000 — the model thinks in explicit <think>…</think> blocks. Avoid greedy (temperature 0): reasoning models degenerate there.

Intended use & limitations

• Built for hard reasoning + code: competition math, STEM, code generation, multi-step logic.
• Reasoning ≠ knowledge. Quantization and the base distillation transfer how to reason, not new facts.
• Quantization loss: 4-bit is lossy vs bf16; for maximum quality use the bf16 base or an 8-bit quant.
• Distillation provenance: the base's traces were generated with Anthropic's Claude Opus 4.7. Confirm compliance with Anthropic's usage policy.

Acknowledgements

• lordx64 — the reasoning-distilled base this repo quantizes. All capability is theirs.
• Qwen team — Qwen3.6-35B-A3B (Apache-2.0).
• Anthropic — Claude Opus 4.7, the reasoning teacher for the base distillation.
• Apple MLX — mlx, mlx-lm (mlx_lm.quant.dwq), mlx-vlm.
• oMLX — the oq/oQe imatrix quantizer, MTP + vision serving runtime.
• Calibration corpora — Open-R1 (math, verified coding), NVIDIA OpenCodeReasoning, BigCode self-oss-instruct, SWE-bench (SWE-smith trajectories).

License

Apache-2.0, inherited from the base model and Qwen3.6-35B-A3B.

Citation

@misc{qwen36_a3b_2026, title={Qwen3.6-35B-A3B}, author={Qwen Team}, year={2026},
  howpublished={\url{https://huggingface.co/Qwen/Qwen3.6-35B-A3B}} }
@misc{lordx64_qwen36_distill_2026,
  title={Qwen3.6-35B-A3B distilled from Claude Opus 4.7 reasoning}, author={lordx64}, year={2026},
  howpublished={\url{https://huggingface.co/lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled}} }