---
license: agpl-3.0
language:
- en
library_name: mlx
pipeline_tag: image-text-to-text
base_model:
- lordx64/Qwable-v2
base_model_relation: quantized
datasets:
- lordx64/fable-sft-combined-v2
- lordx64/agentic-distill-fable-5-sft
- lordx64/fable-tool-use-sft
- lordx64/reasoning-distill-opus-4-7-max-sft
tags:
- mlx
- oq
- imatrix
- dwq
- quantization
- 4-bit
- mtp
- speculative-decoding
- agentic
- tool-use
- reasoning
- chain-of-thought
- qwable-v2
- qwen3.6
- mixture-of-experts
- moe
- image-text-to-text
- vision
---

# Qwable-v2-oQ4e-DWQ-MTP-Vision-MLX

A **4-bit MLX quantization** of [`lordx64/Qwable-v2`](https://huggingface.co/lordx64/Qwable-v2) —
**vision + text**, with an embedded **MTP** head for speculative decoding. Built with a
two-stage recipe: **imatrix mixed-precision (`oQ4e`) → Distilled Weight Quantization (`DWQ`)**,
the strongest 4-bit path we could measure on this model.

> ⚠️ These are quantized weights. **All capability comes from the base model**
> [`lordx64/Qwable-v2`](https://huggingface.co/lordx64/Qwable-v2) — please star/cite it first.
> This repo's contribution is the quantization + MTP/vision packaging and its
> fidelity/speed validation.

## Model lineage

```
Qwen/Qwen3.6-35B-A3B                          (base, 35B MoE · 256 experts · ~3B active · 262k ctx)
  └─ lordx64/…-Claude-4.7-Opus-Reasoning-Distilled   (SFT distill of Claude Opus 4.7 reasoning)
       └─ lordx64/Qwable-v2                            (+ Claude Fable-5 agentic / tool-use LoRA)
            └─ THIS REPO: oQ4e (imatrix) → DWQ 4-bit MLX + MTP + vision
```

Qwable-v2 is an **open-weights agentic coding model**: it **thinks** in explicit
`<think>…</think>` chains-of-thought (from the Opus-4.7 prior) and **acts** like a
Claude-Code-style agent — emitting `<tool_use>` XML with real Claude Code tool names
(`Read`, `Edit`, `Bash`) and correct field signatures. The agentic XML is
**system-prompt-conditional** (reliable with an agent-style system prompt or after a
`<tool_result>` turn; bare prompts fall back to prose).

- **Architecture:** Qwen3.6-35B-A3B — Mixture-of-Experts, 256 experts (8 routed + 1 shared),
  ~3B active params/token, up to **262,144-token context**.
- **This repo:** imatrix 4-bit quant, DWQ-distilled, with MTP and the vision tower preserved.

## Quantization: `oQ4e` (imatrix) → `DWQ`, 4-bit

A **two-stage** learned quant, not a plain round-to-nearest 4-bit:

1. **`oQ4e` — imatrix mixed-precision.** oMLX's *enhanced* quantizer builds an importance
   matrix from ~1k calibration activations (sized for MoE expert coverage) and allocates bits
   per-tensor by sensitivity. Base **4-bit affine (group size 64)**, with sensitive tensors
   promoted: **201 → 8-bit, 105 → 5-bit, 7 → 6-bit** (the rest stay 4-bit).
2. **`DWQ` — distillation.** The `oQ4e` backbone's sub-8-bit affine scales/biases are then
   gradient-distilled toward an **8-bit teacher** of the same model (KL on top-1024 logits,
   temperature 2.0, lr 1e-6, seq 512), recovering fidelity the quant grid loses.

| | |
|---|---|
| Scheme | affine 4-bit base (g64) · imatrix-promoted 5/6/8-bit on sensitive tensors · then DWQ-distilled |
| Effective size | **~4.5 bits/weight** → ~21.6 GB on disk (incl. bf16 vision tower + MTP head) |
| Distillation teacher | 8-bit MLX quant of `lordx64/Qwable-v2` |
| Calibration | **3,965** ≤512-token windows — ~52 % Fable-5 agentic/tool-use + ~48 % Opus-4.7 reasoning (deep tails covered, not just heads) |
| Tooling | oMLX `oq` (`enhanced`) + `mlx_lm.quant.dwq` |

This is a **true `imatrix → DWQ`** build — distillation applied *on top of* a
sensitivity-mixed backbone, rather than on a flat uniform-4-bit student. On this model it is
the best-fidelity 4-bit we measured (see below).

## Evaluation

### Quant fidelity — validation KL-to-8-bit-teacher (lower = better)

Measured identically across recipes on the held-out calibration split (same seed / temperature
/ teacher targets), so the numbers are directly comparable:

| Recipe | KL-to-teacher | vs plain oQ4 |
|---|---:|---:|
| `oQ4` (sensitivity-mixed, no distill) | 0.0317 | — |
| `oQ4` + DWQ | 0.0289 | −9 % |
| `oQ4e` (imatrix, no distill) | 0.0234 | −26 % |
| **`oQ4e` + DWQ — this repo** | **0.0223** | **−30 %** |

imatrix alone (`oQ4e`) already beats `oQ4`+DWQ; adding DWQ on top squeezes out the rest.

### Capability benchmarks — measured on **this quant**

Run by the uploader on this 4-bit build (served via **oMLX**); reported to show that
quantization **preserves** base capability. Confirm harness/shot settings against your own
eval before citing as official base-model numbers.

| Benchmark | This 4-bit quant |
|---|---:|
| MMLU | **90.0 %** |
| MMLU-Pro | **82.0 %** |
| HumanEval (pass@1) | **88.4 %** |

For the base model's own agentic evals (SWE-bench Lite, etc.), see the
[Qwable-v2 card](https://huggingface.co/lordx64/Qwable-v2) — several suites are still in
progress there.

### Speed (M4 Pro, 48 GB, oMLX 0.5.0, MTP on)

| | Engine | Decode speed | MTP accept |
|---|---|---:|---:|
| **This repo (vision + MTP)** | VLM + **Lightning MTP** (depth-k kernels) | **~69–71 tok/s** | ~84 % (2.2 tok/cycle) |
| Text-only sibling | LLM (batched) | ~58–62 tok/s | ~64 % (1.7 tok/cycle) |

> **Why the vision build is faster:** oMLX routes a model *with* a vision tower to its
> **VLM engine**, which runs this `qwen3_5_moe` architecture natively with **Lightning MTP**
> (custom depth-k speculative-decoding kernels). The text-only sibling routes to the LLM
> engine, which uses a fallback decode path for this VLM-native architecture. **The backbone
> and MTP weights are byte-identical between the two builds** — only decode *speed* differs,
> not accuracy. For text tasks this build serves the vision tower unused, so it's a strict
> speed win.

## Sibling repos (same base, pick your trade-off)

| Repo | Quant | Vision | Size | Best for |
|---|---|:--:|---:|---|
| **`Qwable-v2-oQ4e-DWQ-MTP-Vision-MLX`** (this) | imatrix→DWQ 4-bit | ✓ | ~21.6 GB | fastest decode, multimodal, benchmarking |
| `Qwable-v2-oQ4e-DWQ-MTP-MLX` | imatrix→DWQ 4-bit | – | ~20.7 GB | text-only deployments, no vision tower |
| [`lordx64/Qwable-v2`](https://huggingface.co/lordx64/Qwable-v2) | bf16 base | ✓ | ~67 GB | reference / maximum quality |

Both quant builds share the **same distilled backbone + MTP head** (byte-identical) — identical
task quality; they differ only by the vision tower and thus the serving engine (speed).

## How to run

These are **MLX** weights (Apple Silicon). The tested serving path is
**[oMLX](https://omlx.app)** ≥ 0.5.0, which supports this model's **native MTP** speculative
decoding and vision out of the box. This repo includes the **vision tower**
(`vision_tower.*`, bf16) — it accepts images as well as text.

```bash
# 1. place the folder in your oMLX models directory
mv Qwable-v2-oQ4e-DWQ-MTP-Vision-MLX ~/.omlx/models/

# 2. enable MTP once (VLM path), then call the OpenAI-compatible API
curl -X PUT http://127.0.0.1:8003/admin/api/models/Qwable-v2-oQ4e-DWQ-MTP-Vision-MLX/settings \
  -H "Authorization: Bearer sk-local" -H "Content-Type: application/json" \
  -d '{"mtp_enabled": true, "vlm_mtp_enabled": true}'

curl -X POST http://127.0.0.1:8003/v1/chat/completions \
  -H "Authorization: Bearer sk-local" -H "Content-Type: application/json" \
  -d '{"model": "Qwable-v2-oQ4e-DWQ-MTP-Vision-MLX",
       "messages": [{"role": "user", "content": "Prove there are infinitely many primes."}],
       "max_tokens": 8000, "temperature": 0.6}'
```

The distilled backbone also loads directly in stock **`mlx-lm`** as a `qwen3_5_moe` text model.
MTP speculative decoding needs an MTP-aware runtime — **oMLX** (tested).

### Recommended sampling
`temperature 0.6, top_p 0.95, top_k 20`. For hard reasoning / long agent runs set
`max_tokens ≥ 32000` — the model thinks in explicit `<think>…</think>` blocks. For Claude-Code-style
tool use, provide an agent system prompt so it emits `<tool_use>` XML.

## Intended use & limitations

- **Built for agentic coding + hard reasoning:** tool-use loops, SWE-style edits, competition
  math, STEM, multi-step logic.
- **Reasoning/agency ≠ knowledge.** Quantization (and the base distillation) transfer *how to
  reason and act*, not new facts.
- **Quantization loss:** 4-bit is lossy vs bf16; the fidelity ladder above quantifies it
  (small). For maximum quality use the bf16 base or an 8-bit quant.
- **Distillation provenance:** the base's training traces were generated with Anthropic's
  Claude Opus 4.7 / Fable-5. Downstream users should confirm compliance with
  [Anthropic's usage policy](https://www.anthropic.com/legal/usage-policy).

## Datasets

Inherited from the base model (used for its training and for this quant's calibration — no new
knowledge is introduced; calibration only aligns the 4-bit scales to the model's own outputs):

- [`lordx64/fable-sft-combined-v2`](https://huggingface.co/datasets/lordx64/fable-sft-combined-v2) — Fable-5 agentic / tool-use SFT (combined)
- [`lordx64/agentic-distill-fable-5-sft`](https://huggingface.co/datasets/lordx64/agentic-distill-fable-5-sft)
- [`lordx64/fable-tool-use-sft`](https://huggingface.co/datasets/lordx64/fable-tool-use-sft)
- [`lordx64/reasoning-distill-opus-4-7-max-sft`](https://huggingface.co/datasets/lordx64/reasoning-distill-opus-4-7-max-sft) — Opus-4.7 reasoning traces

## Acknowledgements

- **[lordx64](https://huggingface.co/lordx64)** — the `Qwable-v2` base model this repo quantizes. All capability is theirs.
- **Qwen team** — Qwen3.6-35B-A3B.
- **Anthropic** — Claude Opus 4.7 and Fable-5, the reasoning/agentic teachers for the base.
- **Apple MLX** — `mlx`, `mlx-lm` (DWQ · `mlx_lm.quant.dwq`), `mlx-vlm`.
- **oMLX** — the `oq`/`oQe` imatrix quantizer, Lightning-MTP + vision serving runtime, and OpenAI-compatible API.

## License

**AGPL-3.0**, inherited from `lordx64/Qwable-v2` (the base's Fable-5 datasets are AGPL-3.0).
If you serve this model over a network, the AGPL's network-use clause applies — make your
corresponding source available accordingly.

## Citation

```bibtex
@misc{qwable_v2_2026, title={Qwable-v2}, author={lordx64}, year={2026},
  howpublished={\url{https://huggingface.co/lordx64/Qwable-v2}} }
@misc{qwen36_a3b_2026, title={Qwen3.6-35B-A3B}, author={Qwen Team}, year={2026},
  howpublished={\url{https://huggingface.co/Qwen/Qwen3.6-35B-A3B}} }
```