ThinkingCap-Qwen3.8-27B-MLX-4bit-DWQ

MLX build for Apple Silicon, in a mixed 4/8-bit affine layout (group 64) whose quantization parameters are fitted to the bf16 model's own outputs by distillation (DWQ) rather than set by rounding alone. 22.5 GB on disk (21.0 GiB), against ≈56 GB for bf16 — it runs on a 32 GB Mac.

Built from bottlecapai/ThinkingCap-Qwen3.8-27B (bf16). The MLP projections of all but the last 8 layers are 4-bit; self-attention, the wide Gated-DeltaNet projections, the last 8 layers' MLPs, lm_head and the embeddings are 8-bit. The vision tower, the MTP head's fc, the narrow Gated-DeltaNet input gates and all norms stay bf16. Vision input and MTP self-speculative decoding both work from this repo — the drafter is in mtp-drafter/ and is also embedded in the main shards.

Decode on Apple Silicon is bound by memory bandwidth, so bits per weight set the speed; the 8-bit share is placed where a flat 4-bit layout costs the most accuracy.

Serving

Use mlx-vlm or oMLX. mlx-lm drops the MTP head and the vision tower when it loads a checkpoint, so it will serve this one as text-only without speculative decoding.

pip install mlx-vlm

python -m mlx_vlm generate \
  --model bottlecapai/ThinkingCap-Qwen3.8-27B-MLX-4bit-DWQ \
  --prompt "Explain the Cauchy–Schwarz inequality." \
  --enable-thinking --max-tokens 4096

MTP self-speculative decoding

The model's own next-token head drafts for it; there is no draft model to download.

python -m mlx_vlm generate \
  --model bottlecapai/ThinkingCap-Qwen3.8-27B-MLX-4bit-DWQ \
  --draft-model bottlecapai/ThinkingCap-Qwen3.8-27B-MLX-4bit-DWQ/mtp-drafter \
  --draft-kind mtp --draft-block-size 3 \
  --prompt "..." --enable-thinking --max-tokens 4096

Acceptance is 2.41 tokens per round, against 2.43 for the bf16 weights on the same prompts.

Faster prefill on M5

M5 has INT8 matrix units, so you can turn on oMLX's INT8-activation prefill with these weights — no separate build needed, the 4-bit projections are already in the affine group-64 format those kernels take. Turn it on in oMLX under Model Settings → Experimental Features; it can't be combined with ANE prefill. oMLX measured 34% faster prefill at 32K on an M5 Max, 615 → 827 tok/s, with generation slightly slower. It's off by default and can change outputs; the numbers below were measured with it off.

Sampling

Thinking mode, as the base model: temperature 1.0, top_p 0.95, top_k 20, min_p 0.0.

Expected performance

Accuracy and completion length against the bf16 weights, on the same questions at the same sampling seeds — GPQA-Diamond 198, MMLU-Pro 1,500, RealWorldQA 765 (images), IFBench 300, AA-LCR 100 (long-context prompts) — thinking at the chat template's default reasoning effort (xhigh), with sampled decoding (temperature 1.0, top_p 0.95, top_k 20, min_p 0.0). The MMLU-Pro set is a uniform sample of the 12,032-question test split; AA-LCR answers are graded by Gemma-4-26B-A4B-it with thinking off, the same judge the comparison's bf16 row used. median tokens / mean tokens = completion length (reasoning plus answer) over the questions.

Every benchmark lands within about two points of bf16, in both directions, and completion length is unchanged. Compared question by question, wins and losses are evenly split on all five. Evaluated, not proven lossless.

GPQA-Diamond (graduate-level science) — 198 questions × 4 seeds

config	acc	median tokens	mean tokens
ThinkingCap-Qwen3.8-27B bf16	0.880	1033	7115
MLX-4bit-DWQ	0.862	1142	6916

MMLU-Pro (knowledge) — 1,500 questions × 1 seed

config	acc	median tokens	mean tokens
ThinkingCap-Qwen3.8-27B bf16	0.841	166	1436
MLX-4bit-DWQ	0.849	167	1546

RealWorldQA (vision) — 765 questions × 2 seeds

config	acc	median tokens	mean tokens
ThinkingCap-Qwen3.8-27B bf16	0.831	112	488
MLX-4bit-DWQ	0.818	114	484

IFBench (instruction following) — 300 questions × 2 seeds

config	acc	median tokens	mean tokens
ThinkingCap-Qwen3.8-27B bf16	0.797	1819	4531
MLX-4bit-DWQ	0.785	1911	4529

AA-LCR (long-context reasoning) — 100 questions × 1 seed

config	acc	median tokens	mean tokens
ThinkingCap-Qwen3.8-27B bf16	0.810	844	1718
MLX-4bit-DWQ	0.830	899	1843

Where to find us

Need even more efficiency? The open release is production-ready. Our enterprise versions go further — fewer thinking tokens still, tuned to your workload, at matched accuracy on your own tasks. Built for AI labs, inference providers and enterprises running models at scale. Deployed on your infrastructure, or in the cloud and region you choose. Talk to our team

License

ThinkingCap: PolyForm Small Business 1.0.0 + BottleCap personal-use grant (see LICENSE).

Upstream Qwen materials: Apache-2.0 (see NOTICE).

Commercial license: contact BottleCap AI.

Citation

If you use this model, please cite:

@misc{ThinkingCap-Qwen3.8-27B,
  title     = {bottlecapai/ThinkingCap-Qwen3.8-27B},
  author    = {Osusky, Adam and Lindauer, Jan and Jirkovsky, Adam and Mihal, Filip and Platek, Ondrej and Herel, David and Ihnatchenko, Luka and Bartek, Vojtech and Jirak, Jiri and Kubista, Daniel and Krus, Frantisek and Mikolov, Tomas},
  year      = {2026},
}