--- base_model: bottlecapai/ThinkingCap-Qwen3.8-27B base_model_relation: quantized library_name: mlx pipeline_tag: image-text-to-text license: other license_name: polyform-small-business-1.0.0 license_link: LICENSE extra_gated_heading: "Request access to the BottleCap AI model" extra_gated_prompt: | Request access below. Add your company if you're evaluating this for work — we have enterprise versions that go further, and we'll make sure you hear about them first. We'll also send you new ThinkingCap releases and early access before they're public. Tell us how you plan to use the model and we can help you get the most out of it — there's [a short form](https://docs.google.com/forms/d/e/1FAIpQLSdU8MyVP_mVx0_y55d6QCMXVyCKsQ6yg68KEqWm_EIptKB0Nw/viewform) for that too. extra_gated_fields: Name: text "Company name (enter “N/A” if none)": text "Work email (enter your personal email if none)": text extra_gated_button_content: "Agree and request access" tags: - mlx - apple-silicon - quantized - 4-bit --- # ThinkingCap-Qwen3.8-27B-MLX-4bit-DWQ MLX build for Apple Silicon, in a mixed 4/8-bit affine layout (group 64) whose quantization parameters are fitted to the bf16 model's own outputs by distillation (DWQ) rather than set by rounding alone. **22.5 GB** on disk (21.0 GiB), against ≈56 GB for bf16 — it runs on a 32 GB Mac. Built from [bottlecapai/ThinkingCap-Qwen3.8-27B](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-27B) (bf16). The MLP projections of all but the last 8 layers are 4-bit; self-attention, the wide Gated-DeltaNet projections, the last 8 layers' MLPs, `lm_head` and the embeddings are 8-bit. The vision tower, the MTP head's `fc`, the narrow Gated-DeltaNet input gates and all norms stay bf16. Vision input and MTP self-speculative decoding both work from this repo — the drafter is in `mtp-drafter/` and is also embedded in the main shards. Decode on Apple Silicon is bound by memory bandwidth, so bits per weight set the speed; the 8-bit share is placed where a flat 4-bit layout costs the most accuracy. ## Serving Use **mlx-vlm** or **oMLX**. mlx-lm drops the MTP head and the vision tower when it loads a checkpoint, so it will serve this one as text-only without speculative decoding. ```bash pip install mlx-vlm python -m mlx_vlm generate \ --model bottlecapai/ThinkingCap-Qwen3.8-27B-MLX-4bit-DWQ \ --prompt "Explain the Cauchy–Schwarz inequality." \ --enable-thinking --max-tokens 4096 ``` ### MTP self-speculative decoding The model's own next-token head drafts for it; there is no draft model to download. ```bash python -m mlx_vlm generate \ --model bottlecapai/ThinkingCap-Qwen3.8-27B-MLX-4bit-DWQ \ --draft-model bottlecapai/ThinkingCap-Qwen3.8-27B-MLX-4bit-DWQ/mtp-drafter \ --draft-kind mtp --draft-block-size 3 \ --prompt "..." --enable-thinking --max-tokens 4096 ``` Acceptance is **2.41 tokens per round**, against 2.43 for the bf16 weights on the same prompts. ### Faster prefill on M5 M5 has INT8 matrix units, so you can turn on oMLX's INT8-activation prefill with these weights — no separate build needed, the 4-bit projections are already in the affine group-64 format those kernels take. Turn it on in oMLX under Model Settings → Experimental Features; it can't be combined with ANE prefill. oMLX measured 34% faster prefill at 32K on an M5 Max, 615 → 827 tok/s, with generation slightly slower. It's off by default and can change outputs; the numbers below were measured with it off. ### Sampling Thinking mode, as the base model: `temperature 1.0, top_p 0.95, top_k 20, min_p 0.0`. ## Expected performance Accuracy and completion length against the bf16 weights, on the same questions at the same sampling seeds — GPQA-Diamond 198, MMLU-Pro 1,500, RealWorldQA 765 (images), IFBench 300, AA-LCR 100 (long-context prompts) — thinking at the chat template's default reasoning effort (`xhigh`), with sampled decoding (temperature 1.0, top_p 0.95, top_k 20, min_p 0.0). The MMLU-Pro set is a uniform sample of the 12,032-question test split; AA-LCR answers are graded by Gemma-4-26B-A4B-it with thinking off, the same judge the comparison's bf16 row used. `median tokens` / `mean tokens` = completion length (reasoning plus answer) over the questions. Every benchmark lands within about two points of bf16, in both directions, and completion length is unchanged. Compared question by question, wins and losses are evenly split on all five. Evaluated, not proven lossless. **GPQA-Diamond (graduate-level science)** — 198 questions × 4 seeds | config | acc | median tokens | mean tokens | |---|---|---|---| | ThinkingCap-Qwen3.8-27B bf16 | 0.880 | 1033 | 7115 | | MLX-4bit-DWQ | 0.862 | 1142 | 6916 | **MMLU-Pro (knowledge)** — 1,500 questions × 1 seed | config | acc | median tokens | mean tokens | |---|---|---|---| | ThinkingCap-Qwen3.8-27B bf16 | 0.841 | 166 | 1436 | | MLX-4bit-DWQ | 0.849 | 167 | 1546 | **RealWorldQA (vision)** — 765 questions × 2 seeds | config | acc | median tokens | mean tokens | |---|---|---|---| | ThinkingCap-Qwen3.8-27B bf16 | 0.831 | 112 | 488 | | MLX-4bit-DWQ | 0.818 | 114 | 484 | **IFBench (instruction following)** — 300 questions × 2 seeds | config | acc | median tokens | mean tokens | |---|---|---|---| | ThinkingCap-Qwen3.8-27B bf16 | 0.797 | 1819 | 4531 | | MLX-4bit-DWQ | 0.785 | 1911 | 4529 | **AA-LCR (long-context reasoning)** — 100 questions × 1 seed | config | acc | median tokens | mean tokens | |---|---|---|---| | ThinkingCap-Qwen3.8-27B bf16 | 0.810 | 844 | 1718 | | MLX-4bit-DWQ | 0.830 | 899 | 1843 | ## Where to find us
Website LinkedIn Instagram X
Need even more efficiency? The open release is production-ready. Our enterprise versions go further — fewer thinking tokens still, tuned to your workload, at matched accuracy on your own tasks. Built for AI labs, inference providers and enterprises running models at scale. Deployed on your infrastructure, or in the cloud and region you choose. [Talk to our team](mailto:enterprise@bottlecapai.com) ## License ThinkingCap: PolyForm Small Business 1.0.0 + BottleCap personal-use grant (see [LICENSE](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-27B-MLX-4bit-DWQ/blob/main/LICENSE)). Upstream Qwen materials: Apache-2.0 (see [NOTICE](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-27B-MLX-4bit-DWQ/blob/main/NOTICE)). Commercial license: [contact BottleCap AI](mailto:enterprise@bottlecapai.com). ## Citation If you use this model, please cite: ```bibtex @misc{ThinkingCap-Qwen3.8-27B, title = {bottlecapai/ThinkingCap-Qwen3.8-27B}, author = {Osusky, Adam and Lindauer, Jan and Jirkovsky, Adam and Mihal, Filip and Platek, Ondrej and Herel, David and Ihnatchenko, Luka and Bartek, Vojtech and Jirak, Jiri and Kubista, Daniel and Krus, Frantisek and Mikolov, Tomas}, year = {2026}, } ```