Qwen3.8-84B-A53B-Niwaki-v2-experimental-mlx

⚠️ EXPERIMENTAL RESEARCH ARTIFACT — NOT FOR REAL USE. This model is a measurement instrument, not a usable assistant. It is Qwen3.8-2.4T-A95B with 98% of its routed experts deleted (2.4T → 84B total / ~53B active parameters, 63 GB on disk) and a short distillation heal. It generates fluent, well-structured text that is frequently and confidently wrong — knowledge that lived in the deleted experts is gone, and no sampling setting brings it back. Published so the curious can poke at what survives extreme MoE pruning at frontier scale.

Niwaki (庭木) are Japan's garden trees, sculpted by meticulous pruning so that every branch serves the form of the whole. This model applies that spirit to a Mixture-of-Experts — here pruned far past the point of use, to see where the form gives way.

A paper with the full method and measurements is coming soon.

What this is

The 2.4T reference routes each token through 10 of 512 experts per layer across 92 layers. This artifact keeps: the full attention/backbone (8-bit), every shared expert (healed, 8-bit), the original routers, and compact 2-bit banks holding only the top-96 most-routed experts in the first 6 layers and top-40 in the last 3 — 664 of 47,104 routed experts (1.4%). Routing renormalizes exactly over the kept experts. A ~0.5M-token knowledge-distillation heal (teacher logits from the reference model) recovered about half of the deletion damage in log-perplexity before plateauing.

Benchmarks

Screen protocol: WikiText-2 and C4 perplexity over 64 × 1024-token windows each (fp32 loss, no chat template) — smaller than the full protocol of the main Niwaki releases; every row below uses the identical windows.

model	disk	wt2 ppl ↓	c4 ppl ↓
Qwen3.8-2.4T-A95B (teacher, API anchor)	~4,890 GB (bf16)	2.77*	—
shared-experts-only strip (no routed experts)	52.1 GB	46.21	76.90
+ protected expert banks (unhealed)	63.0 GB	39.87	64.04
this model (protected + healed, folded)	63.0 GB	23.13	29.95

* teacher anchor measured via API echo-scoring on the same text, its own tokenization; indicative, not window-identical. Task-suite evaluation was not run for this artifact.

What to expect

Form survives; knowledge does not. The model plans, formats code blocks, self-corrects, and holds discourse structure — then confabulates the substance (asked for Tetris, it confidently invents a different game). Ranking-style capabilities recover much faster than open generation. Expect coherent wrongness, repetition under greedy decoding, and no factual reliability whatsoever.

Run it

Loads with the same 30-line shim as the other Niwaki releases (niwaki_v2_load.py, shipped in this repo); everything else is stock mlx_lm. Needs ~70 GB free unified memory; ~30 tok/s on an M4 Max.

from niwaki_v2_load import load
import mlx_lm
from mlx_lm.sample_utils import make_sampler, make_logits_processors

model, tok = load("neopolita/Qwen3.8-84B-A53B-Niwaki-v2-experimental-mlx")
out = mlx_lm.generate(
    model, tok,
    prompt=tok.apply_chat_template([{"role": "user", "content": "Hello!"}],
                                   tokenize=False, add_generation_prompt=True),
    max_tokens=256,
    sampler=make_sampler(temp=1.0, top_p=0.95, top_k=20),
    logits_processors=make_logits_processors(repetition_penalty=1.15,
                                             repetition_context_size=256),
)

Shipped sampling defaults (temp 1.0, top-k 20, top-p 0.95) follow the base model; the repetition penalty is recommended — the healed model loops without it.

Model dimensions

92 layers (hybrid linear-attention + gated attention), hidden 8192, 512 routed experts top-10 + 1 shared expert per MoE layer in the reference; here 83 layers are shared-expert-only and 9 layers carry kept-expert banks (96×6 early, 40×3 late, 2-bit g64). Healed output maps are folded into the down-projections (zero extra parameters). Storage: 8-bit backbone + 2-bit banks, mixed precision, 63.0 GB total.

Method

Not documented here yet — a paper with the full method and measurements is coming soon.