Brooooooklyn/Qwen3.6-35B-A3B-nvfp4-mlx

Qwen3.6 35B-A3B — Unsloth NVFP4/E4M3 recipe for MLX

This is a weight-only MLX conversion of Qwen/Qwen3.6-35B-A3B using the tensor-class policy from unsloth/Qwen3.6-35B-A3B-NVFP4:

Tensor class	Count	Storage
Routed/shared expert FFNs in layers 0–31	192	NVFP4, 4-bit, group 16
Final-eight FFNs, attention, Gated DeltaNet qkv/z/out, and lm_head	179	E4M3 FP8 plus one BF16 scale per output
Vision, MTP, routers, GDN a/b, embeddings, norms, and state tensors	—	BF16

Update — re-converted 2026-09-01

Expert FFN tensors (switch_mlp, shared_expert) are NOT lifted. The norm they read also drives the softmax router and the sigmoid shared-expert gate, and neither is scale-invariant, so folding a scalar into it would move routing. Those blocks therefore carry the same annihilation as the previous revision -- measured 0.9505% of expert blocks, worst layers.0.mlp.switch_mlp.gate_proj at 41.44%. This is a known open limitation, not a regression.

The imatrix / AWQ pre-scaling is removed. Under a float-scaled format it drove block scales to the zero code and annihilated whole blocks, so mlx-node now refuses --imatrix-path for these class maps. The reproduce command above is updated accordingly; the previous one fails on current mlx-node.

Weight bytes changed; the tensor-class map, shard layout and tensor counts are unchanged. Earlier revisions remain in this repository's history.

Provenance

• Base weights: Qwen/Qwen3.6-35B-A3B revision 995ad96eacd98c81ed38be0c5b274b04031597b0
• Tensor-class reference: unsloth/Qwen3.6-35B-A3B-NVFP4 revision 739af1e7aac320af1682ed1e0cce369af4c5265d
• Converter: current mlx-node source

No imatrix or calibration dataset is used. The FP4 tensor-class maps are data-free, and AWQ pre-scaling is refused under a float-scaled format: dividing a weight column by an AWQ scale drives the block amax down until amax / 6 falls under E4M3's smallest normal value and the shared block scale rounds to the zero code, which decodes the whole block to zero.

Runtime semantics

This artifact preserves the low/high weight classes, but activations remain 16-bit. NVFP4 uses MLX weight-only quantized matmul. Raw E4M3 weights are reconstructed to BF16 once when the model loads, then use ordinary A16 matmul/gather-mm. This is not upstream W4A4/W8A8 execution and does not claim global-scale, FP8 KV-cache, numerical, or performance parity.

The tokenizer, processor, vision encoder, and inline MTP weights are retained; only text generation was smoke-tested for this upload.

Reproduce

yarn mlx convert \
  --input /path/to/qwen3.6-35b-a3b \
  --output ./Qwen3.6-35B-A3B-nvfp4-mlx \
  --model-type qwen3_5_moe \
  --dtype bfloat16 \
  --quantize \
  --q-recipe unsloth \
  --q-mode nvfp4

Validation

• 35,951,822,704 source parameters converted.
• 1,457 serialized tensors across six SafeTensors shards.
• Exact index/shard closure.
• Exact class map: 192 NVFP4 4/16 modules and 179 E4M3 FP8 overrides.
• E4M3 storage verified as raw U8 weights plus BF16 per-output scales.
• Quantization metadata is identical in quantization and quantization_config.
• Text smoke test: loaded in 43.9 seconds and generated one deterministic token, OK.

This model belongs to the Unsloth NVFP4 Tensor-Class Recipe for MLX macOS/DGX collection.