Qwen3.6-35B-A3B Selective NVFP4 MoE-MLP GGUF

An accuracy-oriented GGUF conversion of Qwen/Qwen3.6-35B-A3B for NVIDIA Blackwell GPUs. Only the routed and shared-expert MLP weights use NVFP4 W4A16; attention, embeddings, the output head, vision components, and embedded MTP remain at BF16/F32 precision.

Files

File	Purpose	Size
`Qwen3.6-35B-A3B-NVFP4-W4A16-MoE-MLP-BF16attn-BF16head-MTP.gguf`	Main model with embedded MTP	24.58 GB
`mmproj-Qwen3.6-35B-A3B-BF16.gguf`	BF16 vision projector	0.90 GB
`Qwen3.6-35B-A3B-DFlash-BF16.gguf`	Optional original DFlash speculative drafter	0.78 GB

The main GGUF works by itself for text. Add mmproj for image input. The DFlash GGUF is not a standalone model and is used instead of embedded MTP when DFlash speculation is enabled.

Precision layout

Component	Precision
Routed/shared-expert gate, up, and down weights	NVFP4 W4A16
Full and linear attention	BF16/F32
Embeddings, routers, and `lm_head`	BF16/F32
Vision and MTP/NextN	BF16/F32

The main GGUF contains 240 NVFP4 tensors and retains all other tensors at source precision.

Usage

A recent CUDA build of llama.cpp with Qwen3.6 NVFP4 support is required. DFlash additionally requires a compatible build such as DFlash 2 PR #27342.

Embedded MTP:

llama-server \
  -m Qwen3.6-35B-A3B-NVFP4-W4A16-MoE-MLP-BF16attn-BF16head-MTP.gguf \
  --mmproj mmproj-Qwen3.6-35B-A3B-BF16.gguf \
  --ctx-size 32768 --flash-attn on --n-gpu-layers -1 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --spec-type draft-mtp --spec-draft-n-max 2

Original DFlash:

llama-server \
  -m Qwen3.6-35B-A3B-NVFP4-W4A16-MoE-MLP-BF16attn-BF16head-MTP.gguf \
  -md Qwen3.6-35B-A3B-DFlash-BF16.gguf \
  --mmproj mmproj-Qwen3.6-35B-A3B-BF16.gguf \
  --ctx-size 32768 --flash-attn on --n-gpu-layers -1 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --spec-type draft-dflash --spec-draft-n-max 7 --spec-draft-ngl all \
  --cache-type-k-draft q8_0 --cache-type-v-draft q8_0

Remove --mmproj for text-only use.

Validation

Structural audits passed: 240 intended MoE tensors in NVFP4; attention, lm_head, vision, and MTP retained at BF16/F32.
The target and projector successfully allocated a 262,144-token context and completed a real image request. A near-limit prompt has not yet been tested.
DFlash completed correctness and code-generation smoke tests at 32,768 context. The longer test accepted 210 of 308 drafted tokens (68.18%) and generated at 330.61 tok/s on one RTX PRO 6000 Blackwell; this is not a controlled benchmark.