Myric/Qwen3.8-Flash-Next-APEX-GGUF

Qwen3.8-Flash-Next — GGUF

Work in progress. Only the full-precision BF16 conversion and the vision projector are uploaded so far. APEX quant tiers are not built yet.

Full-precision GGUF conversion of Qwen/Qwen3.8-Flash-Next — a 180 B-total / ~6.8 B-active multimodal MoE (48 layers, 512 routed experts, top-10 routing, hybrid gated-delta-net + full attention every 4th layer, QSA sparse attention, per-layer n-gram embeddings, 262 K context).

⚠️ Requires a patched llama.cpp

qwen4_exp is not in mainline llama.cpp. Support exists only as the unmerged PR #27742. These files will not load on stock builds — the architecture is unknown to them.

A build tracking that PR, plus one extra fix needed to convert this model at all, is at brywil/llama.cpp @ qwen4exp-pr27742-spill-fix.

This caveat is expected to expire when #27742 merges. The PR adds no new ggml ops (git diff master --stat -- ggml/ is empty), and the Qwen-side author closed their own competing PR in favour of it, so it is on track for mainline rather than being a fork.

Files

File	Size	Notes
qwen3.8-flash-next-BF16.gguf	354.0 GB	full precision, 1224 tensors
qwen3.8-flash-next-mmproj-F16.gguf	0.9 GB	vision projector, pass with --mmproj

The BF16 is a lossless conversion of the tensors it contains: the checkpoint is already bfloat16 on disk (config.json → "dtype": "bfloat16"), so the GGUF holds identical bit patterns. It is published as a quantization/imatrix input — there is no reason to run it for inference.

It is a backbone-only source, not the whole model. Two blocks are absent:

	params	where it went
vision tower	0.449 B	the mmproj file above
MTP head	2.607 B	dropped — in neither file

That is 176.944 B of the model's 180.000 B. The MTP head is a 1-layer multi-token-prediction block for speculative decoding; the current converter sets supports_mtp_export = False for this architecture and discards its 31 tensors silently — no warning appears in the conversion log. --mtp is refused outright (--mtp / --no-nextn are not supported for Qwen4ExpForConditionalGeneration), and the runtime graph in #27742 has no MTP handling either, so an exported head would not load even if produced. The closed PR #27739 did implement both halves, and its author deferred to #27742 with the intent that the useful parts be reconciled — so this is a scope cut, not a limitation of the format.

If you need the MTP head, or full-model reproducibility, start from the original safetensors, not from this file.

Notes for quantizers

Two tensor shapes make this model awkward, and both are shape properties that a name-based recipe will not notice:

• ffn_down_exps rows are 640 wide (moe_intermediate_size), and
• per_layer_token_embd rows are 160 wide.

Neither is divisible by 256, so K-quants and every 256-block I-quant (IQ4_XS, IQ3_XXS, IQ2_S, …) are unusable on them and fall back to 32-block types. That is 92.3 B of 180 B params — 51% of the model — with a floor near 4.5 bpw, which flattens the size/quality curve fast: pushing the remaining gate/up experts from IQ4_XS down to IQ2_XXS moves the total only from ~112 GB to ~77 GB.

per_layer_token_embd is a 51.2 B-param sparse lookup table — only a few rows are read per token — and at Q4_0 it is 28.8 GB, roughly a third of any tier here.

On a discrete-GPU system it is worth keeping in host RAM with -ot "per_layer_token_embd=CPU", since host memory is a separate and usually larger pool and the access pattern is cheap over PCIe.

On a unified-memory system (DGX Spark / GB10, Apple silicon) that does nothing: CPU and GPU share one pool, so there is nowhere to offload to. Budget the full file size as the footprint there, plus KV cache and compute buffers.

Provenance

• Source: Qwen/Qwen3.8-Flash-Next safetensors, 144 files, 360.0 GB, all verified byte-exact against the Hub after download.
• Converter: convert_hf_to_gguf.py from PR #27742 at the commit linked above, --outtype bf16.
• Unofficial community conversion — not affiliated with or endorsed by Qwen. Provided as-is, without warranty.