leoncca/Qwen3.8-Flash-Next-AWQ-g32

Qwen3.8 Flash Next AWQ g32 + QSA FP8 E4M3 KV

AWQ derivative of Qwen/Qwen3.8-Flash-Next. Only routed-expert projections are quantized. PLE is reused from the official FP8 checkpoint, and calibrated QSA FP8 E4M3 K/V scales are included.

138.13 GB across 44 model shards: 10 AWQ, 33 PLE, and one K/V-scale shard.

Checkpoint

• Base revision: f5d08274bafd880402bd16f5e3e6c514136ec06c
• PLE revision: bcd9f01ddc9cff2316eb84281bebcd5b058bddce
• Routed experts: asymmetric AWQ W4A16, g32, zero point, GEMM layout
• Quantized: per-expert gate_proj, up_proj, and down_proj
• Kept at source precision: vision, PLE, attention, routers, shared experts, embeddings, LM head, hyper-connections, and MTP
• Indexed tensors: 222,771 with 137,042,968,666 bytes of tensor payload
• Merged index SHA-256: f066a0154a9101b359c3a4d4fa6a83fb12b7126a9cf0f19cb772611d08cc07ee

Machine-readable details are in EXPERT_COVERAGE.json, MODEL_PROVENANCE.json, VALIDATION.json, and SHA256SUMS.

Why AWQ g32

Two properties favor asymmetric AWQ here:

1. On hardware without native FP4 execution, AWQ W4A16 provides a practical weight-only path while keeping activations in FP16.
2. The routed-expert weights frequently have off-center local ranges, which an asymmetric affine codebook can represent with a movable zero point.

A route-frequency-stratified CPU audit covered all 93 layer/expert pairs with fewer than 128 natural tokens, including all eight zero-hit pairs, plus controls from every layer. In total it compared 707 layer/expert pairs, all three routed projections, 2,121 matrices, and 69,500,928 BF16 weight values. It dequantized the actual NVFP4 checkpoint and compared both formats against the same BF16 mother weights; all 48 unquantized router tensors matched bitwise.

Route stratum	Pairs	Matrices	Actual NVFP4 relative RMSE	Affine g32 relative RMSE	Reduction	g32 wins
Zero natural tokens	8	24	0.094704	0.082320	13.08%	24/24
1–31 natural tokens	14	42	0.094387	0.083253	11.80%	42/42
32–63 natural tokens	17	51	0.094737	0.082029	13.41%	51/51
64–127 natural tokens	54	162	0.094747	0.082104	13.34%	162/162
Complete <128 census	93	279	0.094685	0.082289	13.09%	279/279
Non-low-frequency controls	614	1,842	0.094887	0.081588	14.02%	1,842/1,842
All deduplicated selections	707	2,121	0.094859	0.081686	13.89%	2,121/2,121

Affine g32 had lower error in every sampled matrix, including all 279 low-frequency gate/up/down matrices. The complete low-frequency census also reduced NMSE by 24.47%.

The result was not only a group-size effect: asymmetric affine g16 reached 0.069948 relative RMSE, compared with 0.094859 for actual NVFP4. Of the sampled BF16 groups, 56.78% had a range-center shift above 0.1 and 25.00% above 0.2. Only 55.40% of raw affine g32 zero-points and 58.73% of released AWQ zero-points landed at the central 7/8 positions. These weights benefit from a movable asymmetric zero point rather than a fixed zero-centered codebook.

The g32 layout also matches the model's width-640 experts under TP4: 640 / 4 = 160, and 160 is divisible by 32 but not 128. This avoids storing g128 metadata that must later be expanded to effective g32. The cost is about 6.59 GiB more scale and zero-point metadata than g128.

Calibration coverage

AWQ weight calibration used 684 records and 202,750 active tokens, producing 97,320,000 native top-10 token-to-expert assignments across 48 layers. Natural routing covered 24,568 of 24,576 layer/expert pairs (99.9674%), with 45/48 layers at 512/512. The remaining eight zero-hit pairs and 85 low-coverage pairs were explicitly augmented for gate/up/down projection calibration. The released checkpoint requires zero runtime fallback pairs. This corpus is separate from the 4,130,597-token E4M3 K/V-scale calibration described below.

Loading requirements

The loader must support:

• Qwen4Exp per-expert asymmetric AWQ W4A16 g32 in GEMM layout;
• indexed reuse of complete PLE shard files while ignoring unindexed tensors;
• all 24 bundled, finite, positive QSA K/V scale entries;
• complete Qwen4Exp multimodal support, including MRoPE and video-token-pruning initialization, for image and video inference.

The image-text-to-text tag requires the complete multimodal path; text-only loading does not establish image or video support.

Validation

Quality check	FP16 KV	Calibrated E4M3 KV
Basic generation	4/4	—
Needle retrieval, length sweep	6/6: one case each at 1K, 4K, 16K, 32K, 64K, and 128K	—
Needle retrieval, 128K repeat set	—	6/6: three cases × two repeats
Held-out tool selection	10/12	10/12
General repeat set	6/6 exact-stable	—
Held-out tool repeat set	3/6 exact-stable	4/6 exact-stable
GSM8K five-shot subset	29/32	29/32
HumanEval/MBPP functional subset	9/10	9/10
IFEval	3/5 prompts, 9/12 instructions	3/5 prompts, 9/12 instructions
Image and video requests	18/18 semantic; 9/9 cases exact-stable	18/18 semantic; 9/9 cases exact-stable

— means that the check was not run in that K/V arm; it is not a failed result.

The E4M3 scales were calibrated from 4,130,597 tokens. All 24 values are finite and range from 0.0171072837 to 0.0806361660, with zero observed calibration saturation. First tokens matched on 18/18 checks. Selected-block recall averaged 0.995906 with a minimum of 0.991822; QSA outputs had minimum cosine 0.998597 and maximum relative L2 0.052962.

The multimodal set contained five image and four video cases, each repeated twice. All 333 model.visual.* tensors are bitwise identical to the reference checkpoint, so AWQ did not alter the vision tower payload.

Notes

• The bundled K/V scales belong to this exact merged checkpoint and should not be replaced with scales from another weight artifact.
• The reported results are regression checks rather than a full capability benchmark.
• This derivative uses the included Qwen Community License 1.0.

Integrity

Verify the repository with SHA256SUMS. The authoritative artifact identity is the merged index SHA-256 shown above.