Add GSM8K interim benchmark (96.5%, N=200, chat+reasoning)

193/200 correct, 0 request errors, strict == flexible on every item. 3/200 empty on the max_tokens=4096 generation cap. Interim: no BF16 / NVIDIA-reference A/B yet.

README.md

@@ -51,3 +51,13 @@ Serving needs the mamba scheduler knobs to avoid a spec-v2-vs-radix-cache boot c
 Stock SGLang 0.5.15 crashes loading this checkpoint on GB10/sm121; it needs five launch-time source patches, shipped in the public dgxarley repo (https://github.com/vroomfondel/dgxarley) under roles/k8s_dgx/files/sglang_patches/ and applied by sglang_launch.sh: (1) p50 -- allow QUANTIZED attention for modelopt_fp4 (this is a uniform-W4A4 export that also quantizes the Gated-DeltaNet linear-attn, unlike NVIDIA's MoE-only NVFP4); (2) p51 -- load the main model's baked FP8 KV scales onto RadixAttention (remap ...self_attn.k_proj.k_scale -> ...attn.k_scale), else the full-attn layers default to scale 1.0; (3) p43 -- keep quant_config for the QUANTIZED MTP draft head, else SGLang assumes a BF16 MTP, builds it unquantized, and the fused-MoE loader narrows the unpacked intermediate dim over the NVFP4-packed expert weight ("start(0)+length(512) exceeds dimension size(256)"); (4) p44 -- apply the same KV-scale mapper in the MTP load path (qwen3_5_mtp.py has its own load_weights) so the draft attention loads its baked FP8 scales instead of the 1.0 sentinel; (5) the arch-independent sm121 CUTLASS-FP4 mma patch. p43/p44 are specific to a quantized MTP and are inert for a BF16-MTP checkpoint. Independently, config.json pins torch_dtype: bfloat16 (the Qwen3.6 source config omits it) so SGLang's --dtype auto resolves bf16; without it the unquantized in_proj_ba loads as fp16 and crashes at the CUDA-graph capture ("mat1 and mat2 ... BFloat16 != Half"). Validated end to end on GB10/sm121 (image 0.5.15.post1-sm121): loads register-free, NEXTN speculation active, coherent output, MTP draft attn loads its borrowed KV scales (k_scale 0.0398, byte-identical to the last full-attn layer, the requant borrow source).
 
 <!-- END quant:remarks -->
+
+<!-- BEGIN quant:benchmarks -->
+
+## Benchmarks
+
+| Task | Metric | Value | Setup | Hardware | Date | Notes |
+| --- | --- | --- | --- | --- | --- | --- |
+| GSM8K | accuracy (strict == flexible) | 96.5% | chat + reasoning (qwen3 think), 4-shot, temperature 0.6, max_tokens 4096, N=200 (test split head) | 4x DGX Spark GB10 / sm121; SGLang 0.5.15.post1-sm121, NEXTN spec num_steps=2, flashinfer attention + flashinfer_cutlass MoE | 2026-07-25 | 193/200 correct, 0 request errors; strict == flexible on every item (no over-generation artifact). 3/200 came back empty on the generation-length cap, one of them verifiably needs ~8.8k reasoning tokens, so max_tokens 4096 is the binding limit rather than the model. Interim result: 95% CI [93.0%, 98.3%] at this N, and no BF16 / NVIDIA-reference A/B has been run yet, so this does not by itself establish that the quant is lossless. |
+
+<!-- END quant:benchmarks -->