README: full-model eval results; correct serving pin to v4-capable branch

README.md

@@ -2,9 +2,9 @@
 library_name: vllm
 tags: [arvq, nvfp4, experimental]
 ---
-# GLM-5.3-Vision-NVFP4-ARVQ-hybrid
+# GLM-5.3-Vision-NVFP4-ARVQ-v2-hybrid
 
-Per-expert ARVQ v3 cold experts, NVFP4 hot experts, ARVQ-scored REAP allocation.
+Per-expert ARVQ v4 cold experts, NVFP4 hot experts, ARVQ-scored REAP allocation.
 **75/75 MoE layers replaced by the full-corpus sequential PV campaign.**
 Other layers retain their previously published weights; see pv_progress.json.
 Each layer's two tensor files and reports are replaced together in one commit.
@@ -25,17 +25,40 @@ the best checkpoint. Audit non-regression and export replay gate publication.
 Student inputs reflect retained tuned upstream layers. Fixed reference targets
 use original FP8-source routed experts and the donor backbone. A rolling cache
 carries both trajectories. Cold arithmetic emulates FP4 activation planes and
-FP16 boundaries; native SM120 parity and full-model quality remain unverified.
+FP16 boundaries; native SM120 kernel execution has not yet been smoke-tested
+(the evaluation below decodes the published weights to BF16 in software).
 The audit set is historical development data, not an untouched final test.
 Hot experts, backbone, BF16 MTP, vision components and allocation are unchanged.
 
+## Full-model evaluation (2026-09-22)
+
+Teacher-forced perplexity and per-token KL divergence against the NVFP4 donor
+(all 75 cold layers decoded and substituted, context 2048, 8-way sharded over
+four held-out corpora):
+
+| Corpus | Donor ppl | This repo ppl | KLD (nats) | Top-1 agreement |
+|---|---|---|---|---|
+| calib-domain heldout | 2.9005 | 2.9954 | 0.114 | 89.7% |
+| calib-domain heldout-xl | 2.6050 | 2.8028 | 0.150 | 88.7% |
+| wikitext | 3.0160 | 5.3726 | 0.735 | 70.4% |
+| github code | 2.7471 | 2.7950 | 0.097 | 90.8% |
+
+In-domain degradation is small (KLD 0.10–0.15 nats; 35–40% lower than the v1
+ARVQ repo, whose corresponding KLDs are 0.191/0.237/0.733/0.154). Wikitext
+shows a large gap (+78% ppl vs donor) on both this repo and v1, indicating a
+calibration-domain bias of the ARVQ calibration mix rather than a fitting
+regression; treat general-English quality accordingly.
+
 ## Serving (vLLM)
 
 Serve with the SM120 fork `github.com/jarrelscy/vllm-glm52-sm120`, branch
-`experiment/arvq-fp16-block-scales`, commit `00f57ca51`. That build loads the
-v4 cold-expert format `rvq256_256x8_expert_fp16block` used by the PV-replaced
-layers here (FP16 block scales converted at load time and applied in FP32
-after the FP4 MMA; experimental ABI — rebuild loader and libraries together).
-The `glm52-sm120` main branch and the v3 `arvq-hybrid-sm120` branch cannot
-load FP16 block-scale layers, and `experiment/arvq-fp16-scales-rs4` is for
-the v2r repo (quarter-scale residual decode), not this one.
+`experiment/arvq-mcbook16`, commit `d7ada6d` or later. That build loads the
+v4 cold-expert format `rvq256_256x8_expert_fp16block` used by all PV-replaced
+layers here (fitted FP16 block scales serialized directly, applied in FP32
+after the FP4 MMA; experimental ABI — rebuild the ARVQ libraries together
+with the loader via arvq/build.sh). The config.json `arvq` marker declares
+format version 4 to match. Earlier branches cannot load this repo:
+`experiment/arvq-fp16-block-scales` (00f57ca51) registers uint8 E4M3 scales
+and predates the v4 format string; the `glm52-sm120` main branch and the v3
+`arvq-hybrid-sm120` branch are v3-only; `experiment/arvq-fp16-scales-rs4`
+is for the v2r repo (quarter-scale residual decode), not this one.