---
library_name: vllm
tags: [arvq, nvfp4, experimental]
---
# GLM-5.3-Vision-NVFP4-ARVQ-hybrid

Per-expert ARVQ v3 cold experts, NVFP4 hot experts, ARVQ-scored REAP allocation.
**75/75 MoE layers replaced by the full-corpus sequential PV campaign.**
Other layers retain their previously published weights; see pv_progress.json.
Each layer's two tensor files and reports are replaced together in one commit.

Training draws sequentially from 18,001,846 text tokens at context 1024. Fixed validation and
development-audit sets each contain 16,384 tokens. Adam trains FP4-constrained
per-expert books and FP8-constrained per-block scales. The effective batch is
262,144 tokens, accumulated in four 65,536-token passes. Layers 4–26 use 69
updates: book/scale LR .048/.032 through update 45, then .012/.008. From layer27,
69 updates is the maximum: three validation checks more than 0.1% worse than
best trigger an earlier LR reduction; 15 updates without improvement after
the reduction permit stopping. Layer30 retains audit-qualified update15 after
its validation-best update20 failed the development audit. Layer 3
retains its separately qualified lower-LR refinement. Output-gradient index
reassignment occurs every 20 updates; validation every five updates retains
the best checkpoint. Audit non-regression and export replay gate publication.

Student inputs reflect retained tuned upstream layers. Fixed reference targets
use original FP8-source routed experts and the donor backbone. A rolling cache
carries both trajectories. Cold arithmetic emulates FP4 activation planes and
FP16 boundaries; native SM120 parity and full-model quality remain unverified.
The audit set is historical development data, not an untouched final test.
Hot experts, backbone, BF16 MTP, vision components and allocation are unchanged.