Publish quantization/data reproduction source and correct recipe documentation

Recovered source, runbooks, recipe/data/environment provenance, CPU verification and path relocation. Correct ARVQ-v2 corpus, objective, schedule, boundary weighting and FP16 reference decode; preserve model weights and original layer reports. Explicitly document historical reproduction gaps.

METHODOLOGY.md

@@ -1,342 +1,7 @@
-# ARVQ cold-expert quantization: methodology
+# ARVQ-v2 methodology correction
 
-How the cold Mixture-of-Experts weights in this repository
-(`GLM-5.3-Vision-NVFP4-ARVQ-hybrid`) were quantized and tuned. This document
-reconstructs the actual pipeline and tuning decisions from the build transcripts
-and the fitting/serving code; numbers are cited from those sources. It is a
-methodology record, not a quality claim: full-model task quality and native
-SM120 execution were **not** evaluated for this checkpoint.
+The previous file was copied from v1 and described the wrong target, scale dtype, corpus and schedule. The current methodology is documented in [reproduce/README.md](reproduce/README.md), with exact stage order in [RUNBOOK.md](reproduce/RUNBOOK.md) and settings in [recipe.json](reproduce/recipe.json).
 
----
+Key facts: same-input target; original FP8 expert teacher; FP4 per-expert books; FP16 block scales;15,007,754-token cleaned corpus;50× next-boundary loss weighting; reassignment every10; decay by25 or earlier; patience10. Validation selection is unweighted target-relative L2. Most layers stop early. All75 layer receipts remain available. Historical contradictory metadata is archived under reproduce/historical_metadata.
 
-## 1. What this model is (the hybrid)
-
-GLM-5.3 is a large MoE. Each MoE block routes each token to its top-8 of 256
-routed experts (plus a shared expert). This checkpoint is a **hybrid** in which
-each expert is stored in one of two ways:
-
-- **Hot experts** — kept in the NVFP4 donor format (unchanged).
-- **Cold experts** — re-quantized to ~2 bits with **ARVQ** (additive residual
-  vector quantization), the subject of this document.
-
-The hot/cold split is fixed by a REAP allocation that keeps the **5,750** most
-important experts hot; the remaining **13,450** experts across the model are
-cold (per the 09-14 build transcript, which verified "13,450 cold-expert source
-files"). The allocation is 75% text-REAP / 25% multimodal-salience weighted and
-is **not** modified by this campaign.
-
-Only the **75 MoE blocks, layers 3–77**, are touched. Layers 0–2 are dense and
-stay frozen. Everything except the cold experts is inherited unchanged:
-attention, backbone, shared experts, hot NVFP4 experts, BF16 MTP, and the
-vision components.
-
-Cold experts are quantized per (layer, projection). The two projections are the
-fused gate/up `w13` (N=4096, K=6144) and the down projection `w2` (N=6144,
-K=2048).
-
----
-
-## 2. ARVQ representation and bit budget
-
-Each cold expert weight group of **8 contiguous columns** is represented as the
-sum of two codebook atoms times a scale:
-
-```
-w_group = global * block_scale * (c0[a] + c1[b])
-```
-
-- **Two codebooks** `c0`, `c1`, each **256 entries × 8 dims** (512 codewords per
-  pair). This is the "rvq256_256x8" format. Codebooks are **per-expert** in this
-  v3 checkpoint (`codebook_scope = expert`, packed format
-  `rvq256_256x8_expert`, version 3).
-- **Indices** `a`, `b` are one uint8 each per 8-weight group → **16 index bits
-  per group = 2.0 bits/weight**.
-- **Codebook atoms are constrained to the FP4 grid** `{0, ±0.5, ±1, ±1.5, ±2,
-  ±3, ±4, ±6}` (project-to-FP4 after every update). The FP4 grid on the atoms is
-  why no incoherence/Hadamard rotation is used: the target space cannot be
-  rescaled without leaving the grid (see `encoder.py` docstring).
-- **Block scales** are stored as **`float8_e4m3fn`**, one scale per **[row,
-  128-column block]** (48 blocks/row for `w13`, 16 blocks/row for `w2`). At use
-  time each block scale is `repeat_interleave(16)` to cover the 16 eight-wide
-  groups in a 128-column block. Adding the FP8 scale (8 bits / 128 weights =
-  0.0625 bit/weight) gives the reported **2.0625 bits/weight** plus the
-  amortized per-expert codebooks.
-- A single fp32 `global` scalar per (layer, projection) sits above the block
-  scales so the index/scale layout is unchanged from earlier versions.
-
-Packing (`pack.py`) writes MMA-fragment index tiles (64 uint32 words/tile) and
-verifies, per layer, that every expert's indices round-trip exactly and that a
-sample of decoded weights matches bit-for-bit before publishing.
-
----
-
-## 3. Initial fit (Hessian-aware, no PV yet)
-
-Before any gradient tuning, each (layer, projection) gets an
-activation-Hessian-tuned initialization (`encoder.py`, `fit.py`):
-
-1. **Per-expert raw-basis Hessian.** `w13` uses `H = E[x xᵀ]` over routed
-   tokens; `w2` uses `H = E[m mᵀ]` where `m = silu(x·Wgᵀ)·(x·Wuᵀ)` is the BF16
-   SwiGLU intermediate. Experts with <32 routed tokens fall back to all captured
-   tokens. Activations are stored/computed in **float32**; Hessians are half.
-2. **Escalating-damp Cholesky** of `H⁻¹`: damping starts at `1e-3 · mean(diag H)`
-   and multiplies by 4× (up to 12 attempts) until the factorization succeeds.
-3. **Codebook EM** on a Hessian-importance-weighted subsample (20,000 groups/
-   expert) of normalized 8-dim groups: k-means warm start → 8 alternating
-   refine iterations, FP4-projecting the atoms after every M-step.
-4. **Per-expert LDLQ/GPTQ error-feedback column sweep** with the fixed
-   codebooks (col_block=128, 2 sweep passes, 1 inner refine): plain-L2 inner
-   assignment, off-diagonal Hessian carries cross-group error feedback.
-5. **Least-squares refit of the E4M3 block scales** given the chosen codes.
-
-An earlier decision, recorded 09-14: a **tuned Hadamard rotation was tested and
-rejected** — a matched 48-expert down-projection refit gave 0.372 (H128 rotated)
-versus **0.191 unrotated**, so the unrotated raw-basis fit was adopted.
-
-Nonfinite guards are raised as errors throughout (the fit/tuner refuse to
-proceed on NaN/Inf rather than silently scrubbing).
-
----
-
-## 4. The PV objective: `--target reference`
-
-The cold experts are then output-tuned per layer. The objective for this
-checkpoint is **`--target reference`** (`sequential_pv_full_corpus.py`).
-
-For each captured token position the tuner forms:
-
-- `frozen` — the block output from everything that is **not** a cold expert on
-  the **student** input (retained tuned upstream layers → attention → shared
-  expert → hot NVFP4 experts). This is fixed.
-- `reference` — the **original FP8 reference block output**: the unchanged donor
-  backbone with the **original FP8 routed experts** (all 256), evaluated on the
-  **original reference trajectory** for that token position.
-- `required = reference − frozen` — the residual the cold experts must supply.
-
-The loss is the relative squared error of the cold-expert sum against `required`:
-
-```
-loss = || student_cold(x_student) − required ||²  /  (denominator · Σw)
-```
-
-where `denominator` is the per-row mean energy of `reference − frozen` over the
-corpus (so the scale is comparable across layers). Selection metric is
-`reference_rel = ||(frozen + cold) − reference|| / ||reference||`, measured over
-the **whole transformer block including the residual**, not the cold-experts sum
-alone.
-
-**Why reference (and its known weakness).** The transcripts weighed
-`reference` against a `same_input` target (match the block's output on the
-*student's own* drifted input). The rationale for reference (09-16): "Matching
-the original reference trajectory is more directly aligned with preserving the
-model's behavior." The acknowledged risk: because each layer receives a
-**different (student) input** than the original, the cold experts may lack the
-capacity to reproduce the reference output and could "learn corrections that
-work on training examples but fail on unseen ones" — i.e. reference-target is
-the more out-of-distribution objective. It was chosen only after a matched
-layer-4 A/B:
-
-| Layer-4 metric | reference vs same_input |
-|---|---|
-| validation reference error | **0.21% better** with reference |
-| development-audit error | 0.08% **worse** with reference |
-
-A small mixed difference; reference was retained for behavioral alignment. Two
-important design consequences: the target is the **complete block output**
-(including the residual), so matching only the MoE contribution cannot leave the
-incoming residual error untouched; and the loss compares the **routing-weighted
-sum of all cold experts** jointly (gate/up and down together).
-
----
-
-## 5. Sequential per-layer pipeline with trajectory coupling
-
-Layers are processed **sequentially, 3 → 77** (`full_reference.py` /
-`sequential_capture.py`):
-
-1. **Capture** (8-way sequence-parallel). For layer L, the student input is the
-   **retained tuned output of block L-1** (`student_outputs.pt`), while the
-   reference target re-runs the **donor backbone + original FP8 experts** on the
-   original reference hidden state. Both trajectories are carried forward in a
-   rolling cache; layer 3 is the control (no preceding ARVQ layer, so student =
-   reference input). This cross-layer coupling means each layer is tuned against
-   the exact drifted inputs it will see at serving time.
-2. **Fit / PV tune** the cold experts (Section 6).
-3. **Gate** (Section 7).
-4. **Export + serialized replay** — decode to the packed format and require the
-   reloaded forward to match the tuned forward to <2e-6 before use.
-5. **Per-layer HF publish** — `publish_reference.py` re-assembles cold slots
-   across the 8 expert shards, checks slot coverage/allocation/global metadata,
-   runs the exact index round-trip and sampled decoded-weight parity, then
-   **atomically replaces that layer's two tensor files + reports in one commit**
-   with parent-commit protection; remote hashes are re-verified.
-
-Parallelism: cold experts are **sharded across 8 GPUs** (each rank owns distinct
-experts); routed outputs are summed with an all-reduce so a single coupled-output
-objective is optimized without a dense-model replica.
-
----
-
-## 6. PV tuning numerics (the published full-corpus campaign)
-
-Per the model card, all 75 layers were replaced by a **full-corpus sequential
-PV campaign** drawing sequentially from **18,001,846 training tokens** at
-context **1024** (53,248 legacy tokens were removed around 17 exact matches to
-held-out prompts). Fixed **validation** and **development-audit** sets each hold
-**16,384 tokens**.
-
-- **Optimizer:** Adam. Two trainable parameter groups — FP4-constrained
-  per-expert **codebooks** and FP8-constrained per-block **scales**. (The
-  faster fork trains per-row scale deltas instead; the published full-corpus
-  fork trains the per-128-block log-scales.)
-- **Effective batch:** 262,144 tokens, accumulated as **four 65,536-token
-  microbatch passes** (256 whole 1024-token sequences), one shuffled
-  no-replacement pass over the corpus.
-- **Learning-rate schedule:** book/scale LR **0.048 / 0.032** through update 45,
-  then dropped by ×0.25 to **0.012 / 0.008** (`lr_decay_after=45`,
-  `lr_decay_factor=0.25`). (Note: these campaign LRs are higher than the
-  in-repo code defaults of 0.003/0.002, consistent with the much larger
-  262k-token batch.)
-- **Update budget:** layers 4–26 use a fixed **69 updates**. From layer 27, 69
-  is the maximum with an **adaptive early stop**: three validation checks more
-  than 0.1% worse than best trigger an earlier LR reduction; 15 updates without
-  improvement after that reduction permit stopping.
-- **Validation every 5 updates**, retaining the best checkpoint.
-- **Index reassignment every 20 updates** (Section 6.1).
-- **Regularization:** codebook drift-from-anchor penalty + scale drift penalty,
-  weighted 0.01; grad-norm clip 1.0; codebook atoms clamped to [-6, 6];
-  log-scales clamped to within ±0.35 of their initial value.
-- **Coverage gate:** experts with fewer than **256 routed training rows** are
-  frozen (gradient masked) and keep their initial books/scales.
-
-### 6.1 Output-gradient index reassignment
-
-Continuous Adam cannot move the discrete indices, so every 20 updates a discrete
-proposal pass runs (`gradient_indices.py`, method
-`expert_parallel_output_gradient_prefix_backtracking_v1`):
-
-1. Backprop the **output** loss to the reconstructed weight of one expert
-   projection.
-2. Propose alternative `(a,b)` index pairs near a small gradient step, capped at
-   `max_fraction=0.001` of groups, `trust_ratio=0.01`, `target_ratio=0.03`; the
-   current pair is always in the candidate set (changing nothing stays legal).
-3. **Prefix backtracking acceptance:** try the top-k proposals with
-   k∈{full, ¼, 1/16, 1}, accept only if the **training** SSE strictly drops
-   **and** a separate **check batch** SSE does not increase (beyond 1e-7 slack);
-   otherwise revert. Proposal deltas are broadcast sparsely (only routed rows).
-
-The transcripts flag this as the method's main theoretical weakness versus
-published PV-Tuning: codebooks are tuned for output accuracy while index
-proposals originally came from weight-reconstruction Hessians — the
-output-gradient proposal above was added to close that gap.
-
-### 6.2 Cold arithmetic emulation
-
-Both training and evaluation emulate the serving numerics (`activation.py`,
-pinned to serving revision `b1380cf7…`): **four FP4 activation planes**
-(`fp4_planes4_fp16_boundaries_v1`) and **FP16 SwiGLU boundaries** (FP32 GEMM,
-FP16 SiLU/product), with straight-through estimators for gradients. This
-emulates the quantization boundaries only — **native SM120 MMA accumulation and
-TP reduction are not bit-exact** and were not qualified.
-
----
-
-## 7. Acceptance gates
-
-Publication of a layer requires all of:
-
-- **Validation non-regression:** `0 ≤ final_reference_rel ≤ initial·(1+1e-6)`.
-- **Development-audit non-regression:** the held-out audit split (never used for
-  updates or checkpoint selection) must satisfy
-  `audit_reference_rel ≤ initial_audit·(1+1e-6)`; otherwise the layer keeps its
-  initialization. There is no absolute error floor — the rule is purely
-  final ≤ initial.
-- **Serialized-replay parity:** decoded/reloaded forward matches the tuned
-  forward to <2e-6, and remote file hashes are re-verified after upload.
-
-The audit is honestly labeled **development data, not an untouched final test**;
-it is a split of the same calibration tokens the initialization already saw. Two
-notable per-layer decisions recorded in the card: **layer 30** retains its
-audit-qualified update-15 checkpoint after its validation-best update-20 failed
-the development audit; **layer 3** retains a separately qualified lower-LR
-refinement.
-
----
-
-## 8. Calibration corpus
-
-The text corpus (`build_calib_v31.py` + `reasoning_slice.py`) is deterministic
-(seed 42), GLM-tokenized, with the following domain mix by tokens:
-
-| Share | Domain | Sources |
-|---|---|---|
-| ~32% | code | local vLLM sources, m-a-p/CodeFeedback, jtatman/python-code-500k |
-| ~20% | tool-calling / agentic | generated GLM chat-template sessions |
-| ~15% | reasoning | OpenR1-Math-220k + dolphin-r1 `<think>…</think>` → answer |
-| ~13% | instruction chat | tatsu-lab/alpaca + coding chat |
-| ~10% | medical | MedQA textbook continuation + medical Q&A |
-| ~10% | prose | vLLM docs markdown + databricks-dolly-15k |
-
-The **reasoning slice was added specifically** because the earlier calib-v3 mix
-starved the cold 2-bit experts of reasoning-termination behavior: 87% of its
-`<think>` blocks were empty. The slice emits multi-turn conversations where an
-**earlier** assistant turn carries a real `<think>…</think>` that terminates and
-hands off to an answer, followed by a trailing user turn, so the think-close
-token **`</think>` (id 154842)** renders in-stream rather than as a trailing
-generation prompt. Held-out prompts are explicitly excluded (last 25 vLLM docs,
-last 3 MedQA files, vLLM code beyond index 400 of the seed-42 shuffle), and
-17 exact-match prompts were purged from the training set.
-
-The capture/training code also supports **up-weighting boundary rows** (rows
-whose next token is a boundary such as `</think>` / `<|endoftext|>`) via a
-`row_weight` term whose sum normalizes the loss, and an optional matched-mixed
-validation set; the published card describes reasoning-based validation.
-
----
-
-## 9. Results captured during the build
-
-All errors below are **held-out relative L2**, either over the whole transformer
-block (including residual) or over the cold-experts sum, as noted. They are
-local reconstruction errors, **not** token-accuracy or perplexity.
-
-- **Pilot smoke test** (layer 3, one step): full-block error 0.005497 → 0.005456
-  (~0.75%), 304/388 expert-projection proposals accepted; exported weights
-  reproduced the retained result.
-- **Layer 3** (200-step pilot): full-block reference error **0.005497 →
-  0.005271 (−4.12%)**, best at step 200.
-- **Layer 4** (200-step pilot): full-block reference error **0.015760 →
-  0.015596 (−1.05%)**.
-- **reference vs same_input** (layer 4, matched): reference 0.21% better on
-  validation, 0.08% worse on development audit.
-- **Faster fork parity:** the optimized fork (larger microbatch, specialized
-  embedding-lookup backward, sparse proposal broadcast) matched the reference
-  fork's validation and audit **exactly** on layers 3 and 4, with all propagated
-  BF16 outputs equal; training loops fell from ~1525 s → ~337 s (layer 3) and
-  ~1226 s → ~303 s (layer 4).
-- **Early-layer stability example** (layer 18, full-corpus): initial
-  0.017205, step-5 0.017176, step-69 0.017183 — differences ≤0.17%, treated as
-  a signal to watch LR/noise rather than proof of convergence.
-
-Older cold-only checkpoints reported ~0.2211→0.2070 (layer 3) and 0.2364→0.2320,
-but those measured **cold-expert output only on different captures** and are
-**not comparable** to the full-block errors above.
-
-**No comparison against the FP8 donor or an AQLM variant on perplexity / KLD /
-top-1 / wikitext was recorded** in the reviewed transcripts, and no full-model
-task evaluation was performed. Those remain open.
-
----
-
-## 10. Honest limitations
-
-- Full-model quality and native **SM120** execution were **not** evaluated; the
-  arithmetic emulation covers FP4/FP16 boundaries only, not MMA/TP bit-exactness.
-- The development audit is a split of calibration data, not an independent test.
-- The reference target is the more OOD objective; its generalization advantage
-  over same_input was small and mixed on the one matched layer tested.
-- Gates enforce local non-regression, not any absolute quality bar.
-
-*Prepared from the GLM-5.3 ARVQ build transcripts (2026-09-14 and 2026-09-16)
-and the `btx53/arvq88` fitting/serving code. Where a number could not be sourced
-it is stated as unknown rather than estimated.*
+The bundled reference packer/decoder now handles both uint8 E4M3 and actual FP16 scales, tested on CPU. No model weights were changed by this correction.

README.md

@@ -9,26 +9,15 @@ Per-expert ARVQ v4 cold experts, NVFP4 hot experts, ARVQ-scored REAP allocation.
 Other layers retain their previously published weights; see pv_progress.json.
 Each layer's two tensor files and reports are replaced together in one commit.
 
-Training draws sequentially from 18,001,846 text tokens at context 1024. Fixed validation and
-development-audit sets each contain 16,384 tokens. Adam trains FP4-constrained
-per-expert books and FP8-constrained per-block scales. The effective batch is
-262,144 tokens, accumulated in four 65,536-token passes. Layers 4–26 use 69
-updates: book/scale LR .048/.032 through update 45, then .012/.008. From layer27,
-69 updates is the maximum: three validation checks more than 0.1% worse than
-best trigger an earlier LR reduction; 15 updates without improvement after
-the reduction permit stopping. Layer30 retains audit-qualified update15 after
-its validation-best update20 failed the development audit. Layer 3
-retains its separately qualified lower-LR refinement. Output-gradient index
-reassignment occurs every 20 updates; validation every five updates retains
-the best checkpoint. Audit non-regression and export replay gate publication.
-
-Student inputs reflect retained tuned upstream layers. Fixed reference targets
-use original FP8-source routed experts and the donor backbone. A rolling cache
-carries both trajectories. Cold arithmetic emulates FP4 activation planes and
-FP16 boundaries; native SM120 kernel execution has not yet been smoke-tested
-(the evaluation below decodes the published weights to BF16 in software).
-The audit set is historical development data, not an untouched final test.
-Hot experts, backbone, BF16 MTP, vision components and allocation are unchanged.
+The production recipe uses **same-input targets**, **FP16 block scales**, and a **50× boundary loss weight**. It differs from v1; previous versions of this README accidentally retained the v1 training description.
+
+The cleaned v3.2 training stream contains **15,007,754 tokens**, context1024, effective batch262,144 and microbatch65,536. It removes empty think blocks and frames documents with canonical GLM delimiters. Reasoning validation and development audit captures contain16,384 tokens each; audit is WikiText/OOD and was used during development. The maximum one-pass budget is58updates, with adaptive early stopping: LR .048/.032,4× decay by update25 or earlier after three worsening validation checks, then patience10. Reassignment runs every10updates and validation every5. Published receipts show67/75 layers selecting update5; a full corpus is available, but most layers do **not** consume an entire pass.
+
+Boundary weighting is50 on the **position preceding** token154842 (`</think>`) or154820 (`<|endoftext|>`),1 elsewhere. It multiplies squared output residuals once and normalizes by summed weights; it is not2500×. Window-final positions have weight1. Validation metrics and discrete reassignment checks remain unweighted. Books stay on the FP4 grid; learned FP16 block scales serialize directly. The cold index rate is2bpw, or2.125bpw including block scales, plus codebook/global overhead.
+
+Student inputs are propagated through the selected upstream hybrid layers. The target is the original FP8-source routed experts evaluated on **those same student inputs**, with frozen hot contribution subtracted. This is not v1's independent original-reference trajectory objective. Two trajectories may still be carried for diagnostic comparison. Reference labels in some historical reports are generic/stale; the reports' `target: same_input` and fitting code determine the actual objective.
+
+Hot experts, backbone and vision components are inherited; the original bootstrap hot-allocation mismatch was corrected as noted below. Native runtime correctness is a separate qualification from software decoding.
 
 ## Full-model evaluation (2026-09-22)
 
@@ -46,8 +35,7 @@ four held-out corpora):
 In-domain degradation is small (KLD 0.10–0.15 nats; 35–40% lower than the v1
 ARVQ repo, whose corresponding KLDs are 0.191/0.237/0.733/0.154). Wikitext
 shows a large gap (+78% ppl vs donor) on both this repo and v1, indicating a
-calibration-domain bias of the ARVQ calibration mix rather than a fitting
-regression; treat general-English quality accordingly.
+possible calibration-domain mismatch. These results do not isolate corpus bias from quantization format, optimization, or trajectory effects; treat general-English quality accordingly.
 
 **Repo history note (2026-09-22):** the hot tier, `aqlm_layer_books` and
 `cold_assignment.json` were re-exported to match the cold_manifests allocation.
@@ -78,3 +66,9 @@ format version 4 to match. Earlier branches cannot load this repo:
 and predates the v4 format string; the `glm52-sm120` main branch and the v3
 `arvq-hybrid-sm120` branch are v3-only; `experiment/arvq-fp16-scales-rs4`
 is for the v2r repo (quarter-scale residual decode), not this one.
+
+## Reproduce this release
+
+The [reproducibility guide](reproduce/README.md), [stage-by-stage runbook](reproduce/RUNBOOK.md), [recipe](reproduce/recipe.json), [data artifact hashes](reproduce/data_artifacts.json), and [source tree](reproduce/source) now accompany this checkpoint. The package contains202 recovered source files spanning corpus preparation, capture, fitting, PV, export and evaluation. Run `python reproduce/verify.py` for CPU checks.
+
+**Scope:** the2026-09-26 audit adds code/documentation and corrects metadata; it does not change model weights. Exact historical retraining is not yet guaranteed: some original local corpus inputs, streamed-dataset revisions and environment locks were not preserved. Recovered artifacts, missing inputs and changed-source qualifications are explicitly documented rather than concealed.

arvq88_reference.py

@@ -20,7 +20,66 @@ def pack_codebooks(c0,c1):
  levels=torch.tensor(LEVELS,device=cb.device);n=(cb[...,None]-levels).abs().argmin(-1)
  if not torch.equal(levels[n],cb):raise ValueError('Codebooks must be exactly on the FP4 grid')
  return (n.long()<<(torch.arange(8,device=cb.device)*4)).sum(-1).to(torch.uint32)
-def decode(t,layer,proj,e,N=None,K=None):
+def pack_codebooks_mb(cb,factors):
+ """mcbook16: cb [E,256+B*256,8] in EFFECTIVE units. Rows are stored as plain-grid
+ nibbles; decode multiplies residual book m by factors[m] (exact powers of two)."""
+ B=len(factors)
+ if cb.ndim!=3 or cb.shape[1]!=256+B*256 or cb.shape[2]!=8:raise ValueError('Invalid mcbook codebook shape')
+ f=torch.cat([torch.ones(256),torch.tensor(factors).float().repeat_interleave(256)])[None,:,None]
+ grid=cb/f
+ levels=torch.tensor(LEVELS,device=cb.device);n=(grid[...,None]-levels).abs().argmin(-1)
+ if not torch.equal(levels[n]*f,cb):raise ValueError('Codebooks must be exactly on their per-book grids')
+ return (n.long()<<(torch.arange(8,device=cb.device)*4)).sum(-1).to(torch.uint32)
+def decode_mb(t,layer,proj,e,N,K):
+ pre=f'model.layers.{layer}.mlp.experts.arvq_{proj}_'
+ a,b=unpack(t[pre+'packed'][e],N,K)
+ cb=t[pre+'codebooks'][e].long();factors=t[pre+'book_factors'].float();B=len(factors)
+ if cb.shape!=(256+B*256,):raise ValueError('Invalid mcbook packed codebook shape')
+ values=torch.tensor(LEVELS,device=cb.device)[(cb[:,None]>>(torch.arange(8,device=cb.device)*4))&15]
+ values=torch.cat([values[:256],values[256:]*factors.repeat_interleave(256)[:,None]])
+ sel=t[pre+'selectors'][e].long()
+ m=sel.repeat_interleave(16,0).repeat_interleave(8,1)
+ stored=t[pre+'scales'][e].permute(0,2,1).contiguous().reshape(N,K//128)
+ if stored.dtype==torch.float16:scales=stored.float()
+ elif stored.dtype==torch.uint8:scales=stored.view(torch.float8_e4m3fn).float()
+ else:raise ValueError('Unsupported packed block scale dtype')
+ return ((values[a.long()]+values[256+m*256+b.long()])*scales.repeat_interleave(16,1)[...,None]*t[pre+'global']).reshape(N,K)
+def export_layer_mb(source,dest,L):
+ """mcbook16 export (format version 5): 16 residual books, per-tile 4-bit selectors,
+ fp16 block scales; packed index stream and scales unchanged from v4."""
+ from safetensors.torch import save_file,load_file
+ source=Path(source);dest=Path(dest);dest.mkdir(parents=True,exist_ok=True)
+ original=json.loads((source/'arvq-manifest.json').read_text());files={};first_factors=None
+ for proj,tag in [('w13','gateup'),('w2','down')]:
+  d=torch.load(source/f'{proj}.pt',map_location='cpu',weights_only=True)[proj];N,K=d['N'],d['K'];E=len(d['a']);pre=f'model.layers.{L}.mlp.experts.arvq_{proj}_'
+  if d.get('scale_dtype')!='fp16':raise ValueError('mcbook export requires fp16 block scales')
+  if not torch.equal(d['s'],d['s'].half().float()):raise ValueError('Scales are not FP16-representable')
+  factors=[float(f) for f in d['book_factor']]
+  if first_factors is None:first_factors=factors
+  elif factors!=first_factors:raise ValueError('Mixed projection book factors')
+  sel=d['selector']
+  if sel.shape!=(E,N//16,K//64) or int(sel.max())>=len(factors):raise ValueError('Invalid selector tensor')
+  t={pre+'packed':torch.stack([pack(a,b,N,K) for a,b in zip(d['a'],d['b'])]),
+     pre+'codebooks':pack_codebooks_mb(d['cb'],factors),
+     pre+'selectors':sel.to(torch.uint8).contiguous(),
+     pre+'book_factors':torch.tensor(factors,dtype=torch.float32),
+     pre+'scales':d['s'].half().reshape(E,N//16,16,K//128).permute(0,1,3,2).contiguous(),
+     pre+'global':torch.tensor([d['glob']],dtype=torch.float32)}
+  for e in range(E):
+   a,b=unpack(t[pre+'packed'][e],N,K);assert torch.equal(a,d['a'][e]) and torch.equal(b,d['b'][e])
+  for e in sorted({0,E//2,E-1}):
+   m=d['selector'][e].long().repeat_interleave(16,0).repeat_interleave(8,1)
+   cb=d['cb'][e]
+   ref=((cb[d['a'][e].long()]+cb[256+m*256+d['b'][e].long()])*d['s'][e].repeat_interleave(16,1)[...,None]*d['glob']).reshape(N,K)
+   assert torch.equal(ref,decode_mb(t,L,proj,e,N,K))
+  name=f'arvq-layer-{L:03d}-{tag}.safetensors';tmp=dest/(name+'.tmp')
+  save_file(t,str(tmp),metadata={'format':'pt','arvq_format':'rvq256_mb16_256x8_expert_fp16block','codebook_scope':'expert','residual_books':str(len(factors)),'book_factors':json.dumps(factors),'fit':str(original.get('fit','initial Hessian fit, no PV')),'serving_compatibility':'requires mcbook16 v5 loader/kernel (vllm-glm52-sm120 branch experiment/arvq-mcbook16)'})
+  tmp.replace(dest/name)
+  loaded=load_file(str(dest/name));assert loaded[pre+'packed'].shape==(E,N//16,K//64,64) and loaded[pre+'selectors'].shape==(E,N//16,K//64);del loaded
+  files[proj]={'file':name,'sha256':sha(dest/name)};del t,d
+ m={**original,'format':'rvq256_mb16_256x8_expert_fp16block','version':5,'residual_scale_shift':0,'block_scale_dtype':'float16','codebook_scope':'expert','bits':2.125+4/1024,'codebook_sizes':[256,16*256],'index_bits':[8,8],'selector_bits_per_tile':4,'words_per_tile':64,'book_factors':first_factors,'files':files,'fit':str(original.get('fit','initial Hessian fit, no PV')),'production_ready':False,'roundtrip':'all expert indices exact; 3 decoded weights/projection exact','requires_8x8_loader_kernel':True,'requires_mcbook16_loader':True}
+ (dest/f'layer-{L:03d}-manifest.json').write_text(json.dumps(m,indent=2));return m
+def decode(t,layer,proj,e,N=None,K=None,residual_scale_shift=0):
  pre=f'model.layers.{layer}.mlp.experts.arvq_{proj}_'
  default=(4096,6144) if proj=='w13' else (6144,2048)
  N,K=default if N is None else (N,K)
@@ -28,16 +87,27 @@ def decode(t,layer,proj,e,N=None,K=None):
  if cb.ndim==2:cb=cb[e]
  if cb.shape!=(512,):raise ValueError('Invalid packed codebook shape')
  values=torch.tensor(LEVELS,device=cb.device)[(cb[:,None]>>(torch.arange(8,device=cb.device)*4))&15]
- scales=t[pre+'scales'][e].permute(0,2,1).contiguous().reshape(N,K//128).view(torch.float8_e4m3fn).float()
- return ((values[a.long()]+values[256+b.long()])*scales.repeat_interleave(16,1)[...,None]*t[pre+'global']).reshape(N,K)
+ stored=t[pre+'scales'][e].permute(0,2,1).contiguous().reshape(N,K//128)
+ if stored.dtype==torch.uint8:scales=stored.view(torch.float8_e4m3fn).float()
+ elif stored.dtype==torch.float16:scales=stored.float()
+ else:raise ValueError('Unsupported packed block scale dtype')
+ f=2.0**(-int(residual_scale_shift))
+ return ((values[a.long()]+f*values[256+b.long()])*scales.repeat_interleave(16,1)[...,None]*t[pre+'global']).reshape(N,K)
 def sha(p):
  h=hashlib.sha256()
  with open(p,'rb') as f:
   for chunk in iter(lambda:f.read(8<<20),b''):h.update(chunk)
  return h.hexdigest()
-def export_layer(source,dest,L):
+def export_layer(source,dest,L,residual_scale_shift=0):
  from safetensors.torch import save_file,load_file
  source=Path(source);dest=Path(dest);dest.mkdir(parents=True,exist_ok=True)
+ probe=torch.load(source/'w13.pt',map_location='cpu',weights_only=True,mmap=True)['w13']
+ if 'selector' in probe:
+  del probe
+  if int(residual_scale_shift):raise ValueError('mcbook exports are shift-0 (per-book factors)')
+  return export_layer_mb(source,dest,L)
+ del probe
+ rss=int(residual_scale_shift);f=2.0**(-rss)
  original=json.loads((source/'arvq-manifest.json').read_text());files={}
  for proj,tag in [('w13','gateup'),('w2','down')]:
   d=torch.load(source/f'{proj}.pt',map_location='cpu',weights_only=True)[proj];N,K=d['N'],d['K'];E=len(d['a']);pre=f'model.layers.{L}.mlp.experts.arvq_{proj}_'
@@ -45,20 +115,28 @@ def export_layer(source,dest,L):
   if expert and d['c0'].shape!=(E,256,8):raise ValueError('Codebook expert count mismatch')
   if proj=='w13':scope='expert' if expert else 'layer'
   elif scope!=('expert' if expert else 'layer'):raise ValueError('Mixed projection codebook scopes')
+  fp16=d.get('scale_dtype','fp8_e4m3')=='fp16'
+  if proj=='w13':fp16_blocks=fp16
+  elif fp16_blocks!=fp16:raise ValueError('Mixed projection scale precision')
+  if fp16 and not torch.equal(d['s'],d['s'].half().float()):raise ValueError('Scales are not FP16-representable')
+  stored_scales=d['s'].half() if fp16 else d['s'].to(torch.float8_e4m3fn).view(torch.uint8)
   t={pre+'packed':torch.stack([pack(a,b,N,K) for a,b in zip(d['a'],d['b'])]),
      pre+'codebooks':pack_codebooks(d['c0'],d['c1']),
-     pre+'scales':d['s'].to(torch.float8_e4m3fn).view(torch.uint8).reshape(E,N//16,16,K//128).permute(0,1,3,2).contiguous(),
+     pre+'scales':stored_scales.reshape(E,N//16,16,K//128).permute(0,1,3,2).contiguous(),
      pre+'global':torch.tensor([d['glob']],dtype=torch.float32)}
   for e in range(E):
    a,b=unpack(t[pre+'packed'][e],N,K);assert torch.equal(a,d['a'][e]) and torch.equal(b,d['b'][e])
   for e in sorted({0,E//2,E-1}):
    c0,c1=(d['c0'][e],d['c1'][e]) if expert else (d['c0'],d['c1'])
-   ref=((c0[d['a'][e].long()]+c1[d['b'][e].long()])*d['s'][e].repeat_interleave(16,1)[...,None]*d['glob']).reshape(N,K)
-   assert torch.equal(ref,decode(t,L,proj,e,N,K))
+   ref=((c0[d['a'][e].long()]+f*c1[d['b'][e].long()])*d['s'][e].repeat_interleave(16,1)[...,None]*d['glob']).reshape(N,K)
+   assert torch.equal(ref,decode(t,L,proj,e,N,K,rss))
+  base_fmt='rvq256_256x8_expert_fp16block' if fp16 else ('rvq256_256x8_expert' if expert else 'rvq256_256x8')
+  arvq_fmt=base_fmt+(f'_rs{int(2**rss)}' if rss else '')
   name=f'arvq-layer-{L:03d}-{tag}.safetensors';tmp=dest/(name+'.tmp')
-  save_file(t,str(tmp),metadata={'format':'pt','arvq_format':'rvq256_256x8_expert' if expert else 'rvq256_256x8','codebook_scope':scope,'fit':str(original.get('fit','initial Hessian fit, no PV')),'serving_compatibility':'requires per-expert v3 loader/kernel' if expert else 'requires 8+8 loader/kernel'})
+  save_file(t,str(tmp),metadata={'format':'pt','arvq_format':arvq_fmt,'residual_scale_shift':str(rss),'codebook_scope':scope,'fit':str(original.get('fit','initial Hessian fit, no PV')),'serving_compatibility':'requires FP16 block-scale v4 loader/kernel' if fp16 else ('requires per-expert v3 loader/kernel' if expert else 'requires 8+8 loader/kernel')})
   tmp.replace(dest/name);del t,d
   loaded=load_file(str(dest/name));assert loaded[pre+'packed'].shape==(E,N//16,K//64,64);del loaded
   files[proj]={'file':name,'sha256':sha(dest/name)}
- m={**original,'format':'rvq256_256x8_expert' if scope=='expert' else 'rvq256_256x8','version':3 if scope=='expert' else 2,'codebook_scope':scope,'bits':2.0625,'codebook_sizes':[256,256],'index_bits':[8,8],'words_per_tile':64,'files':files,'fit':str(original.get('fit','initial Hessian fit, no PV')),'production_ready':False,'roundtrip':'all expert indices exact; 3 decoded weights/projection exact','requires_8x8_loader_kernel':True}
+ base_mfmt='rvq256_256x8_expert_fp16block' if fp16_blocks else ('rvq256_256x8_expert' if scope=='expert' else 'rvq256_256x8')
+ m={**original,'format':base_mfmt+(f'_rs{int(2**rss)}' if rss else ''),'version':4 if fp16_blocks else (3 if scope=='expert' else 2),'residual_scale_shift':rss,'block_scale_dtype':'float16' if fp16_blocks else 'float8_e4m3fn','codebook_scope':scope,'bits':2.125 if fp16_blocks else 2.0625,'codebook_sizes':[256,256],'index_bits':[8,8],'words_per_tile':64,'files':files,'fit':str(original.get('fit','initial Hessian fit, no PV')),'production_ready':False,'roundtrip':'all expert indices exact; 3 decoded weights/projection exact','requires_8x8_loader_kernel':True}
  (dest/f'layer-{L:03d}-manifest.json').write_text(json.dumps(m,indent=2));return m

calibration_corpus.json

@@ -1,24 +1,17 @@
 {
-  "source": "/tmp/glm53-unc-traces",
-  "reasoning_rollouts": true,
-  "counts": {
-    "train": {
-      "problems": 856,
-      "tokens": 3048596
-    },
-    "validation": {
-      "problems": 89,
-      "tokens": 345875
-    },
-    "audit": {
-      "problems": 104,
-      "tokens": 322277
-    }
-  },
-  "partitions": "disjoint prompt hashes; separately captured",
-  "limitations": "Text-only captures in 2048-token windows; source datasets overlap earlier calibration. Teacher answers are not correctness-verified.",
-  "initial_fit_partition": "train only",
-  "capture_teacher": "RadixArk/GLM-5.3-NVFP4",
-  "rollout_teacher": "unc NVFP4 teacher",
-  "same_token_corpus_as": "/tmp/glm53-kernel-aware-pv/full"
-}
+  "stage": "current recipe provenance correction",
+  "variant": "ARVQ-v2",
+  "full_pv_training_tokens": 15007754,
+  "sequence_length": 1024,
+  "validation_rows": 16384,
+  "audit_rows": 16384,
+  "boundary_boost": 50,
+  "boundary_token_ids": [
+    154842,
+    154820
+  ],
+  "data_pipeline": "reproduce/README.md",
+  "data_hashes": "reproduce/data_artifacts.json",
+  "historical_component_metadata": "reproduce/historical_metadata/calibration_corpus.json",
+  "exact_retraining_guaranteed": false
+}

pv_progress.json

@@ -77,111 +77,28 @@
     77
   ],
   "total_layers": 75,
-  "recipe": "full_corpus_sequential_lr_decay",
-  "training_tokens": 18001846,
+  "recipe": "full_corpus_sequential_same_input_fp16block",
+  "training_tokens": 15007754,
   "batch_tokens": 262144,
   "microbatch_tokens": 65536,
-  "max_updates_per_layer": 69,
+  "max_updates_per_layer": 58,
   "adaptive_schedule": {
-    "from_layer": 27,
     "relative_worsening": 0.001,
     "checks": 3,
-    "patience_updates": 15,
-    "max_updates": 69,
-    "selection": "existing fixed reasoning validation; retain best checkpoint",
-    "description": "Drop LR4x after three consecutive validation checks >0.1% worse than best, or after45 at latest. After reduction stop following15 updates without a new best."
+    "patience_updates": 10
   },
-  "lr_decay_after": 45,
+  "lr_decay_after": 25,
   "lr_decay_factor": 0.25,
-  "layer3_exception": "45 high-LR updates plus retained lower-LR refinement checkpoint",
-  "remaining_layers": "retain their previously published weights",
-  "previous_campaign_progress": {
-    "layers_pv_complete": [
-      3,
-      4,
-      5,
-      6,
-      7,
-      8
-    ],
-    "total_layers": 75,
-    "recipe": "sequential_reference_gradient_pv_v1",
-    "legacy_pv_layers": [
-      9,
-      10,
-      11,
-      12,
-      13,
-      14
-    ],
-    "initial_fit_layers": [
-      15,
-      16,
-      17,
-      18,
-      19,
-      20,
-      21,
-      22,
-      23,
-      24,
-      25,
-      26,
-      27,
-      28,
-      29,
-      30,
-      31,
-      32,
-      33,
-      34,
-      35,
-      36,
-      37,
-      38,
-      39,
-      40,
-      41,
-      42,
-      43,
-      44,
-      45,
-      46,
-      47,
-      48,
-      49,
-      50,
-      51,
-      52,
-      53,
-      54,
-      55,
-      56,
-      57,
-      58,
-      59,
-      60,
-      61,
-      62,
-      63,
-      64,
-      65,
-      66,
-      67,
-      68,
-      69,
-      70,
-      71,
-      72,
-      73,
-      74,
-      75,
-      76,
-      77
-    ],
-    "format": "rvq256_256x8_expert",
-    "full_model_quality": "pending"
-  },
-  "format": "rvq256_256x8_expert",
-  "full_model_quality": "pending"
-}
+  "layer3_exception": "v2 layer3 uses same LR recipe; selected step50 after58 updates, per its published report",
+  "format": "rvq256_256x8_expert_fp16block",
+  "full_model_quality": "See README full-model evaluation; software-dequant evaluation, not a guarantee of native runtime parity",
+  "target": "same_input",
+  "boundary_boost": 50.0,
+  "boundary_token_ids": [
+    154842,
+    154820
+  ],
+  "reassign_every": 10,
+  "block_scale_dtype": "float16",
+  "metadata_corrected_on": "2026-09-26"
+}

reproduce/AUDIT.md

@@ -0,0 +1,13 @@
+# Audit evidence and corrections, 2026-09-26
+
+Evidence precedence: actual checkpoint tensor headers and75 per-layer PV reports; saved campaign config; executable capture/loss/packing source; local RUNLOG53 chronology and Claude build conversation; model-card prose. Private transcripts are not uploaded.
+
+Confirmed v2 defects corrected: copied v1 README and METHODOLOGY; incorrect18,001,846-token claim (actual15,007,754); FP8 vs actual FP16 scales; reference vs same-input target; decay45/patience15/reassign20 vs25/10/10; missing50× boundary loss rule; stale pv_progress; root reference decoder reinterpreting FP16 bytes as E4M3. Header samples show U8 in v1 and F16 in v2. CPU fixtures verify corrected decode in both formats.
+
+v1 methodology now distinguishes ARVQ output-benefit allocation from the old AQLM allocation. Both ARVQ corpus metadata files now distinguish earlier/component provenance from final PV streams. AQLM notes clarify sampled activation PV, global75/25 row mixture,65536-entry shared layer/projection books and absence of boundary weighting.
+
+v2 WikiText degradation is measured; attributing it solely to corpus bias was too strong and has been corrected. Quantization and optimization effects were not isolated by a controlled ablation.
+
+Historical per-layer `checkpoint_selection` strings can be stale even when `target` and recorded metrics are correct; original reports are retained, not retroactively rewritten. Old publisher scripts are archived and can regenerate stale prose; use the corrected root documentation and recipe files as the current documentation contract.
+
+Missing provenance is explicitly listed. This package is a practical reconstruction/runbook and source release, not certification of byte-identical historical retraining.

reproduce/README.md

@@ -0,0 +1,88 @@
+# Reproducing the GLM-5.3 hybrid releases
+
+This package was reconstructed on 2026-09-26 from the local quantization source, saved configurations, per-layer reports, checkpoint tensor headers, and Claude build history. The original private conversations are not redistributed. `checkpoint_before_audit.json` pins the weights/reports audited; the present documentation-only revision does not retrain or replace weights.
+
+## Reproducibility status
+
+The fitting, capture, data-building, evaluation, packing and verification source is included in `source/` with SHA-256 checksums. This is a **recovered working-tree snapshot**, containing subsequent fixes and experiments, not a falsely reconstructed original Git commit. `recipe.json` selects the intended recipe; do not infer it from experimental filenames. Data hashes and current environment versions are included. Existing model weights and original per-layer reports remain the reference artifacts.
+
+A bit-for-bit re-training claim is **not justified**: original streamed dataset revisions, local corpus inputs and the complete original dependency lockfile were not all retained. Some training paths were overwritten by later campaigns. The original AQLM `/data` activation and sample archives are no longer present at their recorded paths; the v1 18,001,846-token flat stream has not been recovered. These gaps must be resolved or a rerun must be labeled a new reproduction, with measured differences. Neither a fixed random seed nor the current package versions restores missing historical inputs.
+
+## Package layout
+
+- `source/tools/`: text corpus builders, source loading/NVFP4 emulation, capture, AQLM fitting, PV and validation.
+- `source/tools/capture53/`: GLM5.3 wrappers, multimodal corpus/capture, allocation, checkpoint assembly, MTP conversion, vision graft and full-model PPL/KL evaluation.
+- `source/btx53/`: FP8 source ingest, raw-basis ARVQ initializer, activation emulation, coupled sequential PV, capture/propagation, REAP allocation and export tooling; `arvq88/perf/` contains the full-corpus controller and its gates.
+- `recipe.json`, `layer_recipe_summary.json`: recovered variant-specific settings and original layer exceptions.
+- `data_artifacts.json`: SHA-256, sizes/shapes and token-boundary counts for recovered token files; paths are historical identities, not downloadable links.
+- `historical_metadata/`: original conflicting metadata retained as historical evidence, not the current recipe.
+- `prepare_workspace.py`: creates a new relocatable copy; does not overwrite the archive or execute a training job.
+- `verify.py`: CPU-only source checksum, boundary-weight semantics and FP8/FP16 reference-decoder checks.
+
+## Environment and paths
+
+Training/capture used eight A100s with software dequantization/emulation. This is not proof of native SM120 kernel equivalence. `environment_observed.json` records the current audit environment, **not** the original training environment. `requirements.in` lists main Python dependencies, not an exact historical lock. Use a CUDA-compatible PyTorch build and a Transformers version supporting the donor tokenizer; install additional optional dependencies demanded by the chosen multimodal backend. Data-source scripts record their imports. Native serving requires the custom serving fork described on the model card.
+
+```bash
+python -m venv .venv
+. .venv/bin/activate
+pip install -r reproduce/requirements.in
+python reproduce/verify.py
+python reproduce/prepare_workspace.py --dest /absolute/path/glm53-reproduction
+cd /absolute/path/glm53-reproduction
+# Use the same activated interpreter for all following commands.
+```
+
+The preparer copies source and maps legacy `/data` and `/tmp` paths into the destination, and the old checkout path into this source tree. It writes the transformed source hashes and path map. Review `recipe.json` and scripts before GPU work: donor, tokenizer, corpus, source-cache and baseline locations must exist. Historical scripts that spawn `ROOT/.venv/bin/python` need that interpreter available (the preparer creates the link). Old publication/watchdog scripts are archival; **do not run them against the original public repos**. Export to a new directory and use your own destination repo after verification.
+
+Acquire donor/base/vision snapshots using pinned revisions in `historical_metadata/source.json`, `backbone_sources.json`, available model-index provenance, and the audited checkpoint metadata. Where a revision is missing or marked `legacy_cache_revision_unverified`, record that gap rather than substituting today's HEAD as if it were historical. The source FP8 checkpoint and NVFP4 donor are distinct teachers. AQLM uses the NVFP4 donor; ARVQ cold fitting uses original FP8 expert weights. `btx53/tools/prefetch_cold_fp8.py`, `arvq88/inputs.py`, `local_source.py` and `tools/aqlm_quantize.py` implement source fetching/dequantization.
+
+## Data pipelines and non-portable inputs
+
+### AQLM text calibration (v3)
+
+`tools/build_calib_v3.py`: seed42, target15M tokens, approximately40% code /25% agentic /15% instruction /10% medical /10% prose. Streams CodeFeedback, python-code-dataset-500k, Alpaca and Dolly; also uses local vLLM sources/docs, local tools, synthetic shell/agent sessions and MedQA material. See the builder for exact generator order and per-category stopping. It excludes reserved vLLM/MedQA files, but many dataset revisions were not pinned. The original `collect_expert_stats_v2.py` also reads host logs, diffs and shell outputs: these **host inputs must be frozen and reviewed** to reproduce the original corpus; they are not included here. Do not publish freshly collected private logs merely to recreate this pipeline.
+
+The initial capture uses nonoverlapping2048-token windows. `stream_capture53.py stats` records routing/salience and sampled activations; it is not a claim that later AQLM PV trains on every15M token. AQLM convergence and PV use the retained activation sample.
+
+### Multimodal branch
+
+`capture53/build_calib_mm.py`: seed42,1200 image/caption samples,300 each from medical, natural images, OCR/documents and screenshots. Primary candidates: eltorio/ROCOv2-radiology, lmms-lab/COCO-Caption2017, naver-clova-ix/synthdog-en and HuggingFaceM4/WebSight v0.2; explicit fallbacks live in the script. Retain generated `samples.jsonl`, selected dataset/config/split, dataset revision, sample IDs and checksums. Images are resized448×448 and processed with the checkpoint KimiK25 processor into256 vision tokens. Source licenses differ (ROCOv2 includes a noncommercial/share-alike restriction); model MIT metadata does not relicense the datasets. No image or caption archive is redistributed here.
+
+`capture_mm53.py` splices the GLM5.2V tower/projector outputs and masks padding. `solve_r3.py` allocates using .75*per-layer-normalized text REAP + .25*per-layer-normalized multimodal salience. Mixed activation files concatenate32768 text rows and10922 multimodal rows. Thus75/25 is a **global row mixture**, not a guarantee that each expert's routed Hessian contains exactly25% visual energy/rows.
+
+### ARVQ v1 full-corpus stream
+
+The published75 layer reports describe18,001,846 tokens (17,580 windows including the short tail), context1024. Historical metadata's3,048,596-token reasoning corpus identifies a component/earlier capture, not the complete final stream. The exact original concatenated stream is not recovered. Do not replace it silently with the current15M v32 file or label v32 a faithful v1 data reproduction. Capture reads `train_token_file`; supply a recovered checksum-verified v1 stream to reproduce that campaign. Use original layer reports for early-stop and recovery exceptions.
+
+### ARVQ v2 v3.2 data rebuild
+
+`tools/build_calib_v32.py` actual executable budget is22% code /15% agentic /30% reasoning /13% instruction /10% medical /10% prose, seed42, target15M; actual complete documents produce15,007,754 tokens. Its old opening docstring predates the30% reasoning increase; the budget constants and this runbook are authoritative. Medical data switches to MedRAG textbooks when old local MedQA is unavailable. Reasoning sources include OpenR1-Math-220k and dolphin-r1. Reasoning is shortened to3500 characters, answers1200. Dataset availability, streaming order and local files can affect exact regeneration.
+
+`clean_think` removes empty/whitespace-only think blocks, trailing generation-only prompts and unmatched think tags while retaining content. `frame_ids` adds canonical `[gMASK]<sop>` prefix and `<|endoftext|>` suffix only when absent. Use donor tokenization; IDs154822/154824/154820 correspond to those delimiters. Concatenate `shard_*.npy` in sorted order into the configured `train.npy`, then compare length and SHA to `data_artifacts.json`.
+
+`tools/build_trace_refit_v32.py` builds three eval-capture sources: training reference from the first1M calibration tokens; held-out reasoning with RNG2025 skipping9000 reasoning documents; and OOD WikiText test text. The capturer uses RNG1234 to select64/16/16 nonoverlapping1024-token sequences, yielding65536 training-probe and16384 validation/audit rows. The procedural skip is not an independent content-deduplication guarantee; the audit split was repeatedly inspected during development.
+
+## Boundary weighting: exact semantics
+
+For v2, within each1024-token sequence, set `w[t]=50` when the **next** token is154842 (`</think>`) or154820 (`<|endoftext|>`); otherwise1. The last row is1 because the implementation does not look across the window boundary. The same rule is in standalone capture and fused handoff. It weights the hidden output responsible for predicting a boundary, not the boundary token's own output.
+
+The PV data term is `sum_t w[t] * ||cold_prediction[t] - required[t]||² / (D * sum_t w[t])`, with D the captured target-energy normalizer. The residual is squared **before** multiplying by50; the multiplier is50, not2500. The small anchor regularizer is separate. Reported validation/audit relative-L2 and boundary/bulk summaries are **unweighted**; checkpoint selection uses validation `target_rel`. Discrete index-reassignment proposal/check objectives are also unweighted in the recovered implementation. The50× multiplier is therefore a continuous PV-loss rule, not a blanket multiplier on all quantization/evaluation stages.
+
+AQLM and ARVQ v1 have no identified50× boundary term in their production recipes. Do not retroactively attribute the v2 change to them. Both the corpus cleanup and weighting were motivated by boundary preservation; no causal guarantee of shorter generation follows from this alone.
+
+## Verification and acceptance
+
+Before fitting: verify corpus hashes/counts, donor identity, tokenizer special IDs, cold/hot partition coverage and target dimensions. Freeze the assignment. Initial-fit output must cover every cold expert. After PV: retain the selected checkpoint, check finite metrics and audit/non-regression policy, then pack and reload for index and decoded-weight parity. Propagate the selected weights and recompute the next layer's inputs; do not mix independently repaired layers without re-evaluation. Save scripts/config hashes, training receipts, all per-layer metrics and environment lockfiles for the new run.
+
+`tools/capture53/eval3_kld.py` / `eval_hybrid53.py` evaluate teacher-forced PPL/KL. `btx53/arvq88/pack.py` verifies packed indices and sample decoded weights; the root `arvq88_reference.py` supports actual uint8-E4M3 and float16 scale storage. v3 cold storage is2+8/128=2.0625 bpw before codebook overhead; v4 is2+16/128=2.125 bpw before overhead. AQLM's2bits likewise describes index rate, excluding its65536×8 codebook and scales. Verify actual bytes if comparing compression budgets.
+
+This audit ran lightweight CPU verification; it did not rerun a multi-day quantization or certify native GPU serving. Historical PPL/task results on the model card retain their original scope.
+
+## Variant-specific commands
+
+Read [RUNBOOK.md](RUNBOOK.md) for this repository’s stage order and recipe.
+
+## External code and optional modules
+
+`external_sources.json` records observed upstream commits. Install b12x at its recorded revision only for the optional BTX comparison modules; these are not part of the ARVQ production recipe. The vLLM checkout is also an input to local code/prose generation and must be placed at `source/vllm` (or the relocated workspace’s `vllm`). Its current commit is not proof of the original dirty training checkout. Dataset IDs/licenses appear in the builders; preserve the resolved revisions and generated sample manifests on a new run. See THIRD_PARTY.md.

reproduce/RUNBOOK.md

@@ -0,0 +1,29 @@
+# ARVQ-v2 runbook
+
+Run the source from the prepared workspace. Preserve all75 original layer reports and the original cold allocation. `recipe.json` is the recovered v2 configuration.
+
+1. Prepare source experts from `zai-org/GLM-5.3` and the `RadixArk/GLM-5.3-NVFP4` donor. The published `source.json` pins a source revision but also marks legacy-cache revision verification unavailable; retain that qualification. Source dequantization uses block128×128 E4M3. Prepare the vision extension and capture inputs using the included capture tools.
+2. Run `tools/build_calib_v32.py`, concatenate sorted shards into `train.npy`, and run `tools/build_trace_refit_v32.py`. Confirm15,007,754 training tokens and the recorded hashes.
+3. Generate initial ARVQ fits and the ARVQ output-benefit allocation. Historical v2 launch (replace paths):
+
+```bash
+python btx53/build_baseline.py --pv-steps 0 --workdir BASELINE \
+ --calib-tokens CALIB_SHARDS --calib-token-count 15000000 \
+ --src-cache FP8_EXPERT_CACHE --nvfp4-donor DONOR \
+ --dst baseline-only --gpus 0,1,2,3,4,5,6,7 --codebook-scope expert
+```
+
+`--codebook-scope expert` is required. `arvq88/encoder.py` and `fit.py` implement raw-basis Hessian/LDLQ fitting; `arvq_reap.py` blends text/MM error-reduction benefit75/25 and keeps5750 hot experts. To reproduce the published assignment rather than resolve it anew, use the published `cold_assignment.json` and verify against every cold manifest. Never substitute the original AQLM allocation merely because the hot count matches.
+
+4. Place `recipe.json` as `WORK/config.json`, configure source/cache/corpus/baseline paths, and launch:
+
+```bash
+python btx53/arvq88/perf/full_pipeline.py --work WORK --first 3 --last 77
+```
+
+The controller captures full training rows plus fixed eval rows, runs coupled expert-parallel PV, gates retained outputs, and propagates the chosen student state. Fused handoff requires its bitwise replay qualification. The source includes later optimizations; preserve receipts/identity hashes and a fresh WORK if changing the recipe.
+
+5. v2: same-input source target, FP16 block scales,50× boundary weighting, LR .048/.032 with4× decay by update25 or earlier, patience10, index reassignment every10, evaluation every5. 58updates would cover the full15,007,754-token corpus; adaptive stopping often uses fewer. The published reports show67layers selecting update5. Do not describe this as one complete pass for every layer.
+6. Export through `arvq88/pack.py` / checkpoint assembly to a **new** destination. Run packed-index and decoded-weight parity, hot/cold consistency, full-model PPL/KL and native serving tests. The historical publishers reference public production repos and are provided for provenance, not as a safe rerun destination. Fixed metadata in this audit is the reference; historical publisher templates can reintroduce stale descriptions.
+
+A v2 model-wide v4 marker requires FP16-scale cold layers. Do not deploy a partially replaced v3/v4 mixture. The old bootstrap also had a hot-tier mismatch, corrected at f53a8dcc; use a later audited snapshot.

reproduce/THIRD_PARTY.md

@@ -0,0 +1 @@
+Source files retain their original notices. External checkouts and dataset material have their own licenses; the model card MIT tag does not override those. See external_sources.json and the dataset builder source for origins. The b12x/quant-toolkit/vLLM repositories are referenced rather than bulk vendored. No new license is asserted over third-party code or datasets by this audit.

reproduce/checkpoint_before_audit.json

@@ -0,0 +1,100 @@
+{
+  "repo": "jarrelscy/GLM-5.3-Vision-NVFP4-ARVQ-v2-hybrid",
+  "revision": "cbde0aed09e981e68bfefc827cb22d5fbdb81cfb",
+  "files": [
+    "METHODOLOGY.md",
+    "README.md",
+    "arvq88_reference.py",
+    "backbone_sources.json",
+    "btx_hybrid_manifest.json",
+    "build_provenance.json",
+    "calibration_corpus.json",
+    "cold_assignment.json",
+    "config.json",
+    "gates_report.json",
+    "generation_config.json",
+    "kimi_k25_processor.py",
+    "kimi_k25_vision_processing.py",
+    "media_utils.py",
+    "model.safetensors.index.json",
+    "preprocessor_config.json",
+    "pv_layers/layer_003.json",
+    "pv_layers/layer_004.json",
+    "pv_layers/layer_005.json",
+    "pv_layers/layer_006.json",
+    "pv_layers/layer_007.json",
+    "pv_layers/layer_008.json",
+    "pv_layers/layer_009.json",
+    "pv_layers/layer_010.json",
+    "pv_layers/layer_011.json",
+    "pv_layers/layer_012.json",
+    "pv_layers/layer_013.json",
+    "pv_layers/layer_014.json",
+    "pv_layers/layer_015.json",
+    "pv_layers/layer_016.json",
+    "pv_layers/layer_017.json",
+    "pv_layers/layer_018.json",
+    "pv_layers/layer_019.json",
+    "pv_layers/layer_020.json",
+    "pv_layers/layer_021.json",
+    "pv_layers/layer_022.json",
+    "pv_layers/layer_023.json",
+    "pv_layers/layer_024.json",
+    "pv_layers/layer_025.json",
+    "pv_layers/layer_026.json",
+    "pv_layers/layer_027.json",
+    "pv_layers/layer_028.json",
+    "pv_layers/layer_029.json",
+    "pv_layers/layer_030.json",
+    "pv_layers/layer_031.json",
+    "pv_layers/layer_032.json",
+    "pv_layers/layer_033.json",
+    "pv_layers/layer_034.json",
+    "pv_layers/layer_035.json",
+    "pv_layers/layer_036.json",
+    "pv_layers/layer_037.json",
+    "pv_layers/layer_038.json",
+    "pv_layers/layer_039.json",
+    "pv_layers/layer_040.json",
+    "pv_layers/layer_041.json",
+    "pv_layers/layer_042.json",
+    "pv_layers/layer_043.json",
+    "pv_layers/layer_044.json",
+    "pv_layers/layer_045.json",
+    "pv_layers/layer_046.json",
+    "pv_layers/layer_047.json",
+    "pv_layers/layer_048.json",
+    "pv_layers/layer_049.json",
+    "pv_layers/layer_050.json",
+    "pv_layers/layer_051.json",
+    "pv_layers/layer_052.json",
+    "pv_layers/layer_053.json",
+    "pv_layers/layer_054.json",
+    "pv_layers/layer_055.json",
+    "pv_layers/layer_056.json",
+    "pv_layers/layer_057.json",
+    "pv_layers/layer_058.json",
+    "pv_layers/layer_059.json",
+    "pv_layers/layer_060.json",
+    "pv_layers/layer_061.json",
+    "pv_layers/layer_062.json",
+    "pv_layers/layer_063.json",
+    "pv_layers/layer_064.json",
+    "pv_layers/layer_065.json",
+    "pv_layers/layer_066.json",
+    "pv_layers/layer_067.json",
+    "pv_layers/layer_068.json",
+    "pv_layers/layer_069.json",
+    "pv_layers/layer_070.json",
+    "pv_layers/layer_071.json",
+    "pv_layers/layer_072.json",
+    "pv_layers/layer_073.json",
+    "pv_layers/layer_074.json",
+    "pv_layers/layer_075.json",
+    "pv_layers/layer_076.json",
+    "pv_layers/layer_077.json",
+    "pv_progress.json",
+    "source.json",
+    "tokenizer_config.json"
+  ]
+}

reproduce/data_artifacts.json

@@ -0,0 +1,333 @@
+{
+  "note": "Recovered local artifact hashes; current paths may have been reused. v32 hashes belong to v2. No raw texts, local logs, private chats or image bytes are included. v1 original 18,001,846-token flat stream is not recovered by this manifest.",
+  "files": [
+    {
+      "path": "/tmp/glm52-calib-v3/shard_00000.npy",
+      "sha256": "84dba15d4fec9e222ab68d2196a926dfd60123a7741530b3cbc02a84e1f20cb8",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v3/shard_00001.npy",
+      "sha256": "25310686c241a8ddfd5463fa240bf0a9549885772cc0d0ff75d15138f00c1ea0",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v3/shard_00002.npy",
+      "sha256": "cbcfa8c283d8e894e83559198c52e65a95455ceee9bb5061c9cce90ec53478b9",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v3/shard_00003.npy",
+      "sha256": "b64418fda40e375922baa5755b8c01ef7b4b14b8d40c1fcd2ebd9c80e1cec381",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v3/shard_00004.npy",
+      "sha256": "b42bd5638c244923766898100cb0e75458919a3a127d42789c925d81ebe59699",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v3/shard_00005.npy",
+      "sha256": "a672f7431ddefb1b5233fc7e53e6f0fc09a36395231c5c2c8d8a7d4056fe74eb",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v3/shard_00006.npy",
+      "sha256": "e6dcafae0e4018eb7d99b7517366f05375570d934476ad0456b4e4448c890719",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v3/shard_00007.npy",
+      "sha256": "eb22593102c11b48a182852989f17399ca32b3bc77196b73b22cb3e3d58190f9",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v3/shard_00008.npy",
+      "sha256": "610bfc9c5dcaf030eabfeca2c4150f409391102634f9dff003a58ca32bb5b0ed",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v3/shard_00009.npy",
+      "sha256": "0464899366bb7429800be3f61486af04b871abde381a28c644a44b14793ff9a8",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v3/shard_00010.npy",
+      "sha256": "9691d0cfce1ec93e5ace299c4045e9c1bb2e53adcf713adccbd1336ebe9287a7",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v3/shard_00011.npy",
+      "sha256": "d48e3552bcbf2251ecbd98febc390167d15514f265d0cf4bab2169a173266438",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v3/shard_00012.npy",
+      "sha256": "255fa27435f0948f0af499e474810aa877951c68c84655afa5a7376c47592c04",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v3/shard_00013.npy",
+      "sha256": "94220e009467d6b645902d2bdd429aa29544a4eb9fa5d4f9b24bff823b8e0496",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v3/shard_00014.npy",
+      "sha256": "37d7b9604e62a12eef2c293ed899f6bb2a8032418e5c5b7c578c82ff29e5bf0c",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v3/shard_00015.npy",
+      "sha256": "a7a3a29377ef4436d6ec5185d5f45f2c7e615fbb093499e59a3651d78ef0181d",
+      "bytes": 26120,
+      "shape": [
+        6498
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v32/shard_00000.npy",
+      "sha256": "587d01f48f9d39719661004f51d288bed7ccba5808c2a86068ad83a72e8e9130",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v32/shard_00001.npy",
+      "sha256": "b34946701d15f586a7c7513ab37aa43c7bdbf1e9a51448a90ddea81850df8011",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v32/shard_00002.npy",
+      "sha256": "42b98201ecbfd0830fc333c6691ec67daaa6b82287b8d33a8ac2670dca14d842",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v32/shard_00003.npy",
+      "sha256": "87c619b34e746d40ab1b32827317aca92a5583d7a9a010c6199ccb63aeb52f8f",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v32/shard_00004.npy",
+      "sha256": "9ea10b0107c7afbb82256c5e00ddc001f5b0288d9f35e69712e88ffd20d70b28",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v32/shard_00005.npy",
+      "sha256": "5941cd2282b9c9eeeffcb500044a20c8bf5cb27d03ca3e0783491d956141af56",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v32/shard_00006.npy",
+      "sha256": "3a83b4ae5266f229e4431193594462f38f82357f0ecb20bb5bf2b79c1703e1ab",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v32/shard_00007.npy",
+      "sha256": "564afecdb45368874c7b371ed4627188b14d7a84ab3a8e2f26c26433fa228548",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v32/shard_00008.npy",
+      "sha256": "0a56d27bcab2ad9149e138c85b513016c97cc0950d8911ae49ed83e12c6f6c8c",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v32/shard_00009.npy",
+      "sha256": "7fe80d4b868c2d7f1d0a162a7fd80782f7d69f8f50ec6976f6a75c1a24dc2aca",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v32/shard_00010.npy",
+      "sha256": "61d886db2533ec343ca240ccdd859118476bd37dd59ba37cfcb0dfb56c6d5f57",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v32/shard_00011.npy",
+      "sha256": "d7e4af049ce7bf3f0186973d73f66990403f365deac2b1ba1c26bd04be917bfb",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v32/shard_00012.npy",
+      "sha256": "23bbeb34c4ac0f753192f1d8d17d3def7ba5cf6449149549bfe1ff9c0ef404e7",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v32/shard_00013.npy",
+      "sha256": "36f1aba2e26747a9227f7d60edde31d4e18b64f402578b42ccd1bbd16258414a",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v32/shard_00014.npy",
+      "sha256": "4d86d75d69d69b86fae7eba6aaf31153d9a043620bb94daee285925b4ce8a353",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm52-calib-v32/shard_00015.npy",
+      "sha256": "6a5fc10154611f997bc9957ced9c43f37c14372289a5c89d850a5f405358c5b9",
+      "bytes": 31144,
+      "shape": [
+        7754
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm53-vision-trace-refit/audit/tokens/shard_000.npy",
+      "sha256": "3f9fead556891373590e7017016833d7395674bdcf60c052dc663fb3190932bc",
+      "bytes": 1160816,
+      "shape": [
+        290172
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm53-vision-trace-refit/train/tokens/shard_000.npy",
+      "sha256": "587d01f48f9d39719661004f51d288bed7ccba5808c2a86068ad83a72e8e9130",
+      "bytes": 4000128,
+      "shape": [
+        1000000
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm53-vision-trace-refit/validation/tokens/shard_000.npy",
+      "sha256": "9f943a73f4a9eed1f8f817b34653f242f212b5df108d66dfeea4e7f0a5357f47",
+      "bytes": 1605812,
+      "shape": [
+        401421
+      ],
+      "dtype": "uint32"
+    },
+    {
+      "path": "/tmp/glm53-layer3-full-corpus/train.npy",
+      "sha256": "26812c90e966c66bbe33b0e0c8bf9572906deb65eda7ce022f73c58f04eda29a",
+      "shape": [
+        15007754
+      ],
+      "dtype": "uint32",
+      "boundary_counts": {
+        "154820": 18854,
+        "154841": 3099,
+        "154842": 3099
+      }
+    }
+  ]
+}

reproduce/environment_observed.json

@@ -0,0 +1,16 @@
+{
+  "captured_on": "2026-09-26",
+  "versions": {
+    "torch": "2.11.0+cu130",
+    "numpy": "2.2.6",
+    "transformers": "5.13.0",
+    "datasets": "5.0.0",
+    "safetensors": "0.8.0",
+    "huggingface_hub": "1.22.0",
+    "Pillow": "12.3.0",
+    "scipy": "1.18.0",
+    "triton": "3.6.0"
+  },
+  "historical_environment_exactly_recovered": false,
+  "note": "Current audit environment, not an original training lockfile. Pin a tested CUDA/PyTorch/Transformers stack before rerunning."
+}

reproduce/external_sources.json

@@ -0,0 +1,18 @@
+{
+  "scope": "Observed checkout commits, not proof of original dataset/training source state",
+  "vllm": {
+    "url": "https://github.com/vllm-project/vllm.git",
+    "revision": "f32e2837ffc41ad369fff3ba8b05461ce5b53b55",
+    "use": "optional serving/code-corpus checkout; local dirty diff not reproduced"
+  },
+  "b12x": {
+    "url": "https://github.com/local-inference-lab/b12x",
+    "revision": "9043b448622764a598969518d413b3fd8b3c0c07",
+    "use": "optional BTX comparison modules, not ARVQ production codec"
+  },
+  "quant-toolkit": {
+    "url": "https://github.com/local-inference-lab/quant-toolkit",
+    "revision": "8bdb1016e52dd15f91e42d35c09096c0f31325f3",
+    "use": "historical external toolkit; not vendored"
+  }
+}

reproduce/historical_metadata/METHODOLOGY.md

@@ -0,0 +1,342 @@
+# ARVQ cold-expert quantization: methodology
+
+How the cold Mixture-of-Experts weights in this repository
+(`GLM-5.3-Vision-NVFP4-ARVQ-hybrid`) were quantized and tuned. This document
+reconstructs the actual pipeline and tuning decisions from the build transcripts
+and the fitting/serving code; numbers are cited from those sources. It is a
+methodology record, not a quality claim: full-model task quality and native
+SM120 execution were **not** evaluated for this checkpoint.
+
+---
+
+## 1. What this model is (the hybrid)
+
+GLM-5.3 is a large MoE. Each MoE block routes each token to its top-8 of 256
+routed experts (plus a shared expert). This checkpoint is a **hybrid** in which
+each expert is stored in one of two ways:
+
+- **Hot experts** — kept in the NVFP4 donor format (unchanged).
+- **Cold experts** — re-quantized to ~2 bits with **ARVQ** (additive residual
+  vector quantization), the subject of this document.
+
+The hot/cold split is fixed by a REAP allocation that keeps the **5,750** most
+important experts hot; the remaining **13,450** experts across the model are
+cold (per the 09-14 build transcript, which verified "13,450 cold-expert source
+files"). The allocation is 75% text-REAP / 25% multimodal-salience weighted and
+is **not** modified by this campaign.
+
+Only the **75 MoE blocks, layers 3–77**, are touched. Layers 0–2 are dense and
+stay frozen. Everything except the cold experts is inherited unchanged:
+attention, backbone, shared experts, hot NVFP4 experts, BF16 MTP, and the
+vision components.
+
+Cold experts are quantized per (layer, projection). The two projections are the
+fused gate/up `w13` (N=4096, K=6144) and the down projection `w2` (N=6144,
+K=2048).
+
+---
+
+## 2. ARVQ representation and bit budget
+
+Each cold expert weight group of **8 contiguous columns** is represented as the
+sum of two codebook atoms times a scale:
+
+```
+w_group = global * block_scale * (c0[a] + c1[b])
+```
+
+- **Two codebooks** `c0`, `c1`, each **256 entries × 8 dims** (512 codewords per
+  pair). This is the "rvq256_256x8" format. Codebooks are **per-expert** in this
+  v3 checkpoint (`codebook_scope = expert`, packed format
+  `rvq256_256x8_expert`, version 3).
+- **Indices** `a`, `b` are one uint8 each per 8-weight group → **16 index bits
+  per group = 2.0 bits/weight**.
+- **Codebook atoms are constrained to the FP4 grid** `{0, ±0.5, ±1, ±1.5, ±2,
+  ±3, ±4, ±6}` (project-to-FP4 after every update). The FP4 grid on the atoms is
+  why no incoherence/Hadamard rotation is used: the target space cannot be
+  rescaled without leaving the grid (see `encoder.py` docstring).
+- **Block scales** are stored as **`float8_e4m3fn`**, one scale per **[row,
+  128-column block]** (48 blocks/row for `w13`, 16 blocks/row for `w2`). At use
+  time each block scale is `repeat_interleave(16)` to cover the 16 eight-wide
+  groups in a 128-column block. Adding the FP8 scale (8 bits / 128 weights =
+  0.0625 bit/weight) gives the reported **2.0625 bits/weight** plus the
+  amortized per-expert codebooks.
+- A single fp32 `global` scalar per (layer, projection) sits above the block
+  scales so the index/scale layout is unchanged from earlier versions.
+
+Packing (`pack.py`) writes MMA-fragment index tiles (64 uint32 words/tile) and
+verifies, per layer, that every expert's indices round-trip exactly and that a
+sample of decoded weights matches bit-for-bit before publishing.
+
+---
+
+## 3. Initial fit (Hessian-aware, no PV yet)
+
+Before any gradient tuning, each (layer, projection) gets an
+activation-Hessian-tuned initialization (`encoder.py`, `fit.py`):
+
+1. **Per-expert raw-basis Hessian.** `w13` uses `H = E[x xᵀ]` over routed
+   tokens; `w2` uses `H = E[m mᵀ]` where `m = silu(x·Wgᵀ)·(x·Wuᵀ)` is the BF16
+   SwiGLU intermediate. Experts with <32 routed tokens fall back to all captured
+   tokens. Activations are stored/computed in **float32**; Hessians are half.
+2. **Escalating-damp Cholesky** of `H⁻¹`: damping starts at `1e-3 · mean(diag H)`
+   and multiplies by 4× (up to 12 attempts) until the factorization succeeds.
+3. **Codebook EM** on a Hessian-importance-weighted subsample (20,000 groups/
+   expert) of normalized 8-dim groups: k-means warm start → 8 alternating
+   refine iterations, FP4-projecting the atoms after every M-step.
+4. **Per-expert LDLQ/GPTQ error-feedback column sweep** with the fixed
+   codebooks (col_block=128, 2 sweep passes, 1 inner refine): plain-L2 inner
+   assignment, off-diagonal Hessian carries cross-group error feedback.
+5. **Least-squares refit of the E4M3 block scales** given the chosen codes.
+
+An earlier decision, recorded 09-14: a **tuned Hadamard rotation was tested and
+rejected** — a matched 48-expert down-projection refit gave 0.372 (H128 rotated)
+versus **0.191 unrotated**, so the unrotated raw-basis fit was adopted.
+
+Nonfinite guards are raised as errors throughout (the fit/tuner refuse to
+proceed on NaN/Inf rather than silently scrubbing).
+
+---
+
+## 4. The PV objective: `--target reference`
+
+The cold experts are then output-tuned per layer. The objective for this
+checkpoint is **`--target reference`** (`sequential_pv_full_corpus.py`).
+
+For each captured token position the tuner forms:
+
+- `frozen` — the block output from everything that is **not** a cold expert on
+  the **student** input (retained tuned upstream layers → attention → shared
+  expert → hot NVFP4 experts). This is fixed.
+- `reference` — the **original FP8 reference block output**: the unchanged donor
+  backbone with the **original FP8 routed experts** (all 256), evaluated on the
+  **original reference trajectory** for that token position.
+- `required = reference − frozen` — the residual the cold experts must supply.
+
+The loss is the relative squared error of the cold-expert sum against `required`:
+
+```
+loss = || student_cold(x_student) − required ||²  /  (denominator · Σw)
+```
+
+where `denominator` is the per-row mean energy of `reference − frozen` over the
+corpus (so the scale is comparable across layers). Selection metric is
+`reference_rel = ||(frozen + cold) − reference|| / ||reference||`, measured over
+the **whole transformer block including the residual**, not the cold-experts sum
+alone.
+
+**Why reference (and its known weakness).** The transcripts weighed
+`reference` against a `same_input` target (match the block's output on the
+*student's own* drifted input). The rationale for reference (09-16): "Matching
+the original reference trajectory is more directly aligned with preserving the
+model's behavior." The acknowledged risk: because each layer receives a
+**different (student) input** than the original, the cold experts may lack the
+capacity to reproduce the reference output and could "learn corrections that
+work on training examples but fail on unseen ones" — i.e. reference-target is
+the more out-of-distribution objective. It was chosen only after a matched
+layer-4 A/B:
+
+| Layer-4 metric | reference vs same_input |
+|---|---|
+| validation reference error | **0.21% better** with reference |
+| development-audit error | 0.08% **worse** with reference |
+
+A small mixed difference; reference was retained for behavioral alignment. Two
+important design consequences: the target is the **complete block output**
+(including the residual), so matching only the MoE contribution cannot leave the
+incoming residual error untouched; and the loss compares the **routing-weighted
+sum of all cold experts** jointly (gate/up and down together).
+
+---
+
+## 5. Sequential per-layer pipeline with trajectory coupling
+
+Layers are processed **sequentially, 3 → 77** (`full_reference.py` /
+`sequential_capture.py`):
+
+1. **Capture** (8-way sequence-parallel). For layer L, the student input is the
+   **retained tuned output of block L-1** (`student_outputs.pt`), while the
+   reference target re-runs the **donor backbone + original FP8 experts** on the
+   original reference hidden state. Both trajectories are carried forward in a
+   rolling cache; layer 3 is the control (no preceding ARVQ layer, so student =
+   reference input). This cross-layer coupling means each layer is tuned against
+   the exact drifted inputs it will see at serving time.
+2. **Fit / PV tune** the cold experts (Section 6).
+3. **Gate** (Section 7).
+4. **Export + serialized replay** — decode to the packed format and require the
+   reloaded forward to match the tuned forward to <2e-6 before use.
+5. **Per-layer HF publish** — `publish_reference.py` re-assembles cold slots
+   across the 8 expert shards, checks slot coverage/allocation/global metadata,
+   runs the exact index round-trip and sampled decoded-weight parity, then
+   **atomically replaces that layer's two tensor files + reports in one commit**
+   with parent-commit protection; remote hashes are re-verified.
+
+Parallelism: cold experts are **sharded across 8 GPUs** (each rank owns distinct
+experts); routed outputs are summed with an all-reduce so a single coupled-output
+objective is optimized without a dense-model replica.
+
+---
+
+## 6. PV tuning numerics (the published full-corpus campaign)
+
+Per the model card, all 75 layers were replaced by a **full-corpus sequential
+PV campaign** drawing sequentially from **18,001,846 training tokens** at
+context **1024** (53,248 legacy tokens were removed around 17 exact matches to
+held-out prompts). Fixed **validation** and **development-audit** sets each hold
+**16,384 tokens**.
+
+- **Optimizer:** Adam. Two trainable parameter groups — FP4-constrained
+  per-expert **codebooks** and FP8-constrained per-block **scales**. (The
+  faster fork trains per-row scale deltas instead; the published full-corpus
+  fork trains the per-128-block log-scales.)
+- **Effective batch:** 262,144 tokens, accumulated as **four 65,536-token
+  microbatch passes** (256 whole 1024-token sequences), one shuffled
+  no-replacement pass over the corpus.
+- **Learning-rate schedule:** book/scale LR **0.048 / 0.032** through update 45,
+  then dropped by ×0.25 to **0.012 / 0.008** (`lr_decay_after=45`,
+  `lr_decay_factor=0.25`). (Note: these campaign LRs are higher than the
+  in-repo code defaults of 0.003/0.002, consistent with the much larger
+  262k-token batch.)
+- **Update budget:** layers 4–26 use a fixed **69 updates**. From layer 27, 69
+  is the maximum with an **adaptive early stop**: three validation checks more
+  than 0.1% worse than best trigger an earlier LR reduction; 15 updates without
+  improvement after that reduction permit stopping.
+- **Validation every 5 updates**, retaining the best checkpoint.
+- **Index reassignment every 20 updates** (Section 6.1).
+- **Regularization:** codebook drift-from-anchor penalty + scale drift penalty,
+  weighted 0.01; grad-norm clip 1.0; codebook atoms clamped to [-6, 6];
+  log-scales clamped to within ±0.35 of their initial value.
+- **Coverage gate:** experts with fewer than **256 routed training rows** are
+  frozen (gradient masked) and keep their initial books/scales.
+
+### 6.1 Output-gradient index reassignment
+
+Continuous Adam cannot move the discrete indices, so every 20 updates a discrete
+proposal pass runs (`gradient_indices.py`, method
+`expert_parallel_output_gradient_prefix_backtracking_v1`):
+
+1. Backprop the **output** loss to the reconstructed weight of one expert
+   projection.
+2. Propose alternative `(a,b)` index pairs near a small gradient step, capped at
+   `max_fraction=0.001` of groups, `trust_ratio=0.01`, `target_ratio=0.03`; the
+   current pair is always in the candidate set (changing nothing stays legal).
+3. **Prefix backtracking acceptance:** try the top-k proposals with
+   k∈{full, ¼, 1/16, 1}, accept only if the **training** SSE strictly drops
+   **and** a separate **check batch** SSE does not increase (beyond 1e-7 slack);
+   otherwise revert. Proposal deltas are broadcast sparsely (only routed rows).
+
+The transcripts flag this as the method's main theoretical weakness versus
+published PV-Tuning: codebooks are tuned for output accuracy while index
+proposals originally came from weight-reconstruction Hessians — the
+output-gradient proposal above was added to close that gap.
+
+### 6.2 Cold arithmetic emulation
+
+Both training and evaluation emulate the serving numerics (`activation.py`,
+pinned to serving revision `b1380cf7…`): **four FP4 activation planes**
+(`fp4_planes4_fp16_boundaries_v1`) and **FP16 SwiGLU boundaries** (FP32 GEMM,
+FP16 SiLU/product), with straight-through estimators for gradients. This
+emulates the quantization boundaries only — **native SM120 MMA accumulation and
+TP reduction are not bit-exact** and were not qualified.
+
+---
+
+## 7. Acceptance gates
+
+Publication of a layer requires all of:
+
+- **Validation non-regression:** `0 ≤ final_reference_rel ≤ initial·(1+1e-6)`.
+- **Development-audit non-regression:** the held-out audit split (never used for
+  updates or checkpoint selection) must satisfy
+  `audit_reference_rel ≤ initial_audit·(1+1e-6)`; otherwise the layer keeps its
+  initialization. There is no absolute error floor — the rule is purely
+  final ≤ initial.
+- **Serialized-replay parity:** decoded/reloaded forward matches the tuned
+  forward to <2e-6, and remote file hashes are re-verified after upload.
+
+The audit is honestly labeled **development data, not an untouched final test**;
+it is a split of the same calibration tokens the initialization already saw. Two
+notable per-layer decisions recorded in the card: **layer 30** retains its
+audit-qualified update-15 checkpoint after its validation-best update-20 failed
+the development audit; **layer 3** retains a separately qualified lower-LR
+refinement.
+
+---
+
+## 8. Calibration corpus
+
+The text corpus (`build_calib_v31.py` + `reasoning_slice.py`) is deterministic
+(seed 42), GLM-tokenized, with the following domain mix by tokens:
+
+| Share | Domain | Sources |
+|---|---|---|
+| ~32% | code | local vLLM sources, m-a-p/CodeFeedback, jtatman/python-code-500k |
+| ~20% | tool-calling / agentic | generated GLM chat-template sessions |
+| ~15% | reasoning | OpenR1-Math-220k + dolphin-r1 `<think>…</think>` → answer |
+| ~13% | instruction chat | tatsu-lab/alpaca + coding chat |
+| ~10% | medical | MedQA textbook continuation + medical Q&A |
+| ~10% | prose | vLLM docs markdown + databricks-dolly-15k |
+
+The **reasoning slice was added specifically** because the earlier calib-v3 mix
+starved the cold 2-bit experts of reasoning-termination behavior: 87% of its
+`<think>` blocks were empty. The slice emits multi-turn conversations where an
+**earlier** assistant turn carries a real `<think>…</think>` that terminates and
+hands off to an answer, followed by a trailing user turn, so the think-close
+token **`</think>` (id 154842)** renders in-stream rather than as a trailing
+generation prompt. Held-out prompts are explicitly excluded (last 25 vLLM docs,
+last 3 MedQA files, vLLM code beyond index 400 of the seed-42 shuffle), and
+17 exact-match prompts were purged from the training set.
+
+The capture/training code also supports **up-weighting boundary rows** (rows
+whose next token is a boundary such as `</think>` / `<|endoftext|>`) via a
+`row_weight` term whose sum normalizes the loss, and an optional matched-mixed
+validation set; the published card describes reasoning-based validation.
+
+---
+
+## 9. Results captured during the build
+
+All errors below are **held-out relative L2**, either over the whole transformer
+block (including residual) or over the cold-experts sum, as noted. They are
+local reconstruction errors, **not** token-accuracy or perplexity.
+
+- **Pilot smoke test** (layer 3, one step): full-block error 0.005497 → 0.005456
+  (~0.75%), 304/388 expert-projection proposals accepted; exported weights
+  reproduced the retained result.
+- **Layer 3** (200-step pilot): full-block reference error **0.005497 →
+  0.005271 (−4.12%)**, best at step 200.
+- **Layer 4** (200-step pilot): full-block reference error **0.015760 →
+  0.015596 (−1.05%)**.
+- **reference vs same_input** (layer 4, matched): reference 0.21% better on
+  validation, 0.08% worse on development audit.
+- **Faster fork parity:** the optimized fork (larger microbatch, specialized
+  embedding-lookup backward, sparse proposal broadcast) matched the reference
+  fork's validation and audit **exactly** on layers 3 and 4, with all propagated
+  BF16 outputs equal; training loops fell from ~1525 s → ~337 s (layer 3) and
+  ~1226 s → ~303 s (layer 4).
+- **Early-layer stability example** (layer 18, full-corpus): initial
+  0.017205, step-5 0.017176, step-69 0.017183 — differences ≤0.17%, treated as
+  a signal to watch LR/noise rather than proof of convergence.
+
+Older cold-only checkpoints reported ~0.2211→0.2070 (layer 3) and 0.2364→0.2320,
+but those measured **cold-expert output only on different captures** and are
+**not comparable** to the full-block errors above.
+
+**No comparison against the FP8 donor or an AQLM variant on perplexity / KLD /
+top-1 / wikitext was recorded** in the reviewed transcripts, and no full-model
+task evaluation was performed. Those remain open.
+
+---
+
+## 10. Honest limitations
+
+- Full-model quality and native **SM120** execution were **not** evaluated; the
+  arithmetic emulation covers FP4/FP16 boundaries only, not MMA/TP bit-exactness.
+- The development audit is a split of calibration data, not an independent test.
+- The reference target is the more OOD objective; its generalization advantage
+  over same_input was small and mixed on the one matched layer tested.
+- Gates enforce local non-regression, not any absolute quality bar.
+
+*Prepared from the GLM-5.3 ARVQ build transcripts (2026-09-14 and 2026-09-16)
+and the `btx53/arvq88` fitting/serving code. Where a number could not be sourced
+it is stated as unknown rather than estimated.*

reproduce/historical_metadata/build_provenance.json

@@ -0,0 +1,178 @@
+{
+  "stage": "per-expert initial fit; alternating PV in progress",
+  "allocation": {
+    "mode": "ARVQ output-benefit allocation",
+    "text_weight": 0.75,
+    "mm_weight": 0.25,
+    "hot_count": 5750,
+    "floor": 8,
+    "cap": 176,
+    "text_partition": "train only",
+    "metric": "positive routing-weighted squared-error reduction from retaining NVFP4 over fitted ARVQ",
+    "scores_sha256": {
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_3.npz": "99e689edad7a7f026759c9c66e682b499ca3263e6d19ad54f59c1f1dccdb3ea9",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_4.npz": "7813ddafdb23f7c4802bf431a2d063f7afc9f3463356079bc97ef8289e30d823",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_5.npz": "551940cef83bd52f2e032bd738a88fe293e74efc3e918cfdabf025e111c0d586",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_6.npz": "1949e2540a0d92169ae9e5374789443d026b0868860149e785342eee19b521b5",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_7.npz": "cb1d4f6597057c32d2d8136700ea357b86a12ccb881d4bb4a613033c6ea0a34e",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_8.npz": "c2d4cfb7641a3a9fe96abffa76ab59774eb37cc629b058b3f9bebb1f98bbb09e",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_9.npz": "c82bff242306c5b27acac0bdd8530e9ad603c1e00642855face4a9aefd03f97a",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_10.npz": "59fcb8922894f0155704577296cd8752323d1f880a51e801a5b5b1c969647aa0",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_11.npz": "45b94e85267b83b94203eddca795a21de68fb359a55dd0e304082597a8cb2cee",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_12.npz": "a277261ee1c2b60dd45de874e0e7e4819356480af132fcb23dfff411a2775077",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_13.npz": "911c1ba89ea9807704759cad0522532127329259b19ba3ade47ce95c1a5c4e49",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_14.npz": "e672d24e525c3c077ff2a9a2617b6435e0d3d97398cd2c901f5298bd4a0b7197",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_15.npz": "e163205f5eb51d0c8796df2070fde3bf80caf3bdf166311336669f53a695c437",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_16.npz": "9cda2a0e36f882a5daa485c698b9d1eb5ff3aba7ffeac81f7706bed09214f6c6",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_17.npz": "0f085320e0e1f09bc52183d619a8d91535bdb33b5d56c3de05f212b4337bc2a7",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_18.npz": "627d542b236da70a5dd2ed098d21cee75dc4cb38db6105369f947361dbe7448a",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_19.npz": "3613a6fa1e852c110ee05ad901a4c89e1ba659c4782b88988bc935a3a69406bf",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_20.npz": "7be6a86b425bb811436bdf0f1e4a9a55a0d64aa8a2a7df163923a1898e88e3d9",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_21.npz": "4abd00a72ff0e76af4cf708caace639e945bce378168ee47892db2645af0de81",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_22.npz": "92deb3e93534084dd0f43a0f3ab6a11bde3156b5a6218128f03ad58c31e5da08",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_23.npz": "6866eee649260c556ad663a9f2b198109d1a3aa65dff69a26ef9a33ebfe90b17",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_24.npz": "9d2a097d7039b8016e50fe48ef7db9175bd6af7652fb8f02d2845e0608c085c8",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_25.npz": "9675f20e29ebc295606eff124e7eea8e512123d3a8ec37774fac5db06797a5c6",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_26.npz": "db25e41b5b130d39bcb6d508f719eaa76164b71813fa2bbdba61a249653f9fe3",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_27.npz": "07d874eae8001ca5fc0e976ba7199afa6e83f54925606b0a8e2ea3836920dbcd",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_28.npz": "b83eafbc8d8cf70d62c805c37cc3b6b6a2554f2236cd858a84d44696a45a526b",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_29.npz": "084317cbf70cdb24996a55bda7423b32eb2ee807209d73a0baba7fcad1307cd7",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_30.npz": "68a6b57cae5acdb0d01fd498ff324ae4e3b01a3c9e3121d7814b1d93fc53cb78",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_31.npz": "b2f421260f518195814fd1f42b10dbf59060dfd08ad726e1afd0de2f6b5f7442",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_32.npz": "88f73b9073cf9b4532aac041fad881f9df4392fea7b367101e361e643183a01a",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_33.npz": "cce07637bfafcc225057a4c7e3c304c167965ca28b551e7da0f0d13f41b00505",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_34.npz": "0251a6abee66ab96080a23ff1b545ad933841525a8782d3de3dac5b472a681bb",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_35.npz": "1dc1d919c3fb1de2ae8ec4765e5576e53de6049d43608d3f2e2c5eb687b01b0b",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_36.npz": "43815d9f766ec63f51268edd90433bf6331327766888a1a089cf7bf31c9a28b6",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_37.npz": "dc7a1f5b296bc6105a8ca969f558e2f1261238889abfc73326b50ef83b2c0216",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_38.npz": "2fb9251355a449817aef26486305cc41edace820a91d6113a2f3a0b8f05258a8",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_39.npz": "f69d1d2da6fd0b3a277e0d3810be2eff1e95050742e26013a30226cb2ba86b50",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_40.npz": "c59aeceb845ee137ebf890bf5789d181198272838d1f794bf2c96ae31502e940",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_41.npz": "d962c915b2b672f58bd5f1bfa65e806f09dd5a842a5986c1ca4ac03422620130",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_42.npz": "acbdc6f83401fac52ce2a5a5e4d174be609842ffba5f1a3e219e9faae913c55a",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_43.npz": "7ffa833b873294a9f6f1f005644b1101fda459b10ecdb5375eb4c85f7cf6ad5b",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_44.npz": "906e81150c5f462b68750f74966df5bb93527a30f3b2cb0ecc410da5c9859840",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_45.npz": "df9f6938632d32af48ceef8013694d0a92bd7208f4335fb06dc74ee2cd38ba12",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_46.npz": "9fd39f469414d71da762d10b7671e5e0bdcbf087808804e2c868b73193702065",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_47.npz": "9d2eb792b6553fa078c5cecac1ac53db461e5f2b7bc3b971246f6b8bc7846477",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_48.npz": "49b108964ddb44d54b4d2375e82772d14275e94ab43da17855953717367ff583",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_49.npz": "6775fdc746a2b11a443645140a86d1a11dcb389df71289fbef3fe300dec787e2",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_50.npz": "f6d604b98a1c161b023f42205abf6d1eeecc9119871a8e920623908af0784fc9",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_51.npz": "0aac9c4e66efaf5c0924107749e82a1e19e70a0b8e31db46434acf38353b8312",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_52.npz": "7601958d27deb50c0bbf21a6ebb79c20859ca08c962b6f751f972b0f3453bbe6",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_53.npz": "7bf959678dfc05ec4ef9f7b063d74a3de80e3cf757a199ecf083988ca55d41b7",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_54.npz": "53097dbffee6e800d22551ed5ea068fe5bae3bad836813e2288f0667612bb6e6",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_55.npz": "1ec4c9979cc55150e3e3163f8bf52f5d4042e351c15429bd11acc8bc7a77d273",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_56.npz": "1675552ccfc8929bd29d7eed04e18e387222858beadf170e9652e17b27966766",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_57.npz": "7e7029ec0de486aad04720d1059b69d74358c59432a33aa16e526041f4506609",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_58.npz": "813756873a97c71e40fdff33313809723d599dbef2b2ebf36af0d56118b4e62d",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_59.npz": "fcaadf941258b398bd6bf6a96f7865fccfe9008bf4ac1b93df8896065f761383",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_60.npz": "e73c14eb000ec4c2fdadb4078011f12ab44bfd470bf690c22c3e0ffdbe89d11a",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_61.npz": "a4c10b888ee7879054c08e4de97540e786f6c44c8823f3b7988dfb0fbe1b8927",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_62.npz": "b73347956fed5d783082fe8dd76027372ec50db12683447b180e9a053f9abe61",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_63.npz": "fe9211a06376cee25e813131672c6101c4564a7c27b5820eafe54b3efa0623e1",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_64.npz": "348ecb20769be5b1bec0cb31157f513b92f911ef36a8439bef7d45b31321f4d9",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_65.npz": "2d37ba213e5e4c5e598aedb928c7f50f16205665253d9fc04cad569aa01384db",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_66.npz": "28615fbdc3c2f0b2458d38a52f05e041dd02501b3d18448ba449b4626a334cd6",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_67.npz": "9e5a6100b7acb907c99f80ce1cf89f2e46e392c7ca48bb000eda2c126ae1413a",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_68.npz": "dc0fdd42f3332b73508943c6f9ed0c4166ca0065c2b444823eccbd37a451be6f",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_69.npz": "4a2b3139ba9f6cd9327285f6ca2417eeacbca49c6e96d8dc525f80e1a7d6eed0",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_70.npz": "6c03c2284504c6b89cd7ea6af120b6236691dee58ecb2ab1680be19e379068e0",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_71.npz": "c51e64b3d7d9347bb8fa91a6f4b74631983b7ded564f291b44ba496ff198622c",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_72.npz": "298c23118c8377d56d7a025cfc59ca782246c1269f0a0b47cdc44f8f53778dc6",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_73.npz": "0e2aaa5468da7aa0f48a0007eb7fdbed01abe6fdcca5f62ccc8c0d0febccd582",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_74.npz": "7c33c7cb1a136dd9f438ab36b83007554227ecbb2488fdf883d545d1c680cc94",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_75.npz": "0c71e68caa217700531d8611731f67fb5842e53df15778bbb480472028d564f6",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_76.npz": "9dc6ec2ca2dd15912b5505a62ea3b296be8a174bc3c69eeeaa0d6ba316dcaf67",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/text/layer_77.npz": "a2c9a43c8566867ede5e47f37b72c2623adc075edb44dd43508037d9a137bd06",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_3.npz": "2e102e3892277e8808818ffa8373389b9e02e3ce3e1b280c8511dbf0e47353be",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_4.npz": "c007140d4edbd361da7e15c6bd74bb49e10786ccddc49f0c343592c3939d5519",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_5.npz": "99e460fca675b7c78122c054e9293b5bfc745d43a4cd57625c4b53dd6fb7dc09",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_6.npz": "f1d5673cb87462244ddf87c79feee9f38887bcf3443092335b3b38c7ee350e0d",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_7.npz": "b7e29194ef43c192fedd848312f2fd33687075be7271fbd5e8e882c600a5e4b7",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_8.npz": "25b9e46d9c422178b825f2d20cf23a1adb09470dea7454e6b70000b927b58f98",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_9.npz": "3757ebe69d33ab85557c141c3a37e5c800fe2c2ee2c966d2cdef3a6e689a1420",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_10.npz": "07ab61f067a9fff3d4a1c9bb5039c745417d5bf305d824be85ddd6b114b9829f",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_11.npz": "269c82abbd5c60440a169962d051b285670e86beef6f60f84a23ddb260e2419f",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_12.npz": "0e2326f560e5e0b3b7441d8e6584720fefc42d6209bbd0ffc5c4af3679b5f542",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_13.npz": "4e8c4f5e081cbe193d0468ad98842f4904ddceef140d61363eadce54bb9d8ce9",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_14.npz": "16ccc9716e778239850f0d8c7f827ab3eefbd4673b775d1bdd17673141550973",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_15.npz": "199256730bd069357ace4ff4a9528f4a4a2b316e5d299071d2b8751a05a0f026",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_16.npz": "4631ef874d195778a08af4f0d03a2e6a6a2d1ed6c830b7016c758a0687a1a235",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_17.npz": "66fbd376505e0f6f921e3f483253f8e733378f8df909733c8fc65194b6eeb092",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_18.npz": "ee3134b30d8287684cdf95d1eb44da997b90c3ef4c681ac998a770109b35adbf",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_19.npz": "611b57f18f927ae09479a29d77425fafba33514d0f15ff69dae366733c91e1d3",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_20.npz": "b3d431eae9b9c3aa5a36c60975378bbec4112377cc858b5ccf0d0946778a6079",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_21.npz": "cefc699b376b5b43c14769884cc481c22f4a284e168c1fa88feeb9614588f505",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_22.npz": "7966b47efd7a838d8f87c2f5697320648c6f0bcdd5ffe7762cef03ff2ffbfa95",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_23.npz": "9b55935db844c788fc59417a1c053722fd5685ad6ec4060a192a0a1fc1582e4f",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_24.npz": "65c1134ad6a7a38dabea537c0b19899126aa8de849306c1385c14a3c09a22d37",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_25.npz": "038eab98b035a9ee810e1c7bb08ad89d227bcb87012bf88164d3dea0ad3bc1cd",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_26.npz": "8e31e623cf17a166d7ee056ba20b5c6f19a693e039c9910a851c0dc3a9cc983b",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_27.npz": "1306ea155b6746f73a4463cb8be7474bb7f80b3870daead9930e37d0a8fa12cf",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_28.npz": "3cd4ce023f5f11199b423dda6484bbb7377617b086e2e0bf5dc57afd29e7f566",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_29.npz": "c7ee73af65346bc85e3c908825198faf4acc4987778f204df0821f4b642c7990",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_30.npz": "81607884d312e7062274e230fec71dffc2a8a9f073ad8479d3ba19ea3df66899",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_31.npz": "514aa850fc3099da75b19b74be73e944d7b58cb3577b4a221f58bef4169d9f75",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_32.npz": "63b2e2a7e689a5efe641536527594ce26be5c807a32acc09a81aa6709150ccdb",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_33.npz": "acdf16dc4552aef9184c5b687a9b7e0a4ea5edda0fa9ae496eb5f4f6bec12db1",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_34.npz": "9c70b6174cf4a7a02a6300b14bd730fc51f6d1520dff274b8a3d6588368624e8",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_35.npz": "25a82c26d0da013eb14be7d9b9f1811eeb9768fa45687e4a71c96c9b9752d9e6",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_36.npz": "aeae4e169f803c42af7030b1395e1db2459fffaf363d4729075302df46d07f26",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_37.npz": "c3787ea553f8bc3d239269f7e329e6f39d02b4ec63d06022405286dc640fbf2e",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_38.npz": "1a486331012e24ff643d0dfec19f2d71bbf29e5de7df3706d7e16da33541ffca",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_39.npz": "811e7c07267a7c80dbbcd51462fa01bae403341761ee9d89fce1ec67d32eec31",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_40.npz": "712935236730a5e4142b5cf4da8dca2ba78416a358d79001b2787ee2a0f5b552",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_41.npz": "e5fcd78bedc942ba734c28e16127c63fb02280ee279a8aee70143e2ec6d16c6e",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_42.npz": "324b4909e5317f0217290669a78b1ef6f21db35665f13dbac7a78650a166a9b5",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_43.npz": "b990182a89192ae58b4cfc82b3dd530bdfefbea01dd517a8d361c28c47980370",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_44.npz": "cdddf30edd971309d21935c342fc353d0fe7b7a45a12c404cf8b7ed5d7a7dece",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_45.npz": "525ef223869db6a843bca47df8fec0d8cf7f08f21d3626f8af03f4a33f0e896a",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_46.npz": "8c4001f106b27f2646c61809d701a29245ee9997b0e291a7487f091e2651a7e3",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_47.npz": "ce82107d4c1575654f41c9bc0bb070aade6d316c5a3bfd73bf84a75e9713fdeb",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_48.npz": "d0a12442d24126677dd652b204f6910aff45d31ed76d368ce0efaff95943069f",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_49.npz": "1566580cf72cf4ffc51c7cf5d9238bb09e92236512e92a7ae4a52fd887d5ac2e",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_50.npz": "55be42b2dee8fecee914584711e405a48bd1be061aee8f8840ef43ed9a8ffe5b",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_51.npz": "eca0da0b8d7522bc39c587cbd4c3f2ee88e7fc7f27370ff1d1b57c2e96d45565",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_52.npz": "53491b6e515ddd86a49a3c4b5dcd5d00cd9687a0eabc595248a0cd28a85b2c3f",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_53.npz": "561e99733a55b073842560ee9847b6b7dcd798d7bc769f3504ee373613708877",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_54.npz": "3d0d33114524ff3a30e83b4d4f5c80aa44b031fd0e4da7fc776eb63fe89ff322",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_55.npz": "3166fea4ab9810e08c4c05e5960c3fb338be355a0b9a583d9dd2a2e121c64982",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_56.npz": "f4123b5dd6c2f722d029139b6f82919ee3c34de8d67326d819c8a6ca65931ff3",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_57.npz": "e32c77a50be665998bd0682742535bd10b6c2478894f42779d141581b09788c0",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_58.npz": "69430a65fc86a0166ccc9193746aa61fb4682ae65983944f8304907b34509bbd",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_59.npz": "c35ebeaf949917077fb132e219558381c9638ef4de3819b95f289918cc4f9c3f",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_60.npz": "6dc83319d29beb864c6f36d92a11ec898870e6fdf173917f0c9e73f6b18cac5c",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_61.npz": "69c7b7440c70f7c4383000d1a798b5ef7eb9d3d9a7c6f28c49bd124f484f5fb5",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_62.npz": "3bf69359b4ad241cb4e6dfdc00205fd637e1f7f959e35ec40eca480f6c41f80b",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_63.npz": "280ad144e82b822b7ebf60af0d367dfcd261531bbccebd5791152aa75ddcf237",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_64.npz": "1303b6bb6d7fad77472424cbc3696cb3368cc9e8e3f442dba0f362885b283a49",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_65.npz": "f396c0ed74ed729685ff8681b2d2029acde74c28db559386b4cf15fff7cb8769",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_66.npz": "b4d92dc502dc505d873f079044a09b8c2c0c310e95bdb2602a8cd572603c9953",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_67.npz": "aa4765dd3124747d07ea1e06d353c12bdf2cf3f67777af71556c81d51e047a48",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_68.npz": "f8df75fb71d9459a84789008ce0b0dc5ee8f0cafce7dad057e6fd6b544da9131",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_69.npz": "339f59f847d5d98d52dd986eada9092a04036f82c4e570af4db66b7c51eb2000",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_70.npz": "1156e221583b3435eb14c4116ef2629e89f84a8a0979253d88556d57424afec0",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_71.npz": "473f3146f0ba8dbd340b44665954354d4d8fac6b5acb9ebcc952cf3f298b345c",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_72.npz": "58179fe2da6eed5b63eac4710ef8bc54ceff4ff9ba2d8108942643585d21b744",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_73.npz": "73fcb0af07e5a5accaa1d165446302cf22c595ffacd45015a460524d8e9cd9b1",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_74.npz": "6e50287f298b566b96d44fe9961ebc7c9f0010c81b035fc4ff7456833ba86b46",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_75.npz": "1a219f6efabaf773a6268b08f82716dc6177d257c728f70f81b0694d01d65cdf",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_76.npz": "5aade66c63087e535fcd2c52bd6d8e075f439185a1745a8c8644bd4133a02eaf",
+      "/tmp/glm53-vision-expert-full/arvq_reap/scores/mm/layer_77.npz": "88e5ec9012bd07e07ef84d675db34d5fccfc45361ecf0ccbef2aca0a2a7e8bda"
+    },
+    "limitation": "per-expert additive error proxy; excludes cross-expert covariance and end-to-end model quality",
+    "candidate_codebook_scope": "expert"
+  },
+  "hub_revisions": {
+    "zai-org/GLM-5.3": "aca966e4e02791568aa6a4ced368624b3d897f42",
+    "RadixArk/GLM-5.3-NVFP4": "11af4cba759e6559eda70358a5778bd1bddddd78",
+    "jarrelscy/GLM-5.3-Vision-NVFP4-AQLM-hybrid-1m": "2b883d28bb9dd13a9511e2bd45a8ad1cbacbad74"
+  },
+  "cold_source": "zai-org/GLM-5.3",
+  "source_nonexpert_mtp": "nvfp4_donor",
+  "source_vision": "base_hybrid",
+  "initial_fit_partition": "train only",
+  "pv_reassign_every": 40,
+  "format": "rvq256_256x8_expert"
+}

reproduce/historical_metadata/calibration_corpus.json

@@ -0,0 +1,24 @@
+{
+  "source": "/tmp/glm53-unc-traces",
+  "reasoning_rollouts": true,
+  "counts": {
+    "train": {
+      "problems": 856,
+      "tokens": 3048596
+    },
+    "validation": {
+      "problems": 89,
+      "tokens": 345875
+    },
+    "audit": {
+      "problems": 104,
+      "tokens": 322277
+    }
+  },
+  "partitions": "disjoint prompt hashes; separately captured",
+  "limitations": "Text-only captures in 2048-token windows; source datasets overlap earlier calibration. Teacher answers are not correctness-verified.",
+  "initial_fit_partition": "train only",
+  "capture_teacher": "RadixArk/GLM-5.3-NVFP4",
+  "rollout_teacher": "unc NVFP4 teacher",
+  "same_token_corpus_as": "/tmp/glm53-kernel-aware-pv/full"
+}

reproduce/historical_metadata/pv_progress.json

@@ -0,0 +1,187 @@
+{
+  "layers_pv_complete": [
+    3,
+    4,
+    5,
+    6,
+    7,
+    8,
+    9,
+    10,
+    11,
+    12,
+    13,
+    14,
+    15,
+    16,
+    17,
+    18,
+    19,
+    20,
+    21,
+    22,
+    23,
+    24,
+    25,
+    26,
+    27,
+    28,
+    29,
+    30,
+    31,
+    32,
+    33,
+    34,
+    35,
+    36,
+    37,
+    38,
+    39,
+    40,
+    41,
+    42,
+    43,
+    44,
+    45,
+    46,
+    47,
+    48,
+    49,
+    50,
+    51,
+    52,
+    53,
+    54,
+    55,
+    56,
+    57,
+    58,
+    59,
+    60,
+    61,
+    62,
+    63,
+    64,
+    65,
+    66,
+    67,
+    68,
+    69,
+    70,
+    71,
+    72,
+    73,
+    74,
+    75,
+    76,
+    77
+  ],
+  "total_layers": 75,
+  "recipe": "full_corpus_sequential_lr_decay",
+  "training_tokens": 18001846,
+  "batch_tokens": 262144,
+  "microbatch_tokens": 65536,
+  "max_updates_per_layer": 69,
+  "adaptive_schedule": {
+    "from_layer": 27,
+    "relative_worsening": 0.001,
+    "checks": 3,
+    "patience_updates": 15,
+    "max_updates": 69,
+    "selection": "existing fixed reasoning validation; retain best checkpoint",
+    "description": "Drop LR4x after three consecutive validation checks >0.1% worse than best, or after45 at latest. After reduction stop following15 updates without a new best."
+  },
+  "lr_decay_after": 45,
+  "lr_decay_factor": 0.25,
+  "layer3_exception": "45 high-LR updates plus retained lower-LR refinement checkpoint",
+  "remaining_layers": "retain their previously published weights",
+  "previous_campaign_progress": {
+    "layers_pv_complete": [
+      3,
+      4,
+      5,
+      6,
+      7,
+      8
+    ],
+    "total_layers": 75,
+    "recipe": "sequential_reference_gradient_pv_v1",
+    "legacy_pv_layers": [
+      9,
+      10,
+      11,
+      12,
+      13,
+      14
+    ],
+    "initial_fit_layers": [
+      15,
+      16,
+      17,
+      18,
+      19,
+      20,
+      21,
+      22,
+      23,
+      24,
+      25,
+      26,
+      27,
+      28,
+      29,
+      30,
+      31,
+      32,
+      33,
+      34,
+      35,
+      36,
+      37,
+      38,
+      39,
+      40,
+      41,
+      42,
+      43,
+      44,
+      45,
+      46,
+      47,
+      48,
+      49,
+      50,
+      51,
+      52,
+      53,
+      54,
+      55,
+      56,
+      57,
+      58,
+      59,
+      60,
+      61,
+      62,
+      63,
+      64,
+      65,
+      66,
+      67,
+      68,
+      69,
+      70,
+      71,
+      72,
+      73,
+      74,
+      75,
+      76,
+      77
+    ],
+    "format": "rvq256_256x8_expert",
+    "full_model_quality": "pending"
+  },
+  "format": "rvq256_256x8_expert",
+  "full_model_quality": "pending"
+}

reproduce/historical_metadata/source.json

@@ -0,0 +1,14 @@
+{
+  "repo": "zai-org/GLM-5.3",
+  "revision": "aca966e4e02791568aa6a4ced368624b3d897f42",
+  "meta": {
+    "kind": "block_fp8",
+    "block": [
+      128,
+      128
+    ],
+    "fmt": "e4m3"
+  },
+  "cache": "/tmp/glm53-fp8-cold",
+  "legacy_cache_revision_unverified": true
+}

reproduce/layer_recipe_summary.json

@@ -0,0 +1,1727 @@
+[
+  {
+    "layer": 3,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 15007754,
+    "training_passes": 1.0,
+    "best_step": 50,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.004300016159961927,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 50,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 4,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.011536361290655902,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 5,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.017510809110214375,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 6,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.018967027905769453,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 7,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.02562102057711342,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 8,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 13107200,
+    "training_passes": 0.8733618634740414,
+    "best_step": 40,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.022436852159198863,
+      "drop_after": 25,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 0,
+      "last_best": 40,
+      "early_drop": false
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 9,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 15007754,
+    "training_passes": 1.0,
+    "best_step": 58,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.0017143418153641949,
+      "drop_after": 25,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 0,
+      "last_best": 58,
+      "early_drop": false
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 10,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.0013590989109528494,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 11,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.0013980529941903324,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 12,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.001638401368651591,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 13,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.00271562211069347,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 14,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.0030091782715398136,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 15,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.003168485440774852,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 16,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.004101365709508885,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 17,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.003969997484537619,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 18,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.00536306292142981,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 19,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.005900326847720345,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 20,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.00827687275281147,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 21,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.007670614413287488,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 22,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.009540958133708182,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 23,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.011060020056539258,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 24,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.009994059305569054,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 25,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.010544123217574992,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 26,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.012252411564001522,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 27,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.017041586186703876,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 28,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 9175040,
+    "training_passes": 0.611353304431829,
+    "best_step": 25,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.018901359302169598,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 25,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 29,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.01730259021141979,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 30,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 9175040,
+    "training_passes": 0.611353304431829,
+    "best_step": 25,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.01953328825434068,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 25,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 31,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.0185387897556427,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 32,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.02538125707815895,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 33,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.02889075275987553,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 34,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.03261215982052183,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 35,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.02849563835939984,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 36,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.033030121256077974,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 37,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.035430789227037796,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 38,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.03938840653001622,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 39,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.04198908163931699,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 40,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.040921155033096596,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 41,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.04285828934035643,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 42,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.04590353936917018,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 43,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.0445766345634676,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 44,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.04553051376240955,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 45,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.04488142473736064,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 46,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.053625666510220674,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 47,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.05097487104130676,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 48,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.05253228264238492,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 49,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.054531234896217015,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 50,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.05778833536756532,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 51,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.046839461506473105,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 52,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.05742379963086141,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 53,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.05401293698903637,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 54,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.05698656719338924,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 55,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.05936456247157864,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 56,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.0527071627420072,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 57,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.04895510529481134,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 58,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.05838005265713665,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 59,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.06076521669703373,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 60,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.05275984301503296,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 61,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.05175780703646056,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 62,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.05643135330733986,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 63,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.0563437113825566,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 64,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.048833171244225114,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 65,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.060005061741192876,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 66,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 1310720,
+    "training_passes": 0.08733618634740414,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.05761269836156414,
+      "drop_after": 25,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 0,
+      "last_best": 5,
+      "early_drop": false
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 67,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.05480543365221943,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 68,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.062059367848489574,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 69,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 10485760,
+    "training_passes": 0.6986894907792331,
+    "best_step": 30,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.056006381621796365,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 30,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 70,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.0733148360257097,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 71,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.06478566802319799,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 72,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.05970967496138404,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 73,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.0664773336538204,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 74,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.05450847628729126,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 75,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 9175040,
+    "training_passes": 0.611353304431829,
+    "best_step": 25,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.04963358343974084,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 25,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 76,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 7864320,
+    "training_passes": 0.5240171180844249,
+    "best_step": 5,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.05268687462948389,
+      "drop_after": 20,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 3,
+      "last_best": 5,
+      "early_drop": true
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  },
+  {
+    "layer": 77,
+    "target": "same_input",
+    "block_scale_dtype": "fp16",
+    "training_sequence_count": 14657,
+    "training_tokens_seen": 14417920,
+    "training_passes": 0.9606980498214457,
+    "best_step": 45,
+    "codebook_lr": 0.048,
+    "scale_lr": 0.032,
+    "lr_decay_after": 25,
+    "adaptive_schedule": {
+      "best": 0.03900097511967364,
+      "drop_after": 25,
+      "relative_worsening": 0.001,
+      "checks": 3,
+      "patience_updates": 10,
+      "bad_checks": 0,
+      "last_best": 45,
+      "early_drop": false
+    },
+    "checkpoint_selection": "held-out original reference trajectory for both arms"
+  }
+]

reproduce/package_manifest.json

@@ -0,0 +1,247 @@
+{
+  "files": {
+    "arvq88_reference.py": "b0606bf8ae91bab655ed5b26bb9b7437f0e28718301fdb1b86f475b1b31924e1",
+    "METHODOLOGY.md": "45ad59ea38585133a8005e169898a4dcd7ae56b9a4ce8ff3ba7e9f1ca00e9253",
+    "pv_progress.json": "583c9ebe0f26dfd4ffee434f1a7abc66af61a149b4f818ab5ccc4d5d78e7e3a9",
+    "README.md": "6d219086224a5c4fdc8085d04bae3292747673408d52275fa7ce17561c0a9c4b",
+    "calibration_corpus.json": "43517a1e4a3580b4da7c628019d0ad5d0a610aa469cfddcafc05f86bff47a770",
+    "reproduce/source_manifest.json": "eadfaac3a4552c341d64293b25f702b66358ba29450bbdba0e9b80f473ac9819",
+    "reproduce/environment_observed.json": "e9ae2da2456329999caf5ff468067e3c47caabd754bd80ae27b74f3c6c6f16e3",
+    "reproduce/requirements.in": "3f58e110a3d1816aef49af4fe3e44e2ff36e4bffd8c3187a099d556ef690d671",
+    "reproduce/data_artifacts.json": "bf1e037960df3435a829526d7145149036553f6e2ebe514d35c308a37e9a4126",
+    "reproduce/checkpoint_before_audit.json": "a8822199bb0af460349ba8c930e7abdbdf23096ba68276f429068c94b325fcce",
+    "reproduce/layer_recipe_summary.json": "1bdb40cd40eb3a80e9b0fc4e7b162be0143cd42cfbdcc592170e24e9f918a68e",
+    "reproduce/recipe.json": "4a6e876f0320909178001af53b086063a1951191e2ddfc26e7c696c76cdbc524",
+    "reproduce/README.md": "007935e4deb8d402f7d43bcd2dc7e2d2d27cf78dfde39fad35a58b4eae96126e",
+    "reproduce/RUNBOOK.md": "d13c303ef1c80eba3bc0aea0df43a263b042a5cd3fd76500e88432fe561b6407",
+    "reproduce/prepare_workspace.py": "cc679deae776de7beacece214c4868e51e35bc4a61a7a2725a4d05d1bb8324e5",
+    "reproduce/verify.py": "418df7c6dc54de146f0ff465618d42e69f8489eecc7743abc9464f8e15fa96e2",
+    "reproduce/test_roundtrip.py": "fdce4358d869c1fb5fe01a7a9a878d6a29710af65779bbf0637be52eefd84735",
+    "reproduce/external_sources.json": "42d948405368409e11c646a313f055e417271f4a828a907743d86a3094bba82a",
+    "reproduce/THIRD_PARTY.md": "b26dc2e38b00f1b7e17881535567ac8e811725abbdc709342f67f2eb08b93ac5",
+    "reproduce/tensor_header_sample.json": "7f804b22fa108706614d8fb7d12302eae7e4c537c746299c7ca00dc5670f4916",
+    "reproduce/AUDIT.md": "ae0e0e3447269352176307811ed4d07c4b9b91525f4d06724e9840cce3af1c8b",
+    "reproduce/historical_metadata/source.json": "c797ff4755b4ef9bf4b1fe8143ea7cfd55e6b52383fe11debaf4a23bf4cb1ffc",
+    "reproduce/historical_metadata/backbone_sources.json": "b9fc2ca4ed0da88b4061aff2d6cf53d59afae80a1d291a1ac8952f9a2063cd51",
+    "reproduce/historical_metadata/cold_assignment.json": "2bc853d8c282e2e7b7a3cb6fe390bda59d912d046063c6ee235cfa551014f695",
+    "reproduce/historical_metadata/calibration_corpus.json": "5c3e0f16c03632cf987d4e6cc4af731dd51abd7f12e18366285dbe0766150743",
+    "reproduce/historical_metadata/build_provenance.json": "d137642397e2c8cab6da61f1a7975453b5bf9d010ba1217a2d2b683d9cf75dbd",
+    "reproduce/historical_metadata/pv_progress.json": "ebec214f80ea0b16365f1aa819f6723a9461f5bf14a80f977f705025140ac6bc",
+    "reproduce/historical_metadata/METHODOLOGY.md": "43591534ca9a1e8db9015fd05368177a37efb1400b5d2ed699e2844e17485805",
+    "reproduce/source/tools/gguf_remote.py": "8befc8a77f2e7964c9bca0c5732a919b58dc5598fe33b00d4708bf0d8547232a",
+    "reproduce/source/tools/budget.py": "562dbf3a01822eb500f92d46f9710c987ba48bd2faf4fe1f0a12474ed1cf08cb",
+    "reproduce/source/tools/make_manifests.py": "b1c92a95a3e49bc5d27207e915a402da6a8713facbae1e09c00b5d1a932a4740",
+    "reproduce/source/tools/range_download.py": "6b8462c4af6372b5899bdae608537a76e5a470fc4938bc3d67ea5de7f16e9581",
+    "reproduce/source/tools/validate_serve.py": "e328e06a20246d58b0225ddb0d6cc00fa7450623c7ebb464da41cb58f29c8ea6",
+    "reproduce/source/tools/build_checkpoint.py": "af00a9e5c1a1b96c302bf7ea7eb7f5f9b5f32060de2c186085067b2bd5868ee5",
+    "reproduce/source/tools/patch_checkpoint.py": "cb5bdf47be506c0293b96055b83cf38f0ca807414aa0b9c640c16c2186246a3b",
+    "reproduce/source/tools/strip_stale.py": "3684269f45eeb2ea1b6698c80774a4a74cf616dea4832004be884ef1d5483b8e",
+    "reproduce/source/tools/collect_expert_stats.py": "979d22afbaf18e920a576857e6aedadbb74a120fdea593f2609a2e813342d16f",
+    "reproduce/source/tools/build_checkpoint_v3.py": "e0dc5a53b242a4a7d54ba7412798792bfc75f95ccd31632f7e589949a27500c2",
+    "reproduce/source/tools/solve_assignment.py": "f95d133a3dd96a797bec1a589e49e405056ffe9a525174994b95ec0a9059bba2",
+    "reproduce/source/tools/collect_expert_stats_v2.py": "7837bca536acd8513b2a7f6f127460b2e490bb615eabcc81fa06675d4915e1c4",
+    "reproduce/source/tools/patch_checkpoint_v3.py": "8e0479ec80071339dcd1ca06a9de4cbc12e661750142c037c24ee3f51df5a60e",
+    "reproduce/source/tools/solve_assignment_2tier.py": "c0de546bfe8ec4698eb8c0ff5c73a7f270513f217670ed44481cf7913cec3430",
+    "reproduce/source/tools/make_hot_manifest.py": "310b088822bbcf423bc4fc67ccd1033b0c8bfbb761f778f9b13f95ec303822d5",
+    "reproduce/source/tools/build_checkpoint_v4.py": "46624935e36542442b15b65d4f27d7cb4fb4ee40759ec59a9823b69785c7d4d6",
+    "reproduce/source/tools/build_checkpoint_v5.py": "b72c276e1977b67ab31f7e269dc1c82529d0431e45df1c1676cca378db19fd29",
+    "reproduce/source/tools/make_hot_manifest2.py": "a7b8ddc26edf59956413e2ba51297b694a9edc33797fd726596798d249637dd6",
+    "reproduce/source/tools/build_checkpoint_v6.py": "bd707467d8d8ff0738eddc4ecb324125b7b6c094fab6532addacca9e7cd26725",
+    "reproduce/source/tools/run_rtx6000.sh": "6ab7fdcffa0fd1f4768134d0f98d670259935fafcef9e478204579b6376ac478",
+    "reproduce/source/tools/capture_golden.py": "353997aca54a26b622d831dc6224199bf9c73bfcdb31be61a9abdb4bfc6bb100",
+    "reproduce/source/tools/make_kernel_vectors.py": "633dcde4ca640a19754baddaa98eea065ee8c45af80c68c075f4cb9ed542f395",
+    "reproduce/source/tools/verify_sm120.py": "51f4f52f28e6424bc094273055c6ad62d138d70780c7632edfba0415524d5433",
+    "reproduce/source/tools/capture_acts.py": "907206bce4a843da2c5d3d3c153dc09c0787a70bae62fd388664fd2d114da632",
+    "reproduce/source/tools/aqlm_converge.py": "9cb4c093a6c8dfd7639b46bd68f98c7b113d39666642a4c117748776cd625cd5",
+    "reproduce/source/tools/CONVERGER.md": "3394b8437e2eb6038dfd8d7159768634dd4c87137847293a38dfb2b252472894",
+    "reproduce/source/tools/score_experts_reap.py": "21e675cc5ad0b2727ccedd5695341541a98a8575fb4ce9797ae8110a9d175b23",
+    "reproduce/source/tools/build_checkpoint_v7.py": "54e4838a08d6b6c8b6e4e9e8cc83c7636b46a1f4511e85140c72dd4280c2d911",
+    "reproduce/source/tools/build_calib_v3.py": "38f51b1d4ada52631d35c6afab1983b3118e4dadcf1a67c5f7ff3ede30ac499c",
+    "reproduce/source/tools/phase15_fit.py": "b2c4fe789e9fc355abcbdecb9a89fe53922f58564cb53af381ff27fb1abb6346",
+    "reproduce/source/tools/build_checkpoint_v8.py": "c30f8265ea2c6a982dcb6e2b432b28f34c4d765e1e55ad06ad42a275992b11b7",
+    "reproduce/source/tools/capture_acts_v2.py": "15ee0237b58c0e1cc68dbfcf832dd7e741fdfb0270678ea30bd1819749d01b8c",
+    "reproduce/source/tools/aqlm_full.py": "37b3757632e3be9e9e88ebb7145769c96ca956835593e1c039a5b36d79b84c0d",
+    "reproduce/source/tools/make_bf16_manifest.py": "7716ffabdd110ad606fe69992d1bc9380d7f4713c1f2dc52c1163e28f1b7be61",
+    "reproduce/source/tools/phase21_pipeline.py": "45fa1b0a34863d5a0112f4d8ced733b4a284dc6a2fe3b4d74308cbb64411b8ad",
+    "reproduce/source/tools/build_heldout_xl.py": "a331c916257890166729bb6b4de470c93a735f06194fd740a58e23f25233bd3a",
+    "reproduce/source/tools/bf16_stream.py": "67b7cdcb13c621f7c53d60d1c4d2d3aef6f64044a7c5e29e19ae6b42955360c4",
+    "reproduce/source/tools/pv_tune.py": "878ee5749eca7cc88ca3591a0b5d1f5d9c2f33e1baf584f7a1ae9631517f5038",
+    "reproduce/source/tools/capture_acts_fresh.py": "e46b3148902f8cee8a82a145ccfeac9a3c21332773b9f8b0e37ffe96b328d9b7",
+    "reproduce/source/tools/build_checkpoint_v9.py": "3f4e177466d57661d3f4f5fa75af22a11d89b46401c4e53a0cea970a82d46131",
+    "reproduce/source/tools/convert_fp8_attn.py": "20ed2e8f8398b1098e54ce953b8234f1484a75e4bd5508bbe2bc4ddd98d4af77",
+    "reproduce/source/tools/fetch_nvfp4_experts.py": "79c8f59d25880eaf288b2403a95c67b95df48144b8ae5a283cb55ff0a65bebaa",
+    "reproduce/source/tools/build_checkpoint_v10.py": "d84b3a38b74a7f6bef2913a03e710eac13a6d434a1a2adc96863b06de45863f8",
+    "reproduce/source/tools/encode_onto_codebook.py": "f9e3343a6cd5d95da2a48f79431cedb70b7c5c311dfd1392e398f8c425425626",
+    "reproduce/source/tools/run_b200_tp8.sh": "a71e0d82b6a3e9cc485f87098c8cb8f9fb86e0755d0bf3845521452f781dffda",
+    "reproduce/source/tools/aqlm_quantize.py": "da0cea94862d7703c028a139bc15fbc41d719390c09633303a1b678e6fcd31cd",
+    "reproduce/source/tools/build_calib_v31.py": "b87f2d9415380e28cfba2a0ce75f876ee48e37bb6e865c4a02d43e2300698fa7",
+    "reproduce/source/tools/reasoning_slice.py": "6878c1f6162cb97a98ee11679f97e39f616c0e61818312670bd8a4ffb5639f23",
+    "reproduce/source/tools/build_calib_v32.py": "6b369e36e210351fbdead6754170a83c6410e9692292e83a29926e50d23d1b08",
+    "reproduce/source/tools/build_trace_refit_v32.py": "0d4bdfa5345c0051bf03a7bcf9798922cb4cccdf3e3a379526e541e2af4f3ab9",
+    "reproduce/source/btx53/pilot_expert.py": "fb09fd073d8168d3b563ed5affb184be22bf6954b15d721fb039a5dd1fb40399",
+    "reproduce/source/btx53/retry_prefetch.sh": "9c27f3c724b803b284bc79dcd8a5d2fba6d145c3e4f7be43aebbf9f09b981da9",
+    "reproduce/source/btx53/pipeline.py": "091bcbe282ac930930704865bb02b2738103c5f42abbb1d2d1e8aed6d2849f24",
+    "reproduce/source/btx53/build_baseline.py": "0fd3b364f5ed470354723342ae766e8695b1804d7a50495adfe68941cfaf9344",
+    "reproduce/source/btx53/build_v2_seed_checkpoint.sh": "d0b4445045ee7900f4f423a54e2c1adcf4634756ba056f73b227b7bd228f7a56",
+    "reproduce/source/btx53/arvq_gate_loop.sh": "c7c8c9527efbafd16d9e11a604e51e505fa8fbbff6079edb9bfbdcc2476837fd",
+    "reproduce/source/btx53/arvq_gate.sh": "530ce8c4f6755a164862cfd1fce5c839e57d523288401bed0ab2ee25b26e1f65",
+    "reproduce/source/btx53/arvq_watchdog.sh": "415cb80615b9d6d0e2918cbfad978c4f312d49b271f37db633eb7e51b978ad60",
+    "reproduce/source/tools/sanity/sc1_schema.py": "78bff17dea376fb982ce8e58e77ce34772b3891b35e0acde5c70ba9a9e3f5cc1",
+    "reproduce/source/tools/sanity/sc2_dequant_stats.py": "f8440c5a4982482a5d5efb10d0b9e390c30bcace65d809ae8da0e4be8c7abc78",
+    "reproduce/source/tools/sanity/sc6_ppl.py": "7cafb6d5c6d9222e7582241c1533bfc6389e55a35079e1c6fee1235d3c7882a1",
+    "reproduce/source/tools/sanity/bench_small.py": "75bb8bec5854d8415aaa7b806ada089af29be7cc36283c6fbbe14a7be65f7647",
+    "reproduce/source/tools/evalsets/build_heldout_53.py": "e913a8b1223044c86665769497bf551be8da10a88e1ae94f64f97c4970619883",
+    "reproduce/source/tools/evalsets/build_neutral_sets.py": "21ec003f850b4397345cbd30ef320feb8c7db7f85ecc73c9fc17f22c824c087e",
+    "reproduce/source/tools/vision/fetch_tensor.py": "40b10e1b5e37096b23663884f0f3d7c9dc7326c2b6a09918e9f53beaf879934f",
+    "reproduce/source/tools/vision/compare_embeds.py": "ba890a855820b3f9cbc1c4e042da183ac1c521b3705921f917439d06b7a708dd",
+    "reproduce/source/tools/vision/compare_tower.py": "2cf6db6c278107837ec16239a4c2dcbe3a93a7cab99d9c592bf599e9d4d685d5",
+    "reproduce/source/tools/capture53/launch_stats.sh": "d4f4453b0ffb48cb7fcbd441bd1cac1e6af132c982cdd4eeb276bd91c10b0d01",
+    "reproduce/source/tools/capture53/solve_assignment53.py": "5cbdb5a27c8b34b194b04083a120d6e0f9bde64171a19d2629ace3349c4e77e8",
+    "reproduce/source/tools/capture53/mtp_requant53.py": "ce982688427c49e3f04ddd1e18e89b6dfa5ee9dcbf454fb9c2e184ded72e6ee6",
+    "reproduce/source/tools/capture53/vision_probe53.py": "b6a772b02b808bf2f72920b5f2678cf09210e91d6210b9e7c501c2691c88c7cc",
+    "reproduce/source/tools/capture53/score_reap53.py": "c711b388826f3fac6a0684d16772e98c54bbc29b35876513ff98feef79a45e58",
+    "reproduce/source/tools/capture53/pv_tune53.py": "9f2c246411878d6af6902c4b7074fc997a70e3f9064e6a235796da2630d03fc7",
+    "reproduce/source/tools/capture53/eval_hybrid53.py": "dd291b3ff47935773ae02b6c22d574065db5af54bc779d3605e9757eb8f98047",
+    "reproduce/source/tools/capture53/solve_blend53.py": "bda421790c9e42292cc4d6c1a023d0d8b60a2777262f9165e54a378eeed86420",
+    "reproduce/source/tools/capture53/build_checkpoint53.py": "f4ef159add9890d34580e62f153b9e8db72e9f8411396477da249c45e4510d40",
+    "reproduce/source/tools/capture53/solve_reap53.py": "29d35e7abbe23985e1adbeeb439015884f8523ba36bd602cecb817a5c9fd0308",
+    "reproduce/source/tools/capture53/build_calib_mm.py": "cbc0364753cef10607327fa3126caa825207882a05d0315d23f52dcc9a03862c",
+    "reproduce/source/tools/capture53/graft_vision53.py": "04fbca9c6bec0c935dad247ac2f06ee3df583ba5053c46235b7ad7b94504b85d",
+    "reproduce/source/tools/capture53/aqlm_converge53.py": "b3ccd5521e0ad3a82d38e56a6ce070625c2a99528d38cd932ac57aadc64a1af0",
+    "reproduce/source/tools/capture53/solve_r3.py": "5979122eab06cbcc1568f0942896f8d056773378064bf24551dc3c1ccb9b6c09",
+    "reproduce/source/tools/capture53/capture_mm53.py": "9201ddd9cbfa89092016f4f51b7859a6cb20d25ab13c575f0bb270d7b786bc4c",
+    "reproduce/source/tools/capture53/stream_capture53.py": "9ca7120673abb2ac0b4fbeaf25811dc36ca25da9d7791cceb65ae53d935d8a08",
+    "reproduce/source/tools/capture53/prepare_unc_trace_model.py": "bb08ea3058856ec31b595f2cf4aa24eb541cbf1ee2fb7f77f2008cc1e2d954af",
+    "reproduce/source/tools/capture53/eval3_kld.py": "0c270ef4822f47cfbc92721456297e74a4ef115608a6d25d7eaba1d91e55cfe7",
+    "reproduce/source/btx53/arvq88/__init__.py": "0bf1139931af3c4357fa592ee2f2abc669bc2ce17826b4ab9cb4f593f773d018",
+    "reproduce/source/btx53/arvq88/inputs.py": "4a8ebeef4ad6a07e47b7d8e28474dd2ea48a192f755cd14abada993c052305a0",
+    "reproduce/source/btx53/arvq88/fit.py": "7288bbe580b307dbc399b161ee5899ea0cab9d57e57d40224ebce74152575497",
+    "reproduce/source/btx53/arvq88/checkpoint.py": "4d1194767fb46102085732eb114e524bf052653ad82f80367a655ba1155b7b90",
+    "reproduce/source/btx53/arvq88/cli.py": "452f32eb2eaa6fde3a8270c79599e185e36ab11e6993901cc7f94f56d5903840",
+    "reproduce/source/btx53/arvq88/gate_worker.py": "05199a15d687bf943ba0e8b9f8d16958a195a5874aff9d1b42ae00bbebda697f",
+    "reproduce/source/btx53/arvq88/reap_worker.py": "ac39da41fb132a026d666c892e300948b7606492610816068b0223848b14049d",
+    "reproduce/source/btx53/arvq88/reap.py": "9e69dbfcf48987b53c8694aa8c32f29e04bea270481a7c98a98b0a2e0a277704",
+    "reproduce/source/btx53/arvq88/pv_campaign.py": "95a483dedc27f61fb767cb8020471a04fd5cfa05078f816efbfc6fdb9cbd15cb",
+    "reproduce/source/btx53/arvq88/validation.py": "829c6a4ece05c189653a18dfe9429f3a48cb29c54e373fcd398706e3439a47ce",
+    "reproduce/source/btx53/arvq88/promote_initial.py": "27b7600139e9f5bf30ce0b3db0cb587284c16a79e77baa1cd3049a03b0e04dc0",
+    "reproduce/source/btx53/arvq88/local_source.py": "0c57365308afcf75bf4b485a1e86fd1ca169ce7c4ddb999f80f2c3f938bb1bd9",
+    "reproduce/source/btx53/arvq88/backbone.py": "88e88930ea9ecec52259f5ab648915dc9b3a4d8144b302dadfbeb736c9dcc45b",
+    "reproduce/source/btx53/arvq88/activation.py": "cf13e2170085459b97907a270abb32d694af3ce69f723b332119a62be00e6641",
+    "reproduce/source/btx53/arvq88/alternating.py": "19e38a8230dba40bc022180ed4234b5a60f53765d36a417677e35711e9f0c80c",
+    "reproduce/source/btx53/arvq88/arvq_reap.py": "c39faf937462d85b2ede04f19f14c2cc8ef875c6788a1dab2445a03834882cfd",
+    "reproduce/source/btx53/arvq88/jobs.py": "4b9021e5763ba15b45cf4c0cf76d6f62f705ac175b076cdae380032a2d9ac59d",
+    "reproduce/source/btx53/arvq88/experiment.py": "65272954df9ef44f88467c05f5ce5f12c8045b38765f67b773353890d7877de4",
+    "reproduce/source/btx53/arvq88/incremental_publish.py": "70526567046728bd2905d059bf475f6a1d4d74d4242f7288cd64091e9137f910",
+    "reproduce/source/btx53/arvq88/gradient_indices.py": "09bd66d77575deec66d1732677355c406cb9b228abfc5002e79cbae5adb72e83",
+    "reproduce/source/btx53/arvq88/sequential_pv.py": "807609be6d53c94ed578ae91f81e74d3be60451d436d6d5f1b5053004ea821f0",
+    "reproduce/source/btx53/arvq88/sequential_pilot.py": "030f3bd1d83ea2f9bcd97ee8514d01dc69d269e7d7b7f87159da667b46b76710",
+    "reproduce/source/btx53/arvq88/encoder.py": "6c20c56fc7a308724ab4faf4174c4e8e32cba5a43c3fccd3f3e6d4da1f0b583e",
+    "reproduce/source/btx53/arvq88/sequential_capture.py": "bcbb6cb450243411e064624c70b1ec460e052680eb7f9958f418b25577d9e671",
+    "reproduce/source/btx53/arvq88/final_publish.py": "e7dcb2859de2e08a5c2bea7b21b9aae61e96bb2361b28ba4270cc145fb0da188",
+    "reproduce/source/btx53/arvq88/pv.py": "82f3f559e44b5db2480e041fa45d983bf1e6afc2d74a68c0511af0d174f9ed68",
+    "reproduce/source/btx53/arvq88/pack.py": "b0606bf8ae91bab655ed5b26bb9b7437f0e28718301fdb1b86f475b1b31924e1",
+    "reproduce/source/btx53/arvqprep/__init__.py": "921c16de3c268ed12dcac8329852f47fb6875d926a2ea665a03e69c81059b23d",
+    "reproduce/source/btx53/arvqprep/pack.py": "ce459847f07a558377872e0f1f83620d9f26d9eec7ec94d08247052876bab656",
+    "reproduce/source/btx53/arvqprep/encoder.py": "ace5d22c886994a0327e8320c6674f71c1caab779d215e3d05295d1f490bfec1",
+    "reproduce/source/btx53/arvqprep/smoke.py": "fb445c69e081b1c23acaf03feef234e215c61cceaf1b6d0b4c47aa63f9032a34",
+    "reproduce/source/btx53/arvqprep/container.py": "3f5ab2a065a1a44826cd0183ac39ba1d016ac87be655ed3b9502b1c3137ae7ad",
+    "reproduce/source/btx53/arvqprep/workflow.py": "778d6782e95cf81ea7d7c44f8f508aab65706e64a18cc3d15adc57eb61e5e095",
+    "reproduce/source/btx53/arvqprep/publish.py": "3249526e87beba17e69612d034cc400c57cf88a09eea3b3622eb8e4f43b868ce",
+    "reproduce/source/btx53/arvqprep/pv_workflow.py": "5a57e8ae4e76f938f6cda3b245321c4812e5e7c16e52b3976adcfa96b3cb9187",
+    "reproduce/source/btx53/btxprep/core.py": "b4fad5c192efa2e9e0088427ad8e90dacac59b9220479809b81ca826556c8348",
+    "reproduce/source/btx53/btxprep/transforms.py": "2f0bda3c93e39836ae6bca82deceb876e87b54797dd696c194a99041f6bd6d1c",
+    "reproduce/source/btx53/btxprep/container.py": "f260e70e4e6fda4c20b8cb85469620ebe67d343983776c370b0bfe7bdb2f967e",
+    "reproduce/source/btx53/btxprep/__init__.py": "f8b773e399d3d5b859d05a85347d8f6a91eed2808447c934d917c1de39d43a40",
+    "reproduce/source/btx53/btxprep/encoder.py": "ac8745d1ba1590817a16bc4ed48acc9af980a7103ad22e32aeb179b48b5813f2",
+    "reproduce/source/btx53/tools/remote_st.py": "4a27ac143fff1cc8cdf6fb349ac58bba1d22933c77c562779bb111b727f0031e",
+    "reproduce/source/btx53/tools/ingest.py": "e05630716d805903f1bfe658512da2d3f63abbd280a3af87ca9baadb5becd52c",
+    "reproduce/source/btx53/tools/upload_btx.py": "72bc32e8f2c2755e9ec5c713d6226bd7021265d1cddaf1e927b3682b508e5cba",
+    "reproduce/source/btx53/tools/filter_shards.py": "749d7b6f8a5b6ba24442508ce66e4108086b837e89c20234c3bf4939b27ada7d",
+    "reproduce/source/btx53/tools/fetch_hyb_kinds.py": "aaa37fd535d26b3772c4e467c55502c810ba39b73ca998078c487aeb1a7be846",
+    "reproduce/source/btx53/tools/gates_btx.py": "bcd2d8b5e4ef9e3779bfbec625d7af5b0727d954502db63a46147de882de170c",
+    "reproduce/source/btx53/tools/upload_btx_full.py": "ee8b46dd631629e15d1f1bb86c0b06d4ad22e573c674e129e58c04b5e7e1dc8d",
+    "reproduce/source/btx53/tools/launch_btx_workers.sh": "4e8484172fe4dea0b48390c72959d81bc4d25ac05265187b9bb90df9c5bc592a",
+    "reproduce/source/btx53/tools/upload_hybrid.py": "5b3e4be4ccf076d569ea5a48f0e79d6168236d4d6eccf84053a4bf183a4cc2f6",
+    "reproduce/source/btx53/tools/gates_arvq.py": "ce6b09045cc8f1c9cea7647083e133a52885b284c09879ccf4b9090d58d27453",
+    "reproduce/source/btx53/tools/launch_arvq_workers.sh": "78b013b3c8f5e2750fc6a3b7e88d58ddc5a61aafdea0d51eb666d0d3dba2000c",
+    "reproduce/source/btx53/tools/arvq_driver.sh": "13d72054ed28f5e112b98ddd7db0cdb7de00e0a4ead5cbd54b03bc0386860cc8",
+    "reproduce/source/btx53/tools/pv_arvq.py": "82dd57c204c057eaacd17aefcd37faa1d6b90ab7f6931aa2872b3acf65822858",
+    "reproduce/source/btx53/tools/prefetch_cold_fp8.py": "5825b1069ee10681516d69a703699abae4ac46011f6e71d6537acfece4345921",
+    "reproduce/source/btx53/arvq88/tests/test_workflow.py": "d9bc78d2a898a9c8dab96ea2a79dcf183cf182fb535992ff106ff452c8961979",
+    "reproduce/source/btx53/arvq88/tests/test_final_publish.py": "6c2b104ec40284415763fea5005510df0b63055bf99f081345e8248de0f7eb24",
+    "reproduce/source/btx53/arvq88/tests/test_validation.py": "f563a433c66412085a191deb8c693865632aedf1de19948567c710d11c3606b7",
+    "reproduce/source/btx53/arvq88/tests/test_local_source.py": "544513082fe0c666ade7ed00a837aeeeb915c52fb1325362a488e1c8c3a050ed",
+    "reproduce/source/btx53/arvq88/tests/test_donor.py": "1948eca1225198bed8964ff9d56e5f2b98c86847537a108670cf59260a20d76e",
+    "reproduce/source/btx53/arvq88/tests/test_backbone.py": "58f3875440e45b9bfd5be915452a7f1a18cd36688909f737d5830e1a5062fa85",
+    "reproduce/source/btx53/arvq88/tests/test_activation.py": "eff65a3e37b8b4b77b8fbb8bd979f6ca5012680a9643716f257f4e8985662944",
+    "reproduce/source/btx53/arvq88/tests/test_expert_books.py": "5942045e2e09a0a50ea1bae852a2f9a411b8dfd949e78e8e67151b8e119ec33f",
+    "reproduce/source/btx53/arvq88/tests/test_incremental_publish.py": "d0296210c666b4c556dcc6139500a1a0a36646186ee7a019d0ca2d41b9a80368",
+    "reproduce/source/btx53/arvq88/tests/test_gradient_indices.py": "6f1527d21b73c882500bb8ff229c54a023abe89d43ad23c6c3176122c188347e",
+    "reproduce/source/btx53/arvq88/tests/test_sequential_pv.py": "7d3c65cf41648b1b564a1939bcbee3d495ec4bd9d0af7028a5d788b052fadd18",
+    "reproduce/source/btx53/arvq88/docs/per_expert_handoff.md": "554425132e253e53a7890ba6f270c2cbfcec4e7115921db2983036c5b6d20664",
+    "reproduce/source/btx53/arvq88/docs/expert_experiments.md": "6d78a3ee627255f13e2be7995bf21df4c36f9703b192bc5020a5c1c7a7a7336e",
+    "reproduce/source/btx53/arvq88/docs/gradient_pv_recipe.md": "77686dc9d94c3a4c5559024598dbbcafe7ca55f17858e7305fb2ad069a4f274f",
+    "reproduce/source/btx53/arvq88/docs/sequential_pilot.md": "079342a060d5fc384e00b724ca3e1189c169ef4b7c590df12b1739c1dceba6d5",
+    "reproduce/source/btx53/arvq88/perf/graph_expert.py": "9d663b49ef7bb23db843f78068f146e2c178f23137360acfaa1aa544c6942d73",
+    "reproduce/source/btx53/arvq88/perf/bench_graph_expert.py": "9b6b5759376fb771caed63e908e256d9b187fec93e0d6bcb6a16a16c97cdb42e",
+    "reproduce/source/btx53/arvq88/perf/sparse_delta.py": "52565e3bdaa477d1676fffaf10bdffd10e1ad2d1b4275ded2cdd1d210bd0f81e",
+    "reproduce/source/btx53/arvq88/perf/test_perf.py": "3f9221722a3d0aa3360ba02ca5823612162aefb27d084f0d4a3bacf4bac45ef9",
+    "reproduce/source/btx53/arvq88/perf/README.md": "ebeeb89781cd162e1711837bab91a9077df496a876bc99ab6d6a0fe53779c21b",
+    "reproduce/source/btx53/arvq88/perf/rerun_pilot.py": "691665c2f6d482d8518f7153124814f8333a810d44c8c03269756ef2bbf8199f",
+    "reproduce/source/btx53/arvq88/perf/full_reference.py": "cabbd41c1be1f2aa77e3b298b5958a79aed88cc73263ec358ef2c64a9d4846b5",
+    "reproduce/source/btx53/arvq88/perf/sequential_pv_repair.py": "74ece4df20d6573ae8bf752736adb12217968098c49e6f5d3bfbb7ebc1d3b3f3",
+    "reproduce/source/btx53/arvq88/perf/test_repair.py": "78112f17972eadf45c7f48f578b8c75170c70e7fe0aa75f63bf861c6161f952c",
+    "reproduce/source/btx53/arvq88/perf/sequential_pv_cached.py": "9c629c8083fe5cc2e760e1d6afc3dc1f60c1320ac30b4dcd95db35dc93d8b163",
+    "reproduce/source/btx53/arvq88/perf/batch_benchmark.py": "c78c5608221c051b93817b373c94c7ca844416851f72d3905f4e231272776829",
+    "reproduce/source/btx53/arvq88/perf/capture_full_layer3.py": "3359edb2956d41de364c32e209275c1d87d047bbfe48ee84f29d76d9ee535a1b",
+    "reproduce/source/btx53/arvq88/perf/test_full_corpus.py": "98491ab3943d70b4ed2536ecbd72bd163561002d6fba61124f829adcbcf5bf1c",
+    "reproduce/source/btx53/arvq88/perf/async_shards.py": "b536e99bf5ea3badd4291f3fbc9b22a0594d334e9b551f3146a71288991d7529",
+    "reproduce/source/btx53/arvq88/perf/test_async_shards.py": "ecee760fa718fa0542a8650bf8a5b7d1b7ccade05b6682f0660401cac4208f34",
+    "reproduce/source/btx53/arvq88/perf/FULL_CORPUS_PIPELINE.md": "2c842e5833b64c1de55b9b3d33756ddb5cc6dfaef808ccd652cbc37f07280040",
+    "reproduce/source/btx53/arvq88/perf/distributed_batch.py": "ff4fcfdc9c74bc46f06bb51a126d2764aa4d983da5b9eebc06885081330fa39e",
+    "reproduce/source/btx53/arvq88/perf/test_distributed_batch.py": "36bc25e59d2ec9045751409ccb74a2f0464a2de502fa5adb4affff5fec3a56cc",
+    "reproduce/source/btx53/arvq88/perf/check_recapture.py": "9b49570cf9dca3d42ee9262822b7992b2d1ddf6025838f4fbe3d9ff5ab090593",
+    "reproduce/source/btx53/arvq88/perf/publish_full_corpus.py": "a361a0fe959a214dc8f7550c90248697b46f1d1fa3aeba1f149503723617b0b7",
+    "reproduce/source/btx53/arvq88/perf/capture_transfer.py": "aa6f8702cbaaf3ebcc9dad65053c422f3ec7f1e1d9c4ac3fc60834769c38b386",
+    "reproduce/source/btx53/arvq88/perf/test_capture_transfer.py": "c33c0ffda4ca906930760cd100d7e0d283d32eeba1fbebd47af34bffa680abfa",
+    "reproduce/source/btx53/arvq88/perf/monitor_output_error.py": "cce2dd878e8e164e8b3a296cc70699cc8eb6e3225bac3f9b8c5171e063af40cb",
+    "reproduce/source/btx53/arvq88/perf/capture_matched_validation.py": "5d0b9d1892923acbd066f0f88030221db39aecee6ca62e8a1f094a29833a473d",
+    "reproduce/source/btx53/arvq88/perf/adaptive_schedule.py": "504f54400931422d706544029ca1c3673224f33f77d7801c89dda1d4ba53dd30",
+    "reproduce/source/btx53/arvq88/perf/test_adaptive_schedule.py": "456bc945a66c6ff1893f86a972a63b2efb44ae3a386e09e02df23673780cc537",
+    "reproduce/source/btx53/arvq88/perf/campaign_watchdog.py": "548d78758937c79d17601772f2c5424a8bbf5d9952144b6fbbde396a8de4e499",
+    "reproduce/source/btx53/arvq88/perf/row_scale_pilot.py": "a84aec6f9aa2824aaa9f00e8d53c19b5f5deca76ddd4c0da69230bebe95c0181",
+    "reproduce/source/btx53/arvq88/perf/four_layer_row_pilot.py": "de817289405b9f2a4eb34e28b315e779dcadadfafc401c1e28d54c75b5a0acf7",
+    "reproduce/source/btx53/arvq88/perf/row_scale_combined_eval.py": "6e2084b496058983fc4b143142ec3ebcb9703bed5801bb7bb0a599017976e17f",
+    "reproduce/source/btx53/arvq88/perf/FP16_BLOCK_SCALE_HANDOFF.md": "9304e796d7fbff92157737b9aa6cf2a6e29e66d16656409af4ab06c5f18e8cfa",
+    "reproduce/source/btx53/arvq88/perf/sequential_pv_same_input.py": "c6fb2c7a2f565c45f326837da2546ebd3fd01d2496353539d3a3303c9c370f13",
+    "reproduce/source/btx53/arvq88/perf/full_corpus.py": "db9e8cb6b3d7ea22fc8490fc355fd9421dfc7429c600431cc37f52017f9811c2",
+    "reproduce/source/btx53/arvq88/perf/full_pipeline.py": "c35476f87e9c2231cc1acc0059e17f91d0c51f04a99d464423054d54b4d793a4",
+    "reproduce/source/btx53/arvq88/perf/fused_handoff.py": "6c62bf5606ed48fa8a1a9d27a5f694f353f44c7a02f651475522f2e431a5c37a",
+    "reproduce/source/btx53/arvq88/perf/fused_evaluation_capture.py": "10ef6f00129bcea2915b41febdb08ba2f942c4f132e61ca4a774d8f913bf8daa",
+    "reproduce/source/btx53/arvq88/perf/capture_full_sequential.py": "ee221434a21fb4818f9ab20925ea66c31a396c8b3557dda1a45174fabd284927",
+    "reproduce/source/btx53/arvq88/perf/publish_v2_streaming.py": "07ea5e0fb78b462590f422895e05c823208da79d929980e3a6b0dde071f69395",
+    "reproduce/source/btx53/arvq88/perf/sequential_pv_fast.py": "a9b04ba27ebfd00f10bfbf96791c816858cef27d560086610e3bf6906fde9356",
+    "reproduce/source/btx53/arvq88/perf/propagate_full.py": "efb52d2784d1fca48beb3404c462255756c3fe499f68a418baa8a790d2018bf9",
+    "reproduce/source/btx53/arvq88/perf/baseline_propagation.py": "a0169695c1993f9c57da6c078e6e22aaefd404e1108a39a1989ceb1142d1cf35",
+    "reproduce/source/btx53/arvq88/perf/sequential_pv_full_corpus.py": "297df5e8e276e2caaae2af049b4dc0bf2e6185e0a9844bfdf705272aa4f96c3c",
+    "reproduce/source/btx53/arvq88/perf/publish_same_input.py": "bc75d242557ad1cac3b2cdc316b5760a501da5b221f7ed1f439a39d09b3de978",
+    "reproduce/source/btx53/arvq88/perf/publish_reference.py": "204d86bd3393bd91b5b43a5f7eb8f5c40d2cb3811962b772ab5cfffca1f63bab",
+    "reproduce/source/btx53/arvq88/perf/sequential_pv_mcbook.py": "f60b89d8325f4afd07349980c9190f78caf7ca0c05494805b04a1032de8fc749",
+    "reproduce/source/btx53/arvqprep/tests/test_workflow.py": "9250f47f793f13098ea48ce9ec51baec71ff65c410b3beac4609f32f8decefeb",
+    "reproduce/source/btx53/btxprep/tests/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+    "reproduce/source/btx53/btxprep/tests/test_roundtrip.py": "fdce4358d869c1fb5fe01a7a9a878d6a29710af65779bbf0637be52eefd84735"
+  },
+  "weights_changed": false,
+  "validated": "2026-09-26",
+  "checks": [
+    "source SHA256",
+    "all Python parses",
+    "CPU FP8/FP16 decode fixtures",
+    "50x next-token boundary semantics",
+    "relocated workspace smoke"
+  ],
+  "not_validated": [
+    "full historical retraining",
+    "native SM120 execution"
+  ]
+}

reproduce/prepare_workspace.py

@@ -0,0 +1,26 @@
+"""Materialize a fresh portable source copy; no fitting or publication is run."""
+import argparse,json,shutil,sys,re,hashlib
+from pathlib import Path
+p=argparse.ArgumentParser();p.add_argument('--dest',type=Path,required=True);a=p.parse_args();dest=a.dest.resolve();root=Path(__file__).resolve().parent
+if dest.exists():raise SystemExit('Destination must not exist; refusing to modify an existing workspace')
+shutil.copytree(root/'source',dest)
+mapping={'/home/coder/git/glm52':str(dest),'/data':str(dest/'data'),'/tmp':str(dest/'tmp')}
+pattern="(?:"+"|".join(re.escape(k) for k in mapping)+")(?=/|$|[\\s\"' ])"
+changed={}
+for f in dest.rglob('*'):
+ if not f.is_file() or f.suffix not in ('.py','.sh','.md'):continue
+ s=f.read_text();t=s
+ # Interpreter locations must resolve to the activated environment.
+ t=re.sub(r'/tmp/(?:venv[^/]*|[^/]+/venv)/bin/python(?:3)?','__GLM53_PYTHON__',t)
+ t=re.sub(pattern,lambda m:mapping[m.group()],t).replace('__GLM53_PYTHON__',sys.executable)
+ if t!=s:f.write_text(t);changed[str(f.relative_to(dest))]=hashlib.sha256(f.read_bytes()).hexdigest()
+(dest/'data').mkdir(exist_ok=True);(dest/'tmp').mkdir(exist_ok=True);(dest/'.venv/bin').mkdir(parents=True,exist_ok=True);(dest/'.venv/bin/python').symlink_to(sys.executable)
+cfg=json.loads((root/'recipe.json').read_text())
+def relocate(v):
+ if isinstance(v,str):
+  return re.sub(pattern,lambda m:mapping[m.group()],v)
+ if isinstance(v,list):return [relocate(x) for x in v]
+ if isinstance(v,dict):return {k:relocate(x) for k,x in v.items()}
+ return v
+(dest/'recipe.json').write_text(json.dumps(relocate(cfg),indent=2));(dest/'relocation.json').write_text(json.dumps({'mapping':mapping,'python':sys.executable,'changed_sha256':changed},indent=2))
+print(f'Prepared {dest}. Populate donor/data/cache paths, inspect recipe.json, then follow RUNBOOK.md. No job launched.')

reproduce/recipe.json

@@ -0,0 +1,116 @@
+{
+  "baseline": "/tmp/glm53-vision-expert-full",
+  "corpus": "/tmp/glm53-vision-trace-refit",
+  "source_cache": "/tmp/glm53-fp8-cold",
+  "sequence_length": 1024,
+  "sequences": [
+    64,
+    16,
+    16
+  ],
+  "world_size": 8,
+  "record_initial_audit": true,
+  "train_token_file": "/tmp/glm53-layer3-full-corpus/train.npy",
+  "batch_tokens": 262144,
+  "microbatch_tokens": 65536,
+  "codebook_lr": 0.048,
+  "scale_lr": 0.032,
+  "reference_only": false,
+  "target": "same_input",
+  "block_scale_dtype": "fp16",
+  "codebook_dtype": "fp4_grid",
+  "boundary_boost": 50.0,
+  "boundary_token_ids": [
+    154842,
+    154820
+  ],
+  "reassign_every": 10,
+  "reassign_max_fraction": 0.005,
+  "reassign_trust_ratio": 0.02,
+  "reassign_target_ratio": 0.1,
+  "lr_decay_after": 25,
+  "lr_decay_factor": 0.25,
+  "adaptive_schedule": {
+    "relative_worsening": 0.001,
+    "checks": 3,
+    "patience_updates": 10
+  },
+  "fuse_next_capture": true,
+  "layers": [
+    3,
+    4,
+    5,
+    6,
+    7,
+    8,
+    9,
+    10,
+    11,
+    12,
+    13,
+    14,
+    15,
+    16,
+    17,
+    18,
+    19,
+    20,
+    21,
+    22,
+    23,
+    24,
+    25,
+    26,
+    27,
+    28,
+    29,
+    30,
+    31,
+    32,
+    33,
+    34,
+    35,
+    36,
+    37,
+    38,
+    39,
+    40,
+    41,
+    42,
+    43,
+    44,
+    45,
+    46,
+    47,
+    48,
+    49,
+    50,
+    51,
+    52,
+    53,
+    54,
+    55,
+    56,
+    57,
+    58,
+    59,
+    60,
+    61,
+    62,
+    63,
+    64,
+    65,
+    66,
+    67,
+    68,
+    69,
+    70,
+    71,
+    72,
+    73,
+    74,
+    75,
+    76,
+    77
+  ]
+}

reproduce/requirements.in

@@ -0,0 +1,10 @@
+torch
+numpy
+transformers
+datasets
+safetensors
+huggingface_hub
+accelerate
+Pillow
+scipy
+triton

reproduce/source/btx53/arvq88/__init__.py

@@ -0,0 +1 @@
+"""GLM-5.3 256+256 FP4 additive-vector quantization workflow."""

reproduce/source/btx53/arvq88/activation.py

@@ -0,0 +1,44 @@
+"""Serving activation-plane reference, pinned to b1380cf74d170b69a8708f1b1287cc09a6eb7df4.
+
+Matches hybrid.cu pack_planes and the default FP16 SwiGLU path. This emulates
+quantization boundaries, not SM120 MMA instruction accumulation bit-for-bit.
+"""
+import torch
+REFERENCE_REVISION='b1380cf74d170b69a8708f1b1287cc09a6eb7df4'
+ARITHMETIC='fp4_planes4_fp16_boundaries_v1'
+
+def fp16_ste(x):
+ rounded=x.to(torch.float16).float()
+ return x+(rounded-x).detach() if x.requires_grad else rounded
+
+def planes(x):
+ """Return FP4 codes, FP8 scale bytes, and decoded *unweighted* planes."""
+ if x.shape[-1]%16:raise ValueError('Activation input dimension must be divisible by 16')
+ v=x.to(torch.float16).float().reshape(*x.shape[:-1],-1,16)
+ if not torch.isfinite(v).all():raise ValueError('Nonfinite FP16 serving activation')
+ cuts=torch.tensor([.25,.75,1.25,1.75,2.5,3.5,5.],device=x.device)
+ table=torch.tensor([0.,.5,1.,1.5,2.,3.,4.,6.],device=x.device)
+ codes=[];scales=[];decoded=[]
+ for _ in range(4):
+  peak=v.abs().amax(-1,keepdim=True)
+  exponent=torch.ceil(torch.log2((peak/6).clamp_min(2**-20))).clamp(-6,8)
+  scale=torch.exp2(exponent)
+  # CUDA compares strictly > each midpoint: ties go toward zero, not ties-to-even.
+  q=(v.abs().div(scale).unsqueeze(-1)>cuts).sum(-1)
+  dec=table[q]*scale*torch.where(v<0,-1.,1.)
+  codes.append((q|((v<0).long()<<3)).to(torch.uint8).reshape(x.shape))
+  scales.append(((exponent.squeeze(-1).long()+7)<<3).to(torch.uint8))
+  decoded.append(dec.reshape(x.shape))
+  v=(v-dec)*16
+ return torch.stack(codes),torch.stack(scales),torch.stack(decoded)
+
+def activation_ste(x):
+ with torch.no_grad():
+  _,_,p=planes(x)
+  result=p[0]+p[1]/16+p[2]/256+p[3]/4096
+ return x+(result-x).detach() if x.requires_grad else result
+
+def swiglu_ste(gu):
+ # Serving: FP32 projection -> FP16; SiLU FP16 result; FP16 product.
+ gu=fp16_ste(gu);gate,up=gu.chunk(2,-1)
+ return fp16_ste(fp16_ste(torch.nn.functional.silu(gate))*up)

reproduce/source/btx53/arvq88/alternating.py

@@ -0,0 +1,64 @@
+"""Discrete LDLQ proposals interleaved with continuous output matching.
+
+This is a bounded alternating optimizer, not a reproduction of the PV-Tuning
+paper. Proposals use training activation Hessians and are retained only if the
+training routed-output objective decreases. Validation still selects checkpoints.
+"""
+import math
+from pathlib import Path
+import torch
+from .encoder import cholesky_inv_upper, hessian_fc1, sweep_expert
+from .activation import ARITHMETIC, activation_ste, swiglu_ste
+
+
+def accept_candidate(before, after):
+    return math.isfinite(after) and after < before - 1e-8
+
+
+@torch.no_grad()
+def reassign_indices(p13, p2, x, ids, train, cold, args, meta, evaluate_train):
+    from ingest import load_expert_bf16
+    before = evaluate_train()
+    # Codes must participate in both validation-best snapshots and rollback.
+    old = [(p.codes_a.clone(), p.codes_b.clone()) for p in (p13, p2)]
+    changed = total = 0
+    try:
+        for e, eid in enumerate(cold):
+            rows = train[(ids[train] == eid).any(1)]
+            if len(rows) < 32:
+                continue  # No held-out-row fallback for rare experts.
+            z = x[rows[:args.reassign_hessian_tokens]]
+            wg, wu, wd = [w.to(x.device).float() for w in load_expert_bf16(
+                str(Path(args.src_cache)/f'layer_{args.layer}'/f'expert_{eid}.safetensors'), meta)]
+            z = activation_ste(z) if args.arithmetic == ARITHMETIC else z
+            # Down sees the current quantized gate/up, not teacher intermediates.
+            gu = z @ p13.weight(e).T
+            if args.arithmetic == ARITHMETIC:
+                down_x = activation_ste(swiglu_ste(gu))
+            else:
+                g, u = gu.chunk(2, -1)
+                down_x = torch.nn.functional.silu(g) * u
+            for p, W, inp in ((p13, torch.cat([wg, wu]), z), (p2, wd, down_x)):
+                H = hessian_fc1(inp)
+                hv = cholesky_inv_upper(H)
+                cb, scale, glob = p.quantized(e)
+                a, b, _ = sweep_expert(W, hv, cb[:256], cb[256:], scale,
+                                      float(glob), col_block=128, refine=2)
+                changed += int(((a != p.codes_a[e]) | (b != p.codes_b[e])).sum())
+                total += a.numel()
+                p.codes_a[e].copy_(a); p.codes_b[e].copy_(b)
+                del H, hv, a, b
+            del wg, wu, wd, gu, down_x, z
+        after = evaluate_train()
+        accepted = accept_candidate(before, after)
+        if not accepted:
+            for p, (a, b) in zip((p13, p2), old):
+                p.codes_a.copy_(a); p.codes_b.copy_(b)
+        return {'method': 'LDLQ training-Hessian proposal with routed-training-output acceptance',
+                'training_before': before, 'training_candidate': after,
+                'accepted': accepted, 'proposed_changed_groups': changed,
+                'groups_considered': total}
+    except BaseException:
+        for p, (a, b) in zip((p13, p2), old):
+            p.codes_a.copy_(a); p.codes_b.copy_(b)
+        raise

reproduce/source/btx53/arvq88/arvq_reap.py

@@ -0,0 +1,174 @@
+"""Score actual ARVQ vs NVFP4 expert outputs on training captures.
+
+Uses fitted ARVQ candidates for ALL 256 experts, including currently hot experts.
+The score is the positive reduction in routing-weighted squared output error
+against original FP8 when an expert is kept NVFP4 instead of ARVQ. Cross-expert
+error covariance and downstream model quality remain outside this proxy.
+"""
+import argparse
+import json
+import sys
+from pathlib import Path
+import numpy as np
+import torch
+
+ROOT = Path(__file__).resolve().parents[1]
+sys.path[:0] = [str(ROOT), str(ROOT/'tools'), str(ROOT.parent/'tools')]
+from arvq88.activation import activation_ste, swiglu_ste, ARITHMETIC
+from arvq88.encoder import EncodedLayerProj, reconstruct
+from arvq88.inputs import write
+from arvq88.pack import sha
+from arvq88.reap import blend_allocate
+
+
+def hot_benefit(reference, cold_output, hot_output, gates):
+    if reference.shape != cold_output.shape or reference.shape != hot_output.shape:
+        raise ValueError('Expert outputs must have matching shapes')
+    weight = gates.double().square()
+    cold = (cold_output.double()-reference.double()).square().sum(-1)
+    hot = (hot_output.double()-reference.double()).square().sum(-1)
+    return float((weight*(cold-hot)).sum())
+
+
+def score_output(x, w13, w2, quantized):
+    if quantized:
+        return activation_ste(swiglu_ste(activation_ste(x) @ w13.T)) @ w2.T
+    g,u=(x @ w13.T).chunk(2,-1)
+    return (torch.nn.functional.silu(g)*u) @ w2.T
+
+
+@torch.no_grad()
+def score_layer(args):
+    from ingest import SourceMeta, load_expert_bf16
+    from aqlm_quantize import ShardReader, dequant_nvfp4, FP4_LUT
+    torch.set_num_threads(2)
+    dev=args.device; L=args.layer; source=Path(args.fit_dir)/f'layer_{L:05d}'
+    manifest=json.loads((source/'arvq-manifest.json').read_text())
+    if manifest['cold_expert_ids']!=list(range(256)):
+        raise ValueError('ARVQ allocation scoring requires candidates for all 256 experts')
+    store={}
+    for name in ('w13','w2'):
+        store.update(torch.load(source/f'{name}.pt',map_location='cpu',weights_only=True))
+    enc={k:EncodedLayerProj(**store[k]) for k in ('w13','w2')}
+    captures=torch.load(args.capture,map_location='cpu',weights_only=True)
+    if 'pv_split' in captures:
+        mask=captures['pv_split']==0
+        captures={k:v[mask] for k,v in captures.items() if k in ('x','topk_ids','topk_weights')}
+    x=captures['x'].to(dev).float();ids=captures['topk_ids'].to(dev);g=captures['topk_weights'].to(dev).float()
+    meta=SourceMeta(**json.loads(Path(args.source_meta).read_text())['meta'])
+    reader=ShardReader(args.donor);lut=torch.tensor(FP4_LUT,device=dev,dtype=torch.float32)
+    benefits=np.zeros(256); counts=np.zeros(256,dtype=np.int64)
+    for e in range(256):
+        rows=(ids==e).any(1).nonzero().flatten()
+        counts[e]=len(rows)
+        if not len(rows):continue
+        cold13=reconstruct(enc['w13'],e,dev);cold2=reconstruct(enc['w2'],e,dev)
+        wg,wu,wd=[w.to(dev).float() for w in load_expert_bf16(str(Path(args.src_cache)/f'layer_{L}'/f'expert_{e}.safetensors'),meta)]
+        pre=f'model.layers.{L}.mlp.experts.{e}'
+        hot=[dequant_nvfp4(reader,pre+'.'+proj,dev,lut).float() for proj in ('gate_proj','up_proj','down_proj')]
+        hot13=torch.cat(hot[:2]);hot2=hot[2];ref13=torch.cat([wg,wu])
+        for batch in rows.split(args.batch):
+            z=x[batch];weight=(g[batch]*(ids[batch]==e)).sum(1)
+            ref=score_output(z,ref13,wd,False)
+            cold=score_output(z,cold13,cold2,True)
+            hot_out=score_output(z,hot13,hot2,True)
+            benefits[e]+=hot_benefit(ref,cold,hot_out,weight)
+        del cold13,cold2,wg,wu,wd,hot,hot13,hot2,ref13
+    out=Path(args.out);out.parent.mkdir(parents=True,exist_ok=True)
+    np.savez(out,scores=benefits.clip(min=0),signed_hot_benefit=benefits,routed_rows=counts,layer=L)
+    write(out.with_suffix('.json'),{'layer':L,'metric':'sum routing_weight^2 * (ARVQ squared error - NVFP4 squared error), clipped at zero',
+        'reference':'original block-FP8 weights','arithmetic':ARITHMETIC,'capture_sha256':sha(args.capture),
+        'candidate_manifest_sha256':sha(source/'arvq-manifest.json'),'scope':'all 256 experts; training tokens only',
+        'limitation':'FP32 GEMMs with serving rounding emulation; no native MMA or cross-expert covariance'})
+    print('SCORED',L,args.capture,flush=True)
+
+
+def allocate_scores(text_files,mm_files,hot_count=5750):
+    if len(text_files)!=75 or len(mm_files)!=75:raise ValueError('Need both modalities for all 75 layers')
+    def read(files):
+        scores=[]
+        for L,p in zip(range(3,78),files):
+            d=np.load(p)
+            if int(d['layer'])!=L:raise ValueError('Wrong score layer order')
+            scores.append(d['scores'])
+        return np.stack(scores)
+    text,mm=read(text_files),read(mm_files)
+    result=blend_allocate(text,mm,hot_count)
+    result['provenance']={'mode':'ARVQ output-benefit allocation','text_weight':.75,'mm_weight':.25,
+        'hot_count':hot_count,'floor':8,'cap':176,'text_partition':'train only',
+        'metric':'positive routing-weighted squared-error reduction from retaining NVFP4 over fitted ARVQ',
+        'scores_sha256':{str(p):sha(p) for p in [*text_files,*mm_files]},
+        'limitation':'per-expert additive error proxy; excludes cross-expert covariance and end-to-end model quality'}
+    return result
+
+
+
+
+def recompute(args, hub, work):
+    """Fit all expert candidates, measure both modalities, then spend hot budget."""
+    from types import SimpleNamespace
+    from arvq88.inputs import ensure_source, LAYERS
+    from arvq88.jobs import run_jobs
+    if not getattr(args,'mm_acts',None):
+        raise ValueError('ARVQ REAP requires --mm-acts with matching donor multimodal captures')
+    mm=Path(args.mm_acts)
+    if not all((mm/f'acts_layer{L}.pt').is_file() for L in LAYERS):
+        raise ValueError('Incomplete multimodal captures; refusing text-only ARVQ REAP')
+    prefit=work/'arvq_reap';prefit.mkdir(exist_ok=True)
+    a=SimpleNamespace(**vars(args));a.workdir=str(prefit)
+    # These are candidates, not a valid final 5750-hot allocation.
+    all_experts={'layers':{str(L):{'cold_5750':list(range(256)),'hot':[]} for L in LAYERS}}
+    write(prefit/'status.json',{'stage':'preparing_original_fp8_sources','experts':19200})
+    ensure_source(a,hub,prefit,all_experts)
+    write(prefit/'assignment.json',all_experts);write(prefit/'run_config.json',vars(a))
+    write(prefit/'status.json',{'stage':'fitting_all_expert_candidates','layers':75,'experts':19200})
+    run_jobs([(f'fit-{L}',lambda gpu,L=L:[sys.executable,'-u',str(ROOT/'arvq88/fit.py'),str(prefit),str(L),f'cuda:{gpu}']) for L in LAYERS],args.gpus,prefit/'logs')
+    jobs=[]
+    for modality,folder in [('text',Path(args.calib)),('mm',mm)]:
+        for L in LAYERS:
+            output=prefit/'scores'/modality/f'layer_{L}.npz'
+            jobs.append((f'score-{modality}-{L}',lambda gpu,L=L,folder=folder,output=output:[
+                sys.executable,'-u',str(ROOT/'arvq88/arvq_reap.py'),
+                '--fit-dir',str(prefit/'initial'),'--capture',str(folder/f'acts_layer{L}.pt'),
+                '--src-cache',args.src_cache,'--source-meta',str(prefit/'source.json'),
+                '--donor',args.donor_cache,'--layer',str(L),'--device',f'cuda:{gpu}','--out',str(output)]))
+    write(prefit/'status.json',{'stage':'scoring_text_and_multimodal','jobs':150})
+    run_jobs(jobs,args.gpus,prefit/'logs')
+    result=allocate_scores([prefit/'scores/text'/f'layer_{L}.npz' for L in LAYERS],
+                           [prefit/'scores/mm'/f'layer_{L}.npz' for L in LAYERS],args.hot_count)
+    result['provenance']['candidate_codebook_scope']=args.codebook_scope
+    write(prefit/'status.json',{'stage':'allocation_complete','hot':args.hot_count,'cold':19200-args.hot_count})
+    return result
+
+
+def select_candidates(prefit,work,allocation):
+    """Keep exactly the fitted candidates used to calculate hot-slot benefit."""
+    from arvq88.pack import export_layer
+    prefit,work=Path(prefit),Path(work)
+    for L in range(3,78):
+        source=prefit/'initial'/f'layer_{L:05d}';dest=work/'initial'/f'layer_{L:05d}'
+        dest.mkdir(parents=True,exist_ok=True)
+        manifest=json.loads((source/'arvq-manifest.json').read_text())
+        if manifest['cold_expert_ids']!=list(range(256)):raise ValueError('Candidates must cover all experts')
+        cold=allocation['layers'][str(L)]['cold_5750'];files={}
+        for proj in ['w13','w2']:
+            d=torch.load(source/f'{proj}.pt',map_location='cpu',weights_only=True)[proj]
+            for key in ['a','b','s']:d[key]=d[key][cold]
+            if d['c0'].ndim==3:
+                for key in ['c0','c1']:d[key]=d[key][cold]
+            path=dest/f'{proj}.pt';tmp=path.with_suffix('.tmp');torch.save({proj:d},tmp);tmp.replace(path)
+            files[proj]={'file':path.name,'sha256':sha(path)}
+        write(dest/'arvq-manifest.json',{**manifest,'cold_expert_ids':cold,'files':files,
+            'selection':'Exact subset of all-expert ARVQ candidates scored for allocation',
+            'candidate_manifest_sha256':sha(source/'arvq-manifest.json')})
+        export_layer(dest,work/'cold',L)
+
+
+if __name__=='__main__':
+    ap=argparse.ArgumentParser(description=__doc__)
+    ap.add_argument('--fit-dir',required=True);ap.add_argument('--capture',required=True)
+    ap.add_argument('--src-cache',required=True);ap.add_argument('--source-meta',required=True)
+    ap.add_argument('--donor',required=True);ap.add_argument('--layer',type=int,required=True)
+    ap.add_argument('--out',required=True);ap.add_argument('--device',default='cuda:0')
+    ap.add_argument('--batch',type=int,default=256)
+    score_layer(ap.parse_args())

reproduce/source/btx53/arvq88/backbone.py

@@ -0,0 +1,44 @@
+"""Explicit tensor provenance: language/MTP from donor, vision from base."""
+import copy,json,re,os
+from pathlib import Path
+
+def vision_tensor(name):
+ return name.startswith(('vision_tower.','mm_projector.'))
+def routed_tensor(name):
+ m=re.match(r'model.layers\.(\d+)\.mlp\.experts\.',name)
+ return bool(m and 3<=int(m[1])<=77)
+
+def copy_backbone(base,donor,dest,save):
+ from safetensors import safe_open
+ from .pack import sha
+ provenance={}
+ for root,label,choose in [(base,'default-vision',vision_tensor),(donor,'nvfp4-donor',lambda n:not vision_tensor(n) and not routed_tensor(n))]:
+  index=json.loads((root/'model.safetensors.index.json').read_text())['weight_map']
+  for i,filename in enumerate(sorted(set(index.values()))):
+   names=sorted(n for n,f in index.items() if f==filename and choose(n))
+   if not names:continue
+   target=f'{label}-{i:05d}.safetensors'
+   with safe_open(str(root/filename),framework='pt') as stream:
+    save(target,{n:stream.get_tensor(n) for n in names})
+   # save_file preserves dtype and values; gate the resulting immutable shard identity.
+   provenance[target]={'source':str(root/filename),'tensor_names':names,'sha256':sha(dest/target),'role':label}
+ expected={n for n in json.loads((base/'model.safetensors.index.json').read_text())['weight_map'] if not routed_tensor(n) and not vision_tensor(n) and not n.endswith(('.weight_scale','.weight_scale_2','.input_scale','.input_scale_2'))}
+ actual={n for f in provenance.values() if f['role']=='nvfp4-donor' for n in f['tensor_names']}
+ if expected-actual:raise ValueError(f'Donor missing language/MTP tensors: {sorted(expected-actual)[:5]}')
+ return provenance
+
+def hybrid_config(base,donor):
+ config=copy.deepcopy(base);text=copy.deepcopy(donor.get('text_config',donor))
+ qc=copy.deepcopy(base.get('quantization_config',{}))
+ donor_qc=text.pop('quantization_config',{})
+ if donor_qc.get('quant_method')=='modelopt':qc['nvfp4']=donor_qc
+ elif donor_qc.get('nvfp4'):qc['nvfp4']=donor_qc['nvfp4']
+ else:raise ValueError('Expected NVFP4 donor quantization config')
+ if 'text_config' in config:
+  config['text_config']=text
+  for name in ['pad_token_id','eos_token_id','tie_word_embeddings','dtype']:
+   if name in text:config[name]=text[name]
+ else:config=text
+ config['quantization_config']=copy.deepcopy(qc)
+ config.get('text_config',config)['quantization_config']=copy.deepcopy(qc)
+ return config

reproduce/source/btx53/arvq88/checkpoint.py

@@ -0,0 +1,168 @@
+"""Assemble/gate/publish an explicitly versioned 8+8 hybrid checkpoint."""
+import json,re,shutil,os
+from pathlib import Path
+from .inputs import LAYERS,complete_snapshot,write,token
+from .pack import sha
+
+def build(args,hub,work,allocation):
+ # Refresh the card renderer after long PV runs so reviewed provenance text is current.
+ import importlib,sys
+ if 'arvq88.pv_campaign' in sys.modules:importlib.reload(sys.modules['arvq88.pv_campaign'])
+ import torch
+ from safetensors import safe_open
+ from safetensors.torch import save_file
+ base=Path(args.local_hot)
+ if not complete_snapshot(base):base=hub.snapshot(args.base_hybrid,work/'base-hybrid')
+ dest=work/'checkpoint'
+ if dest.resolve()==base.resolve():raise ValueError('Output checkpoint cannot overwrite the local hot donor')
+ dest.mkdir(exist_ok=True)
+ manifests=[json.loads((work/'cold'/f'layer-{L:03d}-manifest.json').read_text()) for L in LAYERS]
+ formats={(m['format'],m['version']) for m in manifests}
+ if len(formats)!=1 or not formats<= {('rvq256_256x8',2),('rvq256_256x8_expert',3)}:
+  raise ValueError('Mixed or unsupported cold formats')
+ cold_format,cold_version=next(iter(formats))
+ index=json.loads((base/'model.safetensors.index.json').read_text())['weight_map'];wm={};total=0
+ def save(name,tensors):
+  nonlocal total
+  if not tensors:return
+  save_file(tensors,str(dest/name));wm.update({k:name for k in tensors});total+=sum(t.numel()*t.element_size() for t in tensors.values())
+ from .backbone import copy_backbone,hybrid_config
+ donor=Path(args.donor_cache)
+ if not complete_snapshot(donor):donor=hub.snapshot(args.nvfp4_donor,donor)
+ if dest.resolve()==donor.resolve():raise ValueError('Output cannot overwrite donor')
+ # A stale assembled checkpoint must not retain weights from the previous provenance rule.
+ marker=dest/'backbone_sources.json'
+ if any(dest.glob('*.safetensors')) and not marker.exists():raise ValueError('Use a fresh checkpoint directory for donor-backbone assembly')
+ provenance=copy_backbone(base,donor,dest,save)
+ write(marker,provenance)
+ dwm=json.loads((donor/'model.safetensors.index.json').read_text())['weight_map']
+ def donor_get(name):
+  nonlocal dwm,donor
+  if dwm is None:
+   if not complete_snapshot(donor):donor=hub.snapshot(args.nvfp4_donor,donor)
+   dwm=json.loads((donor/'model.safetensors.index.json').read_text())['weight_map']
+  with safe_open(str(donor/dwm[name]),framework='pt') as f:return f.get_tensor(name)
+ for L in LAYERS:
+  pre=f'model.layers.{L}.mlp.experts.';row=allocation['layers'][str(L)];cold=row['cold_5750'];hot=row['hot']
+  with safe_open(str(base/index[pre+'hyb_kind']),framework='pt') as f:old=f.get_tensor(pre+'hyb_kind')
+  tensors={};expected=torch.zeros(256,dtype=torch.int8);expected[cold]=2;tensors[pre+'hyb_kind']=expected
+  suffixes=['nvfp4_w13_packed','nvfp4_w13_bscale','nvfp4_w13_scale2','nvfp4_w2_packed','nvfp4_w2_bscale','nvfp4_w2_scale2']
+  if torch.equal(old,expected) and args.nvfp4_donor=='RadixArk/GLM-5.3-NVFP4':
+   for suffix in suffixes:
+    name=pre+suffix
+    with safe_open(str(base/index[name]),framework='pt') as f:tensors[name]=f.get_tensor(name)
+  else:
+   parts={k:[] for k in suffixes}
+   for e in hot:
+    ep=pre+str(e)+'.'
+    for suffix,out_suffix in [('weight','packed'),('weight_scale','bscale')]:
+     gu=torch.cat([donor_get(ep+p+'.'+suffix) for p in ['gate_proj','up_proj']]).view(torch.uint8)
+     dn=donor_get(ep+'down_proj.'+suffix).view(torch.uint8)
+     parts['nvfp4_w13_'+out_suffix].append(gu);parts['nvfp4_w2_'+out_suffix].append(dn)
+    parts['nvfp4_w13_scale2'].append(torch.stack([donor_get(ep+p+'.weight_scale_2').float().reshape(()) for p in ['gate_proj','up_proj']]))
+    parts['nvfp4_w2_scale2'].append(donor_get(ep+'down_proj.weight_scale_2').float().reshape(1))
+   tensors.update({pre+k:torch.stack(v) for k,v in parts.items()})
+  save(f'hot-layer-{L:03d}.safetensors',tensors)
+  manifest=json.loads((work/'cold'/f'layer-{L:03d}-manifest.json').read_text())
+  if manifest['cold_expert_ids']!=cold:raise ValueError('Cold fit/assignment mismatch')
+  for f in manifest['files'].values():
+   src=work/'cold'/f['file']
+   if sha(src)!=f['sha256']:raise ValueError('Cold shard identity mismatch')
+   target=dest/src.name
+   if not target.exists():os.link(src,target)
+   elif sha(target)!=f['sha256']:raise ValueError('Stale assembled cold shard')
+   with safe_open(str(target),framework='pt') as stream:
+    for name in stream.keys():
+     t=stream.get_tensor(name);wm[name]=src.name;total+=t.numel()*t.element_size()
+ for p in base.iterdir():
+  if p.is_file() and p.suffix in ['.json','.py','.jinja'] and p.name not in ['config.json','model.safetensors.index.json','backbone_sources.json'] and 'report' not in p.name and 'provenance' not in p.name:shutil.copy2(p,dest/p.name)
+ # Keep the vision wrapper/processor; language config and tokenizer follow donor.
+ for p in donor.iterdir():
+  if p.is_file() and (p.name.startswith(('tokenizer','special_tokens','added_tokens','chat_template','generation_config')) or p.name.endswith('.model')):shutil.copy2(p,dest/p.name)
+ config=hybrid_config(json.loads((base/'config.json').read_text()),json.loads((donor/'config.json').read_text()))
+ books={str(L):{'n_nvfp4':len(allocation['layers'][str(L)]['hot']),'n_base':0,'n_cold':len(allocation['layers'][str(L)]['cold_5750'])} for L in LAYERS}
+ for cfg in [config,config.get('text_config',config)]:
+  qc=cfg.setdefault('quantization_config',{});qc['quant_method']='nvfp4_arvq_hybrid'
+  qc['arvq']={'format':cold_format,'version':cold_version,'activation_planes':4,'weight_scale_group':128,
+              'codebook_scope':'expert' if cold_version==3 else 'layer','codebook_sizes':[256,256]}
+  qc['aqlm_layer_books']=books
+ write(dest/'config.json',config);write(dest/'model.safetensors.index.json',{'metadata':{'total_size':total},'weight_map':wm})
+ shutil.copy2(work/'assignment.json',dest/'cold_assignment.json');shutil.copy2(work/'source.json',dest/'source.json')
+ shutil.copytree(work/'cold',dest/'cold_manifests',dirs_exist_ok=True,ignore=shutil.ignore_patterns('*.safetensors'))
+ return dest
+
+def gates(work,layers=None,device="cpu",report_path=None):
+ import torch
+ from safetensors import safe_open
+ from .pack import decode
+ torch.set_num_threads(2);layers=LAYERS if layers is None else layers;ck=work/'checkpoint';wm=json.loads((ck/'model.safetensors.index.json').read_text())['weight_map'];errors=[];hashes={}
+ if any(k.endswith(('.w13_codes','.w2c_codes','.w2m_codes')) for k in wm):errors.append('Old AQLM indices leaked')
+ for L in layers:
+  m=json.loads((work/'cold'/f'layer-{L:03d}-manifest.json').read_text());E=len(m['cold_expert_ids'])
+  pre=f'model.layers.{L}.mlp.experts.'
+  with safe_open(str(ck/wm[pre+'hyb_kind']),framework='pt') as stream:
+   kind=stream.get_tensor(pre+'hyb_kind')
+   if (kind==2).nonzero().flatten().tolist()!=m['cold_expert_ids']:errors.append('Hot/cold assignment mismatch')
+   for suffix,shape in [('nvfp4_w13_packed',(256-E,4096,3072)),('nvfp4_w2_packed',(256-E,6144,1024)),('nvfp4_w13_bscale',(256-E,4096,384)),('nvfp4_w2_bscale',(256-E,6144,128)),('nvfp4_w13_scale2',(256-E,2)),('nvfp4_w2_scale2',(256-E,1))]:
+    if tuple(stream.get_slice(pre+suffix).get_shape())!=shape:errors.append(f'Hot shape mismatch {L} {suffix}')
+  for proj in ['w13','w2']:
+   f=m['files'][proj];path=ck/f['file'];digest=sha(path);hashes[f['file']]=digest
+   if digest!=f['sha256']:errors.append(f'SHA mismatch {path.name}')
+   with safe_open(str(path),framework='pt') as stream:
+    t={k:stream.get_tensor(k) for k in stream.keys()}
+   N,K=(4096,6144) if proj=='w13' else (6144,2048);pre=f'model.layers.{L}.mlp.experts.arvq_{proj}_'
+   cbshape=(E,512) if m.get('version')==3 else (512,)
+   config=json.loads((ck/'config.json').read_text())
+   qconfig=config.get('text_config',config)['quantization_config']['arvq']
+   if qconfig['format']!=m['format'] or qconfig['version']!=m['version']:errors.append(f'Format metadata mismatch {L}')
+   for suffix,shape,dtype in [('packed',(E,N//16,K//64,64),torch.uint32),('codebooks',cbshape,torch.uint32),('scales',(E,N//16,K//128,16),torch.uint8),('global',(1,),torch.float32)]:
+    if t[pre+suffix].shape!=shape or t[pre+suffix].dtype!=dtype:errors.append(f'Shape/dtype {L} {suffix}')
+    if wm.get(pre+suffix)!=f['file']:errors.append('Index mismatch')
+   scales=t[pre+'scales'].view(torch.float8_e4m3fn).float()
+   if not torch.isfinite(scales).all() or (scales<0).any() or not torch.isfinite(t[pre+'global']).all() or not (t[pre+'global']>0).all():errors.append(f'Invalid scales {L} {proj}')
+   if not torch.isfinite(decode({k:v.to(device) for k,v in t.items()},L,proj,0)).all():errors.append('Nonfinite decoded sample')
+   del t,scales
+ backbone=json.loads((ck/'backbone_sources.json').read_text())
+ for name,entry in backbone.items():
+  if sha(ck/name)!=entry['sha256']:errors.append(f'Backbone SHA mismatch {name}')
+  if any(wm.get(n)!=name for n in entry['tensor_names']):errors.append(f'Backbone index mismatch {name}')
+ report={'passed':not errors,'layers':len(layers),'layer_ids':layers,'errors':errors,'cold_file_sha256':hashes,'scope':'Initial-fit export has exhaustive index roundtrips and sampled weight parity; checkpoint hashes, shapes, scales and decoded finiteness checked. No SM120 serving or model quality gate.'}
+ report['metadata_sha256']={n:sha(ck/n) for n in ['config.json','model.safetensors.index.json','backbone_sources.json']}
+ write(report_path or ck/'gates_report.json',report)
+ if errors:raise RuntimeError(f'8+8 gates failed: {errors[:3]}')
+ return report
+
+def parallel_gates(work,gpus):
+ import subprocess,sys
+ from .inputs import ROOT
+ folder=work/'gate_parts';folder.mkdir(exist_ok=True);jobs=[]
+ for i,gpu in enumerate(gpus):
+  layers=LAYERS[i::len(gpus)]
+  if not layers:continue
+  result=folder/f'gpu{gpu}.json'
+  with open(folder/f'gpu{gpu}.log','a') as log:
+   p=subprocess.Popen([sys.executable,str(ROOT/'arvq88/gate_worker.py'),str(work),','.join(map(str,layers)),f'cuda:{gpu}',str(result)],stdout=log,stderr=subprocess.STDOUT,start_new_session=True)
+  jobs.append((p,result))
+ codes=[p.wait() for p,_ in jobs]
+ if any(codes):raise RuntimeError(f'Gate workers failed: {codes}; inspect {folder}')
+ parts=[json.loads(r.read_text()) for _,r in jobs];ids=[l for p in parts for l in p['layer_ids']]
+ if sorted(ids)!=LAYERS:raise RuntimeError('Gate layer coverage mismatch')
+ if any(p['metadata_sha256']!=parts[0]['metadata_sha256'] for p in parts):raise RuntimeError('Checkpoint metadata changed during gates')
+ report={**parts[0],'passed':all(p['passed'] for p in parts),'layers':75,'layer_ids':LAYERS,
+         'errors':[e for p in parts for e in p['errors']],
+         'cold_file_sha256':{k:v for p in parts for k,v in p['cold_file_sha256'].items()},'workers':len(parts)}
+ write(work/'checkpoint/gates_report.json',report)
+ if not report['passed']:raise RuntimeError('Gates did not pass')
+ return report
+
+def publish(args,hub,work):
+ ck=work/'checkpoint';report=json.loads((ck/'gates_report.json').read_text())
+ if not report['passed'] or len(report['cold_file_sha256'])!=150:raise RuntimeError('Incomplete initial checkpoint gates')
+ for name,digest in {**report['cold_file_sha256'],**report['metadata_sha256']}.items():
+  if sha(ck/name)!=digest:raise RuntimeError('Weights changed after gates')
+ parent=hub.api.model_info(args.dst).sha
+ commit=hub.api.upload_folder(repo_id=args.dst,folder_path=str(ck),revision='main',parent_commit=parent,commit_message='GLM-5.3 NVFP4 + ARVQ 8+8 activation-calibrated initial fit')
+ remote={s.rfilename:s for s in hub.api.model_info(args.dst,revision=commit.oid,files_metadata=True).siblings}
+ for name,digest in report['cold_file_sha256'].items():
+  if remote[name].lfs.sha256!=digest:raise RuntimeError('Remote cold-weight hash mismatch')
+ write(work/'upload.json',{'repo':args.dst,'revision':commit.oid,'cold_files_verified':150});return commit.commit_url

reproduce/source/btx53/arvq88/cli.py

@@ -0,0 +1,154 @@
+"""Local-or-Hub GLM-5.3 to NVFP4/ARVQ 8+8 with audited PV tuning."""
+import argparse,fcntl,json,os,subprocess,sys,time,traceback,shutil
+from pathlib import Path
+from .inputs import ROOT,LAYERS,Hub,assignment,ensure_captures,ensure_source,write
+
+def parser():
+ p=argparse.ArgumentParser(description=__doc__)
+ p.add_argument('--arvq88',action='store_true',help=argparse.SUPPRESS)
+ p.add_argument('--src',default='zai-org/GLM-5.3',help='Hugging Face repo ID or local model directory');p.add_argument('--dst',required=True)
+ p.add_argument('--workdir',default='/tmp/glm53-arvq88');p.add_argument('--gpus',default='0,1,2,3,4,5,6,7')
+ p.add_argument('--assign',default=str(ROOT/'cold_assignment.json'));p.add_argument('--reap-scores')
+ p.add_argument('--mm-stats',help='Cached multimodal salience NPZ for r3 allocation')
+ p.add_argument('--mm-calib',default='/tmp/glm53-calib-mm')
+ p.add_argument('--vision-dir',default='/tmp/glm53-vision')
+ p.add_argument('--reap-parts',help='Historical all-expert 1x16 AQLM warm starts, allocation proxy only')
+ p.add_argument('--reap-metric',choices=['aqlm','arvq'],default='aqlm',help='ARVQ recomputes hot allocation from fitted ARVQ vs NVFP4 output error on both modalities')
+ p.add_argument('--mm-acts',help='Matching donor multimodal activation directory for ARVQ allocation scoring')
+ p.add_argument('--hot-count',type=int,default=5750);p.add_argument('--calib',default='/tmp/glm53-capture/acts')
+ p.add_argument('--calib-tokens',default='/tmp/glm52-calib-v3');p.add_argument('--calib-token-count',type=int,default=15_000_000)
+ p.add_argument('--src-cache',default='/tmp/glm53-fp8-cold')
+ p.add_argument('--nvfp4-donor',default='RadixArk/GLM-5.3-NVFP4',help='NVFP4 Hugging Face repo ID or local sharded checkpoint directory; download cache is automatic')
+ p.add_argument('--base-hybrid',default='jarrelscy/GLM-5.3-Vision-NVFP4-AQLM-hybrid-1m')
+ p.add_argument('--local-hot',default='/tmp/glm53-hybrid-pipeline/checkpoint')
+ p.add_argument('--download-workers',type=int,default=8);p.add_argument('--threads',type=int,default=2)
+ p.add_argument('--col-block',type=int,default=128);p.add_argument('--cb-iters',type=int,default=8)
+ p.add_argument('--sweep-passes',type=int,default=2);p.add_argument('--refine',type=int,default=1)
+ p.add_argument('--codebook-scope',choices=['layer','expert'],default='layer')
+ p.add_argument('--pv-reassign-every',type=int,default=0,help='Alternate index proposals every N PV steps; 0 freezes indices')
+ from .activation import ARITHMETIC
+ p.add_argument('--pv-arithmetic',choices=['fp32',ARITHMETIC],default=ARITHMETIC,help='PV forward arithmetic: serving activation emulation or legacy FP32')
+ p.add_argument('--pv-steps',type=int,default=200,help='PV steps per layer; 0 produces initial fits only')
+ p.add_argument('--clean-repo',action='store_true',help='After verified final PV upload, remove obsolete repository files')
+ p.add_argument('--detach',action='store_true');p.add_argument('--do-upload',action='store_true')
+ p.add_argument('--plan',action='store_true',help='Show resolved workflow without downloading, fitting or uploading')
+ return p
+
+def resolve_donor(args,work):
+ from .local_source import local_directory,fingerprint
+ from .inputs import complete_snapshot
+ local=local_directory(args.nvfp4_donor)
+ args.donor_identity=None
+ if local:
+  if not complete_snapshot(local) or not (local/'config.json').is_file():
+   raise ValueError('Local NVFP4 donor requires config.json, model.safetensors.index.json and all indexed shards')
+  args.nvfp4_donor=str(local);args.donor_cache=str(local)
+  args.donor_identity=fingerprint(local)
+ else:
+  import hashlib
+  legacy=Path('/tmp/glm53-dl/nvfp4-full')
+  args.donor_cache=str(legacy if args.nvfp4_donor=='RadixArk/GLM-5.3-NVFP4' and complete_snapshot(legacy)
+                       else work/'donors'/hashlib.sha256(args.nvfp4_donor.encode()).hexdigest()[:16])
+
+def main():
+ args=parser().parse_args()
+ from .local_source import local_directory,fingerprint
+ local=local_directory(args.src)
+ if local:
+  args.src=str(local);fingerprint(local)  # Validate config/index/shards before capture or GPU work.
+ args.gpus=[int(x) for x in args.gpus.split(',')]
+ if not args.gpus or len(args.gpus)!=len(set(args.gpus)) or min(args.gpus)<0:raise ValueError('GPUs must be a unique nonnegative list')
+ if args.pv_steps!=0 and args.pv_steps<40:raise ValueError('PV steps must be 0 or at least 40')
+ if args.pv_reassign_every<0:raise ValueError('Invalid index reassignment interval')
+ if args.reap_metric=='arvq' and not args.mm_acts:raise ValueError('ARVQ scoring requires --mm-acts')
+ if args.clean_repo and args.pv_steps==0:raise ValueError('Repository cleanup requires final PV weights')
+ if args.threads<1 or args.download_workers<1 or args.calib_token_count<2048:raise ValueError('Invalid CPU/download/calibration count')
+ if args.col_block<=0 or args.col_block%8 or 2048%args.col_block or args.cb_iters<1 or args.sweep_passes<1 or args.refine<0:raise ValueError('Invalid encoder settings')
+ work=Path(args.workdir);work.mkdir(parents=True,exist_ok=True)
+ resolve_donor(args,work)
+ # Do not silently reuse this machine's original-source artifacts for another input.
+ if args.src!='zai-org/GLM-5.3':
+  for name,default,replacement in [('src_cache','/tmp/glm53-fp8-cold',work/'source'),('calib','/tmp/glm53-capture/acts',work/'capture/acts'),('calib_tokens','/tmp/glm52-calib-v3',work/'calib-tokens'),('assign',str(ROOT/'cold_assignment.json'),work/'recomputed-assignment.json')]:
+   if getattr(args,name)==default:setattr(args,name,str(replacement))
+ if args.nvfp4_donor!='RadixArk/GLM-5.3-NVFP4':
+  if args.calib=='/tmp/glm53-capture/acts':args.calib=str(work/'capture/acts')
+  if args.assign==str(ROOT/'cold_assignment.json'):args.assign=str(work/'recomputed-assignment.json')
+ if args.base_hybrid!='jarrelscy/GLM-5.3-Vision-NVFP4-AQLM-hybrid-1m' and args.local_hot=='/tmp/glm53-hybrid-pipeline/checkpoint':args.local_hot=str(work/'base-hybrid')
+ if args.plan:
+  print(json.dumps({'format':'rvq256_256x8_expert' if args.codebook_scope=='expert' else 'rvq256_256x8','stage':'PV-tuned' if args.pv_steps else 'initial fit (no PV)','args':vars(args),'steps':['reuse/download text calibration','reuse/capture NVFP4-teacher activations','reuse allocation or scores, otherwise recalculate 75% text REAP + 25% multimodal salience allocation','reuse/fetch original cold source tensors','fit 8+8 with per-layer cache identities',*(['PV with train/validation/audit splits, early stopping and audit fallback'] if args.pv_steps else []),'assemble donor hot/nonexpert/MTP + ARVQ cold + default vision','gate packed format and hashes','upload main only with --do-upload'],'fork_changes':False},indent=2));return
+ compat='/tmp/compat13/usr/local/cuda-13.2/compat'
+ if Path(compat).exists() and compat not in os.environ.get('LD_LIBRARY_PATH','').split(':'):
+  os.execvpe(sys.executable,[sys.executable]+sys.argv,{**os.environ,'LD_LIBRARY_PATH':compat+':'+os.environ.get('LD_LIBRARY_PATH','')})
+ if args.detach:
+  with open(work/'driver.log','a') as log:
+   child=subprocess.Popen([sys.executable,'-u']+[a for a in sys.argv if a!='--detach'],stdin=subprocess.DEVNULL,stdout=log,stderr=subprocess.STDOUT,start_new_session=True)
+  print(f'Detached 8+8 pipeline PID {child.pid}; {work}/driver.log');return
+ lock=open(work/'.workflow.lock','a');fcntl.flock(lock,fcntl.LOCK_EX|fcntl.LOCK_NB)
+ import torch
+ torch.set_num_threads(args.threads)
+ if not torch.cuda.is_available():raise RuntimeError('CUDA required for 8+8 pipeline')
+ # Never overbook another user's or an existing detached worker's GPU.
+ result=subprocess.run(['nvidia-smi','--query-compute-apps=gpu_uuid,pid','--format=csv,noheader'],capture_output=True,text=True,check=True)
+ gpuinfo=subprocess.run(['nvidia-smi','--query-gpu=index,uuid','--format=csv,noheader'],capture_output=True,text=True,check=True)
+ selected={line.split(',')[1].strip() for line in gpuinfo.stdout.splitlines() if int(line.split(',')[0]) in args.gpus}
+ if any(line.split(',')[0].strip() in selected for line in result.stdout.splitlines() if line.strip()):raise RuntimeError('Requested GPUs have active compute jobs; choose free --gpus or wait. No processes were stopped.')
+ try:
+  binding={'src':args.src,'nvfp4_donor':args.nvfp4_donor,'base_hybrid':args.base_hybrid,'format':'rvq256_256x8_expert' if args.codebook_scope=='expert' else 'rvq256_256x8',
+           'reap_metric':args.reap_metric,'pv_reassign_every':args.pv_reassign_every}
+  if args.donor_identity:binding['local_donor_identity']=args.donor_identity
+  if (work/'binding.json').exists() and json.loads((work/'binding.json').read_text())!=binding:raise ValueError('Workdir belongs to another input/donor/format')
+  write(work/'binding.json',binding);hub=Hub(work/'hfcache')
+  ensure_captures(args,hub,work)
+  allocation=assignment(args,hub,work)
+  ensure_source(args,hub,work,allocation)
+  write(work/'run_config.json',vars(args));pending=list(LAYERS);running={}
+  if args.reap_metric=='arvq':
+   from .arvq_reap import select_candidates
+   select_candidates(work/'arvq_reap',work,allocation);pending=[]
+  while pending or running:
+   for gpu in args.gpus:
+    if gpu in running or not pending:continue
+    L=pending.pop(0)
+    with open(work/f'fit-layer{L}.log','a') as log:
+     child=subprocess.Popen([sys.executable,'-u',str(ROOT/'arvq88/fit.py'),str(work),str(L),f'cuda:{gpu}'],stdout=log,stderr=subprocess.STDOUT,stdin=subprocess.DEVNULL,start_new_session=True)
+    running[gpu]=(L,child);print('FIT',L,'GPU',gpu,'PID',child.pid,flush=True)
+   time.sleep(15)
+   for gpu,(L,child) in list(running.items()):
+    rc=child.poll()
+    if rc is None:continue
+    if rc:raise RuntimeError(f'Fit layer {L} failed; see fit-layer{L}.log')
+    del running[gpu];print('FIT DONE',L,flush=True)
+  if args.pv_steps:
+   from .pv_campaign import run as run_pv
+   run_pv(args,work,work/'initial')
+  from .checkpoint import build,parallel_gates,publish
+  ck=build(args,hub,work,allocation);parallel_gates(work,args.gpus)
+  write(ck/'build_provenance.json',{'inputs':binding,'hub_revisions':hub.revisions,'allocation':allocation['provenance'],'capture_dir':args.calib,'calib_tokens':args.calib_tokens,'stage':'PV-tuned' if args.pv_steps else 'initial fit, no PV','format':binding['format'],'source_nonexpert_mtp':'nvfp4_donor','source_vision':'base_hybrid','capture_teacher':'nvfp4_donor','allocation_recipe':allocation['provenance']})
+  (ck/'README.md').write_text(f'''---
+base_model: {args.src}
+tags: [arvq, nvfp4, experimental]
+---
+# GLM-5.3 NVFP4 / ARVQ 8+8 hybrid
+
+Cold experts use two shared FP4 dictionaries of 256 entries, 8-dimensional groups,
+16 index bits/group and FP8 scales/128 weights (2.0625 bpw plus codebooks).
+This is the activation-calibrated initial Hessian fit, **before PV**.
+Hot experts use the selected NVFP4 donor. Nonexpert and MTP tensors come from the NVFP4 donor; vision comes from the
+base hybrid. See build_provenance.json for every donor and allocation method.
+
+Codebook scope: {args.codebook_scope}. Requires a loader/kernel supporting
+{binding['format']} version {3 if args.codebook_scope=='expert' else 2}: 512 codewords per pair,
+64 uint32 index words per tile and 8+8 index extraction. The prior 8+7 loader is
+incompatible. This pipeline does not modify the serving fork.
+Offline packing/structure checks passed; serving and model-level quality are untested.
+''')
+  shutil.copy2(ROOT/'arvq88/pack.py',ck/'arvq88_reference.py')
+  if args.pv_steps:
+   from .pv_campaign import write_card
+   write_card(args,work)
+   from .final_publish import publish
+  write(work/'complete.json',{'checkpoint':str(ck),'uploaded':False})
+  if args.do_upload:print('UPLOADED',publish(args,hub,work),flush=True)
+  else:print('Checkpoint ready:',ck,'; add --do-upload to publish',flush=True)
+ except BaseException:
+  (work/'run.failed').write_text(traceback.format_exc());raise

reproduce/source/btx53/arvq88/docs/expert_experiments.md

@@ -0,0 +1,76 @@
+# Per-expert books, alternating updates, and ARVQ allocation
+
+These changes are experimental. Production v2 defaults remain shared books and
+fixed indices. No new weights are uploaded by the ablation runner.
+
+The serving-agent prompt is in `per_expert_handoff.md`. Version 3 changes only
+the packed codebook tensor from `[512]` to `[E,512]` per layer/projection.
+The paired index tensor, block scales and single projection global scale retain
+the v2 layout. Expert slots follow the ordered cold-expert manifest.
+
+`pv.py --reassign-every 40` interleaves continuous updates with LDLQ discrete
+proposals computed from training activation Hessians. Proposals are accepted
+only if a fixed training subset improves in routed cold-output error. This is
+an alternating optimizer, not a reproduction of the PV-Tuning paper. Best-state
+snapshots include code indices, so validation selection and audit fallback
+restore complete models, not just codebooks. Rare experts with fewer than 32
+training rows are skipped during discrete proposals rather than using held-out
+rows. This can limit improvement for rare experts.
+
+`fit.py` accepts `codebook_scope: expert` in run_config.json and learns separate
+books for each expert with a common projection global scale. Fitting filters
+merged captures to `pv_split == 0` when labels are present.
+
+Run the controlled pilot:
+
+```bash
+LD_LIBRARY_PATH=/tmp/compat13/usr/local/cuda-13.2/compat \
+  .venv/bin/python -u btx53/arvq88/experiment.py \
+  --base /tmp/glm53-vision-trace-refit \
+  --workdir /tmp/glm53-arvq-expert-ablation --layers 3,40,70
+```
+
+This waits for all selected GPUs to be free, fits independent books, then runs
+shared-fixed/shared-alternating/expert-fixed/expert-alternating variants with
+identical source, split captures, expert selection and continuous step budgets.
+Alternating variants have additional discrete-search work; this is not an
+iso-compute benchmark. Outputs are comparison.json and packed/<variant>/ shards.
+The original shared fit is reused as a baseline; the per-expert fit is fresh.
+No production promotion is automatic. Native serving parity and task evaluation
+are owned by the serving agent. The reused audit corpus is not a new independent
+benchmark; do not tune repeatedly against audit results.
+
+ARVQ allocation is available through these new pipeline arguments:
+
+```bash
+.venv/bin/python btx53/pipeline.py \
+  --src zai-org/GLM-5.3 --dst jarrelscy/GLM-5.3-Vision-NVFP4-ARVQ-hybrid \
+  --workdir /tmp/glm53-arvq-expert-full \
+  --calib /tmp/glm53-vision-trace-refit/trace_capture/acts \
+  --calib-tokens /tmp/glm53-vision-trace-refit/train/tokens \
+  --codebook-scope expert --pv-reassign-every 40 --reap-metric arvq \
+  --mm-acts /tmp/glm53-vision-trace-refit/capture-mm-rlianv4b/acts_mm
+```
+
+Run a full campaign only after the pilot and serving tests warrant it; the above
+command is documented, not launched. It fits candidate ARVQ weights for all 256
+experts in all 75 layers, fetches missing original FP8 experts, scores candidates
+on both text and multimodal captures, then selects exactly 5750 hot experts using
+the 75/25 layer-normalized blend, floor 8, cap 176. The final initial weights are
+an exact subset of those candidates; allocation is followed by alternating PV.
+
+For each expert the new score is:
+
+    max(0, sum_t g[t,e]^2 * (||ARVQ_e(x_t)-FP8_e(x_t)||^2
+                            - ||NVFP4_e(x_t)-FP8_e(x_t)||^2))
+
+Both quantized candidates use four-plane / FP16-boundary emulation, not native
+SM120 MMA. The score measures the local benefit of retaining the NVFP4 expert,
+with no AQLM error proxy. It omits cross-expert error covariance, routing changes,
+and downstream error accumulation. Both modalities are required; an empty or
+nonpositive modality layer fails instead of silently using text-only allocation.
+
+The initial implementation does not use full-model cross-entropy/distillation
+and does not establish end-to-end quality. The old corpus captures use fixed
+teacher routing and 2048-token windows. Independent reasoning/task evaluations
+remain necessary even if all layer audits pass.

reproduce/source/btx53/arvq88/docs/gradient_pv_recipe.md

@@ -0,0 +1,180 @@
+# Proposed Vision ARVQ v3 tuning recipe
+
+Prepared 18 September 2026 (Melbourne). Status: plan, not a production restart.
+The prior full-run driver and publisher were stopped at the user's request.
+Initial fits, REAP allocation, local results, and published weights are preserved.
+
+## Findings verified against code and logs
+
+- Claude transcript `d7bce691-1871-46a6-af7a-61eae60194fc.jsonl`,
+  2026-09-01T22:31:07Z: AQLM converge optimized Hessian-weighted weight error;
+  subsequent PV optimized layer output MSE with fixed indices.
+- `tools/capture53/pv_tune53.py` inherits `tools/pv_tune.py`: per-layer,
+  expert-coordinate gradient updates; cached routing; NVFP4-dequant teacher;
+  validation best restore. Its `reseed()` body is a no-op. This is not the
+  full-model algorithm in the PV-Tuning paper.
+- `PLAN53.md` and `tools/capture53/eval_hybrid53.py`: the old pipeline also
+  performed full-model, layer-streamed perplexity evaluation. The old evaluator
+  decodes AQLM and needs an ARVQ v3 loader and arithmetic changes.
+- Current ARVQ trains all codebooks and per-128-weight scales. Per expert:
+  8,192 dictionary values + 294,912 scales; 13,450 cold experts expose
+  3,966,566,400 scale values. Low storage precision is not immunity to overfit.
+- Current merged capture: 98,304 rows/layer, 32,768 each for train/val/audit.
+  It contains x, topk_ids, topk_weights, pv_split, but no per-row prompt IDs.
+- Current objective matches cold FP8 contributions only. The old AQLM code
+  explicitly subtracted frozen student-hot contributions from the full target.
+  With FP8 reference versus NVFP4 hot weights these are not interchangeable.
+- New sparse gradient prototype passes the 42-test package suite, including
+  gradient direction, trust limit, training-row isolation and improvement tests.
+  A real layer-3/two-expert/two-step smoke completed packing/reload audit, but
+  rejected all 8 expert/projection proposals. This is not quality qualification.
+
+## Fixed scope and baselines
+
+Preserve expert-specific books, v3 serialization, original FP8 weight source,
+current 5,750/13,450 REAP allocation, and existing corpus. No refitting/re-download
+is necessary for this optimizer change. Preserve all old PV artifacts, including
+12 previously published layers. Compare proposed replacements against both the
+initial fit and the currently published layer. If both are candidates, select on
+validation and reserve the new final audit for the frozen recipe.
+
+This remains layer-local alternating optimization, not formal EM and not a
+claim to reproduce full-model published PV-Tuning.
+
+## Phase 1: data and trustworthy objective
+
+1. Recapture or expand from the same training problems to an initial target of
+   131,072 representative training rows/layer. Keep separate prompt-defined
+   validation; retain prompt ID, token position, domain, modality and capture
+   provenance per row. Use reservoir/stratified sampling across conversations,
+   not the first rows or unrestricted token-level resplitting.
+2. Log routed token counts AND distinct prompt counts for every expert. Tentative
+   minimum for expert-specific discrete tuning: 256 training routes from at least
+   16 prompts; below this freeze indices and scale corrections, and initially
+   freeze its books too. Pilot coverage decides whether more capture is needed.
+3. Include multimodal TRAINING captures, not just multimodal REAP scores. Start
+   with the existing 75/25 text/MM mixture as a hypothesis, report both domains
+   separately, and do not treat previously allocation-used MM rows as held-out.
+4. Build full routed FP8 reference outputs, subtract frozen student-hot output,
+   then fit the cold sum to that residual. Report both full routed error and
+   cold-only error. Match actual hot arithmetic or clearly label the emulator.
+5. Normalize training MSE with a fixed training-set reference energy rather than
+   changing denominators per minibatch. Preserve natural routing weights;
+   expert-enriched samples require sampling corrections to avoid silently
+   changing the deployment objective.
+6. Audit forward arithmetic against the serving commit reported as b8db885d9,
+   including routing weights, intermediate rounding, scale layout, cold slots,
+   accumulation and TP reductions. A100 surrogate gradients are acceptable;
+   native forward parity needs the SM120 environment, not a claim of bit parity.
+
+## Phase 2: constrained continuous updates
+
+- Start with books + regularized per-output-row multiplicative scale corrections.
+  Fold corrections into the existing E4M3 per-128 scales at export: no format or
+  kernel change. Freeze projection-global scales to remove avoidable scale
+  ambiguity. This exposes 10,240 row corrections per expert instead of 294,912
+  independent block-scale adjustments.
+- Use FP32 latent parameters, FP4/E4M3 quantized forward and STE backward.
+- Penalize drift from initial quantized weights/books and log-scale corrections;
+  use stronger shrinkage for poorly supported experts. Do not describe this as
+  a guarantee. The tiny prototype's uniform latent anchor is only scaffolding.
+- Pilot conservative book learning rates around 3e-4 versus the prior 3e-3,
+  and row-log-scale LR around 1e-4, with warmup then decay. These are starting
+  hypotheses, not transferred AQLM hyperparameters: parameter units differ.
+- Effective batch 1,024 through four 256-row microbatches; measure wall time.
+  Unfreeze individual block scales only if a later bounded ablation establishes
+  an independent-validation benefit. Capacity should be earned by evidence.
+
+## Phase 3: output-gradient discrete updates
+
+- After a short continuous warmup, propose every 20 continuous updates initially.
+- Compute gradients of the SAME routed-output objective with respect to one
+  expert's decoded gate/up or down matrix at a time, with other experts fixed.
+  The full routed residual retains interactions between experts.
+- Precondition/damp the gradient and construct a temporary nearby target weight.
+  Search code pairs against this gradient-stepped target, NOT original FP8
+  matrices. Weight-space projection itself is legitimate: the target matters.
+- Use beam search over both books (small beam e.g. 4, retain current pair),
+  shortlist by predicted improvement and consider at most 0.1% of groups per
+  expert/projection initially. Hard ceiling 1%; also bound decoded weight change.
+- Backtrack the target step and/or number of changed groups when actual loss
+  worsens. Raw gradient direction plus a global norm constraint was insufficient
+  in the smoke test. No forced replacements and no all-layer all-or-nothing sweep.
+- Accept each expert/projection separately using actual quantized forward loss
+  on proposal-training rows and a second rotating training check batch. Never
+  query validation to accept individual proposals. Both batches remain training.
+- Exploit the down projection's linearity while gate/up is fixed: for a candidate
+  compute its routed output delta and exact squared-loss change
+  `2 * <residual, delta_y> + ||delta_y||^2`. This is a more faithful ranking
+  than weight distance. Batched changes still require a combined check because
+  their output deltas interact. Gate/up proposals require nonlinear forward checks.
+- For gate/up changes recompute downstream activations. Following acceptance,
+  refresh gradients/residuals before the next projection. Restore codes and any
+  affected optimizer state on rollback; codebooks/scales are fixed in this step.
+- Stream one expert/projection's dense gradient/temporary target. Do not allocate
+  persistent dense Adam moments or shadow weights for all cold experts: those
+  are impractical at this model size. Sparse/factorized state needs memory tests.
+
+## Phase 4: stopping, validation and final audit
+
+- Report fixed training-probe and validation loss every 25-32 continuous steps,
+  by domain and at prompt level. Include accepted proposals, actual/predicted
+  improvement, quantized value changes, and coverage. Batch loss alone is not
+  evidence of a train/validation gap.
+- First pilot uses 200 steps for a matched comparison. For the chosen full recipe
+  use a bounded 2-4 passes over the expanded training cache, rather than assuming
+  200 steps is convergence. Always retain the best validation state.
+- Do not stop for a short plateau. After >=2 passes, require sustained training
+  improvement AND validation deterioration over >=3 checks, exceeding uncertainty
+  from paired prompt-level resampling. Separate compute-budget exhaustion and
+  plateau from demonstrated overfitting. Small fixed percentage thresholds in the
+  prototype are provisional, not statistical proof.
+- Existing audit data has already influenced several accepted runs. Build a new
+  disjoint final audit and do not inspect it during optimizer/recipe selection.
+  One final pass/fail; do not tune until that audit passes.
+
+## Phase 5: pilot, rollout and full-model verification
+
+1. Three full representative layers (3,40,70), baseline initial/current weights.
+2. Compare continuous-only against continuous + gradient-guided indices with
+   identical data, scale parameterization, regularization and step budget. This
+   isolates whether discrete optimization actually helps; no shared-book trials.
+3. Qualify on held-out error including text/MM slices, actual serialized forward,
+   proposal improvement, no malformed tensors, deterministic rollback and peak
+   memory/runtime. Rejection rate alone is neither success nor failure.
+4. Launch all 75 only after the pilot demonstrates a held-out benefit or useful
+   nonregression at realistic cost. Incremental uploads retain truthful recipe
+   provenance and current validation status; no stale old-run result reuse.
+5. Port the existing streamed evaluator to ARVQ v3. Compare NVFP4 donor, initial
+   per-expert checkpoint, and final candidate on identical held-out/neutral text
+   and multimodal checks. Weight-only decoded evaluation is a diagnostic; the
+   activation-quantized forward and native SM120 tests remain required.
+6. Measure teacher-input versus student-input activation/routing drift. If local
+   improvements fail to transfer, use one sequential refinement sweep: feed each
+   layer with already-quantized upstream student activations, pair with reference
+   trajectories and recompute appropriate routing/targets. Preserve held-out
+   prompt separation. Do not recapture the teacher and label it student capture.
+7. Consider short multi-layer reconstruction or full-model KL distillation only
+   if measured cross-layer drift justifies it and a memory/runtime prototype fits.
+   Independent layer tuning cannot be claimed to minimize end-to-end loss.
+
+## Code scope
+
+- `gradient_indices.py`: damped/beam proposals, backtracking, two-batch acceptance,
+  memory-bounded state, exact per-expert rollback and diagnostics. Prototype exists.
+- `pv.py`: target residual, restricted scale parameterization, fixed normalization,
+  gradient accumulation, regularization, uncertainty-aware stopping. Partial
+  prototype exists; not approved as the final recipe by its smoke result.
+- New `pv_data.py` / capture extension: prompt provenance, coverage and multimodal
+  partitions, larger representative reservoirs, student-trajectory capture.
+- `validation.py`: per-prompt paired metrics, separate overfit versus plateau,
+  fresh final audit and baseline comparisons.
+- `pv_campaign.py`: recipe hash, unique workdir, pilot selection and true optimizer
+  resume. Restart from initial fits by default; keep old tuned artifacts intact.
+- `incremental_publish.py` / `write_card`: truthful optimizer/coverage/stopping
+  provenance and mixed-revision status; no old Hessian method described as new PV.
+- New `eval_arvq53.py`: adapt `tools/capture53/eval_hybrid53.py`, with explicit
+  weight-only versus activation-emulated modes and serving parity fixtures.
+
+No new completion ETA until the selected full-layer pilot measures iteration,
+proposal, validation and upload costs. All user-facing times: Melbourne.

reproduce/source/btx53/arvq88/docs/per_expert_handoff.md

@@ -0,0 +1,78 @@
+Implement per-expert ARVQ 8+8 codebook support in the active GLM serving fork.
+
+Coordinate with the fitting agent in /home/coder/git/glm52. That agent owns fitting,
+alternating index updates, ARVQ-specific REAP scoring, and checkpoint export. You own
+the serving loader/kernel and actual SM120 tests. Do not duplicate fitter changes.
+
+Context
+- Existing 8+8 checkpoints share two FP4 books per (layer, projection).
+- New experiments give each cold expert its own pair of 256 x 8 FP4 books.
+- gate/up (w13) share a pair; down (w2) has a separate pair.
+- New tensor layout is an explicit version 3 format. Existing version 2 must keep
+  working unchanged. The stale /tmp/arvq-inspect2 checkout only supported 8+7 at
+  inspection time: work from the actual current 8+8 serving branch, not that clone.
+- No new production checkpoint is ready yet. Start with synthetic parity fixtures.
+
+Exact proposed format contract (fitter will export this)
+config.quantization_config.arvq, also under text_config when present:
+  format: rvq256_256x8_expert
+  version: 3
+  codebook_scope: expert
+  codebook_sizes: [256, 256]
+  activation_planes: 4
+  weight_scale_group: 128
+
+For each layer L, prefix model.layers.L.mlp.experts.arvq_{w13|w2}_:
+  packed:    uint32 [E, N/16, K/64, 64]  (identical to v2)
+  codebooks: uint32 [E, 512]              (NEW expert axis)
+  scales:    uint8  [E, N/16, K/128, 16]  (identical to v2)
+  global:    float32 [1]                 (identical to v2; not per expert)
+E is the number of cold experts in this layer.
+w13: N=4096, K=6144. w2: N=6144, K=2048.
+Each book row packs 8 FP4 E2M1 values into one uint32: component j occupies
+bits [4*j,4*j+3]. Codes 0..7 represent [0,.5,1,1.5,2,3,4,6], bit 3 is sign.
+Rows 0..255 are book 0; rows 256..511 are book 1.
+The representation is W[e,r,k] = global * scale[e,r,k//128] *
+  (C[e,a[e,r,k//8],k%8] + C[e,256+b[e,r,k//8],k%8]).
+Index pairs retain a | (b << 8), packed in the existing v2 MMA-fragment order.
+Use the existing v2 row/column packing, not a new natural-order interpretation.
+See btx53/arvq88/pack.py for the exporter/reference decoder.
+
+Critical expert mapping
+- Book row e is the COLD SLOT, matching packed[e], scales[e], and the ordered
+  cold_expert_ids manifest. It is not the original global expert ID.
+- TP shards retain the full local codebook pair for every locally represented
+  cold expert; books are not sliced along input/output axes. Existing N/K index
+  and scale sharding remains unchanged.
+- EP/expert reordering must reorder codebooks with the other expert tensors.
+- Preserve the shared [512] v2 codebook path. Reject incompatible format/shape
+  combinations rather than silently interpreting expert books as shared.
+
+Kernel changes
+- Select cb + cold_slot * 512 for v3; shared cb base for v2.
+- Keep FP4 two-book MMA algebra and four activation planes unchanged.
+- Audit all direct, LUT, fused, prefill/decode, and CUDA-graph paths. A lookup table
+  computed once from layer-shared books cannot be reused across different experts.
+  Make caches/LUT keys include the relevant expert or dispatch to a correct path.
+- Account for any shared-memory sizing that assumed exactly one shared book pair.
+- No additional per-weight index bits are required; actual throughput/cache effects
+  must be measured, not inferred from unchanged MMA count.
+
+Required tests before claiming correctness
+1. Two experts with identical indices/scales and deliberately DIFFERENT books must
+   decode to different known outputs. Exercise permuted original expert IDs.
+2. Compare packed decode to an independent CPU oracle, including a,b=0 and 255,
+   negative FP4 values, extreme valid scales, both projections, and TP slicing.
+3. Duplicated per-expert books must agree with the existing shared-book path within
+   the same kernel's numerical tolerance; test full MoE outputs and routing weights.
+4. On SM120 compare gate/up outputs, FP16 SwiGLU, down outputs, and summed MoE
+   outputs to the FP4-plane reference on identical inputs. Report max abs and
+   relative L2 discrepancies for TP1 and actual deployment TP.
+5. Test both decode and prefill, graphs on/off, expert permutations, and LUT/direct
+   dispatch. Benchmark latency/throughput and memory against version 2.
+6. Run the actual failing model tests once the fitting agent supplies candidate
+   weights. Offline layer audit success is not evidence of end-to-end quality.
+
+Scope: implement and test serving support; return commit/patch, commands, parity
+results, and benchmark results. Do not upload, overwrite, or deploy a production
+checkpoint. Report hardware-dependent checks you cannot run as untested.

reproduce/source/btx53/arvq88/docs/sequential_pilot.md

@@ -0,0 +1,65 @@
+# Two-block reference-trajectory pilot
+
+Work directory: `/tmp/glm53-sequential-pilot`.
+Driver: `python btx53/arvq88/sequential_pilot.py --work /tmp/glm53-sequential-pilot`.
+No Hugging Face publication is performed by this pilot.
+
+The first routed layers are 3 and 4. Frozen donor layers 0–2 generate the prefix.
+Both reference and student use this prefix and unchanged donor attention, router,
+shared experts and norms. Reference routed experts in layers 3/4 come from the
+original FP8 source. This is a controlled hybrid reference, not a claim to run
+all-original FP8 nonexpert weights or native SM120 kernels.
+
+Capture uses 64 training, 16 validation, 16 development-audit windows of 1,024
+ tokens, sampled deterministically from the existing prompt-separated corpus.
+Within-split windows are not guaranteed to represent distinct prompts. The
+historical audit split is development data, not a fresh untouched final test.
+Attention is dense-equivalent at this context length (<= index_topk).
+
+Layer 3 has identical same-input/reference-trajectory targets and is trained once.
+Its retained serialized outputs become the student's layer-4 inputs. The
+reference layer-3 outputs independently propagate into reference layer 4.
+Layer 4 then receives two matched 200-step runs from identical initial weights:
+
+- `same_input`: FP8 reference experts on the student's own block input.
+- `reference`: the original reference trajectory's complete block output.
+
+Both subtract the frozen student's residual/attention/shared/hot contributions
+before fitting the cold sum. Loss includes the complete block residual mismatch.
+The same fixed loss normalization, training minibatches, seed, optimizer settings,
+validation split, allocation and initial codebooks are used in both arms.
+
+Eight GPUs own disjoint cold-expert subsets within each layer. Their partial
+routed outputs are summed, and each rank differentiates the same coupled loss
+with respect to its local experts. No averaging of local expert targets replaces
+this objective. Continuous updates use effective batch 1,024 in four microbatches.
+Books use LR 3e-4, row-log-scale corrections 1e-4, and an initialization anchor.
+Block scales and projection globals are frozen; row corrections fold into the
+existing E4M3 scale layout. Experts with <256 routed training rows are frozen.
+
+Every 20 steps, expert-local output gradients propose sparse code changes.
+Per expert/projection proposals are generated concurrently; their output deltas
+are then accepted in deterministic rank order against the up-to-date coupled
+residual. Prefix backtracking tries smaller changes. Acceptance requires lower
+error on the proposal-training batch and nonregression on a separate training
+check batch. Validation never accepts individual proposals.
+
+Quantized forward emulates FP4 cold books, E4M3 weight scales, four activation
+planes and FP16 SwiGLU boundaries. Hot experts use NVFP4-dequantized weights with
+float32 GEMMs. Cross-GPU sums and donor BF16 backbone are surrogates for native
+serving numerics; record this limitation with results.
+
+Validation every 25 steps retains the best state. Budget is 200 steps. Early
+stopping is allowed only after >=150 steps and 3 consecutive checks where
+training improves >=0.5% while validation worsens >=0.5% versus the validation
+best. This is a conservative heuristic, not statistical proof of overfitting.
+
+`comparison.json` compares both layer-4 candidates against the SAME reference
+trajectory on held-out windows; includes a paired-window bootstrap diagnostic.
+Output errors are full-block relative L2, so they are not numerically comparable
+to earlier cold-contribution relative errors (~0.2–0.3).
+
+Preflight: 8-GPU one-step layer-3 smoke completed; 304/388 proposals accepted,
+118,355 groups changed. Held-out full-block relative L2 0.0054971166 ->
+0.0054560048. Saved/reloaded forward matched. These are smoke measurements,
+not results of the 200-step target comparison.

reproduce/source/btx53/arvq88/encoder.py

@@ -0,0 +1,321 @@
+"""Activation-Hessian-tuned ARVQ fit for one (layer, projection).
+
+Codebooks (c0[256,8], c1[256,8], FP4-constrained) are SHARED across all cold
+experts of a (layer, projection), so this is a joint per-layer fit:
+
+  1. per expert: raw-basis Hessian H_e, escalating-damp Cholesky hinv_e,
+     per-(row,128block) init scale s_e, global scalar (fixed).
+  2. codebook EM on a Hessian-importance-weighted subsample of normalized
+     8-dim weight groups (k-means warm start -> alternating refine, FP4 project
+     after every M-step).  <- "fit codebooks to data" (a)
+  3. per expert final assignment via a GPTQ/LDLQ error-feedback column sweep
+     with the fixed codebooks (plain-L2 inner assignment; cross-group feedback
+     carries the off-diagonal Hessian), then LS refit of the E4M3 scales.
+     <- "Hessian-aware code selection + error feedback" (b)
+
+No incoherence rotation: Hessians and targets are in the raw weight basis
+(the SM120 kernel feeds raw activations). Weighting is in the EM importance +
+error feedback + scale LS, never a sqrt(diag(H)) whitening (the FP4 grid on the
+codebook forbids rescaling the target space).
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+import torch
+import torch.nn.functional as F
+
+from arvqprep.pack import project_to_fp4
+
+
+# --------------------------------------------------------------- Hessians
+def hessian_fc1(x: torch.Tensor) -> torch.Tensor:
+    """gate/up share E[x x^T] over the H axis. x: [T, H] (routed tokens)."""
+    x = x.float()
+    return (x.t() @ x) / max(x.shape[0], 1)
+
+
+def hessian_fc2(x: torch.Tensor, wg: torch.Tensor, wu: torch.Tensor) -> torch.Tensor:
+    """down E[m m^T], m = silu(x Wg^T) * (x Wu^T) over the I axis. x:[T,H]."""
+    x = x.float()
+    m = F.silu(x @ wg.float().t()) * (x @ wu.float().t())    # [T, I]
+    return (m.t() @ m) / max(x.shape[0], 1)
+
+
+def cholesky_inv_upper(hess: torch.Tensor, damp_start: float = 1e-3) -> torch.Tensor:
+    """Escalating-damp upper Cholesky of H^-1 (same recipe as btxprep/encoder)."""
+    k = hess.shape[0]
+    mean_diag = hess.diagonal().mean().clamp(min=1e-12)
+    damp = damp_start
+    eye = torch.eye(k, device=hess.device, dtype=hess.dtype)
+    for _ in range(12):
+        try:
+            return torch.linalg.cholesky(
+                torch.linalg.inv(hess + damp * mean_diag * eye), upper=True)
+        except Exception:  # noqa: BLE001
+            damp *= 4.0
+    raise RuntimeError("ARVQ Cholesky failed at maximum damp")
+
+
+# ---------------------------------------------------------- codebook EM
+def _nearest(x, c):
+    """argmin_k ||x - c_k||^2 (plain L2). x:[M,8], c:[K,8] -> [M]."""
+    return (c.square().sum(1)[None, :] - 2 * x @ c.t()).argmin(1)
+
+
+def _wmeans(x, w, ids, count, prev):
+    """Weighted per-cluster mean of x (scalar sample weights w); keep prev if empty."""
+    sums = torch.zeros_like(prev)
+    sums.index_add_(0, ids, x * w[:, None])
+    wsum = torch.zeros(count, device=x.device, dtype=x.dtype)
+    wsum.index_add_(0, ids, w)
+    return torch.where(wsum[:, None] > 0, sums / wsum.clamp_min(1e-20)[:, None], prev)
+
+
+def fit_codebooks(x, w, *, iters=8, seed=0):
+    """Weighted FP4 additive-RVQ codebook fit on a subsample.
+    x:[M,8] normalized group vectors, w:[M] importance weights.
+    Returns c0[256,8], c1[256,8] (FP4-on-grid), best weighted relative-L2."""
+    g = torch.Generator(device=x.device).manual_seed(seed)
+    perm = torch.randperm(x.shape[0], generator=g, device=x.device)
+    c0 = project_to_fp4(x[perm[:256]].clone())
+    i = _nearest(x, c0)
+    c1 = project_to_fp4(_kmeans(x - c0[i], 256, g))
+    j = _nearest(x - c0[i], c1)
+    wsum = (w * x.square().sum(1)).sum().clamp_min(1e-20)
+    best = None
+    for _ in range(iters):
+        i = _nearest(x - c1[j], c0)
+        c0 = project_to_fp4(_wmeans(x - c1[j], w, i, 256, c0))
+        j = _nearest(x - c0[i], c1)
+        c1 = project_to_fp4(_wmeans(x - c0[i], w, j, 256, c1))
+        err = ((w * (x - c0[i] - c1[j]).square().sum(1)).sum() / wsum).item()
+        if best is None or err < best[0]:
+            best = (err, c0.clone(), c1.clone())
+    err, c0, c1 = best
+    return c0, c1, err ** 0.5
+
+
+def _kmeans(x, count, gen, iters=12):
+    c = x[torch.randperm(x.shape[0], generator=gen, device=x.device)[:count]].clone()
+    for _ in range(iters):
+        ids = _nearest(x, c)
+        sums = torch.zeros_like(c)
+        sums.index_add_(0, ids, x)
+        cnt = torch.bincount(ids, minlength=count).float().clamp_min(1)[:, None]
+        c = torch.where(torch.bincount(ids, minlength=count)[:, None] > 0, sums / cnt, c)
+    return c
+
+
+# ----------------------------------------------------- per-expert sweep
+def _assign(tn, c0, c0n, c1, c1n, refine):
+    """Plain-L2 additive assignment. tn:[N,8]. Returns a,b (long)."""
+    a = (c0n[None, :] - 2 * tn @ c0.t()).argmin(1)
+    r = tn - c0[a]
+    b = (c1n[None, :] - 2 * r @ c1.t()).argmin(1)
+    for _ in range(refine):
+        r = tn - c1[b]
+        a = (c0n[None, :] - 2 * r @ c0.t()).argmin(1)
+        r = tn - c0[a]
+        b = (c1n[None, :] - 2 * r @ c1.t()).argmin(1)
+    return a, b
+
+
+def sweep_expert(W, hinv, c0, c1, s, glob, *, col_block, refine):
+    """LDLQ error-feedback assignment for one expert.
+
+    W:[N,K] fp32 target; hinv:[K,K] upper fp32; s:[N,K/128] fp32 (E4M3-valued)
+    scale; glob: scalar. Returns a,b uint8 [N,K/8] and reconstruction q [N,K]."""
+    N, K = W.shape
+    w = W.clone()
+    q = torch.empty_like(w)
+    a_full = torch.empty(N, K // 8, dtype=torch.uint8, device=W.device)
+    b_full = torch.empty(N, K // 8, dtype=torch.uint8, device=W.device)
+    c0n = c0.square().sum(1)
+    c1n = c1.square().sum(1)
+    for j0 in range(0, K, col_block):
+        j1 = min(j0 + col_block, K)
+        blk = w[:, j0:j1]
+        qhat = torch.empty_like(blk)
+        for gg in range((j1 - j0) // 8):
+            c = j0 + gg * 8
+            sden = (glob * s[:, c // 128]).clamp_min(1e-20)[:, None]      # [N,1]
+            tn = blk[:, gg * 8:gg * 8 + 8] / sden
+            a, b = _assign(tn, c0, c0n, c1, c1n, refine)
+            qhat[:, gg * 8:gg * 8 + 8] = sden * (c0[a] + c1[b])
+            a_full[:, c // 8] = a.to(torch.uint8)
+            b_full[:, c // 8] = b.to(torch.uint8)
+        q[:, j0:j1] = qhat
+        err = (blk - qhat) @ torch.linalg.inv(hinv[j0:j1, j0:j1])
+        if j1 < K:
+            w[:, j1:] -= err @ hinv[j0:j1, j1:]
+    return a_full, b_full, q
+
+
+def refit_scales(W, a, b, c0, c1, glob, hdiag, K):
+    """LS per-(row,128block) scale given codes; E4M3-quantized. Returns [N,K/128]."""
+    N = W.shape[0]
+    u = (c0[a.long()] + c1[b.long()]).reshape(N, K // 8, 8) * glob         # unit recon
+    Wg = W.reshape(N, K // 8, 8)
+    hd = hdiag.reshape(K // 8, 8)[None]                                    # [1,K/8,8]
+    num = (hd * Wg * u).reshape(N, K // 128, 128).sum(-1)
+    den = (hd * u * u).reshape(N, K // 128, 128).sum(-1).clamp_min(1e-20)
+    s = (num / den)
+    s = s.clamp_min(0).to(torch.float8_e4m3fn).float()
+    return s
+
+
+@dataclass
+class EncodedLayerProj:
+    c0: torch.Tensor            # [256,8] fp32 (FP4 grid)
+    c1: torch.Tensor            # [256,8] fp32; [E,256,8] for expert scope
+    glob: float
+    a: torch.Tensor            # [E,N,K/8] uint8
+    b: torch.Tensor            # [E,N,K/8] uint8
+    s: torch.Tensor            # [E,N,K/128] fp32 (E4M3-valued)
+    N: int
+    K: int
+    cb_rel_l2: float
+    recon_rel_fro: float       # weight-space, Hessian-weighted, over experts
+    scale_dtype: str = "fp8_e4m3"
+    codebook_dtype: str = "fp4_grid"   # 'fp4_grid' (v2/v3) or 'fp16' (free atoms)
+
+
+def _e4m3(x):
+    return x.to(torch.float8_e4m3fn).float()
+
+
+def fit_layer_projection(W, H, *, col_block=128, cb_iters=8, sweep_passes=2,
+                         refine=1, subsample_per_expert=20000, seed=0,
+                         device="cuda", verbose=False, global_scale=None,
+                         codebook_scope="layer"):
+    """W: list of E weight targets [N,K] (fp16/fp32, CPU or GPU).
+    H: list of E Hessians [K,K] (fp32). Returns EncodedLayerProj."""
+    dev = torch.device(device)
+    E = len(W)
+    N, K = W[0].shape
+    gen = torch.Generator(device=dev).manual_seed(seed)
+    if codebook_scope not in ('layer','expert'):
+        raise ValueError('Unknown codebook scope')
+    if codebook_scope == 'expert':
+        # Retain one global scale per projection so the v2 index/scale layout
+        # and arithmetic stay unchanged. Only the dictionaries gain an axis.
+        if global_scale is None:
+            pool=[]
+            for weight in W:
+                rms=weight.to(dev).float().reshape(N,K//128,128).square().mean(-1).sqrt().flatten()
+                pick=torch.randperm(len(rms),generator=gen,device=dev)[:4096]
+                pool.append(rms[pick])
+            global_scale=float(torch.cat(pool).median().clamp_min(1e-8))
+            del pool,rms
+        results=[]
+        for e in range(E):
+            result=fit_layer_projection([W[e]],[H[e]],col_block=col_block,
+                cb_iters=cb_iters,sweep_passes=sweep_passes,refine=refine,
+                subsample_per_expert=subsample_per_expert,seed=seed+e,
+                device=device,verbose=verbose,global_scale=global_scale)
+            results.append(result)
+        return EncodedLayerProj(torch.stack([r.c0 for r in results]),
+            torch.stack([r.c1 for r in results]),float(global_scale),
+            torch.cat([r.a for r in results]),torch.cat([r.b for r in results]),
+            torch.cat([r.s for r in results]),N,K,
+            sum(r.cb_rel_l2 for r in results)/E,
+            sum(r.recon_rel_fro for r in results)/E)
+
+    # ---- Phase A: hinv, hdiag, init scale, global
+    hinv, hdiag, s = [], [], []
+    rms_pool = []
+    for e in range(E):
+        We = W[e].to(dev).float()
+        hd = H[e].to(dev).float().diagonal().clamp_min(1e-12)
+        hinv.append(cholesky_inv_upper(H[e].to(dev).float()).half())
+        hdiag.append(hd.half())
+        blk_rms = We.reshape(N, K // 128, 128).square().mean(-1).sqrt()    # [N,K/128]
+        s.append(blk_rms)                                                  # tmp (pre-global)
+        rms_pool.append(blk_rms.flatten()[torch.randperm(N * (K // 128),
+                        generator=gen, device=dev)[:4096]])
+        del We
+    glob = (float(torch.cat(rms_pool).median().clamp_min(1e-8).item())
+            if global_scale is None else float(global_scale))
+    if not __import__('math').isfinite(glob) or glob <= 0:
+        raise ValueError('Global scale must be finite and positive')
+    for e in range(E):
+        s[e] = _e4m3(s[e] / glob)                                          # E4M3 scale
+
+    # ---- Phase B: importance-weighted subsample of normalized group vectors
+    xs, ws = [], []
+    for e in range(E):
+        We = W[e].to(dev).float().reshape(N, K // 8, 8)
+        sden = (glob * s[e]).clamp_min(1e-20)                              # [N,K/128]
+        sden = sden.repeat_interleave(16, dim=1)                          # [N,K/8]
+        tn = (We / sden[:, :, None]).reshape(-1, 8)                       # [N*K/8,8]
+        imp = (hdiag[e].float().reshape(K // 8, 8).mean(1)[None, :]
+               * (s[e].repeat_interleave(16, dim=1) ** 2)).reshape(-1)   # [N*K/8]
+        m = tn.shape[0]
+        idx = torch.randperm(m, generator=gen, device=dev)[:subsample_per_expert]
+        xs.append(tn[idx])
+        ws.append(imp[idx])
+        del We, tn, imp
+    xs = torch.cat(xs)
+    ws = torch.cat(ws).clamp_min(1e-12)
+    c0, c1, cb_rel = fit_codebooks(xs, ws, iters=cb_iters, seed=seed)
+    if verbose:
+        print(f"  codebook subsample rel-L2 {cb_rel:.4f} (M={xs.shape[0]})",
+              flush=True)
+    del xs, ws
+
+    # ---- Phase D: per-expert error-feedback sweep + scale refit
+    a_all = torch.empty(E, N, K // 8, dtype=torch.uint8)
+    b_all = torch.empty(E, N, K // 8, dtype=torch.uint8)
+    s_all = torch.empty(E, N, K // 128, dtype=torch.float32)
+    num_w = den_w = 0.0
+    for e in range(E):
+        We = W[e].to(dev).float()
+        hv = hinv[e].float()
+        hd = hdiag[e].float()
+        se = s[e]
+        a = b = q = None
+        for p in range(sweep_passes):
+            a, b, q = sweep_expert(We, hv, c0, c1, se, glob,
+                                   col_block=col_block, refine=refine)
+            if p + 1 < sweep_passes:
+                se = refit_scales(We, a, b, c0, c1, glob, hd, K)
+        se = refit_scales(We, a, b, c0, c1, glob, hd, K)
+        a, b, q = sweep_expert(We, hv, c0, c1, se, glob,
+                               col_block=col_block, refine=refine)
+        a_all[e], b_all[e], s_all[e] = a.cpu(), b.cpu(), se.cpu()
+        d = (We - q)
+        # Half-stored Hessians can lose positive semidefiniteness. Evaluate
+        # against the stabilized positive metric used by the sweep instead:
+        # H_eff^-1 = U.T @ U, hence d H_eff d.T = ||d U^-1||^2.
+        num_w += positive_quadratic(d, hv)
+        den_w += positive_quadratic(We, hv)
+        del We, hv, q, d
+        hinv[e] = None
+    recon = (num_w / max(den_w, 1e-20)) ** 0.5
+    return EncodedLayerProj(c0.cpu(), c1.cpu(), glob, a_all, b_all, s_all,
+                            N, K, cb_rel, recon)
+
+
+def positive_quadratic(weight, inverse_cholesky):
+    """Nonnegative squared norm under the regularized fitting Hessian."""
+    z = torch.linalg.solve_triangular(inverse_cholesky.T, weight.T,
+                                      upper=False)
+    value = float(z.double().square().sum())
+    if not __import__('math').isfinite(value):
+        raise ValueError('Nonfinite regularized Hessian diagnostic')
+    return value
+
+
+def reconstruct(enc: EncodedLayerProj, e: int, device="cpu") -> torch.Tensor:
+    """Decode one expert's weight [N,K] from stored codes/scale/codebooks."""
+    dev = torch.device(device)
+    c0, c1 = enc.c0.to(dev), enc.c1.to(dev)
+    if c0.ndim == 3:
+        c0, c1 = c0[e], c1[e]
+    a = enc.a[e].to(dev).long()
+    b = enc.b[e].to(dev).long()
+    s = enc.s[e].to(dev).repeat_interleave(16, dim=1)                     # [N,K/8]
+    cb = (c0[a] + c1[b])                                                  # [N,K/8,8]
+    W = (enc.glob * s[:, :, None] * cb).reshape(enc.N, enc.K)
+    return W

reproduce/source/btx53/arvq88/experiment.py

@@ -0,0 +1,129 @@
+"""Matched layer ablation for shared/expert books and fixed/alternating indices.
+
+Waits for unoccupied GPUs; never stops another job. Exports local experimental
+weights only. Full-model serving evaluation must precede a production campaign.
+"""
+import argparse
+import fcntl
+import json
+import os
+import shutil
+import subprocess
+import sys
+import time
+import traceback
+from pathlib import Path
+
+ROOT=Path(__file__).resolve().parents[1]
+sys.path[:0]=[str(ROOT),str(ROOT/'tools')]
+from arvq88.inputs import write
+from arvq88.jobs import run_jobs
+from arvq88.pack import export_layer,sha
+from arvq88.activation import ARITHMETIC
+
+
+def summary(work,layers):
+    rows={}
+    for variant in ['shared_fixed','shared_alternating','expert_fixed','expert_alternating']:
+        rows[variant]={}
+        for L in layers:
+            p=work/variant/f'layer_{L:05d}'/'report.json'
+            if not p.exists():continue
+            r=json.loads(p.read_text())
+            rows[variant][str(L)]={k:r[k] for k in ['initial_output_rel','serialized_output_rel','initial_audit_rel','serialized_audit_rel','audit_passed','audit_candidate_accepted','steps_run','codebook_scope','indices_frozen','index_reassignment']}
+    write(work/'comparison.json',{'variants':rows,'scope':'Same cold allocation, corpus splits, source weights and PV budgets; isolated layer ablation. Not full-model task quality.',
+          'allocation_next_step':'Use --reap-metric arvq --mm-acts with all-expert fitted candidates, after serving parity and task evaluation.'})
+
+
+def main():
+    ap=argparse.ArgumentParser(description=__doc__)
+    ap.add_argument('--base',default='/tmp/glm53-vision-trace-refit')
+    ap.add_argument('--workdir',default='/tmp/glm53-arvq-expert-ablation')
+    ap.add_argument('--layers',default='3,40,70');ap.add_argument('--gpus',default='0,1,2,3,4,5,6,7')
+    ap.add_argument('--steps',type=int,default=200);ap.add_argument('--reassign-every',type=int,default=40)
+    ap.add_argument('--check',action='store_true')
+    a=ap.parse_args();base=Path(a.base);work=Path(a.workdir);work.mkdir(parents=True,exist_ok=True)
+    layers=[int(v) for v in a.layers.split(',')];gpus=[int(v) for v in a.gpus.split(',')]
+    if not layers or len(set(layers))!=len(layers) or any(L not in range(3,78) for L in layers):raise ValueError('Invalid layers')
+    if not gpus or len(set(gpus))!=len(gpus) or min(gpus)<0 or a.reassign_every<=0 or a.steps<40:raise ValueError('Invalid run budget')
+    config=json.loads((base/'config.json').read_text())
+    train=base/'train/capture/acts';captures=base/'trace_capture/acts'
+    for L in layers:
+        for p in [train/f'acts_layer{L}.pt',captures/f'acts_layer{L}.pt',base/'initial'/f'layer_{L:05d}/arvq-manifest.json']:
+            if not p.is_file():raise FileNotFoundError(p)
+    identity={'base':str(base),'assignment_sha256':sha(base/'assignment.json'),'source_sha256':sha(base/'source.json'),
+              'layers':layers,'steps':a.steps,'reassign_every':a.reassign_every,'format':'rvq256_256x8_expert',
+              'inputs_sha256':{str(p):sha(p) for L in layers for p in [train/f'acts_layer{L}.pt',captures/f'acts_layer{L}.pt',base/'initial'/f'layer_{L:05d}/arvq-manifest.json']},
+              'code_sha256':{name:sha(ROOT/'arvq88'/name) for name in ['pv.py','alternating.py','encoder.py','pack.py','fit.py']}}
+    if (work/'binding.json').exists() and json.loads((work/'binding.json').read_text())!=identity:
+        raise ValueError('Experiment inputs or code changed; choose a fresh workdir')
+    if a.check:
+        print('PREFLIGHT OK',json.dumps(identity),flush=True);return
+    lock=open(work/'.experiment.lock','a');fcntl.flock(lock,fcntl.LOCK_EX|fcntl.LOCK_NB)
+    write(work/'binding.json',identity)
+    if (work/'complete.json').exists():return
+    def status(stage,**kw):write(work/'status.json',{'stage':stage,'updated_utc':time.strftime('%Y-%m-%dT%H:%M:%SZ',time.gmtime()),**kw});print(stage,json.dumps(kw),flush=True)
+    try:
+        status('waiting_for_free_gpus',gpus=gpus,layers=layers)
+        while True:
+            rows=subprocess.check_output(['nvidia-smi','--query-gpu=index,memory.used','--format=csv,noheader,nounits'],text=True)
+            mem={int(i):int(m) for i,m in (r.split(',') for r in rows.splitlines())}
+            if any(g not in mem for g in gpus):raise ValueError('Selected GPU does not exist')
+            if all(mem[g]<1024 for g in gpus):break
+            time.sleep(30)
+        expert=work/'expert_initial';expert.mkdir(exist_ok=True)
+        for name in ['assignment.json','source.json']:shutil.copy2(base/name,expert/name)
+        config.update(workdir=str(expert),calib=str(train),codebook_scope='expert',do_upload=False)
+        write(expert/'run_config.json',config)
+        status('fitting_per_expert_books',layers=layers)
+        pending=[]
+        for L in layers:
+            folder=expert/'initial'/f'layer_{L:05d}'
+            path=folder/'arvq-manifest.json'
+            if path.exists():
+                m=json.loads(path.read_text());fit_id=m['source_identity']
+                expected=json.loads((expert/'assignment.json').read_text())['layers'][str(L)]['cold_5750']
+                if (m['cold_expert_ids']!=expected or fit_id['capture_sha256']!=sha(train/f'acts_layer{L}.pt')
+                        or fit_id['source']!=json.loads((expert/'source.json').read_text())
+                        or fit_id['settings'].get('codebook_scope')!='expert'):
+                    raise ValueError('Completed fit inputs do not match experiment')
+                for entry in m['files'].values():
+                    if sha(folder/entry['file'])!=entry['sha256']:raise ValueError('Completed fit weights changed')
+                print('REUSE VALIDATED FIT',L,flush=True)
+            else:pending.append(L)
+        run_jobs([(f'fit-{L}',lambda gpu,L=L:[sys.executable,'-u',str(ROOT/'arvq88/fit.py'),str(expert),str(L),f'cuda:{gpu}']) for L in pending],gpus,work/'logs')
+        jobs=[]
+        for variant in ['shared_fixed','shared_alternating','expert_fixed','expert_alternating']:
+            initial=expert/'initial' if variant.startswith('expert') else base/'initial'
+            reassign=a.reassign_every if variant.endswith('alternating') else 0
+            for L in layers:
+                out=work/variant/f'layer_{L:05d}'
+                if (out/'report.json').exists():continue
+                jobs.append((f'{variant}-{L}',lambda gpu,L=L,out=out,initial=initial,reassign=reassign:[
+                    sys.executable,'-u',str(ROOT/'arvq88/pv.py'),'--arvq-dir',str(initial),
+                    '--layer',str(L),'--experts','0','--steps',str(a.steps),'--device',f'cuda:{gpu}',
+                    '--src-cache',config['src_cache'],'--source-meta',str(base/'source.json'),
+                    '--calib',str(captures),'--codebook-lr','0.003','--scale-lr','0.002',
+                    '--arithmetic',ARITHMETIC,'--reassign-every',str(reassign),'--out',str(out)]))
+        status('matched_pv_ablation',variants=4,layers=layers)
+        run_jobs(jobs,gpus,work/'logs')
+        status('exporting_experimental_weights')
+        for variant in ['shared_fixed','shared_alternating','expert_fixed','expert_alternating']:
+            initial=expert/'initial' if variant.startswith('expert') else base/'initial'
+            for L in layers:
+                out=work/variant/f'layer_{L:05d}';r=json.loads((out/'report.json').read_text())
+                manifest=json.loads((initial/f'layer_{L:05d}/arvq-manifest.json').read_text())
+                stage=work/'exports'/variant/f'layer_{L:05d}';stage.mkdir(parents=True,exist_ok=True)
+                for proj in ['w13','w2']:
+                    target=stage/f'{proj}.pt'
+                    if not target.exists():os.link(out/'offline_weights.pt',target)
+                write(stage/'arvq-manifest.json',{**manifest,'fit':variant,'pv_report':r})
+                export_layer(stage,work/'packed'/variant,L)
+        summary(work,layers)
+        write(work/'complete.json',{'uploaded':False,'comparison':str(work/'comparison.json'),'serving_validation':'pending'})
+        status('complete',uploaded=False)
+    except BaseException:
+        (work/'run.failed').write_text(traceback.format_exc());status('failed',report=str(work/'run.failed'));raise
+
+
+if __name__=='__main__':main()

reproduce/source/btx53/arvq88/final_publish.py

@@ -0,0 +1,69 @@
+"""Publish final PV artifacts, verify them, then remove obsolete repo files."""
+import hashlib,json
+from pathlib import Path
+from .pack import sha
+from .inputs import write
+from .validation import require_audit
+
+def inventory(root):
+ result={}
+ for p in sorted(root.rglob('*')):
+  if not p.is_file():continue
+  name=str(p.relative_to(root))
+  # upload_folder ignores .cache/huggingface/ by default, so it never reaches the remote;
+  # including it here would make verify() fail on a file that was intentionally not uploaded.
+  if name.startswith('.cache/'):continue
+  size=p.stat().st_size
+  entry={'size':size,'sha256':sha(p)}
+  if size<10*1024*1024:
+   data=p.read_bytes();entry['blob_id']=hashlib.sha1(f'blob {len(data)}\0'.encode()+data).hexdigest()
+  result[name]=entry
+ return result
+
+def verify(api,repo,revision,expected):
+ files={s.rfilename:s for s in api.model_info(repo,revision=revision,files_metadata=True).siblings}
+ for name,e in expected.items():
+  if name not in files:raise RuntimeError(f'Remote file missing: {name}')
+  s=files[name]
+  if s.size!=e['size']:raise RuntimeError(f'Remote size mismatch: {name}')
+  if s.lfs:
+   if s.lfs.sha256!=e['sha256']:raise RuntimeError(f'Remote SHA mismatch: {name}')
+  elif s.blob_id!=e.get('blob_id'):raise RuntimeError(f'Remote Git blob mismatch: {name}')
+ return files
+
+def publish(args,hub,work):
+ ck=work/'checkpoint';pv=json.loads((work/'pv_report.json').read_text());gates=json.loads((ck/'gates_report.json').read_text())
+ if not pv['passed'] or set(pv['layers_completed'])!=set(range(3,78)) or not gates['passed']:raise RuntimeError('Final PV/gates incomplete')
+ if set(map(int,pv.get('layers',{})))!=set(range(3,78)):raise RuntimeError('Missing per-layer audits')
+ for report in pv['layers'].values():require_audit(report)
+ for name,digest in {**gates['cold_file_sha256'],**gates['metadata_sha256']}.items():
+  if sha(ck/name)!=digest:raise RuntimeError('Checkpoint changed after gates')
+ return publish_verified(args,hub,work,'complete GLM-5.3 NVFP4 / ARVQ 8+8 PV-tuned hybrid')
+
+def publish_verified(args,hub,work,description):
+ from huggingface_hub import CommitOperationDelete
+ ck=work/'checkpoint'
+ expected=inventory(ck);write(work/'final_upload_inventory.json',expected)
+ from huggingface_hub.errors import RepositoryNotFoundError
+ try:before=hub.api.model_info(args.dst,files_metadata=True)
+ except RepositoryNotFoundError:
+  hub.api.create_repo(repo_id=args.dst,repo_type='model',exist_ok=True)
+  before=hub.api.model_info(args.dst,files_metadata=True)
+ write(work/'repo_before_final.json',{'revision':before.sha,'files':[s.rfilename for s in before.siblings]})
+ commit=hub.api.upload_folder(repo_id=args.dst,folder_path=str(ck),revision='main',parent_commit=before.sha,commit_message='Publish '+description)
+ files=verify(hub.api,args.dst,commit.oid,expected)
+ write(work/'final_upload_verified.json',{'revision':commit.oid,'files':len(expected)})
+ revision=commit.oid
+ if args.clean_repo:
+  obsolete=sorted(set(files)-set(expected)-{'.gitattributes'})
+  write(work/'repo_cleanup_plan.json',{'verified_revision':revision,'delete':obsolete,'keep':sorted(expected)})
+  if obsolete:
+   cleanup=hub.api.create_commit(repo_id=args.dst,revision='main',parent_commit=revision,
+    operations=[CommitOperationDelete(path_in_repo=name) for name in obsolete],
+    commit_message='Keep only current 8+8 checkpoint, metadata and validation reports')
+   revision=cleanup.oid
+  remaining=verify(hub.api,args.dst,revision,expected)
+  if set(remaining)-set(expected)-{'.gitattributes'}:raise RuntimeError('Obsolete files remain after cleanup')
+ write(work/'all.done',{'repo':args.dst,'revision':revision,'files_verified':len(expected),'clean_repo':args.clean_repo})
+ print('FINAL PUBLISHED AND VERIFIED',args.dst,revision,flush=True)
+ return revision

reproduce/source/btx53/arvq88/fit.py

@@ -0,0 +1,48 @@
+"""One 8+8 initial-fit worker. Source and cache identity are pinned by the driver."""
+import sys,json,time,fcntl
+from pathlib import Path
+ROOT=Path(__file__).resolve().parents[1];sys.path[:0]=[str(ROOT),str(ROOT/'tools')]
+import torch
+from arvq88 import encoder as enc
+from arvq88.pack import export_layer
+from arvq88.inputs import write
+from arvqprep.workflow import sha256
+from ingest import SourceMeta,load_expert_bf16
+
+def fit(work,L,dev):
+ args=json.loads((work/'run_config.json').read_text());allocation=json.loads((work/'assignment.json').read_text())
+ cold=allocation['layers'][str(L)]['cold_5750'];base=work/'initial'/f'layer_{L:05d}';base.mkdir(parents=True,exist_ok=True)
+ lock=open(base/'.lock','a');fcntl.flock(lock,fcntl.LOCK_EX|fcntl.LOCK_NB)
+ torch.set_num_threads(args['threads']);source=json.loads((work/'source.json').read_text());meta=SourceMeta(**source['meta'])
+ act_path=Path(args['calib'])/f'acts_layer{L}.pt'
+ settings={k:args[k] for k in ['cb_iters','sweep_passes','refine','col_block']}
+ settings['codebook_scope']=args.get('codebook_scope','layer')
+ fingerprint={'source':source,'capture_sha256':sha256(act_path),'cold_ids':cold,'encoder_sha256':sha256(Path(enc.__file__)),'settings':settings}
+ identity=base/'fit_identity.json'
+ if identity.exists() and json.loads(identity.read_text())!=fingerprint:raise ValueError('Cached initial fit inputs/settings changed')
+ if not identity.exists() and any(base.glob('*.pt')):raise ValueError('Unidentified fit cache; use import-cache command or fresh workdir')
+ write(identity,fingerprint)
+ if (base/'arvq-manifest.json').exists():
+  export_layer(base,work/'cold',L);return
+ acts=torch.load(act_path,weights_only=True,map_location='cpu');x=acts['x'].float();ids=acts['topk_ids'].long()
+ if 'pv_split' in acts:
+  train=acts['pv_split']==0;x=x[train];ids=ids[train]
+ if not len(x):raise ValueError('No training activations for initial fit')
+ files={};metrics={}
+ for proj in ['w13','w2']:
+  path=base/f'{proj}.pt';mp=base/f'{proj}.json'
+  if not path.exists() or not mp.exists():
+   W=[];H=[];start=time.time()
+   for eid in cold:
+    wg,wu,wd=[w.to(dev) for w in load_expert_bf16(str(Path(args['src_cache'])/f'layer_{L}'/f'expert_{eid}.safetensors'),meta)]
+    xr=x[(ids==eid).any(1)];xr=(x if len(xr)<32 else xr).to(dev)
+    H.append((enc.hessian_fc1(xr) if proj=='w13' else enc.hessian_fc2(xr,wg,wu)).half())
+    W.append((torch.cat([wg,wu]) if proj=='w13' else wd).half());del wg,wu,wd,xr
+   result=enc.fit_layer_projection(W,H,device=dev,verbose=True,seed=0,**fingerprint['settings'])
+   tmp=path.with_suffix('.tmp');torch.save({proj:vars(result)},tmp);tmp.replace(path)
+   write(mp,{'cb_rel_l2':result.cb_rel_l2,'hess_recon_rel_fro':result.recon_rel_fro,'diagnostic_metric':'regularized fitting Hessian; mean expert relative error for expert scope','seconds':time.time()-start})
+   del result,W,H;torch.cuda.empty_cache()
+  files[proj]={'file':path.name,'sha256':sha256(path)};metrics[proj]=json.loads(mp.read_text())
+ write(base/'arvq-manifest.json',{'layer':L,'cold_expert_ids':cold,'files':files,'per_proj':metrics,'no_rotation':True,'source_identity':fingerprint,'format':'rvq256_256x8'})
+ export_layer(base,work/'cold',L)
+if __name__=='__main__':fit(Path(sys.argv[1]),int(sys.argv[2]),sys.argv[3])

reproduce/source/btx53/arvq88/gate_worker.py

@@ -0,0 +1,5 @@
+import sys
+from pathlib import Path
+root=Path(__file__).resolve().parents[1];sys.path[:0]=[str(root),str(root/'tools')]
+from arvq88.checkpoint import gates
+gates(Path(sys.argv[1]),[int(x) for x in sys.argv[2].split(',')],sys.argv[3],Path(sys.argv[4]))

reproduce/source/btx53/arvq88/gradient_indices.py

@@ -0,0 +1,120 @@
+"""Sparse proximal code proposals driven by routed-output gradients.
+
+This is a layer-local optimizer, not full-model published PV-Tuning. Each
+expert/projection proposal is evaluated against actual training output loss.
+"""
+import math
+import torch
+from .activation import ARITHMETIC, activation_ste, swiglu_ste
+
+METHOD = 'routed_output_gradient_sparse_v1'
+
+@torch.no_grad()
+def propose_codes(p, e, grad, max_fraction=.01, trust_ratio=.01, target_ratio=.1):
+    """Search single-book alternatives near a gradient step; cap count and norm.
+
+The candidate pool is the highest-gradient 4*budget groups. Search all 256
+choices in either book, with the other book fixed. Score using the proximal
+linearization g*delta + ||delta||^2/(2*eta). Original FP8 weights are never used.
+    """
+    w=p.weight(e).detach().reshape(-1,8);g=grad.detach().reshape(-1,8)
+    if not torch.isfinite(g).all():raise ValueError('Nonfinite index gradient')
+    gn=g.norm();wn=w.norm();budget=int(len(w)*max_fraction)
+    if budget<1 or float(gn)==0 or float(wn)==0:return None
+    eta=target_ratio*wn/gn
+    pool=min(len(w),4*budget)
+    selected=g.square().sum(1).topk(pool,sorted=False).indices
+    cb,s,glob=p.quantized(e);cb=cb.detach();factor=(s.detach().repeat_interleave(16,1)*glob.detach()).flatten()
+    old_a=p.codes_a[e].flatten();old_b=p.codes_b[e].flatten()
+    scores=[];aa=[];bb=[];norms=[]
+    for ix in selected.split(2048):
+        a=old_a[ix].long();b=old_b[ix].long();f=factor[ix,None]
+        current=w[ix];target=current-eta*g[ix]
+        # Distances need only [chunk,256], not [chunk,256,8].
+        best=[]
+        for book,other in ((cb[:256],cb[256+b]),(cb[256:],cb[a])):
+            residual=target-f*other
+            distances=f.square()*book.square().sum(1)[None,:]-2*f*(residual@book.T)
+            choice=distances.argmin(1)
+            candidate=f*(book[choice]+other);delta=candidate-current
+            cost=(g[ix]*delta).sum(1)+delta.square().sum(1)/(2*eta)
+            best.append((choice,cost,delta.square().sum(1)))
+        choose_a=best[0][1]<=best[1][1]
+        scores.append(torch.minimum(best[0][1],best[1][1]))
+        aa.append(torch.where(choose_a,best[0][0],a));bb.append(torch.where(choose_a,b,best[1][0]))
+        norms.append(torch.where(choose_a,best[0][2],best[1][2]))
+    score=torch.cat(scores);a=torch.cat(aa);b=torch.cat(bb);delta2=torch.cat(norms)
+    valid=((score<0)&(delta2>0)).nonzero().flatten()
+    if not len(valid):return None
+    order=valid[score[valid].argsort()[:budget]]
+    # Strict trust radius for the reconstructed projection, not latent values.
+    order=order[delta2[order].cumsum(0)<=(trust_ratio*wn).square()]
+    if not len(order):return None
+    ix=selected[order]
+    return {'positions':ix,'a':a[order].to(old_a.dtype),'b':b[order].to(old_b.dtype),
+            'old_a':old_a[ix].clone(),'old_b':old_b[ix].clone(),
+            'predicted_change':float(score[order].sum()),
+            'relative_weight_change':float(delta2[order].sum().sqrt()/wn)}
+
+
+def expert_output(z,w13,w2,arithmetic):
+    if arithmetic==ARITHMETIC:
+        gu=activation_ste(z)@w13.T
+        return activation_ste(swiglu_ste(gu))@w2.T
+    gu=z@w13.T;gate,up=gu.chunk(2,-1)
+    return (torch.nn.functional.silu(gate)*up)@w2.T
+
+
+@torch.no_grad()
+def reassign_indices(p13,p2,x,rows,cold,args,target,prediction,routes):
+    """Sequential expert/projection updates with exact routed residual checks.
+
+Only the provided training rows enter gradients, candidate scoring or acceptance.
+The routed residual includes every cold expert, so gates and cross-expert
+error interactions are included. Validation/audit are inaccessible here.
+    """
+    ref=target[rows];residual=prediction(rows)-ref
+    denominator=ref.square().sum().clamp_min(1e-20)
+    before=float((residual.square().sum()/denominator).sqrt())
+    accepted=proposed=changed=skipped=0;details=[]
+    for e,eid in enumerate(cold):
+        weights=routes[e][rows];use=weights!=0
+        if int(use.sum())<args.reassign_min_rows:skipped+=1;continue
+        z=x[rows[use]];gate=weights[use,None]
+        for p,name in ((p13,'w13'),(p2,'w2')):
+            w13=p13.weight(e).detach();w2=p2.weight(e).detach()
+            with torch.enable_grad():
+                weight=w13 if name=='w13' else w2;weight.requires_grad_(True)
+                old_output=expert_output(z,w13,w2,args.arithmetic)
+                # At current weights this equals the actual coupled output loss.
+                routed=residual[use].detach()+gate*(old_output-old_output.detach())
+                loss=routed.square().sum()/denominator
+                grad,=torch.autograd.grad(loss,weight)
+            old_output=old_output.detach();w13=w13.detach();w2=w2.detach()
+            proposal=propose_codes(p,e,grad,args.reassign_max_fraction,
+                                   args.reassign_trust_ratio,args.reassign_target_ratio)
+            del grad,loss,routed,weight
+            if proposal is None:continue
+            proposed+=1;ix=proposal['positions']
+            a=p.codes_a[e].view(-1);b=p.codes_b[e].view(-1)
+            a[ix]=proposal['a'];b[ix]=proposal['b']
+            try:
+                new_output=expert_output(z,p13.weight(e).detach(),p2.weight(e).detach(),args.arithmetic)
+                candidate=residual[use]+gate*(new_output-old_output)
+                old_loss=float(residual[use].square().sum());new_loss=float(candidate.square().sum())
+                keep=math.isfinite(new_loss) and new_loss<old_loss-max(1e-12,old_loss*1e-7)
+                if keep:residual[use]=candidate;accepted+=1;changed+=len(ix)
+                else:a[ix]=proposal['old_a'];b[ix]=proposal['old_b']
+            except BaseException:
+                a[ix]=proposal['old_a'];b[ix]=proposal['old_b'];raise
+            details.append({'expert':eid,'projection':name,'accepted':keep,'changed_groups':len(ix),
+                            'relative_weight_change':proposal['relative_weight_change'],
+                            'training_before_local':old_loss,'training_candidate_local':new_loss})
+    # Recompute rather than trust accumulated residual arithmetic. Rollback is
+    # handled by the caller's full index snapshot if the net check regresses.
+    after=float(((prediction(rows)-ref).square().sum()/denominator).sqrt())
+    return {'method':METHOD,'training_before':before,'training_candidate':after,
+            'accepted':math.isfinite(after) and after<before,
+            'accepted_proposals':accepted,'proposals':proposed,'changed_groups':changed,
+            'skipped_experts':skipped,'max_changed_fraction':args.reassign_max_fraction,
+            'trust_ratio':args.reassign_trust_ratio,'details':details}

reproduce/source/btx53/arvq88/incremental_publish.py

@@ -0,0 +1,122 @@
+"""Atomic initial v3 checkpoint publication, then audited layer replacements."""
+import json
+import os
+import shutil
+import time
+from pathlib import Path
+from .inputs import write
+from .pack import sha
+from .validation import require_audit
+from .final_publish import inventory,verify
+
+
+def progress_card(repo,completed):
+    return f'''---
+library_name: vllm
+tags: [arvq, nvfp4, experimental]
+---
+# {repo.rsplit('/',1)[-1]}
+
+Complete Vision NVFP4 / per-expert ARVQ 8+8 checkpoint, format
+`rvq256_256x8_expert`, version 3. Requires the per-expert-book serving loader.
+The new ARVQ-based 75% text / 25% multimodal allocation and all initial fits
+are complete. **{len(completed)}/75 layers have been replaced by audited PV
+weights.** Remaining layers retain their initial fits. See pv_progress.json.
+
+Each cold expert has separate two-book FP4 dictionaries for gate/up and down.
+PV emulates four FP4 activation planes and FP16 boundaries, with discrete
+Hessian-aware index proposals every 40 steps. Validation selects the best
+state; audit regression restores the initial fit. Reference source is original
+GLM-5.3 FP8; activations use the saved NVFP4-teacher conversation corpus.
+Hot experts, language backbone and BF16 MTP come from RadixArk NVFP4; vision
+comes from the existing vision hybrid. Indices, cold slots and hot allocation
+are coherent at every published revision. Each layer replacement is atomic.
+
+Offline audits/packing checks are not full-model quality or native-SM120 parity
+benchmarks. Independent-book serving quality validation remains pending.
+'''
+
+
+def commit_snapshot(hub,repo,root,parent,message,delete=()):
+    from huggingface_hub import CommitOperationAdd,CommitOperationDelete
+    expected=inventory(root)
+    operations=[CommitOperationAdd(path_in_repo=name,path_or_fileobj=str(Path(root)/name)) for name in expected]
+    # Upload LFS objects first, but expose all tensor/config changes in ONE commit.
+    hub.api.preupload_lfs_files(repo,operations,revision='main',num_threads=8)
+    result=hub.api.create_commit(repo_id=repo,revision='main',parent_commit=parent,
+        operations=operations+[CommitOperationDelete(path_in_repo=name) for name in delete],
+        commit_message=message,num_threads=8)
+    verify(hub.api,repo,result.oid,expected)
+    return result.oid
+
+
+def publish_initial(args,hub,work):
+    ck=work/'checkpoint';gates=json.loads((ck/'gates_report.json').read_text())
+    if not gates['passed']:raise ValueError('Initial checkpoint gates failed')
+    for name,digest in {**gates['cold_file_sha256'],**gates['metadata_sha256']}.items():
+        if sha(ck/name)!=digest:raise ValueError('Checkpoint changed after initial gates')
+    remote=hub.api.model_info(args.dst,files_metadata=True)
+    local={str(p.relative_to(ck)) for p in ck.rglob('*') if p.is_file()}
+    obsolete=sorted({s.rfilename for s in remote.siblings}-local-{'.gitattributes'})
+    revision=commit_snapshot(hub,args.dst,ck,remote.sha,'Replace with complete per-expert ARVQ v3 initial fit; PV follows',obsolete)
+    state={'repo':args.dst,'revision':revision,'layers':[]}
+    write(work/'incremental_upload.json',state)
+    print('INITIAL CHECKPOINT UPLOADED AND VERIFIED',revision,flush=True)
+    return state
+
+
+def layer_snapshot(work,L,completed,repo):
+    ck=work/'checkpoint'
+    try:m=json.loads((work/'cold'/f'layer-{L:03d}-manifest.json').read_text())
+    except (FileNotFoundError,json.JSONDecodeError):return None
+    r=m.get('pv_report')
+    if not r:return None
+    require_audit(r)
+    if not r.get('passed') or r.get('indices_frozen',True) or r.get('codebook_scope')!='expert':
+        raise ValueError('Unexpected PV recipe for incremental upload')
+    assignment=json.loads((work/'assignment.json').read_text())['layers'][str(L)]['cold_5750']
+    if m['cold_expert_ids']!=assignment or m.get('version')!=3:raise ValueError('Layer allocation/format mismatch')
+    stage=work/'upload_layers'/f'layer_{L:05d}';stage.mkdir(parents=True,exist_ok=True)
+    gates=json.loads((ck/'gates_report.json').read_text())
+    for entry in m['files'].values():
+        source=work/'cold'/entry['file']
+        if sha(source)!=entry['sha256']:raise ValueError('Packed layer hash mismatch')
+        dst=stage/entry['file']
+        if dst.exists():dst.unlink()
+        os.link(source,dst)
+        gates['cold_file_sha256'][entry['file']]=entry['sha256']
+    gates['scope']='Initial structural gates plus incremental packing/parity and PV audits; final full gates pending'
+    write(stage/'gates_report.json',gates)
+    write(stage/'cold_manifests'/f'layer-{L:03d}-manifest.json',m)
+    write(stage/'pv_layers'/f'layer_{L:03d}.json',r)
+    write(stage/'pv_progress.json',{'layers_pv_complete':sorted(completed),'total_layers':75,
+                                  'remaining_layers':'initial fit','format':'rvq256_256x8_expert'})
+    (stage/'README.md').write_text(progress_card(repo,completed))
+    return stage
+
+
+def publish_layers(args,hub,work):
+    statefile=work/'incremental_upload.json'
+    state=json.loads(statefile.read_text()) if statefile.exists() else publish_initial(args,hub,work)
+    done=set(state['layers'])
+    while len(done)<75:
+        if (work/'run.failed').exists():raise RuntimeError('Fit/PV campaign failed; completed uploads retained')
+        changed=False
+        for L in range(3,78):
+            if L in done:continue
+            stage=layer_snapshot(work,L,done|{L},args.dst)
+            if stage is None:continue
+            revision=commit_snapshot(hub,args.dst,stage,state['revision'],f'Replace layer {L} with audited per-expert alternating PV ({len(done)+1}/75)')
+            # Keep local assembly aligned; atomic replaces avoid mutating initial hardlinks.
+            for source in stage.rglob('*'):
+                if not source.is_file():continue
+                dst=work/'checkpoint'/source.relative_to(stage);dst.parent.mkdir(parents=True,exist_ok=True)
+                tmp=dst.with_suffix(dst.suffix+'.replace')
+                if tmp.exists():tmp.unlink()
+                if source.suffix=='.safetensors':os.link(source,tmp)
+                else:shutil.copy2(source,tmp)
+                tmp.replace(dst)
+            done.add(L);state.update(revision=revision,layers=sorted(done));write(statefile,state)
+            print('PV LAYER UPLOADED AND VERIFIED',L,len(done),'/75',revision,flush=True);changed=True
+        if not changed:time.sleep(20)
+    write(work/'incremental_pv.done.json',state)

reproduce/source/btx53/arvq88/inputs.py

@@ -0,0 +1,198 @@
+"""Validated cache reuse, pinned Hub input fetch and REAP allocation."""
+import json,os,shutil,sys,subprocess
+from pathlib import Path
+from concurrent.futures import ThreadPoolExecutor
+import numpy as np
+from .local_source import local_directory,model_index,fingerprint,LocalShard
+from arvqprep.workflow import sha256
+ROOT=Path(__file__).resolve().parents[1]
+LAYERS=list(range(3,78))
+def write(path,obj):
+ path=Path(path);path.parent.mkdir(parents=True,exist_ok=True)
+ tmp=path.with_suffix(path.suffix+'.tmp');tmp.write_text(json.dumps(obj,indent=2));tmp.replace(path)
+def validate_assignment(obj,hot_count=5750):
+ layers=obj.get('layers',obj)
+ if set(map(int,layers))!=set(LAYERS):raise ValueError('Allocation must cover layers 3–77')
+ result={}
+ for L in LAYERS:
+  d=layers[str(L)];cold=d.get('cold_5750',d.get('cold'))
+  if cold is None or len(set(cold))!=len(cold) or any(type(e)!=int or not 0<=e<256 for e in cold):raise ValueError(f'Invalid cold IDs layer {L}')
+  hot=sorted(set(range(256))-set(cold))
+  if 'hot' in d and sorted(d['hot'])!=hot:raise ValueError('Overlapping/missing expert assignment')
+  result[str(L)]={'cold_5750':sorted(cold),'hot':hot}
+ if sum(len(v['hot']) for v in result.values())!=hot_count:raise ValueError('Allocation hot budget mismatch')
+ return {'layers':result}
+def allocate(scores,hot_count=5750,floor=8,cap=176):
+ scores=np.asarray(scores,dtype=np.float64)
+ if scores.shape!=(75,256) or not np.isfinite(scores).all() or (scores<0).any():raise ValueError('Invalid REAP score array')
+ if not 75*floor<=hot_count<=75*cap:raise ValueError('Infeasible hot budget')
+ norm=scores/np.maximum(scores.sum(1,keepdims=True),1e-30)
+ hot=[set(np.argsort(-row,kind='stable')[:floor].tolist()) for row in norm];placed=75*floor
+ for pos in np.argsort(-norm.ravel(),kind='stable'):
+  if placed==hot_count:break
+  l,e=divmod(int(pos),256)
+  if len(hot[l])<cap and e not in hot[l]:hot[l].add(e);placed+=1
+ return validate_assignment({'layers':{str(L):{'cold':sorted(set(range(256))-hot[i])} for i,L in enumerate(LAYERS)}},hot_count)
+def token():
+ p=Path.home()/'.cache/huggingface/token'
+ return p.read_text().strip() if p.exists() else None
+class Hub:
+ def __init__(self,cache):
+  from huggingface_hub import HfApi
+  self.api=HfApi(token=token());self.cache=Path(cache);self.revisions={}
+ def revision(self,repo):
+  if repo not in self.revisions:
+   local=local_directory(repo)
+   self.revisions[repo]=fingerprint(local) if local else self.api.model_info(repo).sha
+  return self.revisions[repo]
+ def file(self,repo,name):
+  local=local_directory(repo)
+  if local:
+   p=local/name
+   if not p.is_file():raise FileNotFoundError(p)
+   return p
+  from huggingface_hub import hf_hub_download
+  return Path(hf_hub_download(repo,name,revision=self.revision(repo),cache_dir=str(self.cache),token=token()))
+ def snapshot(self,repo,dest):
+  local=local_directory(repo)
+  if local:return local
+  from huggingface_hub import snapshot_download
+  snapshot_download(repo,revision=self.revision(repo),local_dir=str(dest),token=token(),max_workers=8)
+  return Path(dest)
+def complete_snapshot(path):
+ try:
+  wm=json.loads((Path(path)/'model.safetensors.index.json').read_text())['weight_map']
+  return bool(wm) and all((Path(path)/f).is_file() for f in set(wm.values()))
+ except (FileNotFoundError,KeyError,ValueError):return False
+def valid_capture(path):
+ import torch
+ try:
+  d=torch.load(path,weights_only=True,map_location='cpu');n=len(d['x'])
+  return n>=32 and d['x'].shape==(n,6144) and d['topk_ids'].shape==(n,8) and d['topk_weights'].shape==(n,8) and bool(torch.isfinite(d['x']).all()) and bool(torch.isfinite(d['topk_weights']).all()) and bool(((d['topk_ids']>=0)&(d['topk_ids']<256)).all())
+ except (OSError,KeyError,ValueError,RuntimeError,EOFError):return False
+def build_corpus(args,hub):
+ """Download a bounded, deterministic text corpus when local token shards are absent."""
+ from datasets import load_dataset
+ from transformers import AutoTokenizer
+ target=Path(args.calib_tokens);target.mkdir(parents=True,exist_ok=True)
+ marker=target/'corpus.building.json'
+ if marker.exists():
+  for shard in target.glob('shard_*.npy'):shard.unlink()
+ write(marker,{'builder':'arvq88 fallback corpus','target_tokens':args.calib_token_count})
+ tok=AutoTokenizer.from_pretrained(args.src,revision=None if local_directory(args.src) else hub.revision(args.src),token=token(),trust_remote_code=True)
+ specs=[('m-a-p/CodeFeedback-Filtered-Instruction',.4),('tatsu-lab/alpaca',.4),('databricks/databricks-dolly-15k',.2)]
+ pending=[];number=0;actual={};revisions={}
+ for repo,fraction in specs:
+  budget=int(args.calib_token_count*fraction);count=0
+  revisions[repo]=hub.api.dataset_info(repo).sha
+  data=load_dataset(repo,revision=revisions[repo],split='train',streaming=True,token=token()).shuffle(seed=42,buffer_size=10000)
+  while count<budget:
+   before=count
+   for row in data:
+    text='\n'.join(str(v) for v in row.values() if isinstance(v,(str,list)))
+    ids=tok.encode(text,add_special_tokens=False)+[tok.eos_token_id]
+    ids=ids[:budget-count];pending.extend(ids);count+=len(ids)
+    while len(pending)>=1_000_000:
+     np.save(target/f'shard_{number:05d}.npy',np.asarray(pending[:1_000_000],dtype=np.uint32));pending=pending[1_000_000:];number+=1
+    if count>=budget:break
+   if count==before:raise RuntimeError(f'Empty calibration dataset {repo}')
+  actual[repo]=count
+ if pending:np.save(target/f'shard_{number:05d}.npy',np.asarray(pending,dtype=np.uint32))
+ write(target/'corpus.json',{'builder':'fallback text calibration, not legacy calib-v3','sources_token_counts':actual,'dataset_revisions':revisions,'seed':42,'reasoning_rollouts':False,'tokenizer_repo':args.src,'tokenizer_revision':hub.revision(args.src)})
+ marker.unlink()
+def ensure_captures(args,hub,work,need_stats=False):
+ import torch
+ shards=list(Path(args.calib_tokens).glob('shard_*.npy'))
+ if not shards or (Path(args.calib_tokens)/'corpus.building.json').exists():build_corpus(args,hub)
+ else:
+  for p in shards:
+   a=np.load(p,mmap_mode='r')
+   if a.ndim!=1 or not np.issubdtype(a.dtype,np.integer) or not a.size:raise ValueError(f'Invalid token shard {p}')
+ ready=all(valid_capture(Path(args.calib)/f'acts_layer{L}.pt') for L in LAYERS)
+ stats=Path(args.calib).parent/'expert_stats53.npz'
+ if ready and (not need_stats or stats.exists()):return stats
+ if not complete_snapshot(args.donor_cache):hub.snapshot(args.nvfp4_donor,args.donor_cache)
+ # Capture to a fresh directory so partial old ranks cannot enter the merge.
+ dest=work/'capture';dest.mkdir(exist_ok=True)
+ env={**os.environ,'GLM53_DONOR':args.donor_cache,'GLM53_CALIB':args.calib_tokens,'GLM53_CAPTURE_OUT':str(dest)}
+ script=ROOT.parent/'tools/capture53/stream_capture53.py';workers=[]
+ for rank,gpu in enumerate(args.gpus):
+  log=open(work/f'capture-rank{rank}.log','a')
+  workers.append(subprocess.Popen([sys.executable,'-u',str(script),'stats'],env={**env,'CUDA_VISIBLE_DEVICES':str(gpu),'RANK':str(rank),'WORLD':str(len(args.gpus))},stdout=log,stderr=subprocess.STDOUT,start_new_session=True));log.close()
+ codes=[p.wait() for p in workers]
+ if any(codes):raise RuntimeError('Capture workers failed; see capture-rank logs')
+ subprocess.run([sys.executable,str(script),'merge'],env={**env,'WORLD':str(len(args.gpus))},check=True)
+ args.calib=str(dest/'acts')
+ if not all(valid_capture(Path(args.calib)/f'acts_layer{L}.pt') for L in LAYERS):raise RuntimeError('Capture incomplete')
+ return dest/'expert_stats53.npz'
+def assignment(args,hub,work):
+ if getattr(args,'reap_metric','aqlm')=='arvq':
+  from .arvq_reap import recompute
+  obj=recompute(args,hub,work)
+  write(work/'assignment.json',obj);args.assign=str(work/'assignment.json');return obj
+ cached=Path(args.assign)
+ if cached.exists():
+  obj=validate_assignment(json.loads(cached.read_text()),args.hot_count)
+  obj['provenance']={'mode':'reused allocation','path':str(cached),'sha256':sha256(cached)}
+ else:
+  from .reap import recompute
+  obj=recompute(args,hub,work)
+ write(work/'assignment.json',obj);args.assign=str(work/'assignment.json');return obj
+def ensure_source(args,hub,work,allocation):
+ import torch
+ from safetensors import safe_open
+ from safetensors.torch import save_file
+ from ingest import probe_source,validate_meta_against_header
+ from remote_st import RemoteShard
+ import remote_st
+ cfg=json.loads(hub.file(args.src,'config.json').read_text());text=cfg.get('text_config',cfg)
+ if text.get('hidden_size')!=6144 or text.get('moe_intermediate_size')!=2048 or text.get('n_routed_experts',text.get('num_local_experts'))!=256:
+  raise ValueError('Input is not the supported GLM-5.3 expert geometry')
+ meta=probe_source(text);native={**vars(meta),'block':list(meta.block) if meta.block else None}
+ revision=hub.revision(args.src);cache=Path(args.src_cache);cache.mkdir(parents=True,exist_ok=True)
+ had_existing=any(cache.glob('layer_*'))
+ marker=cache/'source_identity.json'
+ if marker.exists():
+  identity=json.loads(marker.read_text())
+  if identity!={'repo':args.src,'revision':revision}:raise ValueError('Source cache identity mismatch; use another --src-cache')
+ elif args.src!='zai-org/GLM-5.3' and any(cache.glob('layer_*')):raise ValueError('Unidentified cache cannot be reused for a different source repo')
+ local=local_directory(args.src)
+ index=model_index(local) if local else json.loads(hub.file(args.src,'model.safetensors.index.json').read_text())['weight_map']
+ remote_st.resolve_url=lambda repo,name:f'https://huggingface.co/{repo}/resolve/{revision}/{name}'
+ headers={}
+ def shard(name):
+  fn=index[name]
+  if fn not in headers:headers[fn]=LocalShard(local/fn) if local else RemoteShard(args.src,fn)
+  return headers[fn]
+ dtype={'F8_E4M3':torch.float8_e4m3fn,'BF16':torch.bfloat16,'F16':torch.float16,'F32':torch.float32,'U8':torch.uint8}
+ def fetch(pair):
+  L,e=pair;path=cache/f'layer_{L}'/f'expert_{e}.safetensors'
+  if path.exists():
+   try:
+    with safe_open(str(path),framework='pt') as f:
+     metadata=f.metadata() or {}
+     if metadata.get('source',args.src)!=args.src:raise ValueError('Expert source mismatch')
+     for proj,shape in [('gate_proj',(2048,6144)),('up_proj',(2048,6144)),('down_proj',(6144,2048))]:
+      t=f.get_slice(proj+'.weight')
+      if tuple(t.get_shape())!=shape:raise ValueError('Invalid cached expert shape')
+      validate_meta_against_header(meta,t.get_dtype())
+      if meta.kind=='block_fp8' and tuple(f.get_slice(proj+'.weight_scale_inv').get_shape())!=(shape[0]//meta.block[0],shape[1]//meta.block[1]):raise ValueError('Invalid cached scales')
+    return
+   except Exception as exc:raise ValueError(f'Invalid cached expert {path}; remove or repair it before retry') from exc
+  tensors={}
+  for proj in ['gate_proj','up_proj','down_proj']:
+   for suffix in ['weight']+(['weight_scale_inv'] if meta.kind=='block_fp8' else []):
+    name=f'model.layers.{L}.mlp.experts.{e}.{proj}.{suffix}';sh=shard(name);m=sh.tensor_meta(name)
+    raw=bytearray(sh.read_tensor_bytes(name));expected=m['data_offsets'][1]-m['data_offsets'][0]
+    if len(raw)!=expected:raise IOError('Short source tensor read')
+    t=torch.frombuffer(raw,dtype=dtype[m['dtype']]).reshape(m['shape']).clone()
+    if suffix=='weight' and meta.kind=='block_fp8' and t.dtype==torch.uint8:t=t.view(torch.float8_e4m3fn)
+    tensors[proj+'.'+suffix]=t
+  path.parent.mkdir(parents=True,exist_ok=True);tmp=path.with_suffix('.part');save_file(tensors,str(tmp),metadata={'source':args.src,'revision':revision});tmp.replace(path)
+ jobs=[(L,e) for L in LAYERS for e in allocation['layers'][str(L)]['cold_5750']]
+ with ThreadPoolExecutor(max_workers=args.download_workers) as pool:list(pool.map(fetch,jobs))
+ # Existing canonical donor files predate revision markers; record their status honestly.
+ if local and fingerprint(local)!=revision:raise ValueError('Local source changed during extraction')
+ write(work/'source.json',{'repo':args.src,'revision':revision,'meta':native,'cache':str(cache),'legacy_cache_revision_unverified':not marker.exists() and had_existing})
+ if not had_existing:write(marker,{'repo':args.src,'revision':revision})
+ return meta

reproduce/source/btx53/arvq88/jobs.py

@@ -0,0 +1,28 @@
+"""Bounded subprocess jobs on explicitly reserved GPUs."""
+import subprocess
+import time
+from pathlib import Path
+
+
+def run_jobs(jobs, gpus, logs):
+    pending=list(jobs);running={};logs=Path(logs);logs.mkdir(parents=True,exist_ok=True)
+    try:
+        while pending or running:
+            for gpu in gpus:
+                if gpu in running or not pending:continue
+                name,command=pending.pop(0)
+                with (logs/(name+'.log')).open('a') as log:
+                    p=subprocess.Popen(command(gpu),stdin=subprocess.DEVNULL,stdout=log,
+                                       stderr=subprocess.STDOUT,start_new_session=True)
+                running[gpu]=(name,p);print('START',name,'GPU',gpu,'PID',p.pid,flush=True)
+            time.sleep(5)
+            for gpu,(name,p) in list(running.items()):
+                rc=p.poll()
+                if rc is None:continue
+                if rc:raise RuntimeError(f'{name} failed ({rc}); see {logs/name}.log')
+                del running[gpu];print('DONE',name,flush=True)
+    except BaseException:
+        for name,p in running.values():
+            if p.poll() is None:p.terminate()
+        for name,p in running.values():p.wait()
+        raise

reproduce/source/btx53/arvq88/local_source.py

@@ -0,0 +1,43 @@
+"""Read standard local safetensors snapshots without Hub requests."""
+import hashlib,json,struct
+from pathlib import Path
+
+def local_directory(source):
+ p=Path(source).expanduser()
+ if p.exists():
+  if not p.is_dir():raise ValueError(f'Model input must be a directory: {p}')
+  return p.resolve()
+ if str(source).startswith(('/', './', '../', '~')):
+  raise FileNotFoundError(f'Local model directory does not exist: {p}')
+ return None
+
+def model_index(root):
+ p=root/'model.safetensors.index.json'
+ if p.is_file():return json.loads(p.read_text())['weight_map']
+ p=root/'model.safetensors'
+ if not p.is_file():raise FileNotFoundError(f'Missing safetensors weights/index in {root}')
+ return {k:p.name for k in LocalShard(p).header if k!='__metadata__'}
+
+def fingerprint(root):
+ # Metadata identity, not a full content hash of a multi-hundred-GB donor.
+ h=hashlib.sha256((root/'config.json').read_bytes())
+ index=model_index(root);h.update(json.dumps(index,sort_keys=True).encode())
+ for name in sorted(set(index.values())):
+  p=root/name;st=p.stat()
+  h.update(json.dumps([name,st.st_size,st.st_mtime_ns,st.st_ctime_ns]).encode())
+ return 'local-stat-sha256:'+h.hexdigest()
+
+class LocalShard:
+ def __init__(self,path):
+  self.path=Path(path)
+  with self.path.open('rb') as f:
+   raw=f.read(8)
+   if len(raw)!=8:raise ValueError('Truncated safetensors header')
+   n=struct.unpack('<Q',raw)[0]
+   if n>100_000_000:raise ValueError('Invalid safetensors header length')
+   self.header=json.loads(f.read(n));self.offset=8+n
+ def tensor_meta(self,name):return self.header[name]
+ def read_tensor_bytes(self,name):
+  a,b=self.tensor_meta(name)['data_offsets']
+  with self.path.open('rb') as f:
+   f.seek(self.offset+a);return f.read(b-a)

reproduce/source/btx53/arvq88/pack.py

@@ -0,0 +1,142 @@
+"""ARVQ 8+8 test format: natural indices -> MMA fragments, 64 uint32 words/tile."""
+import torch,json,hashlib
+from pathlib import Path
+LEVELS=[0.,.5,1.,1.5,2.,3.,4.,6.,-0.,-.5,-1.,-1.5,-2.,-3.,-4.,-6.]
+def maps(device):
+ p=torch.arange(128,device=device);j=p//32;lane=p%32
+ return lane//4+8*(j%2),(j//2)*4+lane%4
+def pack(a,b,N,K):
+ r,k=maps(a.device);rows=torch.arange(N//16,device=a.device)[:,None,None]*16+r[None,None,:];cols=torch.arange(K//64,device=a.device)[None,:,None]*8+k[None,None,:]
+ v=(a.long()|(b.long()<<8))[rows,cols]
+ return (v[...,::2]|(v[...,1::2]<<16)).to(torch.uint32)
+def unpack(w,N,K):
+ w=w.long();v=torch.stack([w&65535,w>>16],-1).reshape(N//16,K//64,128)
+ r,k=maps(w.device);rows=(torch.arange(N//16,device=w.device)[:,None,None]*16+r[None,None,:]).expand_as(v);cols=(torch.arange(K//64,device=w.device)[None,:,None]*8+k[None,None,:]).expand_as(v)
+ a=torch.empty(N,K//8,dtype=torch.uint8,device=w.device);b=torch.empty_like(a)
+ a[rows,cols]=(v&255).byte();b[rows,cols]=(v>>8).byte();return a,b
+def pack_codebooks(c0,c1):
+ cb=torch.cat([c0,c1],dim=-2)
+ if cb.shape[-2:]!=(512,8) or cb.ndim not in (2,3):raise ValueError('Expected shared or per-expert 8+8 books')
+ levels=torch.tensor(LEVELS,device=cb.device);n=(cb[...,None]-levels).abs().argmin(-1)
+ if not torch.equal(levels[n],cb):raise ValueError('Codebooks must be exactly on the FP4 grid')
+ return (n.long()<<(torch.arange(8,device=cb.device)*4)).sum(-1).to(torch.uint32)
+def pack_codebooks_mb(cb,factors):
+ """mcbook16: cb [E,256+B*256,8] in EFFECTIVE units. Rows are stored as plain-grid
+ nibbles; decode multiplies residual book m by factors[m] (exact powers of two)."""
+ B=len(factors)
+ if cb.ndim!=3 or cb.shape[1]!=256+B*256 or cb.shape[2]!=8:raise ValueError('Invalid mcbook codebook shape')
+ f=torch.cat([torch.ones(256),torch.tensor(factors).float().repeat_interleave(256)])[None,:,None]
+ grid=cb/f
+ levels=torch.tensor(LEVELS,device=cb.device);n=(grid[...,None]-levels).abs().argmin(-1)
+ if not torch.equal(levels[n]*f,cb):raise ValueError('Codebooks must be exactly on their per-book grids')
+ return (n.long()<<(torch.arange(8,device=cb.device)*4)).sum(-1).to(torch.uint32)
+def decode_mb(t,layer,proj,e,N,K):
+ pre=f'model.layers.{layer}.mlp.experts.arvq_{proj}_'
+ a,b=unpack(t[pre+'packed'][e],N,K)
+ cb=t[pre+'codebooks'][e].long();factors=t[pre+'book_factors'].float();B=len(factors)
+ if cb.shape!=(256+B*256,):raise ValueError('Invalid mcbook packed codebook shape')
+ values=torch.tensor(LEVELS,device=cb.device)[(cb[:,None]>>(torch.arange(8,device=cb.device)*4))&15]
+ values=torch.cat([values[:256],values[256:]*factors.repeat_interleave(256)[:,None]])
+ sel=t[pre+'selectors'][e].long()
+ m=sel.repeat_interleave(16,0).repeat_interleave(8,1)
+ stored=t[pre+'scales'][e].permute(0,2,1).contiguous().reshape(N,K//128)
+ if stored.dtype==torch.float16:scales=stored.float()
+ elif stored.dtype==torch.uint8:scales=stored.view(torch.float8_e4m3fn).float()
+ else:raise ValueError('Unsupported packed block scale dtype')
+ return ((values[a.long()]+values[256+m*256+b.long()])*scales.repeat_interleave(16,1)[...,None]*t[pre+'global']).reshape(N,K)
+def export_layer_mb(source,dest,L):
+ """mcbook16 export (format version 5): 16 residual books, per-tile 4-bit selectors,
+ fp16 block scales; packed index stream and scales unchanged from v4."""
+ from safetensors.torch import save_file,load_file
+ source=Path(source);dest=Path(dest);dest.mkdir(parents=True,exist_ok=True)
+ original=json.loads((source/'arvq-manifest.json').read_text());files={};first_factors=None
+ for proj,tag in [('w13','gateup'),('w2','down')]:
+  d=torch.load(source/f'{proj}.pt',map_location='cpu',weights_only=True)[proj];N,K=d['N'],d['K'];E=len(d['a']);pre=f'model.layers.{L}.mlp.experts.arvq_{proj}_'
+  if d.get('scale_dtype')!='fp16':raise ValueError('mcbook export requires fp16 block scales')
+  if not torch.equal(d['s'],d['s'].half().float()):raise ValueError('Scales are not FP16-representable')
+  factors=[float(f) for f in d['book_factor']]
+  if first_factors is None:first_factors=factors
+  elif factors!=first_factors:raise ValueError('Mixed projection book factors')
+  sel=d['selector']
+  if sel.shape!=(E,N//16,K//64) or int(sel.max())>=len(factors):raise ValueError('Invalid selector tensor')
+  t={pre+'packed':torch.stack([pack(a,b,N,K) for a,b in zip(d['a'],d['b'])]),
+     pre+'codebooks':pack_codebooks_mb(d['cb'],factors),
+     pre+'selectors':sel.to(torch.uint8).contiguous(),
+     pre+'book_factors':torch.tensor(factors,dtype=torch.float32),
+     pre+'scales':d['s'].half().reshape(E,N//16,16,K//128).permute(0,1,3,2).contiguous(),
+     pre+'global':torch.tensor([d['glob']],dtype=torch.float32)}
+  for e in range(E):
+   a,b=unpack(t[pre+'packed'][e],N,K);assert torch.equal(a,d['a'][e]) and torch.equal(b,d['b'][e])
+  for e in sorted({0,E//2,E-1}):
+   m=d['selector'][e].long().repeat_interleave(16,0).repeat_interleave(8,1)
+   cb=d['cb'][e]
+   ref=((cb[d['a'][e].long()]+cb[256+m*256+d['b'][e].long()])*d['s'][e].repeat_interleave(16,1)[...,None]*d['glob']).reshape(N,K)
+   assert torch.equal(ref,decode_mb(t,L,proj,e,N,K))
+  name=f'arvq-layer-{L:03d}-{tag}.safetensors';tmp=dest/(name+'.tmp')
+  save_file(t,str(tmp),metadata={'format':'pt','arvq_format':'rvq256_mb16_256x8_expert_fp16block','codebook_scope':'expert','residual_books':str(len(factors)),'book_factors':json.dumps(factors),'fit':str(original.get('fit','initial Hessian fit, no PV')),'serving_compatibility':'requires mcbook16 v5 loader/kernel (vllm-glm52-sm120 branch experiment/arvq-mcbook16)'})
+  tmp.replace(dest/name)
+  loaded=load_file(str(dest/name));assert loaded[pre+'packed'].shape==(E,N//16,K//64,64) and loaded[pre+'selectors'].shape==(E,N//16,K//64);del loaded
+  files[proj]={'file':name,'sha256':sha(dest/name)};del t,d
+ m={**original,'format':'rvq256_mb16_256x8_expert_fp16block','version':5,'residual_scale_shift':0,'block_scale_dtype':'float16','codebook_scope':'expert','bits':2.125+4/1024,'codebook_sizes':[256,16*256],'index_bits':[8,8],'selector_bits_per_tile':4,'words_per_tile':64,'book_factors':first_factors,'files':files,'fit':str(original.get('fit','initial Hessian fit, no PV')),'production_ready':False,'roundtrip':'all expert indices exact; 3 decoded weights/projection exact','requires_8x8_loader_kernel':True,'requires_mcbook16_loader':True}
+ (dest/f'layer-{L:03d}-manifest.json').write_text(json.dumps(m,indent=2));return m
+def decode(t,layer,proj,e,N=None,K=None,residual_scale_shift=0):
+ pre=f'model.layers.{layer}.mlp.experts.arvq_{proj}_'
+ default=(4096,6144) if proj=='w13' else (6144,2048)
+ N,K=default if N is None else (N,K)
+ a,b=unpack(t[pre+'packed'][e],N,K);cb=t[pre+'codebooks'].long()
+ if cb.ndim==2:cb=cb[e]
+ if cb.shape!=(512,):raise ValueError('Invalid packed codebook shape')
+ values=torch.tensor(LEVELS,device=cb.device)[(cb[:,None]>>(torch.arange(8,device=cb.device)*4))&15]
+ stored=t[pre+'scales'][e].permute(0,2,1).contiguous().reshape(N,K//128)
+ if stored.dtype==torch.uint8:scales=stored.view(torch.float8_e4m3fn).float()
+ elif stored.dtype==torch.float16:scales=stored.float()
+ else:raise ValueError('Unsupported packed block scale dtype')
+ f=2.0**(-int(residual_scale_shift))
+ return ((values[a.long()]+f*values[256+b.long()])*scales.repeat_interleave(16,1)[...,None]*t[pre+'global']).reshape(N,K)
+def sha(p):
+ h=hashlib.sha256()
+ with open(p,'rb') as f:
+  for chunk in iter(lambda:f.read(8<<20),b''):h.update(chunk)
+ return h.hexdigest()
+def export_layer(source,dest,L,residual_scale_shift=0):
+ from safetensors.torch import save_file,load_file
+ source=Path(source);dest=Path(dest);dest.mkdir(parents=True,exist_ok=True)
+ probe=torch.load(source/'w13.pt',map_location='cpu',weights_only=True,mmap=True)['w13']
+ if 'selector' in probe:
+  del probe
+  if int(residual_scale_shift):raise ValueError('mcbook exports are shift-0 (per-book factors)')
+  return export_layer_mb(source,dest,L)
+ del probe
+ rss=int(residual_scale_shift);f=2.0**(-rss)
+ original=json.loads((source/'arvq-manifest.json').read_text());files={}
+ for proj,tag in [('w13','gateup'),('w2','down')]:
+  d=torch.load(source/f'{proj}.pt',map_location='cpu',weights_only=True)[proj];N,K=d['N'],d['K'];E=len(d['a']);pre=f'model.layers.{L}.mlp.experts.arvq_{proj}_'
+  expert=d['c0'].ndim==3
+  if expert and d['c0'].shape!=(E,256,8):raise ValueError('Codebook expert count mismatch')
+  if proj=='w13':scope='expert' if expert else 'layer'
+  elif scope!=('expert' if expert else 'layer'):raise ValueError('Mixed projection codebook scopes')
+  fp16=d.get('scale_dtype','fp8_e4m3')=='fp16'
+  if proj=='w13':fp16_blocks=fp16
+  elif fp16_blocks!=fp16:raise ValueError('Mixed projection scale precision')
+  if fp16 and not torch.equal(d['s'],d['s'].half().float()):raise ValueError('Scales are not FP16-representable')
+  stored_scales=d['s'].half() if fp16 else d['s'].to(torch.float8_e4m3fn).view(torch.uint8)
+  t={pre+'packed':torch.stack([pack(a,b,N,K) for a,b in zip(d['a'],d['b'])]),
+     pre+'codebooks':pack_codebooks(d['c0'],d['c1']),
+     pre+'scales':stored_scales.reshape(E,N//16,16,K//128).permute(0,1,3,2).contiguous(),
+     pre+'global':torch.tensor([d['glob']],dtype=torch.float32)}
+  for e in range(E):
+   a,b=unpack(t[pre+'packed'][e],N,K);assert torch.equal(a,d['a'][e]) and torch.equal(b,d['b'][e])
+  for e in sorted({0,E//2,E-1}):
+   c0,c1=(d['c0'][e],d['c1'][e]) if expert else (d['c0'],d['c1'])
+   ref=((c0[d['a'][e].long()]+f*c1[d['b'][e].long()])*d['s'][e].repeat_interleave(16,1)[...,None]*d['glob']).reshape(N,K)
+   assert torch.equal(ref,decode(t,L,proj,e,N,K,rss))
+  base_fmt='rvq256_256x8_expert_fp16block' if fp16 else ('rvq256_256x8_expert' if expert else 'rvq256_256x8')
+  arvq_fmt=base_fmt+(f'_rs{int(2**rss)}' if rss else '')
+  name=f'arvq-layer-{L:03d}-{tag}.safetensors';tmp=dest/(name+'.tmp')
+  save_file(t,str(tmp),metadata={'format':'pt','arvq_format':arvq_fmt,'residual_scale_shift':str(rss),'codebook_scope':scope,'fit':str(original.get('fit','initial Hessian fit, no PV')),'serving_compatibility':'requires FP16 block-scale v4 loader/kernel' if fp16 else ('requires per-expert v3 loader/kernel' if expert else 'requires 8+8 loader/kernel')})
+  tmp.replace(dest/name);del t,d
+  loaded=load_file(str(dest/name));assert loaded[pre+'packed'].shape==(E,N//16,K//64,64);del loaded
+  files[proj]={'file':name,'sha256':sha(dest/name)}
+ base_mfmt='rvq256_256x8_expert_fp16block' if fp16_blocks else ('rvq256_256x8_expert' if scope=='expert' else 'rvq256_256x8')
+ m={**original,'format':base_mfmt+(f'_rs{int(2**rss)}' if rss else ''),'version':4 if fp16_blocks else (3 if scope=='expert' else 2),'residual_scale_shift':rss,'block_scale_dtype':'float16' if fp16_blocks else 'float8_e4m3fn','codebook_scope':scope,'bits':2.125 if fp16_blocks else 2.0625,'codebook_sizes':[256,256],'index_bits':[8,8],'words_per_tile':64,'files':files,'fit':str(original.get('fit','initial Hessian fit, no PV')),'production_ready':False,'roundtrip':'all expert indices exact; 3 decoded weights/projection exact','requires_8x8_loader_kernel':True}
+ (dest/f'layer-{L:03d}-manifest.json').write_text(json.dumps(m,indent=2));return m

reproduce/source/btx53/arvq88/perf/FP16_BLOCK_SCALE_HANDOFF.md

@@ -0,0 +1,19 @@
+# ARVQ FP16 block-scale format (v4)
+
+This replaces FP8 E4M3 block scales with FP16 block scales. There are no added row scales. Books remain per-expert packed FP4, indices remain two 8-bit indices per group of eight weights, and global scale remains FP32 per projection. Initial FP8 values are exactly representable in FP16.
+
+Checkpoint tensors use the existing `arvq_{w13,w2}_scales` names and logical tiled shape `[cold_experts, N/16, K/128, 16]`, now dtype F16 instead of U8 (FP8 bytes). Manifest version4, format `rvq256_256x8_expert_fp16block`, `block_scale_dtype: float16`. Packed codebooks and indices retain v3 layout. Index+block-scale storage is2.125 bits/weight, excluding books/global metadata. Each scale belongs to one expert, one output row, and128 input weights. Scales cannot be folded into one whole-row epilogue multiplier.
+
+Decode for a group of8 weights:
+`W[e,r,8*g:8*g+8] = global * scale16[e,r,g//16] * (book0[e,index0] + book1[e,index1])`.
+
+The existing native MMA instruction accepts UE4M3 weight scales, not FP16. A compatible native path must preserve FP16 scale precision, e.g. use unit weight scales in FP4 MMA, accumulate the128-weight block (two K64 tiles, both codebooks and activation planes), multiply its FP32 contribution by the FP16 block scale converted to FP32, and accumulate blocks. Do not silently cast the new scales back to E4M3. Activation quantization and hot NVFP4 paths remain unchanged. Benchmark native latency and validate arithmetic before publication.
+
+Training: `/tmp/glm53-fp16-sequential`, sequential layers3–77 from original pre-PV initialization, full18,001,846-token corpus, batch262144/micro65536, Adam and periodic output-gradient index reassignment. Trainable latent log-scales are FP32; every forward/export projects scales to FP16. Global scale is frozen. Max69 updates; three validation checks >0.1% worse trigger a4x LR drop, otherwise drop after45; stop after15 lower-LR updates without a new best. Validation, development-audit and export-replay gates remain enforced.
+
+Current fitting/export modules:
+- `sequential_pv_full_corpus.py`: config `block_scale_dtype: fp16` selects precision.
+- `arvq88/pv.py`: encoded `scale_dtype` travels with the representation and is honored during replay/propagation.
+- `arvq88/pack.py`: F16 tiled scales, v4 metadata and exact decoded-weight roundtrip.
+
+Local fitting and propagation can proceed. Automatic HF publication is disabled until the loader/kernel supports this format. Do not publish v4 tensors under v3 metadata.

reproduce/source/btx53/arvq88/perf/FULL_CORPUS_PIPELINE.md

@@ -0,0 +1,80 @@
+# Full-corpus sequential PV
+
+Run `full_pipeline.py --work WORK --first 3 --last N` with the repository Python.
+The work directory needs config.json, the initial fixed capture3, and token file.
+The driver runs eight GPU ranks per stage and never uploads. Set explicit last
+layer: a bounded pilot is the default operational choice until it qualifies.
+
+The existing layer-3 fit uses 18,001,846 training tokens, one shuffled sequence
+pass, 262144 effective batch, 65536 microbatch, Adam codebook LR .048 and scale
+LR .032. Gradient accumulation normalizes every microbatch by the full batch
+size; the regularizer, clipping and Adam step execute once per effective batch.
+Validation every five updates retains the best state. Audit is evaluated on the
+retained state; the driver refuses propagation if validation or audit regresses.
+
+Capture stores student normalized inputs/routing, frozen branch output, the
+reference-minus-frozen target, and the BF16 reference output. Both full reference
+and full student trajectories advance one block at a time. Layer 3 needs a
+one-time recapture because the earlier training capture omitted frozen/reference
+outputs, which cannot be reconstructed from their difference alone. Dense prefix
+weights now remain resident during that recapture. Layers 4 onward read cached
+boundaries and never execute the earlier prefix.
+
+Propagation uses the retained serialized FP4 books, FP8 scales and indices with
+the same activation emulation as training. It caches decoded expert weights per
+GPU across all chunks. A fixed-evaluation replay gate allows relative difference
+at most 1e-4 after BF16 output rounding, accounting for different FP32 reduction
+order; it does not claim bitwise parity or native SM120 execution. Partial final
+sequences are zero-padded at boundaries and masked from capture/training losses.
+
+Rank zero alone assembles and prefetches each CPU training batch, then broadcasts
+identical tensors to the expert ranks over NCCL; capture and propagation shard writes
+run on one background thread with bounded outstanding data. Metadata is committed
+only after the data file is atomically renamed. A partial smoke run cannot publish
+a rank-complete marker. `timings.json` records rank-0 data wait, transfer, optimizer,
+reassignment and validation/checkpoint time; these are not all-rank max timings.
+
+The earlier monolithic 256K run exhausted GPU memory on fresh data. Its first five
+updates took 158.9 seconds. The 64K accumulated version with prefetch took 65.0
+seconds with validation 0.0047233582 versus 0.0047233591 (relative difference
+~2e-7). This is an observed first-five-update comparison, not a full-run speedup.
+
+Pending qualification: the scheduled 262144-token integration smoke exercises
+capture3 -> serialized propagation3 -> capture4. Full-layer transition throughput
+must be measured before extrapolating all 75 layers. CUDA graphs are not enabled:
+variable routing and fresh sequence batches require a separately validated static
+buffer/shape design. Native serving-kernel numerical qualification remains separate.
+
+The driver keeps two layers of regenerable full-corpus intermediates and all fit
+outputs/checkpoints. It prunes only directories with its own completed stage
+receipt after a later layer passes held-out checks; it never prunes the original
+legacy training_capture directory or fixed validation/audit captures.
+
+New captures store normalized inputs as BF16 only after an exact FP32 roundtrip
+assertion for every chunk; arithmetic restores FP32 before use. This halves input
+storage and CPU assembly traffic without rounding any captured input values.
+
+## Capture optimizations added during the live campaign
+
+Layer-14 measurements: capture setup ~90 s, capture computation ~154 s,
+CPU copying/statistics ~99 s, input transfers ~31 s. Separate fixed-evaluation
+capture costs another ~100 s. Propagation CPU boundary packing costs ~27 s.
+
+New capture uses two reusable pinned host banks and asynchronous CUDA copies,
+with an event that the shard writer waits on before serialization. The writer's
+single-outstanding-job contract prevents overwriting a bank still being saved.
+Each rank compares its first staged production chunk exactly with the original
+blocking CPU transfer. A real CUDA test also exercises six writes across bank
+reuse and verifies every saved value. Propagation now copies the full output in
+one transfer and reshapes complete sequences without CPU copies; partial tails
+retain explicit zero padding and have exact parity tests.
+
+Full training capture also produces a candidate fixed-evaluation capture using
+resident weights. The existing standalone capture still runs for the first
+qualification layer, compares every tensor on all eight ranks, and publishes
+fused_eval_approved.json only on exact equality. Subsequent fixed capture stages
+reuse that candidate instead of reloading the reference/donor weights. If parity
+fails, standalone capture continues to supply the production tensors.
+
+Production speedups are not yet measured. Inspect training_capture*/rank*_timings.json,
+fused_eval_qualification, fused_eval_approved.json and pipeline stage receipts.

reproduce/source/btx53/arvq88/perf/README.md

@@ -0,0 +1,145 @@
+# PV performance investigation (18 September 2026, Melbourne)
+
+The active sequential quality pilot was not modified. These files are isolated
+prototypes and a separately invocable trainer fork; no changes were published.
+
+## Measured single-expert forward + backward
+
+Actual layer-3 expert 3, 64 routed training rows, original fitted per-expert
+books, float32 GEMMs and the same FP4/FP16 activation emulation. GPU 0 was shared
+with the running pilot. Four timing repeats per path after warmup, no optimizer
+step/communication/validation included. Results repeated in two invocations.
+
+| Path | Median, second invocation |
+| --- | ---: |
+| Original eager arithmetic, four 16-row microbatches | 386.47 ms |
+| Original eager arithmetic, one 64-row batch | 132.02 ms |
+| Graph-compatible arithmetic, graphed one-batch forward/backward | 119.08 ms |
+
+Larger batch: 2.93x throughput for this expert calculation. Graph replay adds
+about 10% lower latency relative to eager one-batch; combined factor 3.25x.
+These are NOT full-layer or whole-campaign speedup measurements.
+
+Graph versus eager one-batch output and parameter-gradient relative differences
+were zero in this test. After in-place index and row-scale changes, replay still
+matched eager output exactly. Four-microbatch versus one-batch maximum parameter
+-gradient relative difference was 5.99e-5 (FP32 accumulation/GEMM differences),
+with losses 0.1189905256 versus 0.1189905405. End-to-end optimization trajectories
+therefore need comparison, not an assertion of bit-identical training.
+
+Artifacts: `/tmp/glm53-pv-graph-benchmark.json` and
+`/tmp/glm53-pv-graph-benchmark-check.json`. Benchmark peak allocated memory:
+0.514 GiB, isolated expert only; not a full-layer graph memory estimate.
+
+## Why larger batches help
+
+The production pilot calls local_prediction separately for each of four
+microbatches. Every call reconstructs full expert matrices and backpropagates
+through their codebook gathers. We repeat this large decode/scatter workload
+four times despite unchanged parameters within the optimizer step. The GPUs
+can report high utilization while doing this redundant work.
+
+`sequential_pv_fast.py` is a separate fork with configurable `--microbatch`
+(default 1024 versus the original 256). Global batch, objective, selected rows,
+regularization and update count remain unchanged. Full-layer peak-memory and
+held-out parity/timing tests are needed before deployment.
+
+## Communication optimization
+
+The pilot broadcasts an 8192 x 6144 float32 delta (192 MiB) for each proposal
+attempt, mostly zeros outside an expert's routed rows. `sparse_delta.py` sends
+only row IDs and their nonzero payload, then reconstructs the same dense buffer
+before the unchanged acceptance arithmetic. For eight-of-256 routing the average
+payload is roughly 1/32 as large, though expert popularity varies.
+
+Two-rank CPU distributed tests confirm bit-exact reconstruction versus dense
+broadcast, for either owner and empty row sets. This optimization is included
+in the separate fast trainer fork. Eight-GPU NCCL qualification remains pending.
+
+## CUDA Graph integration boundary
+
+`graph_expert.py` and `bench_graph_expert.py` demonstrate captured forward and
+backward with live index/parameter buffers. They do not graph the production
+trainer. Original per-expert use.any(), nonzero(), variable routed batch lengths,
+and finite-value host checks prevent simply wrapping the loop in CUDAGraph.
+
+Next integration: precompute routing outside capture, use fixed/padded row-count
+buckets with correct zero masks, capture expert forward/backward, and keep the
+coupled-output all-reduce and discrete acceptance outside the initial graph.
+Stable parameter/code buffer addresses and graph-memory budgets are required.
+Validate finiteness outside captured execution; do not silently remove checks.
+
+## Verification and next qualification
+
+- Graph-compatible activation forward and STE derivative match the existing path.
+- Sparse communication matches dense broadcast exactly in two-rank tests.
+- Real GPU graph benchmark checks loss, gradients, and live code/scale updates.
+- All tests in `perf/test_perf.py` pass; new modules compile.
+
+After the quality pilot, compare baseline and fast fork on the same eight-GPU
+layer and sample schedule, including an actual reassignment cycle. Measure
+continuous steps, proposals, validation, peak memory and held-out error separately.
+Then integrate graph buckets if their additional gain justifies the complexity.
+No full-model runtime estimate should be reduced by the single-expert 3.25x factor.
+
+## Follow-up: specialized codebook backward
+
+An actual CPU/CUDA operator profile (`/tmp/glm53-pv-operator-profile.txt`)
+identified general advanced-index backward as 83% of self CUDA time in the
+measured expert step; matrix multiplication accounted for <1%. This motivated
+using `torch.nn.functional.embedding` for each book lookup, instead of `cb[a]`.
+It changes the lookup/reduction implementation, not the representation or loss.
+
+Two separate real-expert benchmarks while the pilot was active:
+
+| Layer/expert | Four microbatches, original | One batch, original | One batch, embedding | Embedding + graphs |
+| --- | ---: | ---: | ---: | ---: |
+| 3 / 3 | 442.67 ms | 115.18 ms | 35.39 ms | 25.37 ms |
+| 4 / 0 | 373.97 ms | 133.65 ms | 35.19 ms | 34.75 ms |
+
+These include contention and are single-expert forward/backward measurements.
+The combined ratio is about 11–17x for this path, NOT the whole PV campaign.
+Graph benefit is variable here; specialized embedding backward and batching are
+more consistently material than launch replay alone.
+
+Outputs and losses match exactly between original one-batch, embedding and
+graphed embedding. GPU parameter-gradient relative differences are ~1.5–2.2e-6,
+consistent with a changed FP32 reduction order. CPU float64 tests independently
+verify matching parameter gradients, including after changing index buffers.
+Graph replay after changing indices and row corrections still matches eager.
+
+Artifacts: `/tmp/glm53-pv-embedding-benchmark.json` and
+`/tmp/glm53-pv-embedding-layer4.json`. `EmbeddingRowProjection` implements the
+isolated prototype; the specialized embedding lookup is now also present in
+`sequential_pv_fast.py`. All three performance tests pass. The active pilot still
+uses its original code. Full-layer NCCL, memory, quality and timing qualification
+is required before switching a production run to the faster fork.
+
+## Qualified full sequential campaign (18 September 2026)
+
+The completed faster rerun matched both validation and development-audit error
+exactly for reference layers 3 and 4. All propagated BF16 values matched exactly.
+Measured training loops (including reassignment/validation) improved from
+1525.16 to 336.60 seconds for layer 3 and 1226.31 to 303.00 seconds for layer 4.
+This qualifies the larger microbatch, embedding backward and sparse broadcasts;
+CUDA graphs remain a separate prototype and are not in the full trainer.
+
+`full_reference.py --work /tmp/glm53-sequential-reference-full --pilot
+/tmp/glm53-sequential-pilot-fast` reuses those two fitted layers and chains capture
+and 200-step reference tuning through layer 77. `sequential_capture.py` now reads
+reference and retained student outputs from the immediately preceding layer.
+It retains the original corpus, allocation, seeds and tuning hyperparameters.
+An additional initial development-audit evaluation permits an audit regression
+gate before propagation/publication. Validation and audit gates allow only
+1e-6 relative numerical slack; failure stops for investigation. The audit is
+therefore development data, not an untouched final generalization estimate.
+
+`publish_reference.py --work /tmp/glm53-sequential-reference-full` runs separately
+on CPU/network. It reconstructs cold-slot order across eight expert shards,
+checks slot coverage/allocation/global metadata, invokes the existing exact
+index roundtrip and sampled decoded-weight parity checks, then atomically
+replaces both projection files and reports on HF with parent-commit protection.
+Remote hashes are verified before recording completion. Progress explicitly
+identifies remaining legacy-PV and initial-fit layers. The old campaign and
+publisher remain stopped. Full-model quality/native-SM120 qualification remains
+pending; neither pilot proves that outcome.