probe_s12A_440k — decay probe of s12A_arcmix_pool/ at step 440,000
Why it exists: a short side run that measures what the arm would score if training stopped here. The arm (ARC-MIX pool, building pair 1 arm A) trains at a constant learning rate, and constant-LR checkpoints are not comparable with decayed models. This probe copies the arm's step-440,000 checkpoint and runs only the decay phase: the learning rate decays from 3e-4 to 6e-5 over 4,000 steps with a 1−sqrt schedule (as in the trainer's WSD decay) (440,000 → 444,000), on the same data and with the same optimizer state. The arm itself is not affected and keeps training.
- Use: segment 2 of the pair is decided on the difference between this probe and the decay probe of arm B2 (
probe_s12B2_440k/) on the held-out selection sets. - Status: research checkpoint, not a leaderboard submission.
- Format: PyTorch checkpoint dict with
model,opt,step,config;train_gpt_ref.pyin the repository root rebuilds the model fromconfig. - Data: as in
s12A_arcmix_pool/; see the root card of this repository.