Guido 0.5B post-GRPO, step 380

This repository preserves the exact no-TIR, no-Muon, BETA=0 Guido checkpoint from Slurm job 48981274 (grpo_is_cos500). It is the checkpoint used in the reported CP4-versus-post-GRPO evaluations. It must not be confused with the later Muon runs or the legacy GRPO checkpoints 2562 and 2566.

Canonical files

checkpoints/step_000380.pt: inference policy, an Aplos/PyTorch state dict.
checkpoints/step_000380.train.pt: complete resume state, including optimizer state and FP32 master weights.
checkpoints/step_000380.json: machine-readable snapshot metadata.
reproduction/train_grpo_step380.py: exact trainer used by job 48981274.
reproduction/job_48981274.sbatch: exact Slurm submission script.
logs/grpo_is_cos500_g16lg32_4n_48981274.out and .err: complete job logs.
evals/final_cp4_vs_p380/: final three-seed evaluation tables and protocol.

Integrity

Artifact	Bytes	SHA-256
`checkpoints/step_000380.pt`	1,106,938,213	`45c555ea2cab95b6d13aff490c5fae7990a5f9167ce91b0ca39e877a34a8bc91`
`checkpoints/step_000380.train.pt`	6,641,303,821	`ccc9f3bc3ce8e987898d64910faacdf1da03d380177dd1c4b30875e39946ac1c`
`checkpoints/step_000380.json`	1,441	`c8109a4a9ce87e3b5d36c3b68a0414c4103f77c2d6d15f80ec2d39163ece550b`
`reproduction/train_grpo_step380.py`	see repository	`66400316e47aa0c1f46e59358278316c2c647263effca3428148585d55da4b4c`

The source checkpoint actually loaded by the run was BossCalabrese/guido-sft-10b/checkpoints/sft_final.pt, SHA-256 6e2685fa0942bda88e17aa7ef023dc59d4bac87cadfe4d310810077fbfaee5bc.

Snapshot state

rollout batch: 380
optimizer step: 760
usable groups: 12160 / 16000
cumulative raw reward: 0.5026031199758842
batch raw reward: 0.5872395833333334
batch usable-group reward: 0.515625
current policy LR: about 4.2e-6

The overall job later failed after batch 383, so this repository deliberately uses the durable step-380 snapshot that all post-GRPO evaluations referenced.

Training configuration

The run used 4 Leonardo nodes, 4 A100 GPUs per node (world_size=16):

Setting	Value
`G`	16
`LOCAL_GROUPS`	32
`MU`	2
`MICRO`	2
`T` / max new tokens	2048
`TARGET_USABLE`	16000
`LOSS_NORM`	`dr_grpo`
`ADV_SCALE_BY_STD`	0
`LR`	`3e-5`
scheduler	cosine, 20 warmup optimizer steps, 1000 total optimizer steps, floor 0
`BETA`	0
`DUAL_CLIP`	3
PPO clipping	0.2 / 0.2
policy dtype	BF16
optimizer	`torch.optim.Adam`, default betas, no weight decay; no Muon
rollout sampling	`temperature=1.0`, `top_p=1.0`
behavior correction	generation log-probs returned, `RATIO_DENOM=rollout`
stale queue	dropped; `ASSERT_QAGE_ZERO=1`
decode	compiled
seed	123

Other provenance:

data: Guido-GRPO-pipeline/grpo_pool_FINAL.jsonl, SHA-256 604cbe4a1fc34e67814a399ab6284e47b90c1941913a6a761bdd83539bf0c18b
Aplos git commit: 47492b2
Aplos Vathos/blocks.py SHA-256: 2b3a9029516da1edec835817003b978b89d15a6ce229f5a68ad1b799c3a0fcb3
Aplos Vathos/functions.py SHA-256: 6beda892946f9a013719563ab324788f8c808d2ffa037c32bdf6448d75857179
the loader reported missing=0, unexpected=0, and patched all 24 FFN blocks to SiLU.

Final sampled evaluation

Protocol: 16 samples per problem, temperature 0.6, top-p 0.95, no top-k, 2048 generated tokens, prompt cap 512, compiled decoding, seeds 1234/2027/3407. Values below are mean +/- sample standard deviation across seeds.

Benchmark	maj@16 strict	pass@1 strict	pass@16 strict	boxed
MATH500	69.09 +/- 0.12	60.82 +/- 1.11	83.94 +/- 0.31	100.00 +/- 0.00
GSM8K	77.86 +/- 0.20	71.19 +/- 0.66	90.12 +/- 0.19	100.00 +/- 0.00
AIME	4.81 +/- 0.64	4.44 +/- 1.92	13.70 +/- 0.64	100.00 +/- 0.00
AMC	38.55 +/- 1.20	31.33 +/- 4.17	59.44 +/- 4.23	100.00 +/- 0.00
AIME 2025	11.54 +/- 0.00	8.97 +/- 2.22	20.51 +/- 2.22	100.00 +/- 0.00

The complete tables, paired CP4 deltas, settings, and raw evaluation material are also preserved in the private dataset repository BossCalabrese/Guido-GRPO-pipeline.

Loading

The policy file is a plain ordered PyTorch state dict:

import torch

state_dict = torch.load("checkpoints/step_000380.pt", map_location="cpu")
model.load_state_dict(state_dict, strict=True)

Use the exact trainer and Aplos commit recorded above for training resume or strict reproduction. step_000380.train.pt is not required for inference.

Downloads last month: -; Downloads are not tracked for this model. How to track

Video Preview

Reinforcement Learning