S1: vanilla PE vs GAB (matched steps)
Date: 2026-08-03
Worktree: .worktrees/chessformer-lite
Command: python -m experiments.chessformer_lite.run_s1_compare --train-path /mnt/c/Users/jun/chessdb/data/train/data.parquet --out runs/chessformer_lite/s1 --steps 20000 --device cuda
Artifact: runs/chessformer_lite/s1/compare.json
Setup
| Knob | Value |
|---|---|
| steps | 20,000 |
| batch | 256 |
| warmup | 2,000 |
| seed | 0 (shared trunk init matched vanilla→GAB) |
| d_model / layers / heads / d_ff | 256 / 6 / 8 / 384 |
| probe | 4,096 held-out train rows |
Results
| Arm | params | wall (s) | ms/step | final loss | policy CE (final) | probe top-1 | probe policy CE | train FLOPs |
|---|---|---|---|---|---|---|---|---|
| vanilla | 2,992,071 | 1074 | 53.7 | 3.134 | 2.455 | 0.464 | 1.758 | 5.97e15 |
| GAB | 7,566,471 | 1171 | 58.6 | 2.776 | 2.127 | 0.488 | 1.626 | 1.51e16 |
ms/step ratio GAB/vanilla = 1.09 (cap 1.30) → speed OK.
Decision
GO — scale GAB.
- Better final loss and better probe policy CE.
- Quality win under matched steps/data/seed with shared trunk init.
- Step-time overhead well under 1.3× despite ~2.5× params (GAB templates dominate param count, not step cost).
Next
S2: run_s2_scale --use-gab true with flop_cap 4.665e17, then eval_play vs v1-final.