Speedrun architecture: 8 pretraining seeds/size, anneal marks every 10 TPP as revisions. Plain ladder: “Scaling Ladder — 4 sizes, 135M–973M, 200 TPP”
Julian Minder
jkminder
AI & ML interests
None yet
Recent Activity
updated a model about 1 month ago
jkminder/pretraining-priors-doormail4k-d32-treated-base published a model about 1 month ago
jkminder/pretraining-priors-doormail4k-d32-treated-base updated a model about 1 month ago
jkminder/d20_477m_seed4_sftOrganizations
Speedrun architecture: 8 pretraining seeds/size, anneal marks every 10 TPP as revisions. Plain ladder: “Scaling Ladder — 4 sizes, 135M–973M, 200 TPP”
The whole ladder, 200 TPP on ClimbMix; plain GPT (nanochat mechanisms disabled). Bases, then chat-SFTs (_sft).
d26 models with the pirate-2x2 corpora inserted in pretraining: exp-056 anchor + exp-074 dose/window sweep, base and instruction-SFT pairs.