YifeiZuo/attention-0.6b-adamw-wsd-step500
0.6B • Updated
Attention 0.6B AdamW-WSD training trajectory
Per-step record (every 500 steps, 40 ckpts) of the 0.6B Qwen3 softmax-attention baseline trained AdamW + WSD on 80B tokens.
-
-
YifeiZuo/attention-0.6b-adamw-wsd-step1000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step1500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step2000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step2500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step3000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step3500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step4000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step4500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step5000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step5500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step6000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step6500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step7000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step7500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step8000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step8500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step9000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step9500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step10000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step10500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step11000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step11500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step12000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step12500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step13000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step13500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step14000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step14500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step15000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step15500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step16000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step16500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step17000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step17500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step18000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step18500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step19000
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step19500
0.6B • Updated -
YifeiZuo/attention-0.6b-adamw-wsd-step20000
0.6B • Updated • 6