train.log

Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
delta: warm start from /__modal/volumes/vo-kEMu8BkBAIrorAQI6V8f2D/hub/models--jaredpalmer--kev-0.8b/snapshots/c917edefdfd72b3e9ba71455584700acc70595f6: 372 adapter tensors and the pointer head loaded
device=cuda trainable params=11.3M
replay: 2000 of 12576 suite training records mixed with 1425 from evals/night2/dates_unknowable.jsonl
3425 training requests (holdout=[]), questions by type {'noul': 1268, 'score': 1431, 'choice': 1214}
[transformers] `causal_conv1d_fn` is falling back to its reference PyTorch implementation because `causal_conv1d` is not installed. This is correct but much slower; install `causal_conv1d` for the optimized kernel.
ep0 step 10/429 loss 0.364 kl 0.000 anchor 0.000 1.620s/rec
ep0 step 20/429 loss 0.487 kl 0.000 anchor 0.000 0.946s/rec
ep0 step 30/429 loss 0.310 kl 0.000 anchor 0.000 0.650s/rec
ep0 step 40/429 loss 0.476 kl 0.000 anchor 0.000 0.506s/rec
ep0 step 50/429 loss 0.393 kl 0.000 anchor 0.000 0.416s/rec
ep0 step 60/429 loss 0.418 kl 0.000 anchor 0.000 0.346s/rec
ep0 step 70/429 loss 0.449 kl 0.000 anchor 0.000 0.301s/rec
ep0 step 80/429 loss 0.461 kl 0.000 anchor 0.000 0.266s/rec
ep0 step 90/429 loss 0.257 kl 0.000 anchor 0.000 0.240s/rec
ep0 step 100/429 loss 0.734 kl 0.000 anchor 0.000 0.222s/rec
ep0 step 110/429 loss 0.427 kl 0.000 anchor 0.000 0.205s/rec
ep0 step 120/429 loss 0.269 kl 0.000 anchor 0.000 0.190s/rec
ep0 step 130/429 loss 0.658 kl 0.000 anchor 0.000 0.177s/rec
ep0 step 140/429 loss 0.192 kl 0.000 anchor 0.000 0.181s/rec
ep0 step 150/429 loss 0.390 kl 0.000 anchor 0.000 0.171s/rec
ep0 step 160/429 loss 0.528 kl 0.000 anchor 0.000 0.161s/rec
ep0 step 170/429 loss 0.385 kl 0.000 anchor 0.000 0.153s/rec
ep0 step 180/429 loss 0.368 kl 0.000 anchor 0.000 0.146s/rec
ep0 step 190/429 loss 0.362 kl 0.000 anchor 0.000 0.140s/rec
ep0 step 200/429 loss 0.365 kl 0.000 anchor 0.000 0.135s/rec
ep0 step 210/429 loss 0.267 kl 0.000 anchor 0.000 0.130s/rec
ep0 step 220/429 loss 0.463 kl 0.000 anchor 0.000 0.125s/rec
ep0 step 230/429 loss 0.235 kl 0.000 anchor 0.000 0.121s/rec
ep0 step 240/429 loss 0.228 kl 0.000 anchor 0.000 0.117s/rec
ep0 step 250/429 loss 0.267 kl 0.000 anchor 0.000 0.113s/rec
ep0 step 260/429 loss 0.424 kl 0.000 anchor 0.000 0.110s/rec
ep0 step 270/429 loss 0.231 kl 0.000 anchor 0.000 0.107s/rec
ep0 step 280/429 loss 0.409 kl 0.000 anchor 0.000 0.104s/rec
ep0 step 290/429 loss 0.401 kl 0.000 anchor 0.000 0.102s/rec
ep0 step 300/429 loss 0.402 kl 0.000 anchor 0.000 0.099s/rec
ep0 step 310/429 loss 0.264 kl 0.000 anchor 0.000 0.097s/rec
ep0 step 320/429 loss 0.337 kl 0.000 anchor 0.000 0.095s/rec
ep0 step 330/429 loss 0.552 kl 0.000 anchor 0.000 0.093s/rec
ep0 step 340/429 loss 0.436 kl 0.000 anchor 0.000 0.091s/rec
ep0 step 350/429 loss 0.309 kl 0.000 anchor 0.000 0.089s/rec
ep0 step 360/429 loss 0.239 kl 0.000 anchor 0.000 0.088s/rec
ep0 step 370/429 loss 0.449 kl 0.000 anchor 0.000 0.087s/rec
ep0 step 380/429 loss 0.207 kl 0.000 anchor 0.000 0.085s/rec
ep0 step 390/429 loss 0.430 kl 0.000 anchor 0.000 0.084s/rec
ep0 step 400/429 loss 0.216 kl 0.000 anchor 0.000 0.082s/rec
ep0 step 410/429 loss 0.219 kl 0.000 anchor 0.000 0.081s/rec
ep0 step 420/429 loss 0.284 kl 0.000 anchor 0.000 0.080s/rec
saved /runs/night2-08b-du2/00-trial-0/checkpoint