Device: cuda ObjA sources: Resolving data files: 100%|██████████████████| 48/48 [00:00<00:00, 50801.56it/s] Resolving data files: 100%|██████████████████| 48/48 [00:00<00:00, 40410.80it/s] LibriSpeech (train.100): 28,539 utterances Twi HuggingFace (train): +15,560 utterances ObjB sources: twi-english-paragraph-dataset_news: +256,797 pairs english-twi-sentences-non-nouns: +1,763,936 pairs english-twi-nouns-v2: +625,988 pairs ObjC sources: Resolving data files: 100%|██████████████████| 48/48 [00:00<00:00, 32901.88it/s] Resolving data files: 100%|██████████████████| 48/48 [00:00<00:00, 45374.49it/s] LibriSpeech (train.100): 28,539 utterances Twi HuggingFace (train): +15,560 utterances Steps/epoch: 85,466 total: 854,660 epoch 1/10: 1%|▏ | 499/85466 [01:59<5:57:42, 3.96it/s] step 00500 B loss 0.0002 std 0.029 cos 0.955 lr 3.00e-04 epoch 1/10: 1%|▎ | 999/85466 [03:52<4:44:35, 4.95it/s] step 01000 A loss 0.0001 std 0.027 cos 0.964 lr 3.00e-04 epoch 1/10: 2%|▍ | 1498/85466 [05:47<6:09:34, 3.79it/s] step 01500 C loss 0.0001 std 0.025 cos 0.977 lr 3.00e-04 epoch 1/10: 2%|▌ | 1999/85466 [07:43<6:23:40, 3.63it/s] step 02000 B loss 0.0002 std 0.037 cos 0.942 lr 3.00e-04 epoch 1/10: 3%|▋ | 2499/85466 [09:37<4:50:48, 4.75it/s] step 02500 A loss 0.0006 std 0.033 cos 0.857 lr 3.00e-04 epoch 1/10: 4%|▊ | 2998/85466 [11:31<5:52:13, 3.90it/s] step 03000 C loss 0.0004 std 0.031 cos 0.904 lr 3.00e-04 epoch 1/10: 4%|▉ | 3499/85466 [13:26<5:50:11, 3.90it/s] step 03500 B loss 0.0002 std 0.037 cos 0.956 lr 3.00e-04 epoch 1/10: 5%|█ | 3999/85466 [15:21<4:29:14, 5.04it/s] step 04000 A loss 0.0005 std 0.036 cos 0.871 lr 3.00e-04 epoch 1/10: 6%|█▌ | 5226/85466 [16:21<33:33, 39.85it/s] step 04500 B loss 0.0002 std 0.036 cos 0.943 lr 3.00e-04 epoch 1/10: 8%|██ | 6725/85466 [17:00<27:55, 47.01it/s] step 05000 B loss 0.0003 std 0.035 cos 0.930 lr 3.00e-04 epoch 1/10: 10%|██▌ | 8225/85466 [17:39<29:28, 43.69it/s] step 05500 B loss 0.0001 std 0.038 cos 0.977 lr 3.00e-04 epoch 1/10: 11%|██▉ | 9728/85466 [18:19<29:14, 43.16it/s] step 06000 B loss 0.0001 std 0.038 cos 0.970 lr 3.00e-04 epoch 1/10: 13%|███▎ | 11225/85466 [18:58<30:38, 40.38it/s] step 06500 B loss 0.0001 std 0.038 cos 0.973 lr 3.00e-04 epoch 1/10: 15%|███▋ | 12725/85466 [19:37<31:52, 38.03it/s] step 07000 B loss 0.0002 std 0.040 cos 0.943 lr 3.00e-04 epoch 1/10: 17%|████▏ | 14226/85466 [20:15<31:21, 37.86it/s] step 07500 B loss 0.0003 std 0.040 cos 0.916 lr 3.00e-04 epoch 1/10: 18%|████▌ | 15728/85466 [20:53<29:59, 38.75it/s] step 08000 B loss 0.0002 std 0.041 cos 0.942 lr 3.00e-04 epoch 1/10: 20%|█████ | 17228/85466 [21:31<24:17, 46.81it/s] step 08500 B loss 0.0002 std 0.040 cos 0.941 lr 3.00e-04 epoch 1/10: 22%|█████▍ | 18728/85466 [22:10<29:16, 38.00it/s] step 09000 B loss 0.0002 std 0.038 cos 0.947 lr 3.00e-04 epoch 1/10: 24%|█████▉ | 20225/85466 [22:49<28:57, 37.55it/s] step 09500 B loss 0.0002 std 0.040 cos 0.952 lr 3.00e-04 epoch 1/10: 25%|██████▎ | 21725/85466 [23:28<25:07, 42.29it/s] step 10000 B loss 0.0002 std 0.040 cos 0.955 lr 3.00e-04 epoch 1/10: 27%|██████▊ | 23228/85466 [24:06<34:22, 30.18it/s] step 10500 B loss 0.0003 std 0.041 cos 0.923 lr 3.00e-04 epoch 1/10: 29%|███████▏ | 24725/85466 [24:45<24:13, 41.78it/s] step 11000 B loss 0.0001 std 0.041 cos 0.966 lr 3.00e-04 epoch 1/10: 31%|███████▋ | 26225/85466 [25:25<28:09, 35.07it/s] step 11500 B loss 0.0001 std 0.040 cos 0.970 lr 3.00e-04 epoch 1/10: 32%|████████ | 27725/85466 [26:04<27:02, 35.58it/s] step 12000 B loss 0.0002 std 0.040 cos 0.947 lr 3.00e-04 epoch 1/10: 34%|████████▌ | 29227/85466 [26:42<23:06, 40.57it/s] step 12500 B loss 0.0001 std 0.041 cos 0.962 lr 3.00e-04 epoch 1/10: 36%|████████▉ | 30728/85466 [27:21<24:54, 36.62it/s] step 13000 B loss 0.0002 std 0.041 cos 0.946 lr 3.00e-04 epoch 1/10: 38%|█████████▍ | 32228/85466 [28:00<29:49, 29.76it/s] step 13500 B loss 0.0004 std 0.039 cos 0.904 lr 3.00e-04 epoch 1/10: 39%|█████████▊ | 33725/85466 [28:39<22:18, 38.66it/s] step 14000 B loss 0.0002 std 0.041 cos 0.962 lr 3.00e-04 epoch 1/10: 41%|██████████▎ | 35228/85466 [29:17<19:51, 42.16it/s] step 14500 B loss 0.0002 std 0.041 cos 0.957 lr 3.00e-04 epoch 1/10: 43%|██████████▋ | 36728/85466 [29:57<23:29, 34.57it/s] step 15000 B loss 0.0004 std 0.041 cos 0.903 lr 3.00e-04 epoch 1/10: 45%|███████████▏ | 38228/85466 [30:35<18:06, 43.47it/s] step 15500 B loss 0.0003 std 0.040 cos 0.921 lr 3.00e-04 epoch 1/10: 46%|███████████▌ | 39728/85466 [31:15<19:47, 38.52it/s] step 16000 B loss 0.0003 std 0.040 cos 0.915 lr 3.00e-04 epoch 1/10: 48%|████████████ | 41228/85466 [31:55<23:13, 31.74it/s] step 16500 B loss 0.0002 std 0.041 cos 0.948 lr 3.00e-04 epoch 1/10: 50%|████████████▍ | 42728/85466 [32:34<19:15, 36.99it/s] step 17000 B loss 0.0003 std 0.041 cos 0.935 lr 3.00e-04 epoch 1/10: 52%|████████████▉ | 44228/85466 [33:12<16:38, 41.30it/s] step 17500 B loss 0.0002 std 0.040 cos 0.953 lr 3.00e-04 epoch 1/10: 54%|█████████████▍ | 45728/85466 [33:50<14:31, 45.57it/s] step 18000 B loss 0.0003 std 0.042 cos 0.934 lr 3.00e-04 epoch 1/10: 55%|█████████████▊ | 47228/85466 [34:28<17:35, 36.24it/s] step 18500 B loss 0.0003 std 0.040 cos 0.927 lr 3.00e-04 epoch 1/10: 57%|██████████████▎ | 48728/85466 [35:07<14:38, 41.82it/s] step 19000 B loss 0.0002 std 0.041 cos 0.960 lr 3.00e-04 epoch 1/10: 59%|██████████████▋ | 50228/85466 [35:46<13:40, 42.92it/s] step 19500 B loss 0.0005 std 0.042 cos 0.872 lr 3.00e-04 epoch 1/10: 61%|███████████████▏ | 51725/85466 [36:24<13:06, 42.89it/s] step 20000 B loss 0.0001 std 0.042 cos 0.963 lr 3.00e-04 epoch 1/10: 62%|███████████████▌ | 53228/85466 [37:02<16:10, 33.20it/s] step 20500 B loss 0.0003 std 0.041 cos 0.930 lr 3.00e-04 epoch 1/10: 64%|████████████████ | 54725/85466 [37:40<13:11, 38.85it/s] step 21000 B loss 0.0003 std 0.040 cos 0.931 lr 3.00e-04 epoch 1/10: 66%|████████████████▍ | 56228/85466 [38:20<14:29, 33.63it/s] step 21500 B loss 0.0004 std 0.042 cos 0.909 lr 3.00e-04 epoch 1/10: 68%|████████████████▉ | 57725/85466 [38:59<10:31, 43.93it/s] step 22000 B loss 0.0003 std 0.041 cos 0.936 lr 3.00e-04 epoch 1/10: 69%|█████████████████▎ | 59225/85466 [39:37<13:56, 31.38it/s] step 22500 B loss 0.0004 std 0.040 cos 0.906 lr 3.00e-04 epoch 1/10: 71%|█████████████████▊ | 60728/85466 [40:16<11:50, 34.82it/s] step 23000 B loss 0.0003 std 0.040 cos 0.917 lr 2.99e-04 epoch 1/10: 73%|██████████████████▏ | 62228/85466 [40:53<08:42, 44.46it/s] step 23500 B loss 0.0004 std 0.041 cos 0.909 lr 2.99e-04 epoch 1/10: 75%|██████████████████▋ | 63728/85466 [41:33<09:07, 39.69it/s] step 24000 B loss 0.0003 std 0.041 cos 0.917 lr 2.99e-04 epoch 1/10: 76%|███████████████████ | 65228/85466 [42:12<08:25, 40.04it/s] step 24500 B loss 0.0003 std 0.041 cos 0.918 lr 2.99e-04 epoch 1/10: 78%|███████████████████▌ | 66728/85466 [42:50<07:16, 42.90it/s] step 25000 B loss 0.0003 std 0.042 cos 0.934 lr 2.99e-04 epoch 1/10: 80%|███████████████████▉ | 68227/85466 [43:29<06:45, 42.47it/s] step 25500 B loss 0.0003 std 0.040 cos 0.927 lr 2.99e-04 epoch 1/10: 82%|████████████████████▍ | 69725/85466 [44:07<08:39, 30.28it/s] step 26000 B loss 0.0004 std 0.042 cos 0.901 lr 2.99e-04 epoch 1/10: 83%|████████████████████▊ | 71228/85466 [44:46<06:58, 34.00it/s] step 26500 B loss 0.0003 std 0.041 cos 0.925 lr 2.99e-04 epoch 1/10: 85%|█████████████████████▎ | 72726/85466 [45:25<06:32, 32.45it/s] step 27000 B loss 0.0002 std 0.042 cos 0.950 lr 2.99e-04 epoch 1/10: 87%|█████████████████████▋ | 74228/85466 [46:05<04:22, 42.89it/s] step 27500 B loss 0.0003 std 0.042 cos 0.929 lr 2.99e-04 epoch 1/10: 89%|██████████████████████▏ | 75728/85466 [46:44<04:20, 37.32it/s] step 28000 B loss 0.0002 std 0.041 cos 0.951 lr 2.99e-04 epoch 1/10: 90%|██████████████████████▌ | 77228/85466 [47:24<03:36, 38.14it/s] step 28500 B loss 0.0003 std 0.040 cos 0.932 lr 2.99e-04 epoch 1/10: 92%|███████████████████████ | 78728/85466 [48:02<02:58, 37.80it/s] step 29000 B loss 0.0002 std 0.042 cos 0.943 lr 2.99e-04 epoch 1/10: 94%|███████████████████████▍ | 80225/85466 [48:41<02:14, 38.91it/s] step 29500 B loss 0.0002 std 0.043 cos 0.953 lr 2.99e-04 epoch 1/10: 96%|███████████████████████▉ | 81725/85466 [49:20<01:32, 40.51it/s] step 30000 B loss 0.0002 std 0.043 cos 0.956 lr 2.99e-04 epoch 1/10: 97%|████████████████████████▎| 83228/85466 [49:59<01:00, 37.27it/s] step 30500 B loss 0.0002 std 0.041 cos 0.945 lr 2.99e-04 epoch 1/10: 99%|████████████████████████▊| 84728/85466 [50:39<00:19, 38.77it/s] step 31000 B loss 0.0004 std 0.042 cos 0.889 lr 2.99e-04 epoch 1/10: 100%|█████████████████████████| 85466/85466 [50:59<00:00, 27.94it/s] epoch 1/10 avg_loss 0.0001 3060s epoch 2/10: 0%| | 253/85466 [01:00<6:10:49, 3.83it/s] step 31500 C loss 0.0002 std 0.025 cos 0.954 lr 2.99e-04 epoch 2/10: 1%|▏ | 754/85466 [02:56<5:58:04, 3.94it/s] step 32000 B loss 0.0002 std 0.041 cos 0.939 lr 2.99e-04 epoch 2/10: 1%|▎ | 1254/85466 [04:51<4:44:48, 4.93it/s] step 32500 A loss 0.0003 std 0.028 cos 0.919 lr 2.99e-04 epoch 2/10: 2%|▍ | 1753/85466 [06:46<6:09:23, 3.78it/s] step 33000 C loss 0.0003 std 0.031 cos 0.930 lr 2.99e-04 epoch 2/10: 3%|▋ | 2254/85466 [08:41<5:49:34, 3.97it/s] step 33500 B loss 0.0003 std 0.041 cos 0.932 lr 2.99e-04 epoch 2/10: 3%|▊ | 2754/85466 [10:36<4:35:39, 5.00it/s] step 34000 A loss 0.0004 std 0.032 cos 0.909 lr 2.99e-04 epoch 2/10: 4%|▉ | 3253/85466 [12:31<6:01:09, 3.79it/s] step 34500 C loss 0.0002 std 0.029 cos 0.948 lr 2.99e-04 epoch 2/10: 4%|█ | 3754/85466 [14:27<5:58:58, 3.79it/s] step 35000 B loss 0.0002 std 0.041 cos 0.945 lr 2.99e-04 epoch 2/10: 5%|█▎ | 4490/85466 [16:04<35:34, 37.93it/s] step 35500 B loss 0.0004 std 0.042 cos 0.907 lr 2.99e-04 epoch 2/10: 7%|█▊ | 5993/85466 [16:43<35:42, 37.10it/s] step 36000 B loss 0.0002 std 0.042 cos 0.961 lr 2.99e-04 epoch 2/10: 9%|██▎ | 7491/85466 [17:20<35:41, 36.40it/s] step 36500 B loss 0.0003 std 0.041 cos 0.920 lr 2.99e-04 epoch 2/10: 11%|██▋ | 8993/85466 [17:59<32:00, 39.81it/s] step 37000 B loss 0.0003 std 0.042 cos 0.918 lr 2.99e-04 epoch 2/10: 12%|███ | 10493/85466 [18:38<37:11, 33.60it/s] step 37500 B loss 0.0003 std 0.041 cos 0.931 lr 2.99e-04 epoch 2/10: 14%|███▌ | 11990/85466 [19:17<30:56, 39.59it/s] step 38000 B loss 0.0002 std 0.042 cos 0.952 lr 2.99e-04 epoch 2/10: 16%|███▉ | 13493/85466 [19:57<28:28, 42.13it/s] step 38500 B loss 0.0002 std 0.042 cos 0.945 lr 2.99e-04 epoch 2/10: 18%|████▍ | 14994/85466 [20:37<34:52, 33.68it/s] step 39000 B loss 0.0004 std 0.041 cos 0.897 lr 2.98e-04 epoch 2/10: 19%|████▊ | 16493/85466 [21:16<26:08, 43.97it/s] step 39500 B loss 0.0003 std 0.042 cos 0.935 lr 2.98e-04 epoch 2/10: 21%|█████▎ | 17990/85466 [21:54<29:21, 38.30it/s] step 40000 B loss 0.0004 std 0.043 cos 0.892 lr 2.98e-04 epoch 2/10: 23%|█████▋ | 19493/85466 [22:33<30:29, 36.07it/s] step 40500 B loss 0.0003 std 0.041 cos 0.924 lr 2.98e-04 epoch 2/10: 25%|██████▏ | 20993/85466 [23:11<26:19, 40.82it/s] step 41000 B loss 0.0002 std 0.042 cos 0.937 lr 2.98e-04 epoch 2/10: 26%|██████▌ | 22490/85466 [23:50<25:48, 40.68it/s] step 41500 B loss 0.0004 std 0.042 cos 0.910 lr 2.98e-04 epoch 2/10: 28%|███████ | 23990/85466 [24:29<30:27, 33.64it/s] step 42000 B loss 0.0002 std 0.043 cos 0.942 lr 2.98e-04 epoch 2/10: 30%|███████▍ | 25490/85466 [25:09<25:28, 39.23it/s] step 42500 B loss 0.0003 std 0.041 cos 0.929 lr 2.98e-04 epoch 2/10: 32%|███████▉ | 26990/85466 [25:48<23:37, 41.26it/s] step 43000 B loss 0.0003 std 0.041 cos 0.927 lr 2.98e-04 epoch 2/10: 33%|████████▎ | 28493/85466 [26:29<26:44, 35.50it/s] step 43500 B loss 0.0002 std 0.043 cos 0.956 lr 2.98e-04 epoch 2/10: 35%|████████▊ | 29990/85466 [27:09<25:28, 36.29it/s] step 44000 B loss 0.0003 std 0.042 cos 0.927 lr 2.98e-04 epoch 2/10: 37%|█████████▏ | 31493/85466 [27:49<21:38, 41.58it/s] step 44500 B loss 0.0003 std 0.042 cos 0.913 lr 2.98e-04 epoch 2/10: 39%|█████████▋ | 32990/85466 [28:27<21:35, 40.50it/s] step 45000 B loss 0.0006 std 0.042 cos 0.856 lr 2.98e-04 epoch 2/10: 40%|██████████ | 34490/85466 [29:06<22:07, 38.40it/s] step 45500 B loss 0.0004 std 0.040 cos 0.909 lr 2.98e-04 epoch 2/10: 42%|██████████▌ | 35990/85466 [29:45<20:29, 40.23it/s] step 46000 B loss 0.0002 std 0.043 cos 0.938 lr 2.98e-04 epoch 2/10: 44%|██████████▉ | 37493/85466 [30:24<19:30, 41.00it/s] step 46500 B loss 0.0002 std 0.042 cos 0.955 lr 2.98e-04 epoch 2/10: 46%|███████████▍ | 38990/85466 [31:03<19:56, 38.83it/s] step 47000 B loss 0.0005 std 0.042 cos 0.872 lr 2.98e-04 epoch 2/10: 47%|███████████▊ | 40490/85466 [31:43<16:57, 44.22it/s] step 47500 B loss 0.0002 std 0.042 cos 0.943 lr 2.98e-04 epoch 2/10: 49%|████████████▎ | 41993/85466 [32:22<18:49, 38.48it/s] step 48000 B loss 0.0004 std 0.042 cos 0.904 lr 2.98e-04 epoch 2/10: 51%|████████████▋ | 43493/85466 [33:01<20:11, 34.65it/s] step 48500 B loss 0.0003 std 0.041 cos 0.934 lr 2.98e-04 epoch 2/10: 53%|█████████████▏ | 44990/85466 [33:40<18:32, 36.40it/s] step 49000 B loss 0.0003 std 0.042 cos 0.927 lr 2.98e-04 epoch 2/10: 54%|█████████████▌ | 46490/85466 [34:20<17:42, 36.68it/s] step 49500 B loss 0.0004 std 0.042 cos 0.903 lr 2.98e-04 epoch 2/10: 56%|██████████████ | 47990/85466 [35:00<20:45, 30.10it/s] step 50000 B loss 0.0002 std 0.042 cos 0.945 lr 2.98e-04 epoch 2/10: 58%|██████████████▍ | 49493/85466 [35:39<13:42, 43.73it/s] step 50500 B loss 0.0005 std 0.042 cos 0.881 lr 2.97e-04 epoch 2/10: 60%|██████████████▉ | 50990/85466 [36:18<15:02, 38.20it/s] step 51000 B loss 0.0003 std 0.042 cos 0.930 lr 2.97e-04 epoch 2/10: 61%|███████████████▎ | 52492/85466 [36:58<13:07, 41.89it/s] step 51500 B loss 0.0003 std 0.042 cos 0.926 lr 2.97e-04 epoch 2/10: 63%|███████████████▊ | 53993/85466 [37:36<14:07, 37.13it/s] step 52000 B loss 0.0002 std 0.042 cos 0.943 lr 2.97e-04 epoch 2/10: 65%|████████████████▏ | 55490/85466 [38:16<12:13, 40.87it/s] step 52500 B loss 0.0002 std 0.042 cos 0.937 lr 2.97e-04 epoch 2/10: 67%|████████████████▋ | 56990/85466 [38:56<11:58, 39.63it/s] step 53000 B loss 0.0002 std 0.041 cos 0.943 lr 2.97e-04 epoch 2/10: 68%|█████████████████ | 58490/85466 [39:36<10:38, 42.24it/s] step 53500 B loss 0.0003 std 0.042 cos 0.911 lr 2.97e-04 epoch 2/10: 70%|█████████████████▌ | 59990/85466 [40:15<10:23, 40.84it/s] step 54000 B loss 0.0003 std 0.041 cos 0.934 lr 2.97e-04 epoch 2/10: 72%|█████████████████▉ | 61493/85466 [40:55<09:10, 43.54it/s] step 54500 B loss 0.0004 std 0.042 cos 0.898 lr 2.97e-04 epoch 2/10: 74%|██████████████████▍ | 62993/85466 [41:34<08:47, 42.60it/s] step 55000 B loss 0.0003 std 0.042 cos 0.931 lr 2.97e-04 epoch 2/10: 75%|██████████████████▊ | 64490/85466 [42:14<10:01, 34.89it/s] step 55500 B loss 0.0002 std 0.043 cos 0.944 lr 2.97e-04 epoch 2/10: 77%|███████████████████▎ | 65993/85466 [42:54<09:58, 32.54it/s] step 56000 B loss 0.0003 std 0.042 cos 0.925 lr 2.97e-04 epoch 2/10: 79%|███████████████████▋ | 67490/85466 [43:33<07:40, 39.03it/s] step 56500 B loss 0.0003 std 0.042 cos 0.926 lr 2.97e-04 epoch 2/10: 81%|████████████████████▏ | 68993/85466 [44:11<06:07, 44.84it/s] step 57000 B loss 0.0004 std 0.042 cos 0.909 lr 2.97e-04 epoch 2/10: 82%|████████████████████▌ | 70493/85466 [44:51<06:21, 39.20it/s] step 57500 B loss 0.0004 std 0.043 cos 0.904 lr 2.97e-04 epoch 2/10: 84%|█████████████████████ | 71990/85466 [45:30<05:17, 42.43it/s] step 58000 B loss 0.0003 std 0.041 cos 0.933 lr 2.97e-04 epoch 2/10: 86%|█████████████████████▍ | 73493/85466 [46:09<05:35, 35.67it/s] step 58500 B loss 0.0002 std 0.043 cos 0.954 lr 2.97e-04 epoch 2/10: 88%|█████████████████████▉ | 74994/85466 [46:50<04:07, 42.38it/s] step 59000 B loss 0.0002 std 0.041 cos 0.954 lr 2.97e-04 epoch 2/10: 90%|██████████████████████▍ | 76493/85466 [47:30<05:38, 26.50it/s] step 59500 B loss 0.0002 std 0.042 cos 0.945 lr 2.96e-04 epoch 2/10: 91%|██████████████████████▊ | 77995/85466 [48:09<03:21, 37.05it/s] step 60000 B loss 0.0003 std 0.043 cos 0.919 lr 2.96e-04 epoch 2/10: 93%|███████████████████████▎ | 79490/85466 [48:49<02:28, 40.33it/s] step 60500 B loss 0.0002 std 0.042 cos 0.957 lr 2.96e-04 epoch 2/10: 95%|███████████████████████▋ | 80993/85466 [49:28<01:49, 40.77it/s] step 61000 B loss 0.0003 std 0.042 cos 0.913 lr 2.96e-04 epoch 2/10: 97%|████████████████████████▏| 82493/85466 [50:07<01:13, 40.67it/s] step 61500 B loss 0.0003 std 0.042 cos 0.914 lr 2.96e-04 epoch 2/10: 98%|████████████████████████▌| 83993/85466 [50:46<00:38, 38.33it/s] step 62000 B loss 0.0002 std 0.042 cos 0.949 lr 2.96e-04 epoch 2/10: 100%|█████████████████████████| 85466/85466 [51:25<00:00, 27.70it/s] epoch 2/10 avg_loss 0.0001 3086s epoch 3/10: 0%| | 9/85466 [00:04<6:39:08, 3.57it/s] step 62500 A loss 0.0010 std 0.024 cos 0.747 lr 2.96e-04 epoch 3/10: 1%|▏ | 508/85466 [01:58<5:57:06, 3.97it/s] step 63000 C loss 0.0002 std 0.029 cos 0.939 lr 2.96e-04 epoch 3/10: 1%|▎ | 1009/85466 [03:53<5:48:07, 4.04it/s] step 63500 B loss 0.0003 std 0.041 cos 0.929 lr 2.96e-04 epoch 3/10: 2%|▍ | 1509/85466 [05:46<4:38:09, 5.03it/s] step 64000 A loss 0.0003 std 0.030 cos 0.916 lr 2.96e-04 epoch 3/10: 2%|▌ | 2008/85466 [07:40<5:57:03, 3.90it/s] step 64500 C loss 0.0004 std 0.032 cos 0.886 lr 2.96e-04 epoch 3/10: 3%|▋ | 2509/85466 [09:35<5:55:54, 3.88it/s] step 65000 B loss 0.0003 std 0.041 cos 0.919 lr 2.96e-04 epoch 3/10: 4%|▊ | 3009/85466 [11:28<4:45:30, 4.81it/s] step 65500 A loss 0.0003 std 0.032 cos 0.935 lr 2.96e-04 epoch 3/10: 4%|▉ | 3508/85466 [13:22<5:47:11, 3.93it/s] step 66000 C loss 0.0002 std 0.032 cos 0.959 lr 2.96e-04 epoch 3/10: 5%|█▏ | 4009/85466 [15:16<5:37:30, 4.02it/s] step 66500 B loss 0.0003 std 0.042 cos 0.926 lr 2.96e-04 epoch 3/10: 6%|█▌ | 5259/85466 [16:14<37:28, 35.67it/s] step 67000 B loss 0.0003 std 0.042 cos 0.924 lr 2.96e-04 epoch 3/10: 8%|██ | 6758/85466 [16:53<31:17, 41.93it/s] step 67500 B loss 0.0003 std 0.043 cos 0.919 lr 2.95e-04 epoch 3/10: 10%|██▌ | 8256/85466 [17:31<32:53, 39.12it/s] step 68000 B loss 0.0002 std 0.040 cos 0.937 lr 2.95e-04 epoch 3/10: 11%|██▉ | 9758/85466 [18:09<31:05, 40.58it/s] step 68500 B loss 0.0003 std 0.043 cos 0.933 lr 2.95e-04 epoch 3/10: 13%|███▎ | 11258/85466 [18:47<30:55, 39.99it/s] step 69000 B loss 0.0005 std 0.043 cos 0.881 lr 2.95e-04 epoch 3/10: 15%|███▋ | 12755/85466 [19:26<34:13, 35.42it/s] step 69500 B loss 0.0004 std 0.043 cos 0.905 lr 2.95e-04 epoch 3/10: 17%|████▏ | 14258/85466 [20:05<32:22, 36.67it/s] step 70000 B loss 0.0002 std 0.042 cos 0.955 lr 2.95e-04 epoch 3/10: 18%|████▌ | 15758/85466 [20:42<31:24, 36.99it/s] step 70500 B loss 0.0003 std 0.042 cos 0.930 lr 2.95e-04 epoch 3/10: 20%|█████ | 17255/85466 [21:20<28:58, 39.25it/s] step 71000 B loss 0.0006 std 0.042 cos 0.857 lr 2.95e-04 epoch 3/10: 22%|█████▍ | 18758/85466 [22:00<28:05, 39.58it/s] step 71500 B loss 0.0003 std 0.042 cos 0.928 lr 2.95e-04 epoch 3/10: 24%|█████▉ | 20258/85466 [22:38<26:53, 40.40it/s] step 72000 B loss 0.0003 std 0.042 cos 0.922 lr 2.95e-04 epoch 3/10: 25%|██████▎ | 21755/85466 [23:16<26:24, 40.20it/s] step 72500 B loss 0.0002 std 0.042 cos 0.938 lr 2.95e-04 epoch 3/10: 27%|██████▊ | 23258/85466 [23:54<28:13, 36.73it/s] step 73000 B loss 0.0004 std 0.042 cos 0.899 lr 2.95e-04 epoch 3/10: 29%|███████▏ | 24758/85466 [24:31<25:02, 40.40it/s] step 73500 B loss 0.0002 std 0.042 cos 0.936 lr 2.95e-04 epoch 3/10: 31%|███████▋ | 26255/85466 [25:09<24:11, 40.79it/s] step 74000 B loss 0.0002 std 0.042 cos 0.950 lr 2.95e-04 epoch 3/10: 32%|████████ | 27758/85466 [25:48<20:31, 46.84it/s] step 74500 B loss 0.0003 std 0.042 cos 0.920 lr 2.94e-04 epoch 3/10: 34%|████████▌ | 29258/85466 [26:26<24:15, 38.63it/s] step 75000 B loss 0.0002 std 0.042 cos 0.953 lr 2.94e-04 epoch 3/10: 36%|████████▉ | 30755/85466 [27:04<24:04, 37.86it/s] step 75500 B loss 0.0006 std 0.043 cos 0.834 lr 2.94e-04 epoch 3/10: 38%|█████████▍ | 32258/85466 [27:43<25:07, 35.29it/s] step 76000 B loss 0.0003 std 0.042 cos 0.924 lr 2.94e-04 epoch 3/10: 39%|█████████▊ | 33755/85466 [28:21<22:40, 38.00it/s] step 76500 B loss 0.0001 std 0.042 cos 0.964 lr 2.94e-04 epoch 3/10: 41%|██████████▎ | 35255/85466 [29:00<18:51, 44.37it/s] step 77000 B loss 0.0003 std 0.041 cos 0.913 lr 2.94e-04 epoch 3/10: 43%|██████████▊ | 36758/85466 [29:37<23:17, 34.85it/s] step 77500 B loss 0.0003 std 0.042 cos 0.913 lr 2.94e-04 epoch 3/10: 45%|███████████▏ | 38258/85466 [30:15<20:41, 38.01it/s] step 78000 B loss 0.0002 std 0.043 cos 0.951 lr 2.94e-04 epoch 3/10: 47%|███████████▋ | 39758/85466 [30:54<17:01, 44.76it/s] step 78500 B loss 0.0004 std 0.043 cos 0.908 lr 2.94e-04 epoch 3/10: 48%|████████████ | 41255/85466 [31:32<17:18, 42.58it/s] step 79000 B loss 0.0002 std 0.042 cos 0.961 lr 2.94e-04 epoch 3/10: 50%|████████████▌ | 42755/85466 [32:10<15:29, 45.93it/s] step 79500 B loss 0.0002 std 0.041 cos 0.938 lr 2.94e-04 epoch 3/10: 52%|████████████▉ | 44259/85466 [32:49<17:25, 39.40it/s] step 80000 B loss 0.0002 std 0.043 cos 0.947 lr 2.94e-04 epoch 3/10: 54%|█████████████▍ | 45756/85466 [33:27<20:12, 32.76it/s] step 80500 B loss 0.0004 std 0.042 cos 0.905 lr 2.94e-04 epoch 3/10: 55%|█████████████▊ | 47255/85466 [34:05<13:41, 46.50it/s] step 81000 B loss 0.0004 std 0.042 cos 0.898 lr 2.93e-04 epoch 3/10: 57%|██████████████▎ | 48755/85466 [34:44<16:05, 38.02it/s] step 81500 B loss 0.0003 std 0.043 cos 0.931 lr 2.93e-04 epoch 3/10: 59%|██████████████▋ | 50255/85466 [35:22<14:15, 41.15it/s] step 82000 B loss 0.0002 std 0.042 cos 0.941 lr 2.93e-04 epoch 3/10: 61%|███████████████▏ | 51755/85466 [36:01<15:13, 36.92it/s] step 82500 B loss 0.0005 std 0.042 cos 0.865 lr 2.93e-04 epoch 3/10: 62%|███████████████▌ | 53255/85466 [36:39<12:32, 42.79it/s] step 83000 B loss 0.0004 std 0.042 cos 0.902 lr 2.93e-04 epoch 3/10: 64%|████████████████ | 54755/85466 [37:18<14:22, 35.61it/s] step 83500 B loss 0.0003 std 0.043 cos 0.921 lr 2.93e-04 epoch 3/10: 66%|████████████████▍ | 56255/85466 [37:56<12:02, 40.45it/s] step 84000 B loss 0.0004 std 0.042 cos 0.907 lr 2.93e-04 epoch 3/10: 68%|████████████████▉ | 57758/85466 [38:34<12:26, 37.11it/s] step 84500 B loss 0.0003 std 0.042 cos 0.916 lr 2.93e-04 epoch 3/10: 69%|█████████████████▎ | 59258/85466 [39:12<11:36, 37.61it/s] step 85000 B loss 0.0004 std 0.041 cos 0.894 lr 2.93e-04 epoch 3/10: 71%|█████████████████▊ | 60755/85466 [39:51<09:10, 44.89it/s] step 85500 B loss 0.0004 std 0.041 cos 0.907 lr 2.93e-04 epoch 3/10: 73%|██████████████████▏ | 62258/85466 [40:29<11:44, 32.94it/s] step 86000 B loss 0.0002 std 0.041 cos 0.942 lr 2.93e-04 epoch 3/10: 75%|██████████████████▋ | 63757/85466 [41:07<08:06, 44.64it/s] step 86500 B loss 0.0003 std 0.043 cos 0.921 lr 2.93e-04 epoch 3/10: 76%|███████████████████ | 65258/85466 [41:45<08:07, 41.41it/s] step 87000 B loss 0.0002 std 0.042 cos 0.954 lr 2.92e-04 epoch 3/10: 78%|███████████████████▌ | 66758/85466 [42:24<07:38, 40.80it/s] step 87500 B loss 0.0004 std 0.042 cos 0.887 lr 2.92e-04 epoch 3/10: 80%|███████████████████▉ | 68255/85466 [43:03<07:16, 39.47it/s] step 88000 B loss 0.0002 std 0.042 cos 0.954 lr 2.92e-04 epoch 3/10: 82%|████████████████████▍ | 69758/85466 [43:41<07:57, 32.87it/s] step 88500 B loss 0.0006 std 0.042 cos 0.845 lr 2.92e-04 epoch 3/10: 83%|████████████████████▊ | 71255/85466 [44:18<07:16, 32.56it/s] step 89000 B loss 0.0002 std 0.043 cos 0.951 lr 2.92e-04 epoch 3/10: 85%|█████████████████████▎ | 72758/85466 [44:57<04:48, 44.07it/s] step 89500 B loss 0.0004 std 0.042 cos 0.908 lr 2.92e-04 epoch 3/10: 87%|█████████████████████▋ | 74258/85466 [45:36<05:17, 35.31it/s] step 90000 B loss 0.0003 std 0.043 cos 0.920 lr 2.92e-04 epoch 3/10: 89%|██████████████████████▏ | 75755/85466 [46:15<04:43, 34.20it/s] step 90500 B loss 0.0004 std 0.043 cos 0.892 lr 2.92e-04 epoch 3/10: 90%|██████████████████████▌ | 77258/85466 [46:53<02:58, 45.87it/s] step 91000 B loss 0.0003 std 0.041 cos 0.932 lr 2.92e-04 epoch 3/10: 92%|███████████████████████ | 78755/85466 [47:31<03:30, 31.93it/s] step 91500 B loss 0.0002 std 0.042 cos 0.961 lr 2.92e-04 epoch 3/10: 94%|███████████████████████▍ | 80259/85466 [48:08<02:25, 35.87it/s] step 92000 B loss 0.0002 std 0.043 cos 0.942 lr 2.92e-04 epoch 3/10: 96%|███████████████████████▉ | 81758/85466 [48:47<01:47, 34.56it/s] step 92500 B loss 0.0002 std 0.043 cos 0.957 lr 2.91e-04 epoch 3/10: 97%|████████████████████████▎| 83255/85466 [49:25<00:59, 37.47it/s] step 93000 B loss 0.0003 std 0.043 cos 0.912 lr 2.91e-04 epoch 3/10: 99%|████████████████████████▊| 84757/85466 [50:03<00:20, 35.06it/s] step 93500 B loss 0.0003 std 0.043 cos 0.921 lr 2.91e-04 epoch 3/10: 100%|█████████████████████████| 85466/85466 [50:21<00:00, 28.28it/s] epoch 3/10 avg_loss 0.0001 3023s epoch 4/10: 0%| | 264/85466 [01:02<4:53:47, 4.83it/s] step 94000 A loss 0.0001 std 0.029 cos 0.965 lr 2.91e-04 epoch 4/10: 1%|▏ | 763/85466 [02:55<5:56:03, 3.96it/s] step 94500 C loss 0.0001 std 0.029 cos 0.964 lr 2.91e-04 epoch 4/10: 1%|▎ | 1264/85466 [04:49<6:07:39, 3.82it/s] step 95000 B loss 0.0004 std 0.041 cos 0.899 lr 2.91e-04 epoch 4/10: 2%|▍ | 1764/85466 [06:43<4:35:07, 5.07it/s] step 95500 A loss 0.0003 std 0.031 cos 0.928 lr 2.91e-04 epoch 4/10: 3%|▋ | 2263/85466 [08:37<6:09:28, 3.75it/s] step 96000 C loss 0.0003 std 0.032 cos 0.923 lr 2.91e-04 epoch 4/10: 3%|▊ | 2764/85466 [10:31<5:52:07, 3.91it/s] step 96500 B loss 0.0003 std 0.042 cos 0.927 lr 2.91e-04 epoch 4/10: 4%|▉ | 3264/85466 [12:24<4:40:17, 4.89it/s] step 97000 A loss 0.0005 std 0.031 cos 0.882 lr 2.91e-04 epoch 4/10: 4%|█ | 3763/85466 [14:18<5:59:23, 3.79it/s] step 97500 C loss 0.0004 std 0.030 cos 0.904 lr 2.91e-04 epoch 4/10: 5%|█▍ | 4520/85466 [15:53<36:48, 36.65it/s] step 98000 B loss 0.0004 std 0.042 cos 0.896 lr 2.90e-04 epoch 4/10: 7%|█▊ | 6023/85466 [16:31<37:41, 35.13it/s] step 98500 B loss 0.0005 std 0.042 cos 0.872 lr 2.90e-04 epoch 4/10: 9%|██▎ | 7520/85466 [17:09<35:17, 36.82it/s] step 99000 B loss 0.0002 std 0.042 cos 0.940 lr 2.90e-04 epoch 4/10: 11%|██▋ | 9023/85466 [17:48<34:27, 36.98it/s] step 99500 B loss 0.0004 std 0.042 cos 0.906 lr 2.90e-04 epoch 4/10: 12%|███ | 10523/85466 [18:26<27:55, 44.73it/s] step 100000 B loss 0.0002 std 0.042 cos 0.960 lr 2.90e-04 epoch 4/10: 14%|███▌ | 12022/85466 [19:04<26:25, 46.31it/s] step 100500 B loss 0.0004 std 0.043 cos 0.903 lr 2.90e-04 epoch 4/10: 16%|███▉ | 13520/85466 [19:42<28:47, 41.64it/s] step 101000 B loss 0.0002 std 0.042 cos 0.953 lr 2.90e-04 epoch 4/10: 18%|████▍ | 15020/85466 [20:21<26:41, 43.99it/s] step 101500 B loss 0.0003 std 0.042 cos 0.921 lr 2.90e-04 epoch 4/10: 19%|████▊ | 16523/85466 [20:59<27:12, 42.24it/s] step 102000 B loss 0.0002 std 0.043 cos 0.948 lr 2.90e-04 epoch 4/10: 21%|█████▎ | 18023/85466 [21:38<26:44, 42.04it/s] step 102500 B loss 0.0001 std 0.043 cos 0.967 lr 2.90e-04 epoch 4/10: 23%|█████▋ | 19520/85466 [22:16<24:53, 44.16it/s] step 103000 B loss 0.0003 std 0.043 cos 0.921 lr 2.89e-04 epoch 4/10: 25%|██████▏ | 21020/85466 [22:54<31:24, 34.19it/s] step 103500 B loss 0.0004 std 0.043 cos 0.897 lr 2.89e-04 epoch 4/10: 26%|██████▌ | 22520/85466 [23:32<30:54, 33.94it/s] step 104000 B loss 0.0004 std 0.043 cos 0.909 lr 2.89e-04 epoch 4/10: 28%|███████ | 24024/85466 [24:11<32:13, 31.78it/s] step 104500 B loss 0.0002 std 0.042 cos 0.946 lr 2.89e-04 epoch 4/10: 30%|███████▍ | 25520/85466 [24:50<22:57, 43.51it/s] step 105000 B loss 0.0003 std 0.043 cos 0.911 lr 2.89e-04 epoch 4/10: 32%|███████▉ | 27023/85466 [25:28<23:30, 41.42it/s] step 105500 B loss 0.0002 std 0.043 cos 0.940 lr 2.89e-04 epoch 4/10: 33%|████████▎ | 28523/85466 [26:07<26:01, 36.46it/s] step 106000 B loss 0.0003 std 0.043 cos 0.917 lr 2.89e-04 epoch 4/10: 35%|████████▊ | 30020/85466 [26:45<24:42, 37.40it/s] step 106500 B loss 0.0004 std 0.041 cos 0.887 lr 2.89e-04 epoch 4/10: 37%|█████████▏ | 31523/85466 [27:23<23:31, 38.21it/s] step 107000 B loss 0.0003 std 0.042 cos 0.915 lr 2.89e-04 epoch 4/10: 39%|█████████▋ | 33023/85466 [28:01<22:34, 38.72it/s] step 107500 B loss 0.0003 std 0.043 cos 0.931 lr 2.89e-04 epoch 4/10: 40%|██████████ | 34518/85466 [28:39<21:00, 40.41it/s] step 108000 B loss 0.0007 std 0.041 cos 0.812 lr 2.88e-04 epoch 4/10: 42%|██████████▌ | 36023/85466 [29:19<20:02, 41.11it/s] step 108500 B loss 0.0001 std 0.043 cos 0.967 lr 2.88e-04 epoch 4/10: 44%|██████████▉ | 37522/85466 [29:57<20:00, 39.93it/s] step 109000 B loss 0.0001 std 0.042 cos 0.969 lr 2.88e-04 epoch 4/10: 46%|███████████▍ | 39020/85466 [30:36<14:34, 53.11it/s] step 109500 B loss 0.0004 std 0.043 cos 0.894 lr 2.88e-04 epoch 4/10: 47%|███████████▊ | 40525/85466 [31:14<20:07, 37.22it/s] step 110000 B loss 0.0003 std 0.043 cos 0.934 lr 2.88e-04 epoch 4/10: 49%|████████████▎ | 42023/85466 [31:53<15:47, 45.84it/s] step 110500 B loss 0.0005 std 0.041 cos 0.882 lr 2.88e-04 epoch 4/10: 51%|████████████▋ | 43523/85466 [32:32<18:39, 37.46it/s] step 111000 B loss 0.0003 std 0.042 cos 0.912 lr 2.88e-04 epoch 4/10: 53%|█████████████▏ | 45023/85466 [33:11<17:20, 38.89it/s] step 111500 B loss 0.0003 std 0.043 cos 0.929 lr 2.88e-04 epoch 4/10: 54%|█████████████▌ | 46523/85466 [33:49<16:16, 39.89it/s] step 112000 B loss 0.0003 std 0.043 cos 0.934 lr 2.88e-04 epoch 4/10: 56%|██████████████ | 48024/85466 [34:28<18:02, 34.58it/s] step 112500 B loss 0.0003 std 0.043 cos 0.917 lr 2.87e-04 epoch 4/10: 58%|██████████████▍ | 49523/85466 [35:08<16:02, 37.35it/s] step 113000 B loss 0.0003 std 0.042 cos 0.928 lr 2.87e-04 epoch 4/10: 60%|██████████████▉ | 51020/85466 [35:46<12:21, 46.47it/s] step 113500 B loss 0.0005 std 0.042 cos 0.880 lr 2.87e-04 epoch 4/10: 61%|███████████████▎ | 52523/85466 [36:26<10:44, 51.11it/s] step 114000 B loss 0.0006 std 0.042 cos 0.857 lr 2.87e-04 epoch 4/10: 63%|███████████████▊ | 54025/85466 [37:05<13:01, 40.21it/s] step 114500 B loss 0.0003 std 0.042 cos 0.916 lr 2.87e-04 epoch 4/10: 65%|████████████████▏ | 55520/85466 [37:43<10:20, 48.23it/s] step 115000 B loss 0.0002 std 0.042 cos 0.939 lr 2.87e-04 epoch 4/10: 67%|████████████████▋ | 57023/85466 [38:22<12:01, 39.40it/s] step 115500 B loss 0.0003 std 0.042 cos 0.911 lr 2.87e-04 epoch 4/10: 68%|█████████████████ | 58520/85466 [39:01<10:42, 41.92it/s] step 116000 B loss 0.0002 std 0.042 cos 0.952 lr 2.87e-04 epoch 4/10: 70%|█████████████████▌ | 60023/85466 [39:39<09:42, 43.71it/s] step 116500 B loss 0.0004 std 0.042 cos 0.896 lr 2.87e-04 epoch 4/10: 72%|█████████████████▉ | 61522/85466 [40:17<09:29, 42.06it/s] step 117000 B loss 0.0002 std 0.042 cos 0.938 lr 2.86e-04 epoch 4/10: 74%|██████████████████▍ | 63020/85466 [40:55<09:24, 39.75it/s] step 117500 B loss 0.0004 std 0.043 cos 0.902 lr 2.86e-04 epoch 4/10: 75%|██████████████████▊ | 64520/85466 [41:34<10:03, 34.68it/s] step 118000 B loss 0.0002 std 0.043 cos 0.943 lr 2.86e-04 epoch 4/10: 77%|███████████████████▎ | 66020/85466 [42:12<08:36, 37.67it/s] step 118500 B loss 0.0002 std 0.043 cos 0.949 lr 2.86e-04 epoch 4/10: 79%|███████████████████▊ | 67523/85466 [42:51<08:07, 36.84it/s] step 119000 B loss 0.0007 std 0.042 cos 0.832 lr 2.86e-04 epoch 4/10: 81%|████████████████████▏ | 69020/85466 [43:29<05:53, 46.52it/s] step 119500 B loss 0.0006 std 0.043 cos 0.848 lr 2.86e-04 epoch 4/10: 83%|████████████████████▋ | 70523/85466 [44:08<06:56, 35.85it/s] step 120000 B loss 0.0002 std 0.043 cos 0.940 lr 2.86e-04 epoch 4/10: 84%|█████████████████████ | 72020/85466 [44:46<05:32, 40.38it/s] step 120500 B loss 0.0003 std 0.041 cos 0.914 lr 2.86e-04 epoch 4/10: 86%|█████████████████████▌ | 73520/85466 [45:24<05:10, 38.48it/s] step 121000 B loss 0.0004 std 0.042 cos 0.887 lr 2.86e-04 epoch 4/10: 88%|█████████████████████▉ | 75023/85466 [46:04<05:44, 30.29it/s] step 121500 B loss 0.0005 std 0.042 cos 0.881 lr 2.85e-04 epoch 4/10: 90%|██████████████████████▍ | 76523/85466 [46:42<04:02, 36.88it/s] step 122000 B loss 0.0001 std 0.043 cos 0.963 lr 2.85e-04 epoch 4/10: 91%|██████████████████████▊ | 78023/85466 [47:21<03:40, 33.71it/s] step 122500 B loss 0.0003 std 0.043 cos 0.911 lr 2.85e-04 epoch 4/10: 93%|███████████████████████▎ | 79523/85466 [47:59<02:24, 41.10it/s] step 123000 B loss 0.0001 std 0.043 cos 0.962 lr 2.85e-04 epoch 4/10: 95%|███████████████████████▋ | 81020/85466 [48:38<01:49, 40.53it/s] step 123500 B loss 0.0003 std 0.041 cos 0.931 lr 2.85e-04 epoch 4/10: 97%|████████████████████████▏| 82521/85466 [49:16<01:30, 32.47it/s] step 124000 B loss 0.0002 std 0.043 cos 0.953 lr 2.85e-04 epoch 4/10: 98%|████████████████████████▌| 84020/85466 [49:55<00:39, 36.79it/s] step 124500 B loss 0.0001 std 0.042 cos 0.963 lr 2.85e-04 epoch 4/10: 100%|█████████████████████████| 85466/85466 [50:33<00:00, 28.18it/s] epoch 4/10 avg_loss 0.0001 3034s epoch 5/10: 0%| | 19/85466 [00:06<6:11:14, 3.84it/s] step 125000 B loss 0.0005 std 0.042 cos 0.882 lr 2.85e-04 epoch 5/10: 1%|▏ | 519/85466 [02:00<4:54:38, 4.81it/s] step 125500 A loss 0.0002 std 0.025 cos 0.949 lr 2.84e-04 epoch 5/10: 1%|▎ | 1019/85466 [03:54<5:21:04, 4.38it/s] step 126000 C loss 0.0002 std 0.028 cos 0.945 lr 2.84e-04 epoch 5/10: 2%|▍ | 1519/85466 [05:50<5:52:21, 3.97it/s] step 126500 B loss 0.0003 std 0.042 cos 0.934 lr 2.84e-04 epoch 5/10: 2%|▌ | 2019/85466 [07:44<4:34:48, 5.06it/s] step 127000 A loss 0.0003 std 0.034 cos 0.932 lr 2.84e-04 epoch 5/10: 3%|▋ | 2518/85466 [09:38<6:09:09, 3.74it/s] step 127500 C loss 0.0003 std 0.034 cos 0.923 lr 2.84e-04 epoch 5/10: 4%|▊ | 3019/85466 [11:33<5:57:52, 3.84it/s] step 128000 B loss 0.0002 std 0.042 cos 0.945 lr 2.84e-04 epoch 5/10: 4%|▉ | 3519/85466 [13:28<4:41:28, 4.85it/s] step 128500 A loss 0.0005 std 0.033 cos 0.880 lr 2.84e-04 epoch 5/10: 5%|█▏ | 4018/85466 [15:22<5:41:59, 3.97it/s] step 129000 C loss 0.0003 std 0.034 cos 0.930 lr 2.84e-04 epoch 5/10: 6%|█▌ | 5288/85466 [16:19<40:02, 33.38it/s] step 129500 B loss 0.0003 std 0.042 cos 0.928 lr 2.83e-04 epoch 5/10: 8%|██ | 6785/85466 [16:58<33:17, 39.39it/s] step 130000 B loss 0.0003 std 0.041 cos 0.921 lr 2.83e-04 epoch 5/10: 10%|██▌ | 8288/85466 [17:37<31:24, 40.96it/s] step 130500 B loss 0.0003 std 0.043 cos 0.919 lr 2.83e-04 epoch 5/10: 11%|██▉ | 9785/85466 [18:16<35:05, 35.95it/s] step 131000 B loss 0.0002 std 0.042 cos 0.939 lr 2.83e-04 epoch 5/10: 13%|███▎ | 11286/85466 [18:54<35:05, 35.23it/s] step 131500 B loss 0.0004 std 0.043 cos 0.895 lr 2.83e-04 epoch 5/10: 15%|███▋ | 12787/85466 [19:33<30:42, 39.45it/s] step 132000 B loss 0.0004 std 0.043 cos 0.898 lr 2.83e-04 epoch 5/10: 17%|████▏ | 14285/85466 [20:12<30:31, 38.87it/s] step 132500 B loss 0.0003 std 0.043 cos 0.931 lr 2.83e-04 epoch 5/10: 18%|████▌ | 15785/85466 [20:51<32:32, 35.69it/s] step 133000 B loss 0.0004 std 0.041 cos 0.893 lr 2.83e-04 epoch 5/10: 20%|█████ | 17285/85466 [21:29<27:37, 41.15it/s] step 133500 B loss 0.0003 std 0.043 cos 0.929 lr 2.82e-04 epoch 5/10: 22%|█████▍ | 18785/85466 [22:08<28:16, 39.31it/s] step 134000 B loss 0.0002 std 0.043 cos 0.960 lr 2.82e-04 epoch 5/10: 24%|█████▉ | 20288/85466 [22:46<30:17, 35.85it/s] step 134500 B loss 0.0003 std 0.042 cos 0.911 lr 2.82e-04 epoch 5/10: 25%|██████▎ | 21788/85466 [23:25<29:00, 36.59it/s] step 135000 B loss 0.0003 std 0.042 cos 0.933 lr 2.82e-04 epoch 5/10: 27%|██████▊ | 23285/85466 [24:03<31:17, 33.13it/s] step 135500 B loss 0.0003 std 0.043 cos 0.935 lr 2.82e-04 epoch 5/10: 29%|███████▏ | 24785/85466 [24:42<25:30, 39.65it/s] step 136000 B loss 0.0003 std 0.040 cos 0.918 lr 2.82e-04 epoch 5/10: 31%|███████▋ | 26288/85466 [25:21<24:47, 39.77it/s] step 136500 B loss 0.0003 std 0.043 cos 0.913 lr 2.82e-04 epoch 5/10: 33%|████████▏ | 27785/85466 [26:00<28:09, 34.13it/s] step 137000 B loss 0.0003 std 0.043 cos 0.917 lr 2.81e-04 epoch 5/10: 34%|████████▌ | 29288/85466 [26:39<28:26, 32.92it/s] step 137500 B loss 0.0003 std 0.043 cos 0.913 lr 2.81e-04 epoch 5/10: 36%|█████████ | 30785/85466 [27:18<21:29, 42.40it/s] step 138000 B loss 0.0002 std 0.043 cos 0.947 lr 2.81e-04 epoch 5/10: 38%|█████████▍ | 32288/85466 [27:57<24:40, 35.92it/s] step 138500 B loss 0.0005 std 0.043 cos 0.872 lr 2.81e-04 epoch 5/10: 40%|█████████▉ | 33786/85466 [28:35<25:15, 34.10it/s] step 139000 B loss 0.0002 std 0.042 cos 0.951 lr 2.81e-04 epoch 5/10: 41%|██████████▎ | 35288/85466 [29:15<19:49, 42.18it/s] step 139500 B loss 0.0002 std 0.042 cos 0.947 lr 2.81e-04 epoch 5/10: 43%|██████████▊ | 36785/85466 [29:54<23:16, 34.87it/s] step 140000 B loss 0.0002 std 0.043 cos 0.954 lr 2.81e-04 epoch 5/10: 45%|███████████▏ | 38285/85466 [30:33<20:03, 39.21it/s] step 140500 B loss 0.0005 std 0.043 cos 0.874 lr 2.81e-04 epoch 5/10: 47%|███████████▋ | 39788/85466 [31:12<17:21, 43.86it/s] step 141000 B loss 0.0002 std 0.043 cos 0.939 lr 2.80e-04 epoch 5/10: 48%|████████████ | 41285/85466 [31:51<18:51, 39.04it/s] step 141500 B loss 0.0003 std 0.042 cos 0.922 lr 2.80e-04 epoch 5/10: 50%|████████████▌ | 42788/85466 [32:29<19:08, 37.16it/s] step 142000 B loss 0.0005 std 0.041 cos 0.863 lr 2.80e-04 epoch 5/10: 52%|████████████▉ | 44288/85466 [33:08<19:34, 35.06it/s] step 142500 B loss 0.0002 std 0.042 cos 0.938 lr 2.80e-04 epoch 5/10: 52%|████████████▉ | 44288/85466 [33:08<19:34, 35.06it/s] step 142500 B loss 0.0002 std 0.042 cos 0.938 lr 2.80e-04 epoch 5/10: 54%|█████████████▍ | 45788/85466 [33:48<17:59, 36.77it/s] step 143000 B loss 0.0004 std 0.043 cos 0.900 lr 2.80e-04 epoch 5/10: 55%|█████████████▊ | 47288/85466 [34:28<16:24, 38.79it/s] step 143500 B loss 0.0003 std 0.042 cos 0.912 lr 2.80e-04 epoch 5/10: 57%|██████████████▎ | 48788/85466 [35:07<15:15, 40.06it/s] step 144000 B loss 0.0001 std 0.043 cos 0.969 lr 2.80e-04 epoch 5/10: 59%|██████████████▋ | 50289/85466 [35:46<17:01, 34.42it/s] step 144500 B loss 0.0004 std 0.043 cos 0.910 lr 2.79e-04 epoch 5/10: 61%|███████████████▏ | 51785/85466 [36:25<13:53, 40.43it/s] step 145000 B loss 0.0003 std 0.043 cos 0.931 lr 2.79e-04 epoch 5/10: 62%|███████████████▌ | 53288/85466 [37:04<13:58, 38.40it/s] step 145500 B loss 0.0003 std 0.041 cos 0.926 lr 2.79e-04 epoch 5/10: 64%|████████████████ | 54788/85466 [37:42<12:23, 41.25it/s] step 146000 B loss 0.0002 std 0.043 cos 0.939 lr 2.79e-04 epoch 5/10: 66%|████████████████▍ | 56285/85466 [38:21<11:56, 40.70it/s] step 146500 B loss 0.0004 std 0.043 cos 0.900 lr 2.79e-04 epoch 5/10: 68%|████████████████▉ | 57785/85466 [39:00<12:51, 35.87it/s] step 147000 B loss 0.0003 std 0.043 cos 0.917 lr 2.79e-04 epoch 5/10: 69%|█████████████████▎ | 59288/85466 [39:39<12:51, 33.92it/s] step 147500 B loss 0.0002 std 0.043 cos 0.938 lr 2.79e-04 epoch 5/10: 71%|█████████████████▊ | 60785/85466 [40:17<10:44, 38.28it/s] step 148000 B loss 0.0003 std 0.042 cos 0.928 lr 2.78e-04 epoch 5/10: 73%|██████████████████▏ | 62285/85466 [40:56<10:26, 36.97it/s] step 148500 B loss 0.0002 std 0.041 cos 0.945 lr 2.78e-04 epoch 5/10: 75%|██████████████████▋ | 63788/85466 [41:34<10:14, 35.27it/s] step 149000 B loss 0.0001 std 0.043 cos 0.969 lr 2.78e-04 epoch 5/10: 76%|███████████████████ | 65285/85466 [42:13<08:35, 39.12it/s] step 149500 B loss 0.0004 std 0.042 cos 0.899 lr 2.78e-04 epoch 5/10: 78%|███████████████████▌ | 66788/85466 [42:52<08:52, 35.07it/s] step 150000 B loss 0.0005 std 0.042 cos 0.875 lr 2.78e-04 epoch 5/10: 80%|███████████████████▉ | 68285/85466 [43:30<07:31, 38.09it/s] step 150500 B loss 0.0002 std 0.043 cos 0.954 lr 2.78e-04 epoch 5/10: 82%|████████████████████▍ | 69785/85466 [44:09<07:19, 35.69it/s] step 151000 B loss 0.0004 std 0.043 cos 0.909 lr 2.78e-04 epoch 5/10: 83%|████████████████████▊ | 71285/85466 [44:48<06:07, 38.63it/s] step 151500 B loss 0.0003 std 0.043 cos 0.932 lr 2.77e-04 epoch 5/10: 85%|█████████████████████▎ | 72788/85466 [45:28<04:37, 45.73it/s] step 152000 B loss 0.0002 std 0.043 cos 0.954 lr 2.77e-04 epoch 5/10: 87%|█████████████████████▋ | 74288/85466 [46:06<04:05, 45.48it/s] step 152500 B loss 0.0003 std 0.043 cos 0.923 lr 2.77e-04 epoch 5/10: 89%|██████████████████████▏ | 75785/85466 [46:45<03:56, 40.92it/s] step 153000 B loss 0.0002 std 0.042 cos 0.954 lr 2.77e-04 epoch 5/10: 90%|██████████████████████▌ | 77285/85466 [47:24<03:32, 38.54it/s] step 153500 B loss 0.0003 std 0.042 cos 0.922 lr 2.77e-04 epoch 5/10: 92%|███████████████████████ | 78788/85466 [48:03<02:33, 43.57it/s] step 154000 B loss 0.0003 std 0.042 cos 0.925 lr 2.77e-04 epoch 5/10: 94%|███████████████████████▍ | 80288/85466 [48:42<02:20, 36.87it/s] step 154500 B loss 0.0003 std 0.043 cos 0.929 lr 2.77e-04 epoch 5/10: 96%|███████████████████████▉ | 81788/85466 [49:21<01:25, 43.15it/s] step 155000 B loss 0.0002 std 0.042 cos 0.944 lr 2.76e-04 epoch 5/10: 97%|████████████████████████▎| 83290/85466 [50:00<01:06, 32.84it/s] step 155500 B loss 0.0002 std 0.042 cos 0.962 lr 2.76e-04 epoch 5/10: 99%|████████████████████████▊| 84788/85466 [50:38<00:16, 41.29it/s] step 156000 B loss 0.0001 std 0.043 cos 0.962 lr 2.76e-04 epoch 5/10: 100%|█████████████████████████| 85466/85466 [50:56<00:00, 27.96it/s] epoch 5/10 avg_loss 0.0001 3057s epoch 6/10: 0%| | 274/85466 [01:05<6:08:58, 3.85it/s] step 156500 B loss 0.0003 std 0.042 cos 0.932 lr 2.76e-04 epoch 6/10: 1%|▏ | 774/85466 [03:00<4:51:22, 4.84it/s] step 157000 A loss 0.0004 std 0.031 cos 0.888 lr 2.76e-04 epoch 6/10: 1%|▎ | 1273/85466 [04:55<5:51:08, 4.00it/s] step 157500 C loss 0.0005 std 0.032 cos 0.873 lr 2.76e-04 epoch 6/10: 2%|▍ | 1774/85466 [06:51<6:15:29, 3.71it/s] step 158000 B loss 0.0003 std 0.043 cos 0.932 lr 2.76e-04 epoch 6/10: 3%|▋ | 2274/85466 [08:46<4:50:17, 4.78it/s] step 158500 A loss 0.0005 std 0.032 cos 0.876 lr 2.75e-04 epoch 6/10: 3%|▊ | 2773/85466 [10:42<5:56:38, 3.86it/s] step 159000 C loss 0.0002 std 0.033 cos 0.937 lr 2.75e-04 epoch 6/10: 4%|▉ | 3274/85466 [12:36<6:27:50, 3.53it/s] step 159500 B loss 0.0003 std 0.042 cos 0.925 lr 2.75e-04 epoch 6/10: 4%|█ | 3774/85466 [14:32<4:48:22, 4.72it/s] step 160000 A loss 0.0005 std 0.033 cos 0.877 lr 2.75e-04 epoch 6/10: 5%|█▍ | 4550/85466 [16:05<31:36, 42.67it/s] step 160500 B loss 0.0003 std 0.043 cos 0.933 lr 2.75e-04 epoch 6/10: 7%|█▊ | 6050/85466 [16:44<31:31, 42.00it/s] step 161000 B loss 0.0002 std 0.043 cos 0.941 lr 2.75e-04 epoch 6/10: 9%|██▎ | 7554/85466 [17:25<40:21, 32.18it/s] step 161500 B loss 0.0002 std 0.043 cos 0.943 lr 2.74e-04 epoch 6/10: 11%|██▊ | 9050/85466 [18:04<29:39, 42.95it/s] step 162000 B loss 0.0005 std 0.042 cos 0.862 lr 2.74e-04 epoch 6/10: 12%|███ | 10550/85466 [18:42<29:33, 42.24it/s] step 162500 B loss 0.0004 std 0.042 cos 0.907 lr 2.74e-04 epoch 6/10: 14%|███▌ | 12050/85466 [19:22<33:28, 36.55it/s] step 163000 B loss 0.0005 std 0.042 cos 0.880 lr 2.74e-04 epoch 6/10: 16%|███▉ | 13550/85466 [20:02<31:08, 38.49it/s] step 163500 B loss 0.0001 std 0.043 cos 0.963 lr 2.74e-04 epoch 6/10: 18%|████▍ | 15053/85466 [20:41<26:38, 44.06it/s] step 164000 B loss 0.0005 std 0.043 cos 0.880 lr 2.74e-04 epoch 6/10: 19%|████▊ | 16554/85466 [21:20<41:29, 27.68it/s] step 164500 B loss 0.0002 std 0.043 cos 0.947 lr 2.74e-04 epoch 6/10: 21%|█████▎ | 18053/85466 [22:00<32:41, 34.36it/s] step 165000 B loss 0.0002 std 0.039 cos 0.939 lr 2.73e-04 epoch 6/10: 23%|█████▋ | 19553/85466 [22:39<32:51, 33.43it/s] step 165500 B loss 0.0003 std 0.043 cos 0.933 lr 2.73e-04 epoch 6/10: 25%|██████▏ | 21053/85466 [23:18<25:20, 42.37it/s] step 166000 B loss 0.0003 std 0.043 cos 0.912 lr 2.73e-04 epoch 6/10: 26%|██████▌ | 22555/85466 [23:57<27:30, 38.11it/s] step 166500 B loss 0.0003 std 0.043 cos 0.932 lr 2.73e-04 epoch 6/10: 28%|███████ | 24050/85466 [24:36<29:35, 34.59it/s] step 167000 B loss 0.0003 std 0.043 cos 0.931 lr 2.73e-04 epoch 6/10: 30%|███████▍ | 25554/85466 [25:15<24:32, 40.68it/s] step 167500 B loss 0.0003 std 0.043 cos 0.931 lr 2.73e-04 epoch 6/10: 32%|███████▉ | 27053/85466 [25:54<26:06, 37.28it/s] step 168000 B loss 0.0003 std 0.042 cos 0.922 lr 2.72e-04 epoch 6/10: 33%|████████▎ | 28552/85466 [26:33<24:54, 38.08it/s] step 168500 B loss 0.0003 std 0.043 cos 0.926 lr 2.72e-04 epoch 6/10: 35%|████████▊ | 30050/85466 [27:13<24:02, 38.41it/s] step 169000 B loss 0.0003 std 0.043 cos 0.922 lr 2.72e-04 epoch 6/10: 37%|█████████▏ | 31550/85466 [27:50<20:55, 42.93it/s] step 169500 B loss 0.0003 std 0.043 cos 0.925 lr 2.72e-04 epoch 6/10: 39%|█████████▋ | 33050/85466 [28:30<26:01, 33.56it/s] step 170000 B loss 0.0004 std 0.043 cos 0.897 lr 2.72e-04 epoch 6/10: 40%|██████████ | 34550/85466 [29:09<26:50, 31.61it/s] step 170500 B loss 0.0002 std 0.042 cos 0.954 lr 2.72e-04 epoch 6/10: 42%|██████████▌ | 36053/85466 [29:50<23:00, 35.79it/s] step 171000 B loss 0.0001 std 0.043 cos 0.973 lr 2.71e-04 epoch 6/10: 44%|██████████▉ | 37553/85466 [30:28<22:07, 36.10it/s] step 171500 B loss 0.0003 std 0.041 cos 0.934 lr 2.71e-04 epoch 6/10: 46%|███████████▍ | 39053/85466 [31:06<18:47, 41.15it/s] step 172000 B loss 0.0004 std 0.043 cos 0.908 lr 2.71e-04 epoch 6/10: 47%|███████████▊ | 40550/85466 [31:45<21:33, 34.74it/s] step 172500 B loss 0.0003 std 0.041 cos 0.916 lr 2.71e-04 epoch 6/10: 49%|████████████▎ | 42050/85466 [32:24<18:04, 40.03it/s] step 173000 B loss 0.0003 std 0.042 cos 0.928 lr 2.71e-04 epoch 6/10: 51%|████████████▋ | 43553/85466 [33:03<21:30, 32.48it/s] step 173500 B loss 0.0004 std 0.043 cos 0.896 lr 2.71e-04 epoch 6/10: 53%|█████████████▏ | 45050/85466 [33:42<17:32, 38.39it/s] step 174000 B loss 0.0003 std 0.043 cos 0.934 lr 2.70e-04 epoch 6/10: 54%|█████████████▌ | 46550/85466 [34:20<17:25, 37.22it/s] step 174500 B loss 0.0002 std 0.041 cos 0.940 lr 2.70e-04 epoch 6/10: 56%|██████████████ | 48053/85466 [34:59<17:29, 35.65it/s] step 175000 B loss 0.0003 std 0.042 cos 0.930 lr 2.70e-04 epoch 6/10: 58%|██████████████▍ | 49553/85466 [35:39<16:42, 35.83it/s] step 175500 B loss 0.0003 std 0.040 cos 0.920 lr 2.70e-04 epoch 6/10: 60%|██████████████▉ | 51050/85466 [36:17<13:53, 41.29it/s] step 176000 B loss 0.0004 std 0.043 cos 0.885 lr 2.70e-04 epoch 6/10: 61%|███████████████▎ | 52550/85466 [36:56<14:10, 38.68it/s] step 176500 B loss 0.0002 std 0.042 cos 0.950 lr 2.70e-04 epoch 6/10: 63%|███████████████▊ | 54053/85466 [37:34<14:58, 34.97it/s] step 177000 B loss 0.0004 std 0.041 cos 0.903 lr 2.69e-04 epoch 6/10: 65%|████████████████▏ | 55551/85466 [38:13<16:09, 30.84it/s] step 177500 B loss 0.0003 std 0.043 cos 0.931 lr 2.69e-04 epoch 6/10: 67%|████████████████▋ | 57053/85466 [38:52<12:37, 37.52it/s] step 178000 B loss 0.0003 std 0.042 cos 0.926 lr 2.69e-04 epoch 6/10: 69%|█████████████████▏ | 58553/85466 [39:31<10:05, 44.45it/s] step 178500 B loss 0.0002 std 0.043 cos 0.948 lr 2.69e-04 epoch 6/10: 70%|█████████████████▌ | 60050/85466 [40:10<10:22, 40.82it/s] step 179000 B loss 0.0002 std 0.042 cos 0.952 lr 2.69e-04 epoch 6/10: 72%|██████████████████ | 61553/85466 [40:48<10:58, 36.33it/s] step 179500 B loss 0.0003 std 0.042 cos 0.911 lr 2.69e-04 epoch 6/10: 74%|██████████████████▍ | 63050/85466 [41:27<08:34, 43.54it/s] step 180000 B loss 0.0002 std 0.043 cos 0.958 lr 2.68e-04 epoch 6/10: 76%|██████████████████▉ | 64551/85466 [42:05<08:29, 41.09it/s] step 180500 B loss 0.0004 std 0.043 cos 0.891 lr 2.68e-04 epoch 6/10: 77%|███████████████████▎ | 66053/85466 [42:44<08:48, 36.74it/s] step 181000 B loss 0.0005 std 0.043 cos 0.867 lr 2.68e-04 epoch 6/10: 79%|███████████████████▊ | 67550/85466 [43:23<06:34, 45.39it/s] step 181500 B loss 0.0004 std 0.043 cos 0.896 lr 2.68e-04 epoch 6/10: 81%|████████████████████▏ | 69053/85466 [44:02<06:34, 41.61it/s] step 182000 B loss 0.0003 std 0.042 cos 0.916 lr 2.68e-04 epoch 6/10: 83%|████████████████████▋ | 70550/85466 [44:40<05:27, 45.49it/s] step 182500 B loss 0.0002 std 0.043 cos 0.961 lr 2.68e-04 epoch 6/10: 84%|█████████████████████ | 72050/85466 [45:19<05:50, 38.32it/s] step 183000 B loss 0.0002 std 0.042 cos 0.941 lr 2.67e-04 epoch 6/10: 86%|█████████████████████▌ | 73550/85466 [45:58<05:01, 39.47it/s] step 183500 B loss 0.0004 std 0.042 cos 0.888 lr 2.67e-04 epoch 6/10: 88%|█████████████████████▉ | 75053/85466 [46:36<04:04, 42.62it/s] step 184000 B loss 0.0002 std 0.042 cos 0.946 lr 2.67e-04 epoch 6/10: 90%|██████████████████████▍ | 76550/85466 [47:15<04:33, 32.55it/s] step 184500 B loss 0.0002 std 0.042 cos 0.955 lr 2.67e-04 epoch 6/10: 91%|██████████████████████▊ | 78053/85466 [47:54<02:53, 42.75it/s] step 185000 B loss 0.0005 std 0.043 cos 0.874 lr 2.67e-04 epoch 6/10: 93%|███████████████████████▎ | 79553/85466 [48:33<02:39, 37.08it/s] step 185500 B loss 0.0003 std 0.043 cos 0.914 lr 2.67e-04 epoch 6/10: 95%|███████████████████████▋ | 81053/85466 [49:11<01:50, 39.88it/s] step 186000 B loss 0.0002 std 0.043 cos 0.937 lr 2.66e-04 epoch 6/10: 97%|████████████████████████▏| 82550/85466 [49:50<01:09, 42.25it/s] step 186500 B loss 0.0004 std 0.042 cos 0.903 lr 2.66e-04 epoch 6/10: 98%|████████████████████████▌| 84050/85466 [50:28<00:32, 43.55it/s] step 187000 B loss 0.0001 std 0.043 cos 0.968 lr 2.66e-04 epoch 6/10: 100%|█████████████████████████| 85466/85466 [51:03<00:00, 27.90it/s] epoch 6/10 avg_loss 0.0001 3065s epoch 7/10: 0%| | 28/85466 [00:08<6:03:23, 3.92it/s] step 187500 C loss 0.0005 std 0.030 cos 0.868 lr 2.66e-04 epoch 7/10: 1%|▏ | 529/85466 [02:02<6:18:37, 3.74it/s] step 188000 B loss 0.0003 std 0.043 cos 0.935 lr 2.66e-04 epoch 7/10: 1%|▎ | 1029/85466 [03:56<4:50:22, 4.85it/s] step 188500 A loss 0.0004 std 0.032 cos 0.902 lr 2.66e-04 epoch 7/10: 2%|▍ | 1528/85466 [05:50<6:00:09, 3.88it/s] step 189000 C loss 0.0002 std 0.033 cos 0.956 lr 2.65e-04 epoch 7/10: 2%|▌ | 2029/85466 [07:44<5:49:09, 3.98it/s] step 189500 B loss 0.0004 std 0.042 cos 0.904 lr 2.65e-04 epoch 7/10: 3%|▋ | 2529/85466 [09:39<4:34:38, 5.03it/s] step 190000 A loss 0.0005 std 0.034 cos 0.884 lr 2.65e-04 epoch 7/10: 4%|▊ | 3028/85466 [11:32<5:33:52, 4.12it/s] step 190500 C loss 0.0002 std 0.033 cos 0.945 lr 2.65e-04 epoch 7/10: 4%|▉ | 3529/85466 [13:27<6:11:11, 3.68it/s] step 191000 B loss 0.0003 std 0.042 cos 0.918 lr 2.65e-04 epoch 7/10: 5%|█▏ | 4029/85466 [15:21<4:31:06, 5.01it/s] step 191500 A loss 0.0006 std 0.034 cos 0.840 lr 2.64e-04 epoch 7/10: 6%|█▌ | 5315/85466 [16:16<35:01, 38.14it/s] step 192000 B loss 0.0001 std 0.043 cos 0.965 lr 2.64e-04 epoch 7/10: 8%|██ | 6818/85466 [16:55<35:50, 36.57it/s] step 192500 B loss 0.0004 std 0.043 cos 0.907 lr 2.64e-04 epoch 7/10: 10%|██▌ | 8318/85466 [17:34<31:39, 40.62it/s] step 193000 B loss 0.0004 std 0.043 cos 0.887 lr 2.64e-04 epoch 7/10: 11%|██▉ | 9815/85466 [18:11<31:09, 40.47it/s] step 193500 B loss 0.0003 std 0.043 cos 0.936 lr 2.64e-04 epoch 7/10: 13%|███▎ | 11315/85466 [18:48<30:11, 40.92it/s] step 194000 B loss 0.0004 std 0.042 cos 0.897 lr 2.64e-04 epoch 7/10: 15%|███▋ | 12815/85466 [19:27<27:55, 43.37it/s] step 194500 B loss 0.0003 std 0.043 cos 0.923 lr 2.63e-04 epoch 7/10: 17%|████▏ | 14319/85466 [20:06<33:44, 35.14it/s] step 195000 B loss 0.0002 std 0.043 cos 0.947 lr 2.63e-04 epoch 7/10: 19%|████▋ | 15818/85466 [20:44<30:03, 38.61it/s] step 195500 B loss 0.0004 std 0.043 cos 0.889 lr 2.63e-04 epoch 7/10: 20%|█████ | 17320/85466 [21:23<24:42, 45.98it/s] step 196000 B loss 0.0003 std 0.043 cos 0.933 lr 2.63e-04 epoch 7/10: 22%|█████▌ | 18818/85466 [22:01<30:21, 36.59it/s] step 196500 B loss 0.0002 std 0.042 cos 0.936 lr 2.63e-04 epoch 7/10: 24%|█████▉ | 20318/85466 [22:40<24:54, 43.61it/s] step 197000 B loss 0.0002 std 0.043 cos 0.947 lr 2.63e-04 epoch 7/10: 26%|██████▍ | 21818/85466 [23:18<24:51, 42.67it/s] step 197500 B loss 0.0003 std 0.042 cos 0.923 lr 2.62e-04 epoch 7/10: 27%|██████▊ | 23312/85466 [23:55<22:38, 45.76it/s] step 198000 B loss 0.0002 std 0.043 cos 0.949 lr 2.62e-04 epoch 7/10: 29%|███████▎ | 24818/85466 [24:35<27:20, 36.97it/s] step 198500 B loss 0.0002 std 0.043 cos 0.949 lr 2.62e-04 epoch 7/10: 31%|███████▋ | 26315/85466 [25:13<24:14, 40.68it/s] step 199000 B loss 0.0003 std 0.043 cos 0.917 lr 2.62e-04 epoch 7/10: 33%|████████▏ | 27818/85466 [25:51<24:52, 38.62it/s] step 199500 B loss 0.0003 std 0.043 cos 0.925 lr 2.62e-04 epoch 7/10: 34%|████████▌ | 29315/85466 [26:29<23:57, 39.05it/s] step 200000 B loss 0.0002 std 0.043 cos 0.944 lr 2.61e-04 epoch 7/10: 36%|█████████ | 30818/85466 [27:08<19:39, 46.33it/s] step 200500 B loss 0.0001 std 0.042 cos 0.965 lr 2.61e-04 epoch 7/10: 38%|█████████▍ | 32315/85466 [27:47<25:13, 35.12it/s] step 201000 B loss 0.0003 std 0.042 cos 0.918 lr 2.61e-04 epoch 7/10: 40%|█████████▉ | 33818/85466 [28:26<20:49, 41.32it/s] step 201500 B loss 0.0002 std 0.043 cos 0.943 lr 2.61e-04 epoch 7/10: 41%|██████████▎ | 35318/85466 [29:05<19:42, 42.41it/s] step 202000 B loss 0.0002 std 0.042 cos 0.953 lr 2.61e-04 epoch 7/10: 43%|██████████▊ | 36817/85466 [29:43<18:59, 42.68it/s] step 202500 B loss 0.0002 std 0.043 cos 0.947 lr 2.60e-04 epoch 7/10: 45%|███████████▏ | 38315/85466 [30:21<19:27, 40.40it/s] step 203000 B loss 0.0003 std 0.042 cos 0.913 lr 2.60e-04 epoch 7/10: 47%|███████████▋ | 39815/85466 [31:00<17:50, 42.63it/s] step 203500 B loss 0.0002 std 0.043 cos 0.943 lr 2.60e-04 epoch 7/10: 48%|████████████ | 41319/85466 [31:39<19:47, 37.19it/s] step 204000 B loss 0.0003 std 0.043 cos 0.917 lr 2.60e-04 epoch 7/10: 50%|████████████▌ | 42815/85466 [32:17<17:36, 40.39it/s] step 204500 B loss 0.0003 std 0.043 cos 0.916 lr 2.60e-04 epoch 7/10: 52%|████████████▉ | 44313/85466 [32:55<21:54, 31.31it/s] step 205000 B loss 0.0003 std 0.043 cos 0.933 lr 2.60e-04 epoch 7/10: 54%|█████████████▍ | 45815/85466 [33:34<16:50, 39.25it/s] step 205500 B loss 0.0004 std 0.043 cos 0.894 lr 2.59e-04 epoch 7/10: 55%|█████████████▊ | 47318/85466 [34:12<15:43, 40.45it/s] step 206000 B loss 0.0002 std 0.043 cos 0.943 lr 2.59e-04 epoch 7/10: 57%|██████████████▎ | 48818/85466 [34:51<15:42, 38.90it/s] step 206500 B loss 0.0002 std 0.041 cos 0.943 lr 2.59e-04 epoch 7/10: 59%|██████████████▋ | 50315/85466 [35:29<13:03, 44.84it/s] step 207000 B loss 0.0002 std 0.043 cos 0.944 lr 2.59e-04 epoch 7/10: 61%|███████████████▏ | 51815/85466 [36:06<15:43, 35.65it/s] step 207500 B loss 0.0003 std 0.042 cos 0.922 lr 2.59e-04 epoch 7/10: 62%|███████████████▌ | 53315/85466 [36:46<14:14, 37.64it/s] step 208000 B loss 0.0002 std 0.042 cos 0.954 lr 2.58e-04 epoch 7/10: 64%|████████████████ | 54815/85466 [37:26<14:28, 35.29it/s] step 208500 B loss 0.0003 std 0.043 cos 0.931 lr 2.58e-04 epoch 7/10: 66%|████████████████▍ | 56318/85466 [38:04<11:24, 42.60it/s] step 209000 B loss 0.0002 std 0.043 cos 0.953 lr 2.58e-04 epoch 7/10: 68%|████████████████▉ | 57815/85466 [38:42<11:32, 39.90it/s] step 209500 B loss 0.0001 std 0.042 cos 0.963 lr 2.58e-04 epoch 7/10: 69%|█████████████████▎ | 59318/85466 [39:21<12:35, 34.59it/s] step 210000 B loss 0.0002 std 0.043 cos 0.949 lr 2.58e-04 epoch 7/10: 71%|█████████████████▊ | 60818/85466 [40:00<10:19, 39.80it/s] step 210500 B loss 0.0003 std 0.042 cos 0.935 lr 2.57e-04 epoch 7/10: 73%|██████████████████▏ | 62318/85466 [40:38<08:52, 43.50it/s] step 211000 B loss 0.0005 std 0.042 cos 0.877 lr 2.57e-04 epoch 7/10: 75%|██████████████████▋ | 63815/85466 [41:16<10:11, 35.40it/s] step 211500 B loss 0.0004 std 0.043 cos 0.907 lr 2.57e-04 epoch 7/10: 76%|███████████████████ | 65318/85466 [41:55<08:40, 38.73it/s] step 212000 B loss 0.0003 std 0.043 cos 0.918 lr 2.57e-04 epoch 7/10: 78%|███████████████████▌ | 66818/85466 [42:34<07:02, 44.13it/s] step 212500 B loss 0.0003 std 0.042 cos 0.918 lr 2.57e-04 epoch 7/10: 80%|███████████████████▉ | 68318/85466 [43:13<07:11, 39.72it/s] step 213000 B loss 0.0004 std 0.043 cos 0.900 lr 2.56e-04 epoch 7/10: 82%|████████████████████▍ | 69815/85466 [43:52<07:24, 35.24it/s] step 213500 B loss 0.0002 std 0.042 cos 0.945 lr 2.56e-04 epoch 7/10: 83%|████████████████████▊ | 71318/85466 [44:30<05:56, 39.70it/s] step 214000 B loss 0.0003 std 0.043 cos 0.934 lr 2.56e-04 epoch 7/10: 85%|█████████████████████▎ | 72815/85466 [45:08<04:50, 43.52it/s] step 214500 B loss 0.0003 std 0.042 cos 0.920 lr 2.56e-04 epoch 7/10: 87%|█████████████████████▋ | 74315/85466 [45:47<05:28, 33.96it/s] step 215000 B loss 0.0002 std 0.042 cos 0.945 lr 2.56e-04 epoch 7/10: 89%|██████████████████████▏ | 75815/85466 [46:27<04:08, 38.90it/s] step 215500 B loss 0.0003 std 0.043 cos 0.929 lr 2.55e-04 epoch 7/10: 90%|██████████████████████▌ | 77315/85466 [47:06<03:54, 34.83it/s] step 216000 B loss 0.0004 std 0.043 cos 0.890 lr 2.55e-04 epoch 7/10: 92%|███████████████████████ | 78815/85466 [47:46<02:46, 40.02it/s] step 216500 B loss 0.0002 std 0.042 cos 0.943 lr 2.55e-04 epoch 7/10: 94%|███████████████████████▍ | 80316/85466 [48:25<02:29, 34.37it/s] step 217000 B loss 0.0003 std 0.041 cos 0.930 lr 2.55e-04 epoch 7/10: 96%|███████████████████████▉ | 81815/85466 [49:04<01:28, 41.06it/s] step 217500 B loss 0.0002 std 0.042 cos 0.941 lr 2.55e-04 epoch 7/10: 97%|████████████████████████▎| 83315/85466 [49:44<00:57, 37.72it/s] step 218000 B loss 0.0002 std 0.043 cos 0.948 lr 2.55e-04 epoch 7/10: 99%|████████████████████████▊| 84818/85466 [50:24<00:18, 35.75it/s] step 218500 B loss 0.0002 std 0.043 cos 0.944 lr 2.54e-04 epoch 7/10: 100%|█████████████████████████| 85466/85466 [50:40<00:00, 28.11it/s] epoch 7/10 avg_loss 0.0001 3041s epoch 8/10: 0%| | 283/85466 [01:07<6:04:37, 3.89it/s] step 219000 C loss 0.0003 std 0.030 cos 0.935 lr 2.54e-04 epoch 8/10: 1%|▏ | 784/85466 [03:02<5:50:00, 4.03it/s] step 219500 B loss 0.0002 std 0.043 cos 0.943 lr 2.54e-04 epoch 8/10: 2%|▎ | 1284/85466 [04:56<4:37:59, 5.05it/s] step 220000 A loss 0.0004 std 0.033 cos 0.901 lr 2.54e-04 epoch 8/10: 2%|▌ | 1783/85466 [06:51<5:55:24, 3.92it/s] step 220500 C loss 0.0003 std 0.035 cos 0.920 lr 2.54e-04 epoch 8/10: 3%|▋ | 2284/85466 [08:47<5:51:23, 3.95it/s] step 221000 B loss 0.0003 std 0.043 cos 0.920 lr 2.53e-04 epoch 8/10: 3%|▊ | 2784/85466 [10:43<4:48:40, 4.77it/s] step 221500 A loss 0.0005 std 0.037 cos 0.883 lr 2.53e-04 epoch 8/10: 4%|▉ | 3283/85466 [12:37<5:49:06, 3.92it/s] step 222000 C loss 0.0005 std 0.034 cos 0.865 lr 2.53e-04 epoch 8/10: 4%|█ | 3784/85466 [14:33<5:49:19, 3.90it/s] step 222500 B loss 0.0002 std 0.043 cos 0.953 lr 2.53e-04 epoch 8/10: 5%|█▍ | 4583/85466 [16:05<34:21, 39.24it/s] step 223000 B loss 0.0001 std 0.043 cos 0.969 lr 2.53e-04 epoch 8/10: 7%|█▊ | 6083/85466 [16:43<38:53, 34.01it/s] step 223500 B loss 0.0001 std 0.041 cos 0.967 lr 2.52e-04 epoch 8/10: 9%|██▎ | 7583/85466 [17:21<36:01, 36.03it/s] step 224000 B loss 0.0004 std 0.043 cos 0.894 lr 2.52e-04 epoch 8/10: 11%|██▊ | 9080/85466 [18:00<33:39, 37.83it/s] step 224500 B loss 0.0004 std 0.043 cos 0.893 lr 2.52e-04 epoch 8/10: 12%|███ | 10580/85466 [18:39<38:07, 32.73it/s] step 225000 B loss 0.0004 std 0.042 cos 0.899 lr 2.52e-04 epoch 8/10: 14%|███▌ | 12082/85466 [19:18<29:42, 41.17it/s] step 225500 B loss 0.0003 std 0.042 cos 0.930 lr 2.52e-04 epoch 8/10: 16%|███▉ | 13580/85466 [19:57<34:26, 34.79it/s] step 226000 B loss 0.0003 std 0.043 cos 0.933 lr 2.51e-04 epoch 8/10: 18%|████▍ | 15083/85466 [20:36<30:02, 39.04it/s] step 226500 B loss 0.0003 std 0.041 cos 0.932 lr 2.51e-04 epoch 8/10: 19%|████▊ | 16580/85466 [21:15<27:33, 41.67it/s] step 227000 B loss 0.0003 std 0.042 cos 0.927 lr 2.51e-04 epoch 8/10: 21%|█████▎ | 18080/85466 [21:54<30:41, 36.59it/s] step 227500 B loss 0.0004 std 0.042 cos 0.888 lr 2.51e-04 epoch 8/10: 23%|█████▋ | 19580/85466 [22:32<28:32, 38.48it/s] step 228000 B loss 0.0004 std 0.042 cos 0.909 lr 2.50e-04 epoch 8/10: 25%|██████▏ | 21084/85466 [23:10<34:12, 31.37it/s] step 228500 B loss 0.0002 std 0.043 cos 0.960 lr 2.50e-04 epoch 8/10: 26%|██████▌ | 22581/85466 [23:49<26:33, 39.46it/s] step 229000 B loss 0.0003 std 0.043 cos 0.919 lr 2.50e-04 epoch 8/10: 28%|███████ | 24080/85466 [24:27<26:44, 38.27it/s] step 229500 B loss 0.0004 std 0.042 cos 0.909 lr 2.50e-04 epoch 8/10: 30%|███████▍ | 25581/85466 [25:04<27:42, 36.02it/s] step 230000 B loss 0.0002 std 0.043 cos 0.939 lr 2.50e-04 epoch 8/10: 32%|███████▉ | 27084/85466 [25:43<25:28, 38.20it/s] step 230500 B loss 0.0001 std 0.042 cos 0.965 lr 2.49e-04 epoch 8/10: 33%|████████▎ | 28580/85466 [26:22<23:25, 40.48it/s] step 231000 B loss 0.0001 std 0.043 cos 0.971 lr 2.49e-04 epoch 8/10: 35%|████████▊ | 30080/85466 [27:00<24:36, 37.51it/s] step 231500 B loss 0.0002 std 0.043 cos 0.936 lr 2.49e-04 epoch 8/10: 37%|█████████▏ | 31583/85466 [27:39<19:43, 45.52it/s] step 232000 B loss 0.0003 std 0.043 cos 0.918 lr 2.49e-04 epoch 8/10: 39%|█████████▋ | 33080/85466 [28:18<23:14, 37.58it/s] step 232500 B loss 0.0001 std 0.043 cos 0.962 lr 2.49e-04 epoch 8/10: 40%|██████████ | 34580/85466 [28:57<24:04, 35.22it/s] step 233000 B loss 0.0002 std 0.043 cos 0.940 lr 2.48e-04 epoch 8/10: 42%|██████████▌ | 36083/85466 [29:36<20:55, 39.34it/s] step 233500 B loss 0.0003 std 0.043 cos 0.921 lr 2.48e-04 epoch 8/10: 44%|██████████▉ | 37583/85466 [30:15<20:40, 38.60it/s] step 234000 B loss 0.0004 std 0.043 cos 0.897 lr 2.48e-04 epoch 8/10: 46%|███████████▍ | 39080/85466 [30:53<18:25, 41.96it/s] step 234500 B loss 0.0003 std 0.043 cos 0.923 lr 2.48e-04 epoch 8/10: 47%|███████████▊ | 40583/85466 [31:32<20:44, 36.07it/s] step 235000 B loss 0.0005 std 0.043 cos 0.867 lr 2.48e-04 epoch 8/10: 49%|████████████▎ | 42083/85466 [32:11<21:30, 33.61it/s] step 235500 B loss 0.0002 std 0.043 cos 0.937 lr 2.47e-04 epoch 8/10: 51%|████████████▋ | 43583/85466 [32:50<17:57, 38.88it/s] step 236000 B loss 0.0002 std 0.042 cos 0.937 lr 2.47e-04 epoch 8/10: 53%|█████████████▏ | 45078/85466 [33:28<18:18, 36.78it/s] step 236500 B loss 0.0003 std 0.043 cos 0.921 lr 2.47e-04 epoch 8/10: 55%|█████████████▋ | 46583/85466 [34:07<15:32, 41.69it/s] step 237000 B loss 0.0003 std 0.043 cos 0.925 lr 2.47e-04 epoch 8/10: 56%|██████████████ | 48080/85466 [34:47<13:54, 44.81it/s] step 237500 B loss 0.0004 std 0.042 cos 0.898 lr 2.47e-04 epoch 8/10: 58%|██████████████▌ | 49578/85466 [35:26<16:21, 36.57it/s] step 238000 B loss 0.0003 std 0.042 cos 0.911 lr 2.46e-04 epoch 8/10: 60%|██████████████▉ | 51083/85466 [36:04<17:54, 32.01it/s] step 238500 B loss 0.0002 std 0.043 cos 0.956 lr 2.46e-04 epoch 8/10: 62%|███████████████▍ | 52580/85466 [36:43<13:44, 39.91it/s] step 239000 B loss 0.0003 std 0.041 cos 0.919 lr 2.46e-04 epoch 8/10: 63%|███████████████▊ | 54083/85466 [37:20<14:36, 35.81it/s] step 239500 B loss 0.0002 std 0.043 cos 0.948 lr 2.46e-04 epoch 8/10: 65%|████████████████▎ | 55583/85466 [37:58<11:23, 43.72it/s] step 240000 B loss 0.0002 std 0.043 cos 0.936 lr 2.45e-04 epoch 8/10: 67%|████████████████▋ | 57080/85466 [38:37<13:37, 34.71it/s] step 240500 B loss 0.0003 std 0.042 cos 0.935 lr 2.45e-04 epoch 8/10: 69%|█████████████████▏ | 58583/85466 [39:15<12:23, 36.16it/s] step 241000 B loss 0.0003 std 0.043 cos 0.923 lr 2.45e-04 epoch 8/10: 70%|█████████████████▌ | 60083/85466 [39:54<08:33, 49.41it/s] step 241500 B loss 0.0001 std 0.043 cos 0.967 lr 2.45e-04 epoch 8/10: 72%|██████████████████ | 61584/85466 [40:33<10:19, 38.58it/s] step 242000 B loss 0.0003 std 0.042 cos 0.915 lr 2.45e-04 epoch 8/10: 74%|██████████████████▍ | 63080/85466 [41:12<10:00, 37.26it/s] step 242500 B loss 0.0003 std 0.042 cos 0.927 lr 2.44e-04 epoch 8/10: 76%|██████████████████▉ | 64580/85466 [41:51<07:59, 43.55it/s] step 243000 B loss 0.0001 std 0.043 cos 0.968 lr 2.44e-04 epoch 8/10: 77%|███████████████████▎ | 66083/85466 [42:30<07:58, 40.47it/s] step 243500 B loss 0.0003 std 0.043 cos 0.917 lr 2.44e-04 epoch 8/10: 79%|███████████████████▊ | 67580/85466 [43:09<07:45, 38.39it/s] step 244000 B loss 0.0003 std 0.041 cos 0.927 lr 2.44e-04 epoch 8/10: 81%|████████████████████▏ | 69083/85466 [43:49<06:20, 43.03it/s] step 244500 B loss 0.0002 std 0.043 cos 0.944 lr 2.44e-04 epoch 8/10: 83%|████████████████████▋ | 70583/85466 [44:28<05:49, 42.53it/s] step 245000 B loss 0.0003 std 0.042 cos 0.923 lr 2.43e-04 epoch 8/10: 84%|█████████████████████ | 72083/85466 [45:07<04:58, 44.81it/s] step 245500 B loss 0.0005 std 0.043 cos 0.884 lr 2.43e-04 epoch 8/10: 86%|█████████████████████▌ | 73583/85466 [45:46<05:07, 38.62it/s] step 246000 B loss 0.0002 std 0.043 cos 0.956 lr 2.43e-04 epoch 8/10: 88%|█████████████████████▉ | 75081/85466 [46:26<04:33, 37.97it/s] step 246500 B loss 0.0003 std 0.042 cos 0.931 lr 2.43e-04 epoch 8/10: 90%|██████████████████████▍ | 76584/85466 [47:05<03:27, 42.83it/s] step 247000 B loss 0.0002 std 0.043 cos 0.956 lr 2.42e-04 epoch 8/10: 91%|██████████████████████▊ | 78080/85466 [47:44<02:58, 41.35it/s] step 247500 B loss 0.0003 std 0.042 cos 0.929 lr 2.42e-04 epoch 8/10: 93%|███████████████████████▎ | 79580/85466 [48:22<02:53, 33.85it/s] step 248000 B loss 0.0002 std 0.043 cos 0.959 lr 2.42e-04 epoch 8/10: 95%|███████████████████████▋ | 81080/85466 [49:00<01:43, 42.58it/s] step 248500 B loss 0.0003 std 0.043 cos 0.925 lr 2.42e-04 epoch 8/10: 97%|████████████████████████▏| 82584/85466 [49:39<01:10, 41.01it/s] step 249000 B loss 0.0003 std 0.040 cos 0.934 lr 2.42e-04 epoch 8/10: 98%|████████████████████████▌| 84084/85466 [50:19<00:35, 38.47it/s] step 249500 B loss 0.0001 std 0.043 cos 0.967 lr 2.41e-04 epoch 8/10: 100%|█████████████████████████| 85466/85466 [50:53<00:00, 27.99it/s] epoch 8/10 avg_loss 0.0001 3055s epoch 9/10: 0%| | 39/85466 [00:11<4:51:03, 4.89it/s] step 250000 A loss 0.0007 std 0.033 cos 0.817 lr 2.41e-04 epoch 9/10: 1%|▏ | 538/85466 [02:05<6:16:22, 3.76it/s] step 250500 C loss 0.0002 std 0.032 cos 0.957 lr 2.41e-04 epoch 9/10: 1%|▎ | 1039/85466 [03:59<5:44:17, 4.09it/s] step 251000 B loss 0.0004 std 0.042 cos 0.907 lr 2.41e-04 epoch 9/10: 2%|▍ | 1539/85466 [05:53<4:48:03, 4.86it/s] step 251500 A loss 0.0005 std 0.034 cos 0.880 lr 2.40e-04 epoch 9/10: 2%|▌ | 2038/85466 [07:47<5:50:41, 3.96it/s] step 252000 C loss 0.0003 std 0.036 cos 0.911 lr 2.40e-04 epoch 9/10: 3%|▋ | 2539/85466 [09:42<5:57:56, 3.86it/s] step 252500 B loss 0.0003 std 0.041 cos 0.934 lr 2.40e-04 epoch 9/10: 4%|▊ | 3039/85466 [11:36<4:37:22, 4.95it/s] step 253000 A loss 0.0006 std 0.034 cos 0.839 lr 2.40e-04 epoch 9/10: 4%|▉ | 3538/85466 [13:30<5:33:42, 4.09it/s] step 253500 C loss 0.0004 std 0.036 cos 0.896 lr 2.40e-04 epoch 9/10: 5%|█▏ | 4039/85466 [15:24<5:54:26, 3.83it/s] step 254000 B loss 0.0004 std 0.042 cos 0.910 lr 2.39e-04 epoch 9/10: 6%|█▋ | 5348/85466 [16:17<34:09, 39.08it/s] step 254500 B loss 0.0002 std 0.043 cos 0.960 lr 2.39e-04 epoch 9/10: 8%|██ | 6845/85466 [16:56<38:50, 33.74it/s] step 255000 B loss 0.0003 std 0.043 cos 0.911 lr 2.39e-04 epoch 9/10: 10%|██▌ | 8345/85466 [17:36<36:37, 35.09it/s] step 255500 B loss 0.0003 std 0.042 cos 0.929 lr 2.39e-04 epoch 9/10: 12%|██▉ | 9845/85466 [18:14<26:30, 47.53it/s] step 256000 B loss 0.0002 std 0.043 cos 0.948 lr 2.39e-04 epoch 9/10: 13%|███▎ | 11348/85466 [18:52<32:33, 37.93it/s] step 256500 B loss 0.0003 std 0.043 cos 0.921 lr 2.38e-04 epoch 9/10: 15%|███▊ | 12848/85466 [19:31<31:55, 37.90it/s] step 257000 B loss 0.0004 std 0.040 cos 0.901 lr 2.38e-04 epoch 9/10: 17%|████▏ | 14348/85466 [20:10<26:21, 44.97it/s] step 257500 B loss 0.0003 std 0.043 cos 0.915 lr 2.38e-04 epoch 9/10: 19%|████▋ | 15848/85466 [20:48<34:19, 33.80it/s] step 258000 B loss 0.0004 std 0.042 cos 0.898 lr 2.38e-04 epoch 9/10: 20%|█████ | 17348/85466 [21:26<27:48, 40.81it/s] step 258500 B loss 0.0002 std 0.043 cos 0.959 lr 2.37e-04 epoch 9/10: 22%|█████▌ | 18848/85466 [22:05<31:31, 35.22it/s] step 259000 B loss 0.0004 std 0.043 cos 0.904 lr 2.37e-04 epoch 9/10: 24%|█████▉ | 20345/85466 [22:44<28:43, 37.79it/s] step 259500 B loss 0.0002 std 0.043 cos 0.959 lr 2.37e-04 epoch 9/10: 26%|██████▍ | 21845/85466 [23:23<25:11, 42.09it/s] step 260000 B loss 0.0002 std 0.043 cos 0.946 lr 2.37e-04 epoch 9/10: 27%|██████▊ | 23345/85466 [24:04<23:58, 43.18it/s] step 260500 B loss 0.0001 std 0.043 cos 0.966 lr 2.36e-04 epoch 9/10: 29%|███████▎ | 24848/85466 [24:42<29:09, 34.64it/s] step 261000 B loss 0.0001 std 0.043 cos 0.964 lr 2.36e-04 epoch 9/10: 31%|███████▋ | 26348/85466 [25:21<22:17, 44.21it/s] step 261500 B loss 0.0003 std 0.042 cos 0.931 lr 2.36e-04 epoch 9/10: 33%|████████▏ | 27848/85466 [25:59<22:03, 43.54it/s] step 262000 B loss 0.0003 std 0.043 cos 0.922 lr 2.36e-04 epoch 9/10: 34%|████████▌ | 29345/85466 [26:37<22:36, 41.37it/s] step 262500 B loss 0.0002 std 0.043 cos 0.953 lr 2.36e-04 epoch 9/10: 36%|█████████ | 30848/85466 [27:16<24:12, 37.59it/s] step 263000 B loss 0.0004 std 0.043 cos 0.885 lr 2.35e-04 epoch 9/10: 38%|█████████▍ | 32348/85466 [27:54<21:56, 40.36it/s] step 263500 B loss 0.0002 std 0.043 cos 0.952 lr 2.35e-04 epoch 9/10: 40%|█████████▉ | 33848/85466 [28:32<20:25, 42.13it/s] step 264000 B loss 0.0003 std 0.042 cos 0.933 lr 2.35e-04 epoch 9/10: 41%|██████████▎ | 35345/85466 [29:11<19:40, 42.46it/s] step 264500 B loss 0.0004 std 0.042 cos 0.897 lr 2.35e-04 epoch 9/10: 43%|██████████▊ | 36848/85466 [29:49<20:56, 38.70it/s] step 265000 B loss 0.0003 std 0.042 cos 0.926 lr 2.34e-04 epoch 9/10: 45%|███████████▏ | 38345/85466 [30:29<23:24, 33.56it/s] step 265500 B loss 0.0006 std 0.043 cos 0.848 lr 2.34e-04 epoch 9/10: 47%|███████████▋ | 39845/85466 [31:06<17:50, 42.63it/s] step 266000 B loss 0.0002 std 0.043 cos 0.949 lr 2.34e-04 epoch 9/10: 48%|████████████ | 41348/85466 [31:45<20:23, 36.05it/s] step 266500 B loss 0.0003 std 0.043 cos 0.918 lr 2.34e-04 epoch 9/10: 50%|████████████▌ | 42845/85466 [32:23<18:22, 38.67it/s] step 267000 B loss 0.0004 std 0.043 cos 0.907 lr 2.34e-04 epoch 9/10: 52%|████████████▉ | 44346/85466 [33:04<18:32, 36.96it/s] step 267500 B loss 0.0003 std 0.042 cos 0.911 lr 2.33e-04 epoch 9/10: 54%|█████████████▍ | 45845/85466 [33:43<15:10, 43.49it/s] step 268000 B loss 0.0002 std 0.043 cos 0.942 lr 2.33e-04 epoch 9/10: 55%|█████████████▊ | 47348/85466 [34:21<15:12, 41.75it/s] step 268500 B loss 0.0002 std 0.043 cos 0.942 lr 2.33e-04 epoch 9/10: 57%|██████████████▎ | 48848/85466 [35:00<16:22, 37.27it/s] step 269000 B loss 0.0001 std 0.042 cos 0.964 lr 2.33e-04 epoch 9/10: 59%|██████████████▋ | 50345/85466 [35:39<13:49, 42.33it/s] step 269500 B loss 0.0002 std 0.043 cos 0.949 lr 2.32e-04 epoch 9/10: 61%|███████████████▏ | 51848/85466 [36:17<12:37, 44.40it/s] step 270000 B loss 0.0003 std 0.043 cos 0.932 lr 2.32e-04 epoch 9/10: 62%|███████████████▌ | 53348/85466 [36:55<13:04, 40.93it/s] step 270500 B loss 0.0002 std 0.043 cos 0.959 lr 2.32e-04 epoch 9/10: 64%|████████████████ | 54850/85466 [37:34<12:08, 42.05it/s] step 271000 B loss 0.0007 std 0.043 cos 0.815 lr 2.32e-04 epoch 9/10: 66%|████████████████▍ | 56349/85466 [38:14<12:58, 37.40it/s] step 271500 B loss 0.0003 std 0.043 cos 0.934 lr 2.31e-04 epoch 9/10: 68%|████████████████▉ | 57848/85466 [38:52<12:04, 38.10it/s] step 272000 B loss 0.0003 std 0.043 cos 0.923 lr 2.31e-04 epoch 9/10: 69%|█████████████████▎ | 59348/85466 [39:31<10:12, 42.62it/s] step 272500 B loss 0.0002 std 0.043 cos 0.938 lr 2.31e-04 epoch 9/10: 71%|█████████████████▊ | 60845/85466 [40:10<11:16, 36.37it/s] step 273000 B loss 0.0003 std 0.041 cos 0.935 lr 2.31e-04 epoch 9/10: 73%|██████████████████▏ | 62348/85466 [40:49<10:45, 35.81it/s] step 273500 B loss 0.0003 std 0.042 cos 0.930 lr 2.31e-04 epoch 9/10: 75%|██████████████████▋ | 63848/85466 [41:27<10:07, 35.57it/s] step 274000 B loss 0.0002 std 0.042 cos 0.950 lr 2.30e-04 epoch 9/10: 76%|███████████████████ | 65348/85466 [42:06<09:05, 36.86it/s] step 274500 B loss 0.0001 std 0.043 cos 0.967 lr 2.30e-04 epoch 9/10: 78%|███████████████████▌ | 66848/85466 [42:45<08:24, 36.93it/s] step 275000 B loss 0.0001 std 0.043 cos 0.976 lr 2.30e-04 epoch 9/10: 80%|███████████████████▉ | 68348/85466 [43:25<08:04, 35.32it/s] step 275500 B loss 0.0002 std 0.042 cos 0.959 lr 2.30e-04 epoch 9/10: 82%|████████████████████▍ | 69848/85466 [44:04<06:59, 37.25it/s] step 276000 B loss 0.0002 std 0.043 cos 0.942 lr 2.29e-04 epoch 9/10: 83%|████████████████████▊ | 71348/85466 [44:42<05:50, 40.32it/s] step 276500 B loss 0.0003 std 0.042 cos 0.923 lr 2.29e-04 epoch 9/10: 85%|█████████████████████▎ | 72848/85466 [45:22<05:08, 40.84it/s] step 277000 B loss 0.0003 std 0.042 cos 0.927 lr 2.29e-04 epoch 9/10: 87%|█████████████████████▋ | 74345/85466 [46:01<04:42, 39.33it/s] step 277500 B loss 0.0003 std 0.043 cos 0.920 lr 2.29e-04 epoch 9/10: 89%|██████████████████████▏ | 75849/85466 [46:39<05:11, 30.87it/s] step 278000 B loss 0.0003 std 0.043 cos 0.920 lr 2.28e-04 epoch 9/10: 90%|██████████████████████▌ | 77345/85466 [47:18<03:42, 36.52it/s] step 278500 B loss 0.0004 std 0.043 cos 0.909 lr 2.28e-04 epoch 9/10: 92%|███████████████████████ | 78845/85466 [47:58<02:26, 45.19it/s] step 279000 B loss 0.0003 std 0.042 cos 0.911 lr 2.28e-04 epoch 9/10: 94%|███████████████████████▌ | 80348/85466 [48:37<02:12, 38.68it/s] step 279500 B loss 0.0003 std 0.043 cos 0.930 lr 2.28e-04 epoch 9/10: 96%|███████████████████████▉ | 81848/85466 [49:16<01:28, 40.93it/s] step 280000 B loss 0.0001 std 0.042 cos 0.963 lr 2.27e-04 epoch 9/10: 98%|████████████████████████▍| 83350/85466 [49:55<00:54, 38.55it/s] step 280500 B loss 0.0002 std 0.043 cos 0.956 lr 2.27e-04 epoch 9/10: 99%|████████████████████████▊| 84845/85466 [50:33<00:14, 42.30it/s] step 281000 B loss 0.0003 std 0.043 cos 0.922 lr 2.27e-04 epoch 9/10: 100%|█████████████████████████| 85466/85466 [50:49<00:00, 28.03it/s] epoch 9/10 avg_loss 0.0001 3051s epoch 10/10: 0%| | 294/85466 [01:08<4:46:57, 4.95it/s] step 281500 A loss 0.0004 std 0.029 cos 0.889 lr 2.27e-04 epoch 10/10: 1%|▏ | 793/85466 [03:03<5:59:58, 3.92it/s] step 282000 C loss 0.0002 std 0.032 cos 0.937 lr 2.27e-04 epoch 10/10: 2%|▎ | 1294/85466 [04:58<6:12:37, 3.76it/s] step 282500 B loss 0.0004 std 0.042 cos 0.901 lr 2.26e-04 epoch 10/10: 2%|▍ | 1794/85466 [06:51<4:38:12, 5.01it/s] step 283000 A loss 0.0003 std 0.036 cos 0.934 lr 2.26e-04 epoch 10/10: 3%|▌ | 2293/85466 [08:44<6:14:14, 3.70it/s] step 283500 C loss 0.0003 std 0.037 cos 0.925 lr 2.26e-04 epoch 10/10: 3%|▊ | 2794/85466 [10:39<5:46:52, 3.97it/s] step 284000 B loss 0.0003 std 0.043 cos 0.914 lr 2.26e-04 epoch 10/10: 4%|▉ | 3294/85466 [12:33<4:31:17, 5.05it/s] step 284500 A loss 0.0004 std 0.037 cos 0.888 lr 2.25e-04 epoch 10/10: 4%|█ | 3793/85466 [14:27<5:42:17, 3.98it/s] step 285000 C loss 0.0005 std 0.036 cos 0.860 lr 2.25e-04 epoch 10/10: 5%|█▎ | 4610/85466 [15:57<36:41, 36.72it/s] step 285500 B loss 0.0002 std 0.042 cos 0.938 lr 2.25e-04 epoch 10/10: 7%|█▊ | 6113/85466 [16:36<41:34, 31.81it/s] step 286000 B loss 0.0003 std 0.043 cos 0.916 lr 2.25e-04 epoch 10/10: 9%|██▏ | 7613/85466 [17:14<29:39, 43.75it/s] step 286500 B loss 0.0002 std 0.043 cos 0.958 lr 2.24e-04 epoch 10/10: 11%|██▋ | 9113/85466 [17:53<35:36, 35.74it/s] step 287000 B loss 0.0003 std 0.043 cos 0.928 lr 2.24e-04 epoch 10/10: 12%|██▉ | 10613/85466 [18:33<30:53, 40.37it/s] step 287500 B loss 0.0002 std 0.043 cos 0.949 lr 2.24e-04 epoch 10/10: 14%|███▍ | 12115/85466 [19:12<29:25, 41.55it/s] step 288000 B loss 0.0003 std 0.043 cos 0.918 lr 2.24e-04 epoch 10/10: 16%|███▊ | 13610/85466 [19:52<28:54, 41.43it/s] step 288500 B loss 0.0003 std 0.042 cos 0.928 lr 2.23e-04 epoch 10/10: 18%|████▏ | 15113/85466 [20:30<28:54, 40.56it/s] step 289000 B loss 0.0004 std 0.043 cos 0.901 lr 2.23e-04 epoch 10/10: 19%|████▋ | 16613/85466 [21:10<28:38, 40.07it/s] step 289500 B loss 0.0002 std 0.043 cos 0.961 lr 2.23e-04 epoch 10/10: 21%|█████ | 18114/85466 [21:48<29:56, 37.49it/s] step 290000 B loss 0.0002 std 0.042 cos 0.949 lr 2.23e-04 epoch 10/10: 23%|█████▌ | 19613/85466 [22:26<38:11, 28.74it/s] step 290500 B loss 0.0004 std 0.042 cos 0.909 lr 2.22e-04 epoch 10/10: 25%|█████▉ | 21114/85466 [23:06<32:47, 32.71it/s] step 291000 B loss 0.0005 std 0.043 cos 0.874 lr 2.22e-04 epoch 10/10: 26%|██████▎ | 22611/85466 [23:44<31:11, 33.58it/s] step 291500 B loss 0.0002 std 0.042 cos 0.937 lr 2.22e-04 epoch 10/10: 28%|██████▊ | 24113/85466 [24:24<29:10, 35.06it/s] step 292000 B loss 0.0003 std 0.042 cos 0.914 lr 2.22e-04 epoch 10/10: 30%|███████▏ | 25610/85466 [25:03<24:59, 39.91it/s] step 292500 B loss 0.0002 std 0.043 cos 0.956 lr 2.21e-04 epoch 10/10: 32%|███████▌ | 27114/85466 [25:42<24:12, 40.17it/s] step 293000 B loss 0.0003 std 0.043 cos 0.924 lr 2.21e-04 epoch 10/10: 33%|████████ | 28610/85466 [26:21<24:40, 38.40it/s] step 293500 B loss 0.0002 std 0.042 cos 0.953 lr 2.21e-04 epoch 10/10: 35%|████████▍ | 30113/85466 [27:00<23:15, 39.66it/s] step 294000 B loss 0.0005 std 0.042 cos 0.860 lr 2.21e-04 epoch 10/10: 37%|████████▉ | 31610/85466 [27:39<22:33, 39.78it/s] step 294500 B loss 0.0002 std 0.042 cos 0.961 lr 2.21e-04 epoch 10/10: 39%|█████████▎ | 33113/85466 [28:17<20:19, 42.93it/s] step 295000 B loss 0.0002 std 0.043 cos 0.938 lr 2.20e-04 epoch 10/10: 40%|█████████▋ | 34610/85466 [28:55<19:08, 44.29it/s] step 295500 B loss 0.0003 std 0.041 cos 0.915 lr 2.20e-04 epoch 10/10: 42%|██████████▏ | 36110/85466 [29:35<21:02, 39.09it/s] step 296000 B loss 0.0003 std 0.043 cos 0.933 lr 2.20e-04 epoch 10/10: 44%|██████████▌ | 37613/85466 [30:14<19:18, 41.31it/s] step 296500 B loss 0.0003 std 0.043 cos 0.926 lr 2.20e-04 epoch 10/10: 46%|██████████▉ | 39113/85466 [30:52<20:53, 36.98it/s] step 297000 B loss 0.0004 std 0.042 cos 0.901 lr 2.19e-04 epoch 10/10: 48%|███████████▍ | 40613/85466 [31:32<20:48, 35.92it/s] step 297500 B loss 0.0003 std 0.042 cos 0.928 lr 2.19e-04 epoch 10/10: 49%|███████████▊ | 42110/85466 [32:10<15:40, 46.11it/s] step 298000 B loss 0.0002 std 0.043 cos 0.941 lr 2.19e-04 epoch 10/10: 51%|████████████▏ | 43611/85466 [32:49<19:34, 35.63it/s] step 298500 B loss 0.0002 std 0.042 cos 0.961 lr 2.19e-04 epoch 10/10: 53%|████████████▋ | 45110/85466 [33:27<14:08, 47.53it/s] step 299000 B loss 0.0004 std 0.043 cos 0.888 lr 2.18e-04 epoch 10/10: 55%|█████████████ | 46610/85466 [34:06<17:27, 37.08it/s] step 299500 B loss 0.0002 std 0.043 cos 0.942 lr 2.18e-04 epoch 10/10: 56%|█████████████▌ | 48110/85466 [34:45<13:47, 45.17it/s] step 300000 B loss 0.0001 std 0.043 cos 0.969 lr 2.18e-04 epoch 10/10: 58%|█████████████▉ | 49613/85466 [35:24<15:08, 39.48it/s] step 300500 B loss 0.0003 std 0.042 cos 0.935 lr 2.18e-04 epoch 10/10: 60%|██████████████▎ | 51113/85466 [36:03<16:01, 35.74it/s] step 301000 B loss 0.0002 std 0.043 cos 0.943 lr 2.17e-04 epoch 10/10: 62%|██████████████▊ | 52610/85466 [36:41<14:34, 37.57it/s] step 301500 B loss 0.0001 std 0.043 cos 0.971 lr 2.17e-04 epoch 10/10: 63%|███████████████▏ | 54113/85466 [37:20<13:55, 37.52it/s] step 302000 B loss 0.0002 std 0.043 cos 0.955 lr 2.17e-04 epoch 10/10: 65%|███████████████▌ | 55613/85466 [37:59<12:18, 40.43it/s] step 302500 B loss 0.0003 std 0.042 cos 0.915 lr 2.17e-04 epoch 10/10: 67%|████████████████ | 57115/85466 [38:38<11:20, 41.63it/s] step 303000 B loss 0.0002 std 0.042 cos 0.959 lr 2.16e-04 epoch 10/10: 69%|████████████████▍ | 58614/85466 [39:17<12:41, 35.26it/s] step 303500 B loss 0.0005 std 0.042 cos 0.879 lr 2.16e-04 epoch 10/10: 70%|████████████████▉ | 60112/85466 [39:55<10:19, 40.95it/s] step 304000 B loss 0.0002 std 0.043 cos 0.960 lr 2.16e-04 epoch 10/10: 72%|█████████████████▎ | 61613/85466 [40:35<10:21, 38.41it/s] step 304500 B loss 0.0003 std 0.043 cos 0.931 lr 2.16e-04 epoch 10/10: 74%|█████████████████▋ | 63110/85466 [41:13<10:28, 35.57it/s] step 305000 B loss 0.0003 std 0.042 cos 0.924 lr 2.15e-04 epoch 10/10: 76%|██████████████████▏ | 64610/85466 [41:52<08:12, 42.36it/s] step 305500 B loss 0.0003 std 0.043 cos 0.919 lr 2.15e-04 epoch 10/10: 77%|██████████████████▌ | 66113/85466 [42:33<09:22, 34.38it/s] step 306000 B loss 0.0003 std 0.043 cos 0.935 lr 2.15e-04 epoch 10/10: 79%|██████████████████▉ | 67613/85466 [43:13<08:35, 34.63it/s] step 306500 B loss 0.0005 std 0.042 cos 0.878 lr 2.15e-04 epoch 10/10: 81%|███████████████████▍ | 69110/85466 [43:52<07:02, 38.70it/s] step 307000 B loss 0.0002 std 0.042 cos 0.949 lr 2.14e-04 epoch 10/10: 83%|███████████████████▊ | 70610/85466 [44:32<06:17, 39.30it/s] step 307500 B loss 0.0002 std 0.043 cos 0.940 lr 2.14e-04 epoch 10/10: 84%|████████████████████▎ | 72114/85466 [45:11<05:39, 39.31it/s] step 308000 B loss 0.0003 std 0.043 cos 0.930 lr 2.14e-04 epoch 10/10: 86%|████████████████████▋ | 73615/85466 [45:50<05:26, 36.33it/s] step 308500 B loss 0.0002 std 0.043 cos 0.957 lr 2.14e-04 epoch 10/10: 88%|█████████████████████ | 75113/85466 [46:30<05:00, 34.42it/s] step 309000 B loss 0.0001 std 0.043 cos 0.966 lr 2.13e-04 epoch 10/10: 90%|█████████████████████▌ | 76610/85466 [47:09<05:11, 28.48it/s] step 309500 B loss 0.0002 std 0.043 cos 0.942 lr 2.13e-04 epoch 10/10: 91%|█████████████████████▉ | 78110/85466 [47:48<03:00, 40.75it/s] step 310000 B loss 0.0003 std 0.043 cos 0.921 lr 2.13e-04 epoch 10/10: 93%|██████████████████████▎ | 79613/85466 [48:29<02:30, 38.91it/s] step 310500 B loss 0.0002 std 0.043 cos 0.951 lr 2.13e-04 epoch 10/10: 95%|██████████████████████▊ | 81110/85466 [49:09<02:08, 33.99it/s] step 311000 B loss 0.0002 std 0.042 cos 0.952 lr 2.12e-04 epoch 10/10: 97%|███████████████████████▏| 82610/85466 [49:47<01:05, 43.49it/s] step 311500 B loss 0.0002 std 0.043 cos 0.958 lr 2.12e-04 epoch 10/10: 98%|███████████████████████▌| 84110/85466 [50:27<00:37, 36.26it/s] step 312000 B loss 0.0003 std 0.043 cos 0.925 lr 2.12e-04 epoch 10/10: 100%|████████████████████████| 85466/85466 [51:01<00:00, 27.92it/s] epoch 10/10 avg_loss 0.0001 3062s