3: W0820 17:12:00.367000 3475112 torch/distributed/run.py:792] 
3: W0820 17:12:00.367000 3475112 torch/distributed/run.py:792] *****************************************
3: W0820 17:12:00.367000 3475112 torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
3: W0820 17:12:00.367000 3475112 torch/distributed/run.py:792] *****************************************
1: W0820 17:12:00.368000 1669907 torch/distributed/run.py:792] 
1: W0820 17:12:00.368000 1669907 torch/distributed/run.py:792] *****************************************
1: W0820 17:12:00.368000 1669907 torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
1: W0820 17:12:00.368000 1669907 torch/distributed/run.py:792] *****************************************
2: W0820 17:12:00.368000 366223 torch/distributed/run.py:792] 
2: W0820 17:12:00.368000 366223 torch/distributed/run.py:792] *****************************************
2: W0820 17:12:00.368000 366223 torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
2: W0820 17:12:00.368000 366223 torch/distributed/run.py:792] *****************************************
0: W0820 17:12:00.385000 3534577 torch/distributed/run.py:792] 
0: W0820 17:12:00.385000 3534577 torch/distributed/run.py:792] *****************************************
0: W0820 17:12:00.385000 3534577 torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
0: W0820 17:12:00.385000 3534577 torch/distributed/run.py:792] *****************************************
0: [2025-08-20 17:12:38,142] [INFO] [axolotl.utils.schemas.validation.check_eval_packing:118] [PID:3534656] [RANK:0] explicitly setting `eval_sample_packing` to match `sample_packing`
0: [2025-08-20 17:12:38,143] [INFO] [axolotl.utils.schemas.validation.hint_sample_packing_padding:217] [PID:3534656] [RANK:0] Setting `pad_to_sequence_len: true` to prevent memory leaks when sample_packing
2: [2025-08-20 17:12:38,142] [INFO] [axolotl.utils.schemas.validation.check_eval_packing:118] [PID:366299] [RANK:0] explicitly setting `eval_sample_packing` to match `sample_packing`
1: [2025-08-20 17:12:38,143] [INFO] [axolotl.utils.schemas.validation.check_eval_packing:118] [PID:1669982] [RANK:0] explicitly setting `eval_sample_packing` to match `sample_packing`
1: [2025-08-20 17:12:38,143] [INFO] [axolotl.utils.schemas.validation.hint_sample_packing_padding:217] [PID:1669982] [RANK:0] Setting `pad_to_sequence_len: true` to prevent memory leaks when sample_packing
2: [2025-08-20 17:12:38,143] [INFO] [axolotl.utils.schemas.validation.hint_sample_packing_padding:217] [PID:366299] [RANK:0] Setting `pad_to_sequence_len: true` to prevent memory leaks when sample_packing
3: [2025-08-20 17:12:38,143] [INFO] [axolotl.utils.schemas.validation.check_eval_packing:118] [PID:3475187] [RANK:0] explicitly setting `eval_sample_packing` to match `sample_packing`
3: [2025-08-20 17:12:38,143] [INFO] [axolotl.utils.schemas.validation.hint_sample_packing_padding:217] [PID:3475187] [RANK:0] Setting `pad_to_sequence_len: true` to prevent memory leaks when sample_packing
0: [2025-08-20 17:12:43,948] [INFO] [axolotl.cli.config.load_cfg:244] [PID:3534656] [RANK:0] config:
0: {
0:   "activation_offloading": false,
0:   "auto_resume_from_checkpoints": true,
0:   "axolotl_config_path": "/lustre/fswork/projects/rech/dgo/udv55np/train/tmp/1755698958515502361.yaml",
0:   "base_model": "/lustre/fswork/projects/rech/qwv/udv55np/Qwen/Qwen2.5-7B",
0:   "base_model_config": "/lustre/fswork/projects/rech/qwv/udv55np/Qwen/Qwen2.5-7B",
0:   "batch_size": 16,
0:   "bf16": true,
0:   "capabilities": {
0:     "bf16": true,
0:     "compute_capability": "sm_90",
0:     "fp8": false,
0:     "n_gpu": 16,
0:     "n_node": 1
0:   },
0:   "chat_template": "qwen_25",
0:   "dataloader_num_workers": 16,
0:   "dataloader_pin_memory": true,
0:   "dataloader_prefetch_factor": 256,
0:   "dataset_prepared_path": "/lustre/fsn1/projects/rech/dgo/udv55np/dataset/Qwen3-235B-A22B/Qwen2.5-7B/mix_0",
0:   "dataset_processes": 192,
0:   "datasets": [
0:     {
0:       "chat_template": "tokenizer_default",
0:       "data_files": [
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0007.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0009.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0005.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0006.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0014.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0010.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0012.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0008.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0001.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0002.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0013.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0015.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0004.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0011.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0000.jsonl",
0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking/0003.jsonl"
0:       ],
0:       "ds_type": "json",
0:       "field_messages": "conversations",
0:       "message_property_mappings": {
0:         "content": "content",
0:         "role": "role"
0:       },
0:       "path": "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking",
0:       "trust_remote_code": false,
0:       "type": "chat_template"
0:     }
0:   ],
0:   "ddp": true,
0:   "deepspeed": {
0:     "bf16": {
0:       "enabled": true
0:     },
0:     "gradient_accumulation_steps": "auto",
0:     "gradient_clipping": "auto",
0:     "train_batch_size": "auto",
0:     "train_micro_batch_size_per_gpu": "auto",
0:     "wall_clock_breakdown": false,
0:     "zero_optimization": {
0:       "contiguous_gradients": true,
0:       "overlap_comm": true,
0:       "reduce_bucket_size": "auto",
0:       "stage": 3,
0:       "stage3_gather_16bit_weights_on_model_save": true,
0:       "stage3_param_persistence_threshold": "auto",
0:       "stage3_prefetch_bucket_size": "auto",
0:       "sub_group_size": 0
0:     }
0:   },
0:   "device": "cuda:0",
0:   "device_map": {
0:     "": 0
0:   },
0:   "env_capabilities": {
0:     "torch_version": "2.6.0"
0:   },
0:   "eval_batch_size": 1,
0:   "eval_causal_lm_metrics": [
0:     "sacrebleu",
0:     "comet",
0:     "ter",
0:     "chrf"
0:   ],
0:   "eval_max_new_tokens": 128,
0:   "eval_sample_packing": true,
0:   "eval_table_size": 0,
0:   "evals_per_epoch": 0,
0:   "flash_attention": true,
0:   "fp16": false,
0:   "gradient_accumulation_steps": 1,
0:   "gradient_checkpointing": true,
0:   "gradient_checkpointing_kwargs": {
0:     "use_reentrant": true
0:   },
0:   "learning_rate": 3e-06,
0:   "lisa_layers_attribute": "model.layers",
0:   "load_best_model_at_end": false,
0:   "load_in_4bit": false,
0:   "load_in_8bit": false,
0:   "local_rank": 0,
0:   "logging_steps": 10,
0:   "lora_dropout": 0.0,
0:   "loraplus_lr_embedding": 1e-06,
0:   "lr_scheduler": "warmup_stable_decay",
0:   "lr_scheduler_kwargs": {
0:     "min_lr_ratio": 0.1,
0:     "num_decay_steps": 300
0:   },
0:   "max_prompt_len": 512,
0:   "mean_resizing_embeddings": false,
0:   "micro_batch_size": 1,
0:   "model_config_type": "qwen2",
0:   "num_epochs": 1.0,
0:   "optimizer": "adamw_torch_fused",
0:   "output_dir": "/lustre/fswork/projects/rech/dgo/udv55np/ift/Qwen3-235B-A22B/Qwen2.5-7B/0",
0:   "pad_to_sequence_len": true,
0:   "pretrain_multipack_attn": true,
0:   "pretrain_multipack_buffer_size": 10000,
0:   "profiler_steps_start": 0,
0:   "qlora_sharded_model_loading": false,
0:   "ray_num_workers": 1,
0:   "resources_per_worker": {
0:     "GPU": 1
0:   },
0:   "sample_packing": true,
0:   "sample_packing_bin_size": 200,
0:   "sample_packing_group_size": 100000,
0:   "save_only_model": false,
0:   "save_safetensors": true,
0:   "save_steps": 0.2,
0:   "save_total_limit": 20,
0:   "sequence_len": 16384,
0:   "sequence_parallel_degree": 1,
0:   "shuffle_merged_datasets": true,
0:   "skip_prepare_dataset": false,
0:   "special_tokens": {
0:     "bos_token": "<|im_start|>",
0:     "eos_token": "<|im_end|>",
0:     "pad_token": "<|endoftext|>"
0:   },
0:   "strict": false,
0:   "tensor_parallel_size": 1,
0:   "tf32": false,
0:   "tiled_mlp_use_original_mlp": true,
0:   "tokenizer_config": "/lustre/fswork/projects/rech/qwv/udv55np/Qwen/Qwen2.5-7B",
0:   "torch_dtype": "torch.bfloat16",
0:   "train_on_inputs": false,
0:   "trl": {
0:     "log_completions": false,
0:     "mask_truncated_completions": false,
0:     "ref_model_mixup_alpha": 0.9,
0:     "ref_model_sync_steps": 64,
0:     "scale_rewards": true,
0:     "sync_ref_model": false,
0:     "use_vllm": false,
0:     "vllm_server_host": "0.0.0.0",
0:     "vllm_server_port": 8000
0:   },
0:   "use_ray": false,
0:   "use_tensorboard": true,
0:   "val_set_size": 0.0,
0:   "vllm": {
0:     "device": "auto",
0:     "dtype": "auto",
0:     "gpu_memory_utilization": 0.9,
0:     "host": "0.0.0.0",
0:     "port": 8000
0:   },
0:   "warmup_steps": 150,
0:   "weight_decay": 0.0,
0:   "world_size": 16
0: }
0: [2025-08-20 17:12:43,949] [INFO] [axolotl.cli.checks.check_user_token:35] [PID:3534656] [RANK:0] Skipping HuggingFace token verification because HF_HUB_OFFLINE is set to True. Only local files will be used.
3: [2025-08-20 17:12:44,243] [INFO] [axolotl.utils.data.sft._load_raw_datasets:310] [PID:3475187] [RANK:0] Loading raw datasets...
3: [2025-08-20 17:12:44,529] [INFO] [axolotl.utils.data.wrappers.get_dataset_wrapper:88] [PID:3475187] [RANK:0] Loading dataset: /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Qwen3-235B-A22B/no_thinking with base_type: chat_template and prompt_style: None
3: 
Dropping Long Sequences (>16384) (num_proc=192):   0%|          | 0/1393784 [00:00<?, ? examples/s]
Dropping Long Sequences (>16384) (num_proc=192):   0%|          | 1000/1393784 [00:01<33:21, 695.79 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):   1%|▏         | 19000/1393784 [00:01<01:22, 16762.85 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):   2%|▏         | 34000/1393784 [00:01<00:42, 31697.89 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):   4%|▎         | 52000/1393784 [00:01<00:25, 52045.13 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):   5%|▌         | 71000/1393784 [00:01<00:17, 74626.81 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):   6%|▌         | 87000/1393784 [00:01<00:14, 90093.82 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):   7%|▋         | 103000/1393784 [00:02<00:12, 101970.13 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):   9%|▊         | 120000/1393784 [00:02<00:10, 116194.07 
3: examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  10%|▉         | 137000/1393784 [00:02<00:09, 126871.50 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  11%|█         | 153000/1393784 [00:02<00:09, 133387.94 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  12%|█▏        | 170000/1393784 [00:02<00:08, 141312.82 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  15%|█▍        | 207000/1393784 [00:02<00:05, 201476.53 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  20%|█▉        | 272000/1393784 [00:02<00:03, 323979.16 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  23%|██▎       | 316000/1393784 [00:02<00:03, 351280.21 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  26%|██▋       | 368000/1393784 [00:02<00:02, 399241.10 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  36%|███▋      | 508260/1393784 [00:03<00:01, 687617.23 examples/s]
Dropping Long Sequences (>16384) (num_pr
3: oc=192):  42%|████▏     | 579040/1393784 [00:03<00:01, 532980.67 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  46%|████▌     | 639080/1393784 [00:03<00:01, 481939.44 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  50%|████▉     | 693380/1393784 [00:03<00:01, 478819.95 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  53%|█████▎    | 745460/1393784 [00:03<00:01, 443868.45 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  57%|█████▋    | 793019/1393784 [00:03<00:01, 424008.78 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  60%|██████    | 837579/1393784 [00:03<00:01, 424637.59 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  64%|██████▎   | 886139/1393784 [00:03<00:01, 440239.37 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  67%|██████▋   | 931476/1393784 [00:04<00:01, 420876.42 examples/s]
Dropping Long Sequences (>16384) (
3: num_proc=192):  70%|██████▉   | 974774/1393784 [00:04<00:01, 407744.97 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  73%|███████▎  | 1019070/1393784 [00:04<00:00, 415806.67 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  76%|███████▋  | 1063404/1393784 [00:04<00:00, 421243.00 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  80%|███████▉  | 1108478/1393784 [00:04<00:00, 427551.73 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  83%|████████▎ | 1152293/1393784 [00:04<00:00, 411849.29 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  86%|████████▌ | 1194401/1393784 [00:04<00:00, 407459.98 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  89%|████████▉ | 1238805/1393784 [00:04<00:00, 416990.28 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  92%|█████████▏| 1281208/1393784 [00:04<00:00, 3
3: 92638.65 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  95%|█████████▍| 1320870/1393784 [00:05<00:00, 393284.98 examples/s]
Dropping Long Sequences (>16384) (num_proc=192):  98%|█████████▊| 1360568/1393784 [00:05<00:00, 322809.54 examples/s]
Dropping Long Sequences (>16384) (num_proc=192): 100%|██████████| 1393784/1393784 [00:05<00:00, 234425.67 examples/s]
3: 
Drop Samples with Zero Trainable Tokens (num_proc=192):   0%|          | 0/1393229 [00:00<?, ? examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   0%|          | 1000/1393229 [00:02<47:23, 489.69 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   1%|          | 11000/1393229 [00:02<03:19, 6942.87 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   1%|▏         | 20000/1393229 [00:02<01:39, 13846.73 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   2%|▏         | 30000/1393229 [00:02<01:00, 22407.86 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   3%|▎         | 38000/1393229 [00:02<00:49, 27472.20 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   3%|▎         | 46000/1393229 [00:02<00:40, 33152.90 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   4%|▍         | 60000/1393229 [00:02<00:26, 50480.30 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192): 
3:   5%|▌         | 74000/1393229 [00:02<00:19, 67375.66 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   6%|▋         | 88000/1393229 [00:03<00:15, 81607.30 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   7%|▋         | 102000/1393229 [00:03<00:13, 94112.45 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   8%|▊         | 115000/1393229 [00:03<00:12, 100610.10 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):   9%|▉         | 131000/1393229 [00:03<00:10, 114818.67 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  10%|█         | 144000/1393229 [00:03<00:10, 114607.83 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  11%|█▏        | 157000/1393229 [00:03<00:10, 114550.47 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  12%|█▏        | 172000/1393229 [00:03<00:09, 122722.69 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  13%|█▎      
3:   | 185000/1393229 [00:03<00:09, 122659.03 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  14%|█▍        | 198000/1393229 [00:03<00:09, 124650.14 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  15%|█▌        | 214000/1393229 [00:03<00:08, 131479.65 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  16%|█▋        | 228000/1393229 [00:04<00:09, 125932.89 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  17%|█▋        | 241000/1393229 [00:04<00:09, 124469.16 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  18%|█▊        | 255000/1393229 [00:04<00:08, 128552.25 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  19%|█▉        | 269000/1393229 [00:04<00:08, 129934.58 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  24%|██▎       | 329000/1393229 [00:04<00:04, 263995.10 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  30%|███ 
3:       | 419000/1393229 [00:04<00:02, 442400.12 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  33%|███▎      | 464000/1393229 [00:04<00:02, 366599.51 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  36%|███▌      | 504000/1393229 [00:04<00:02, 351126.15 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  39%|███▉      | 541000/1393229 [00:05<00:02, 338755.19 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  41%|████▏     | 577000/1393229 [00:05<00:02, 337549.37 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  44%|████▍     | 612000/1393229 [00:05<00:02, 311719.91 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  46%|████▌     | 644000/1393229 [00:05<00:02, 309369.94 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  49%|████▉     | 680000/1393229 [00:05<00:02, 322457.83 examples/s]
Drop Samples with Zero Trainable 
3: Tokens (num_proc=192):  52%|█████▏    | 718000/1393229 [00:05<00:02, 327662.53 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  54%|█████▍    | 755000/1393229 [00:05<00:01, 336458.57 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  57%|█████▋    | 789000/1393229 [00:05<00:01, 331775.27 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  59%|█████▉    | 824000/1393229 [00:05<00:01, 334508.69 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  62%|██████▏   | 858000/1393229 [00:06<00:01, 322445.51 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  65%|██████▍   | 899028/1393229 [00:06<00:01, 347118.05 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  67%|██████▋   | 937855/1393229 [00:06<00:01, 354030.80 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  70%|██████▉   | 974425/13932
3: 29 [00:06<00:01, 351899.83 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  73%|███████▎  | 1022224/1393229 [00:06<00:00, 387292.32 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  76%|███████▌  | 1061794/1393229 [00:06<00:00, 366344.36 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  79%|███████▉  | 1099392/1393229 [00:06<00:00, 363088.13 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  82%|████████▏ | 1146246/1393229 [00:06<00:00, 388811.58 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  85%|████████▌ | 1188605/1393229 [00:06<00:00, 394967.44 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  88%|████████▊ | 1228445/1393229 [00:07<00:00, 360983.98 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  91%|█████████ | 1269261/1393229 [00:07<00:00, 373493.38 ex
3: amples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  94%|█████████▍| 1307357/1393229 [00:07<00:00, 347889.93 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  96%|█████████▋| 1342965/1393229 [00:07<00:00, 334668.32 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192):  99%|█████████▉| 1377109/1393229 [00:07<00:00, 300099.69 examples/s]
Drop Samples with Zero Trainable Tokens (num_proc=192): 100%|██████████| 1393229/1393229 [00:08<00:00, 173125.80 examples/s]
3: 
Add position_id column (Sample Packing) (num_proc=192):   0%|          | 0/1393229 [00:00<?, ? examples/s]
Add position_id column (Sample Packing) (num_proc=192):   0%|          | 1000/1393229 [00:01<46:03, 503.83 examples/s]
Add position_id column (Sample Packing) (num_proc=192):   1%|▏         | 18000/1393229 [00:02<01:56, 11843.87 examples/s]
Add position_id column (Sample Packing) (num_proc=192):   2%|▏         | 33000/1393229 [00:02<00:57, 23808.95 examples/s]
Add position_id column (Sample Packing) (num_proc=192):   3%|▎         | 45000/1393229 [00:02<00:39, 33775.36 examples/s]
Add position_id column (Sample Packing) (num_proc=192):   4%|▍         | 58000/1393229 [00:02<00:28, 46210.39 examples/s]
Add position_id column (Sample Packing) (num_proc=192):   5%|▌         | 72000/1393229 [00:02<00:21, 61035.31 examples/s]
Add position_id column (Sample Packing) (num_proc=192):   6%|▌         | 86000/1393229 [00:02<00:17, 75281.98 examples/s]
Add position_id column (Sample Packing) (num_proc=192
3: ):   7%|▋         | 99000/1393229 [00:02<00:15, 85348.87 examples/s]
Add position_id column (Sample Packing) (num_proc=192):   8%|▊         | 115000/1393229 [00:02<00:12, 100641.85 examples/s]
Add position_id column (Sample Packing) (num_proc=192):   9%|▉         | 132000/1393229 [00:02<00:10, 114965.36 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  11%|█▏        | 160000/1393229 [00:03<00:08, 152734.12 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  13%|█▎        | 187000/1393229 [00:03<00:06, 183114.78 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  16%|█▌        | 224000/1393229 [00:03<00:05, 230457.34 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  19%|█▊        | 260000/1393229 [00:03<00:04, 265070.09 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  21%|██        | 289000/1393229 [00:03<00:04, 256450.95 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  23%|
3: ██▎       | 316000/1393229 [00:03<00:04, 240001.74 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  25%|██▍       | 342000/1393229 [00:03<00:04, 240959.62 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  26%|██▋       | 367000/1393229 [00:03<00:04, 233665.40 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  28%|██▊       | 391000/1393229 [00:03<00:04, 231891.40 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  30%|██▉       | 417000/1393229 [00:04<00:04, 237145.38 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  32%|███▏      | 441000/1393229 [00:04<00:04, 235031.73 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  33%|███▎      | 465000/1393229 [00:04<00:04, 229355.46 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  35%|███▌      | 491000/1393229 [00:04<00:03, 237303.60 examples/s]
Add position_id column (Sample Packing)
3:  (num_proc=192):  37%|███▋      | 515000/1393229 [00:04<00:03, 231192.95 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  39%|███▉      | 540000/1393229 [00:04<00:03, 234115.52 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  40%|████      | 564000/1393229 [00:04<00:03, 229330.33 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  42%|████▏     | 588000/1393229 [00:04<00:03, 226990.67 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  44%|████▍     | 619000/1393229 [00:04<00:03, 249583.86 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  46%|████▋     | 645000/1393229 [00:05<00:03, 225660.10 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  48%|████▊     | 671000/1393229 [00:05<00:03, 234741.09 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  50%|████▉     | 695000/1393229 [00:05<00:03, 219056.47 examples/
3: s]
Add position_id column (Sample Packing) (num_proc=192):  52%|█████▏    | 722000/1393229 [00:05<00:02, 231746.73 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  54%|█████▎    | 746000/1393229 [00:05<00:02, 228730.32 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  55%|█████▌    | 770000/1393229 [00:05<00:02, 225365.87 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  58%|█████▊    | 802000/1393229 [00:05<00:02, 247991.41 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  59%|█████▉    | 828000/1393229 [00:05<00:02, 242459.81 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  61%|██████    | 853000/1393229 [00:05<00:02, 240301.72 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  63%|██████▎   | 878000/1393229 [00:06<00:02, 225129.97 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  65%|██
3: ████▍   | 903000/1393229 [00:06<00:02, 231295.41 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  67%|██████▋   | 928000/1393229 [00:06<00:01, 234068.21 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  68%|██████▊   | 952000/1393229 [00:06<00:01, 223951.14 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  70%|███████   | 982000/1393229 [00:06<00:01, 244937.41 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  72%|███████▏  | 1007514/1393229 [00:06<00:01, 240594.65 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  74%|███████▍  | 1032542/1393229 [00:06<00:01, 242060.90 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  76%|███████▌  | 1058341/1393229 [00:06<00:01, 241528.44 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  78%|███████▊  | 1082654/1393229 [00:06<00:0
3: 1, 230908.62 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  81%|████████  | 1123481/1393229 [00:07<00:00, 272243.44 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  83%|████████▎ | 1151079/1393229 [00:07<00:00, 255564.83 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  85%|████████▌ | 1184702/1393229 [00:07<00:00, 275271.67 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  87%|████████▋ | 1213063/1393229 [00:07<00:00, 263820.88 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  89%|████████▉ | 1239883/1393229 [00:07<00:00, 256689.29 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  91%|█████████ | 1271188/1393229 [00:07<00:00, 270558.50 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  93%|█████████▎| 1299029/1393229 [00:07<00:00, 243983.69 examples/s
3: ]
Add position_id column (Sample Packing) (num_proc=192):  95%|█████████▌| 1324613/1393229 [00:07<00:00, 238242.43 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  97%|█████████▋| 1348941/1393229 [00:07<00:00, 234087.19 examples/s]
Add position_id column (Sample Packing) (num_proc=192):  99%|█████████▊| 1372805/1393229 [00:08<00:00, 230127.21 examples/s]
Add position_id column (Sample Packing) (num_proc=192): 100%|██████████| 1393229/1393229 [00:08<00:00, 161099.31 examples/s]
3: 
Saving the dataset (0/192 shards):   0%|          | 0/1393229 [00:00<?, ? examples/s]
Saving the dataset (0/192 shards):   0%|          | 4000/1393229 [00:01<10:43, 2157.65 examples/s]
Saving the dataset (1/192 shards):   7%|▋         | 101771/1393229 [00:01<09:58, 2157.65 examples/s]
Saving the dataset (2/192 shards):   8%|▊         | 105028/1393229 [00:01<09:57, 2157.65 examples/s]
Saving the dataset (3/192 shards):   8%|▊         | 108028/1393229 [00:01<09:55, 2157.65 examples/s]
Saving the dataset (4/192 shards):   8%|▊         | 111028/1393229 [00:01<09:54, 2157.65 examples/s]
Saving the dataset (5/192 shards):  10%|▉         | 134799/1393229 [00:01<09:43, 2157.65 examples/s]
Saving the dataset (6/192 shards):  10%|▉         | 137799/1393229 [00:01<09:41, 2157.65 examples/s]
Saving the dataset (7/192 shards):  10%|▉         | 137799/1393229 [00:01<09:41, 2157.65 examples/s]
Saving the dataset (8/192 shards):  11%|█▏        | 157313/1393229 [00:01<09:32, 2157.65 examples/s]
Saving the d
3: ataset (9/192 shards):  11%|█▏        | 159313/1393229 [00:01<09:31, 2157.65 examples/s]
Saving the dataset (10/192 shards):  13%|█▎        | 177341/1393229 [00:01<09:23, 2157.65 examples/s]
Saving the dataset (11/192 shards):  13%|█▎        | 183341/1393229 [00:01<09:20, 2157.65 examples/s]
Saving the dataset (12/192 shards):  13%|█▎        | 186341/1393229 [00:01<09:19, 2157.65 examples/s]
Saving the dataset (13/192 shards):  14%|█▎        | 191341/1393229 [00:01<09:17, 2157.65 examples/s]
Saving the dataset (14/192 shards):  15%|█▌        | 213598/1393229 [00:01<09:06, 2157.65 examples/s]
Saving the dataset (15/192 shards):  17%|█▋        | 234112/1393229 [00:01<08:57, 2157.65 examples/s]
Saving the dataset (16/192 shards):  17%|█▋        | 234112/1393229 [00:01<08:57, 2157.65 examples/s]
Saving the dataset (17/192 shards):  18%|█▊        | 247626/1393229 [00:01<08:50, 2157.65 examples/s]
Saving the dataset (18/192 shards):  19%|█▊        | 259626/1393229 [00:01<08:4
3: 5, 2157.65 examples/s]
Saving the dataset (19/192 shards):  20%|██        | 283140/1393229 [00:01<08:34, 2157.65 examples/s]
Saving the dataset (20/192 shards):  21%|██        | 289654/1393229 [00:01<08:31, 2157.65 examples/s]
Saving the dataset (21/192 shards):  21%|██▏       | 299168/1393229 [00:01<08:27, 2157.65 examples/s]
Saving the dataset (22/192 shards):  22%|██▏       | 300425/1393229 [00:01<08:26, 2157.65 examples/s]
Saving the dataset (23/192 shards):  22%|██▏       | 311425/1393229 [00:01<08:21, 2157.65 examples/s]
Saving the dataset (24/192 shards):  23%|██▎       | 315425/1393229 [00:01<08:19, 2157.65 examples/s]
Saving the dataset (25/192 shards):  23%|██▎       | 317425/1393229 [00:01<08:18, 2157.65 examples/s]
Saving the dataset (26/192 shards):  24%|██▎       | 327939/1393229 [00:01<08:13, 2157.65 examples/s]
Saving the dataset (27/192 shards):  24%|██▎       | 327939/1393229 [00:01<08:13, 2157.65 examples/s]
Saving the dataset (28/192 shards
3: ):  26%|██▌       | 356196/1393229 [00:01<08:00, 2157.65 examples/s]
Saving the dataset (29/192 shards):  27%|██▋       | 376710/1393229 [00:01<07:51, 2157.65 examples/s]
Saving the dataset (30/192 shards):  29%|██▊       | 400224/1393229 [00:01<07:40, 2157.65 examples/s]
Saving the dataset (31/192 shards):  29%|██▉       | 402224/1393229 [00:01<07:39, 2157.65 examples/s]
Saving the dataset (32/192 shards):  29%|██▉       | 405481/1393229 [00:01<07:37, 2157.65 examples/s]
Saving the dataset (33/192 shards):  30%|██▉       | 415738/1393229 [00:01<07:33, 2157.65 examples/s]
Saving the dataset (34/192 shards):  30%|███       | 421995/1393229 [00:01<07:30, 2157.65 examples/s]
Saving the dataset (35/192 shards):  31%|███       | 432509/1393229 [00:01<07:25, 2157.65 examples/s]
Saving the dataset (36/192 shards):  31%|███▏      | 436509/1393229 [00:01<07:23, 2157.65 examples/s]
Saving the dataset (37/192 shards):  31%|███▏      | 436509/1393229 [00:01<
3: 07:23, 2157.65 examples/s]
Saving the dataset (38/192 shards):  34%|███▍      | 474794/1393229 [00:01<07:05, 2157.65 examples/s]
Saving the dataset (39/192 shards):  34%|███▍      | 479308/1393229 [00:01<07:03, 2157.65 examples/s]
Saving the dataset (40/192 shards):  34%|███▍      | 479308/1393229 [00:01<07:03, 2157.65 examples/s]
Saving the dataset (41/192 shards):  35%|███▍      | 481308/1393229 [00:01<07:02, 2157.65 examples/s]
Saving the dataset (42/192 shards):  35%|███▍      | 481308/1393229 [00:01<07:02, 2157.65 examples/s]
Saving the dataset (43/192 shards):  35%|███▌      | 491308/1393229 [00:01<06:58, 2157.65 examples/s]
Saving the dataset (44/192 shards):  35%|███▌      | 491308/1393229 [00:01<06:58, 2157.65 examples/s]
Saving the dataset (45/192 shards):  37%|███▋      | 514593/1393229 [00:01<06:47, 2157.65 examples/s]
Saving the dataset (46/192 shards):  37%|███▋      | 520593/1393229 [00:01<06:44, 2157.65 examples/s]
Saving 
3: the dataset (47/192 shards):  38%|███▊      | 522593/1393229 [00:01<06:43, 2157.65 examples/s]
Saving the dataset (48/192 shards):  38%|███▊      | 527850/1393229 [00:01<06:41, 2157.65 examples/s]
Saving the dataset (49/192 shards):  38%|███▊      | 527850/1393229 [00:01<06:41, 2157.65 examples/s]
Saving the dataset (50/192 shards):  39%|███▊      | 536621/1393229 [00:01<06:37, 2157.65 examples/s]
Saving the dataset (51/192 shards):  39%|███▉      | 546621/1393229 [00:01<06:32, 2157.65 examples/s]
Saving the dataset (52/192 shards):  39%|███▉      | 546878/1393229 [00:01<06:32, 2157.65 examples/s]
Saving the dataset (53/192 shards):  39%|███▉      | 549135/1393229 [00:01<06:31, 2157.65 examples/s]
Saving the dataset (54/192 shards):  40%|███▉      | 553135/1393229 [00:01<06:29, 2157.65 examples/s]
Saving the dataset (55/192 shards):  40%|███▉      | 555135/1393229 [00:01<06:28, 2157.65 examples/s]
Saving the dataset (56/192 shards):  41%|
3: ████▏     | 577649/1393229 [00:01<06:17, 2157.65 examples/s]
Saving the dataset (57/192 shards):  43%|████▎     | 595163/1393229 [00:01<06:09, 2157.65 examples/s]
Saving the dataset (58/192 shards):  43%|████▎     | 599420/1393229 [00:01<06:07, 2157.65 examples/s]
Saving the dataset (59/192 shards):  43%|████▎     | 599420/1393229 [00:01<06:07, 2157.65 examples/s]
Saving the dataset (60/192 shards):  44%|████▍     | 613934/1393229 [00:01<06:01, 2157.65 examples/s]
Saving the dataset (61/192 shards):  45%|████▍     | 620191/1393229 [00:01<05:58, 2157.65 examples/s]
Saving the dataset (62/192 shards):  45%|████▌     | 631448/1393229 [00:01<05:53, 2157.65 examples/s]
Saving the dataset (63/192 shards):  45%|████▌     | 633705/1393229 [00:01<05:52, 2157.65 examples/s]
Saving the dataset (64/192 shards):  46%|████▌     | 637219/1393229 [00:01<05:50, 2157.65 examples/s]
Saving the dataset (65/192 shards):  46%|████▌ 
3:     | 637219/1393229 [00:01<05:50, 2157.65 examples/s]
Saving the dataset (66/192 shards):  47%|████▋     | 656733/1393229 [00:01<05:41, 2157.65 examples/s]
Saving the dataset (67/192 shards):  47%|████▋     | 658733/1393229 [00:01<05:40, 2157.65 examples/s]
Saving the dataset (68/192 shards):  48%|████▊     | 663990/1393229 [00:01<05:37, 2157.65 examples/s]
Saving the dataset (69/192 shards):  48%|████▊     | 663990/1393229 [00:01<05:37, 2157.65 examples/s]
Saving the dataset (70/192 shards):  48%|████▊     | 671990/1393229 [00:01<05:34, 2157.65 examples/s]
Saving the dataset (71/192 shards):  50%|█████     | 701016/1393229 [00:01<05:20, 2157.65 examples/s]
Saving the dataset (72/192 shards):  50%|█████     | 701016/1393229 [00:01<05:20, 2157.65 examples/s]
Saving the dataset (73/192 shards):  51%|█████     | 709016/1393229 [00:01<05:17, 2157.65 examples/s]
Saving the dataset (74/192 shards):  51%|█████     | 713016/139
3: 3229 [00:01<05:15, 2157.65 examples/s]
Saving the dataset (75/192 shards):  52%|█████▏    | 728043/1393229 [00:01<05:08, 2157.65 examples/s]
Saving the dataset (76/192 shards):  52%|█████▏    | 729300/1393229 [00:01<05:07, 2157.65 examples/s]
Saving the dataset (77/192 shards):  53%|█████▎    | 732556/1393229 [00:01<05:06, 2157.65 examples/s]
Saving the dataset (78/192 shards):  53%|█████▎    | 742556/1393229 [00:01<05:01, 2157.65 examples/s]
Saving the dataset (79/192 shards):  54%|█████▍    | 750556/1393229 [00:01<04:57, 2157.65 examples/s]
Saving the dataset (80/192 shards):  54%|█████▍    | 752556/1393229 [00:01<04:56, 2157.65 examples/s]
Saving the dataset (81/192 shards):  55%|█████▍    | 763580/1393229 [00:01<04:51, 2157.65 examples/s]
Saving the dataset (82/192 shards):  55%|█████▍    | 765580/1393229 [00:01<04:50, 2157.65 examples/s]
Saving the dataset (83/192 shards):  55%|█████▌    | 770580/1
3: 393229 [00:01<04:48, 2157.65 examples/s]
Saving the dataset (84/192 shards):  55%|█████▌    | 770580/1393229 [00:01<04:48, 2157.65 examples/s]
Saving the dataset (85/192 shards):  57%|█████▋    | 787604/1393229 [00:01<04:40, 2157.65 examples/s]
Saving the dataset (86/192 shards):  57%|█████▋    | 787604/1393229 [00:01<04:40, 2157.65 examples/s]
Saving the dataset (87/192 shards):  57%|█████▋    | 789604/1393229 [00:01<04:39, 2157.65 examples/s]
Saving the dataset (88/192 shards):  57%|█████▋    | 797604/1393229 [00:01<04:36, 2157.65 examples/s]
Saving the dataset (89/192 shards):  59%|█████▉    | 824373/1393229 [00:01<04:23, 2157.65 examples/s]
Saving the dataset (90/192 shards):  59%|█████▉    | 826629/1393229 [00:01<04:22, 2157.65 examples/s]
Saving the dataset (91/192 shards):  60%|██████    | 840141/1393229 [00:01<04:16, 2157.65 examples/s]
Saving the dataset (92/192 shards):  60%|██████    | 840141
3: /1393229 [00:01<04:16, 2157.65 examples/s]
Saving the dataset (93/192 shards):  60%|██████    | 840141/1393229 [00:01<04:16, 2157.65 examples/s]
Saving the dataset (94/192 shards):  60%|██████    | 840141/1393229 [00:01<04:16, 2157.65 examples/s]
Saving the dataset (95/192 shards):  63%|██████▎   | 880421/1393229 [00:01<03:57, 2157.65 examples/s]
Saving the dataset (96/192 shards):  63%|██████▎   | 884421/1393229 [00:01<03:55, 2157.65 examples/s]
Saving the dataset (97/192 shards):  64%|██████▎   | 885677/1393229 [00:01<03:55, 2157.65 examples/s]
Saving the dataset (98/192 shards):  64%|██████▎   | 885677/1393229 [00:01<03:55, 2157.65 examples/s]
Saving the dataset (98/192 shards):  64%|██████▎   | 886933/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (99/192 shards):  64%|██████▍   | 888933/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (100/192 shards):  64%|███�
3: �██▍   | 891933/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (101/192 shards):  64%|██████▍   | 894701/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (102/192 shards):  65%|██████▍   | 898701/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (103/192 shards):  65%|██████▍   | 900701/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (104/192 shards):  65%|██████▍   | 900701/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (105/192 shards):  67%|██████▋   | 931981/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (106/192 shards):  67%|██████▋   | 937749/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (107/192 shards):  68%|██████▊   | 943749/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (108/192 shards):  68%|██████▊   | 948749/1393229 [00:01<00:00, 633505.90 examples/s]
Saving
3:  the dataset (109/192 shards):  68%|██████▊   | 948749/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (110/192 shards):  68%|██████▊   | 948749/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (111/192 shards):  68%|██████▊   | 951005/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (112/192 shards):  69%|██████▉   | 959005/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (113/192 shards):  70%|██████▉   | 972517/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (114/192 shards):  70%|███████   | 977773/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (115/192 shards):  70%|███████   | 977773/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (116/192 shards):  71%|███████   | 983029/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (117/192 shards):  71%|███████   | 985541/139
3: 3229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (118/192 shards):  71%|███████▏  | 994053/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (119/192 shards):  71%|███████▏  | 994053/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (120/192 shards):  72%|███████▏  | 999053/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (121/192 shards):  72%|███████▏  | 999053/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (122/192 shards):  73%|███████▎  | 1014821/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (123/192 shards):  73%|███████▎  | 1017077/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (124/192 shards):  74%|███████▍  | 1029333/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (125/192 shards):  74%|███████▍  | 1032589/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the 
3: dataset (126/192 shards):  74%|███████▍  | 1032845/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (127/192 shards):  74%|███████▍  | 1037845/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (128/192 shards):  74%|███████▍  | 1037845/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (129/192 shards):  75%|███████▌  | 1050101/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (130/192 shards):  78%|███████▊  | 1086613/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (131/192 shards):  78%|███████▊  | 1090125/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (132/192 shards):  79%|███████▉  | 1097637/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (133/192 shards):  79%|███████▉  | 1101893/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (134/192 shards):  80%|█████�
3: �█▉  | 1107893/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (135/192 shards):  80%|███████▉  | 1112893/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (136/192 shards):  80%|████████  | 1119405/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (137/192 shards):  81%|████████  | 1129661/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (138/192 shards):  81%|████████  | 1129661/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (139/192 shards):  81%|████████  | 1129661/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (140/192 shards):  82%|████████▏ | 1140173/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (141/192 shards):  82%|████████▏ | 1148173/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (142/192 shards):  84%|████████▍ | 1174941/1393229 [00:01<00:00, 6
3: 33505.90 examples/s]
Saving the dataset (143/192 shards):  84%|████████▍ | 1174941/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (144/192 shards):  84%|████████▍ | 1176941/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (145/192 shards):  85%|████████▍ | 1182197/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (146/192 shards):  87%|████████▋ | 1215221/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (147/192 shards):  87%|████████▋ | 1216477/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (148/192 shards):  87%|████████▋ | 1218733/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (149/192 shards):  87%|████████▋ | 1218733/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (150/192 shards):  88%|████████▊ | 1222245/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the 
3: dataset (151/192 shards):  88%|████████▊ | 1222245/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (152/192 shards):  88%|████████▊ | 1228245/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (153/192 shards):  89%|████████▊ | 1234245/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (154/192 shards):  89%|████████▉ | 1243013/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (155/192 shards):  89%|████████▉ | 1243013/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (156/192 shards):  89%|████████▉ | 1243269/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (157/192 shards):  90%|█████████ | 1259037/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (158/192 shards):  91%|█████████ | 1269549/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (159/192 shards):  91%|�
3: ��████████ | 1269549/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (160/192 shards):  92%|█████████▏| 1275549/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (161/192 shards):  92%|█████████▏| 1275549/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (162/192 shards):  92%|█████████▏| 1275805/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (163/192 shards):  92%|█████████▏| 1278061/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (164/192 shards):  92%|█████████▏| 1283317/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (165/192 shards):  92%|█████████▏| 1286317/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (166/192 shards):  94%|█████████▍| 1311085/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (167/192 shards):  94%|██████�
3: ��██▍| 1314085/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (168/192 shards):  95%|█████████▍| 1321341/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (169/192 shards):  95%|█████████▌| 1324853/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (170/192 shards):  95%|█████████▌| 1324853/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (171/192 shards):  95%|█████████▌| 1327109/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (172/192 shards):  95%|█████████▌| 1327109/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (173/192 shards):  96%|█████████▌| 1335621/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (174/192 shards):  96%|█████████▌| 1340133/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (175/192 shards):  96%|█████████▋| 134
3: 2645/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (176/192 shards):  96%|█████████▋| 1342645/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (177/192 shards):  96%|█████████▋| 1342645/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (178/192 shards):  96%|█████████▋| 1342645/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (179/192 shards):  97%|█████████▋| 1356669/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (180/192 shards):  97%|█████████▋| 1356669/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (181/192 shards):  97%|█████████▋| 1356669/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (182/192 shards):  98%|█████████▊| 1360669/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (183/192 shards):  98%|█████████▊| 1363437/1393229 [00
3: :01<00:00, 633505.90 examples/s]
Saving the dataset (184/192 shards):  98%|█████████▊| 1363437/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (185/192 shards):  98%|█████████▊| 1363437/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (186/192 shards):  98%|█████████▊| 1371693/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (187/192 shards):  98%|█████████▊| 1371949/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (188/192 shards): 100%|█████████▉| 1388717/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (189/192 shards): 100%|█████████▉| 1388717/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (190/192 shards): 100%|██████████| 1393229/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (191/192 shards): 100%|██████████| 1393229/1393229 [00:01<00:00, 63350
3: 5.90 examples/s]
Saving the dataset (192/192 shards): 100%|██████████| 1393229/1393229 [00:01<00:00, 633505.90 examples/s]
Saving the dataset (192/192 shards): 100%|██████████| 1393229/1393229 [00:02<00:00, 675379.04 examples/s]
0: [2025-08-20 17:13:34,679] [INFO] [axolotl.utils.data.shared.load_preprocessed_dataset:471] [PID:3534656] [RANK:0] Loading prepared dataset from disk at /lustre/fsn1/projects/rech/dgo/udv55np/dataset/Qwen3-235B-A22B/Qwen2.5-7B/mix_0/37fc2b771f43bc2991c54330ec6e7d7a...
0: [2025-08-20 17:14:47,518] [INFO] [axolotl.utils.samplers.multipack.calc_min_len:435] [PID:3534656] [RANK:0] gather_len_batches: [47617, 47616, 47616, 47616, 47617, 47617, 47617, 47616, 47615, 47616, 47617, 47616, 47617, 47616, 47616, 47616]
0: [2025-08-20 17:14:47,558] [INFO] [axolotl.utils.trainer.calc_sample_packing_eff_est:496] [PID:3534656] [RANK:0] sample_packing_eff_est across ranks: [0.9988111257553101, 0.9987901449203491, 0.9987901449203491, 0.9987691640853882, 0.9987901449203491, 0.9987691640853882, 0.9987691640853882, 0.9988111257553101, 0.9987691640853882, 0.9988111257553101, 0.9987691640853882, 0.9987691640853882, 0.9987691640853882, 0.9987691640853882, 0.9987901449203491, 0.9988111257553101]
0: [2025-08-20 17:14:47,573] [INFO] [axolotl.utils.data.sft._prepare_standard_dataset:123] [PID:3534656] [RANK:0] Maximum number of steps set at 2975
0: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
0: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
0: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
0: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
3: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
3: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
1: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
1: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
1: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
3: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
1: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
3: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
2: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
2: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
2: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
2: You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
1: 
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:03<00:11,  3.88s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:03<00:11,  3.88s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:03<00:11,  3.88s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:03<00:11,  3.88s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:08<00:08,  4.15s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:08<00:08,  4.15s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:08<00:08,  4.15s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:08<00:08,  4.15s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:12<00:04,  4.05s/it]
Loading checkpoint shards:
3: 
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:03<00:11,  3.86s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:03<00:11,  3.87s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:03<00:11,  3.87s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:03<00:11,  3.87s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:08<00:08,  4.14s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:08<00:08,  4.14s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:08<00:08,  4.14s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:08<00:08,  4.14s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:12<00:04,  4.05s/it]
Loading checkpoint shards:
2: 
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:03<00:11,  3.88s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:03<00:11,  3.88s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:03<00:11,  3.88s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:03<00:11,  3.88s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:08<00:08,  4.15s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:08<00:08,  4.15s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:08<00:08,  4.15s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:08<00:08,  4.15s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:12<00:04,  4.05s/it]
Loading checkpoint shards:
0: 
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:03<00:11,  3.88s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:03<00:11,  3.88s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:03<00:11,  3.88s/it]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:03<00:11,  3.98s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:08<00:08,  4.15s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:08<00:08,  4.15s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:08<00:08,  4.15s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:08<00:08,  4.22s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:12<00:04,  4.05s/it]
Loading checkpoint shards:
3:   75%|███████▌  | 3/4 [00:12<00:04,  4.05s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:12<00:04,  4.05s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:12<00:04,  4.05s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.43s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.66s/it]
0:   75%|███████▌  | 3/4 [00:12<00:04,  4.05s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:12<00:04,  4.05s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:12<00:04,  4.11s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.44s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.66s/it]
1:   75%|███████▌  | 3/4 [00:12<00:04,  4.05s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:12<00:04,  4.05s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:12<00:04,  4.05s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.44s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.66s/it]
2:   75%|███████▌  | 3/4 [00:12<00:04,  4.05s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:12<00:04,  4.05s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:12<00:04,  4.05s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.44s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.44s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.44s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.66s/it]
2: 
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.66s/it]
1: 
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.44s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.66s/it]
2: 
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.66s/it]
0: 
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.44s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.66s/it]
3: 
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.43s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.43s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.66s/it]
3: 
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.66s/it]
1: 
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.44s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.66s/it]
0: 
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.44s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.66s/it]
2: 
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.44s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.66s/it]
3: 
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.43s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.66s/it]
1: 
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.44s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00,  3.66s/it]
0: 
Loading checkpoint shards: 100%|██████████| 4/4 [00:15<00:00,  3.80s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:15<00:00,  3.92s/it]
0: [2025-08-20 17:15:04,798] [INFO] [axolotl.loaders.model._configure_embedding_dtypes:317] [PID:3534656] [RANK:0] Converting modules to torch.bfloat16
0: [2025-08-20 17:15:11,315] [INFO] [axolotl.train.save_initial_configs:397] [PID:3534656] [RANK:0] Pre-saving tokenizer to /lustre/fswork/projects/rech/dgo/udv55np/ift/Qwen3-235B-A22B/Qwen2.5-7B/0...
0: [2025-08-20 17:15:11,686] [INFO] [axolotl.train.save_initial_configs:400] [PID:3534656] [RANK:0] Pre-saving model config to /lustre/fswork/projects/rech/dgo/udv55np/ift/Qwen3-235B-A22B/Qwen2.5-7B/0...
0: [2025-08-20 17:15:11,692] [INFO] [axolotl.train.execute_training:221] [PID:3534656] [RANK:0] Starting trainer...
0: [2025-08-20 17:19:03,373] [INFO] [axolotl.utils.samplers.multipack.calc_min_len:435] [PID:3534656] [RANK:0] gather_len_batches: [47617, 47617, 47617, 47617, 47617, 47617, 47617, 47617, 47617, 47617, 47617, 47617, 47617, 47617, 47617, 47617]
0: Parameter Offload - Persistent parameters statistics: param_count = 141, numel = 333312
0: {'loss': 0.7768, 'grad_norm': 6.308251322994207, 'learning_rate': 4.62e-07, 'epoch': 0.0}
0: 
  0%|          | 0/2975 [00:00<?, ?it/s]
  0%|          | 1/2975 [03:50<190:40:48, 230.82s/it]
  0%|          | 2/2975 [03:53<79:46:53, 96.61s/it]  
  0%|          | 3/2975 [03:55<44:12:07, 53.54s/it]
  0%|          | 4/2975 [03:58<27:29:46, 33.32s/it]
  0%|          | 5/2975 [04:00<18:15:08, 22.12s/it]
  0%|          | 6/2975 [04:02<12:42:04, 15.40s/it]
  0%|          | 7/2975 [04:04<9:09:15, 11.10s/it] 
  0%|          | 8/2975 [04:07<6:50:33,  8.30s/it]
  0%|          | 9/2975 [04:09<5:17:26,  6.42s/it]
  0%|          | 10/2975 [04:11<4:14:07,  5.14s/it]
                                                   

  0%|          | 10/2975 [04:11<4:14:07,  5.14s/it]
  0%|          | 11/2975 [04:14<3:30:36,  4.26s/it]
  0%|          | 12/2975 [04:16<3:00:39,  3.66s/it]
  0%|          | 13/2975 [04:18<2:39:51,  3.24s/it]
  0%|          | 14/2975 [04:20<2:25:51,  2.96s/it]
  1%|          | 15/2975 [04:23<2:19:06,  2.82s/it]
  1%|          | 16/2975 [04:25<2:11:05,  2.66s/it]
  1%|          | 17/2975 [04:28<2:05:18,  2
0: {'loss': 0.7532, 'grad_norm': 5.83017316333207, 'learning_rate': 6.420000000000001e-07, 'epoch': 0.01}
0: {'loss': 0.6823, 'grad_norm': 3.3989663797741545, 'learning_rate': 8.220000000000001e-07, 'epoch': 0.01}
0: .54s/it]
  1%|          | 18/2975 [04:30<2:01:14,  2.46s/it]
  1%|          | 19/2975 [04:32<1:58:35,  2.41s/it]
  1%|          | 20/2975 [04:34<1:56:57,  2.37s/it]
                                                   

  1%|          | 20/2975 [04:34<1:56:57,  2.37s/it]
  1%|          | 21/2975 [04:37<1:55:35,  2.35s/it]
  1%|          | 22/2975 [04:39<1:54:42,  2.33s/it]
  1%|          | 23/2975 [04:41<1:54:03,  2.32s/it]
  1%|          | 24/2975 [04:44<1:53:17,  2.30s/it]
  1%|          | 25/2975 [04:46<1:52:57,  2.30s/it]
  1%|          | 26/2975 [04:48<1:52:34,  2.29s/it]
  1%|          | 27/2975 [04:50<1:52:27,  2.29s/it]
  1%|          | 28/2975 [04:53<1:52:07,  2.28s/it]
  1%|          | 29/2975 [04:55<1:52:01,  2.28s/it]
  1%|          | 30/2975 [04:57<1:52:00,  2.28s/it]
                                                   

  1%|          | 30/2975 [04:57<1:52:00,  2.28s/it]
  1%|          | 31/2975 [04:59<1:52:26,  2.29s/it]
  1%|          | 32/2975 [05:02<2:01:19,  2.47s/it]
  1%|          | 33/2975 
0: {'loss': 0.6256, 'grad_norm': 2.0163913620661096, 'learning_rate': 1.002e-06, 'epoch': 0.01}
0: [05:05<1:58:37,  2.42s/it]
  1%|          | 34/2975 [05:07<1:56:23,  2.37s/it]
  1%|          | 35/2975 [05:09<1:55:05,  2.35s/it]
  1%|          | 36/2975 [05:12<1:53:52,  2.32s/it]
  1%|          | 37/2975 [05:14<1:53:27,  2.32s/it]
  1%|▏         | 38/2975 [05:16<1:55:29,  2.36s/it]
  1%|▏         | 39/2975 [05:19<1:55:23,  2.36s/it]
  1%|▏         | 40/2975 [05:21<1:57:17,  2.40s/it]
                                                   

  1%|▏         | 40/2975 [05:21<1:57:17,  2.40s/it]
  1%|▏         | 41/2975 [05:23<1:55:57,  2.37s/it]
  1%|▏         | 42/2975 [05:26<1:54:49,  2.35s/it]
  1%|▏         | 43/2975 [05:28<1:53:47,  2.33s/it]
  1%|▏         | 44/2975 [05:30<1:53:21,  2.32s/it]
  2%|▏         | 45/2975 [05:33<1:52:53,  2.31s/it]
  2%|▏         | 46/2975 [05:35<1:53:21,  2.32s/it]
  2%|▏         | 47/2975 [05:37<1:52:34,  2.31s/it]
  2%|▏         | 48/2975 [05:39<1:52:16,  2.30s/it]
  2%|▏         | 49/2975 [05:42<1:51:53,  2.29s/it]
  2%|▏         | 50/2975 [05:44<
0: {'loss': 0.599, 'grad_norm': 1.2114875762010269, 'learning_rate': 1.182e-06, 'epoch': 0.02}
0: {'loss': 0.5547, 'grad_norm': 1.050559421856475, 'learning_rate': 1.3619999999999998e-06, 'epoch': 0.02}
0: 1:52:18,  2.30s/it]
                                                   

  2%|▏         | 50/2975 [05:44<1:52:18,  2.30s/it]
  2%|▏         | 51/2975 [05:46<1:52:02,  2.30s/it]
  2%|▏         | 52/2975 [05:49<1:51:53,  2.30s/it]
  2%|▏         | 53/2975 [05:51<1:51:40,  2.29s/it]
  2%|▏         | 54/2975 [05:53<1:51:07,  2.28s/it]
  2%|▏         | 55/2975 [05:56<1:51:07,  2.28s/it]
  2%|▏         | 56/2975 [05:58<1:51:12,  2.29s/it]
  2%|▏         | 57/2975 [06:00<1:51:45,  2.30s/it]
  2%|▏         | 58/2975 [06:02<1:51:30,  2.29s/it]
  2%|▏         | 59/2975 [06:05<1:51:19,  2.29s/it]
  2%|▏         | 60/2975 [06:07<1:51:06,  2.29s/it]
                                                   

  2%|▏         | 60/2975 [06:07<1:51:06,  2.29s/it]
  2%|▏         | 61/2975 [06:09<1:51:36,  2.30s/it]
  2%|▏         | 62/2975 [06:12<1:52:13,  2.31s/it]
  2%|▏         | 63/2975 [06:14<1:51:48,  2.30s/it]
  2%|▏         | 64/2975 [06:16<1:51:14,  2.29s/it]
  2%|▏         | 65/2975 [06:18<
0: {'loss': 0.5385, 'grad_norm': 0.8517727468506996, 'learning_rate': 1.542e-06, 'epoch': 0.02}
0: {'loss': 0.5151, 'grad_norm': 0.8013494726361342, 'learning_rate': 1.722e-06, 'epoch': 0.03}
0: 1:50:55,  2.29s/it]
  2%|▏         | 66/2975 [06:21<1:50:57,  2.29s/it]
  2%|▏         | 67/2975 [06:23<1:50:57,  2.29s/it]
  2%|▏         | 68/2975 [06:25<1:50:39,  2.28s/it]
  2%|▏         | 69/2975 [06:28<1:50:44,  2.29s/it]
  2%|▏         | 70/2975 [06:30<1:50:36,  2.28s/it]
                                                   

  2%|▏         | 70/2975 [06:30<1:50:36,  2.28s/it]
  2%|▏         | 71/2975 [06:32<1:50:31,  2.28s/it]
  2%|▏         | 72/2975 [06:34<1:50:42,  2.29s/it]
  2%|▏         | 73/2975 [06:37<1:51:03,  2.30s/it]
  2%|▏         | 74/2975 [06:39<1:51:19,  2.30s/it]
  3%|▎         | 75/2975 [06:41<1:51:06,  2.30s/it]
  3%|▎         | 76/2975 [06:44<1:50:32,  2.29s/it]
  3%|▎         | 77/2975 [06:46<1:50:20,  2.28s/it]
  3%|▎         | 78/2975 [06:48<1:50:13,  2.28s/it]
  3%|▎         | 79/2975 [06:51<1:52:42,  2.34s/it]
  3%|▎         | 80/2975 [06:53<1:52:14,  2.33s/it]
                                                   

  3%|▎         | 80/2975 [06:53<
0: {'loss': 0.516, 'grad_norm': 0.8645615452486105, 'learning_rate': 1.9020000000000002e-06, 'epoch': 0.03}
0: 1:52:14,  2.33s/it]
  3%|▎         | 81/2975 [06:55<1:51:26,  2.31s/it]
  3%|▎         | 82/2975 [06:58<1:51:03,  2.30s/it]
  3%|▎         | 83/2975 [07:00<1:50:39,  2.30s/it]
  3%|▎         | 84/2975 [07:02<1:50:24,  2.29s/it]
  3%|▎         | 85/2975 [07:04<1:50:23,  2.29s/it]
  3%|▎         | 86/2975 [07:07<1:50:20,  2.29s/it]
  3%|▎         | 87/2975 [07:09<1:50:05,  2.29s/it]
  3%|▎         | 88/2975 [07:11<1:53:44,  2.36s/it]
  3%|▎         | 89/2975 [07:14<1:52:23,  2.34s/it]
  3%|▎         | 90/2975 [07:16<1:51:31,  2.32s/it]
                                                   

  3%|▎         | 90/2975 [07:16<1:51:31,  2.32s/it]
  3%|▎         | 91/2975 [07:18<1:51:07,  2.31s/it]
  3%|▎         | 92/2975 [07:21<1:51:03,  2.31s/it]
  3%|▎         | 93/2975 [07:23<1:50:54,  2.31s/it]
  3%|▎         | 94/2975 [07:25<1:51:35,  2.32s/it]
  3%|▎         | 95/2975 [07:28<1:50:59,  2.31s/it]
  3%|▎         | 96/2975 [07:30<1:50:28,  2.30s/it]
  3%|▎         | 97/2975 [07:32
0: {'loss': 0.5083, 'grad_norm': 0.8494867414135814, 'learning_rate': 2.082e-06, 'epoch': 0.03}
0: {'loss': 0.5047, 'grad_norm': 0.8330517914588346, 'learning_rate': 2.262e-06, 'epoch': 0.04}
0: <1:50:37,  2.31s/it]
  3%|▎         | 98/2975 [07:35<1:51:30,  2.33s/it]
  3%|▎         | 99/2975 [07:37<1:50:51,  2.31s/it]
  3%|▎         | 100/2975 [07:39<1:50:35,  2.31s/it]
                                                    

  3%|▎         | 100/2975 [07:39<1:50:35,  2.31s/it]
  3%|▎         | 101/2975 [07:41<1:51:05,  2.32s/it]
  3%|▎         | 102/2975 [07:44<1:50:27,  2.31s/it]
  3%|▎         | 103/2975 [07:46<1:50:09,  2.30s/it]
  3%|▎         | 104/2975 [07:48<1:49:43,  2.29s/it]
  4%|▎         | 105/2975 [07:51<1:49:53,  2.30s/it]
  4%|▎         | 106/2975 [07:53<1:49:59,  2.30s/it]
  4%|▎         | 107/2975 [07:55<1:49:44,  2.30s/it]
  4%|▎         | 108/2975 [07:58<1:49:36,  2.29s/it]
  4%|▎         | 109/2975 [08:00<1:49:19,  2.29s/it]
  4%|▎         | 110/2975 [08:02<1:49:11,  2.29s/it]
                                                    

  4%|▎         | 110/2975 [08:02<1:49:11,  2.29s/it]
  4%|▎         | 111/2975 [08:04<1:49:20,  2.29s/it]
  4%|▍         
0: {'loss': 0.5012, 'grad_norm': 0.8279500959732156, 'learning_rate': 2.442e-06, 'epoch': 0.04}
0: | 112/2975 [08:07<1:49:10,  2.29s/it]
  4%|▍         | 113/2975 [08:09<1:48:59,  2.29s/it]
  4%|▍         | 114/2975 [08:11<1:52:03,  2.35s/it]
  4%|▍         | 115/2975 [08:14<1:50:42,  2.32s/it]
  4%|▍         | 116/2975 [08:16<1:50:03,  2.31s/it]
  4%|▍         | 117/2975 [08:18<1:50:09,  2.31s/it]
  4%|▍         | 118/2975 [08:21<1:49:52,  2.31s/it]
  4%|▍         | 119/2975 [08:23<1:49:19,  2.30s/it]
  4%|▍         | 120/2975 [08:25<1:49:01,  2.29s/it]
                                                    

  4%|▍         | 120/2975 [08:25<1:49:01,  2.29s/it]
  4%|▍         | 121/2975 [08:27<1:48:42,  2.29s/it]
  4%|▍         | 122/2975 [08:30<1:48:40,  2.29s/it]
  4%|▍         | 123/2975 [08:32<1:48:43,  2.29s/it]
  4%|▍         | 124/2975 [08:34<1:48:44,  2.29s/it]
  4%|▍         | 125/2975 [08:37<1:49:00,  2.29s/it]
  4%|▍         | 126/2975 [08:39<1:48:48,  2.29s/it]
  4%|▍         | 127/2975 [08:41<1:48:28,  2.29s/it]
  4%|▍         | 128/2975 [08:44<1:58:51,  2.51s/i
0: {'loss': 0.4874, 'grad_norm': 0.7992561766710217, 'learning_rate': 2.622e-06, 'epoch': 0.04}
0: {'loss': 0.4813, 'grad_norm': 0.7225169262352843, 'learning_rate': 2.802e-06, 'epoch': 0.05}
0: t]
  4%|▍         | 129/2975 [08:46<1:55:35,  2.44s/it]
  4%|▍         | 130/2975 [08:49<1:56:08,  2.45s/it]
                                                    

  4%|▍         | 130/2975 [08:49<1:56:08,  2.45s/it]
  4%|▍         | 131/2975 [08:51<1:54:19,  2.41s/it]
  4%|▍         | 132/2975 [08:54<1:52:34,  2.38s/it]
  4%|▍         | 133/2975 [08:56<1:55:44,  2.44s/it]
  5%|▍         | 134/2975 [08:59<1:54:37,  2.42s/it]
  5%|▍         | 135/2975 [09:01<1:52:32,  2.38s/it]
  5%|▍         | 136/2975 [09:04<2:00:39,  2.55s/it]
  5%|▍         | 137/2975 [09:06<1:56:56,  2.47s/it]
  5%|▍         | 138/2975 [09:09<1:58:26,  2.50s/it]
  5%|▍         | 139/2975 [09:11<1:56:31,  2.47s/it]
  5%|▍         | 140/2975 [09:13<1:53:45,  2.41s/it]
                                                    

  5%|▍         | 140/2975 [09:13<1:53:45,  2.41s/it]
  5%|▍         | 141/2975 [09:16<1:52:01,  2.37s/it]
  5%|▍         | 142/2975 [09:18<1:50:37,  2.34s/it]
  5%|▍         | 143/2975 [09:2
0: {'loss': 0.4873, 'grad_norm': 0.732286582435689, 'learning_rate': 2.982e-06, 'epoch': 0.05}
0: 0<1:51:03,  2.35s/it]
  5%|▍         | 144/2975 [09:22<1:49:55,  2.33s/it]
  5%|▍         | 145/2975 [09:25<1:49:09,  2.31s/it]
  5%|▍         | 146/2975 [09:27<1:48:39,  2.30s/it]
  5%|▍         | 147/2975 [09:29<1:48:33,  2.30s/it]
  5%|▍         | 148/2975 [09:32<1:49:52,  2.33s/it]
  5%|▌         | 149/2975 [09:34<1:49:11,  2.32s/it]
  5%|▌         | 150/2975 [09:36<1:49:33,  2.33s/it]
                                                    

  5%|▌         | 150/2975 [09:36<1:49:33,  2.33s/it]
  5%|▌         | 151/2975 [09:39<1:49:22,  2.32s/it]
  5%|▌         | 152/2975 [09:41<1:48:38,  2.31s/it]
  5%|▌         | 153/2975 [09:43<1:48:06,  2.30s/it]
  5%|▌         | 154/2975 [09:46<1:48:08,  2.30s/it]
  5%|▌         | 155/2975 [09:48<1:47:43,  2.29s/it]
  5%|▌         | 156/2975 [09:50<1:47:51,  2.30s/it]
  5%|▌         | 157/2975 [09:53<1:49:23,  2.33s/it]
  5%|▌         | 158/2975 [09:55<1:49:03,  2.32s/it]
  5%|▌         | 159/2975 [09:57<1:48:24,  2.31s/it]
  5%|▌     
0: {'loss': 0.4788, 'grad_norm': 0.6583918593540916, 'learning_rate': 3e-06, 'epoch': 0.05}
0: {'loss': 0.4848, 'grad_norm': 0.6077976602686953, 'learning_rate': 3e-06, 'epoch': 0.06}
0:     | 160/2975 [09:59<1:48:30,  2.31s/it]
                                                    

  5%|▌         | 160/2975 [09:59<1:48:30,  2.31s/it]
  5%|▌         | 161/2975 [10:02<1:48:12,  2.31s/it]
  5%|▌         | 162/2975 [10:04<1:47:28,  2.29s/it]
  5%|▌         | 163/2975 [10:06<1:47:09,  2.29s/it]
  6%|▌         | 164/2975 [10:09<1:47:02,  2.28s/it]
  6%|▌         | 165/2975 [10:11<1:46:50,  2.28s/it]
  6%|▌         | 166/2975 [10:13<1:46:51,  2.28s/it]
  6%|▌         | 167/2975 [10:15<1:46:55,  2.28s/it]
  6%|▌         | 168/2975 [10:18<1:47:01,  2.29s/it]
  6%|▌         | 169/2975 [10:20<1:47:11,  2.29s/it]
  6%|▌         | 170/2975 [10:22<1:47:22,  2.30s/it]
                                                    

  6%|▌         | 170/2975 [10:22<1:47:22,  2.30s/it]
  6%|▌         | 171/2975 [10:25<1:47:21,  2.30s/it]
  6%|▌         | 172/2975 [10:27<1:47:05,  2.29s/it]
  6%|▌         | 173/2975 [10:29<1:48:07,  2.32s/it]
  6%|▌         | 174/2975 [10:32<1:47:44,  2.31
0: {'loss': 0.4672, 'grad_norm': 0.5307576082197036, 'learning_rate': 3e-06, 'epoch': 0.06}
0: s/it]
  6%|▌         | 175/2975 [10:34<1:47:29,  2.30s/it]
  6%|▌         | 176/2975 [10:36<1:49:28,  2.35s/it]
  6%|▌         | 177/2975 [10:39<1:48:30,  2.33s/it]
  6%|▌         | 178/2975 [10:41<1:48:01,  2.32s/it]
  6%|▌         | 179/2975 [10:43<1:47:41,  2.31s/it]
  6%|▌         | 180/2975 [10:45<1:46:53,  2.29s/it]
                                                    

  6%|▌         | 180/2975 [10:45<1:46:53,  2.29s/it]
  6%|▌         | 181/2975 [10:48<1:46:46,  2.29s/it]
  6%|▌         | 182/2975 [10:50<1:46:34,  2.29s/it]
  6%|▌         | 183/2975 [10:52<1:46:35,  2.29s/it]
  6%|▌         | 184/2975 [10:55<1:47:05,  2.30s/it]
  6%|▌         | 185/2975 [10:57<1:46:55,  2.30s/it]
  6%|▋         | 186/2975 [10:59<1:46:38,  2.29s/it]
  6%|▋         | 187/2975 [11:01<1:46:33,  2.29s/it]
  6%|▋         | 188/2975 [11:04<1:46:25,  2.29s/it]
  6%|▋         | 189/2975 [11:06<1:46:15,  2.29s/it]
  6%|▋         | 190/2975 [11:08<1:46:30,  2.29s/it]
                             
0: {'loss': 0.4718, 'grad_norm': 0.5207150873198968, 'learning_rate': 3e-06, 'epoch': 0.06}
0: {'loss': 0.4596, 'grad_norm': 0.49237911213075464, 'learning_rate': 3e-06, 'epoch': 0.07}
0:                        

  6%|▋         | 190/2975 [11:08<1:46:30,  2.29s/it]
  6%|▋         | 191/2975 [11:11<1:46:28,  2.29s/it]
  6%|▋         | 192/2975 [11:13<1:46:13,  2.29s/it]
  6%|▋         | 193/2975 [11:15<1:46:22,  2.29s/it]
  7%|▋         | 194/2975 [11:17<1:46:18,  2.29s/it]
  7%|▋         | 195/2975 [11:20<1:46:09,  2.29s/it]
  7%|▋         | 196/2975 [11:22<1:46:00,  2.29s/it]
  7%|▋         | 197/2975 [11:24<1:45:51,  2.29s/it]
  7%|▋         | 198/2975 [11:27<1:45:40,  2.28s/it]
  7%|▋         | 199/2975 [11:29<1:45:44,  2.29s/it]
  7%|▋         | 200/2975 [11:31<1:46:16,  2.30s/it]
                                                    

  7%|▋         | 200/2975 [11:31<1:46:16,  2.30s/it]
  7%|▋         | 201/2975 [11:34<1:47:37,  2.33s/it]
  7%|▋         | 202/2975 [11:36<1:46:58,  2.31s/it]
  7%|▋         | 203/2975 [11:38<1:46:24,  2.30s/it]
  7%|▋         | 204/2975 [11:40<1:46:14,  2.30s/it]
  7%|▋         | 205/2975 [11:43<1:46:26,  2.31s/it]
  7%|▋  
0: {'loss': 0.4712, 'grad_norm': 0.5064764361486164, 'learning_rate': 3e-06, 'epoch': 0.07}
0: {'loss': 0.4722, 'grad_norm': 0.5157321664880986, 'learning_rate': 3e-06, 'epoch': 0.07}
0:        | 206/2975 [11:45<1:45:56,  2.30s/it]
  7%|▋         | 207/2975 [11:47<1:45:37,  2.29s/it]
  7%|▋         | 208/2975 [11:50<1:45:29,  2.29s/it]
  7%|▋         | 209/2975 [11:52<1:45:16,  2.28s/it]
  7%|▋         | 210/2975 [11:54<1:45:23,  2.29s/it]
                                                    

  7%|▋         | 210/2975 [11:54<1:45:23,  2.29s/it]
  7%|▋         | 211/2975 [11:57<1:45:42,  2.29s/it]
  7%|▋         | 212/2975 [11:59<1:45:31,  2.29s/it]
  7%|▋         | 213/2975 [12:01<1:46:24,  2.31s/it]
  7%|▋         | 214/2975 [12:04<1:48:28,  2.36s/it]
  7%|▋         | 215/2975 [12:06<1:47:08,  2.33s/it]
  7%|▋         | 216/2975 [12:08<1:46:17,  2.31s/it]
  7%|▋         | 217/2975 [12:10<1:45:52,  2.30s/it]
  7%|▋         | 218/2975 [12:13<1:45:34,  2.30s/it]
  7%|▋         | 219/2975 [12:15<1:46:33,  2.32s/it]
  7%|▋         | 220/2975 [12:17<1:46:10,  2.31s/it]
                                                    

  7%|▋         | 220/2975 [12:17<1:46:10,  2
0: {'loss': 0.4673, 'grad_norm': 0.5424118604532722, 'learning_rate': 3e-06, 'epoch': 0.08}
0: .31s/it]
  7%|▋         | 221/2975 [12:20<1:45:51,  2.31s/it]
  7%|▋         | 222/2975 [12:22<1:45:39,  2.30s/it]
  7%|▋         | 223/2975 [12:24<1:45:21,  2.30s/it]
  8%|▊         | 224/2975 [12:27<1:45:33,  2.30s/it]
  8%|▊         | 225/2975 [12:29<1:45:15,  2.30s/it]
  8%|▊         | 226/2975 [12:32<1:55:40,  2.52s/it]
  8%|▊         | 227/2975 [12:35<1:57:00,  2.55s/it]
  8%|▊         | 228/2975 [12:37<1:53:24,  2.48s/it]
  8%|▊         | 229/2975 [12:39<1:50:36,  2.42s/it]
  8%|▊         | 230/2975 [12:42<1:54:28,  2.50s/it]
                                                    

  8%|▊         | 230/2975 [12:42<1:54:28,  2.50s/it]
  8%|▊         | 231/2975 [12:44<1:51:35,  2.44s/it]
  8%|▊         | 232/2975 [12:47<1:52:20,  2.46s/it]
  8%|▊         | 233/2975 [12:49<1:50:06,  2.41s/it]
  8%|▊         | 234/2975 [12:51<1:48:16,  2.37s/it]
  8%|▊         | 235/2975 [12:53<1:47:09,  2.35s/it]
  8%|▊         | 236/2975 [12:56<1:46:06,  2.32s/it]
  8%|▊         | 237/297
0: {'loss': 0.4676, 'grad_norm': 0.526643671926225, 'learning_rate': 3e-06, 'epoch': 0.08}
0: {'loss': 0.4691, 'grad_norm': 0.508126293526943, 'learning_rate': 3e-06, 'epoch': 0.08}
0: 5 [12:59<1:54:13,  2.50s/it]
  8%|▊         | 238/2975 [13:01<1:51:12,  2.44s/it]
  8%|▊         | 239/2975 [13:03<1:49:15,  2.40s/it]
  8%|▊         | 240/2975 [13:06<1:47:58,  2.37s/it]
                                                    

  8%|▊         | 240/2975 [13:06<1:47:58,  2.37s/it]
  8%|▊         | 241/2975 [13:08<1:46:57,  2.35s/it]
  8%|▊         | 242/2975 [13:10<1:46:25,  2.34s/it]
  8%|▊         | 243/2975 [13:12<1:45:48,  2.32s/it]
  8%|▊         | 244/2975 [13:15<1:45:10,  2.31s/it]
  8%|▊         | 245/2975 [13:17<1:44:55,  2.31s/it]
  8%|▊         | 246/2975 [13:19<1:44:29,  2.30s/it]
  8%|▊         | 247/2975 [13:22<1:44:19,  2.29s/it]
  8%|▊         | 248/2975 [13:24<1:44:07,  2.29s/it]
  8%|▊         | 249/2975 [13:26<1:43:53,  2.29s/it]
  8%|▊         | 250/2975 [13:29<1:44:46,  2.31s/it]
                                                    

  8%|▊         | 250/2975 [13:29<1:44:46,  2.31s/it]
  8%|▊         | 251/2975 [13:31<1:46:06,  2.34s/it]
  8%|�
0: {'loss': 0.4728, 'grad_norm': 0.4966204597578199, 'learning_rate': 3e-06, 'epoch': 0.09}
0: �         | 252/2975 [13:33<1:45:29,  2.32s/it]
  9%|▊         | 253/2975 [13:36<1:45:02,  2.32s/it]
  9%|▊         | 254/2975 [13:38<1:44:28,  2.30s/it]
  9%|▊         | 255/2975 [13:40<1:44:43,  2.31s/it]
  9%|▊         | 256/2975 [13:42<1:44:22,  2.30s/it]
  9%|▊         | 257/2975 [13:45<1:46:02,  2.34s/it]
  9%|▊         | 258/2975 [13:47<1:45:29,  2.33s/it]
  9%|▊         | 259/2975 [13:49<1:44:58,  2.32s/it]
  9%|▊         | 260/2975 [13:52<1:44:29,  2.31s/it]
                                                    

  9%|▊         | 260/2975 [13:52<1:44:29,  2.31s/it]
  9%|▉         | 261/2975 [13:54<1:46:48,  2.36s/it]
  9%|▉         | 262/2975 [13:56<1:45:50,  2.34s/it]
  9%|▉         | 263/2975 [13:59<1:45:27,  2.33s/it]
  9%|▉         | 264/2975 [14:01<1:47:00,  2.37s/it]
  9%|▉         | 265/2975 [14:04<1:47:17,  2.38s/it]
  9%|▉         | 266/2975 [14:06<1:46:15,  2.35s/it]
  9%|▉         | 267/2975 [14:08<1:46:22,  2.36s/it]
  9%|▉         | 268/2975 [14:11<1:45:31
0: {'loss': 0.4695, 'grad_norm': 0.6349683843325821, 'learning_rate': 3e-06, 'epoch': 0.09}
0: {'loss': 0.4647, 'grad_norm': 0.46700898978489686, 'learning_rate': 3e-06, 'epoch': 0.09}
0: ,  2.34s/it]
  9%|▉         | 269/2975 [14:13<1:44:40,  2.32s/it]
  9%|▉         | 270/2975 [14:16<1:54:14,  2.53s/it]
                                                    

  9%|▉         | 270/2975 [14:16<1:54:14,  2.53s/it]
  9%|▉         | 271/2975 [14:18<1:50:39,  2.46s/it]
  9%|▉         | 272/2975 [14:20<1:48:14,  2.40s/it]
  9%|▉         | 273/2975 [14:23<1:46:54,  2.37s/it]
  9%|▉         | 274/2975 [14:25<1:45:44,  2.35s/it]
  9%|▉         | 275/2975 [14:27<1:45:03,  2.33s/it]
  9%|▉         | 276/2975 [14:30<1:44:18,  2.32s/it]
  9%|▉         | 277/2975 [14:32<1:43:41,  2.31s/it]
  9%|▉         | 278/2975 [14:34<1:43:36,  2.31s/it]
  9%|▉         | 279/2975 [14:37<1:48:04,  2.41s/it]
  9%|▉         | 280/2975 [14:39<1:49:11,  2.43s/it]
                                                    

  9%|▉         | 280/2975 [14:39<1:49:11,  2.43s/it]
  9%|▉         | 281/2975 [14:42<1:49:23,  2.44s/it]
  9%|▉         | 282/2975 [14:44<1:47:22,  2.39s/it]
 10%|▉         | 283/
0: {'loss': 0.4715, 'grad_norm': 0.5029127429084231, 'learning_rate': 3e-06, 'epoch': 0.1}
0: 2975 [14:46<1:46:16,  2.37s/it]
 10%|▉         | 284/2975 [14:49<1:51:08,  2.48s/it]
 10%|▉         | 285/2975 [14:51<1:48:39,  2.42s/it]
 10%|▉         | 286/2975 [14:54<1:46:45,  2.38s/it]
 10%|▉         | 287/2975 [14:56<1:45:13,  2.35s/it]
 10%|▉         | 288/2975 [14:58<1:44:17,  2.33s/it]
 10%|▉         | 289/2975 [15:01<1:43:46,  2.32s/it]
 10%|▉         | 290/2975 [15:03<1:43:32,  2.31s/it]
                                                    

 10%|▉         | 290/2975 [15:03<1:43:32,  2.31s/it]
 10%|▉         | 291/2975 [15:06<1:48:09,  2.42s/it]
 10%|▉         | 292/2975 [15:08<1:55:17,  2.58s/it]
 10%|▉         | 293/2975 [15:11<1:51:35,  2.50s/it]
 10%|▉         | 294/2975 [15:13<1:48:36,  2.43s/it]
 10%|▉         | 295/2975 [15:15<1:46:43,  2.39s/it]
 10%|▉         | 296/2975 [15:18<1:45:15,  2.36s/it]
 10%|▉         | 297/2975 [15:20<1:44:12,  2.33s/it]
 10%|█         | 298/2975 [15:22<1:43:17,  2.32s/it]
 10%|█         | 299/2975 [15:25<1:43:13,  2.31s/it]
 10
0: {'loss': 0.4595, 'grad_norm': 0.4964938220971449, 'learning_rate': 3e-06, 'epoch': 0.1}
0: {'loss': 0.4612, 'grad_norm': 0.5043358306340271, 'learning_rate': 3e-06, 'epoch': 0.1}
0: %|█         | 300/2975 [15:27<1:42:39,  2.30s/it]
                                                    

 10%|█         | 300/2975 [15:27<1:42:39,  2.30s/it]
 10%|█         | 301/2975 [15:29<1:44:28,  2.34s/it]
 10%|█         | 302/2975 [15:32<1:43:43,  2.33s/it]
 10%|█         | 303/2975 [15:34<1:44:16,  2.34s/it]
 10%|█         | 304/2975 [15:36<1:43:31,  2.33s/it]
 10%|█         | 305/2975 [15:39<1:44:22,  2.35s/it]
 10%|█         | 306/2975 [15:41<1:43:45,  2.33s/it]
 10%|█         | 307/2975 [15:43<1:44:40,  2.35s/it]
 10%|█         | 308/2975 [15:46<1:43:38,  2.33s/it]
 10%|█         | 309/2975 [15:48<1:43:06,  2.32s/it]
 10%|█         | 310/2975 [15:50<1:42:31,  2.31s/it]
                                                    

 10%|█         | 310/2975 [15:50<1:42:31,  2.31s/it]
 10%|█         | 311/2975 [15:53<1:47:54,  2.43s/it]
 10%|█         | 312/2975 [15:55<1:46:36,  2.40s/it]
 11%|█         | 313/2975 [15:57<1:44:51,  2.36s/it]
 11%|█         | 314/2975 [16:00<1:43
0: {'loss': 0.4647, 'grad_norm': 0.47909595165748897, 'learning_rate': 3e-06, 'epoch': 0.11}
0: :48,  2.34s/it]
 11%|█         | 315/2975 [16:02<1:43:25,  2.33s/it]
 11%|█         | 316/2975 [16:04<1:42:40,  2.32s/it]
 11%|█         | 317/2975 [16:07<1:41:50,  2.30s/it]
 11%|█         | 318/2975 [16:09<1:41:32,  2.29s/it]
 11%|█         | 319/2975 [16:12<1:50:18,  2.49s/it]
 11%|█         | 320/2975 [16:14<1:48:04,  2.44s/it]
                                                    

 11%|█         | 320/2975 [16:14<1:48:04,  2.44s/it]
 11%|█         | 321/2975 [16:16<1:46:09,  2.40s/it]
 11%|█         | 322/2975 [16:19<1:45:09,  2.38s/it]
 11%|█         | 323/2975 [16:21<1:44:14,  2.36s/it]
 11%|█         | 324/2975 [16:23<1:43:02,  2.33s/it]
 11%|█         | 325/2975 [16:26<1:43:32,  2.34s/it]
 11%|█         | 326/2975 [16:28<1:42:48,  2.33s/it]
 11%|█         | 327/2975 [16:30<1:42:14,  2.32s/it]
 11%|█         | 328/2975 [16:33<1:41:59,  2.31s/it]
 11%|█         | 329/2975 [16:35<1:41:40,  2.31s/it]
 11%|█         | 330/2975 [16:37<1:41:27,  2.30s/it]
                   
0: {'loss': 0.4603, 'grad_norm': 0.5103808924234785, 'learning_rate': 3e-06, 'epoch': 0.11}
0: {'loss': 0.4647, 'grad_norm': 0.5127297948797209, 'learning_rate': 3e-06, 'epoch': 0.11}
0:                                  

 11%|█         | 330/2975 [16:37<1:41:27,  2.30s/it]
 11%|█         | 331/2975 [16:40<1:41:19,  2.30s/it]
 11%|█         | 332/2975 [16:42<1:41:27,  2.30s/it]
 11%|█         | 333/2975 [16:44<1:41:32,  2.31s/it]
 11%|█         | 334/2975 [16:46<1:41:19,  2.30s/it]
 11%|█▏        | 335/2975 [16:49<1:43:57,  2.36s/it]
 11%|█▏        | 336/2975 [16:51<1:42:38,  2.33s/it]
 11%|█▏        | 337/2975 [16:53<1:42:11,  2.32s/it]
 11%|█▏        | 338/2975 [16:56<1:42:32,  2.33s/it]
 11%|█▏        | 339/2975 [16:58<1:41:49,  2.32s/it]
 11%|█▏        | 340/2975 [17:00<1:41:23,  2.31s/it]
                                                    

 11%|█▏        | 340/2975 [17:00<1:41:23,  2.31s/it]
 11%|█▏        | 341/2975 [17:03<1:41:05,  2.30s/it]
 11%|█▏        | 342/2975 [17:05<1:41:03,  2.30s/it]
 12%|█▏        | 343/2975 [17:07<1:41:56,  2.32s/it]
 12%|█▏        | 344/2975 [17:10<1:41:19,  2.31s/it]
 12%|█▏        | 345/2975 [17
0: {'loss': 0.4618, 'grad_norm': 0.46927762799100936, 'learning_rate': 3e-06, 'epoch': 0.12}
0: {'loss': 0.4649, 'grad_norm': 0.4988019980035848, 'learning_rate': 3e-06, 'epoch': 0.12}
0: :12<1:41:12,  2.31s/it]
 12%|█▏        | 346/2975 [17:14<1:40:53,  2.30s/it]
 12%|█▏        | 347/2975 [17:17<1:40:34,  2.30s/it]
 12%|█▏        | 348/2975 [17:19<1:40:20,  2.29s/it]
 12%|█▏        | 349/2975 [17:21<1:40:13,  2.29s/it]
 12%|█▏        | 350/2975 [17:23<1:40:20,  2.29s/it]
                                                    

 12%|█▏        | 350/2975 [17:23<1:40:20,  2.29s/it]
 12%|█▏        | 351/2975 [17:26<1:40:19,  2.29s/it]
 12%|█▏        | 352/2975 [17:28<1:40:09,  2.29s/it]
 12%|█▏        | 353/2975 [17:30<1:40:28,  2.30s/it]
 12%|█▏        | 354/2975 [17:33<1:40:10,  2.29s/it]
 12%|█▏        | 355/2975 [17:35<1:40:06,  2.29s/it]
 12%|█▏        | 356/2975 [17:37<1:40:06,  2.29s/it]
 12%|█▏        | 357/2975 [17:40<1:42:11,  2.34s/it]
 12%|█▏        | 358/2975 [17:42<1:41:27,  2.33s/it]
 12%|█▏        | 359/2975 [17:45<1:47:11,  2.46s/it]
 12%|█▏        | 360/2975 [17:47<1:44:54,  2.41s/it]
                                  
0: {'loss': 0.4601, 'grad_norm': 0.5013571479232268, 'learning_rate': 3e-06, 'epoch': 0.12}
0:                   

 12%|█▏        | 360/2975 [17:47<1:44:54,  2.41s/it]
 12%|█▏        | 361/2975 [17:49<1:43:10,  2.37s/it]
 12%|█▏        | 362/2975 [17:52<1:41:54,  2.34s/it]
 12%|█▏        | 363/2975 [17:54<1:41:08,  2.32s/it]
 12%|█▏        | 364/2975 [17:56<1:41:29,  2.33s/it]
 12%|█▏        | 365/2975 [17:58<1:41:04,  2.32s/it]
 12%|█▏        | 366/2975 [18:01<1:40:17,  2.31s/it]
 12%|█▏        | 367/2975 [18:03<1:39:49,  2.30s/it]
 12%|█▏        | 368/2975 [18:05<1:39:34,  2.29s/it]
 12%|█▏        | 369/2975 [18:08<1:39:37,  2.29s/it]
 12%|█▏        | 370/2975 [18:10<1:39:25,  2.29s/it]
                                                    

 12%|█▏        | 370/2975 [18:10<1:39:25,  2.29s/it]
 12%|█▏        | 371/2975 [18:12<1:39:37,  2.30s/it]
 13%|█▎        | 372/2975 [18:14<1:39:12,  2.29s/it]
 13%|█▎        | 373/2975 [18:17<1:39:07,  2.29s/it]
 13%|█▎        | 374/2975 [18:19<1:39:17,  2.29s/it]
 13%|█▎        | 375/2975 [18:21<1
0: {'loss': 0.4521, 'grad_norm': 0.47911325852472864, 'learning_rate': 3e-06, 'epoch': 0.13}
0: :39:55,  2.31s/it]
 13%|█▎        | 376/2975 [18:24<1:39:15,  2.29s/it]
 13%|█▎        | 377/2975 [18:26<1:40:12,  2.31s/it]
 13%|█▎        | 378/2975 [18:28<1:39:31,  2.30s/it]
 13%|█▎        | 379/2975 [18:31<1:38:48,  2.28s/it]
 13%|█▎        | 380/2975 [18:33<1:39:18,  2.30s/it]
                                                    

 13%|█▎        | 380/2975 [18:33<1:39:18,  2.30s/it]
 13%|█▎        | 381/2975 [18:35<1:39:32,  2.30s/it]
 13%|█▎        | 382/2975 [18:37<1:39:10,  2.29s/it]
 13%|█▎        | 383/2975 [18:40<1:38:54,  2.29s/it]
 13%|█▎        | 384/2975 [18:42<1:39:32,  2.31s/it]
 13%|█▎        | 385/2975 [18:44<1:39:37,  2.31s/it]
 13%|█▎        | 386/2975 [18:47<1:39:41,  2.31s/it]
 13%|█▎        | 387/2975 [18:49<1:40:15,  2.32s/it]
 13%|█▎        | 388/2975 [18:51<1:39:49,  2.32s/it]
 13%|█▎        | 389/2975 [18:54<1:39:42,  2.31s/it]
 13%|█▎        | 390/2975 [18:56<1:42:02,  2.37s/it]
                                       
0: {'loss': 0.4629, 'grad_norm': 0.46351782554956134, 'learning_rate': 3e-06, 'epoch': 0.13}
0: {'loss': 0.4589, 'grad_norm': 0.4879289994086694, 'learning_rate': 3e-06, 'epoch': 0.13}
0:              

 13%|█▎        | 390/2975 [18:56<1:42:02,  2.37s/it]
 13%|█▎        | 391/2975 [18:58<1:41:06,  2.35s/it]
 13%|█▎        | 392/2975 [19:01<1:40:20,  2.33s/it]
 13%|█▎        | 393/2975 [19:03<1:40:00,  2.32s/it]
 13%|█▎        | 394/2975 [19:05<1:39:11,  2.31s/it]
 13%|█▎        | 395/2975 [19:08<1:38:47,  2.30s/it]
 13%|█▎        | 396/2975 [19:10<1:39:56,  2.32s/it]
 13%|█▎        | 397/2975 [19:12<1:39:17,  2.31s/it]
 13%|█▎        | 398/2975 [19:15<1:38:56,  2.30s/it]
 13%|█▎        | 399/2975 [19:17<1:38:43,  2.30s/it]
 13%|█▎        | 400/2975 [19:19<1:38:35,  2.30s/it]
                                                    

 13%|█▎        | 400/2975 [19:19<1:38:35,  2.30s/it]
 13%|█▎        | 401/2975 [19:21<1:38:28,  2.30s/it]
 14%|█▎        | 402/2975 [19:24<1:40:11,  2.34s/it]
 14%|█▎        | 403/2975 [19:26<1:39:28,  2.32s/it]
 14%|█▎        | 404/2975 [19:29<1:46:29,  2.49s/it]
 14%|█▎        | 405/2975 [19:31<1:43:5
0: {'loss': 0.4396, 'grad_norm': 0.4882611726520583, 'learning_rate': 3e-06, 'epoch': 0.14}
0: 1,  2.42s/it]
 14%|█▎        | 406/2975 [19:34<1:42:09,  2.39s/it]
 14%|█▎        | 407/2975 [19:36<1:40:45,  2.35s/it]
 14%|█▎        | 408/2975 [19:38<1:39:52,  2.33s/it]
 14%|█▎        | 409/2975 [19:40<1:39:16,  2.32s/it]
 14%|█▍        | 410/2975 [19:43<1:38:47,  2.31s/it]
                                                    

 14%|█▍        | 410/2975 [19:43<1:38:47,  2.31s/it]
 14%|█▍        | 411/2975 [19:45<1:40:25,  2.35s/it]
 14%|█▍        | 412/2975 [19:48<1:40:32,  2.35s/it]
 14%|█▍        | 413/2975 [19:50<1:39:40,  2.33s/it]
 14%|█▍        | 414/2975 [19:53<1:46:14,  2.49s/it]
 14%|█▍        | 415/2975 [19:55<1:44:29,  2.45s/it]
 14%|█▍        | 416/2975 [19:57<1:42:44,  2.41s/it]
 14%|█▍        | 417/2975 [20:00<1:42:54,  2.41s/it]
 14%|█▍        | 418/2975 [20:02<1:41:17,  2.38s/it]
 14%|█▍        | 419/2975 [20:05<1:42:58,  2.42s/it]
 14%|█▍        | 420/2975 [20:07<1:41:39,  2.39s/it]
                                            
0: {'loss': 0.4598, 'grad_norm': 0.5041791303795379, 'learning_rate': 3e-06, 'epoch': 0.14}
0: {'loss': 0.4512, 'grad_norm': 0.5007792487637535, 'learning_rate': 3e-06, 'epoch': 0.14}
0:         

 14%|█▍        | 420/2975 [20:07<1:41:39,  2.39s/it]
 14%|█▍        | 421/2975 [20:09<1:40:16,  2.36s/it]
 14%|█▍        | 422/2975 [20:11<1:39:17,  2.33s/it]
 14%|█▍        | 423/2975 [20:14<1:38:41,  2.32s/it]
 14%|█▍        | 424/2975 [20:16<1:38:19,  2.31s/it]
 14%|█▍        | 425/2975 [20:18<1:37:53,  2.30s/it]
 14%|█▍        | 426/2975 [20:21<1:39:43,  2.35s/it]
 14%|█▍        | 427/2975 [20:23<1:40:07,  2.36s/it]
 14%|█▍        | 428/2975 [20:25<1:39:36,  2.35s/it]
 14%|█▍        | 429/2975 [20:28<1:40:56,  2.38s/it]
 14%|█▍        | 430/2975 [20:30<1:41:56,  2.40s/it]
                                                    

 14%|█▍        | 430/2975 [20:30<1:41:56,  2.40s/it]
 14%|█▍        | 431/2975 [20:33<1:40:37,  2.37s/it]
 15%|█▍        | 432/2975 [20:35<1:39:27,  2.35s/it]
 15%|█▍        | 433/2975 [20:37<1:39:06,  2.34s/it]
 15%|█▍        | 434/2975 [20:40<1:38:20,  2.32s/it]
 15%|█▍        | 435/2975 [20:42<1:37:55,  2
0: {'loss': 0.4495, 'grad_norm': 0.4749731397872052, 'learning_rate': 3e-06, 'epoch': 0.15}
0: .31s/it]
 15%|█▍        | 436/2975 [20:44<1:37:32,  2.31s/it]
 15%|█▍        | 437/2975 [20:46<1:37:24,  2.30s/it]
 15%|█▍        | 438/2975 [20:49<1:37:07,  2.30s/it]
 15%|█▍        | 439/2975 [20:51<1:36:49,  2.29s/it]
 15%|█▍        | 440/2975 [20:53<1:36:39,  2.29s/it]
                                                    

 15%|█▍        | 440/2975 [20:53<1:36:39,  2.29s/it]
 15%|█▍        | 441/2975 [20:56<1:36:32,  2.29s/it]
 15%|█▍        | 442/2975 [20:58<1:36:33,  2.29s/it]
 15%|█▍        | 443/2975 [21:00<1:36:36,  2.29s/it]
 15%|█▍        | 444/2975 [21:02<1:36:33,  2.29s/it]
 15%|█▍        | 445/2975 [21:05<1:36:17,  2.28s/it]
 15%|█▍        | 446/2975 [21:07<1:36:11,  2.28s/it]
 15%|█▌        | 447/2975 [21:09<1:36:51,  2.30s/it]
 15%|█▌        | 448/2975 [21:12<1:37:17,  2.31s/it]
 15%|█▌        | 449/2975 [21:14<1:37:45,  2.32s/it]
 15%|█▌        | 450/2975 [21:16<1:37:49,  2.32s/it]
                                                 
0: {'loss': 0.4472, 'grad_norm': 0.4697539248964525, 'learning_rate': 3e-06, 'epoch': 0.15}
0: {'loss': 0.4611, 'grad_norm': 0.49047577553224775, 'learning_rate': 3e-06, 'epoch': 0.15}
0:    

 15%|█▌        | 450/2975 [21:16<1:37:49,  2.32s/it]
 15%|█▌        | 451/2975 [21:19<1:38:11,  2.33s/it]
 15%|█▌        | 452/2975 [21:21<1:40:14,  2.38s/it]
 15%|█▌        | 453/2975 [21:23<1:39:07,  2.36s/it]
 15%|█▌        | 454/2975 [21:26<1:38:31,  2.34s/it]
 15%|█▌        | 455/2975 [21:28<1:37:50,  2.33s/it]
 15%|█▌        | 456/2975 [21:30<1:37:26,  2.32s/it]
 15%|█▌        | 457/2975 [21:33<1:36:40,  2.30s/it]
 15%|█▌        | 458/2975 [21:35<1:36:23,  2.30s/it]
 15%|█▌        | 459/2975 [21:37<1:36:08,  2.29s/it]
 15%|█▌        | 460/2975 [21:40<1:36:30,  2.30s/it]
                                                    

 15%|█▌        | 460/2975 [21:40<1:36:30,  2.30s/it]
 15%|█▌        | 461/2975 [21:42<1:36:17,  2.30s/it]
 16%|█▌        | 462/2975 [21:44<1:36:18,  2.30s/it]
 16%|█▌        | 463/2975 [21:46<1:36:19,  2.30s/it]
 16%|█▌        | 464/2975 [21:49<1:37:40,  2.33s/it]
 16%|█▌        | 465/2975 [21:51<1:37:50,  2.34s/
0: {'loss': 0.4627, 'grad_norm': 0.5206772885806493, 'learning_rate': 3e-06, 'epoch': 0.16}
0: {'loss': 0.4502, 'grad_norm': 0.5026730828056636, 'learning_rate': 3e-06, 'epoch': 0.16}
0: it]
 16%|█▌        | 466/2975 [21:54<1:37:25,  2.33s/it]
 16%|█▌        | 467/2975 [21:56<1:37:10,  2.32s/it]
 16%|█▌        | 468/2975 [21:58<1:37:12,  2.33s/it]
 16%|█▌        | 469/2975 [22:01<1:37:31,  2.34s/it]
 16%|█▌        | 470/2975 [22:03<1:37:16,  2.33s/it]
                                                    

 16%|█▌        | 470/2975 [22:03<1:37:16,  2.33s/it]
 16%|█▌        | 471/2975 [22:05<1:37:14,  2.33s/it]
 16%|█▌        | 472/2975 [22:07<1:36:50,  2.32s/it]
 16%|█▌        | 473/2975 [22:10<1:37:03,  2.33s/it]
 16%|█▌        | 474/2975 [22:12<1:38:40,  2.37s/it]
 16%|█▌        | 475/2975 [22:15<1:37:41,  2.34s/it]
 16%|█▌        | 476/2975 [22:17<1:36:43,  2.32s/it]
 16%|█▌        | 477/2975 [22:19<1:36:21,  2.31s/it]
 16%|█▌        | 478/2975 [22:21<1:35:54,  2.30s/it]
 16%|█▌        | 479/2975 [22:24<1:37:28,  2.34s/it]
 16%|█▌        | 480/2975 [22:26<1:37:08,  2.34s/it]
                                                    

0: {'loss': 0.4548, 'grad_norm': 2.574826829553421, 'learning_rate': 3e-06, 'epoch': 0.16}
0:  16%|█▌        | 480/2975 [22:26<1:37:08,  2.34s/it]
 16%|█▌        | 481/2975 [22:28<1:36:20,  2.32s/it]
 16%|█▌        | 482/2975 [22:31<1:36:56,  2.33s/it]
 16%|█▌        | 483/2975 [22:33<1:36:30,  2.32s/it]
 16%|█▋        | 484/2975 [22:35<1:35:53,  2.31s/it]
 16%|█▋        | 485/2975 [22:38<1:35:28,  2.30s/it]
 16%|█▋        | 486/2975 [22:40<1:35:12,  2.29s/it]
 16%|█▋        | 487/2975 [22:42<1:35:23,  2.30s/it]
 16%|█▋        | 488/2975 [22:45<1:36:47,  2.34s/it]
 16%|█▋        | 489/2975 [22:47<1:36:47,  2.34s/it]
 16%|█▋        | 490/2975 [22:49<1:36:44,  2.34s/it]
                                                    

 16%|█▋        | 490/2975 [22:49<1:36:44,  2.34s/it]
 17%|█▋        | 491/2975 [22:52<1:39:10,  2.40s/it]
 17%|█▋        | 492/2975 [22:54<1:38:26,  2.38s/it]
 17%|█▋        | 493/2975 [22:57<1:37:39,  2.36s/it]
 17%|█▋        | 494/2975 [22:59<1:36:53,  2.34s/it]
 17%|█▋        | 495/2975 [23:01<1:36:02,  2.32s/it]
 
0: {'loss': 0.4508, 'grad_norm': 0.505548979908201, 'learning_rate': 3e-06, 'epoch': 0.17}
0: {'loss': 0.4567, 'grad_norm': 0.4727771556423643, 'learning_rate': 3e-06, 'epoch': 0.17}
0: 17%|█▋        | 496/2975 [23:03<1:35:50,  2.32s/it]
 17%|█▋        | 497/2975 [23:06<1:35:25,  2.31s/it]
 17%|█▋        | 498/2975 [23:08<1:37:47,  2.37s/it]
 17%|█▋        | 499/2975 [23:11<1:37:12,  2.36s/it]
 17%|█▋        | 500/2975 [23:13<1:36:13,  2.33s/it]
                                                    

 17%|█▋        | 500/2975 [23:13<1:36:13,  2.33s/it]
 17%|█▋        | 501/2975 [23:15<1:35:29,  2.32s/it]
 17%|█▋        | 502/2975 [23:17<1:35:00,  2.31s/it]
 17%|█▋        | 503/2975 [23:20<1:35:15,  2.31s/it]
 17%|█▋        | 504/2975 [23:23<1:43:05,  2.50s/it]
 17%|█▋        | 505/2975 [23:25<1:40:17,  2.44s/it]
 17%|█▋        | 506/2975 [23:27<1:38:26,  2.39s/it]
 17%|█▋        | 507/2975 [23:30<1:37:10,  2.36s/it]
 17%|█▋        | 508/2975 [23:32<1:36:20,  2.34s/it]
 17%|█▋        | 509/2975 [23:34<1:36:30,  2.35s/it]
 17%|█▋        | 510/2975 [23:37<1:37:20,  2.37s/it]
                                                    

 17%|
0: {'loss': 0.4589, 'grad_norm': 0.4784078950454549, 'learning_rate': 3e-06, 'epoch': 0.17}
0: █▋        | 510/2975 [23:37<1:37:20,  2.37s/it]
 17%|█▋        | 511/2975 [23:39<1:37:06,  2.36s/it]
 17%|█▋        | 512/2975 [23:41<1:36:34,  2.35s/it]
 17%|█▋        | 513/2975 [23:44<1:36:07,  2.34s/it]
 17%|█▋        | 514/2975 [23:46<1:35:39,  2.33s/it]
 17%|█▋        | 515/2975 [23:48<1:36:01,  2.34s/it]
 17%|█▋        | 516/2975 [23:51<1:34:59,  2.32s/it]
 17%|█▋        | 517/2975 [23:53<1:34:42,  2.31s/it]
 17%|█▋        | 518/2975 [23:55<1:34:15,  2.30s/it]
 17%|█▋        | 519/2975 [23:58<1:35:58,  2.34s/it]
 17%|█▋        | 520/2975 [24:00<1:35:03,  2.32s/it]
                                                    

 17%|█▋        | 520/2975 [24:00<1:35:03,  2.32s/it]
 18%|█▊        | 521/2975 [24:02<1:34:25,  2.31s/it]
 18%|█▊        | 522/2975 [24:05<1:35:56,  2.35s/it]
 18%|█▊        | 523/2975 [24:07<1:35:48,  2.34s/it]
 18%|█▊        | 524/2975 [24:09<1:34:57,  2.32s/it]
 18%|█▊        | 525/2975 [24:11<1:34:42,  2.32s/it]
 18%|�
0: {'loss': 0.4523, 'grad_norm': 0.49174370897331376, 'learning_rate': 3e-06, 'epoch': 0.18}
0: {'loss': 0.4484, 'grad_norm': 0.4903968027161027, 'learning_rate': 3e-06, 'epoch': 0.18}
0: ��▊        | 526/2975 [24:14<1:34:21,  2.31s/it]
 18%|█▊        | 527/2975 [24:16<1:37:45,  2.40s/it]
 18%|█▊        | 528/2975 [24:19<1:36:26,  2.36s/it]
 18%|█▊        | 529/2975 [24:21<1:37:22,  2.39s/it]
 18%|█▊        | 530/2975 [24:23<1:36:47,  2.38s/it]
                                                    

 18%|█▊        | 530/2975 [24:23<1:36:47,  2.38s/it]
 18%|█▊        | 531/2975 [24:26<1:36:17,  2.36s/it]
 18%|█▊        | 532/2975 [24:28<1:35:41,  2.35s/it]
 18%|█▊        | 533/2975 [24:30<1:35:42,  2.35s/it]
 18%|█▊        | 534/2975 [24:33<1:35:10,  2.34s/it]
 18%|█▊        | 535/2975 [24:35<1:36:37,  2.38s/it]
 18%|█▊        | 536/2975 [24:37<1:35:17,  2.34s/it]
 18%|█▊        | 537/2975 [24:40<1:34:35,  2.33s/it]
 18%|█▊        | 538/2975 [24:42<1:34:14,  2.32s/it]
 18%|█▊        | 539/2975 [24:45<1:41:51,  2.51s/it]
 18%|█▊        | 540/2975 [24:48<1:42:00,  2.51s/it]
                                                    

 18%|█�
0: {'loss': 0.4471, 'grad_norm': 0.48502208580553163, 'learning_rate': 3e-06, 'epoch': 0.18}
0: �        | 540/2975 [24:48<1:42:00,  2.51s/it]
 18%|█▊        | 541/2975 [24:50<1:43:22,  2.55s/it]
 18%|█▊        | 542/2975 [24:52<1:40:19,  2.47s/it]
 18%|█▊        | 543/2975 [24:55<1:38:06,  2.42s/it]
 18%|█▊        | 544/2975 [24:57<1:36:44,  2.39s/it]
 18%|█▊        | 545/2975 [24:59<1:35:49,  2.37s/it]
 18%|█▊        | 546/2975 [25:02<1:35:49,  2.37s/it]
 18%|█▊        | 547/2975 [25:04<1:34:32,  2.34s/it]
 18%|█▊        | 548/2975 [25:06<1:34:22,  2.33s/it]
 18%|█▊        | 549/2975 [25:09<1:34:49,  2.35s/it]
 18%|█▊        | 550/2975 [25:11<1:34:40,  2.34s/it]
                                                    

 18%|█▊        | 550/2975 [25:11<1:34:40,  2.34s/it]
 19%|█▊        | 551/2975 [25:13<1:34:35,  2.34s/it]
 19%|█▊        | 552/2975 [25:16<1:34:07,  2.33s/it]
 19%|█▊        | 553/2975 [25:18<1:34:05,  2.33s/it]
 19%|█▊        | 554/2975 [25:20<1:34:08,  2.33s/it]
 19%|█▊        | 555/2975 [25:23<1:33:32,  2.32s/it]
 19%|█▊
0: {'loss': 0.4468, 'grad_norm': 0.4876953010443886, 'learning_rate': 3e-06, 'epoch': 0.19}
0: {'loss': 0.4586, 'grad_norm': 0.47893766256441384, 'learning_rate': 3e-06, 'epoch': 0.19}
0:         | 556/2975 [25:25<1:33:05,  2.31s/it]
 19%|█▊        | 557/2975 [25:27<1:32:52,  2.30s/it]
 19%|█▉        | 558/2975 [25:30<1:33:11,  2.31s/it]
 19%|█▉        | 559/2975 [25:32<1:32:53,  2.31s/it]
 19%|█▉        | 560/2975 [25:34<1:32:57,  2.31s/it]
                                                    

 19%|█▉        | 560/2975 [25:34<1:32:57,  2.31s/it]
 19%|█▉        | 561/2975 [25:36<1:32:54,  2.31s/it]
 19%|█▉        | 562/2975 [25:39<1:33:20,  2.32s/it]
 19%|█▉        | 563/2975 [25:41<1:32:45,  2.31s/it]
 19%|█▉        | 564/2975 [25:43<1:32:38,  2.31s/it]
 19%|█▉        | 565/2975 [25:46<1:32:21,  2.30s/it]
 19%|█▉        | 566/2975 [25:48<1:32:17,  2.30s/it]
 19%|█▉        | 567/2975 [25:50<1:32:13,  2.30s/it]
 19%|█▉        | 568/2975 [25:53<1:32:22,  2.30s/it]
 19%|█▉        | 569/2975 [25:56<1:39:51,  2.49s/it]
 19%|█▉        | 570/2975 [25:58<1:38:05,  2.45s/it]
                                                    

 19%|█▉    
0: {'loss': 0.4464, 'grad_norm': 0.4611486426021631, 'learning_rate': 3e-06, 'epoch': 0.19}
0:     | 570/2975 [25:58<1:38:05,  2.45s/it]
 19%|█▉        | 571/2975 [26:00<1:36:56,  2.42s/it]
 19%|█▉        | 572/2975 [26:03<1:36:03,  2.40s/it]
 19%|█▉        | 573/2975 [26:05<1:35:19,  2.38s/it]
 19%|█▉        | 574/2975 [26:07<1:34:58,  2.37s/it]
 19%|█▉        | 575/2975 [26:10<1:36:41,  2.42s/it]
 19%|█▉        | 576/2975 [26:12<1:36:12,  2.41s/it]
 19%|█▉        | 577/2975 [26:14<1:34:37,  2.37s/it]
 19%|█▉        | 578/2975 [26:17<1:33:34,  2.34s/it]
 19%|█▉        | 579/2975 [26:19<1:33:01,  2.33s/it]
 19%|█▉        | 580/2975 [26:21<1:32:28,  2.32s/it]
                                                    

 19%|█▉        | 580/2975 [26:21<1:32:28,  2.32s/it]
 20%|█▉        | 581/2975 [26:24<1:34:31,  2.37s/it]
 20%|█▉        | 582/2975 [26:26<1:33:45,  2.35s/it]
 20%|█▉        | 583/2975 [26:28<1:33:45,  2.35s/it]
 20%|█▉        | 584/2975 [26:31<1:33:10,  2.34s/it]
 20%|█▉        | 585/2975 [26:33<1:33:55,  2.36s/it]
 20%|█▉     
0: {'loss': 0.4521, 'grad_norm': 0.47964598417001625, 'learning_rate': 3e-06, 'epoch': 0.2}
0: {'loss': 0.4405, 'grad_norm': 0.4874147577494643, 'learning_rate': 3e-06, 'epoch': 0.2}
0:    | 586/2975 [26:35<1:32:58,  2.34s/it]
 20%|█▉        | 587/2975 [26:38<1:32:13,  2.32s/it]
 20%|█▉        | 588/2975 [26:40<1:31:52,  2.31s/it]
 20%|█▉        | 589/2975 [26:42<1:32:57,  2.34s/it]
 20%|█▉        | 590/2975 [26:45<1:33:42,  2.36s/it]
                                                    

 20%|█▉        | 590/2975 [26:45<1:33:42,  2.36s/it]
 20%|█▉        | 591/2975 [26:47<1:33:45,  2.36s/it]
 20%|█▉        | 592/2975 [26:50<1:33:14,  2.35s/it]
 20%|█▉        | 593/2975 [26:52<1:33:15,  2.35s/it]
 20%|█▉        | 594/2975 [26:54<1:33:21,  2.35s/it]
 20%|██        | 595/2975 [26:57<1:33:21,  2.35s/it]
 20%|██        | 596/2975 [27:21<5:51:50,  8.87s/it]
 20%|██        | 597/2975 [27:23<4:33:06,  6.89s/it]
 20%|██        | 598/2975 [27:25<3:38:02,  5.50s/it]
 20%|██        | 599/2975 [27:28<2:59:37,  4.54s/it]
 20%|██        | 600/2975 [27:30<2:32:48,  3.86s/it]
                                                    

 20%|██        |
0: {'loss': 0.4388, 'grad_norm': 0.5212938447997459, 'learning_rate': 3e-06, 'epoch': 0.2}
0:  600/2975 [27:30<2:32:48,  3.86s/it]
 20%|██        | 601/2975 [27:32<2:14:19,  3.40s/it]
 20%|██        | 602/2975 [27:34<2:00:57,  3.06s/it]
 20%|██        | 603/2975 [27:37<1:52:30,  2.85s/it]
 20%|██        | 604/2975 [27:39<1:45:32,  2.67s/it]
 20%|██        | 605/2975 [27:41<1:41:13,  2.56s/it]
 20%|██        | 606/2975 [27:44<1:38:00,  2.48s/it]
 20%|██        | 607/2975 [27:46<1:35:43,  2.43s/it]
 20%|██        | 608/2975 [27:48<1:34:44,  2.40s/it]
 20%|██        | 609/2975 [27:51<1:33:18,  2.37s/it]
 21%|██        | 610/2975 [27:53<1:35:46,  2.43s/it]
                                                    

 21%|██        | 610/2975 [27:53<1:35:46,  2.43s/it]
 21%|██        | 611/2975 [27:55<1:34:09,  2.39s/it]
 21%|██        | 612/2975 [27:58<1:32:46,  2.36s/it]
 21%|██        | 613/2975 [28:00<1:31:57,  2.34s/it]
 21%|██        | 614/2975 [28:02<1:31:20,  2.32s/it]
 21%|██        | 615/2975 [28:05<1:31:37,  2.33s/it]
 21%|██        | 
0: {'loss': 0.46, 'grad_norm': 0.49205805451271506, 'learning_rate': 3e-06, 'epoch': 0.21}
0: {'loss': 0.4512, 'grad_norm': 0.46527759632882904, 'learning_rate': 3e-06, 'epoch': 0.21}
0: 616/2975 [28:07<1:30:59,  2.31s/it]
 21%|██        | 617/2975 [28:09<1:33:09,  2.37s/it]
 21%|██        | 618/2975 [28:12<1:32:17,  2.35s/it]
 21%|██        | 619/2975 [28:14<1:31:56,  2.34s/it]
 21%|██        | 620/2975 [28:16<1:31:22,  2.33s/it]
                                                    

 21%|██        | 620/2975 [28:16<1:31:22,  2.33s/it]
 21%|██        | 621/2975 [28:19<1:33:26,  2.38s/it]
 21%|██        | 622/2975 [28:21<1:32:02,  2.35s/it]
 21%|██        | 623/2975 [28:23<1:31:17,  2.33s/it]
 21%|██        | 624/2975 [28:26<1:30:38,  2.31s/it]
 21%|██        | 625/2975 [28:28<1:30:22,  2.31s/it]
 21%|██        | 626/2975 [28:30<1:30:06,  2.30s/it]
 21%|██        | 627/2975 [28:33<1:31:23,  2.34s/it]
 21%|██        | 628/2975 [28:35<1:30:49,  2.32s/it]
 21%|██        | 629/2975 [28:37<1:33:20,  2.39s/it]
 21%|██        | 630/2975 [28:40<1:32:00,  2.35s/it]
                                                    

 21%|██        | 630/
0: {'loss': 0.4444, 'grad_norm': 0.46977336172967155, 'learning_rate': 3e-06, 'epoch': 0.22}
0: 2975 [28:40<1:32:00,  2.35s/it]
 21%|██        | 631/2975 [28:42<1:31:42,  2.35s/it]
 21%|██        | 632/2975 [28:44<1:30:49,  2.33s/it]
 21%|██▏       | 633/2975 [28:47<1:30:22,  2.32s/it]
 21%|██▏       | 634/2975 [28:49<1:29:55,  2.30s/it]
 21%|██▏       | 635/2975 [28:51<1:29:41,  2.30s/it]
 21%|██▏       | 636/2975 [28:54<1:30:12,  2.31s/it]
 21%|██▏       | 637/2975 [28:56<1:29:57,  2.31s/it]
 21%|██▏       | 638/2975 [28:58<1:29:32,  2.30s/it]
 21%|██▏       | 639/2975 [29:00<1:29:13,  2.29s/it]
 22%|██▏       | 640/2975 [29:03<1:29:03,  2.29s/it]
                                                    

 22%|██▏       | 640/2975 [29:03<1:29:03,  2.29s/it]
 22%|██▏       | 641/2975 [29:05<1:30:12,  2.32s/it]
 22%|██▏       | 642/2975 [29:07<1:29:46,  2.31s/it]
 22%|██▏       | 643/2975 [29:10<1:29:21,  2.30s/it]
 22%|██▏       | 644/2975 [29:12<1:29:26,  2.30s/it]
 22%|██▏       | 645/2975 [29:14<1:28:58,  2.29s/it
0: {'loss': 0.4421, 'grad_norm': 0.4899453998489135, 'learning_rate': 3e-06, 'epoch': 0.22}
0: ]
 22%|██▏       | 646/2975 [29:17<1:29:55,  2.32s/it]
 22%|██▏       | 647/2975 [29:19<1:34:05,  2.42s/it]
 22%|██▏       | 648/2975 [29:22<1:32:36,  2.39s/it]
 22%|██▏       | 649/2975 [29:24<1:31:29,  2.36s/it]
 22%|██▏       | 650/2975 [29:26<1:30:57,  2.35s/it]
                                                    

 22%|██▏       | 650/2975 [29:26<1:30:57,  2.35s/it]
 22%|██▏       | 651/2975 [29:28<1:30:16,  2.33s/it]
 22%|██▏       | 652/2975 [29:31<1:29:38,  2.32s/it]
 22%|██▏       | 653/2975 [29:33<1:32:58,  2.40s/it]
 22%|██▏       | 654/2975 [29:36<1:31:31,  2.37s/it]
 22%|██▏       | 655/2975 [29:38<1:30:48,  2.35s/it]
 22%|██▏       | 656/2975 [29:40<1:29:59,  2.33s/it]
 22%|██▏       | 657/2975 [29:43<1:31:17,  2.36s/it]
 22%|██▏       | 658/2975 [29:45<1:30:13,  2.34s/it]
 22%|██▏       | 659/2975 [29:48<1:35:00,  2.46s/it]
 22%|██▏       | 660/2975 [29:50<1:33:10,  2.41s/it]
                        
0: {'loss': 0.4445, 'grad_norm': 0.5105882160713487, 'learning_rate': 3e-06, 'epoch': 0.22}
0: {'loss': 0.445, 'grad_norm': 0.5258581165172035, 'learning_rate': 3e-06, 'epoch': 0.23}
0:                             

 22%|██▏       | 660/2975 [29:50<1:33:10,  2.41s/it]
 22%|██▏       | 661/2975 [29:52<1:31:33,  2.37s/it]
 22%|██▏       | 662/2975 [29:55<1:36:46,  2.51s/it]
 22%|██▏       | 663/2975 [29:57<1:34:34,  2.45s/it]
 22%|██▏       | 664/2975 [30:00<1:32:51,  2.41s/it]
 22%|██▏       | 665/2975 [30:02<1:31:35,  2.38s/it]
 22%|██▏       | 666/2975 [30:04<1:30:39,  2.36s/it]
 22%|██▏       | 667/2975 [30:07<1:30:17,  2.35s/it]
 22%|██▏       | 668/2975 [30:09<1:29:31,  2.33s/it]
 22%|██▏       | 669/2975 [30:11<1:29:59,  2.34s/it]
 23%|██▎       | 670/2975 [30:14<1:29:14,  2.32s/it]
                                                    

 23%|██▎       | 670/2975 [30:14<1:29:14,  2.32s/it]
 23%|██▎       | 671/2975 [30:16<1:28:53,  2.32s/it]
 23%|██▎       | 672/2975 [30:18<1:28:29,  2.31s/it]
 23%|██▎       | 673/2975 [30:20<1:28:21,  2.30s/it]
 23%|██▎       | 674/2975 [30:23<1:28:00,  2.30s/
0: {'loss': 0.4423, 'grad_norm': 0.4648290833520505, 'learning_rate': 3e-06, 'epoch': 0.23}
0: it]
 23%|██▎       | 675/2975 [30:25<1:27:54,  2.29s/it]
 23%|██▎       | 676/2975 [30:28<1:30:44,  2.37s/it]
 23%|██▎       | 677/2975 [30:30<1:29:46,  2.34s/it]
 23%|██▎       | 678/2975 [30:32<1:29:03,  2.33s/it]
 23%|██▎       | 679/2975 [30:34<1:28:26,  2.31s/it]
 23%|██▎       | 680/2975 [30:37<1:28:35,  2.32s/it]
                                                    

 23%|██▎       | 680/2975 [30:37<1:28:35,  2.32s/it]
 23%|██▎       | 681/2975 [30:39<1:28:18,  2.31s/it]
 23%|██▎       | 682/2975 [30:41<1:27:59,  2.30s/it]
 23%|██▎       | 683/2975 [30:44<1:27:55,  2.30s/it]
 23%|██▎       | 684/2975 [30:46<1:27:35,  2.29s/it]
 23%|██▎       | 685/2975 [30:48<1:27:20,  2.29s/it]
 23%|██▎       | 686/2975 [30:50<1:27:02,  2.28s/it]
 23%|██▎       | 687/2975 [30:53<1:27:09,  2.29s/it]
 23%|██▎       | 688/2975 [30:55<1:27:11,  2.29s/it]
 23%|██▎       | 689/2975 [30:57<1:27:08,  2.29s/it]
 23%|██▎       |
0: {'loss': 0.4494, 'grad_norm': 0.47622285682997406, 'learning_rate': 3e-06, 'epoch': 0.23}
0: {'loss': 0.4449, 'grad_norm': 0.47570139182180654, 'learning_rate': 3e-06, 'epoch': 0.24}
0:  690/2975 [31:00<1:27:48,  2.31s/it]
                                                    

 23%|██▎       | 690/2975 [31:00<1:27:48,  2.31s/it]
 23%|██▎       | 691/2975 [31:02<1:27:37,  2.30s/it]
 23%|██▎       | 692/2975 [31:05<1:30:13,  2.37s/it]
 23%|██▎       | 693/2975 [31:07<1:29:24,  2.35s/it]
 23%|██▎       | 694/2975 [31:09<1:28:47,  2.34s/it]
 23%|██▎       | 695/2975 [31:12<1:29:26,  2.35s/it]
 23%|██▎       | 696/2975 [31:14<1:28:54,  2.34s/it]
 23%|██▎       | 697/2975 [31:16<1:28:11,  2.32s/it]
 23%|██▎       | 698/2975 [31:18<1:27:57,  2.32s/it]
 23%|██▎       | 699/2975 [31:21<1:28:00,  2.32s/it]
 24%|██▎       | 700/2975 [31:23<1:27:30,  2.31s/it]
                                                    

 24%|██▎       | 700/2975 [31:23<1:27:30,  2.31s/it]
 24%|██▎       | 701/2975 [31:25<1:27:23,  2.31s/it]
 24%|██▎       | 702/2975 [31:28<1:27:22,  2.31s/it]
 24%|██▎       | 703/2975 [31:30<1:27:11,  2.30
0: {'loss': 0.4407, 'grad_norm': 0.48222576416004126, 'learning_rate': 3e-06, 'epoch': 0.24}
0: s/it]
 24%|██▎       | 704/2975 [31:32<1:27:13,  2.30s/it]
 24%|██▎       | 705/2975 [31:35<1:27:14,  2.31s/it]
 24%|██▎       | 706/2975 [31:37<1:27:21,  2.31s/it]
 24%|██▍       | 707/2975 [31:39<1:28:47,  2.35s/it]
 24%|██▍       | 708/2975 [31:42<1:28:02,  2.33s/it]
 24%|██▍       | 709/2975 [31:44<1:27:29,  2.32s/it]
 24%|██▍       | 710/2975 [31:46<1:27:10,  2.31s/it]
                                                    

 24%|██▍       | 710/2975 [31:46<1:27:10,  2.31s/it]
 24%|██▍       | 711/2975 [31:48<1:27:11,  2.31s/it]
 24%|██▍       | 712/2975 [31:51<1:27:02,  2.31s/it]
 24%|██▍       | 713/2975 [31:53<1:26:50,  2.30s/it]
 24%|██▍       | 714/2975 [31:55<1:28:09,  2.34s/it]
 24%|██▍       | 715/2975 [31:58<1:27:41,  2.33s/it]
 24%|██▍       | 716/2975 [32:00<1:27:15,  2.32s/it]
 24%|██▍       | 717/2975 [32:02<1:27:07,  2.32s/it]
 24%|██▍       | 718/2975 [32:05<1:26:46,  2.31s/it]
 24%|██▍      
0: {'loss': 0.4453, 'grad_norm': 0.4718883886551205, 'learning_rate': 3e-06, 'epoch': 0.24}
0: {'loss': 0.4265, 'grad_norm': 0.5184973556350014, 'learning_rate': 3e-06, 'epoch': 0.25}
0:  | 719/2975 [32:07<1:26:15,  2.29s/it]
 24%|██▍       | 720/2975 [32:09<1:26:35,  2.30s/it]
                                                    

 24%|██▍       | 720/2975 [32:09<1:26:35,  2.30s/it]
 24%|██▍       | 721/2975 [32:12<1:26:24,  2.30s/it]
 24%|██▍       | 722/2975 [32:14<1:26:02,  2.29s/it]
 24%|██▍       | 723/2975 [32:16<1:27:18,  2.33s/it]
 24%|██▍       | 724/2975 [32:19<1:26:44,  2.31s/it]
 24%|██▍       | 725/2975 [32:21<1:27:46,  2.34s/it]
 24%|██▍       | 726/2975 [32:23<1:27:04,  2.32s/it]
 24%|██▍       | 727/2975 [32:26<1:27:25,  2.33s/it]
 24%|██▍       | 728/2975 [32:28<1:26:46,  2.32s/it]
 25%|██▍       | 729/2975 [32:30<1:28:24,  2.36s/it]
 25%|██▍       | 730/2975 [32:33<1:30:37,  2.42s/it]
                                                    

 25%|██▍       | 730/2975 [32:33<1:30:37,  2.42s/it]
 25%|██▍       | 731/2975 [32:35<1:29:06,  2.38s/it]
 25%|██▍       | 732/2975 [32:37<1:28:01,  2.
0: {'loss': 0.4456, 'grad_norm': 0.497846824225062, 'learning_rate': 3e-06, 'epoch': 0.25}
0: 35s/it]
 25%|██▍       | 733/2975 [32:40<1:27:14,  2.33s/it]
 25%|██▍       | 734/2975 [32:42<1:27:47,  2.35s/it]
 25%|██▍       | 735/2975 [32:44<1:27:20,  2.34s/it]
 25%|██▍       | 736/2975 [32:47<1:27:19,  2.34s/it]
 25%|██▍       | 737/2975 [32:49<1:26:41,  2.32s/it]
 25%|██▍       | 738/2975 [32:51<1:26:06,  2.31s/it]
 25%|██▍       | 739/2975 [32:54<1:25:51,  2.30s/it]
 25%|██▍       | 740/2975 [32:56<1:28:30,  2.38s/it]
                                                    

 25%|██▍       | 740/2975 [32:56<1:28:30,  2.38s/it]
 25%|██▍       | 741/2975 [32:58<1:27:35,  2.35s/it]
 25%|██▍       | 742/2975 [33:01<1:28:05,  2.37s/it]
 25%|██▍       | 743/2975 [33:03<1:27:13,  2.34s/it]
 25%|██▌       | 744/2975 [33:05<1:26:43,  2.33s/it]
 25%|██▌       | 745/2975 [33:08<1:26:10,  2.32s/it]
 25%|██▌       | 746/2975 [33:10<1:25:44,  2.31s/it]
 25%|██▌       | 747/2975 [33:12<1:25:23,  2.30s/it]
 25%|██▌    
0: {'loss': 0.4394, 'grad_norm': 0.4923507655353028, 'learning_rate': 3e-06, 'epoch': 0.25}
0: {'loss': 0.441, 'grad_norm': 0.4987374565128982, 'learning_rate': 3e-06, 'epoch': 0.26}
0:    | 748/2975 [33:15<1:25:10,  2.29s/it]
 25%|██▌       | 749/2975 [33:17<1:25:22,  2.30s/it]
 25%|██▌       | 750/2975 [33:19<1:26:18,  2.33s/it]
                                                    

 25%|██▌       | 750/2975 [33:19<1:26:18,  2.33s/it]
 25%|██▌       | 751/2975 [33:22<1:25:53,  2.32s/it]
 25%|██▌       | 752/2975 [33:24<1:25:38,  2.31s/it]
 25%|██▌       | 753/2975 [33:26<1:25:40,  2.31s/it]
 25%|██▌       | 754/2975 [33:29<1:26:19,  2.33s/it]
 25%|██▌       | 755/2975 [33:31<1:25:51,  2.32s/it]
 25%|██▌       | 756/2975 [33:33<1:25:32,  2.31s/it]
 25%|██▌       | 757/2975 [33:35<1:25:15,  2.31s/it]
 25%|██▌       | 758/2975 [33:38<1:24:57,  2.30s/it]
 26%|██▌       | 759/2975 [33:40<1:24:37,  2.29s/it]
 26%|██▌       | 760/2975 [33:42<1:24:34,  2.29s/it]
                                                    

 26%|██▌       | 760/2975 [33:42<1:24:34,  2.29s/it]
 26%|██▌       | 761/2975 [33:45<1:24:58,  
0: {'loss': 0.4501, 'grad_norm': 0.5249077702142555, 'learning_rate': 3e-06, 'epoch': 0.26}
0: 2.30s/it]
 26%|██▌       | 762/2975 [33:47<1:25:13,  2.31s/it]
 26%|██▌       | 763/2975 [33:49<1:24:52,  2.30s/it]
 26%|██▌       | 764/2975 [33:52<1:24:36,  2.30s/it]
 26%|██▌       | 765/2975 [33:54<1:24:29,  2.29s/it]
 26%|██▌       | 766/2975 [33:56<1:24:22,  2.29s/it]
 26%|██▌       | 767/2975 [33:58<1:24:30,  2.30s/it]
 26%|██▌       | 768/2975 [34:01<1:24:16,  2.29s/it]
 26%|██▌       | 769/2975 [34:03<1:25:36,  2.33s/it]
 26%|██▌       | 770/2975 [34:05<1:25:15,  2.32s/it]
                                                    

 26%|██▌       | 770/2975 [34:05<1:25:15,  2.32s/it]
 26%|██▌       | 771/2975 [34:08<1:24:56,  2.31s/it]
 26%|██▌       | 772/2975 [34:10<1:24:33,  2.30s/it]
 26%|██▌       | 773/2975 [34:12<1:24:33,  2.30s/it]
 26%|██▌       | 774/2975 [34:15<1:24:23,  2.30s/it]
 26%|██▌       | 775/2975 [34:17<1:24:05,  2.29s/it]
 26%|██▌       | 776/2975 [34:19<1:25:14,  2.33s/it]
 26%|██▌  
0: {'loss': 0.4447, 'grad_norm': 0.49930616817972, 'learning_rate': 3e-06, 'epoch': 0.26}
0: {'loss': 0.4442, 'grad_norm': 0.48234600838891334, 'learning_rate': 3e-06, 'epoch': 0.27}
0:      | 777/2975 [34:22<1:26:22,  2.36s/it]
 26%|██▌       | 778/2975 [34:24<1:25:33,  2.34s/it]
 26%|██▌       | 779/2975 [34:26<1:24:52,  2.32s/it]
 26%|██▌       | 780/2975 [34:29<1:24:32,  2.31s/it]
                                                    

 26%|██▌       | 780/2975 [34:29<1:24:32,  2.31s/it]
 26%|██▋       | 781/2975 [34:31<1:24:22,  2.31s/it]
 26%|██▋       | 782/2975 [34:33<1:24:23,  2.31s/it]
 26%|██▋       | 783/2975 [34:36<1:24:39,  2.32s/it]
 26%|██▋       | 784/2975 [34:38<1:24:23,  2.31s/it]
 26%|██▋       | 785/2975 [34:40<1:24:00,  2.30s/it]
 26%|██▋       | 786/2975 [34:43<1:26:52,  2.38s/it]
 26%|██▋       | 787/2975 [34:45<1:26:27,  2.37s/it]
 26%|██▋       | 788/2975 [34:47<1:25:32,  2.35s/it]
 27%|██▋       | 789/2975 [34:50<1:27:14,  2.39s/it]
 27%|██▋       | 790/2975 [34:52<1:26:05,  2.36s/it]
                                                    

 27%|██▋       | 790/2975 [34:52<1:26:05,
0: {'loss': 0.4417, 'grad_norm': 0.49868746403713865, 'learning_rate': 3e-06, 'epoch': 0.27}
0:   2.36s/it]
 27%|██▋       | 791/2975 [34:54<1:25:26,  2.35s/it]
 27%|██▋       | 792/2975 [34:57<1:24:59,  2.34s/it]
 27%|██▋       | 793/2975 [34:59<1:24:30,  2.32s/it]
 27%|██▋       | 794/2975 [35:02<1:29:03,  2.45s/it]
 27%|██▋       | 795/2975 [35:04<1:27:15,  2.40s/it]
 27%|██▋       | 796/2975 [35:06<1:26:03,  2.37s/it]
 27%|██▋       | 797/2975 [35:09<1:25:10,  2.35s/it]
 27%|██▋       | 798/2975 [35:11<1:24:11,  2.32s/it]
 27%|██▋       | 799/2975 [35:13<1:23:50,  2.31s/it]
 27%|██▋       | 800/2975 [35:15<1:23:28,  2.30s/it]
                                                    

 27%|██▋       | 800/2975 [35:15<1:23:28,  2.30s/it]
 27%|██▋       | 801/2975 [35:18<1:23:18,  2.30s/it]
 27%|██▋       | 802/2975 [35:20<1:23:07,  2.30s/it]
 27%|██▋       | 803/2975 [35:22<1:23:03,  2.29s/it]
 27%|██▋       | 804/2975 [35:25<1:22:55,  2.29s/it]
 27%|██▋       | 805/2975 [35:28<1:30:17,  2.50s/it]
 27%|██▋
0: {'loss': 0.419, 'grad_norm': 0.4896106788289829, 'learning_rate': 3e-06, 'epoch': 0.27}
0:        | 806/2975 [35:30<1:28:06,  2.44s/it]
 27%|██▋       | 807/2975 [35:33<1:34:26,  2.61s/it]
 27%|██▋       | 808/2975 [35:35<1:30:44,  2.51s/it]
 27%|██▋       | 809/2975 [35:38<1:34:00,  2.60s/it]
 27%|██▋       | 810/2975 [35:40<1:30:34,  2.51s/it]
                                                    

 27%|██▋       | 810/2975 [35:40<1:30:34,  2.51s/it]
 27%|██▋       | 811/2975 [35:43<1:28:07,  2.44s/it]
 27%|██▋       | 812/2975 [35:45<1:27:18,  2.42s/it]
 27%|██▋       | 813/2975 [35:47<1:25:44,  2.38s/it]
 27%|██▋       | 814/2975 [35:50<1:26:48,  2.41s/it]
 27%|██▋       | 815/2975 [35:52<1:25:38,  2.38s/it]
 27%|██▋       | 816/2975 [35:54<1:24:52,  2.36s/it]
 27%|██▋       | 817/2975 [35:57<1:24:01,  2.34s/it]
 27%|██▋       | 818/2975 [35:59<1:23:27,  2.32s/it]
 28%|██▊       | 819/2975 [36:01<1:23:12,  2.32s/it]
 28%|██▊       | 820/2975 [36:03<1:22:49,  2.31s/it]
                                        
0: {'loss': 0.4442, 'grad_norm': 0.506424139494785, 'learning_rate': 3e-06, 'epoch': 0.28}
0: {'loss': 0.4338, 'grad_norm': 0.4781523266180852, 'learning_rate': 3e-06, 'epoch': 0.28}
0:             

 28%|██▊       | 820/2975 [36:03<1:22:49,  2.31s/it]
 28%|██▊       | 821/2975 [36:06<1:23:05,  2.31s/it]
 28%|██▊       | 822/2975 [36:08<1:22:49,  2.31s/it]
 28%|██▊       | 823/2975 [36:11<1:24:06,  2.35s/it]
 28%|██▊       | 824/2975 [36:13<1:24:15,  2.35s/it]
 28%|██▊       | 825/2975 [36:16<1:31:45,  2.56s/it]
 28%|██▊       | 826/2975 [36:18<1:29:03,  2.49s/it]
 28%|██▊       | 827/2975 [36:21<1:26:45,  2.42s/it]
 28%|██▊       | 828/2975 [36:23<1:25:16,  2.38s/it]
 28%|██▊       | 829/2975 [36:25<1:24:11,  2.35s/it]
 28%|██▊       | 830/2975 [36:27<1:24:20,  2.36s/it]
                                                    

 28%|██▊       | 830/2975 [36:28<1:24:20,  2.36s/it]
 28%|██▊       | 831/2975 [36:30<1:23:33,  2.34s/it]
 28%|██▊       | 832/2975 [36:32<1:23:05,  2.33s/it]
 28%|██▊       | 833/2975 [36:34<1:22:39,  2.32s/it]
 28%|██▊       | 834/2975 [36:37<1:22:26,  2.31s/it]
 28%|██�
0: {'loss': 0.4439, 'grad_norm': 0.49693332147004304, 'learning_rate': 3e-06, 'epoch': 0.28}
0: ��       | 835/2975 [36:39<1:22:20,  2.31s/it]
 28%|██▊       | 836/2975 [36:41<1:22:02,  2.30s/it]
 28%|██▊       | 837/2975 [36:44<1:22:36,  2.32s/it]
 28%|██▊       | 838/2975 [36:46<1:22:14,  2.31s/it]
 28%|██▊       | 839/2975 [36:48<1:22:39,  2.32s/it]
 28%|██▊       | 840/2975 [36:51<1:22:19,  2.31s/it]
                                                    

 28%|██▊       | 840/2975 [36:51<1:22:19,  2.31s/it]
 28%|██▊       | 841/2975 [36:53<1:22:03,  2.31s/it]
 28%|██▊       | 842/2975 [36:55<1:21:48,  2.30s/it]
 28%|██▊       | 843/2975 [36:57<1:21:52,  2.30s/it]
 28%|██▊       | 844/2975 [37:00<1:21:38,  2.30s/it]
 28%|██▊       | 845/2975 [37:02<1:21:28,  2.30s/it]
 28%|██▊       | 846/2975 [37:04<1:21:14,  2.29s/it]
 28%|██▊       | 847/2975 [37:07<1:21:11,  2.29s/it]
 29%|██▊       | 848/2975 [37:09<1:21:18,  2.29s/it]
 29%|██▊       | 849/2975 [37:11<1:21:03,  2.29s/it]
 29%|██▊       | 850/2975 [37:13
0: {'loss': 0.425, 'grad_norm': 0.45166131739794957, 'learning_rate': 3e-06, 'epoch': 0.29}
0: {'loss': 0.4432, 'grad_norm': 0.48047534997074515, 'learning_rate': 3e-06, 'epoch': 0.29}
0: <1:21:09,  2.29s/it]
                                                    

 29%|██▊       | 850/2975 [37:13<1:21:09,  2.29s/it]
 29%|██▊       | 851/2975 [37:16<1:21:49,  2.31s/it]
 29%|██▊       | 852/2975 [37:18<1:23:48,  2.37s/it]
 29%|██▊       | 853/2975 [37:21<1:22:55,  2.34s/it]
 29%|██▊       | 854/2975 [37:23<1:24:55,  2.40s/it]
 29%|██▊       | 855/2975 [37:25<1:23:31,  2.36s/it]
 29%|██▉       | 856/2975 [37:28<1:23:11,  2.36s/it]
 29%|██▉       | 857/2975 [37:30<1:22:07,  2.33s/it]
 29%|██▉       | 858/2975 [37:32<1:21:46,  2.32s/it]
 29%|██▉       | 859/2975 [37:35<1:21:12,  2.30s/it]
 29%|██▉       | 860/2975 [37:37<1:20:52,  2.29s/it]
                                                    

 29%|██▉       | 860/2975 [37:37<1:20:52,  2.29s/it]
 29%|██▉       | 861/2975 [37:39<1:21:11,  2.30s/it]
 29%|██▉       | 862/2975 [37:41<1:20:50,  2.30s/it]
 29%|██▉       | 863/2975 [37:44<1:20:44,  2.29s/it]
 29%|█�
0: {'loss': 0.4409, 'grad_norm': 0.48496438904564465, 'learning_rate': 3e-06, 'epoch': 0.29}
0: �▉       | 864/2975 [37:46<1:20:36,  2.29s/it]
 29%|██▉       | 865/2975 [37:48<1:20:43,  2.30s/it]
 29%|██▉       | 866/2975 [37:51<1:20:32,  2.29s/it]
 29%|██▉       | 867/2975 [37:53<1:20:31,  2.29s/it]
 29%|██▉       | 868/2975 [37:55<1:20:22,  2.29s/it]
 29%|██▉       | 869/2975 [37:57<1:20:23,  2.29s/it]
 29%|██▉       | 870/2975 [38:00<1:20:22,  2.29s/it]
                                                    

 29%|██▉       | 870/2975 [38:00<1:20:22,  2.29s/it]
 29%|██▉       | 871/2975 [38:02<1:20:27,  2.29s/it]
 29%|██▉       | 872/2975 [38:04<1:20:22,  2.29s/it]
 29%|██▉       | 873/2975 [38:07<1:20:18,  2.29s/it]
 29%|██▉       | 874/2975 [38:09<1:20:14,  2.29s/it]
 29%|██▉       | 875/2975 [38:11<1:20:24,  2.30s/it]
 29%|██▉       | 876/2975 [38:14<1:20:15,  2.29s/it]
 29%|██▉       | 877/2975 [38:16<1:20:01,  2.29s/it]
 30%|██▉       | 878/2975 [38:18<1:20:02,  2.29s/it]
 30%|██▉       | 879/2975 [38:
0: {'loss': 0.4479, 'grad_norm': 0.5000309791652109, 'learning_rate': 3e-06, 'epoch': 0.3}
0: {'loss': 0.4406, 'grad_norm': 0.47390915616109125, 'learning_rate': 3e-06, 'epoch': 0.3}
0: 20<1:20:11,  2.30s/it]
 30%|██▉       | 880/2975 [38:23<1:21:14,  2.33s/it]
                                                    

 30%|██▉       | 880/2975 [38:23<1:21:14,  2.33s/it]
 30%|██▉       | 881/2975 [38:25<1:20:52,  2.32s/it]
 30%|██▉       | 882/2975 [38:27<1:20:29,  2.31s/it]
 30%|██▉       | 883/2975 [38:30<1:20:18,  2.30s/it]
 30%|██▉       | 884/2975 [38:32<1:20:02,  2.30s/it]
 30%|██▉       | 885/2975 [38:34<1:19:45,  2.29s/it]
 30%|██▉       | 886/2975 [38:37<1:19:40,  2.29s/it]
 30%|██▉       | 887/2975 [38:39<1:19:32,  2.29s/it]
 30%|██▉       | 888/2975 [38:41<1:19:34,  2.29s/it]
 30%|██▉       | 889/2975 [38:43<1:19:25,  2.28s/it]
 30%|██▉       | 890/2975 [38:46<1:19:24,  2.28s/it]
                                                    

 30%|██▉       | 890/2975 [38:46<1:19:24,  2.28s/it]
 30%|██▉       | 891/2975 [38:48<1:19:28,  2.29s/it]
 30%|██▉       | 892/2975 [38:50<1:19:24,  2.29s/it]
 30%|█
0: {'loss': 0.4485, 'grad_norm': 0.4498228085037058, 'learning_rate': 3e-06, 'epoch': 0.3}
0: ██       | 893/2975 [38:53<1:19:16,  2.28s/it]
 30%|███       | 894/2975 [38:55<1:19:27,  2.29s/it]
 30%|███       | 895/2975 [38:57<1:19:22,  2.29s/it]
 30%|███       | 896/2975 [38:59<1:19:09,  2.28s/it]
 30%|███       | 897/2975 [39:02<1:19:38,  2.30s/it]
 30%|███       | 898/2975 [39:04<1:19:20,  2.29s/it]
 30%|███       | 899/2975 [39:06<1:19:14,  2.29s/it]
 30%|███       | 900/2975 [39:09<1:19:59,  2.31s/it]
                                                    

 30%|███       | 900/2975 [39:09<1:19:59,  2.31s/it]
 30%|███       | 901/2975 [39:11<1:20:37,  2.33s/it]
 30%|███       | 902/2975 [39:13<1:20:41,  2.34s/it]
 30%|███       | 903/2975 [39:16<1:19:57,  2.32s/it]
 30%|███       | 904/2975 [39:18<1:19:17,  2.30s/it]
 30%|███       | 905/2975 [39:20<1:19:06,  2.29s/it]
 30%|███       | 906/2975 [39:23<1:19:20,  2.30s/it]
 30%|███       | 907/2975 [39:25<1:19:25,  2.30s/it]
 31%|███       | 908/2975 [3
0: {'loss': 0.4374, 'grad_norm': 0.5407483804115147, 'learning_rate': 3e-06, 'epoch': 0.31}
0: {'loss': 0.4267, 'grad_norm': 0.474936803729911, 'learning_rate': 3e-06, 'epoch': 0.31}
0: 9:27<1:19:14,  2.30s/it]
 31%|███       | 909/2975 [39:29<1:19:04,  2.30s/it]
 31%|███       | 910/2975 [39:32<1:21:29,  2.37s/it]
                                                    

 31%|███       | 910/2975 [39:32<1:21:29,  2.37s/it]
 31%|███       | 911/2975 [39:34<1:20:44,  2.35s/it]
 31%|███       | 912/2975 [39:37<1:20:02,  2.33s/it]
 31%|███       | 913/2975 [39:39<1:19:54,  2.33s/it]
 31%|███       | 914/2975 [39:41<1:19:34,  2.32s/it]
 31%|███       | 915/2975 [39:43<1:19:20,  2.31s/it]
 31%|███       | 916/2975 [39:46<1:18:44,  2.29s/it]
 31%|███       | 917/2975 [39:48<1:18:37,  2.29s/it]
 31%|███       | 918/2975 [39:50<1:18:29,  2.29s/it]
 31%|███       | 919/2975 [39:53<1:18:27,  2.29s/it]
 31%|███       | 920/2975 [39:55<1:18:36,  2.29s/it]
                                                    

 31%|███       | 920/2975 [39:55<1:18:36,  2.29s/it]
 31%|███       | 921/2975 [39:57<1:18:32,  2.29s/it]
 31%|�
0: {'loss': 0.445, 'grad_norm': 0.49732813056462455, 'learning_rate': 3e-06, 'epoch': 0.31}
0: ��██       | 922/2975 [39:59<1:18:27,  2.29s/it]
 31%|███       | 923/2975 [40:02<1:23:11,  2.43s/it]
 31%|███       | 924/2975 [40:04<1:21:39,  2.39s/it]
 31%|███       | 925/2975 [40:07<1:20:32,  2.36s/it]
 31%|███       | 926/2975 [40:09<1:19:58,  2.34s/it]
 31%|███       | 927/2975 [40:11<1:19:45,  2.34s/it]
 31%|███       | 928/2975 [40:14<1:19:16,  2.32s/it]
 31%|███       | 929/2975 [40:16<1:18:58,  2.32s/it]
 31%|███▏      | 930/2975 [40:18<1:18:40,  2.31s/it]
                                                    

 31%|███▏      | 930/2975 [40:18<1:18:40,  2.31s/it]
 31%|███▏      | 931/2975 [40:21<1:18:40,  2.31s/it]
 31%|███▏      | 932/2975 [40:23<1:18:33,  2.31s/it]
 31%|███▏      | 933/2975 [40:25<1:18:41,  2.31s/it]
 31%|███▏      | 934/2975 [40:28<1:18:46,  2.32s/it]
 31%|███▏      | 935/2975 [40:30<1:18:36,  2.31s/it]
 31%|███▏      | 936/2975 [40:32<1:18:11,  2.30s/it]
 31%|███�
0: {'loss': 0.437, 'grad_norm': 0.4654293791913678, 'learning_rate': 3e-06, 'epoch': 0.32}
0: {'loss': 0.4478, 'grad_norm': 0.47986740758028557, 'learning_rate': 3e-06, 'epoch': 0.32}
0: �      | 937/2975 [40:34<1:18:00,  2.30s/it]
 32%|███▏      | 938/2975 [40:37<1:18:02,  2.30s/it]
 32%|███▏      | 939/2975 [40:39<1:17:57,  2.30s/it]
 32%|███▏      | 940/2975 [40:41<1:18:59,  2.33s/it]
                                                    

 32%|███▏      | 940/2975 [40:41<1:18:59,  2.33s/it]
 32%|███▏      | 941/2975 [40:44<1:18:30,  2.32s/it]
 32%|███▏      | 942/2975 [40:46<1:18:12,  2.31s/it]
 32%|███▏      | 943/2975 [40:48<1:18:02,  2.30s/it]
 32%|███▏      | 944/2975 [40:51<1:18:18,  2.31s/it]
 32%|███▏      | 945/2975 [40:53<1:18:07,  2.31s/it]
 32%|███▏      | 946/2975 [40:55<1:17:53,  2.30s/it]
 32%|███▏      | 947/2975 [40:58<1:24:35,  2.50s/it]
 32%|███▏      | 948/2975 [41:00<1:22:37,  2.45s/it]
 32%|███▏      | 949/2975 [41:03<1:22:46,  2.45s/it]
 32%|███▏      | 950/2975 [41:05<1:21:01,  2.40s/it]
                                                    

 32%|███▏
0: {'loss': 0.438, 'grad_norm': 0.481247574193838, 'learning_rate': 3e-06, 'epoch': 0.32}
0:       | 950/2975 [41:05<1:21:01,  2.40s/it]
 32%|███▏      | 951/2975 [41:08<1:21:39,  2.42s/it]
 32%|███▏      | 952/2975 [41:10<1:20:05,  2.38s/it]
 32%|███▏      | 953/2975 [41:12<1:19:11,  2.35s/it]
 32%|███▏      | 954/2975 [41:15<1:18:57,  2.34s/it]
 32%|███▏      | 955/2975 [41:17<1:18:24,  2.33s/it]
 32%|███▏      | 956/2975 [41:19<1:18:01,  2.32s/it]
 32%|███▏      | 957/2975 [41:21<1:17:39,  2.31s/it]
 32%|███▏      | 958/2975 [41:24<1:17:30,  2.31s/it]
 32%|███▏      | 959/2975 [41:26<1:17:58,  2.32s/it]
 32%|███▏      | 960/2975 [41:28<1:17:44,  2.32s/it]
                                                    

 32%|███▏      | 960/2975 [41:28<1:17:44,  2.32s/it]
 32%|███▏      | 961/2975 [41:31<1:22:08,  2.45s/it]
 32%|███▏      | 962/2975 [41:33<1:20:21,  2.40s/it]
 32%|███▏      | 963/2975 [41:36<1:23:53,  2.50s/it]
 32%|███▏      | 964/2975 [41:39<1:23:40,  2.50s/it]
 32%|██
0: {'loss': 0.4386, 'grad_norm': 0.48682734230890734, 'learning_rate': 3e-06, 'epoch': 0.33}
0: █▏      | 965/2975 [41:41<1:21:38,  2.44s/it]
 32%|███▏      | 966/2975 [41:43<1:20:32,  2.41s/it]
 33%|███▎      | 967/2975 [41:46<1:19:21,  2.37s/it]
 33%|███▎      | 968/2975 [41:48<1:18:29,  2.35s/it]
 33%|███▎      | 969/2975 [41:50<1:17:54,  2.33s/it]
 33%|███▎      | 970/2975 [41:52<1:17:31,  2.32s/it]
                                                    

 33%|███▎      | 970/2975 [41:52<1:17:31,  2.32s/it]
 33%|███▎      | 971/2975 [41:55<1:18:05,  2.34s/it]
 33%|███▎      | 972/2975 [41:57<1:17:44,  2.33s/it]
 33%|███▎      | 973/2975 [41:59<1:17:24,  2.32s/it]
 33%|███▎      | 974/2975 [42:02<1:18:53,  2.37s/it]
 33%|███▎      | 975/2975 [42:04<1:18:03,  2.34s/it]
 33%|███▎      | 976/2975 [42:07<1:17:42,  2.33s/it]
 33%|███▎      | 977/2975 [42:09<1:16:56,  2.31s/it]
 33%|███▎      | 978/2975 [42:11<1:17:59,  2.34s/it]
 33%|███▎      | 979/2975 [42:14<1:17:26,  2.33s/it]
 33%|
0: {'loss': 0.4388, 'grad_norm': 0.4678458505276334, 'learning_rate': 3e-06, 'epoch': 0.33}
0: {'loss': 0.4366, 'grad_norm': 0.48564107444981697, 'learning_rate': 3e-06, 'epoch': 0.33}
0: ███▎      | 980/2975 [42:16<1:16:57,  2.31s/it]
                                                    

 33%|███▎      | 980/2975 [42:16<1:16:57,  2.31s/it]
 33%|███▎      | 981/2975 [42:18<1:16:39,  2.31s/it]
 33%|███▎      | 982/2975 [42:20<1:16:32,  2.30s/it]
 33%|███▎      | 983/2975 [42:23<1:16:39,  2.31s/it]
 33%|███▎      | 984/2975 [42:25<1:16:20,  2.30s/it]
 33%|███▎      | 985/2975 [42:27<1:16:04,  2.29s/it]
 33%|███▎      | 986/2975 [42:30<1:16:43,  2.31s/it]
 33%|███▎      | 987/2975 [42:32<1:16:35,  2.31s/it]
 33%|███▎      | 988/2975 [42:34<1:16:22,  2.31s/it]
 33%|███▎      | 989/2975 [42:36<1:15:54,  2.29s/it]
 33%|███▎      | 990/2975 [42:39<1:15:46,  2.29s/it]
                                                    

 33%|███▎      | 990/2975 [42:39<1:15:46,  2.29s/it]
 33%|███▎      | 991/2975 [42:41<1:15:56,  2.30s/it]
 33%|███▎      | 992/2975 [42:43<1:15:51,  2.30s/it]
 33%|�
0: {'loss': 0.4304, 'grad_norm': 0.49241645209571544, 'learning_rate': 3e-06, 'epoch': 0.34}
0: ��██▎      | 993/2975 [42:46<1:17:14,  2.34s/it]
 33%|███▎      | 994/2975 [42:48<1:16:44,  2.32s/it]
 33%|███▎      | 995/2975 [42:51<1:18:03,  2.37s/it]
 33%|███▎      | 996/2975 [42:53<1:17:16,  2.34s/it]
 34%|███▎      | 997/2975 [42:55<1:16:38,  2.33s/it]
 34%|███▎      | 998/2975 [42:57<1:16:11,  2.31s/it]
 34%|███▎      | 999/2975 [43:00<1:16:57,  2.34s/it]
 34%|███▎      | 1000/2975 [43:02<1:18:41,  2.39s/it]
                                                     

 34%|███▎      | 1000/2975 [43:02<1:18:41,  2.39s/it]
 34%|███▎      | 1001/2975 [43:05<1:18:15,  2.38s/it]
 34%|███▎      | 1002/2975 [43:07<1:17:38,  2.36s/it]
 34%|███▎      | 1003/2975 [43:09<1:16:52,  2.34s/it]
 34%|███▎      | 1004/2975 [43:12<1:16:22,  2.33s/it]
 34%|███▍      | 1005/2975 [43:14<1:16:21,  2.33s/it]
 34%|███▍      | 1006/2975 [43:16<1:16:01,  2.32s/it]
 34%|███▍      | 1007/2975 [43:18<1:15:36,  
0: {'loss': 0.4307, 'grad_norm': 0.4784706831192375, 'learning_rate': 3e-06, 'epoch': 0.34}
0: {'loss': 0.4344, 'grad_norm': 0.47368755200976964, 'learning_rate': 3e-06, 'epoch': 0.34}
0: 2.31s/it]
 34%|███▍      | 1008/2975 [43:21<1:15:22,  2.30s/it]
 34%|███▍      | 1009/2975 [43:23<1:15:23,  2.30s/it]
 34%|███▍      | 1010/2975 [43:25<1:15:18,  2.30s/it]
                                                     

 34%|███▍      | 1010/2975 [43:25<1:15:18,  2.30s/it]
 34%|███▍      | 1011/2975 [43:28<1:15:52,  2.32s/it]
 34%|███▍      | 1012/2975 [43:30<1:15:33,  2.31s/it]
 34%|███▍      | 1013/2975 [43:32<1:15:26,  2.31s/it]
 34%|███▍      | 1014/2975 [43:35<1:17:12,  2.36s/it]
 34%|███▍      | 1015/2975 [43:37<1:16:26,  2.34s/it]
 34%|███▍      | 1016/2975 [43:39<1:16:22,  2.34s/it]
 34%|███▍      | 1017/2975 [43:42<1:15:51,  2.32s/it]
 34%|███▍      | 1018/2975 [43:44<1:15:33,  2.32s/it]
 34%|███▍      | 1019/2975 [43:46<1:15:29,  2.32s/it]
 34%|███▍      | 1020/2975 [43:49<1:15:28,  2.32s/it]
                                                     

 34%|███▍      | 1020/2975 [
0: {'loss': 0.4342, 'grad_norm': 0.4662442914531778, 'learning_rate': 3e-06, 'epoch': 0.35}
0: 43:49<1:15:28,  2.32s/it]
 34%|███▍      | 1021/2975 [43:51<1:15:15,  2.31s/it]
 34%|███▍      | 1022/2975 [43:53<1:15:05,  2.31s/it]
 34%|███▍      | 1023/2975 [43:56<1:14:46,  2.30s/it]
 34%|███▍      | 1024/2975 [43:58<1:14:34,  2.29s/it]
 34%|███▍      | 1025/2975 [44:01<1:19:32,  2.45s/it]
 34%|███▍      | 1026/2975 [44:03<1:17:55,  2.40s/it]
 35%|███▍      | 1027/2975 [44:05<1:16:51,  2.37s/it]
 35%|███▍      | 1028/2975 [44:07<1:16:11,  2.35s/it]
 35%|███▍      | 1029/2975 [44:10<1:22:13,  2.53s/it]
 35%|███▍      | 1030/2975 [44:13<1:24:08,  2.60s/it]
                                                     

 35%|███▍      | 1030/2975 [44:13<1:24:08,  2.60s/it]
 35%|███▍      | 1031/2975 [44:15<1:21:05,  2.50s/it]
 35%|███▍      | 1032/2975 [44:18<1:18:58,  2.44s/it]
 35%|███▍      | 1033/2975 [44:20<1:18:24,  2.42s/it]
 35%|███▍      | 1034/2975 [44:22<1:17:27,  2.39s/it]
 35%|██�
0: {'loss': 0.4227, 'grad_norm': 0.4722392797979577, 'learning_rate': 3e-06, 'epoch': 0.35}
0: �▍      | 1035/2975 [44:25<1:16:41,  2.37s/it]
 35%|███▍      | 1036/2975 [44:27<1:15:54,  2.35s/it]
 35%|███▍      | 1037/2975 [44:29<1:15:14,  2.33s/it]
 35%|███▍      | 1038/2975 [44:32<1:14:46,  2.32s/it]
 35%|███▍      | 1039/2975 [44:34<1:14:26,  2.31s/it]
 35%|███▍      | 1040/2975 [44:36<1:14:50,  2.32s/it]
                                                     

 35%|███▍      | 1040/2975 [44:36<1:14:50,  2.32s/it]
 35%|███▍      | 1041/2975 [44:39<1:14:39,  2.32s/it]
 35%|███▌      | 1042/2975 [44:41<1:14:45,  2.32s/it]
 35%|███▌      | 1043/2975 [44:43<1:14:59,  2.33s/it]
 35%|███▌      | 1044/2975 [44:46<1:14:57,  2.33s/it]
 35%|███▌      | 1045/2975 [44:48<1:14:09,  2.31s/it]
 35%|███▌      | 1046/2975 [44:50<1:14:07,  2.31s/it]
 35%|███▌      | 1047/2975 [44:52<1:13:57,  2.30s/it]
 35%|███▌      | 1048/2975 [44:55<1:15:33,  2.35s/it]
 35%|███▌      | 1049/2975 [44:57<1:14:53,  
0: {'loss': 0.4352, 'grad_norm': 0.4774442141668582, 'learning_rate': 3e-06, 'epoch': 0.35}
0: {'loss': 0.4401, 'grad_norm': 0.48415308770214666, 'learning_rate': 3e-06, 'epoch': 0.36}
0: 2.33s/it]
 35%|███▌      | 1050/2975 [45:00<1:14:56,  2.34s/it]
                                                     

 35%|███▌      | 1050/2975 [45:00<1:14:56,  2.34s/it]
 35%|███▌      | 1051/2975 [45:02<1:14:23,  2.32s/it]
 35%|███▌      | 1052/2975 [45:04<1:13:42,  2.30s/it]
 35%|███▌      | 1053/2975 [45:06<1:13:37,  2.30s/it]
 35%|███▌      | 1054/2975 [45:09<1:15:58,  2.37s/it]
 35%|███▌      | 1055/2975 [45:11<1:14:50,  2.34s/it]
 35%|███▌      | 1056/2975 [45:13<1:14:15,  2.32s/it]
 36%|███▌      | 1057/2975 [45:16<1:14:15,  2.32s/it]
 36%|███▌      | 1058/2975 [45:18<1:13:50,  2.31s/it]
 36%|███▌      | 1059/2975 [45:20<1:13:34,  2.30s/it]
 36%|███▌      | 1060/2975 [45:23<1:13:26,  2.30s/it]
                                                     

 36%|███▌      | 1060/2975 [45:23<1:13:26,  2.30s/it]
 36%|███▌      | 1061/2975 [45:25<1:13:25,  2.30s/it]
 36%|███▌      | 1062/2975 [
0: {'loss': 0.4329, 'grad_norm': 0.48020334071196685, 'learning_rate': 3e-06, 'epoch': 0.36}
0: 45:27<1:13:15,  2.30s/it]
 36%|███▌      | 1063/2975 [45:30<1:13:08,  2.30s/it]
 36%|███▌      | 1064/2975 [45:32<1:13:08,  2.30s/it]
 36%|███▌      | 1065/2975 [45:34<1:14:53,  2.35s/it]
 36%|███▌      | 1066/2975 [45:37<1:14:32,  2.34s/it]
 36%|███▌      | 1067/2975 [45:39<1:16:58,  2.42s/it]
 36%|███▌      | 1068/2975 [45:42<1:15:40,  2.38s/it]
 36%|███▌      | 1069/2975 [45:44<1:14:52,  2.36s/it]
 36%|███▌      | 1070/2975 [45:46<1:14:22,  2.34s/it]
                                                     

 36%|███▌      | 1070/2975 [45:46<1:14:22,  2.34s/it]
 36%|███▌      | 1071/2975 [45:49<1:14:58,  2.36s/it]
 36%|███▌      | 1072/2975 [45:51<1:14:16,  2.34s/it]
 36%|███▌      | 1073/2975 [45:53<1:14:36,  2.35s/it]
 36%|███▌      | 1074/2975 [45:56<1:13:56,  2.33s/it]
 36%|███▌      | 1075/2975 [45:58<1:13:21,  2.32s/it]
 36%|███▌      | 1076/2975 [46:00<1:13:05,  2.31s/it]
 36%|██�
0: {'loss': 0.4278, 'grad_norm': 0.4805934417220243, 'learning_rate': 3e-06, 'epoch': 0.36}
0: �▌      | 1077/2975 [46:02<1:12:54,  2.30s/it]
 36%|███▌      | 1078/2975 [46:05<1:12:42,  2.30s/it]
 36%|███▋      | 1079/2975 [46:07<1:12:41,  2.30s/it]
 36%|███▋      | 1080/2975 [46:09<1:12:33,  2.30s/it]
                                                     

 36%|███▋      | 1080/2975 [46:09<1:12:33,  2.30s/it]
 36%|███▋      | 1081/2975 [46:12<1:13:26,  2.33s/it]
 36%|███▋      | 1082/2975 [46:14<1:13:57,  2.34s/it]
 36%|███▋      | 1083/2975 [46:16<1:13:17,  2.32s/it]
 36%|███▋      | 1084/2975 [46:19<1:12:50,  2.31s/it]
 36%|███▋      | 1085/2975 [46:21<1:12:35,  2.30s/it]
 37%|███▋      | 1086/2975 [46:23<1:12:25,  2.30s/it]
 37%|███▋      | 1087/2975 [46:25<1:12:14,  2.30s/it]
 37%|███▋      | 1088/2975 [46:28<1:12:06,  2.29s/it]
 37%|███▋      | 1089/2975 [46:30<1:12:08,  2.30s/it]
 37%|███▋      | 1090/2975 [46:32<1:12:31,  2.31s/it]
                                                    
0: {'loss': 0.4304, 'grad_norm': 0.47524658305451983, 'learning_rate': 3e-06, 'epoch': 0.37}
0: {'loss': 0.4485, 'grad_norm': 0.462654469002214, 'learning_rate': 3e-06, 'epoch': 0.37}
0:  

 37%|███▋      | 1090/2975 [46:32<1:12:31,  2.31s/it]
 37%|███▋      | 1091/2975 [46:35<1:14:57,  2.39s/it]
 37%|███▋      | 1092/2975 [46:37<1:14:01,  2.36s/it]
 37%|███▋      | 1093/2975 [46:40<1:13:26,  2.34s/it]
 37%|███▋      | 1094/2975 [46:42<1:13:03,  2.33s/it]
 37%|███▋      | 1095/2975 [46:44<1:12:42,  2.32s/it]
 37%|███▋      | 1096/2975 [46:46<1:12:20,  2.31s/it]
 37%|███▋      | 1097/2975 [46:49<1:12:08,  2.30s/it]
 37%|███▋      | 1098/2975 [46:51<1:11:53,  2.30s/it]
 37%|███▋      | 1099/2975 [46:53<1:11:43,  2.29s/it]
 37%|███▋      | 1100/2975 [46:56<1:11:40,  2.29s/it]
                                                     

 37%|███▋      | 1100/2975 [46:56<1:11:40,  2.29s/it]
 37%|███▋      | 1101/2975 [46:58<1:12:10,  2.31s/it]
 37%|███▋      | 1102/2975 [47:00<1:12:05,  2.31s/it]
 37%|███▋      | 1103/2975 [47:03<1:11:49,  2.30s/it]
 37%|███▋      | 1104/2975 [
0: {'loss': 0.4402, 'grad_norm': 0.4824316991937445, 'learning_rate': 3e-06, 'epoch': 0.37}
0: 47:05<1:13:33,  2.36s/it]
 37%|███▋      | 1105/2975 [47:07<1:12:57,  2.34s/it]
 37%|███▋      | 1106/2975 [47:10<1:12:28,  2.33s/it]
 37%|███▋      | 1107/2975 [47:12<1:14:45,  2.40s/it]
 37%|███▋      | 1108/2975 [47:15<1:13:49,  2.37s/it]
 37%|███▋      | 1109/2975 [47:17<1:13:02,  2.35s/it]
 37%|███▋      | 1110/2975 [47:19<1:12:31,  2.33s/it]
                                                     

 37%|███▋      | 1110/2975 [47:19<1:12:31,  2.33s/it]
 37%|███▋      | 1111/2975 [47:21<1:12:45,  2.34s/it]
 37%|███▋      | 1112/2975 [47:24<1:12:09,  2.32s/it]
 37%|███▋      | 1113/2975 [47:26<1:11:53,  2.32s/it]
 37%|███▋      | 1114/2975 [47:28<1:11:46,  2.31s/it]
 37%|███▋      | 1115/2975 [47:31<1:12:03,  2.32s/it]
 38%|███▊      | 1116/2975 [47:33<1:11:42,  2.31s/it]
 38%|███▊      | 1117/2975 [47:35<1:11:26,  2.31s/it]
 38%|███▊      | 1118/2975 [47:38<1:12:59,  2.36s/it]
 38%|██�
0: {'loss': 0.4323, 'grad_norm': 0.4851279421667443, 'learning_rate': 3e-06, 'epoch': 0.38}
0: {'loss': 0.4304, 'grad_norm': 0.46468907986472585, 'learning_rate': 3e-06, 'epoch': 0.38}
0: �▊      | 1119/2975 [47:40<1:12:09,  2.33s/it]
 38%|███▊      | 1120/2975 [47:43<1:13:37,  2.38s/it]
                                                     

 38%|███▊      | 1120/2975 [47:43<1:13:37,  2.38s/it]
 38%|███▊      | 1121/2975 [47:45<1:13:42,  2.39s/it]
 38%|███▊      | 1122/2975 [47:47<1:12:42,  2.35s/it]
 38%|███▊      | 1123/2975 [47:50<1:12:09,  2.34s/it]
 38%|███▊      | 1124/2975 [47:52<1:11:39,  2.32s/it]
 38%|███▊      | 1125/2975 [47:54<1:11:27,  2.32s/it]
 38%|███▊      | 1126/2975 [47:56<1:11:13,  2.31s/it]
 38%|███▊      | 1127/2975 [47:59<1:10:56,  2.30s/it]
 38%|███▊      | 1128/2975 [48:01<1:11:07,  2.31s/it]
 38%|███▊      | 1129/2975 [48:03<1:10:51,  2.30s/it]
 38%|███▊      | 1130/2975 [48:06<1:12:22,  2.35s/it]
                                                     

 38%|███▊      | 1130/2975 [48:06<1:12:22,  2.35s/it]
 38%|███▊      | 1131/2975 [48:08<1:11:46,  2.34s/i
0: {'loss': 0.4346, 'grad_norm': 0.46347319052212044, 'learning_rate': 3e-06, 'epoch': 0.38}
0: t]
 38%|███▊      | 1132/2975 [48:10<1:11:34,  2.33s/it]
 38%|███▊      | 1133/2975 [48:13<1:11:05,  2.32s/it]
 38%|███▊      | 1134/2975 [48:15<1:11:33,  2.33s/it]
 38%|███▊      | 1135/2975 [48:17<1:11:13,  2.32s/it]
 38%|███▊      | 1136/2975 [48:20<1:10:56,  2.31s/it]
 38%|███▊      | 1137/2975 [48:22<1:10:42,  2.31s/it]
 38%|███▊      | 1138/2975 [48:24<1:10:31,  2.30s/it]
 38%|███▊      | 1139/2975 [48:27<1:10:20,  2.30s/it]
 38%|███▊      | 1140/2975 [48:29<1:10:16,  2.30s/it]
                                                     

 38%|███▊      | 1140/2975 [48:29<1:10:16,  2.30s/it]
 38%|███▊      | 1141/2975 [48:31<1:10:10,  2.30s/it]
 38%|███▊      | 1142/2975 [48:33<1:10:15,  2.30s/it]
 38%|███▊      | 1143/2975 [48:36<1:10:09,  2.30s/it]
 38%|███▊      | 1144/2975 [48:38<1:10:06,  2.30s/it]
 38%|███▊      | 1145/2975 [48:40<1:10:07,  2.30s/it]
 39%|███▊      | 1146/2975 [
0: {'loss': 0.4357, 'grad_norm': 0.4833487359211495, 'learning_rate': 3e-06, 'epoch': 0.39}
0: 48:43<1:09:55,  2.29s/it]
 39%|███▊      | 1147/2975 [48:45<1:09:45,  2.29s/it]
 39%|███▊      | 1148/2975 [48:47<1:09:40,  2.29s/it]
 39%|███▊      | 1149/2975 [48:50<1:11:08,  2.34s/it]
 39%|███▊      | 1150/2975 [48:52<1:10:30,  2.32s/it]
                                                     

 39%|███▊      | 1150/2975 [48:52<1:10:30,  2.32s/it]
 39%|███▊      | 1151/2975 [48:54<1:10:14,  2.31s/it]
 39%|███▊      | 1152/2975 [48:56<1:10:04,  2.31s/it]
 39%|███▉      | 1153/2975 [48:59<1:09:50,  2.30s/it]
 39%|███▉      | 1154/2975 [49:01<1:11:42,  2.36s/it]
 39%|███▉      | 1155/2975 [49:04<1:11:14,  2.35s/it]
 39%|███▉      | 1156/2975 [49:06<1:10:35,  2.33s/it]
 39%|███▉      | 1157/2975 [49:08<1:10:19,  2.32s/it]
 39%|███▉      | 1158/2975 [49:10<1:09:56,  2.31s/it]
 39%|███▉      | 1159/2975 [49:13<1:09:44,  2.30s/it]
 39%|███▉      | 1160/2975 [49:15<1:09:40,  2.30s/it]
             
0: {'loss': 0.435, 'grad_norm': 0.4925189972105456, 'learning_rate': 3e-06, 'epoch': 0.39}
0: {'loss': 0.4348, 'grad_norm': 0.48454164129386396, 'learning_rate': 3e-06, 'epoch': 0.39}
0:                                         

 39%|███▉      | 1160/2975 [49:15<1:09:40,  2.30s/it]
 39%|███▉      | 1161/2975 [49:17<1:09:25,  2.30s/it]
 39%|███▉      | 1162/2975 [49:20<1:09:16,  2.29s/it]
 39%|███▉      | 1163/2975 [49:22<1:09:11,  2.29s/it]
 39%|███▉      | 1164/2975 [49:24<1:09:31,  2.30s/it]
 39%|███▉      | 1165/2975 [49:27<1:09:17,  2.30s/it]
 39%|███▉      | 1166/2975 [49:29<1:09:15,  2.30s/it]
 39%|███▉      | 1167/2975 [49:31<1:09:17,  2.30s/it]
 39%|███▉      | 1168/2975 [49:34<1:10:15,  2.33s/it]
 39%|███▉      | 1169/2975 [49:36<1:09:51,  2.32s/it]
 39%|███▉      | 1170/2975 [49:38<1:09:33,  2.31s/it]
                                                     

 39%|███▉      | 1170/2975 [49:38<1:09:33,  2.31s/it]
 39%|███▉      | 1171/2975 [49:40<1:09:58,  2.33s/it]
 39%|███▉      | 1172/2975 [49:43<1:09:25,  2.31s/it]
 39%|███▉      | 1173/2975 [49:45<1:09:07,  2.30s/i
0: {'loss': 0.4362, 'grad_norm': 0.49503692220705686, 'learning_rate': 3e-06, 'epoch': 0.4}
0: t]
 39%|███▉      | 1174/2975 [49:47<1:08:55,  2.30s/it]
 39%|███▉      | 1175/2975 [49:50<1:08:41,  2.29s/it]
 40%|███▉      | 1176/2975 [49:52<1:08:43,  2.29s/it]
 40%|███▉      | 1177/2975 [49:54<1:08:31,  2.29s/it]
 40%|███▉      | 1178/2975 [49:56<1:08:32,  2.29s/it]
 40%|███▉      | 1179/2975 [49:59<1:08:24,  2.29s/it]
 40%|███▉      | 1180/2975 [50:01<1:08:41,  2.30s/it]
                                                     

 40%|███▉      | 1180/2975 [50:01<1:08:41,  2.30s/it]
 40%|███▉      | 1181/2975 [50:03<1:08:36,  2.29s/it]
 40%|███▉      | 1182/2975 [50:06<1:08:29,  2.29s/it]
 40%|███▉      | 1183/2975 [50:08<1:08:46,  2.30s/it]
 40%|███▉      | 1184/2975 [50:10<1:08:28,  2.29s/it]
 40%|███▉      | 1185/2975 [50:12<1:08:14,  2.29s/it]
 40%|███▉      | 1186/2975 [50:15<1:08:09,  2.29s/it]
 40%|███▉      | 1187/2975 [50:17<1:08:04,  2.28s/it]
 40%|███▉      | 1188/2975 [
0: {'loss': 0.4308, 'grad_norm': 0.5258417035827336, 'learning_rate': 3e-06, 'epoch': 0.4}
0: {'loss': 0.4276, 'grad_norm': 0.48173200589468174, 'learning_rate': 3e-06, 'epoch': 0.4}
0: 50:19<1:08:42,  2.31s/it]
 40%|███▉      | 1189/2975 [50:22<1:08:29,  2.30s/it]
 40%|████      | 1190/2975 [50:24<1:10:24,  2.37s/it]
                                                     

 40%|████      | 1190/2975 [50:24<1:10:24,  2.37s/it]
 40%|████      | 1191/2975 [50:58<5:48:51, 11.73s/it]
 40%|████      | 1192/2975 [51:00<4:24:24,  8.90s/it]
 40%|████      | 1193/2975 [51:02<3:25:23,  6.92s/it]
 40%|████      | 1194/2975 [51:05<2:44:16,  5.53s/it]
 40%|████      | 1195/2975 [51:07<2:15:16,  4.56s/it]
 40%|████      | 1196/2975 [51:09<1:55:00,  3.88s/it]
 40%|████      | 1197/2975 [51:12<1:42:43,  3.47s/it]
 40%|████      | 1198/2975 [51:14<1:32:14,  3.11s/it]
 40%|████      | 1199/2975 [51:16<1:24:49,  2.87s/it]
 40%|████      | 1200/2975 [51:19<1:19:35,  2.69s/it]
                                                     

 40%|████      | 1200/2975 [51:19<1:19:35,  2.69s/it]
 40%|████   
0: {'loss': 0.431, 'grad_norm': 0.48881223750978037, 'learning_rate': 3e-06, 'epoch': 0.41}
0:    | 1201/2975 [51:21<1:15:50,  2.56s/it]
 40%|████      | 1202/2975 [51:23<1:13:28,  2.49s/it]
 40%|████      | 1203/2975 [51:25<1:11:27,  2.42s/it]
 40%|████      | 1204/2975 [51:28<1:10:42,  2.40s/it]
 41%|████      | 1205/2975 [51:30<1:09:50,  2.37s/it]
 41%|████      | 1206/2975 [51:32<1:09:06,  2.34s/it]
 41%|████      | 1207/2975 [51:35<1:08:31,  2.33s/it]
 41%|████      | 1208/2975 [51:37<1:08:11,  2.32s/it]
 41%|████      | 1209/2975 [51:39<1:07:54,  2.31s/it]
 41%|████      | 1210/2975 [51:42<1:07:43,  2.30s/it]
                                                     

 41%|████      | 1210/2975 [51:42<1:07:43,  2.30s/it]
 41%|████      | 1211/2975 [51:44<1:08:07,  2.32s/it]
 41%|████      | 1212/2975 [51:46<1:07:40,  2.30s/it]
 41%|████      | 1213/2975 [51:48<1:07:23,  2.30s/it]
 41%|████      | 1214/2975 [51:51<1:07:56,  2.31s/it]
 41%|████      | 1215/2975 [51:53<1:08:22,  2.33s/i
0: {'loss': 0.4334, 'grad_norm': 0.48210921135458945, 'learning_rate': 3e-06, 'epoch': 0.41}
0: t]
 41%|████      | 1216/2975 [51:55<1:08:03,  2.32s/it]
 41%|████      | 1217/2975 [51:58<1:07:41,  2.31s/it]
 41%|████      | 1218/2975 [52:00<1:08:54,  2.35s/it]
 41%|████      | 1219/2975 [52:02<1:08:14,  2.33s/it]
 41%|████      | 1220/2975 [52:05<1:07:58,  2.32s/it]
                                                     

 41%|████      | 1220/2975 [52:05<1:07:58,  2.32s/it]
 41%|████      | 1221/2975 [52:07<1:07:40,  2.32s/it]
 41%|████      | 1222/2975 [52:09<1:07:18,  2.30s/it]
 41%|████      | 1223/2975 [52:12<1:07:32,  2.31s/it]
 41%|████      | 1224/2975 [52:14<1:07:09,  2.30s/it]
 41%|████      | 1225/2975 [52:16<1:07:19,  2.31s/it]
 41%|████      | 1226/2975 [52:19<1:07:08,  2.30s/it]
 41%|████      | 1227/2975 [52:21<1:06:54,  2.30s/it]
 41%|████▏     | 1228/2975 [52:23<1:07:06,  2.30s/it]
 41%|████▏     | 1229/2975 [52:25<1:06:51,  2.30s/it]
 41%|████▏     | 1230/
0: {'loss': 0.4301, 'grad_norm': 0.4986002324103987, 'learning_rate': 3e-06, 'epoch': 0.41}
0: {'loss': 0.4369, 'grad_norm': 0.5641394689789232, 'learning_rate': 3e-06, 'epoch': 0.42}
0: 2975 [52:28<1:06:42,  2.29s/it]
                                                     

 41%|████▏     | 1230/2975 [52:28<1:06:42,  2.29s/it]
 41%|████▏     | 1231/2975 [52:30<1:06:38,  2.29s/it]
 41%|████▏     | 1232/2975 [52:32<1:06:30,  2.29s/it]
 41%|████▏     | 1233/2975 [52:35<1:06:26,  2.29s/it]
 41%|████▏     | 1234/2975 [52:37<1:06:17,  2.28s/it]
 42%|████▏     | 1235/2975 [52:39<1:06:22,  2.29s/it]
 42%|████▏     | 1236/2975 [52:42<1:06:27,  2.29s/it]
 42%|████▏     | 1237/2975 [52:44<1:06:27,  2.29s/it]
 42%|████▏     | 1238/2975 [52:46<1:06:22,  2.29s/it]
 42%|████▏     | 1239/2975 [52:48<1:06:17,  2.29s/it]
 42%|████▏     | 1240/2975 [52:51<1:10:20,  2.43s/it]
                                                     

 42%|████▏     | 1240/2975 [52:51<1:10:20,  2.43s/it]
 42%|████▏     | 1241/2975 [52:53<1:09:09,  2.39s/it]
 42%|████▏     | 1242/2975 [52:56<1:08:0
0: {'loss': 0.4474, 'grad_norm': 0.474600637460658, 'learning_rate': 3e-06, 'epoch': 0.42}
0: 2,  2.36s/it]
 42%|████▏     | 1243/2975 [52:58<1:07:39,  2.34s/it]
 42%|████▏     | 1244/2975 [53:00<1:07:02,  2.32s/it]
 42%|████▏     | 1245/2975 [53:03<1:06:56,  2.32s/it]
 42%|████▏     | 1246/2975 [53:05<1:06:35,  2.31s/it]
 42%|████▏     | 1247/2975 [53:07<1:06:25,  2.31s/it]
 42%|████▏     | 1248/2975 [53:09<1:06:05,  2.30s/it]
 42%|████▏     | 1249/2975 [53:12<1:06:11,  2.30s/it]
 42%|████▏     | 1250/2975 [53:14<1:06:30,  2.31s/it]
                                                     

 42%|████▏     | 1250/2975 [53:14<1:06:30,  2.31s/it]
 42%|████▏     | 1251/2975 [53:16<1:06:29,  2.31s/it]
 42%|████▏     | 1252/2975 [53:19<1:06:10,  2.30s/it]
 42%|████▏     | 1253/2975 [53:21<1:06:02,  2.30s/it]
 42%|████▏     | 1254/2975 [53:23<1:05:50,  2.30s/it]
 42%|████▏     | 1255/2975 [53:26<1:05:44,  2.29s/it]
 42%|████▏     | 1256/2975 [53:28<1:05:38,  2.29s
0: {'loss': 0.44, 'grad_norm': 0.4836126093179131, 'learning_rate': 3e-06, 'epoch': 0.42}
0: /it]
 42%|████▏     | 1257/2975 [53:30<1:05:35,  2.29s/it]
 42%|████▏     | 1258/2975 [53:32<1:05:49,  2.30s/it]
 42%|████▏     | 1259/2975 [53:35<1:06:42,  2.33s/it]
 42%|████▏     | 1260/2975 [53:37<1:06:33,  2.33s/it]
                                                     

 42%|████▏     | 1260/2975 [53:37<1:06:33,  2.33s/it]
 42%|████▏     | 1261/2975 [53:40<1:06:13,  2.32s/it]
 42%|████▏     | 1262/2975 [53:42<1:06:20,  2.32s/it]
 42%|████▏     | 1263/2975 [53:44<1:06:00,  2.31s/it]
 42%|████▏     | 1264/2975 [53:46<1:05:53,  2.31s/it]
 43%|████▎     | 1265/2975 [53:49<1:05:37,  2.30s/it]
 43%|████▎     | 1266/2975 [53:51<1:05:38,  2.30s/it]
 43%|████▎     | 1267/2975 [53:53<1:05:28,  2.30s/it]
 43%|████▎     | 1268/2975 [53:56<1:05:16,  2.29s/it]
 43%|████▎     | 1269/2975 [53:58<1:05:28,  2.30s/it]
 43%|████▎     | 1270/2975 [54:00<1:05:41,  2.31s/it]
    
0: {'loss': 0.4308, 'grad_norm': 0.47081501374166773, 'learning_rate': 3e-06, 'epoch': 0.43}
0: {'loss': 0.4428, 'grad_norm': 0.4847343404367392, 'learning_rate': 3e-06, 'epoch': 0.43}
0:                                                  

 43%|████▎     | 1270/2975 [54:00<1:05:41,  2.31s/it]
 43%|████▎     | 1271/2975 [54:03<1:05:28,  2.31s/it]
 43%|████▎     | 1272/2975 [54:05<1:05:14,  2.30s/it]
 43%|████▎     | 1273/2975 [54:07<1:05:07,  2.30s/it]
 43%|████▎     | 1274/2975 [54:09<1:04:58,  2.29s/it]
 43%|████▎     | 1275/2975 [54:12<1:05:17,  2.30s/it]
 43%|████▎     | 1276/2975 [54:14<1:05:01,  2.30s/it]
 43%|████▎     | 1277/2975 [54:16<1:04:53,  2.29s/it]
 43%|████▎     | 1278/2975 [54:19<1:04:48,  2.29s/it]
 43%|████▎     | 1279/2975 [54:21<1:05:48,  2.33s/it]
 43%|████▎     | 1280/2975 [54:23<1:05:23,  2.31s/it]
                                                     

 43%|████▎     | 1280/2975 [54:23<1:05:23,  2.31s/it]
 43%|████▎     | 1281/2975 [54:26<1:07:46,  2.40s/it]
 43%|████▎     | 1282/2975 [54:28<1:06:42,  2.36s/it]
 43%|████▎  
0: {'loss': 0.4252, 'grad_norm': 0.4890090437099711, 'learning_rate': 3e-06, 'epoch': 0.43}
0:    | 1283/2975 [54:30<1:05:58,  2.34s/it]
 43%|████▎     | 1284/2975 [54:33<1:05:38,  2.33s/it]
 43%|████▎     | 1285/2975 [54:35<1:05:47,  2.34s/it]
 43%|████▎     | 1286/2975 [54:37<1:05:29,  2.33s/it]
 43%|████▎     | 1287/2975 [54:40<1:11:16,  2.53s/it]
 43%|████▎     | 1288/2975 [54:43<1:09:13,  2.46s/it]
 43%|████▎     | 1289/2975 [54:45<1:07:47,  2.41s/it]
 43%|████▎     | 1290/2975 [54:47<1:06:40,  2.37s/it]
                                                     

 43%|████▎     | 1290/2975 [54:47<1:06:40,  2.37s/it]
 43%|████▎     | 1291/2975 [54:50<1:05:56,  2.35s/it]
 43%|████▎     | 1292/2975 [54:52<1:06:07,  2.36s/it]
 43%|████▎     | 1293/2975 [54:54<1:05:33,  2.34s/it]
 43%|████▎     | 1294/2975 [54:57<1:05:21,  2.33s/it]
 44%|████▎     | 1295/2975 [54:59<1:04:57,  2.32s/it]
 44%|████▎     | 1296/2975 [55:01<1:04:47,  2.32s/it]
 44%|████▎     | 1297
0: {'loss': 0.4309, 'grad_norm': 0.4716372570132661, 'learning_rate': 3e-06, 'epoch': 0.44}
0: /2975 [55:03<1:04:36,  2.31s/it]
 44%|████▎     | 1298/2975 [55:06<1:04:23,  2.30s/it]
 44%|████▎     | 1299/2975 [55:08<1:05:00,  2.33s/it]
 44%|████▎     | 1300/2975 [55:10<1:04:37,  2.32s/it]
                                                     

 44%|████▎     | 1300/2975 [55:10<1:04:37,  2.32s/it]
 44%|████▎     | 1301/2975 [55:13<1:04:22,  2.31s/it]
 44%|████▍     | 1302/2975 [55:15<1:04:09,  2.30s/it]
 44%|████▍     | 1303/2975 [55:17<1:04:01,  2.30s/it]
 44%|████▍     | 1304/2975 [55:20<1:03:34,  2.28s/it]
 44%|████▍     | 1305/2975 [55:22<1:04:13,  2.31s/it]
 44%|████▍     | 1306/2975 [55:24<1:04:39,  2.32s/it]
 44%|████▍     | 1307/2975 [55:27<1:04:14,  2.31s/it]
 44%|████▍     | 1308/2975 [55:29<1:04:06,  2.31s/it]
 44%|████▍     | 1309/2975 [55:31<1:04:00,  2.31s/it]
 44%|████▍     | 1310/2975 [55:33<1:03:56,  2.30s/it]
                                        
0: {'loss': 0.4402, 'grad_norm': 0.4715712364491822, 'learning_rate': 3e-06, 'epoch': 0.44}
0: {'loss': 0.4415, 'grad_norm': 0.4671912118214157, 'learning_rate': 3e-06, 'epoch': 0.44}
0:              

 44%|████▍     | 1310/2975 [55:33<1:03:56,  2.30s/it]
 44%|████▍     | 1311/2975 [55:36<1:03:56,  2.31s/it]
 44%|████▍     | 1312/2975 [55:38<1:03:44,  2.30s/it]
 44%|████▍     | 1313/2975 [55:40<1:03:35,  2.30s/it]
 44%|████▍     | 1314/2975 [55:43<1:03:30,  2.29s/it]
 44%|████▍     | 1315/2975 [55:45<1:03:26,  2.29s/it]
 44%|████▍     | 1316/2975 [55:47<1:03:23,  2.29s/it]
 44%|████▍     | 1317/2975 [55:50<1:03:19,  2.29s/it]
 44%|████▍     | 1318/2975 [55:52<1:03:11,  2.29s/it]
 44%|████▍     | 1319/2975 [55:54<1:04:34,  2.34s/it]
 44%|████▍     | 1320/2975 [55:56<1:03:51,  2.31s/it]
                                                     

 44%|████▍     | 1320/2975 [55:57<1:03:51,  2.31s/it]
 44%|████▍     | 1321/2975 [55:59<1:03:39,  2.31s/it]
 44%|████▍     | 1322/2975 [56:01<1:03:33,  2.31s/it]
 44%|████▍     | 1323/2975 [56:03<1:03:17,  2.30
0: {'loss': 0.441, 'grad_norm': 0.47977495181005003, 'learning_rate': 3e-06, 'epoch': 0.45}
0: s/it]
 45%|████▍     | 1324/2975 [56:06<1:03:02,  2.29s/it]
 45%|████▍     | 1325/2975 [56:08<1:03:02,  2.29s/it]
 45%|████▍     | 1326/2975 [56:10<1:02:59,  2.29s/it]
 45%|████▍     | 1327/2975 [56:13<1:03:21,  2.31s/it]
 45%|████▍     | 1328/2975 [56:15<1:03:09,  2.30s/it]
 45%|████▍     | 1329/2975 [56:17<1:03:44,  2.32s/it]
 45%|████▍     | 1330/2975 [56:20<1:04:49,  2.36s/it]
                                                     

 45%|████▍     | 1330/2975 [56:20<1:04:49,  2.36s/it]
 45%|████▍     | 1331/2975 [56:22<1:05:52,  2.40s/it]
 45%|████▍     | 1332/2975 [56:24<1:04:56,  2.37s/it]
 45%|████▍     | 1333/2975 [56:27<1:04:19,  2.35s/it]
 45%|████▍     | 1334/2975 [56:29<1:03:46,  2.33s/it]
 45%|████▍     | 1335/2975 [56:31<1:03:30,  2.32s/it]
 45%|████▍     | 1336/2975 [56:34<1:03:14,  2.31s/it]
 45%|████▍     | 1337/2975 [56:36<1:02:57,  2.31s/it]
 45
0: {'loss': 0.436, 'grad_norm': 0.4901822221494393, 'learning_rate': 3e-06, 'epoch': 0.45}
0: {'loss': 0.4432, 'grad_norm': 0.4688083937554462, 'learning_rate': 3e-06, 'epoch': 0.45}
0: %|████▍     | 1338/2975 [56:38<1:02:49,  2.30s/it]
 45%|████▌     | 1339/2975 [56:41<1:02:47,  2.30s/it]
 45%|████▌     | 1340/2975 [56:43<1:05:53,  2.42s/it]
                                                     

 45%|████▌     | 1340/2975 [56:43<1:05:53,  2.42s/it]
 45%|████▌     | 1341/2975 [56:46<1:06:44,  2.45s/it]
 45%|████▌     | 1342/2975 [56:48<1:05:20,  2.40s/it]
 45%|████▌     | 1343/2975 [56:50<1:04:19,  2.36s/it]
 45%|████▌     | 1344/2975 [56:53<1:03:46,  2.35s/it]
 45%|████▌     | 1345/2975 [56:55<1:03:22,  2.33s/it]
 45%|████▌     | 1346/2975 [56:57<1:03:00,  2.32s/it]
 45%|████▌     | 1347/2975 [57:00<1:02:41,  2.31s/it]
 45%|████▌     | 1348/2975 [57:02<1:04:34,  2.38s/it]
 45%|████▌     | 1349/2975 [57:04<1:03:41,  2.35s/it]
 45%|████▌     | 1350/2975 [57:07<1:03:09,  2.33s/it]
                                                     

 45%|████▌ 
0: {'loss': 0.4259, 'grad_norm': 0.48516547252312525, 'learning_rate': 3e-06, 'epoch': 0.46}
0:     | 1350/2975 [57:07<1:03:09,  2.33s/it]
 45%|████▌     | 1351/2975 [57:09<1:03:27,  2.34s/it]
 45%|████▌     | 1352/2975 [57:11<1:02:57,  2.33s/it]
 45%|████▌     | 1353/2975 [57:14<1:02:34,  2.31s/it]
 46%|████▌     | 1354/2975 [57:16<1:02:15,  2.30s/it]
 46%|████▌     | 1355/2975 [57:18<1:02:04,  2.30s/it]
 46%|████▌     | 1356/2975 [57:21<1:04:58,  2.41s/it]
 46%|████▌     | 1357/2975 [57:23<1:04:38,  2.40s/it]
 46%|████▌     | 1358/2975 [57:26<1:04:17,  2.39s/it]
 46%|████▌     | 1359/2975 [57:28<1:03:30,  2.36s/it]
 46%|████▌     | 1360/2975 [57:30<1:02:58,  2.34s/it]
                                                     

 46%|████▌     | 1360/2975 [57:30<1:02:58,  2.34s/it]
 46%|████▌     | 1361/2975 [57:32<1:02:28,  2.32s/it]
 46%|████▌     | 1362/2975 [57:35<1:02:04,  2.31s/it]
 46%|████▌     | 1363/2975 [57:37<1:01:57,  2.31s/it]
 46%|████▌     | 136
0: {'loss': 0.4322, 'grad_norm': 0.46219354631287807, 'learning_rate': 3e-06, 'epoch': 0.46}
0: 4/2975 [57:39<1:01:48,  2.30s/it]
 46%|████▌     | 1365/2975 [57:42<1:01:37,  2.30s/it]
 46%|████▌     | 1366/2975 [57:44<1:01:51,  2.31s/it]
 46%|████▌     | 1367/2975 [57:46<1:01:59,  2.31s/it]
 46%|████▌     | 1368/2975 [57:49<1:01:59,  2.31s/it]
 46%|████▌     | 1369/2975 [57:51<1:02:05,  2.32s/it]
 46%|████▌     | 1370/2975 [57:53<1:02:02,  2.32s/it]
                                                     

 46%|████▌     | 1370/2975 [57:53<1:02:02,  2.32s/it]
 46%|████▌     | 1371/2975 [57:56<1:02:52,  2.35s/it]
 46%|████▌     | 1372/2975 [57:58<1:02:26,  2.34s/it]
 46%|████▌     | 1373/2975 [58:00<1:02:08,  2.33s/it]
 46%|████▌     | 1374/2975 [58:03<1:01:41,  2.31s/it]
 46%|████▌     | 1375/2975 [58:05<1:02:08,  2.33s/it]
 46%|████▋     | 1376/2975 [58:07<1:01:36,  2.31s/it]
 46%|████▋     | 1377/2975 [58:10<1:02:06,  2.33s/it]
 46%|████▋     | 1378/2975 [5
0: {'loss': 0.4341, 'grad_norm': 0.47165676155251657, 'learning_rate': 3e-06, 'epoch': 0.46}
0: {'loss': 0.4456, 'grad_norm': 0.48547765403180504, 'learning_rate': 3e-06, 'epoch': 0.47}
0: 8:12<1:02:07,  2.33s/it]
 46%|████▋     | 1379/2975 [58:14<1:01:42,  2.32s/it]
 46%|████▋     | 1380/2975 [58:17<1:05:10,  2.45s/it]
                                                     

 46%|████▋     | 1380/2975 [58:17<1:05:10,  2.45s/it]
 46%|████▋     | 1381/2975 [58:19<1:03:58,  2.41s/it]
 46%|████▋     | 1382/2975 [58:21<1:02:52,  2.37s/it]
 46%|████▋     | 1383/2975 [58:24<1:02:09,  2.34s/it]
 47%|████▋     | 1384/2975 [58:26<1:01:35,  2.32s/it]
 47%|████▋     | 1385/2975 [58:28<1:01:17,  2.31s/it]
 47%|████▋     | 1386/2975 [58:31<1:03:05,  2.38s/it]
 47%|████▋     | 1387/2975 [58:33<1:03:01,  2.38s/it]
 47%|████▋     | 1388/2975 [58:36<1:02:54,  2.38s/it]
 47%|████▋     | 1389/2975 [58:38<1:02:25,  2.36s/it]
 47%|████▋     | 1390/2975 [58:40<1:02:19,  2.36s/it]
                                                     

 47%|████▋     | 1390/2975 [58:40<1:02:19,  2.3
0: {'loss': 0.4412, 'grad_norm': 0.45488137965619074, 'learning_rate': 3e-06, 'epoch': 0.47}
0: 6s/it]
 47%|████▋     | 1391/2975 [58:43<1:02:08,  2.35s/it]
 47%|████▋     | 1392/2975 [58:45<1:01:47,  2.34s/it]
 47%|████▋     | 1393/2975 [58:47<1:01:26,  2.33s/it]
 47%|████▋     | 1394/2975 [58:50<1:01:07,  2.32s/it]
 47%|████▋     | 1395/2975 [58:52<1:00:59,  2.32s/it]
 47%|████▋     | 1396/2975 [58:54<1:01:01,  2.32s/it]
 47%|████▋     | 1397/2975 [58:57<1:02:07,  2.36s/it]
 47%|████▋     | 1398/2975 [58:59<1:01:30,  2.34s/it]
 47%|████▋     | 1399/2975 [59:01<1:01:24,  2.34s/it]
 47%|████▋     | 1400/2975 [59:04<1:00:59,  2.32s/it]
                                                     

 47%|████▋     | 1400/2975 [59:04<1:00:59,  2.32s/it]
 47%|████▋     | 1401/2975 [59:06<1:01:48,  2.36s/it]
 47%|████▋     | 1402/2975 [59:08<1:01:33,  2.35s/it]
 47%|████▋     | 1403/2975 [59:11<1:00:59,  2.33s/it]
 47%|████▋     | 1404/2975 [59:13<1:00:38,  2.32s/it]
 4
0: {'loss': 0.4253, 'grad_norm': 0.46804984615716283, 'learning_rate': 3e-06, 'epoch': 0.47}
0: 7%|████▋     | 1405/2975 [59:15<1:00:52,  2.33s/it]
 47%|████▋     | 1406/2975 [59:18<1:01:22,  2.35s/it]
 47%|████▋     | 1407/2975 [59:20<1:00:51,  2.33s/it]
 47%|████▋     | 1408/2975 [59:23<1:05:30,  2.51s/it]
 47%|████▋     | 1409/2975 [59:25<1:04:59,  2.49s/it]
 47%|████▋     | 1410/2975 [59:28<1:03:28,  2.43s/it]
                                                     

 47%|████▋     | 1410/2975 [59:28<1:03:28,  2.43s/it]
 47%|████▋     | 1411/2975 [59:30<1:02:43,  2.41s/it]
 47%|████▋     | 1412/2975 [59:32<1:02:45,  2.41s/it]
 47%|████▋     | 1413/2975 [59:35<1:02:18,  2.39s/it]
 48%|████▊     | 1414/2975 [59:37<1:01:48,  2.38s/it]
 48%|████▊     | 1415/2975 [59:39<1:01:34,  2.37s/it]
 48%|████▊     | 1416/2975 [59:42<1:03:35,  2.45s/it]
 48%|████▊     | 1417/2975 [59:44<1:02:19,  2.40s/it]
 48%|████▊     | 1418/2975 [59:47<1:01:20,  2.36s/it]
 48%|██
0: {'loss': 0.4266, 'grad_norm': 0.5038561025685049, 'learning_rate': 3e-06, 'epoch': 0.48}
0: {'loss': 0.4189, 'grad_norm': 0.44939836372729763, 'learning_rate': 3e-06, 'epoch': 0.48}
0: ██▊     | 1419/2975 [59:49<1:00:51,  2.35s/it]
 48%|████▊     | 1420/2975 [59:51<1:00:25,  2.33s/it]
                                                     

 48%|████▊     | 1420/2975 [59:51<1:00:25,  2.33s/it]
 48%|████▊     | 1421/2975 [59:54<1:00:03,  2.32s/it]
 48%|████▊     | 1422/2975 [59:56<59:44,  2.31s/it]  
 48%|████▊     | 1423/2975 [59:58<1:01:54,  2.39s/it]
 48%|████▊     | 1424/2975 [1:00:01<1:03:00,  2.44s/it]
 48%|████▊     | 1425/2975 [1:00:03<1:01:30,  2.38s/it]
 48%|████▊     | 1426/2975 [1:00:05<1:00:45,  2.35s/it]
 48%|████▊     | 1427/2975 [1:00:08<1:00:07,  2.33s/it]
 48%|████▊     | 1428/2975 [1:00:10<59:30,  2.31s/it]  
 48%|████▊     | 1429/2975 [1:00:12<1:00:06,  2.33s/it]
 48%|████▊     | 1430/2975 [1:00:15<1:00:41,  2.36s/it]
                                                       

 48%|████▊     | 1430/2975 [1:00:15<1:00:41,  2.36s/it]
 48%|██
0: {'loss': 0.4356, 'grad_norm': 0.4754076661664157, 'learning_rate': 3e-06, 'epoch': 0.48}
0: ██▊     | 1431/2975 [1:00:17<1:00:28,  2.35s/it]
 48%|████▊     | 1432/2975 [1:00:20<1:00:37,  2.36s/it]
 48%|████▊     | 1433/2975 [1:00:22<1:00:22,  2.35s/it]
 48%|████▊     | 1434/2975 [1:00:24<1:00:08,  2.34s/it]
 48%|████▊     | 1435/2975 [1:00:26<59:51,  2.33s/it]  
 48%|████▊     | 1436/2975 [1:00:29<59:33,  2.32s/it]
 48%|████▊     | 1437/2975 [1:00:31<59:52,  2.34s/it]
 48%|████▊     | 1438/2975 [1:00:34<1:00:05,  2.35s/it]
 48%|████▊     | 1439/2975 [1:00:36<59:29,  2.32s/it]  
 48%|████▊     | 1440/2975 [1:00:38<59:20,  2.32s/it]
                                                     

 48%|████▊     | 1440/2975 [1:00:38<59:20,  2.32s/it]
 48%|████▊     | 1441/2975 [1:00:40<59:03,  2.31s/it]
 48%|████▊     | 1442/2975 [1:00:43<1:00:22,  2.36s/it]
 49%|████▊     | 1443/2975 [1:00:46<1:03:02,  2.47s/it]
 49%|████▊     | 1444/2975 [1:00:48<1:01:47,  2.42s/it]
0: {'loss': 0.4293, 'grad_norm': 0.4807707857905617, 'learning_rate': 3e-06, 'epoch': 0.49}
0:  49%|████▊     | 1445/2975 [1:00:50<1:00:27,  2.37s/it]
 49%|████▊     | 1446/2975 [1:00:53<1:00:31,  2.38s/it]
 49%|████▊     | 1447/2975 [1:00:55<59:49,  2.35s/it]  
 49%|████▊     | 1448/2975 [1:00:57<59:19,  2.33s/it]
 49%|████▊     | 1449/2975 [1:01:00<1:01:42,  2.43s/it]
 49%|████▊     | 1450/2975 [1:01:02<1:01:39,  2.43s/it]
                                                       

 49%|████▊     | 1450/2975 [1:01:02<1:01:39,  2.43s/it]
 49%|████▉     | 1451/2975 [1:01:05<1:01:03,  2.40s/it]
 49%|████▉     | 1452/2975 [1:01:07<1:00:48,  2.40s/it]
 49%|████▉     | 1453/2975 [1:01:09<1:00:22,  2.38s/it]
 49%|████▉     | 1454/2975 [1:01:12<1:00:07,  2.37s/it]
 49%|████▉     | 1455/2975 [1:01:14<59:42,  2.36s/it]  
 49%|████▉     | 1456/2975 [1:01:16<1:00:56,  2.41s/it]
 49%|████▉     | 1457/2975 [1:01:19<59:57,  2.37s/it]  
 49%|████▉     | 1458/2975 [1:01:21
0: {'loss': 0.4276, 'grad_norm': 0.4581839964297145, 'learning_rate': 3e-06, 'epoch': 0.49}
0: {'loss': 0.4233, 'grad_norm': 0.47248471015740856, 'learning_rate': 3e-06, 'epoch': 0.49}
0: <59:19,  2.35s/it]
 49%|████▉     | 1459/2975 [1:01:24<1:00:37,  2.40s/it]
 49%|████▉     | 1460/2975 [1:01:26<1:01:21,  2.43s/it]
                                                       

 49%|████▉     | 1460/2975 [1:01:26<1:01:21,  2.43s/it]
 49%|████▉     | 1461/2975 [1:01:28<1:00:19,  2.39s/it]
 49%|████▉     | 1462/2975 [1:01:31<59:27,  2.36s/it]  
 49%|████▉     | 1463/2975 [1:01:33<58:46,  2.33s/it]
 49%|████▉     | 1464/2975 [1:01:35<58:21,  2.32s/it]
 49%|████▉     | 1465/2975 [1:01:37<58:06,  2.31s/it]
 49%|████▉     | 1466/2975 [1:01:40<1:01:22,  2.44s/it]
 49%|████▉     | 1467/2975 [1:01:43<1:00:07,  2.39s/it]
 49%|████▉     | 1468/2975 [1:01:45<59:17,  2.36s/it]  
 49%|████▉     | 1469/2975 [1:01:47<58:33,  2.33s/it]
 49%|████▉     | 1470/2975 [1:01:49<58:11,  2.32s/it]
                                                     

 49%|████▉     | 1470/2975 [1:01:49
0: {'loss': 0.4355, 'grad_norm': 0.47533168756329824, 'learning_rate': 3e-06, 'epoch': 0.5}
0: <58:11,  2.32s/it]
 49%|████▉     | 1471/2975 [1:01:52<58:48,  2.35s/it]
 49%|████▉     | 1472/2975 [1:01:54<58:41,  2.34s/it]
 50%|████▉     | 1473/2975 [1:01:56<58:53,  2.35s/it]
 50%|████▉     | 1474/2975 [1:01:59<1:00:53,  2.43s/it]
 50%|████▉     | 1475/2975 [1:02:01<1:00:29,  2.42s/it]
 50%|████▉     | 1476/2975 [1:02:04<59:28,  2.38s/it]  
 50%|████▉     | 1477/2975 [1:02:06<58:49,  2.36s/it]
 50%|████▉     | 1478/2975 [1:02:09<59:35,  2.39s/it]
 50%|████▉     | 1479/2975 [1:02:11<58:46,  2.36s/it]
 50%|████▉     | 1480/2975 [1:02:13<58:24,  2.34s/it]
                                                     

 50%|████▉     | 1480/2975 [1:02:13<58:24,  2.34s/it]
 50%|████▉     | 1481/2975 [1:02:15<57:50,  2.32s/it]
 50%|████▉     | 1482/2975 [1:02:18<57:28,  2.31s/it]
 50%|████▉     | 1483/2975 [1:02:20<57:23,  2.31s/it]
 50%|████▉     | 1484/2975 [1:02:22<57
0: {'loss': 0.4328, 'grad_norm': 0.4955991779236201, 'learning_rate': 3e-06, 'epoch': 0.5}
0: :07,  2.30s/it]
 50%|████▉     | 1485/2975 [1:02:25<57:16,  2.31s/it]
 50%|████▉     | 1486/2975 [1:02:27<57:06,  2.30s/it]
 50%|████▉     | 1487/2975 [1:02:29<56:56,  2.30s/it]
 50%|█████     | 1488/2975 [1:02:31<56:52,  2.29s/it]
 50%|█████     | 1489/2975 [1:02:34<56:51,  2.30s/it]
 50%|█████     | 1490/2975 [1:02:36<57:10,  2.31s/it]
                                                     

 50%|█████     | 1490/2975 [1:02:36<57:10,  2.31s/it]
 50%|█████     | 1491/2975 [1:02:38<57:31,  2.33s/it]
 50%|█████     | 1492/2975 [1:02:41<57:58,  2.35s/it]
 50%|█████     | 1493/2975 [1:02:43<57:50,  2.34s/it]
 50%|█████     | 1494/2975 [1:02:46<57:38,  2.34s/it]
 50%|█████     | 1495/2975 [1:02:49<1:02:55,  2.55s/it]
 50%|█████     | 1496/2975 [1:02:51<1:01:29,  2.49s/it]
 50%|█████     | 1497/2975 [1:02:53<1:00:06,  2.44s/it]
 50%|█████     | 1498/2975 [1:02:56<59:01
0: {'loss': 0.4316, 'grad_norm': 0.501343830093561, 'learning_rate': 3e-06, 'epoch': 0.5}
0: {'loss': 0.4207, 'grad_norm': 0.455357077997908, 'learning_rate': 3e-06, 'epoch': 0.51}
0: ,  2.40s/it]  
 50%|█████     | 1499/2975 [1:02:58<58:20,  2.37s/it]
 50%|█████     | 1500/2975 [1:03:00<57:45,  2.35s/it]
                                                     

 50%|█████     | 1500/2975 [1:03:00<57:45,  2.35s/it]
 50%|█████     | 1501/2975 [1:03:02<57:15,  2.33s/it]
 50%|█████     | 1502/2975 [1:03:05<56:53,  2.32s/it]
 51%|█████     | 1503/2975 [1:03:07<56:46,  2.31s/it]
 51%|█████     | 1504/2975 [1:03:09<56:27,  2.30s/it]
 51%|█████     | 1505/2975 [1:03:12<56:10,  2.29s/it]
 51%|█████     | 1506/2975 [1:03:14<56:09,  2.29s/it]
 51%|█████     | 1507/2975 [1:03:16<56:01,  2.29s/it]
 51%|█████     | 1508/2975 [1:03:18<56:20,  2.30s/it]
 51%|█████     | 1509/2975 [1:03:21<56:09,  2.30s/it]
 51%|█████     | 1510/2975 [1:03:23<56:10,  2.30s/it]
                                                     

 51%|█████     | 1510/2975 [1:03:23<56:10,  2.30s/it]
 51
0: {'loss': 0.4314, 'grad_norm': 0.4644767178235877, 'learning_rate': 3e-06, 'epoch': 0.51}
0: %|█████     | 1511/2975 [1:03:25<56:06,  2.30s/it]
 51%|█████     | 1512/2975 [1:03:28<56:07,  2.30s/it]
 51%|█████     | 1513/2975 [1:03:30<56:40,  2.33s/it]
 51%|█████     | 1514/2975 [1:03:32<56:56,  2.34s/it]
 51%|█████     | 1515/2975 [1:03:35<57:10,  2.35s/it]
 51%|█████     | 1516/2975 [1:03:37<56:59,  2.34s/it]
 51%|█████     | 1517/2975 [1:03:40<1:02:12,  2.56s/it]
 51%|█████     | 1518/2975 [1:03:43<1:00:22,  2.49s/it]
 51%|█████     | 1519/2975 [1:03:45<59:05,  2.43s/it]  
 51%|█████     | 1520/2975 [1:03:47<57:58,  2.39s/it]
                                                     

 51%|█████     | 1520/2975 [1:03:47<57:58,  2.39s/it]
 51%|█████     | 1521/2975 [1:03:49<57:17,  2.36s/it]
 51%|█████     | 1522/2975 [1:03:52<57:10,  2.36s/it]
 51%|█████     | 1523/2975 [1:03:54<56:33,  2.34s/it]
 51%|█████     | 1524/2975 [1:03:56<56:31,  2.34s/it]
 51%|�
0: {'loss': 0.4364, 'grad_norm': 0.4705171849817411, 'learning_rate': 3e-06, 'epoch': 0.51}
0: ��████▏    | 1525/2975 [1:03:59<56:06,  2.32s/it]
 51%|█████▏    | 1526/2975 [1:04:01<56:08,  2.32s/it]
 51%|█████▏    | 1527/2975 [1:04:03<55:51,  2.31s/it]
 51%|█████▏    | 1528/2975 [1:04:06<55:39,  2.31s/it]
 51%|█████▏    | 1529/2975 [1:04:08<55:27,  2.30s/it]
 51%|█████▏    | 1530/2975 [1:04:10<55:16,  2.29s/it]
                                                     

 51%|█████▏    | 1530/2975 [1:04:10<55:16,  2.29s/it]
 51%|█████▏    | 1531/2975 [1:04:12<55:13,  2.29s/it]
 51%|█████▏    | 1532/2975 [1:04:15<55:11,  2.29s/it]
 52%|█████▏    | 1533/2975 [1:04:17<57:07,  2.38s/it]
 52%|█████▏    | 1534/2975 [1:04:20<56:57,  2.37s/it]
 52%|█████▏    | 1535/2975 [1:04:22<57:01,  2.38s/it]
 52%|█████▏    | 1536/2975 [1:04:25<58:28,  2.44s/it]
 52%|█████▏    | 1537/2975 [1:04:27<57:45,  2.41s/it]
 52%|█████▏    | 1538/2975 [1:04:29<57:
0: {'loss': 0.4322, 'grad_norm': 0.4696059116768773, 'learning_rate': 3e-06, 'epoch': 0.52}
0: {'loss': 0.4439, 'grad_norm': 0.47234656731855723, 'learning_rate': 3e-06, 'epoch': 0.52}
0: 28,  2.40s/it]
 52%|█████▏    | 1539/2975 [1:04:32<56:45,  2.37s/it]
 52%|█████▏    | 1540/2975 [1:04:34<56:16,  2.35s/it]
                                                     

 52%|█████▏    | 1540/2975 [1:04:34<56:16,  2.35s/it]
 52%|█████▏    | 1541/2975 [1:04:36<55:51,  2.34s/it]
 52%|█████▏    | 1542/2975 [1:04:39<55:24,  2.32s/it]
 52%|█████▏    | 1543/2975 [1:04:41<56:37,  2.37s/it]
 52%|█████▏    | 1544/2975 [1:04:43<55:54,  2.34s/it]
 52%|█████▏    | 1545/2975 [1:04:46<56:53,  2.39s/it]
 52%|█████▏    | 1546/2975 [1:04:48<56:07,  2.36s/it]
 52%|█████▏    | 1547/2975 [1:04:50<55:44,  2.34s/it]
 52%|█████▏    | 1548/2975 [1:04:53<55:16,  2.32s/it]
 52%|█████▏    | 1549/2975 [1:04:55<54:54,  2.31s/it]
 52%|█████▏    | 1550/2975 [1:04:57<54:41,  2.30s/it]
                                                     

 52%|█████▏    | 1550/2975 [1
0: {'loss': 0.4305, 'grad_norm': 0.48167814074702764, 'learning_rate': 3e-06, 'epoch': 0.52}
0: :04:57<54:41,  2.30s/it]
 52%|█████▏    | 1551/2975 [1:05:00<54:37,  2.30s/it]
 52%|█████▏    | 1552/2975 [1:05:02<54:36,  2.30s/it]
 52%|█████▏    | 1553/2975 [1:05:04<54:25,  2.30s/it]
 52%|█████▏    | 1554/2975 [1:05:07<55:49,  2.36s/it]
 52%|█████▏    | 1555/2975 [1:05:09<56:08,  2.37s/it]
 52%|█████▏    | 1556/2975 [1:05:11<55:47,  2.36s/it]
 52%|█████▏    | 1557/2975 [1:05:14<55:29,  2.35s/it]
 52%|█████▏    | 1558/2975 [1:05:16<55:18,  2.34s/it]
 52%|█████▏    | 1559/2975 [1:05:18<55:30,  2.35s/it]
 52%|█████▏    | 1560/2975 [1:05:21<55:18,  2.35s/it]
                                                     

 52%|█████▏    | 1560/2975 [1:05:21<55:18,  2.35s/it]
 52%|█████▏    | 1561/2975 [1:05:23<54:40,  2.32s/it]
 53%|█████▎    | 1562/2975 [1:05:25<54:35,  2.32s/it]
 53%|█████▎    | 1563/2975 [1:05:28<54:17,  2.31s/it]
 53%|█████
0: {'loss': 0.4287, 'grad_norm': 0.48855150878716774, 'learning_rate': 3e-06, 'epoch': 0.53}
0: ▎    | 1564/2975 [1:05:30<54:13,  2.31s/it]
 53%|█████▎    | 1565/2975 [1:05:32<54:41,  2.33s/it]
 53%|█████▎    | 1566/2975 [1:05:35<54:21,  2.31s/it]
 53%|█████▎    | 1567/2975 [1:05:37<54:09,  2.31s/it]
 53%|█████▎    | 1568/2975 [1:05:39<53:57,  2.30s/it]
 53%|█████▎    | 1569/2975 [1:05:41<53:56,  2.30s/it]
 53%|█████▎    | 1570/2975 [1:05:44<55:04,  2.35s/it]
                                                     

 53%|█████▎    | 1570/2975 [1:05:44<55:04,  2.35s/it]
 53%|█████▎    | 1571/2975 [1:05:46<54:34,  2.33s/it]
 53%|█████▎    | 1572/2975 [1:05:49<54:26,  2.33s/it]
 53%|█████▎    | 1573/2975 [1:05:51<54:07,  2.32s/it]
 53%|█████▎    | 1574/2975 [1:05:53<53:53,  2.31s/it]
 53%|█████▎    | 1575/2975 [1:05:55<53:45,  2.30s/it]
 53%|█████▎    | 1576/2975 [1:05:58<54:01,  2.32s/it]
 53%|█████▎    | 1577/2975 [1:06:00<54:22,  2.33s/it]
0: {'loss': 0.4207, 'grad_norm': 0.4513122512966505, 'learning_rate': 3e-06, 'epoch': 0.53}
0: 
 53%|█████▎    | 1578/2975 [1:06:03<55:08,  2.37s/it]
 53%|█████▎    | 1579/2975 [1:06:05<55:04,  2.37s/it]
 53%|█████▎    | 1580/2975 [1:06:07<54:47,  2.36s/it]
                                                     

 53%|█████▎    | 1580/2975 [1:06:07<54:47,  2.36s/it]
 53%|█████▎    | 1581/2975 [1:06:10<54:16,  2.34s/it]
 53%|█████▎    | 1582/2975 [1:06:12<54:00,  2.33s/it]
 53%|█████▎    | 1583/2975 [1:06:14<53:36,  2.31s/it]
 53%|█████▎    | 1584/2975 [1:06:16<53:30,  2.31s/it]
 53%|█████▎    | 1585/2975 [1:06:19<54:44,  2.36s/it]
 53%|█████▎    | 1586/2975 [1:06:21<54:09,  2.34s/it]
 53%|█████▎    | 1587/2975 [1:06:23<53:42,  2.32s/it]
 53%|█████▎    | 1588/2975 [1:06:26<53:28,  2.31s/it]
 53%|█████▎    | 1589/2975 [1:06:28<53:12,  2.30s/it]
 53%|█████▎    | 1590/2975 [1:06:30<53:07,  2.30s/it]
                                            
0: {'loss': 0.4236, 'grad_norm': 0.48273292599370643, 'learning_rate': 3e-06, 'epoch': 0.53}
0: {'loss': 0.4178, 'grad_norm': 0.47971692280731976, 'learning_rate': 3e-06, 'epoch': 0.54}
0:          

 53%|█████▎    | 1590/2975 [1:06:30<53:07,  2.30s/it]
 53%|█████▎    | 1591/2975 [1:06:33<53:06,  2.30s/it]
 54%|█████▎    | 1592/2975 [1:06:35<52:58,  2.30s/it]
 54%|█████▎    | 1593/2975 [1:06:37<52:51,  2.29s/it]
 54%|█████▎    | 1594/2975 [1:06:40<52:49,  2.30s/it]
 54%|█████▎    | 1595/2975 [1:06:42<52:40,  2.29s/it]
 54%|█████▎    | 1596/2975 [1:06:45<57:25,  2.50s/it]
 54%|█████▎    | 1597/2975 [1:06:47<56:10,  2.45s/it]
 54%|█████▎    | 1598/2975 [1:06:49<55:22,  2.41s/it]
 54%|█████▎    | 1599/2975 [1:06:52<55:02,  2.40s/it]
 54%|█████▍    | 1600/2975 [1:06:54<54:34,  2.38s/it]
                                                     

 54%|█████▍    | 1600/2975 [1:06:54<54:34,  2.38s/it]
 54%|█████▍    | 1601/2975 [1:06:56<54:07,  2.36s/it]
 54%|█████▍    | 1602/2975 [1:06:59<53:50,  2.35s/it]
 54%|█████▍    | 1603/
0: {'loss': 0.4184, 'grad_norm': 0.4439687145548118, 'learning_rate': 3e-06, 'epoch': 0.54}
0: 2975 [1:07:01<53:27,  2.34s/it]
 54%|█████▍    | 1604/2975 [1:07:03<53:03,  2.32s/it]
 54%|█████▍    | 1605/2975 [1:07:06<52:46,  2.31s/it]
 54%|█████▍    | 1606/2975 [1:07:08<52:28,  2.30s/it]
 54%|█████▍    | 1607/2975 [1:07:10<52:14,  2.29s/it]
 54%|█████▍    | 1608/2975 [1:07:13<52:14,  2.29s/it]
 54%|█████▍    | 1609/2975 [1:07:15<52:07,  2.29s/it]
 54%|█████▍    | 1610/2975 [1:07:17<52:07,  2.29s/it]
                                                     

 54%|█████▍    | 1610/2975 [1:07:17<52:07,  2.29s/it]
 54%|█████▍    | 1611/2975 [1:07:19<52:14,  2.30s/it]
 54%|█████▍    | 1612/2975 [1:07:22<52:09,  2.30s/it]
 54%|█████▍    | 1613/2975 [1:07:24<51:50,  2.28s/it]
 54%|█████▍    | 1614/2975 [1:07:26<51:48,  2.28s/it]
 54%|█████▍    | 1615/2975 [1:07:29<51:48,  2.29s/it]
 54%|█████▍    | 1616/2975 [1:07:31<51:44,  2.28s/it]
 54%|██�
0: {'loss': 0.4228, 'grad_norm': 0.4775473836909844, 'learning_rate': 3e-06, 'epoch': 0.54}
0: �██▍    | 1617/2975 [1:07:33<51:45,  2.29s/it]
 54%|█████▍    | 1618/2975 [1:07:36<52:47,  2.33s/it]
 54%|█████▍    | 1619/2975 [1:07:38<52:58,  2.34s/it]
 54%|█████▍    | 1620/2975 [1:07:40<52:53,  2.34s/it]
                                                     

 54%|█████▍    | 1620/2975 [1:07:40<52:53,  2.34s/it]
 54%|█████▍    | 1621/2975 [1:07:43<52:40,  2.33s/it]
 55%|█████▍    | 1622/2975 [1:07:45<52:46,  2.34s/it]
 55%|█████▍    | 1623/2975 [1:07:47<52:33,  2.33s/it]
 55%|█████▍    | 1624/2975 [1:07:50<53:24,  2.37s/it]
 55%|█████▍    | 1625/2975 [1:07:52<52:47,  2.35s/it]
 55%|█████▍    | 1626/2975 [1:07:54<52:16,  2.33s/it]
 55%|█████▍    | 1627/2975 [1:07:57<52:19,  2.33s/it]
 55%|█████▍    | 1628/2975 [1:07:59<51:57,  2.31s/it]
 55%|█████▍    | 1629/2975 [1:08:01<51:41,  2.30s/it]
 55%|█████▍    | 1630/2975 [1:08:03<51:33,  2.
0: {'loss': 0.4341, 'grad_norm': 0.4687349957870858, 'learning_rate': 3e-06, 'epoch': 0.55}
0: {'loss': 0.4214, 'grad_norm': 0.46179715742733896, 'learning_rate': 3e-06, 'epoch': 0.55}
0: 30s/it]
                                                     

 55%|█████▍    | 1630/2975 [1:08:03<51:33,  2.30s/it]
 55%|█████▍    | 1631/2975 [1:08:06<51:29,  2.30s/it]
 55%|█████▍    | 1632/2975 [1:08:08<51:28,  2.30s/it]
 55%|█████▍    | 1633/2975 [1:08:10<51:21,  2.30s/it]
 55%|█████▍    | 1634/2975 [1:08:13<51:55,  2.32s/it]
 55%|█████▍    | 1635/2975 [1:08:15<51:36,  2.31s/it]
 55%|█████▍    | 1636/2975 [1:08:17<51:31,  2.31s/it]
 55%|█████▌    | 1637/2975 [1:08:20<51:19,  2.30s/it]
 55%|█████▌    | 1638/2975 [1:08:22<51:13,  2.30s/it]
 55%|█████▌    | 1639/2975 [1:08:24<52:09,  2.34s/it]
 55%|█████▌    | 1640/2975 [1:08:27<52:16,  2.35s/it]
                                                     

 55%|█████▌    | 1640/2975 [1:08:27<52:16,  2.35s/it]
 55%|█████▌    | 1641/2975 [1:08:29<52:10,  2.35s/it]
 55%|█████▌    | 1642/2975 [1:08:32<
0: {'loss': 0.4249, 'grad_norm': 0.47686487887012546, 'learning_rate': 3e-06, 'epoch': 0.55}
0: 54:45,  2.46s/it]
 55%|█████▌    | 1643/2975 [1:08:34<54:50,  2.47s/it]
 55%|█████▌    | 1644/2975 [1:08:38<1:02:51,  2.83s/it]
 55%|█████▌    | 1645/2975 [1:08:40<1:01:00,  2.75s/it]
 55%|█████▌    | 1646/2975 [1:08:43<57:55,  2.61s/it]  
 55%|█████▌    | 1647/2975 [1:08:45<56:04,  2.53s/it]
 55%|█████▌    | 1648/2975 [1:08:47<54:25,  2.46s/it]
 55%|█████▌    | 1649/2975 [1:08:50<54:43,  2.48s/it]
 55%|█████▌    | 1650/2975 [1:08:52<53:28,  2.42s/it]
                                                     

 55%|█████▌    | 1650/2975 [1:08:52<53:28,  2.42s/it]
 55%|█████▌    | 1651/2975 [1:08:55<52:59,  2.40s/it]
 56%|█████▌    | 1652/2975 [1:08:57<52:16,  2.37s/it]
 56%|█████▌    | 1653/2975 [1:08:59<53:21,  2.42s/it]
 56%|█████▌    | 1654/2975 [1:09:02<53:33,  2.43s/it]
 56%|█████▌    | 1655/2975 [1:09:04<52:31,  2.39s/it]
 56%|█████�
0: {'loss': 0.4136, 'grad_norm': 0.5066736828414815, 'learning_rate': 3e-06, 'epoch': 0.56}
0: ��    | 1656/2975 [1:09:06<51:47,  2.36s/it]
 56%|█████▌    | 1657/2975 [1:09:09<51:25,  2.34s/it]
 56%|█████▌    | 1658/2975 [1:09:11<50:58,  2.32s/it]
 56%|█████▌    | 1659/2975 [1:09:13<50:41,  2.31s/it]
 56%|█████▌    | 1660/2975 [1:09:16<50:46,  2.32s/it]
                                                     

 56%|█████▌    | 1660/2975 [1:09:16<50:46,  2.32s/it]
 56%|█████▌    | 1661/2975 [1:09:18<51:02,  2.33s/it]
 56%|█████▌    | 1662/2975 [1:09:20<50:38,  2.31s/it]
 56%|█████▌    | 1663/2975 [1:09:23<50:38,  2.32s/it]
 56%|█████▌    | 1664/2975 [1:09:25<51:36,  2.36s/it]
 56%|█████▌    | 1665/2975 [1:09:27<51:08,  2.34s/it]
 56%|█████▌    | 1666/2975 [1:09:30<52:05,  2.39s/it]
 56%|█████▌    | 1667/2975 [1:09:32<51:21,  2.36s/it]
 56%|█████▌    | 1668/2975 [1:09:34<50:52,  2.34s/it]
 56%|█████▌    | 1669/2975 [1:09:37<50:25,  2.32s/it]
0: {'loss': 0.4201, 'grad_norm': 0.46987836666478017, 'learning_rate': 3e-06, 'epoch': 0.56}
0: {'loss': 0.4211, 'grad_norm': 0.4461312874633456, 'learning_rate': 3e-06, 'epoch': 0.56}
0:  56%|█████▌    | 1670/2975 [1:09:39<50:12,  2.31s/it]
                                                     

 56%|█████▌    | 1670/2975 [1:09:39<50:12,  2.31s/it]
 56%|█████▌    | 1671/2975 [1:09:41<50:04,  2.30s/it]
 56%|█████▌    | 1672/2975 [1:09:44<50:00,  2.30s/it]
 56%|█████▌    | 1673/2975 [1:09:46<49:46,  2.29s/it]
 56%|█████▋    | 1674/2975 [1:09:48<51:31,  2.38s/it]
 56%|█████▋    | 1675/2975 [1:09:51<51:08,  2.36s/it]
 56%|█████▋    | 1676/2975 [1:09:53<50:32,  2.33s/it]
 56%|█████▋    | 1677/2975 [1:09:55<50:11,  2.32s/it]
 56%|█████▋    | 1678/2975 [1:09:58<51:09,  2.37s/it]
 56%|█████▋    | 1679/2975 [1:10:00<50:36,  2.34s/it]
 56%|█████▋    | 1680/2975 [1:10:02<50:36,  2.34s/it]
                                                     

 56%|█████▋    | 1680/2975 [1:10:02<50:36,  2.34s/it]
 57%|█████▋    | 1681/2975 [1:10:05<50:19,  
0: {'loss': 0.4361, 'grad_norm': 0.49184909323384485, 'learning_rate': 3e-06, 'epoch': 0.57}
0: 2.33s/it]
 57%|█████▋    | 1682/2975 [1:10:07<50:05,  2.32s/it]
 57%|█████▋    | 1683/2975 [1:10:09<49:49,  2.31s/it]
 57%|█████▋    | 1684/2975 [1:10:12<49:33,  2.30s/it]
 57%|█████▋    | 1685/2975 [1:10:14<50:13,  2.34s/it]
 57%|█████▋    | 1686/2975 [1:10:16<49:46,  2.32s/it]
 57%|█████▋    | 1687/2975 [1:10:19<49:34,  2.31s/it]
 57%|█████▋    | 1688/2975 [1:10:21<49:23,  2.30s/it]
 57%|█████▋    | 1689/2975 [1:10:23<49:16,  2.30s/it]
 57%|█████▋    | 1690/2975 [1:10:25<49:12,  2.30s/it]
                                                     

 57%|█████▋    | 1690/2975 [1:10:25<49:12,  2.30s/it]
 57%|█████▋    | 1691/2975 [1:10:28<49:02,  2.29s/it]
 57%|█████▋    | 1692/2975 [1:10:30<49:06,  2.30s/it]
 57%|█████▋    | 1693/2975 [1:10:32<48:55,  2.29s/it]
 57%|█████▋    | 1694/2975 [1:10:35<48:57,  2.29s/it]
 57%|█████▋    | 1695/2
0: {'loss': 0.4178, 'grad_norm': 0.471831708501751, 'learning_rate': 3e-06, 'epoch': 0.57}
0: 975 [1:10:37<49:15,  2.31s/it]
 57%|█████▋    | 1696/2975 [1:10:39<49:03,  2.30s/it]
 57%|█████▋    | 1697/2975 [1:10:42<48:53,  2.30s/it]
 57%|█████▋    | 1698/2975 [1:10:44<48:44,  2.29s/it]
 57%|█████▋    | 1699/2975 [1:10:46<48:35,  2.28s/it]
 57%|█████▋    | 1700/2975 [1:10:48<48:26,  2.28s/it]
                                                     

 57%|█████▋    | 1700/2975 [1:10:48<48:26,  2.28s/it]
 57%|█████▋    | 1701/2975 [1:10:51<48:30,  2.28s/it]
 57%|█████▋    | 1702/2975 [1:10:53<49:34,  2.34s/it]
 57%|█████▋    | 1703/2975 [1:10:55<49:27,  2.33s/it]
 57%|█████▋    | 1704/2975 [1:10:58<49:09,  2.32s/it]
 57%|█████▋    | 1705/2975 [1:11:00<48:57,  2.31s/it]
 57%|█████▋    | 1706/2975 [1:11:02<48:45,  2.31s/it]
 57%|█████▋    | 1707/2975 [1:11:05<48:35,  2.30s/it]
 57%|█████▋    | 1708/2975 [1:11:07<48:26,  2.29s/it]
 57%|███
0: {'loss': 0.4229, 'grad_norm': 0.4643538913870113, 'learning_rate': 3e-06, 'epoch': 0.57}
0: {'loss': 0.4203, 'grad_norm': 0.46966087607039025, 'learning_rate': 3e-06, 'epoch': 0.58}
0: ██▋    | 1709/2975 [1:11:09<49:07,  2.33s/it]
 57%|█████▋    | 1710/2975 [1:11:12<48:48,  2.32s/it]
                                                     

 57%|█████▋    | 1710/2975 [1:11:12<48:48,  2.32s/it]
 58%|█████▊    | 1711/2975 [1:11:14<48:36,  2.31s/it]
 58%|█████▊    | 1712/2975 [1:11:16<48:22,  2.30s/it]
 58%|█████▊    | 1713/2975 [1:11:18<48:17,  2.30s/it]
 58%|█████▊    | 1714/2975 [1:11:21<48:15,  2.30s/it]
 58%|█████▊    | 1715/2975 [1:11:23<48:16,  2.30s/it]
 58%|█████▊    | 1716/2975 [1:11:25<48:09,  2.30s/it]
 58%|█████▊    | 1717/2975 [1:11:28<48:01,  2.29s/it]
 58%|█████▊    | 1718/2975 [1:11:30<48:13,  2.30s/it]
 58%|█████▊    | 1719/2975 [1:11:32<47:59,  2.29s/it]
 58%|█████▊    | 1720/2975 [1:11:34<47:59,  2.29s/it]
                                                     

 58%|█████▊    | 1720/2975 [1:11:34<47:59,  2.29s/it]
 58%
0: {'loss': 0.4304, 'grad_norm': 0.4817207323688443, 'learning_rate': 3e-06, 'epoch': 0.58}
0: |█████▊    | 1721/2975 [1:11:37<47:53,  2.29s/it]
 58%|█████▊    | 1722/2975 [1:11:39<47:47,  2.29s/it]
 58%|█████▊    | 1723/2975 [1:11:41<47:38,  2.28s/it]
 58%|█████▊    | 1724/2975 [1:11:44<47:43,  2.29s/it]
 58%|█████▊    | 1725/2975 [1:11:46<48:01,  2.30s/it]
 58%|█████▊    | 1726/2975 [1:11:48<47:45,  2.29s/it]
 58%|█████▊    | 1727/2975 [1:11:51<47:38,  2.29s/it]
 58%|█████▊    | 1728/2975 [1:11:53<47:36,  2.29s/it]
 58%|█████▊    | 1729/2975 [1:11:55<47:33,  2.29s/it]
 58%|█████▊    | 1730/2975 [1:11:57<47:29,  2.29s/it]
                                                     

 58%|█████▊    | 1730/2975 [1:11:57<47:29,  2.29s/it]
 58%|█████▊    | 1731/2975 [1:12:00<47:23,  2.29s/it]
 58%|█████▊    | 1732/2975 [1:12:02<47:22,  2.29s/it]
 58%|█████▊    | 1733/2975 [1:12:04<47:45,  2.31s/it]
 58%|█████▊    | 1734/2975 [1:12:07<5
0: {'loss': 0.4275, 'grad_norm': 0.47486958281430086, 'learning_rate': 3e-06, 'epoch': 0.58}
0: 0:23,  2.44s/it]
 58%|█████▊    | 1735/2975 [1:12:10<53:25,  2.59s/it]
 58%|█████▊    | 1736/2975 [1:12:12<52:54,  2.56s/it]
 58%|█████▊    | 1737/2975 [1:12:15<51:25,  2.49s/it]
 58%|█████▊    | 1738/2975 [1:12:17<50:05,  2.43s/it]
 58%|█████▊    | 1739/2975 [1:12:19<49:16,  2.39s/it]
 58%|█████▊    | 1740/2975 [1:12:22<48:58,  2.38s/it]
                                                     

 58%|█████▊    | 1740/2975 [1:12:22<48:58,  2.38s/it]
 59%|█████▊    | 1741/2975 [1:12:24<50:18,  2.45s/it]
 59%|█████▊    | 1742/2975 [1:12:27<49:16,  2.40s/it]
 59%|█████▊    | 1743/2975 [1:12:29<48:27,  2.36s/it]
 59%|█████▊    | 1744/2975 [1:12:31<47:58,  2.34s/it]
 59%|█████▊    | 1745/2975 [1:12:33<47:42,  2.33s/it]
 59%|█████▊    | 1746/2975 [1:12:36<47:21,  2.31s/it]
 59%|█████▊    | 1747/2975 [1:12:38<48:03,  2.35s/it]
 59%|█████▉    |
0: {'loss': 0.4192, 'grad_norm': 0.446317707572024, 'learning_rate': 3e-06, 'epoch': 0.59}
0: {'loss': 0.4143, 'grad_norm': 0.46892027639720124, 'learning_rate': 3e-06, 'epoch': 0.59}
0:  1748/2975 [1:12:40<47:33,  2.33s/it]
 59%|█████▉    | 1749/2975 [1:12:43<47:13,  2.31s/it]
 59%|█████▉    | 1750/2975 [1:12:45<46:59,  2.30s/it]
                                                     

 59%|█████▉    | 1750/2975 [1:12:45<46:59,  2.30s/it]
 59%|█████▉    | 1751/2975 [1:12:47<46:51,  2.30s/it]
 59%|█████▉    | 1752/2975 [1:12:50<46:43,  2.29s/it]
 59%|█████▉    | 1753/2975 [1:12:52<46:43,  2.29s/it]
 59%|█████▉    | 1754/2975 [1:12:54<46:36,  2.29s/it]
 59%|█████▉    | 1755/2975 [1:12:56<46:31,  2.29s/it]
 59%|█████▉    | 1756/2975 [1:12:59<47:20,  2.33s/it]
 59%|█████▉    | 1757/2975 [1:13:01<47:04,  2.32s/it]
 59%|█████▉    | 1758/2975 [1:13:04<47:29,  2.34s/it]
 59%|█████▉    | 1759/2975 [1:13:06<47:33,  2.35s/it]
 59%|█████▉    | 1760/2975 [1:13:08<47:10,  2.33s/it]
                                                     

 59%|████�
0: {'loss': 0.4231, 'grad_norm': 0.4721161161492819, 'learning_rate': 3e-06, 'epoch': 0.59}
0: ��▉    | 1760/2975 [1:13:08<47:10,  2.33s/it]
 59%|█████▉    | 1761/2975 [1:13:11<46:53,  2.32s/it]
 59%|█████▉    | 1762/2975 [1:13:13<46:42,  2.31s/it]
 59%|█████▉    | 1763/2975 [1:13:15<46:31,  2.30s/it]
 59%|█████▉    | 1764/2975 [1:13:18<47:47,  2.37s/it]
 59%|█████▉    | 1765/2975 [1:13:20<47:16,  2.34s/it]
 59%|█████▉    | 1766/2975 [1:13:22<48:30,  2.41s/it]
 59%|█████▉    | 1767/2975 [1:13:25<47:53,  2.38s/it]
 59%|█████▉    | 1768/2975 [1:13:27<47:17,  2.35s/it]
 59%|█████▉    | 1769/2975 [1:13:29<46:49,  2.33s/it]
 59%|█████▉    | 1770/2975 [1:13:32<46:28,  2.31s/it]
                                                     

 59%|█████▉    | 1770/2975 [1:13:32<46:28,  2.31s/it]
 60%|█████▉    | 1771/2975 [1:13:34<46:35,  2.32s/it]
 60%|█████▉    | 1772/2975 [1:13:36<46:21,  2.31s/it]
 60%|█████▉    | 1773/2975 [1:13:39<46:15,  2.31s/i
0: {'loss': 0.4365, 'grad_norm': 0.4743627600258426, 'learning_rate': 3e-06, 'epoch': 0.6}
0: t]
 60%|█████▉    | 1774/2975 [1:13:41<46:04,  2.30s/it]
 60%|█████▉    | 1775/2975 [1:13:43<46:25,  2.32s/it]
 60%|█████▉    | 1776/2975 [1:13:45<46:14,  2.31s/it]
 60%|█████▉    | 1777/2975 [1:13:48<46:04,  2.31s/it]
 60%|█████▉    | 1778/2975 [1:13:51<50:23,  2.53s/it]
 60%|█████▉    | 1779/2975 [1:13:53<48:54,  2.45s/it]
 60%|█████▉    | 1780/2975 [1:13:55<47:49,  2.40s/it]
                                                     

 60%|█████▉    | 1780/2975 [1:13:55<47:49,  2.40s/it]
 60%|█████▉    | 1781/2975 [1:13:58<47:04,  2.37s/it]
 60%|█████▉    | 1782/2975 [1:14:00<46:30,  2.34s/it]
 60%|█████▉    | 1783/2975 [1:14:02<46:09,  2.32s/it]
 60%|█████▉    | 1784/2975 [1:14:05<46:18,  2.33s/it]
 60%|██████    | 1785/2975 [1:14:07<45:57,  2.32s/it]
 60%|██████    | 1786/2975 [1:14:34<3:11:19,  9.65s/it]
 60%|██████    | 1787/2975 [
0: {'loss': 0.4257, 'grad_norm': 0.4786419703551591, 'learning_rate': 3e-06, 'epoch': 0.6}
0: 1:14:36<2:27:25,  7.45s/it]
 60%|██████    | 1788/2975 [1:14:38<1:56:50,  5.91s/it]
 60%|██████    | 1789/2975 [1:14:41<1:35:41,  4.84s/it]
 60%|██████    | 1790/2975 [1:14:43<1:20:29,  4.08s/it]
                                                       

 60%|██████    | 1790/2975 [1:14:43<1:20:29,  4.08s/it]
 60%|██████    | 1791/2975 [1:14:45<1:09:47,  3.54s/it]
 60%|██████    | 1792/2975 [1:14:47<1:02:19,  3.16s/it]
 60%|██████    | 1793/2975 [1:14:50<57:07,  2.90s/it]  
 60%|██████    | 1794/2975 [1:14:52<53:24,  2.71s/it]
 60%|██████    | 1795/2975 [1:14:54<50:46,  2.58s/it]
 60%|██████    | 1796/2975 [1:14:57<48:56,  2.49s/it]
 60%|██████    | 1797/2975 [1:14:59<47:44,  2.43s/it]
 60%|██████    | 1798/2975 [1:15:01<46:56,  2.39s/it]
 60%|██████    | 1799/2975 [1:15:03<46:17,  2.36s/it]
 61%|██████    | 1800/2975 [1:15:06<45:45,  2.34s/it]
 
0: {'loss': 0.4226, 'grad_norm': 0.4952295185095891, 'learning_rate': 3e-06, 'epoch': 0.6}
0: {'loss': 0.4274, 'grad_norm': 0.4666489309874668, 'learning_rate': 3e-06, 'epoch': 0.61}
0:                                                     

 61%|██████    | 1800/2975 [1:15:06<45:45,  2.34s/it]
 61%|██████    | 1801/2975 [1:15:08<45:18,  2.32s/it]
 61%|██████    | 1802/2975 [1:15:10<45:01,  2.30s/it]
 61%|██████    | 1803/2975 [1:15:13<44:56,  2.30s/it]
 61%|██████    | 1804/2975 [1:15:15<44:56,  2.30s/it]
 61%|██████    | 1805/2975 [1:15:17<44:55,  2.30s/it]
 61%|██████    | 1806/2975 [1:15:19<44:51,  2.30s/it]
 61%|██████    | 1807/2975 [1:15:22<44:44,  2.30s/it]
 61%|██████    | 1808/2975 [1:15:24<44:40,  2.30s/it]
 61%|██████    | 1809/2975 [1:15:26<44:38,  2.30s/it]
 61%|██████    | 1810/2975 [1:15:29<44:38,  2.30s/it]
                                                     

 61%|██████    | 1810/2975 [1:15:29<44:38,  2.30s/it]
 61%|██████    | 1811/2975 [1:15:31<44:26,  2.29s/it]
 61%|██████    | 1812/2975 [1:15:33<44:19,  2
0: {'loss': 0.4188, 'grad_norm': 0.46058288624184335, 'learning_rate': 3e-06, 'epoch': 0.61}
0: .29s/it]
 61%|██████    | 1813/2975 [1:15:36<45:13,  2.33s/it]
 61%|██████    | 1814/2975 [1:15:38<44:54,  2.32s/it]
 61%|██████    | 1815/2975 [1:15:40<44:40,  2.31s/it]
 61%|██████    | 1816/2975 [1:15:43<44:56,  2.33s/it]
 61%|██████    | 1817/2975 [1:15:45<44:43,  2.32s/it]
 61%|██████    | 1818/2975 [1:15:47<44:46,  2.32s/it]
 61%|██████    | 1819/2975 [1:15:50<44:29,  2.31s/it]
 61%|██████    | 1820/2975 [1:15:52<44:14,  2.30s/it]
                                                     

 61%|██████    | 1820/2975 [1:15:52<44:14,  2.30s/it]
 61%|██████    | 1821/2975 [1:15:54<44:03,  2.29s/it]
 61%|██████    | 1822/2975 [1:15:56<44:24,  2.31s/it]
 61%|██████▏   | 1823/2975 [1:15:59<44:40,  2.33s/it]
 61%|██████▏   | 1824/2975 [1:16:01<44:43,  2.33s/it]
 61%|██████▏   | 1825/2975 [1:16:03<44:26,  2.32s/it]
 61%|██████▏   |
0: {'loss': 0.4309, 'grad_norm': 0.4634175974309771, 'learning_rate': 3e-06, 'epoch': 0.61}
0:  1826/2975 [1:16:06<44:07,  2.30s/it]
 61%|██████▏   | 1827/2975 [1:16:08<44:00,  2.30s/it]
 61%|██████▏   | 1828/2975 [1:16:10<44:01,  2.30s/it]
 61%|██████▏   | 1829/2975 [1:16:13<43:57,  2.30s/it]
 62%|██████▏   | 1830/2975 [1:16:15<43:51,  2.30s/it]
                                                     

 62%|██████▏   | 1830/2975 [1:16:15<43:51,  2.30s/it]
 62%|██████▏   | 1831/2975 [1:16:17<43:52,  2.30s/it]
 62%|██████▏   | 1832/2975 [1:16:19<43:45,  2.30s/it]
 62%|██████▏   | 1833/2975 [1:16:22<43:36,  2.29s/it]
 62%|██████▏   | 1834/2975 [1:16:24<43:29,  2.29s/it]
 62%|██████▏   | 1835/2975 [1:16:26<43:40,  2.30s/it]
 62%|██████▏   | 1836/2975 [1:16:29<43:30,  2.29s/it]
 62%|██████▏   | 1837/2975 [1:16:31<43:23,  2.29s/it]
 62%|██████▏   | 1838/2975 [1:16:33<43:22,  2.29s/it]
 62%|██████▏   | 1839/2975 [1:16:
0: {'loss': 0.4323, 'grad_norm': 0.4654200341672867, 'learning_rate': 3e-06, 'epoch': 0.62}
0: {'loss': 0.4291, 'grad_norm': 0.476262707929569, 'learning_rate': 3e-06, 'epoch': 0.62}
0: 35<43:23,  2.29s/it]
 62%|██████▏   | 1840/2975 [1:16:38<43:18,  2.29s/it]
                                                     

 62%|██████▏   | 1840/2975 [1:16:38<43:18,  2.29s/it]
 62%|██████▏   | 1841/2975 [1:16:40<43:18,  2.29s/it]
 62%|██████▏   | 1842/2975 [1:16:42<43:14,  2.29s/it]
 62%|██████▏   | 1843/2975 [1:16:45<43:02,  2.28s/it]
 62%|██████▏   | 1844/2975 [1:16:47<42:57,  2.28s/it]
 62%|██████▏   | 1845/2975 [1:16:49<42:57,  2.28s/it]
 62%|██████▏   | 1846/2975 [1:16:51<42:53,  2.28s/it]
 62%|██████▏   | 1847/2975 [1:16:54<42:56,  2.28s/it]
 62%|██████▏   | 1848/2975 [1:16:56<42:55,  2.29s/it]
 62%|██████▏   | 1849/2975 [1:16:58<42:56,  2.29s/it]
 62%|██████▏   | 1850/2975 [1:17:01<42:50,  2.29s/it]
                                                     

 62%|██████▏   | 1850/2975 [1:17:01<42:50,  2.29s/it]
 62%|█�
0: {'loss': 0.4175, 'grad_norm': 0.45440629833691026, 'learning_rate': 3e-06, 'epoch': 0.62}
0: ��████▏   | 1851/2975 [1:17:03<43:10,  2.30s/it]
 62%|██████▏   | 1852/2975 [1:17:05<43:35,  2.33s/it]
 62%|██████▏   | 1853/2975 [1:17:08<43:26,  2.32s/it]
 62%|██████▏   | 1854/2975 [1:17:10<43:08,  2.31s/it]
 62%|██████▏   | 1855/2975 [1:17:12<42:56,  2.30s/it]
 62%|██████▏   | 1856/2975 [1:17:15<42:48,  2.30s/it]
 62%|██████▏   | 1857/2975 [1:17:17<42:40,  2.29s/it]
 62%|██████▏   | 1858/2975 [1:17:19<42:44,  2.30s/it]
 62%|██████▏   | 1859/2975 [1:17:22<43:48,  2.36s/it]
 63%|██████▎   | 1860/2975 [1:17:24<43:21,  2.33s/it]
                                                     

 63%|██████▎   | 1860/2975 [1:17:24<43:21,  2.33s/it]
 63%|██████▎   | 1861/2975 [1:17:26<44:33,  2.40s/it]
 63%|██████▎   | 1862/2975 [1:17:29<44:03,  2.37s/it]
 63%|██████▎   | 1863/2975 [1:17:31<43:33,  2.35s/it]
 63%|██████▎
0: {'loss': 0.4347, 'grad_norm': 0.4674441565642017, 'learning_rate': 3e-06, 'epoch': 0.63}
0:    | 1864/2975 [1:17:34<45:23,  2.45s/it]
 63%|██████▎   | 1865/2975 [1:17:36<44:17,  2.39s/it]
 63%|██████▎   | 1866/2975 [1:17:38<43:43,  2.37s/it]
 63%|██████▎   | 1867/2975 [1:17:41<43:18,  2.35s/it]
 63%|██████▎   | 1868/2975 [1:17:43<42:54,  2.33s/it]
 63%|██████▎   | 1869/2975 [1:17:45<42:54,  2.33s/it]
 63%|██████▎   | 1870/2975 [1:17:47<42:45,  2.32s/it]
                                                     

 63%|██████▎   | 1870/2975 [1:17:47<42:45,  2.32s/it]
 63%|██████▎   | 1871/2975 [1:17:50<42:26,  2.31s/it]
 63%|██████▎   | 1872/2975 [1:17:52<42:10,  2.29s/it]
 63%|██████▎   | 1873/2975 [1:17:54<42:02,  2.29s/it]
 63%|██████▎   | 1874/2975 [1:17:57<42:13,  2.30s/it]
 63%|██████▎   | 1875/2975 [1:17:59<42:05,  2.30s/it]
 63%|██████▎   | 1876/2975 [1:18:01<42:15,  2.31s/it]
 63%|██████▎   | 1877/2975 [1
0: {'loss': 0.4261, 'grad_norm': 0.47916604426773474, 'learning_rate': 3e-06, 'epoch': 0.63}
0: :18:04<42:07,  2.30s/it]
 63%|██████▎   | 1878/2975 [1:18:06<41:57,  2.30s/it]
 63%|██████▎   | 1879/2975 [1:18:08<41:54,  2.29s/it]
 63%|██████▎   | 1880/2975 [1:18:10<41:42,  2.29s/it]
                                                     

 63%|██████▎   | 1880/2975 [1:18:10<41:42,  2.29s/it]
 63%|██████▎   | 1881/2975 [1:18:13<41:40,  2.29s/it]
 63%|██████▎   | 1882/2975 [1:18:15<41:37,  2.28s/it]
 63%|██████▎   | 1883/2975 [1:18:17<41:36,  2.29s/it]
 63%|██████▎   | 1884/2975 [1:18:20<41:36,  2.29s/it]
 63%|██████▎   | 1885/2975 [1:18:22<41:34,  2.29s/it]
 63%|██████▎   | 1886/2975 [1:18:24<41:51,  2.31s/it]
 63%|██████▎   | 1887/2975 [1:18:26<41:40,  2.30s/it]
 63%|██████▎   | 1888/2975 [1:18:29<41:34,  2.30s/it]
 63%|██████▎   | 1889/2975 [1:18:31<42:36,  2.35s/it]
 64%|██████▎   | 1890/2975 [1:18:34<43:27,  2.
0: {'loss': 0.4315, 'grad_norm': 0.5246101370263686, 'learning_rate': 3e-06, 'epoch': 0.64}
0: {'loss': 0.4322, 'grad_norm': 0.48400445697233857, 'learning_rate': 3e-06, 'epoch': 0.64}
0: 40s/it]
                                                     

 64%|██████▎   | 1890/2975 [1:18:34<43:27,  2.40s/it]
 64%|██████▎   | 1891/2975 [1:18:36<42:45,  2.37s/it]
 64%|██████▎   | 1892/2975 [1:18:38<42:07,  2.33s/it]
 64%|██████▎   | 1893/2975 [1:18:41<41:45,  2.32s/it]
 64%|██████▎   | 1894/2975 [1:18:43<41:35,  2.31s/it]
 64%|██████▎   | 1895/2975 [1:18:45<41:39,  2.31s/it]
 64%|██████▎   | 1896/2975 [1:18:47<41:19,  2.30s/it]
 64%|██████▍   | 1897/2975 [1:18:50<41:09,  2.29s/it]
 64%|██████▍   | 1898/2975 [1:18:52<41:37,  2.32s/it]
 64%|██████▍   | 1899/2975 [1:18:54<41:17,  2.30s/it]
 64%|██████▍   | 1900/2975 [1:18:57<41:13,  2.30s/it]
                                                     

 64%|██████▍   | 1900/2975 [1:18:57<41:13,  2.30s/it]
 64%|██████▍   | 1901/2975 [1:18:59<41:08,  2.30s/it]
 64%|█████�
0: {'loss': 0.4388, 'grad_norm': 0.4750300117526481, 'learning_rate': 3e-06, 'epoch': 0.64}
0: �▍   | 1902/2975 [1:19:01<42:21,  2.37s/it]
 64%|██████▍   | 1903/2975 [1:19:04<42:14,  2.36s/it]
 64%|██████▍   | 1904/2975 [1:19:06<41:41,  2.34s/it]
 64%|██████▍   | 1905/2975 [1:19:08<41:24,  2.32s/it]
 64%|██████▍   | 1906/2975 [1:19:11<41:12,  2.31s/it]
 64%|██████▍   | 1907/2975 [1:19:13<41:08,  2.31s/it]
 64%|██████▍   | 1908/2975 [1:19:15<40:56,  2.30s/it]
 64%|██████▍   | 1909/2975 [1:19:18<40:49,  2.30s/it]
 64%|██████▍   | 1910/2975 [1:19:20<40:41,  2.29s/it]
                                                     

 64%|██████▍   | 1910/2975 [1:19:20<40:41,  2.29s/it]
 64%|██████▍   | 1911/2975 [1:19:22<40:36,  2.29s/it]
 64%|██████▍   | 1912/2975 [1:19:24<40:34,  2.29s/it]
 64%|██████▍   | 1913/2975 [1:19:27<40:29,  2.29s/it]
 64%|██████▍   | 1914/2975 [1:19:29<40:33,  2.29s/it]
 64%|██████▍   | 1915/297
0: {'loss': 0.4181, 'grad_norm': 0.45659573641739076, 'learning_rate': 3e-06, 'epoch': 0.65}
0: 5 [1:19:31<40:25,  2.29s/it]
 64%|██████▍   | 1916/2975 [1:19:34<40:25,  2.29s/it]
 64%|██████▍   | 1917/2975 [1:19:36<43:30,  2.47s/it]
 64%|██████▍   | 1918/2975 [1:19:39<42:33,  2.42s/it]
 65%|██████▍   | 1919/2975 [1:19:41<41:55,  2.38s/it]
 65%|██████▍   | 1920/2975 [1:19:43<41:45,  2.37s/it]
                                                     

 65%|██████▍   | 1920/2975 [1:19:43<41:45,  2.37s/it]
 65%|██████▍   | 1921/2975 [1:19:46<42:11,  2.40s/it]
 65%|██████▍   | 1922/2975 [1:19:48<41:31,  2.37s/it]
 65%|██████▍   | 1923/2975 [1:19:51<44:56,  2.56s/it]
 65%|██████▍   | 1924/2975 [1:19:53<43:32,  2.49s/it]
 65%|██████▍   | 1925/2975 [1:19:56<42:31,  2.43s/it]
 65%|██████▍   | 1926/2975 [1:19:58<41:37,  2.38s/it]
 65%|██████▍   | 1927/2975 [1:20:00<41:04,  2.35s/it]
 65%|██████▍   | 1928/2975 [1:20:03<40:41,
0: {'loss': 0.4392, 'grad_norm': 0.46765059784836605, 'learning_rate': 3e-06, 'epoch': 0.65}
0: {'loss': 0.4281, 'grad_norm': 0.46775101107223993, 'learning_rate': 3e-06, 'epoch': 0.65}
0:   2.33s/it]
 65%|██████▍   | 1929/2975 [1:20:05<40:28,  2.32s/it]
 65%|██████▍   | 1930/2975 [1:20:07<40:11,  2.31s/it]
                                                     

 65%|██████▍   | 1930/2975 [1:20:07<40:11,  2.31s/it]
 65%|██████▍   | 1931/2975 [1:20:09<40:01,  2.30s/it]
 65%|██████▍   | 1932/2975 [1:20:12<39:52,  2.29s/it]
 65%|██████▍   | 1933/2975 [1:20:15<42:33,  2.45s/it]
 65%|██████▌   | 1934/2975 [1:20:17<41:37,  2.40s/it]
 65%|██████▌   | 1935/2975 [1:20:19<41:05,  2.37s/it]
 65%|██████▌   | 1936/2975 [1:20:21<40:38,  2.35s/it]
 65%|██████▌   | 1937/2975 [1:20:24<40:14,  2.33s/it]
 65%|██████▌   | 1938/2975 [1:20:26<40:02,  2.32s/it]
 65%|██████▌   | 1939/2975 [1:20:28<40:03,  2.32s/it]
 65%|██████▌   | 1940/2975 [1:20:31<39:47,  2.31s/it]
                                                     

 65%|████�
0: {'loss': 0.4237, 'grad_norm': 0.44406299055443454, 'learning_rate': 3e-06, 'epoch': 0.66}
0: ��█▌   | 1940/2975 [1:20:31<39:47,  2.31s/it]
 65%|██████▌   | 1941/2975 [1:20:33<40:22,  2.34s/it]
 65%|██████▌   | 1942/2975 [1:20:35<40:03,  2.33s/it]
 65%|██████▌   | 1943/2975 [1:20:38<39:47,  2.31s/it]
 65%|██████▌   | 1944/2975 [1:20:40<39:35,  2.30s/it]
 65%|██████▌   | 1945/2975 [1:20:42<39:43,  2.31s/it]
 65%|██████▌   | 1946/2975 [1:20:45<39:30,  2.30s/it]
 65%|██████▌   | 1947/2975 [1:20:47<39:22,  2.30s/it]
 65%|██████▌   | 1948/2975 [1:20:49<39:12,  2.29s/it]
 66%|██████▌   | 1949/2975 [1:20:51<39:09,  2.29s/it]
 66%|██████▌   | 1950/2975 [1:20:54<39:03,  2.29s/it]
                                                     

 66%|██████▌   | 1950/2975 [1:20:54<39:03,  2.29s/it]
 66%|██████▌   | 1951/2975 [1:20:56<39:03,  2.29s/it]
 66%|██████▌   | 1952/2975 [1:20:58<39:01,  2.29s/it]
 66%|██████▌   | 1953
0: {'loss': 0.4293, 'grad_norm': 0.46692222474571243, 'learning_rate': 3e-06, 'epoch': 0.66}
0: /2975 [1:21:01<38:59,  2.29s/it]
 66%|██████▌   | 1954/2975 [1:21:03<39:02,  2.29s/it]
 66%|██████▌   | 1955/2975 [1:21:05<39:38,  2.33s/it]
 66%|██████▌   | 1956/2975 [1:21:08<39:16,  2.31s/it]
 66%|██████▌   | 1957/2975 [1:21:10<39:01,  2.30s/it]
 66%|██████▌   | 1958/2975 [1:21:12<38:54,  2.30s/it]
 66%|██████▌   | 1959/2975 [1:21:15<39:50,  2.35s/it]
 66%|██████▌   | 1960/2975 [1:21:17<39:29,  2.33s/it]
                                                     

 66%|██████▌   | 1960/2975 [1:21:17<39:29,  2.33s/it]
 66%|██████▌   | 1961/2975 [1:21:19<39:10,  2.32s/it]
 66%|██████▌   | 1962/2975 [1:21:21<39:22,  2.33s/it]
 66%|██████▌   | 1963/2975 [1:21:24<39:10,  2.32s/it]
 66%|██████▌   | 1964/2975 [1:21:26<39:17,  2.33s/it]
 66%|██████▌   | 1965/2975 [1:21:28<38:52,  2.31s/it]
 66%|██████▌   | 1966/2975 [1:21:31<38
0: {'loss': 0.4165, 'grad_norm': 0.44471266527204173, 'learning_rate': 3e-06, 'epoch': 0.66}
0: :39,  2.30s/it]
 66%|██████▌   | 1967/2975 [1:21:33<39:34,  2.36s/it]
 66%|██████▌   | 1968/2975 [1:21:35<39:08,  2.33s/it]
 66%|██████▌   | 1969/2975 [1:21:38<38:53,  2.32s/it]
 66%|██████▌   | 1970/2975 [1:21:40<39:37,  2.37s/it]
                                                     

 66%|██████▌   | 1970/2975 [1:21:40<39:37,  2.37s/it]
 66%|██████▋   | 1971/2975 [1:21:43<39:16,  2.35s/it]
 66%|██████▋   | 1972/2975 [1:21:45<39:07,  2.34s/it]
 66%|██████▋   | 1973/2975 [1:21:47<38:52,  2.33s/it]
 66%|██████▋   | 1974/2975 [1:21:49<39:01,  2.34s/it]
 66%|██████▋   | 1975/2975 [1:21:52<38:39,  2.32s/it]
 66%|██████▋   | 1976/2975 [1:21:54<38:23,  2.31s/it]
 66%|██████▋   | 1977/2975 [1:21:56<38:14,  2.30s/it]
 66%|██████▋   | 1978/2975 [1:21:59<38:09,  2.30s/it]
 67%|██████▋   | 1979/2975 [1:22:01<40:14,  2.42s/it]
 
0: {'loss': 0.412, 'grad_norm': 0.47609234132254197, 'learning_rate': 3e-06, 'epoch': 0.67}
0: {'loss': 0.4266, 'grad_norm': 0.4921688959999424, 'learning_rate': 3e-06, 'epoch': 0.67}
0: 67%|██████▋   | 1980/2975 [1:22:04<39:46,  2.40s/it]
                                                     

 67%|██████▋   | 1980/2975 [1:22:04<39:46,  2.40s/it]
 67%|██████▋   | 1981/2975 [1:22:06<39:17,  2.37s/it]
 67%|██████▋   | 1982/2975 [1:22:08<38:47,  2.34s/it]
 67%|██████▋   | 1983/2975 [1:22:11<38:24,  2.32s/it]
 67%|██████▋   | 1984/2975 [1:22:13<38:07,  2.31s/it]
 67%|██████▋   | 1985/2975 [1:22:15<38:02,  2.31s/it]
 67%|██████▋   | 1986/2975 [1:22:17<37:51,  2.30s/it]
 67%|██████▋   | 1987/2975 [1:22:20<37:41,  2.29s/it]
 67%|██████▋   | 1988/2975 [1:22:22<37:32,  2.28s/it]
 67%|██████▋   | 1989/2975 [1:22:24<37:35,  2.29s/it]
 67%|██████▋   | 1990/2975 [1:22:27<37:36,  2.29s/it]
                                                     

 67%|██████▋   | 1990/2975 [1:22:27<37:36,  2.29s/it]
 67%|██████▋   | 
0: {'loss': 0.433, 'grad_norm': 0.47248132013501054, 'learning_rate': 3e-06, 'epoch': 0.67}
0: 1991/2975 [1:22:29<37:33,  2.29s/it]
 67%|██████▋   | 1992/2975 [1:22:31<37:27,  2.29s/it]
 67%|██████▋   | 1993/2975 [1:22:33<37:24,  2.29s/it]
 67%|██████▋   | 1994/2975 [1:22:36<37:19,  2.28s/it]
 67%|██████▋   | 1995/2975 [1:22:38<37:17,  2.28s/it]
 67%|██████▋   | 1996/2975 [1:22:40<37:54,  2.32s/it]
 67%|██████▋   | 1997/2975 [1:22:43<37:44,  2.32s/it]
 67%|██████▋   | 1998/2975 [1:22:45<37:31,  2.30s/it]
 67%|██████▋   | 1999/2975 [1:22:47<37:13,  2.29s/it]
 67%|██████▋   | 2000/2975 [1:22:50<37:32,  2.31s/it]
                                                     

 67%|██████▋   | 2000/2975 [1:22:50<37:32,  2.31s/it]
 67%|██████▋   | 2001/2975 [1:22:52<37:21,  2.30s/it]
 67%|██████▋   | 2002/2975 [1:22:54<37:15,  2.30s/it]
 67%|██████▋   | 2003/2975 [1:22:57<37:46,  2.33s/it]
 67%|██████▋   | 2004/2975 [1:22:5
0: {'loss': 0.4208, 'grad_norm': 0.45417164146030786, 'learning_rate': 3e-06, 'epoch': 0.68}
0: 9<37:33,  2.32s/it]
 67%|██████▋   | 2005/2975 [1:23:01<37:18,  2.31s/it]
 67%|██████▋   | 2006/2975 [1:23:03<37:21,  2.31s/it]
 67%|██████▋   | 2007/2975 [1:23:06<37:07,  2.30s/it]
 67%|██████▋   | 2008/2975 [1:23:08<37:40,  2.34s/it]
 68%|██████▊   | 2009/2975 [1:23:11<38:41,  2.40s/it]
 68%|██████▊   | 2010/2975 [1:23:13<38:02,  2.37s/it]
                                                     

 68%|██████▊   | 2010/2975 [1:23:13<38:02,  2.37s/it]
 68%|██████▊   | 2011/2975 [1:23:15<37:40,  2.35s/it]
 68%|██████▊   | 2012/2975 [1:23:18<37:18,  2.32s/it]
 68%|██████▊   | 2013/2975 [1:23:20<37:08,  2.32s/it]
 68%|██████▊   | 2014/2975 [1:23:22<36:57,  2.31s/it]
 68%|██████▊   | 2015/2975 [1:23:24<37:06,  2.32s/it]
 68%|██████▊   | 2016/2975 [1:23:27<36:51,  2.31s/it]
 68%|██████▊   | 2017/2975 [1:23:29<36:56,  2.31s/i
0: {'loss': 0.4103, 'grad_norm': 0.47851279940369545, 'learning_rate': 3e-06, 'epoch': 0.68}
0: t]
 68%|██████▊   | 2018/2975 [1:23:31<36:48,  2.31s/it]
 68%|██████▊   | 2019/2975 [1:23:34<36:38,  2.30s/it]
 68%|██████▊   | 2020/2975 [1:23:36<36:33,  2.30s/it]
                                                     

 68%|██████▊   | 2020/2975 [1:23:36<36:33,  2.30s/it]
 68%|██████▊   | 2021/2975 [1:23:38<36:29,  2.29s/it]
 68%|██████▊   | 2022/2975 [1:23:41<36:28,  2.30s/it]
 68%|██████▊   | 2023/2975 [1:23:43<36:23,  2.29s/it]
 68%|██████▊   | 2024/2975 [1:23:45<36:14,  2.29s/it]
 68%|██████▊   | 2025/2975 [1:23:48<38:14,  2.41s/it]
 68%|██████▊   | 2026/2975 [1:23:50<37:30,  2.37s/it]
 68%|██████▊   | 2027/2975 [1:23:52<37:01,  2.34s/it]
 68%|██████▊   | 2028/2975 [1:23:55<36:42,  2.33s/it]
 68%|██████▊   | 2029/2975 [1:23:57<36:28,  2.31s/it]
 68%|██████▊   | 2030/2975 [1:23:59<36:21,  2.31s/it]
              
0: {'loss': 0.4314, 'grad_norm': 0.4793011107758793, 'learning_rate': 3e-06, 'epoch': 0.68}
0: {'loss': 0.4122, 'grad_norm': 0.43588147553183026, 'learning_rate': 3e-06, 'epoch': 0.69}
0:                                        

 68%|██████▊   | 2030/2975 [1:23:59<36:21,  2.31s/it]
 68%|██████▊   | 2031/2975 [1:24:01<36:11,  2.30s/it]
 68%|██████▊   | 2032/2975 [1:24:04<36:04,  2.29s/it]
 68%|██████▊   | 2033/2975 [1:24:06<37:25,  2.38s/it]
 68%|██████▊   | 2034/2975 [1:24:09<36:55,  2.35s/it]
 68%|██████▊   | 2035/2975 [1:24:11<36:31,  2.33s/it]
 68%|██████▊   | 2036/2975 [1:24:13<36:21,  2.32s/it]
 68%|██████▊   | 2037/2975 [1:24:16<36:09,  2.31s/it]
 69%|██████▊   | 2038/2975 [1:24:18<35:56,  2.30s/it]
 69%|██████▊   | 2039/2975 [1:24:20<35:49,  2.30s/it]
 69%|██████▊   | 2040/2975 [1:24:22<36:17,  2.33s/it]
                                                     

 69%|██████▊   | 2040/2975 [1:24:22<36:17,  2.33s/it]
 69%|██████▊   | 2041/2975 [1:24:25<36:05,  2.32s/it]
 69%|██████▊   | 2042/2975 [1:
0: {'loss': 0.4168, 'grad_norm': 0.4685522571296016, 'learning_rate': 3e-06, 'epoch': 0.69}
0: 24:27<36:01,  2.32s/it]
 69%|██████▊   | 2043/2975 [1:24:29<36:19,  2.34s/it]
 69%|██████▊   | 2044/2975 [1:24:32<36:13,  2.34s/it]
 69%|██████▊   | 2045/2975 [1:24:34<36:07,  2.33s/it]
 69%|██████▉   | 2046/2975 [1:24:36<35:50,  2.31s/it]
 69%|██████▉   | 2047/2975 [1:24:39<35:41,  2.31s/it]
 69%|██████▉   | 2048/2975 [1:24:41<35:28,  2.30s/it]
 69%|██████▉   | 2049/2975 [1:24:43<35:19,  2.29s/it]
 69%|██████▉   | 2050/2975 [1:24:46<35:17,  2.29s/it]
                                                     

 69%|██████▉   | 2050/2975 [1:24:46<35:17,  2.29s/it]
 69%|██████▉   | 2051/2975 [1:24:48<35:17,  2.29s/it]
 69%|██████▉   | 2052/2975 [1:24:50<35:18,  2.30s/it]
 69%|██████▉   | 2053/2975 [1:24:52<35:15,  2.29s/it]
 69%|██████▉   | 2054/2975 [1:24:55<35:10,  2.29s/it]
 69%|██████▉   | 2055/2975 [1:24:57<35:04,  2.2
0: {'loss': 0.4194, 'grad_norm': 0.4718215796429598, 'learning_rate': 3e-06, 'epoch': 0.69}
0: 9s/it]
 69%|██████▉   | 2056/2975 [1:24:59<35:02,  2.29s/it]
 69%|██████▉   | 2057/2975 [1:25:02<34:57,  2.28s/it]
 69%|██████▉   | 2058/2975 [1:25:04<34:53,  2.28s/it]
 69%|██████▉   | 2059/2975 [1:25:06<35:02,  2.29s/it]
 69%|██████▉   | 2060/2975 [1:25:08<34:55,  2.29s/it]
                                                     

 69%|██████▉   | 2060/2975 [1:25:08<34:55,  2.29s/it]
 69%|██████▉   | 2061/2975 [1:25:11<34:50,  2.29s/it]
 69%|██████▉   | 2062/2975 [1:25:13<34:44,  2.28s/it]
 69%|██████▉   | 2063/2975 [1:25:15<34:45,  2.29s/it]
 69%|██████▉   | 2064/2975 [1:25:18<34:49,  2.29s/it]
 69%|██████▉   | 2065/2975 [1:25:20<34:46,  2.29s/it]
 69%|██████▉   | 2066/2975 [1:25:22<34:40,  2.29s/it]
 69%|██████▉   | 2067/2975 [1:25:24<34:38,  2.29s/it]
 70%|██████▉   | 2068/2975 [1:25:27<34:42,  2.30s/it]
 70%|█�
0: {'loss': 0.4171, 'grad_norm': 0.4596872163115101, 'learning_rate': 3e-06, 'epoch': 0.7}
0: {'loss': 0.4216, 'grad_norm': 0.4495281474686137, 'learning_rate': 3e-06, 'epoch': 0.7}
0: �████▉   | 2069/2975 [1:25:29<35:43,  2.37s/it]
 70%|██████▉   | 2070/2975 [1:25:32<35:23,  2.35s/it]
                                                     

 70%|██████▉   | 2070/2975 [1:25:32<35:23,  2.35s/it]
 70%|██████▉   | 2071/2975 [1:25:34<35:07,  2.33s/it]
 70%|██████▉   | 2072/2975 [1:25:36<35:09,  2.34s/it]
 70%|██████▉   | 2073/2975 [1:25:39<34:52,  2.32s/it]
 70%|██████▉   | 2074/2975 [1:25:41<34:41,  2.31s/it]
 70%|██████▉   | 2075/2975 [1:25:43<34:31,  2.30s/it]
 70%|██████▉   | 2076/2975 [1:25:46<35:43,  2.38s/it]
 70%|██████▉   | 2077/2975 [1:25:48<35:11,  2.35s/it]
 70%|██████▉   | 2078/2975 [1:25:50<34:50,  2.33s/it]
 70%|██████▉   | 2079/2975 [1:25:53<34:38,  2.32s/it]
 70%|██████▉   | 2080/2975 [1:25:55<34:26,  2.31s/it]
                                                     

 70%|██████▉   | 2080/2975
0: {'loss': 0.4095, 'grad_norm': 0.4728783957794599, 'learning_rate': 3e-06, 'epoch': 0.7}
0:  [1:25:55<34:26,  2.31s/it]
 70%|██████▉   | 2081/2975 [1:25:57<34:21,  2.31s/it]
 70%|██████▉   | 2082/2975 [1:25:59<34:12,  2.30s/it]
 70%|███████   | 2083/2975 [1:26:02<34:06,  2.29s/it]
 70%|███████   | 2084/2975 [1:26:04<34:14,  2.31s/it]
 70%|███████   | 2085/2975 [1:26:06<34:27,  2.32s/it]
 70%|███████   | 2086/2975 [1:26:09<34:52,  2.35s/it]
 70%|███████   | 2087/2975 [1:26:11<34:33,  2.33s/it]
 70%|███████   | 2088/2975 [1:26:14<35:32,  2.40s/it]
 70%|███████   | 2089/2975 [1:26:16<34:59,  2.37s/it]
 70%|███████   | 2090/2975 [1:26:18<34:30,  2.34s/it]
                                                     

 70%|███████   | 2090/2975 [1:26:18<34:30,  2.34s/it]
 70%|███████   | 2091/2975 [1:26:20<34:13,  2.32s/it]
 70%|███████   | 2092/2975 [1:26:23<33:59,  2.31s/it]
 70%|███████   | 2093/2975 [1:26:25<33:54, 
0: {'loss': 0.4149, 'grad_norm': 0.47346837393040664, 'learning_rate': 3e-06, 'epoch': 0.71}
0:  2.31s/it]
 70%|███████   | 2094/2975 [1:26:27<33:44,  2.30s/it]
 70%|███████   | 2095/2975 [1:26:30<33:38,  2.29s/it]
 70%|███████   | 2096/2975 [1:26:32<33:33,  2.29s/it]
 70%|███████   | 2097/2975 [1:26:34<33:27,  2.29s/it]
 71%|███████   | 2098/2975 [1:26:37<33:37,  2.30s/it]
 71%|███████   | 2099/2975 [1:26:39<33:31,  2.30s/it]
 71%|███████   | 2100/2975 [1:26:41<33:41,  2.31s/it]
                                                     

 71%|███████   | 2100/2975 [1:26:41<33:41,  2.31s/it]
 71%|███████   | 2101/2975 [1:26:43<33:47,  2.32s/it]
 71%|███████   | 2102/2975 [1:26:46<34:14,  2.35s/it]
 71%|███████   | 2103/2975 [1:26:48<33:51,  2.33s/it]
 71%|███████   | 2104/2975 [1:26:50<33:30,  2.31s/it]
 71%|███████   | 2105/2975 [1:26:53<33:15,  2.29s/it]
 71%|███████   | 2106/2975 [1:26:55<33:07,  2.29s/it]
 71%|�
0: {'loss': 0.4211, 'grad_norm': 0.5059943827327955, 'learning_rate': 3e-06, 'epoch': 0.71}
0: ��██████   | 2107/2975 [1:26:57<32:58,  2.28s/it]
 71%|███████   | 2108/2975 [1:27:00<33:08,  2.29s/it]
 71%|███████   | 2109/2975 [1:27:02<33:00,  2.29s/it]
 71%|███████   | 2110/2975 [1:27:04<33:29,  2.32s/it]
                                                     

 71%|███████   | 2110/2975 [1:27:04<33:29,  2.32s/it]
 71%|███████   | 2111/2975 [1:27:07<34:07,  2.37s/it]
 71%|███████   | 2112/2975 [1:27:09<33:57,  2.36s/it]
 71%|███████   | 2113/2975 [1:27:11<33:31,  2.33s/it]
 71%|███████   | 2114/2975 [1:27:14<33:15,  2.32s/it]
 71%|███████   | 2115/2975 [1:27:16<33:05,  2.31s/it]
 71%|███████   | 2116/2975 [1:27:18<32:53,  2.30s/it]
 71%|███████   | 2117/2975 [1:27:20<32:46,  2.29s/it]
 71%|███████   | 2118/2975 [1:27:23<32:44,  2.29s/it]
 71%|███████   | 2119/2975 [1:27:25<32:56,  2.31s/it]
 71%|██████
0: {'loss': 0.4201, 'grad_norm': 0.48239588934696226, 'learning_rate': 3e-06, 'epoch': 0.71}
0: {'loss': 0.4133, 'grad_norm': 0.4634604695287565, 'learning_rate': 3e-06, 'epoch': 0.72}
0: █▏  | 2120/2975 [1:27:27<32:41,  2.29s/it]
                                                     

 71%|███████▏  | 2120/2975 [1:27:27<32:41,  2.29s/it]
 71%|███████▏  | 2121/2975 [1:27:30<32:37,  2.29s/it]
 71%|███████▏  | 2122/2975 [1:27:32<32:46,  2.31s/it]
 71%|███████▏  | 2123/2975 [1:27:34<32:36,  2.30s/it]
 71%|███████▏  | 2124/2975 [1:27:37<32:32,  2.29s/it]
 71%|███████▏  | 2125/2975 [1:27:39<32:26,  2.29s/it]
 71%|███████▏  | 2126/2975 [1:27:41<32:14,  2.28s/it]
 71%|███████▏  | 2127/2975 [1:27:43<32:12,  2.28s/it]
 72%|███████▏  | 2128/2975 [1:27:46<32:24,  2.30s/it]
 72%|███████▏  | 2129/2975 [1:27:48<32:40,  2.32s/it]
 72%|███████▏  | 2130/2975 [1:27:50<32:27,  2.30s/it]
                                                     

 72%|███████▏  | 2130/2975 [1:27:50<32:27,  2.30s/it]
 72%|███████�
0: {'loss': 0.4312, 'grad_norm': 0.46983223666425034, 'learning_rate': 3e-06, 'epoch': 0.72}
0: ��  | 2131/2975 [1:27:53<32:55,  2.34s/it]
 72%|███████▏  | 2132/2975 [1:27:55<32:34,  2.32s/it]
 72%|███████▏  | 2133/2975 [1:27:57<32:21,  2.31s/it]
 72%|███████▏  | 2134/2975 [1:28:00<32:13,  2.30s/it]
 72%|███████▏  | 2135/2975 [1:28:02<32:01,  2.29s/it]
 72%|███████▏  | 2136/2975 [1:28:04<32:08,  2.30s/it]
 72%|███████▏  | 2137/2975 [1:28:07<34:06,  2.44s/it]
 72%|███████▏  | 2138/2975 [1:28:09<34:13,  2.45s/it]
 72%|███████▏  | 2139/2975 [1:28:12<33:30,  2.40s/it]
 72%|███████▏  | 2140/2975 [1:28:14<33:03,  2.38s/it]
                                                     

 72%|███████▏  | 2140/2975 [1:28:14<33:03,  2.38s/it]
 72%|███████▏  | 2141/2975 [1:28:17<33:41,  2.42s/it]
 72%|███████▏  | 2142/2975 [1:28:19<33:00,  2.38s/it]
 72%|███████▏  | 2143/2975 [1:28:21<32:35,  2.35s/it]
 72%|███�
0: {'loss': 0.4153, 'grad_norm': 0.48371598239817115, 'learning_rate': 3e-06, 'epoch': 0.72}
0: �███▏  | 2144/2975 [1:28:23<32:21,  2.34s/it]
 72%|███████▏  | 2145/2975 [1:28:26<32:10,  2.33s/it]
 72%|███████▏  | 2146/2975 [1:28:28<31:56,  2.31s/it]
 72%|███████▏  | 2147/2975 [1:28:30<32:21,  2.35s/it]
 72%|███████▏  | 2148/2975 [1:28:33<33:38,  2.44s/it]
 72%|███████▏  | 2149/2975 [1:28:35<33:01,  2.40s/it]
 72%|███████▏  | 2150/2975 [1:28:38<32:31,  2.37s/it]
                                                     

 72%|███████▏  | 2150/2975 [1:28:38<32:31,  2.37s/it]
 72%|███████▏  | 2151/2975 [1:28:40<32:12,  2.34s/it]
 72%|███████▏  | 2152/2975 [1:28:42<31:55,  2.33s/it]
 72%|███████▏  | 2153/2975 [1:28:45<31:44,  2.32s/it]
 72%|███████▏  | 2154/2975 [1:28:47<31:36,  2.31s/it]
 72%|███████▏  | 2155/2975 [1:28:49<31:36,  2.31s/it]
 72%|███████▏  | 2156/2975 [1:28:51<31:25,  2.30s/it]
 73%|
0: {'loss': 0.4118, 'grad_norm': 0.44650159360375086, 'learning_rate': 3e-06, 'epoch': 0.73}
0: ███████▎  | 2157/2975 [1:28:54<31:20,  2.30s/it]
 73%|███████▎  | 2158/2975 [1:28:56<31:10,  2.29s/it]
 73%|███████▎  | 2159/2975 [1:28:58<31:09,  2.29s/it]
 73%|███████▎  | 2160/2975 [1:29:01<31:02,  2.29s/it]
                                                     

 73%|███████▎  | 2160/2975 [1:29:01<31:02,  2.29s/it]
 73%|███████▎  | 2161/2975 [1:29:03<30:59,  2.28s/it]
 73%|███████▎  | 2162/2975 [1:29:05<31:13,  2.30s/it]
 73%|███████▎  | 2163/2975 [1:29:07<31:06,  2.30s/it]
 73%|███████▎  | 2164/2975 [1:29:10<30:51,  2.28s/it]
 73%|███████▎  | 2165/2975 [1:29:12<30:48,  2.28s/it]
 73%|███████▎  | 2166/2975 [1:29:14<30:50,  2.29s/it]
 73%|███████▎  | 2167/2975 [1:29:17<30:46,  2.29s/it]
 73%|███████▎  | 2168/2975 [1:29:19<30:43,  2.28s/it]
 73%|███████▎  | 2169/2975 [1:29:21<30:43,  2.29
0: {'loss': 0.4242, 'grad_norm': 0.4698561936467557, 'learning_rate': 3e-06, 'epoch': 0.73}
0: {'loss': 0.421, 'grad_norm': 0.4614595676350832, 'learning_rate': 3e-06, 'epoch': 0.73}
0: s/it]
 73%|███████▎  | 2170/2975 [1:29:23<30:37,  2.28s/it]
                                                     

 73%|███████▎  | 2170/2975 [1:29:23<30:37,  2.28s/it]
 73%|███████▎  | 2171/2975 [1:29:26<30:37,  2.29s/it]
 73%|███████▎  | 2172/2975 [1:29:28<30:37,  2.29s/it]
 73%|███████▎  | 2173/2975 [1:29:30<30:34,  2.29s/it]
 73%|███████▎  | 2174/2975 [1:29:33<30:30,  2.29s/it]
 73%|███████▎  | 2175/2975 [1:29:35<30:25,  2.28s/it]
 73%|███████▎  | 2176/2975 [1:29:37<30:45,  2.31s/it]
 73%|███████▎  | 2177/2975 [1:29:40<30:35,  2.30s/it]
 73%|███████▎  | 2178/2975 [1:29:42<30:26,  2.29s/it]
 73%|███████▎  | 2179/2975 [1:29:44<30:24,  2.29s/it]
 73%|███████▎  | 2180/2975 [1:29:46<30:22,  2.29s/it]
                                                     

 73%|███████▎  | 2180/2975 [1:29:46<30:22,  2.29s/it
0: {'loss': 0.4226, 'grad_norm': 0.4538688555235033, 'learning_rate': 3e-06, 'epoch': 0.74}
0: ]
 73%|███████▎  | 2181/2975 [1:29:49<30:44,  2.32s/it]
 73%|███████▎  | 2182/2975 [1:29:51<30:43,  2.33s/it]
 73%|███████▎  | 2183/2975 [1:29:53<30:48,  2.33s/it]
 73%|███████▎  | 2184/2975 [1:29:56<30:36,  2.32s/it]
 73%|███████▎  | 2185/2975 [1:29:58<30:25,  2.31s/it]
 73%|███████▎  | 2186/2975 [1:30:00<30:17,  2.30s/it]
 74%|███████▎  | 2187/2975 [1:30:03<30:12,  2.30s/it]
 74%|███████▎  | 2188/2975 [1:30:05<30:08,  2.30s/it]
 74%|███████▎  | 2189/2975 [1:30:07<30:04,  2.30s/it]
 74%|███████▎  | 2190/2975 [1:30:09<30:01,  2.29s/it]
                                                     

 74%|███████▎  | 2190/2975 [1:30:09<30:01,  2.29s/it]
 74%|███████▎  | 2191/2975 [1:30:12<30:20,  2.32s/it]
 74%|███████▎  | 2192/2975 [1:30:14<30:31,  2.34s/it]
 74%|███████▎  | 2193/2975 [1:30:17<30:18
0: {'loss': 0.4201, 'grad_norm': 0.4480957465107131, 'learning_rate': 3e-06, 'epoch': 0.74}
0: ,  2.32s/it]
 74%|███████▎  | 2194/2975 [1:30:19<30:07,  2.31s/it]
 74%|███████▍  | 2195/2975 [1:30:21<29:54,  2.30s/it]
 74%|███████▍  | 2196/2975 [1:30:23<29:50,  2.30s/it]
 74%|███████▍  | 2197/2975 [1:30:26<29:45,  2.30s/it]
 74%|███████▍  | 2198/2975 [1:30:28<29:43,  2.29s/it]
 74%|███████▍  | 2199/2975 [1:30:31<31:31,  2.44s/it]
 74%|███████▍  | 2200/2975 [1:30:33<31:11,  2.41s/it]
                                                     

 74%|███████▍  | 2200/2975 [1:30:33<31:11,  2.41s/it]
 74%|███████▍  | 2201/2975 [1:30:35<30:40,  2.38s/it]
 74%|███████▍  | 2202/2975 [1:30:38<30:17,  2.35s/it]
 74%|███████▍  | 2203/2975 [1:30:40<30:50,  2.40s/it]
 74%|███████▍  | 2204/2975 [1:30:42<30:22,  2.36s/it]
 74%|███████▍  | 2205/2975 [1:30:45<30:02,  2.34s/it]
 74%|███████▍  | 2206/2975 [1:
0: {'loss': 0.4316, 'grad_norm': 0.47704621716676593, 'learning_rate': 3e-06, 'epoch': 0.74}
0: 30:47<29:46,  2.32s/it]
 74%|███████▍  | 2207/2975 [1:30:49<29:36,  2.31s/it]
 74%|███████▍  | 2208/2975 [1:30:52<29:24,  2.30s/it]
 74%|███████▍  | 2209/2975 [1:30:54<29:15,  2.29s/it]
 74%|███████▍  | 2210/2975 [1:30:56<29:13,  2.29s/it]
                                                     

 74%|███████▍  | 2210/2975 [1:30:56<29:13,  2.29s/it]
 74%|███████▍  | 2211/2975 [1:30:59<29:52,  2.35s/it]
 74%|███████▍  | 2212/2975 [1:31:01<29:38,  2.33s/it]
 74%|███████▍  | 2213/2975 [1:31:03<30:05,  2.37s/it]
 74%|███████▍  | 2214/2975 [1:31:06<29:42,  2.34s/it]
 74%|███████▍  | 2215/2975 [1:31:08<29:27,  2.33s/it]
 74%|███████▍  | 2216/2975 [1:31:10<29:18,  2.32s/it]
 75%|███████▍  | 2217/2975 [1:31:13<29:02,  2.30s/it]
 75%|███████▍  | 2218/2975 [1:31:15<28:56,  2.29s/it]
 75%|███████▍  | 22
0: {'loss': 0.4103, 'grad_norm': 0.46881019660891904, 'learning_rate': 3e-06, 'epoch': 0.75}
0: {'loss': 0.4254, 'grad_norm': 0.47189960872281445, 'learning_rate': 3e-06, 'epoch': 0.75}
0: 19/2975 [1:31:17<28:53,  2.29s/it]
 75%|███████▍  | 2220/2975 [1:31:20<29:52,  2.37s/it]
                                                     

 75%|███████▍  | 2220/2975 [1:31:20<29:52,  2.37s/it]
 75%|███████▍  | 2221/2975 [1:31:22<29:45,  2.37s/it]
 75%|███████▍  | 2222/2975 [1:31:24<29:25,  2.34s/it]
 75%|███████▍  | 2223/2975 [1:31:27<29:11,  2.33s/it]
 75%|███████▍  | 2224/2975 [1:31:29<28:54,  2.31s/it]
 75%|███████▍  | 2225/2975 [1:31:31<28:47,  2.30s/it]
 75%|███████▍  | 2226/2975 [1:31:33<28:41,  2.30s/it]
 75%|███████▍  | 2227/2975 [1:31:36<28:35,  2.29s/it]
 75%|███████▍  | 2228/2975 [1:31:38<29:28,  2.37s/it]
 75%|███████▍  | 2229/2975 [1:31:41<29:04,  2.34s/it]
 75%|███████▍  | 2230/2975 [1:31:43<28:47,  2.32s/it]
                                                     

 75%|███████▍  | 2230/2
0: {'loss': 0.4107, 'grad_norm': 0.4612617245971157, 'learning_rate': 3e-06, 'epoch': 0.75}
0: 975 [1:31:43<28:47,  2.32s/it]
 75%|███████▍  | 2231/2975 [1:31:46<30:51,  2.49s/it]
 75%|███████▌  | 2232/2975 [1:31:48<30:02,  2.43s/it]
 75%|███████▌  | 2233/2975 [1:31:50<29:27,  2.38s/it]
 75%|███████▌  | 2234/2975 [1:31:53<29:15,  2.37s/it]
 75%|███████▌  | 2235/2975 [1:31:55<30:29,  2.47s/it]
 75%|███████▌  | 2236/2975 [1:31:58<29:49,  2.42s/it]
 75%|███████▌  | 2237/2975 [1:32:00<29:14,  2.38s/it]
 75%|███████▌  | 2238/2975 [1:32:02<28:50,  2.35s/it]
 75%|███████▌  | 2239/2975 [1:32:04<28:36,  2.33s/it]
 75%|███████▌  | 2240/2975 [1:32:07<28:22,  2.32s/it]
                                                     

 75%|███████▌  | 2240/2975 [1:32:07<28:22,  2.32s/it]
 75%|███████▌  | 2241/2975 [1:32:09<28:12,  2.31s/it]
 75%|███████▌  | 2242/2975 [1:32:11<28:08,  2.30s/it]
 75%|███████�
0: {'loss': 0.4203, 'grad_norm': 0.4611286337505057, 'learning_rate': 3e-06, 'epoch': 0.76}
0: �  | 2243/2975 [1:32:14<28:02,  2.30s/it]
 75%|███████▌  | 2244/2975 [1:32:16<28:07,  2.31s/it]
 75%|███████▌  | 2245/2975 [1:32:18<27:58,  2.30s/it]
 75%|███████▌  | 2246/2975 [1:32:21<28:04,  2.31s/it]
 76%|███████▌  | 2247/2975 [1:32:24<30:30,  2.51s/it]
 76%|███████▌  | 2248/2975 [1:32:26<29:31,  2.44s/it]
 76%|███████▌  | 2249/2975 [1:32:28<29:00,  2.40s/it]
 76%|███████▌  | 2250/2975 [1:32:30<28:32,  2.36s/it]
                                                     

 76%|███████▌  | 2250/2975 [1:32:30<28:32,  2.36s/it]
 76%|███████▌  | 2251/2975 [1:32:33<28:12,  2.34s/it]
 76%|███████▌  | 2252/2975 [1:32:35<28:10,  2.34s/it]
 76%|███████▌  | 2253/2975 [1:32:37<27:55,  2.32s/it]
 76%|███████▌  | 2254/2975 [1:32:40<28:02,  2.33s/it]
 76%|███████▌  | 2255/2975 [1:32:42<27:50,  2.32s/it]
 76%|████
0: {'loss': 0.4242, 'grad_norm': 0.4767939116203499, 'learning_rate': 3e-06, 'epoch': 0.76}
0: ███▌  | 2256/2975 [1:32:44<27:40,  2.31s/it]
 76%|███████▌  | 2257/2975 [1:32:47<27:47,  2.32s/it]
 76%|███████▌  | 2258/2975 [1:32:49<27:59,  2.34s/it]
 76%|███████▌  | 2259/2975 [1:32:51<27:56,  2.34s/it]
 76%|███████▌  | 2260/2975 [1:32:54<27:36,  2.32s/it]
                                                     

 76%|███████▌  | 2260/2975 [1:32:54<27:36,  2.32s/it]
 76%|███████▌  | 2261/2975 [1:32:56<28:19,  2.38s/it]
 76%|███████▌  | 2262/2975 [1:32:58<28:09,  2.37s/it]
 76%|███████▌  | 2263/2975 [1:33:01<27:52,  2.35s/it]
 76%|███████▌  | 2264/2975 [1:33:03<27:39,  2.33s/it]
 76%|███████▌  | 2265/2975 [1:33:05<27:24,  2.32s/it]
 76%|███████▌  | 2266/2975 [1:33:08<27:17,  2.31s/it]
 76%|███████▌  | 2267/2975 [1:33:10<28:49,  2.44s/it]
 76%|███████▌  | 2268/2975 [1:33:13<29:01,  2.46s/it]
 76%|�
0: {'loss': 0.4257, 'grad_norm': 0.46261714805945003, 'learning_rate': 3e-06, 'epoch': 0.76}
0: {'loss': 0.4284, 'grad_norm': 0.47905853364456774, 'learning_rate': 3e-06, 'epoch': 0.77}
0: ��██████▋  | 2269/2975 [1:33:15<28:21,  2.41s/it]
 76%|███████▋  | 2270/2975 [1:33:17<27:53,  2.37s/it]
                                                     

 76%|███████▋  | 2270/2975 [1:33:17<27:53,  2.37s/it]
 76%|███████▋  | 2271/2975 [1:33:20<27:33,  2.35s/it]
 76%|███████▋  | 2272/2975 [1:33:22<27:17,  2.33s/it]
 76%|███████▋  | 2273/2975 [1:33:24<27:05,  2.32s/it]
 76%|███████▋  | 2274/2975 [1:33:27<26:53,  2.30s/it]
 76%|███████▋  | 2275/2975 [1:33:29<26:47,  2.30s/it]
 77%|███████▋  | 2276/2975 [1:33:31<26:38,  2.29s/it]
 77%|███████▋  | 2277/2975 [1:33:33<26:35,  2.29s/it]
 77%|███████▋  | 2278/2975 [1:33:36<26:33,  2.29s/it]
 77%|███████▋  | 2279/2975 [1:33:38<26:31,  2.29s/it]
 77%|███████▋  | 2280/2975 [1:33:40<26:28,  2.29s/it]
                                                     

 77%|█�
0: {'loss': 0.4131, 'grad_norm': 0.46705631329497216, 'learning_rate': 3e-06, 'epoch': 0.77}
0: �█████▋  | 2280/2975 [1:33:40<26:28,  2.29s/it]
 77%|███████▋  | 2281/2975 [1:33:43<26:27,  2.29s/it]
 77%|███████▋  | 2282/2975 [1:33:45<26:24,  2.29s/it]
 77%|███████▋  | 2283/2975 [1:33:47<26:19,  2.28s/it]
 77%|███████▋  | 2284/2975 [1:33:49<26:16,  2.28s/it]
 77%|███████▋  | 2285/2975 [1:33:52<26:13,  2.28s/it]
 77%|███████▋  | 2286/2975 [1:33:54<26:10,  2.28s/it]
 77%|███████▋  | 2287/2975 [1:33:56<26:02,  2.27s/it]
 77%|███████▋  | 2288/2975 [1:33:58<26:03,  2.28s/it]
 77%|███████▋  | 2289/2975 [1:34:01<26:00,  2.28s/it]
 77%|███████▋  | 2290/2975 [1:34:03<26:07,  2.29s/it]
                                                     

 77%|███████▋  | 2290/2975 [1:34:03<26:07,  2.29s/it]
 77%|███████▋  | 2291/2975 [1:34:05<26:04,  2.29s/it]
 77%|███████▋  | 2292/2975 [1:34:08<26:01,  2.29s/it]
0: {'loss': 0.4129, 'grad_norm': 0.4426197647582955, 'learning_rate': 3e-06, 'epoch': 0.77}
0: 
 77%|███████▋  | 2293/2975 [1:34:10<26:00,  2.29s/it]
 77%|███████▋  | 2294/2975 [1:34:12<25:56,  2.29s/it]
 77%|███████▋  | 2295/2975 [1:34:15<26:08,  2.31s/it]
 77%|███████▋  | 2296/2975 [1:34:17<26:00,  2.30s/it]
 77%|███████▋  | 2297/2975 [1:34:19<26:28,  2.34s/it]
 77%|███████▋  | 2298/2975 [1:34:22<26:13,  2.32s/it]
 77%|███████▋  | 2299/2975 [1:34:24<26:03,  2.31s/it]
 77%|███████▋  | 2300/2975 [1:34:26<25:52,  2.30s/it]
                                                     

 77%|███████▋  | 2300/2975 [1:34:26<25:52,  2.30s/it]
 77%|███████▋  | 2301/2975 [1:34:28<25:56,  2.31s/it]
 77%|███████▋  | 2302/2975 [1:34:31<25:44,  2.30s/it]
 77%|███████▋  | 2303/2975 [1:34:33<25:38,  2.29s/it]
 77%|███████▋  | 2304/2975 [1:34:35<25:35,  2.29s/it]
 77%|███████▋  | 2305/2975 [1:34:38<25:33,
0: {'loss': 0.416, 'grad_norm': 0.47027342473369754, 'learning_rate': 3e-06, 'epoch': 0.78}
0:   2.29s/it]
 78%|███████▊  | 2306/2975 [1:34:40<25:29,  2.29s/it]
 78%|███████▊  | 2307/2975 [1:34:42<25:25,  2.28s/it]
 78%|███████▊  | 2308/2975 [1:34:44<25:21,  2.28s/it]
 78%|███████▊  | 2309/2975 [1:34:47<25:18,  2.28s/it]
 78%|███████▊  | 2310/2975 [1:34:49<25:14,  2.28s/it]
                                                     

 78%|███████▊  | 2310/2975 [1:34:49<25:14,  2.28s/it]
 78%|███████▊  | 2311/2975 [1:34:51<25:14,  2.28s/it]
 78%|███████▊  | 2312/2975 [1:34:54<25:41,  2.32s/it]
 78%|███████▊  | 2313/2975 [1:34:56<25:30,  2.31s/it]
 78%|███████▊  | 2314/2975 [1:34:58<25:26,  2.31s/it]
 78%|███████▊  | 2315/2975 [1:35:01<25:17,  2.30s/it]
 78%|███████▊  | 2316/2975 [1:35:03<25:53,  2.36s/it]
 78%|███████▊  | 2317/2975 [1:35:05<25:44,  2.35s/it]
 78%|███████▊  | 2318/2975 [1:3
0: {'loss': 0.4179, 'grad_norm': 0.46545586021530844, 'learning_rate': 3e-06, 'epoch': 0.78}
0: 5:08<25:29,  2.33s/it]
 78%|███████▊  | 2319/2975 [1:35:10<25:17,  2.31s/it]
 78%|███████▊  | 2320/2975 [1:35:12<25:15,  2.31s/it]
                                                     

 78%|███████▊  | 2320/2975 [1:35:12<25:15,  2.31s/it]
 78%|███████▊  | 2321/2975 [1:35:14<25:05,  2.30s/it]
 78%|███████▊  | 2322/2975 [1:35:17<24:57,  2.29s/it]
 78%|███████▊  | 2323/2975 [1:35:19<25:25,  2.34s/it]
 78%|███████▊  | 2324/2975 [1:35:21<25:09,  2.32s/it]
 78%|███████▊  | 2325/2975 [1:35:24<24:58,  2.31s/it]
 78%|███████▊  | 2326/2975 [1:35:26<24:54,  2.30s/it]
 78%|███████▊  | 2327/2975 [1:35:28<24:48,  2.30s/it]
 78%|███████▊  | 2328/2975 [1:35:31<24:43,  2.29s/it]
 78%|███████▊  | 2329/2975 [1:35:33<24:37,  2.29s/it]
 78%|███████▊  | 2330/2975 [1:35:35<24:35,  2.29s/it]
                                    
0: {'loss': 0.4235, 'grad_norm': 0.48923755282576986, 'learning_rate': 3e-06, 'epoch': 0.78}
0: {'loss': 0.4182, 'grad_norm': 0.48879393948799565, 'learning_rate': 3e-06, 'epoch': 0.79}
0:                  

 78%|███████▊  | 2330/2975 [1:35:35<24:35,  2.29s/it]
 78%|███████▊  | 2331/2975 [1:35:37<24:35,  2.29s/it]
 78%|███████▊  | 2332/2975 [1:35:40<25:11,  2.35s/it]
 78%|███████▊  | 2333/2975 [1:35:42<24:55,  2.33s/it]
 78%|███████▊  | 2334/2975 [1:35:45<24:44,  2.32s/it]
 78%|███████▊  | 2335/2975 [1:35:47<24:36,  2.31s/it]
 79%|███████▊  | 2336/2975 [1:35:49<24:26,  2.30s/it]
 79%|███████▊  | 2337/2975 [1:35:51<24:19,  2.29s/it]
 79%|███████▊  | 2338/2975 [1:35:54<24:18,  2.29s/it]
 79%|███████▊  | 2339/2975 [1:35:56<24:14,  2.29s/it]
 79%|███████▊  | 2340/2975 [1:35:58<24:10,  2.28s/it]
                                                     

 79%|███████▊  | 2340/2975 [1:35:58<24:10,  2.28s/it]
 79%|███████▊  | 2341/2975 [1:36:01<25:01,  2.37s/it]
 79%|███████▊  | 2342/29
0: {'loss': 0.4258, 'grad_norm': 0.4574659212331354, 'learning_rate': 3e-06, 'epoch': 0.79}
0: 75 [1:36:03<24:43,  2.34s/it]
 79%|███████▉  | 2343/2975 [1:36:05<24:41,  2.34s/it]
 79%|███████▉  | 2344/2975 [1:36:08<24:24,  2.32s/it]
 79%|███████▉  | 2345/2975 [1:36:10<24:16,  2.31s/it]
 79%|███████▉  | 2346/2975 [1:36:12<24:24,  2.33s/it]
 79%|███████▉  | 2347/2975 [1:36:15<24:10,  2.31s/it]
 79%|███████▉  | 2348/2975 [1:36:17<24:03,  2.30s/it]
 79%|███████▉  | 2349/2975 [1:36:19<24:07,  2.31s/it]
 79%|███████▉  | 2350/2975 [1:36:22<24:04,  2.31s/it]
                                                     

 79%|███████▉  | 2350/2975 [1:36:22<24:04,  2.31s/it]
 79%|███████▉  | 2351/2975 [1:36:24<24:00,  2.31s/it]
 79%|███████▉  | 2352/2975 [1:36:26<24:02,  2.31s/it]
 79%|███████▉  | 2353/2975 [1:36:28<23:52,  2.30s/it]
 79%|███████▉  | 2354/2975 [1:36:31<23:44,  2.29s/it]
 79%|███████▉
0: {'loss': 0.4203, 'grad_norm': 0.46794069040919006, 'learning_rate': 3e-06, 'epoch': 0.79}
0:   | 2355/2975 [1:36:33<23:40,  2.29s/it]
 79%|███████▉  | 2356/2975 [1:36:35<23:35,  2.29s/it]
 79%|███████▉  | 2357/2975 [1:36:38<24:40,  2.40s/it]
 79%|███████▉  | 2358/2975 [1:36:40<24:25,  2.38s/it]
 79%|███████▉  | 2359/2975 [1:36:43<24:06,  2.35s/it]
 79%|███████▉  | 2360/2975 [1:36:45<23:45,  2.32s/it]
                                                     

 79%|███████▉  | 2360/2975 [1:36:45<23:45,  2.32s/it]
 79%|███████▉  | 2361/2975 [1:36:47<24:06,  2.36s/it]
 79%|███████▉  | 2362/2975 [1:36:49<23:47,  2.33s/it]
 79%|███████▉  | 2363/2975 [1:36:52<24:04,  2.36s/it]
 79%|███████▉  | 2364/2975 [1:36:54<23:48,  2.34s/it]
 79%|███████▉  | 2365/2975 [1:36:56<23:35,  2.32s/it]
 80%|███████▉  | 2366/2975 [1:36:59<23:27,  2.31s/it]
 80%|███████▉  | 2367/2975 [1:37:01<23:23,  2.31s/it]
 80%|████�
0: {'loss': 0.4321, 'grad_norm': 0.45736454682272043, 'learning_rate': 3e-06, 'epoch': 0.8}
0: ��██▉  | 2368/2975 [1:37:03<23:21,  2.31s/it]
 80%|███████▉  | 2369/2975 [1:37:06<23:12,  2.30s/it]
 80%|███████▉  | 2370/2975 [1:37:08<23:07,  2.29s/it]
                                                     

 80%|███████▉  | 2370/2975 [1:37:08<23:07,  2.29s/it]
 80%|███████▉  | 2371/2975 [1:37:10<23:02,  2.29s/it]
 80%|███████▉  | 2372/2975 [1:37:13<22:57,  2.28s/it]
 80%|███████▉  | 2373/2975 [1:37:15<22:59,  2.29s/it]
 80%|███████▉  | 2374/2975 [1:37:17<22:55,  2.29s/it]
 80%|███████▉  | 2375/2975 [1:37:19<22:54,  2.29s/it]
 80%|███████▉  | 2376/2975 [1:37:22<22:51,  2.29s/it]
 80%|███████▉  | 2377/2975 [1:37:24<22:47,  2.29s/it]
 80%|███████▉  | 2378/2975 [1:37:26<22:54,  2.30s/it]
 80%|███████▉  | 2379/2975 [1:37:29<22:48,  2.30s/it]
 80%|████████  | 2380/2975 [1:37:31<22:43,  2.29s/it]
       
0: {'loss': 0.4213, 'grad_norm': 0.4612672384885213, 'learning_rate': 3e-06, 'epoch': 0.8}
0: {'loss': 0.4207, 'grad_norm': 0.48352964113775476, 'learning_rate': 3e-06, 'epoch': 0.8}
0:                                               

 80%|████████  | 2380/2975 [1:37:31<22:43,  2.29s/it]
 80%|████████  | 2381/2975 [1:37:55<1:27:26,  8.83s/it]
 80%|████████  | 2382/2975 [1:37:57<1:07:47,  6.86s/it]
 80%|████████  | 2383/2975 [1:38:00<54:14,  5.50s/it]  
 80%|████████  | 2384/2975 [1:38:02<44:38,  4.53s/it]
 80%|████████  | 2385/2975 [1:38:04<37:53,  3.85s/it]
 80%|████████  | 2386/2975 [1:38:06<33:12,  3.38s/it]
 80%|████████  | 2387/2975 [1:38:09<29:56,  3.06s/it]
 80%|████████  | 2388/2975 [1:38:11<27:41,  2.83s/it]
 80%|████████  | 2389/2975 [1:38:13<26:03,  2.67s/it]
 80%|████████  | 2390/2975 [1:38:16<24:55,  2.56s/it]
                                                     

 80%|████████  | 2390/2975 [1:38:16<24:55,  2.56s/it]
 80%|████████  | 2391/2975 [1:38:18<24:08,  2.48s/it]
 80%|
0: {'loss': 0.4232, 'grad_norm': 0.4499089667784455, 'learning_rate': 3e-06, 'epoch': 0.81}
0: ████████  | 2392/2975 [1:38:20<23:32,  2.42s/it]
 80%|████████  | 2393/2975 [1:38:22<23:04,  2.38s/it]
 80%|████████  | 2394/2975 [1:38:25<22:44,  2.35s/it]
 81%|████████  | 2395/2975 [1:38:27<23:08,  2.39s/it]
 81%|████████  | 2396/2975 [1:38:29<22:46,  2.36s/it]
 81%|████████  | 2397/2975 [1:38:32<22:31,  2.34s/it]
 81%|████████  | 2398/2975 [1:38:34<22:30,  2.34s/it]
 81%|████████  | 2399/2975 [1:38:36<22:16,  2.32s/it]
 81%|████████  | 2400/2975 [1:38:39<22:09,  2.31s/it]
                                                     

 81%|████████  | 2400/2975 [1:38:39<22:09,  2.31s/it]
 81%|████████  | 2401/2975 [1:38:41<22:02,  2.30s/it]
 81%|████████  | 2402/2975 [1:38:43<21:55,  2.30s/it]
 81%|████████  | 2403/2975 [1:38:46<21:50,  2.29s/it]
 81%|████████  | 2404/2975 [1:38:48<21:46,  2.29
0: {'loss': 0.4162, 'grad_norm': 0.47161955021142793, 'learning_rate': 3e-06, 'epoch': 0.81}
0: s/it]
 81%|████████  | 2405/2975 [1:38:50<21:42,  2.28s/it]
 81%|████████  | 2406/2975 [1:38:52<21:37,  2.28s/it]
 81%|████████  | 2407/2975 [1:38:55<21:44,  2.30s/it]
 81%|████████  | 2408/2975 [1:38:57<21:40,  2.29s/it]
 81%|████████  | 2409/2975 [1:38:59<21:32,  2.28s/it]
 81%|████████  | 2410/2975 [1:39:02<21:31,  2.29s/it]
                                                     

 81%|████████  | 2410/2975 [1:39:02<21:31,  2.29s/it]
 81%|████████  | 2411/2975 [1:39:04<21:28,  2.28s/it]
 81%|████████  | 2412/2975 [1:39:06<21:26,  2.29s/it]
 81%|████████  | 2413/2975 [1:39:08<21:26,  2.29s/it]
 81%|████████  | 2414/2975 [1:39:11<22:11,  2.37s/it]
 81%|████████  | 2415/2975 [1:39:13<21:53,  2.34s/it]
 81%|████████  | 2416/2975 [1:39:16<21:37,  2.32s/it]
 81%|████████  | 2417/2975 [1:39:18<2
0: {'loss': 0.4234, 'grad_norm': 0.4691225219891306, 'learning_rate': 3e-06, 'epoch': 0.81}
0: 1:27,  2.31s/it]
 81%|████████▏ | 2418/2975 [1:39:20<21:19,  2.30s/it]
 81%|████████▏ | 2419/2975 [1:39:22<21:17,  2.30s/it]
 81%|████████▏ | 2420/2975 [1:39:25<21:20,  2.31s/it]
                                                     

 81%|████████▏ | 2420/2975 [1:39:25<21:20,  2.31s/it]
 81%|████████▏ | 2421/2975 [1:39:27<21:12,  2.30s/it]
 81%|████████▏ | 2422/2975 [1:39:29<21:42,  2.35s/it]
 81%|████████▏ | 2423/2975 [1:39:32<21:28,  2.33s/it]
 81%|████████▏ | 2424/2975 [1:39:34<21:19,  2.32s/it]
 82%|████████▏ | 2425/2975 [1:39:36<21:09,  2.31s/it]
 82%|████████▏ | 2426/2975 [1:39:39<21:00,  2.30s/it]
 82%|████████▏ | 2427/2975 [1:39:41<21:03,  2.31s/it]
 82%|████████▏ | 2428/2975 [1:39:43<21:01,  2.31s/it]
 82%|████████▏ | 2429/2975 [1:39:45<20:56,  2.30s/it]
 82%|███�
0: {'loss': 0.4175, 'grad_norm': 0.46049034147534973, 'learning_rate': 3e-06, 'epoch': 0.82}
0: {'loss': 0.4165, 'grad_norm': 0.4536770172213815, 'learning_rate': 3e-06, 'epoch': 0.82}
0: �████▏ | 2430/2975 [1:39:48<20:48,  2.29s/it]
                                                     

 82%|████████▏ | 2430/2975 [1:39:48<20:48,  2.29s/it]
 82%|████████▏ | 2431/2975 [1:39:50<20:46,  2.29s/it]
 82%|████████▏ | 2432/2975 [1:39:52<20:45,  2.29s/it]
 82%|████████▏ | 2433/2975 [1:39:55<20:41,  2.29s/it]
 82%|████████▏ | 2434/2975 [1:39:57<20:39,  2.29s/it]
 82%|████████▏ | 2435/2975 [1:39:59<20:33,  2.28s/it]
 82%|████████▏ | 2436/2975 [1:40:01<20:30,  2.28s/it]
 82%|████████▏ | 2437/2975 [1:40:04<20:26,  2.28s/it]
 82%|████████▏ | 2438/2975 [1:40:06<20:26,  2.28s/it]
 82%|████████▏ | 2439/2975 [1:40:09<21:23,  2.40s/it]
 82%|████████▏ | 2440/2975 [1:40:11<21:12,  2.38s/it]
                                                     

 82%|████████▏ | 2440/2975 [1:40:11<21:12,  2.38
0: {'loss': 0.4235, 'grad_norm': 0.49198334182329034, 'learning_rate': 3e-06, 'epoch': 0.82}
0: s/it]
 82%|████████▏ | 2441/2975 [1:40:13<20:54,  2.35s/it]
 82%|████████▏ | 2442/2975 [1:40:16<20:40,  2.33s/it]
 82%|████████▏ | 2443/2975 [1:40:18<20:28,  2.31s/it]
 82%|████████▏ | 2444/2975 [1:40:20<20:23,  2.30s/it]
 82%|████████▏ | 2445/2975 [1:40:22<20:16,  2.30s/it]
 82%|████████▏ | 2446/2975 [1:40:25<20:19,  2.31s/it]
 82%|████████▏ | 2447/2975 [1:40:27<20:16,  2.30s/it]
 82%|████████▏ | 2448/2975 [1:40:29<20:08,  2.29s/it]
 82%|████████▏ | 2449/2975 [1:40:32<20:04,  2.29s/it]
 82%|████████▏ | 2450/2975 [1:40:34<20:07,  2.30s/it]
                                                     

 82%|████████▏ | 2450/2975 [1:40:34<20:07,  2.30s/it]
 82%|████████▏ | 2451/2975 [1:40:36<20:00,  2.29s/it]
 82%|████████▏ | 2452/2975 [1:40:38<19:58,  2.29s/it]
 82%|███████�
0: {'loss': 0.4236, 'grad_norm': 0.47938735650423275, 'learning_rate': 3e-06, 'epoch': 0.83}
0: ��▏ | 2453/2975 [1:40:41<19:54,  2.29s/it]
 82%|████████▏ | 2454/2975 [1:40:43<19:50,  2.28s/it]
 83%|████████▎ | 2455/2975 [1:40:45<19:46,  2.28s/it]
 83%|████████▎ | 2456/2975 [1:40:48<19:48,  2.29s/it]
 83%|████████▎ | 2457/2975 [1:40:50<19:42,  2.28s/it]
 83%|████████▎ | 2458/2975 [1:40:52<19:49,  2.30s/it]
 83%|████████▎ | 2459/2975 [1:40:55<19:44,  2.30s/it]
 83%|████████▎ | 2460/2975 [1:40:57<19:39,  2.29s/it]
                                                     

 83%|████████▎ | 2460/2975 [1:40:57<19:39,  2.29s/it]
 83%|████████▎ | 2461/2975 [1:40:59<19:35,  2.29s/it]
 83%|████████▎ | 2462/2975 [1:41:01<19:45,  2.31s/it]
 83%|████████▎ | 2463/2975 [1:41:04<19:38,  2.30s/it]
 83%|████████▎ | 2464/2975 [1:41:06<19:32,  2.29s/it]
 83%|████████▎ | 2465/2975 [1:41:08<19:26,
0: {'loss': 0.4167, 'grad_norm': 0.492021093680105, 'learning_rate': 3e-06, 'epoch': 0.83}
0:   2.29s/it]
 83%|████████▎ | 2466/2975 [1:41:11<19:25,  2.29s/it]
 83%|████████▎ | 2467/2975 [1:41:13<19:29,  2.30s/it]
 83%|████████▎ | 2468/2975 [1:41:15<19:21,  2.29s/it]
 83%|████████▎ | 2469/2975 [1:41:17<19:16,  2.29s/it]
 83%|████████▎ | 2470/2975 [1:41:20<19:15,  2.29s/it]
                                                     

 83%|████████▎ | 2470/2975 [1:41:20<19:15,  2.29s/it]
 83%|████████▎ | 2471/2975 [1:41:22<19:12,  2.29s/it]
 83%|████████▎ | 2472/2975 [1:41:24<19:08,  2.28s/it]
 83%|████████▎ | 2473/2975 [1:41:27<19:04,  2.28s/it]
 83%|████████▎ | 2474/2975 [1:41:29<19:02,  2.28s/it]
 83%|████████▎ | 2475/2975 [1:41:32<20:16,  2.43s/it]
 83%|████████▎ | 2476/2975 [1:41:34<19:50,  2.39s/it]
 83%|████████▎ | 2477/2975 [1:41:36<19:32,  2.35s/it]
 83%|█████�
0: {'loss': 0.4051, 'grad_norm': 0.4506413526003913, 'learning_rate': 3e-06, 'epoch': 0.83}
0: ��██▎ | 2478/2975 [1:41:39<19:27,  2.35s/it]
 83%|████████▎ | 2479/2975 [1:41:41<19:14,  2.33s/it]
 83%|████████▎ | 2480/2975 [1:41:43<19:05,  2.31s/it]
                                                     

 83%|████████▎ | 2480/2975 [1:41:43<19:05,  2.31s/it]
 83%|████████▎ | 2481/2975 [1:41:45<19:08,  2.33s/it]
 83%|████████▎ | 2482/2975 [1:41:48<19:05,  2.32s/it]
 83%|████████▎ | 2483/2975 [1:41:50<18:56,  2.31s/it]
 83%|████████▎ | 2484/2975 [1:41:52<18:49,  2.30s/it]
 84%|████████▎ | 2485/2975 [1:41:55<18:47,  2.30s/it]
 84%|████████▎ | 2486/2975 [1:41:57<18:43,  2.30s/it]
 84%|████████▎ | 2487/2975 [1:41:59<18:36,  2.29s/it]
 84%|████████▎ | 2488/2975 [1:42:01<18:33,  2.29s/it]
 84%|████████▎ | 2489/2975 [1:42:04<18:29,  2.28s/it]
 84%|████████▎ | 2490/2975 [1:42:06<
0: {'loss': 0.4209, 'grad_norm': 0.48106685840340063, 'learning_rate': 3e-06, 'epoch': 0.84}
0: {'loss': 0.4213, 'grad_norm': 0.46459379649289273, 'learning_rate': 3e-06, 'epoch': 0.84}
0: 18:37,  2.30s/it]
                                                     

 84%|████████▎ | 2490/2975 [1:42:06<18:37,  2.30s/it]
 84%|████████▎ | 2491/2975 [1:42:08<18:38,  2.31s/it]
 84%|████████▍ | 2492/2975 [1:42:11<18:55,  2.35s/it]
 84%|████████▍ | 2493/2975 [1:42:13<18:43,  2.33s/it]
 84%|████████▍ | 2494/2975 [1:42:15<18:35,  2.32s/it]
 84%|████████▍ | 2495/2975 [1:42:18<18:32,  2.32s/it]
 84%|████████▍ | 2496/2975 [1:42:20<18:29,  2.32s/it]
 84%|████████▍ | 2497/2975 [1:42:22<18:20,  2.30s/it]
 84%|████████▍ | 2498/2975 [1:42:25<18:16,  2.30s/it]
 84%|████████▍ | 2499/2975 [1:42:27<18:16,  2.30s/it]
 84%|████████▍ | 2500/2975 [1:42:29<18:13,  2.30s/it]
                                                     

 84%|████████▍ | 2500/2975 [1:42:29<18:13,  2.30s/it]
 84%|████████▍
0: {'loss': 0.41, 'grad_norm': 0.459840020124343, 'learning_rate': 3e-06, 'epoch': 0.84}
0:  | 2501/2975 [1:42:31<18:04,  2.29s/it]
 84%|████████▍ | 2502/2975 [1:42:34<17:57,  2.28s/it]
 84%|████████▍ | 2503/2975 [1:42:36<17:52,  2.27s/it]
 84%|████████▍ | 2504/2975 [1:42:38<17:54,  2.28s/it]
 84%|████████▍ | 2505/2975 [1:42:41<17:59,  2.30s/it]
 84%|████████▍ | 2506/2975 [1:42:43<17:54,  2.29s/it]
 84%|████████▍ | 2507/2975 [1:42:45<17:58,  2.31s/it]
 84%|████████▍ | 2508/2975 [1:42:48<17:50,  2.29s/it]
 84%|████████▍ | 2509/2975 [1:42:50<17:46,  2.29s/it]
 84%|████████▍ | 2510/2975 [1:42:52<17:44,  2.29s/it]
                                                     

 84%|████████▍ | 2510/2975 [1:42:52<17:44,  2.29s/it]
 84%|████████▍ | 2511/2975 [1:42:54<17:42,  2.29s/it]
 84%|████████▍ | 2512/2975 [1:42:57<18:33,  2.41s/it]
 84%|████████▍ | 2513/2975 [1:43:00<18:51,  2.4
0: {'loss': 0.4207, 'grad_norm': 0.4762942795070499, 'learning_rate': 3e-06, 'epoch': 0.85}
0: 5s/it]
 85%|████████▍ | 2514/2975 [1:43:02<18:26,  2.40s/it]
 85%|████████▍ | 2515/2975 [1:43:04<18:14,  2.38s/it]
 85%|████████▍ | 2516/2975 [1:43:06<17:57,  2.35s/it]
 85%|████████▍ | 2517/2975 [1:43:09<17:44,  2.32s/it]
 85%|████████▍ | 2518/2975 [1:43:11<17:45,  2.33s/it]
 85%|████████▍ | 2519/2975 [1:43:13<17:34,  2.31s/it]
 85%|████████▍ | 2520/2975 [1:43:16<17:26,  2.30s/it]
                                                     

 85%|████████▍ | 2520/2975 [1:43:16<17:26,  2.30s/it]
 85%|████████▍ | 2521/2975 [1:43:18<17:23,  2.30s/it]
 85%|████████▍ | 2522/2975 [1:43:20<17:29,  2.32s/it]
 85%|████████▍ | 2523/2975 [1:43:23<17:21,  2.30s/it]
 85%|████████▍ | 2524/2975 [1:43:25<17:14,  2.29s/it]
 85%|████████▍ | 2525/2975 [1:43:27<17:12,  2.29s/it]
 85%|███████
0: {'loss': 0.4085, 'grad_norm': 0.45640687765812316, 'learning_rate': 3e-06, 'epoch': 0.85}
0: █▍ | 2526/2975 [1:43:29<17:09,  2.29s/it]
 85%|████████▍ | 2527/2975 [1:43:32<17:06,  2.29s/it]
 85%|████████▍ | 2528/2975 [1:43:34<17:01,  2.29s/it]
 85%|████████▌ | 2529/2975 [1:43:36<16:59,  2.29s/it]
 85%|████████▌ | 2530/2975 [1:43:39<16:55,  2.28s/it]
                                                     

 85%|████████▌ | 2530/2975 [1:43:39<16:55,  2.28s/it]
 85%|████████▌ | 2531/2975 [1:43:41<16:55,  2.29s/it]
 85%|████████▌ | 2532/2975 [1:43:43<16:53,  2.29s/it]
 85%|████████▌ | 2533/2975 [1:43:45<16:48,  2.28s/it]
 85%|████████▌ | 2534/2975 [1:43:48<16:48,  2.29s/it]
 85%|████████▌ | 2535/2975 [1:43:50<16:46,  2.29s/it]
 85%|████████▌ | 2536/2975 [1:43:52<16:43,  2.29s/it]
 85%|████████▌ | 2537/2975 [1:43:55<17:04,  2.34s/it]
 85%|████████▌ | 2538/2975 [1:43:57<16:54
0: {'loss': 0.408, 'grad_norm': 0.4678675459484072, 'learning_rate': 3e-06, 'epoch': 0.85}
0: ,  2.32s/it]
 85%|████████▌ | 2539/2975 [1:43:59<16:43,  2.30s/it]
 85%|████████▌ | 2540/2975 [1:44:02<16:36,  2.29s/it]
                                                     

 85%|████████▌ | 2540/2975 [1:44:02<16:36,  2.29s/it]
 85%|████████▌ | 2541/2975 [1:44:04<16:32,  2.29s/it]
 85%|████████▌ | 2542/2975 [1:44:06<16:29,  2.29s/it]
 85%|████████▌ | 2543/2975 [1:44:08<16:26,  2.28s/it]
 86%|████████▌ | 2544/2975 [1:44:11<16:23,  2.28s/it]
 86%|████████▌ | 2545/2975 [1:44:13<16:21,  2.28s/it]
 86%|████████▌ | 2546/2975 [1:44:15<16:19,  2.28s/it]
 86%|████████▌ | 2547/2975 [1:44:17<16:16,  2.28s/it]
 86%|████████▌ | 2548/2975 [1:44:20<16:12,  2.28s/it]
 86%|████████▌ | 2549/2975 [1:44:22<16:11,  2.28s/it]
 86%|████████▌ | 2550/2975 [1:44:24<16:08,  2.28s/it]
                    
0: {'loss': 0.4302, 'grad_norm': 0.48480614521230114, 'learning_rate': 3e-06, 'epoch': 0.86}
0: {'loss': 0.4213, 'grad_norm': 0.47393976407481114, 'learning_rate': 3e-06, 'epoch': 0.86}
0:                                  

 86%|████████▌ | 2550/2975 [1:44:24<16:08,  2.28s/it]
 86%|████████▌ | 2551/2975 [1:44:27<16:14,  2.30s/it]
 86%|████████▌ | 2552/2975 [1:44:29<16:09,  2.29s/it]
 86%|████████▌ | 2553/2975 [1:44:31<16:05,  2.29s/it]
 86%|████████▌ | 2554/2975 [1:44:34<16:05,  2.29s/it]
 86%|████████▌ | 2555/2975 [1:44:36<16:01,  2.29s/it]
 86%|████████▌ | 2556/2975 [1:44:38<15:56,  2.28s/it]
 86%|████████▌ | 2557/2975 [1:44:40<15:57,  2.29s/it]
 86%|████████▌ | 2558/2975 [1:44:43<15:52,  2.28s/it]
 86%|████████▌ | 2559/2975 [1:44:45<15:47,  2.28s/it]
 86%|████████▌ | 2560/2975 [1:44:47<15:45,  2.28s/it]
                                                     

 86%|████████▌ | 2560/2975 [1:44:47<15:45,  2.28s/it]
 86%|████████▌ | 2561/2975 [1:44:49<15:43,  2.28s/it
0: {'loss': 0.4084, 'grad_norm': 0.48942382938620305, 'learning_rate': 3e-06, 'epoch': 0.86}
0: ]
 86%|████████▌ | 2562/2975 [1:44:52<15:40,  2.28s/it]
 86%|████████▌ | 2563/2975 [1:44:54<15:38,  2.28s/it]
 86%|████████▌ | 2564/2975 [1:44:56<15:45,  2.30s/it]
 86%|████████▌ | 2565/2975 [1:44:59<15:40,  2.29s/it]
 86%|████████▋ | 2566/2975 [1:45:01<15:39,  2.30s/it]
 86%|████████▋ | 2567/2975 [1:45:03<15:38,  2.30s/it]
 86%|████████▋ | 2568/2975 [1:45:06<16:53,  2.49s/it]
 86%|████████▋ | 2569/2975 [1:45:08<16:24,  2.42s/it]
 86%|████████▋ | 2570/2975 [1:45:11<16:08,  2.39s/it]
                                                     

 86%|████████▋ | 2570/2975 [1:45:11<16:08,  2.39s/it]
 86%|████████▋ | 2571/2975 [1:45:13<15:53,  2.36s/it]
 86%|████████▋ | 2572/2975 [1:45:15<15:47,  2.35s/it]
 86%|████████▋ | 2573/2975 [1:45:18<16:15,  2.43s/it]
 87%|████████�
0: {'loss': 0.4149, 'grad_norm': 0.5081481462218561, 'learning_rate': 3e-06, 'epoch': 0.87}
0: � | 2574/2975 [1:45:20<15:56,  2.39s/it]
 87%|████████▋ | 2575/2975 [1:45:23<15:41,  2.35s/it]
 87%|████████▋ | 2576/2975 [1:45:25<15:29,  2.33s/it]
 87%|████████▋ | 2577/2975 [1:45:27<15:19,  2.31s/it]
 87%|████████▋ | 2578/2975 [1:45:29<15:15,  2.31s/it]
 87%|████████▋ | 2579/2975 [1:45:32<15:11,  2.30s/it]
 87%|████████▋ | 2580/2975 [1:45:34<15:05,  2.29s/it]
                                                     

 87%|████████▋ | 2580/2975 [1:45:34<15:05,  2.29s/it]
 87%|████████▋ | 2581/2975 [1:45:36<15:08,  2.31s/it]
 87%|████████▋ | 2582/2975 [1:45:39<15:03,  2.30s/it]
 87%|████████▋ | 2583/2975 [1:45:41<14:59,  2.30s/it]
 87%|████████▋ | 2584/2975 [1:45:43<15:23,  2.36s/it]
 87%|████████▋ | 2585/2975 [1:45:46<15:10,  2.34s/it]
 87%|████████▋ | 2586/2975 [1:45:48<15:00,  2.
0: {'loss': 0.4126, 'grad_norm': 0.4733266136166966, 'learning_rate': 3e-06, 'epoch': 0.87}
0: 31s/it]
 87%|████████▋ | 2587/2975 [1:45:50<14:53,  2.30s/it]
 87%|████████▋ | 2588/2975 [1:45:52<14:47,  2.29s/it]
 87%|████████▋ | 2589/2975 [1:45:55<14:45,  2.29s/it]
 87%|████████▋ | 2590/2975 [1:45:57<14:40,  2.29s/it]
                                                     

 87%|████████▋ | 2590/2975 [1:45:57<14:40,  2.29s/it]
 87%|████████▋ | 2591/2975 [1:45:59<14:37,  2.29s/it]
 87%|████████▋ | 2592/2975 [1:46:02<14:33,  2.28s/it]
 87%|████████▋ | 2593/2975 [1:46:04<14:31,  2.28s/it]
 87%|████████▋ | 2594/2975 [1:46:06<14:31,  2.29s/it]
 87%|████████▋ | 2595/2975 [1:46:08<14:30,  2.29s/it]
 87%|████████▋ | 2596/2975 [1:46:11<14:25,  2.28s/it]
 87%|████████▋ | 2597/2975 [1:46:13<14:28,  2.30s/it]
 87%|████████▋ | 2598/2975 [1:46:15<14:24,  2.29s/it]
 87%|██████�
0: {'loss': 0.4154, 'grad_norm': 0.45885178055145326, 'learning_rate': 3e-06, 'epoch': 0.87}
0: {'loss': 0.4234, 'grad_norm': 0.5025398099944667, 'learning_rate': 3e-06, 'epoch': 0.88}
0: �█▋ | 2599/2975 [1:46:18<14:21,  2.29s/it]
 87%|████████▋ | 2600/2975 [1:46:20<14:18,  2.29s/it]
                                                     

 87%|████████▋ | 2600/2975 [1:46:20<14:18,  2.29s/it]
 87%|████████▋ | 2601/2975 [1:46:22<14:14,  2.28s/it]
 87%|████████▋ | 2602/2975 [1:46:25<14:11,  2.28s/it]
 87%|████████▋ | 2603/2975 [1:46:27<14:11,  2.29s/it]
 88%|████████▊ | 2604/2975 [1:46:29<14:08,  2.29s/it]
 88%|████████▊ | 2605/2975 [1:46:31<14:07,  2.29s/it]
 88%|████████▊ | 2606/2975 [1:46:34<15:07,  2.46s/it]
 88%|████████▊ | 2607/2975 [1:46:37<14:44,  2.40s/it]
 88%|████████▊ | 2608/2975 [1:46:39<14:28,  2.37s/it]
 88%|████████▊ | 2609/2975 [1:46:41<14:16,  2.34s/it]
 88%|████████▊ | 2610/2975 [1:46:43<14:19,  2.36s/it]
                                                     

 88
0: {'loss': 0.4117, 'grad_norm': 0.4468022422661684, 'learning_rate': 3e-06, 'epoch': 0.88}
0: %|████████▊ | 2610/2975 [1:46:43<14:19,  2.36s/it]
 88%|████████▊ | 2611/2975 [1:46:46<14:12,  2.34s/it]
 88%|████████▊ | 2612/2975 [1:46:48<14:02,  2.32s/it]
 88%|████████▊ | 2613/2975 [1:46:51<14:19,  2.38s/it]
 88%|████████▊ | 2614/2975 [1:46:53<14:09,  2.35s/it]
 88%|████████▊ | 2615/2975 [1:46:55<14:02,  2.34s/it]
 88%|████████▊ | 2616/2975 [1:46:57<13:53,  2.32s/it]
 88%|████████▊ | 2617/2975 [1:47:00<13:48,  2.31s/it]
 88%|████████▊ | 2618/2975 [1:47:02<13:43,  2.31s/it]
 88%|████████▊ | 2619/2975 [1:47:04<13:37,  2.30s/it]
 88%|████████▊ | 2620/2975 [1:47:07<13:35,  2.30s/it]
                                                     

 88%|████████▊ | 2620/2975 [1:47:07<13:35,  2.30s/it]
 88%|████████▊ | 2621/2975 [1:47:09<13:33,  2.30s/it]
 88%|████████▊ | 2
0: {'loss': 0.4232, 'grad_norm': 0.4600945791322977, 'learning_rate': 3e-06, 'epoch': 0.88}
0: 622/2975 [1:47:11<13:37,  2.32s/it]
 88%|████████▊ | 2623/2975 [1:47:14<13:31,  2.31s/it]
 88%|████████▊ | 2624/2975 [1:47:16<13:25,  2.29s/it]
 88%|████████▊ | 2625/2975 [1:47:18<13:21,  2.29s/it]
 88%|████████▊ | 2626/2975 [1:47:20<13:31,  2.33s/it]
 88%|████████▊ | 2627/2975 [1:47:23<13:28,  2.32s/it]
 88%|████████▊ | 2628/2975 [1:47:25<13:21,  2.31s/it]
 88%|████████▊ | 2629/2975 [1:47:27<13:14,  2.29s/it]
 88%|████████▊ | 2630/2975 [1:47:30<13:27,  2.34s/it]
                                                     

 88%|████████▊ | 2630/2975 [1:47:30<13:27,  2.34s/it]
 88%|████████▊ | 2631/2975 [1:47:32<13:18,  2.32s/it]
 88%|████████▊ | 2632/2975 [1:47:34<13:10,  2.31s/it]
 89%|████████▊ | 2633/2975 [1:47:37<13:05,  2.30s/it]
 89%|████████▊ | 2634/2975 [1:47:39<13:02,  2.29s/i
0: {'loss': 0.4215, 'grad_norm': 0.4803460345269387, 'learning_rate': 3e-06, 'epoch': 0.89}
0: t]
 89%|████████▊ | 2635/2975 [1:47:41<12:58,  2.29s/it]
 89%|████████▊ | 2636/2975 [1:47:43<12:55,  2.29s/it]
 89%|████████▊ | 2637/2975 [1:47:46<13:24,  2.38s/it]
 89%|████████▊ | 2638/2975 [1:47:48<13:11,  2.35s/it]
 89%|████████▊ | 2639/2975 [1:47:51<13:02,  2.33s/it]
 89%|████████▊ | 2640/2975 [1:47:53<12:54,  2.31s/it]
                                                     

 89%|████████▊ | 2640/2975 [1:47:53<12:54,  2.31s/it]
 89%|████████▉ | 2641/2975 [1:47:55<12:49,  2.30s/it]
 89%|████████▉ | 2642/2975 [1:47:57<12:42,  2.29s/it]
 89%|████████▉ | 2643/2975 [1:48:00<12:39,  2.29s/it]
 89%|████████▉ | 2644/2975 [1:48:02<12:54,  2.34s/it]
 89%|████████▉ | 2645/2975 [1:48:04<12:46,  2.32s/it]
 89%|████████▉ | 2646/2975 [1:48:07<12:40,  2.31s/it]
 89%|████████�
0: {'loss': 0.4136, 'grad_norm': 0.49471253270059085, 'learning_rate': 3e-06, 'epoch': 0.89}
0: �� | 2647/2975 [1:48:09<12:35,  2.30s/it]
 89%|████████▉ | 2648/2975 [1:48:11<12:31,  2.30s/it]
 89%|████████▉ | 2649/2975 [1:48:14<12:25,  2.29s/it]
 89%|████████▉ | 2650/2975 [1:48:16<12:19,  2.28s/it]
                                                     

 89%|████████▉ | 2650/2975 [1:48:16<12:19,  2.28s/it]
 89%|████████▉ | 2651/2975 [1:48:18<12:18,  2.28s/it]
 89%|████████▉ | 2652/2975 [1:48:20<12:18,  2.28s/it]
 89%|████████▉ | 2653/2975 [1:48:23<12:16,  2.29s/it]
 89%|████████▉ | 2654/2975 [1:48:25<12:12,  2.28s/it]
 89%|████████▉ | 2655/2975 [1:48:27<12:10,  2.28s/it]
 89%|████████▉ | 2656/2975 [1:48:30<13:17,  2.50s/it]
 89%|████████▉ | 2657/2975 [1:48:33<12:53,  2.43s/it]
 89%|████████▉ | 2658/2975 [1:48:35<12:38,  2.39s/it]
 89%|████████▉ | 2659/2975 [1:48:37<12:29,  2
0: {'loss': 0.4142, 'grad_norm': 0.5009920312887763, 'learning_rate': 3e-06, 'epoch': 0.89}
0: {'loss': 0.4084, 'grad_norm': 0.4700024092737027, 'learning_rate': 3e-06, 'epoch': 0.9}
0: .37s/it]
 89%|████████▉ | 2660/2975 [1:48:39<12:18,  2.35s/it]
                                                     

 89%|████████▉ | 2660/2975 [1:48:39<12:18,  2.35s/it]
 89%|████████▉ | 2661/2975 [1:48:42<12:09,  2.32s/it]
 89%|████████▉ | 2662/2975 [1:48:44<12:01,  2.31s/it]
 90%|████████▉ | 2663/2975 [1:48:46<11:56,  2.30s/it]
 90%|████████▉ | 2664/2975 [1:48:49<11:55,  2.30s/it]
 90%|████████▉ | 2665/2975 [1:48:51<11:53,  2.30s/it]
 90%|████████▉ | 2666/2975 [1:48:53<11:49,  2.30s/it]
 90%|████████▉ | 2667/2975 [1:48:55<11:44,  2.29s/it]
 90%|████████▉ | 2668/2975 [1:48:58<11:52,  2.32s/it]
 90%|████████▉ | 2669/2975 [1:49:00<11:46,  2.31s/it]
 90%|████████▉ | 2670/2975 [1:49:02<11:42,  2.30s/it]
                                                     

 90%|████████▉ | 2670/2
0: {'loss': 0.4105, 'grad_norm': 0.46467499366937726, 'learning_rate': 2.998815820633459e-06, 'epoch': 0.9}
0: 975 [1:49:02<11:42,  2.30s/it]
 90%|████████▉ | 2671/2975 [1:49:05<11:37,  2.30s/it]
 90%|████████▉ | 2672/2975 [1:49:07<11:32,  2.29s/it]
 90%|████████▉ | 2673/2975 [1:49:10<12:38,  2.51s/it]
 90%|████████▉ | 2674/2975 [1:49:12<12:15,  2.44s/it]
 90%|████████▉ | 2675/2975 [1:49:15<11:57,  2.39s/it]
 90%|████████▉ | 2676/2975 [1:49:17<11:44,  2.36s/it]
 90%|████████▉ | 2677/2975 [1:49:19<11:34,  2.33s/it]
 90%|█████████ | 2678/2975 [1:49:21<11:26,  2.31s/it]
 90%|█████████ | 2679/2975 [1:49:24<11:20,  2.30s/it]
 90%|█████████ | 2680/2975 [1:49:26<11:16,  2.29s/it]
                                                     

 90%|█████████ | 2680/2975 [1:49:26<11:16,  2.29s/it]
 90%|█████████ | 2681/2975 [1:49:28<11:12,  2.29s/it]
 90%|█████████ | 2682/2975 [1:49:30<11:07,  2.28s/it]
 9
0: {'loss': 0.4148, 'grad_norm': 0.49811470180786943, 'learning_rate': 2.9855176495000345e-06, 'epoch': 0.9}
0: 0%|█████████ | 2683/2975 [1:49:33<11:03,  2.27s/it]
 90%|█████████ | 2684/2975 [1:49:35<11:01,  2.27s/it]
 90%|█████████ | 2685/2975 [1:49:37<10:59,  2.27s/it]
 90%|█████████ | 2686/2975 [1:49:39<10:57,  2.27s/it]
 90%|█████████ | 2687/2975 [1:49:42<10:55,  2.28s/it]
 90%|█████████ | 2688/2975 [1:49:44<10:55,  2.28s/it]
 90%|█████████ | 2689/2975 [1:49:46<10:52,  2.28s/it]
 90%|█████████ | 2690/2975 [1:49:49<10:49,  2.28s/it]
                                                     

 90%|█████████ | 2690/2975 [1:49:49<10:49,  2.28s/it]
 90%|█████████ | 2691/2975 [1:49:51<11:03,  2.34s/it]
 90%|█████████ | 2692/2975 [1:49:53<10:55,  2.32s/it]
 91%|█████████ | 2693/2975 [1:49:56<10:49,  2.30s/it]
 91%|█████████ | 2694/2975 [1:49:58<10:44,  2.29s/it]
 91%|█████████ | 
0: {'loss': 0.4112, 'grad_norm': 0.46449759477800795, 'learning_rate': 2.9575872675236522e-06, 'epoch': 0.91}
0: 2695/2975 [1:50:00<10:50,  2.32s/it]
 91%|█████████ | 2696/2975 [1:50:03<10:45,  2.31s/it]
 91%|█████████ | 2697/2975 [1:50:05<10:40,  2.30s/it]
 91%|█████████ | 2698/2975 [1:50:07<10:56,  2.37s/it]
 91%|█████████ | 2699/2975 [1:50:10<10:44,  2.34s/it]
 91%|█████████ | 2700/2975 [1:50:13<11:39,  2.54s/it]
                                                     

 91%|█████████ | 2700/2975 [1:50:13<11:39,  2.54s/it]
 91%|█████████ | 2701/2975 [1:50:15<11:32,  2.53s/it]
 91%|█████████ | 2702/2975 [1:50:17<11:12,  2.46s/it]
 91%|█████████ | 2703/2975 [1:50:20<11:41,  2.58s/it]
 91%|█████████ | 2704/2975 [1:50:23<11:14,  2.49s/it]
 91%|█████████ | 2705/2975 [1:50:25<10:56,  2.43s/it]
 91%|█████████ | 2706/2975 [1:50:27<10:41,  2.39s/it]
 91%|█████████ | 2707/2975 [1:50:29<10:30,  2.35s/
0: {'loss': 0.4095, 'grad_norm': 0.4692134110917298, 'learning_rate': 2.915330685814054e-06, 'epoch': 0.91}
0: it]
 91%|█████████ | 2708/2975 [1:50:32<10:21,  2.33s/it]
 91%|█████████ | 2709/2975 [1:50:35<10:58,  2.48s/it]
 91%|█████████ | 2710/2975 [1:50:37<10:44,  2.43s/it]
                                                     

 91%|█████████ | 2710/2975 [1:50:37<10:44,  2.43s/it]
 91%|█████████ | 2711/2975 [1:50:39<10:30,  2.39s/it]
 91%|█████████ | 2712/2975 [1:50:41<10:19,  2.35s/it]
 91%|█████████ | 2713/2975 [1:50:44<10:10,  2.33s/it]
 91%|█████████ | 2714/2975 [1:50:46<10:04,  2.32s/it]
 91%|█████████▏| 2715/2975 [1:50:48<09:59,  2.30s/it]
 91%|█████████▏| 2716/2975 [1:50:51<09:56,  2.30s/it]
 91%|█████████▏| 2717/2975 [1:50:53<09:52,  2.30s/it]
 91%|█████████▏| 2718/2975 [1:50:55<09:48,  2.29s/it]
 91%|█████████▏| 2719/2975 [1:50:57<09:44,  2.28s/it]
 91%|████�
0: {'loss': 0.4132, 'grad_norm': 0.49539755148015086, 'learning_rate': 2.859210876323207e-06, 'epoch': 0.91}
0: {'loss': 0.4167, 'grad_norm': 0.4599663926841582, 'learning_rate': 2.7898426994277207e-06, 'epoch': 0.92}
0: �████▏| 2720/2975 [1:51:00<09:43,  2.29s/it]
                                                     

 91%|█████████▏| 2720/2975 [1:51:00<09:43,  2.29s/it]
 91%|█████████▏| 2721/2975 [1:51:02<09:40,  2.29s/it]
 91%|█████████▏| 2722/2975 [1:51:04<09:46,  2.32s/it]
 92%|█████████▏| 2723/2975 [1:51:07<09:49,  2.34s/it]
 92%|█████████▏| 2724/2975 [1:51:09<09:43,  2.32s/it]
 92%|█████████▏| 2725/2975 [1:51:11<09:36,  2.31s/it]
 92%|█████████▏| 2726/2975 [1:51:14<09:35,  2.31s/it]
 92%|█████████▏| 2727/2975 [1:51:16<09:29,  2.30s/it]
 92%|█████████▏| 2728/2975 [1:51:18<09:33,  2.32s/it]
 92%|█████████▏| 2729/2975 [1:51:21<09:28,  2.31s/it]
 92%|█████████▏| 2730/2975 [1:51:23<09:25,  2.31s/it]
                                                     

 92%|█████████▏| 2730/297
0: {'loss': 0.4103, 'grad_norm': 0.48513842812044483, 'learning_rate': 2.707986167389884e-06, 'epoch': 0.92}
0: 5 [1:51:23<09:25,  2.31s/it]
 92%|█████████▏| 2731/2975 [1:51:25<09:20,  2.30s/it]
 92%|█████████▏| 2732/2975 [1:51:28<09:46,  2.41s/it]
 92%|█████████▏| 2733/2975 [1:51:30<09:34,  2.37s/it]
 92%|█████████▏| 2734/2975 [1:51:32<09:23,  2.34s/it]
 92%|█████████▏| 2735/2975 [1:51:35<09:41,  2.42s/it]
 92%|█████████▏| 2736/2975 [1:51:37<09:29,  2.38s/it]
 92%|█████████▏| 2737/2975 [1:51:40<09:18,  2.35s/it]
 92%|█████████▏| 2738/2975 [1:51:42<09:10,  2.32s/it]
 92%|█████████▏| 2739/2975 [1:51:44<09:05,  2.31s/it]
 92%|█████████▏| 2740/2975 [1:51:46<09:05,  2.32s/it]
                                                     

 92%|█████████▏| 2740/2975 [1:51:46<09:05,  2.32s/it]
 92%|█████████▏| 2741/2975 [1:51:49<09:01,  2.31s/it]
 92%|█████████▏| 2742/2975 [1:51
0: {'loss': 0.4164, 'grad_norm': 0.4404565751895873, 'learning_rate': 2.6145381175042847e-06, 'epoch': 0.92}
0: :51<09:08,  2.35s/it]
 92%|█████████▏| 2743/2975 [1:51:53<09:01,  2.33s/it]
 92%|█████████▏| 2744/2975 [1:51:56<08:55,  2.32s/it]
 92%|█████████▏| 2745/2975 [1:51:58<08:58,  2.34s/it]
 92%|█████████▏| 2746/2975 [1:52:00<08:51,  2.32s/it]
 92%|█████████▏| 2747/2975 [1:52:03<08:46,  2.31s/it]
 92%|█████████▏| 2748/2975 [1:52:05<08:42,  2.30s/it]
 92%|█████████▏| 2749/2975 [1:52:07<08:39,  2.30s/it]
 92%|█████████▏| 2750/2975 [1:52:10<08:34,  2.29s/it]
                                                     

 92%|█████████▏| 2750/2975 [1:52:10<08:34,  2.29s/it]
 92%|█████████▏| 2751/2975 [1:52:12<08:33,  2.29s/it]
 93%|█████████▎| 2752/2975 [1:52:14<08:32,  2.30s/it]
 93%|█████████▎| 2753/2975 [1:52:16<08:32,  2.31s/it]
 93%|█████████▎| 2754/2975 [1:52:19<08:
0: {'loss': 0.4174, 'grad_norm': 0.44437805967514776, 'learning_rate': 2.5105223861607313e-06, 'epoch': 0.93}
0: 28,  2.30s/it]
 93%|█████████▎| 2755/2975 [1:52:21<08:23,  2.29s/it]
 93%|█████████▎| 2756/2975 [1:52:23<08:30,  2.33s/it]
 93%|█████████▎| 2757/2975 [1:52:26<08:24,  2.32s/it]
 93%|█████████▎| 2758/2975 [1:52:28<08:19,  2.30s/it]
 93%|█████████▎| 2759/2975 [1:52:30<08:17,  2.30s/it]
 93%|█████████▎| 2760/2975 [1:52:33<08:11,  2.29s/it]
                                                     

 93%|█████████▎| 2760/2975 [1:52:33<08:11,  2.29s/it]
 93%|█████████▎| 2761/2975 [1:52:35<08:09,  2.29s/it]
 93%|█████████▎| 2762/2975 [1:52:37<08:06,  2.28s/it]
 93%|█████████▎| 2763/2975 [1:52:39<08:04,  2.28s/it]
 93%|█████████▎| 2764/2975 [1:52:42<08:03,  2.29s/it]
 93%|█████████▎| 2765/2975 [1:52:44<08:00,  2.29s/it]
 93%|█████████▎| 2766/2975 [1:52:46<07:58,  2.
0: {'loss': 0.4179, 'grad_norm': 0.4557315628025563, 'learning_rate': 2.3970785914785146e-06, 'epoch': 0.93}
0: 29s/it]
 93%|█████████▎| 2767/2975 [1:52:49<07:55,  2.29s/it]
 93%|█████████▎| 2768/2975 [1:52:51<07:52,  2.28s/it]
 93%|█████████▎| 2769/2975 [1:52:53<07:49,  2.28s/it]
 93%|█████████▎| 2770/2975 [1:52:55<07:50,  2.29s/it]
                                                     

 93%|█████████▎| 2770/2975 [1:52:55<07:50,  2.29s/it]
 93%|█████████▎| 2771/2975 [1:52:58<07:47,  2.29s/it]
 93%|█████████▎| 2772/2975 [1:53:00<07:44,  2.29s/it]
 93%|█████████▎| 2773/2975 [1:53:02<07:42,  2.29s/it]
 93%|█████████▎| 2774/2975 [1:53:05<07:40,  2.29s/it]
 93%|█████████▎| 2775/2975 [1:53:07<07:38,  2.29s/it]
 93%|█████████▎| 2776/2975 [1:53:09<07:35,  2.29s/it]
 93%|█████████▎| 2777/2975 [1:53:12<07:55,  2.40s/it]
 93%|█████████▎| 2778/2975 [1:53:14<07:46,  2.37s/it]
0: {'loss': 0.4192, 'grad_norm': 0.4683155002154871, 'learning_rate': 2.2754496474118135e-06, 'epoch': 0.93}
0: 
 93%|█████████▎| 2779/2975 [1:53:16<07:42,  2.36s/it]
 93%|█████████▎| 2780/2975 [1:53:19<07:35,  2.33s/it]
                                                     

 93%|█████████▎| 2780/2975 [1:53:19<07:35,  2.33s/it]
 93%|█████████▎| 2781/2975 [1:53:21<07:31,  2.32s/it]
 94%|█████████▎| 2782/2975 [1:53:23<07:25,  2.31s/it]
 94%|█████████▎| 2783/2975 [1:53:26<07:20,  2.30s/it]
 94%|█████████▎| 2784/2975 [1:53:28<07:17,  2.29s/it]
 94%|█████████▎| 2785/2975 [1:53:30<07:15,  2.29s/it]
 94%|█████████▎| 2786/2975 [1:53:32<07:13,  2.30s/it]
 94%|█████████▎| 2787/2975 [1:53:35<07:21,  2.35s/it]
 94%|█████████▎| 2788/2975 [1:53:37<07:15,  2.33s/it]
 94%|█████████▎| 2789/2975 [1:53:40<07:16,  2.35s/it]
 94%|█████████▍| 2790/2975 [1:53:42<07:11,  2.33s/it]
      
0: {'loss': 0.4177, 'grad_norm': 0.45294960546087315, 'learning_rate': 2.1469681461243154e-06, 'epoch': 0.94}
0: {'loss': 0.4148, 'grad_norm': 0.4658564943345454, 'learning_rate': 2.0130417578306083e-06, 'epoch': 0.94}
0:                                                

 94%|█████████▍| 2790/2975 [1:53:42<07:11,  2.33s/it]
 94%|█████████▍| 2791/2975 [1:53:44<07:05,  2.31s/it]
 94%|█████████▍| 2792/2975 [1:53:46<07:01,  2.30s/it]
 94%|█████████▍| 2793/2975 [1:53:49<06:58,  2.30s/it]
 94%|█████████▍| 2794/2975 [1:53:51<06:53,  2.29s/it]
 94%|█████████▍| 2795/2975 [1:53:53<06:55,  2.31s/it]
 94%|█████████▍| 2796/2975 [1:53:56<06:53,  2.31s/it]
 94%|█████████▍| 2797/2975 [1:53:58<07:15,  2.44s/it]
 94%|█████████▍| 2798/2975 [1:54:01<07:03,  2.39s/it]
 94%|█████████▍| 2799/2975 [1:54:03<06:55,  2.36s/it]
 94%|█████████▍| 2800/2975 [1:54:05<06:49,  2.34s/it]
                                                     

 94%|█████████▍| 2800/2975 [1:54:05<06:49,  2.34s/it]
 94%|█████████
0: {'loss': 0.4112, 'grad_norm': 0.44911216339602816, 'learning_rate': 1.8751378080667381e-06, 'epoch': 0.94}
0: ▍| 2801/2975 [1:54:08<06:43,  2.32s/it]
 94%|█████████▍| 2802/2975 [1:54:10<06:39,  2.31s/it]
 94%|█████████▍| 2803/2975 [1:54:12<06:37,  2.31s/it]
 94%|█████████▍| 2804/2975 [1:54:14<06:33,  2.30s/it]
 94%|█████████▍| 2805/2975 [1:54:17<06:30,  2.30s/it]
 94%|█████████▍| 2806/2975 [1:54:19<06:26,  2.29s/it]
 94%|█████████▍| 2807/2975 [1:54:21<06:23,  2.28s/it]
 94%|█████████▍| 2808/2975 [1:54:24<06:21,  2.28s/it]
 94%|█████████▍| 2809/2975 [1:54:26<06:18,  2.28s/it]
 94%|█████████▍| 2810/2975 [1:54:28<06:15,  2.28s/it]
                                                     

 94%|█████████▍| 2810/2975 [1:54:28<06:15,  2.28s/it]
 94%|█████████▍| 2811/2975 [1:54:30<06:13,  2.28s/it]
 95%|█████████▍| 2812/2975 [1:54:33<06:11,  2.28s/it]
 95%|█████████▍| 28
0: {'loss': 0.4216, 'grad_norm': 0.47128136123627, 'learning_rate': 1.7347672013645732e-06, 'epoch': 0.95}
0: 13/2975 [1:54:35<06:11,  2.29s/it]
 95%|█████████▍| 2814/2975 [1:54:37<06:07,  2.28s/it]
 95%|█████████▍| 2815/2975 [1:54:40<06:07,  2.30s/it]
 95%|█████████▍| 2816/2975 [1:54:42<06:03,  2.29s/it]
 95%|█████████▍| 2817/2975 [1:54:44<06:00,  2.28s/it]
 95%|█████████▍| 2818/2975 [1:54:46<05:57,  2.28s/it]
 95%|█████████▍| 2819/2975 [1:54:49<05:57,  2.29s/it]
 95%|█████████▍| 2820/2975 [1:54:51<05:54,  2.28s/it]
                                                     

 95%|█████████▍| 2820/2975 [1:54:51<05:54,  2.28s/it]
 95%|█████████▍| 2821/2975 [1:54:53<05:51,  2.28s/it]
 95%|█████████▍| 2822/2975 [1:54:56<05:48,  2.28s/it]
 95%|█████████▍| 2823/2975 [1:54:58<05:47,  2.28s/it]
 95%|█████████▍| 2824/2975 [1:55:00<05:44,  2.28s/it]
 95%|█████████▍| 2825/2975
0: {'loss': 0.421, 'grad_norm': 0.4492829435970908, 'learning_rate': 1.5934678674655809e-06, 'epoch': 0.95}
0:  [1:55:02<05:41,  2.28s/it]
 95%|█████████▍| 2826/2975 [1:55:05<05:39,  2.28s/it]
 95%|█████████▌| 2827/2975 [1:55:07<05:37,  2.28s/it]
 95%|█████████▌| 2828/2975 [1:55:09<05:35,  2.28s/it]
 95%|█████████▌| 2829/2975 [1:55:11<05:32,  2.28s/it]
 95%|█████████▌| 2830/2975 [1:55:14<05:31,  2.29s/it]
                                                     

 95%|█████████▌| 2830/2975 [1:55:14<05:31,  2.29s/it]
 95%|█████████▌| 2831/2975 [1:55:16<05:29,  2.29s/it]
 95%|█████████▌| 2832/2975 [1:55:18<05:26,  2.28s/it]
 95%|█████████▌| 2833/2975 [1:55:21<05:24,  2.28s/it]
 95%|█████████▌| 2834/2975 [1:55:23<05:21,  2.28s/it]
 95%|█████████▌| 2835/2975 [1:55:25<05:20,  2.29s/it]
 95%|█████████▌| 2836/2975 [1:55:27<05:18,  2.29s/it]
 95%|█████████▌| 2837/2975 [1:55:
0: {'loss': 0.4085, 'grad_norm': 0.45247226107883587, 'learning_rate': 1.4527879114407447e-06, 'epoch': 0.95}
0: 30<05:16,  2.29s/it]
 95%|█████████▌| 2838/2975 [1:55:32<05:13,  2.29s/it]
 95%|█████████▌| 2839/2975 [1:55:34<05:11,  2.29s/it]
 95%|█████████▌| 2840/2975 [1:55:37<05:08,  2.28s/it]
                                                     

 95%|█████████▌| 2840/2975 [1:55:37<05:08,  2.28s/it]
 95%|█████████▌| 2841/2975 [1:55:39<05:07,  2.30s/it]
 96%|█████████▌| 2842/2975 [1:55:41<05:04,  2.29s/it]
 96%|█████████▌| 2843/2975 [1:55:44<05:01,  2.28s/it]
 96%|█████████▌| 2844/2975 [1:55:46<04:59,  2.28s/it]
 96%|█████████▌| 2845/2975 [1:55:48<05:02,  2.32s/it]
 96%|█████████▌| 2846/2975 [1:55:50<04:58,  2.31s/it]
 96%|█████████▌| 2847/2975 [1:55:53<04:56,  2.32s/it]
 96%|█████████▌| 2848/2975 [1:55:55<04:52,  2.31s/it]
 96%|█████████▌| 2849/2975 [1:55:57<04:4
0: {'loss': 0.4144, 'grad_norm': 0.445445383609263, 'learning_rate': 1.3142686523274465e-06, 'epoch': 0.96}
0: {'loss': 0.4193, 'grad_norm': 0.43811132527565, 'learning_rate': 1.1794277361155496e-06, 'epoch': 0.96}
0: 9,  2.30s/it]
 96%|█████████▌| 2850/2975 [1:56:00<04:48,  2.31s/it]
                                                     

 96%|█████████▌| 2850/2975 [1:56:00<04:48,  2.31s/it]
 96%|█████████▌| 2851/2975 [1:56:02<04:45,  2.30s/it]
 96%|█████████▌| 2852/2975 [1:56:04<04:41,  2.29s/it]
 96%|█████████▌| 2853/2975 [1:56:07<04:39,  2.29s/it]
 96%|█████████▌| 2854/2975 [1:56:09<04:37,  2.29s/it]
 96%|█████████▌| 2855/2975 [1:56:11<04:35,  2.29s/it]
 96%|█████████▌| 2856/2975 [1:56:13<04:33,  2.30s/it]
 96%|█████████▌| 2857/2975 [1:56:16<04:31,  2.30s/it]
 96%|█████████▌| 2858/2975 [1:56:18<04:28,  2.29s/it]
 96%|█████████▌| 2859/2975 [1:56:20<04:26,  2.30s/it]
 96%|█████████▌| 2860/2975 [1:56:23<04:32,  2.37s/it]
                                                     

 96%|██�
0: {'loss': 0.4139, 'grad_norm': 0.46032460681351256, 'learning_rate': 1.0497425081003483e-06, 'epoch': 0.96}
0: ��██████▌| 2860/2975 [1:56:23<04:32,  2.37s/it]
 96%|█████████▌| 2861/2975 [1:56:25<04:33,  2.40s/it]
 96%|█████████▌| 2862/2975 [1:56:28<04:26,  2.36s/it]
 96%|█████████▌| 2863/2975 [1:56:30<04:22,  2.35s/it]
 96%|█████████▋| 2864/2975 [1:56:32<04:18,  2.33s/it]
 96%|█████████▋| 2865/2975 [1:56:35<04:37,  2.52s/it]
 96%|█████████▋| 2866/2975 [1:56:37<04:26,  2.45s/it]
 96%|█████████▋| 2867/2975 [1:56:40<04:18,  2.39s/it]
 96%|█████████▋| 2868/2975 [1:56:42<04:12,  2.36s/it]
 96%|█████████▋| 2869/2975 [1:56:44<04:10,  2.36s/it]
 96%|█████████▋| 2870/2975 [1:56:47<04:05,  2.34s/it]
                                                     

 96%|█████████▋| 2870/2975 [1:56:47<04:05,  2.34s/it]
 97%|█████████▋| 2871/2975 [1:56:49<04:01,  2.33s/it]
 97%|████�
0: {'loss': 0.4166, 'grad_norm': 0.45373908391590473, 'learning_rate': 9.266338267783543e-07, 'epoch': 0.97}
0: �████▋| 2872/2975 [1:56:51<03:58,  2.31s/it]
 97%|█████████▋| 2873/2975 [1:56:54<03:54,  2.30s/it]
 97%|█████████▋| 2874/2975 [1:56:56<03:51,  2.29s/it]
 97%|█████████▋| 2875/2975 [1:56:58<03:49,  2.30s/it]
 97%|█████████▋| 2876/2975 [1:57:00<03:46,  2.29s/it]
 97%|█████████▋| 2877/2975 [1:57:03<03:44,  2.29s/it]
 97%|█████████▋| 2878/2975 [1:57:05<03:42,  2.29s/it]
 97%|█████████▋| 2879/2975 [1:57:07<03:40,  2.30s/it]
 97%|█████████▋| 2880/2975 [1:57:10<03:38,  2.30s/it]
                                                     

 97%|█████████▋| 2880/2975 [1:57:10<03:38,  2.30s/it]
 97%|█████████▋| 2881/2975 [1:57:12<03:36,  2.30s/it]
 97%|█████████▋| 2882/2975 [1:57:14<03:33,  2.30s/it]
 97%|█████████▋| 2883/2975 [1:57:16<03:31,  2.30s/it]
 97%|███████
0: {'loss': 0.416, 'grad_norm': 0.48820830829380396, 'learning_rate': 8.114504966242812e-07, 'epoch': 0.97}
0: ██▋| 2884/2975 [1:57:19<03:29,  2.30s/it]
 97%|█████████▋| 2885/2975 [1:57:21<03:28,  2.32s/it]
 97%|█████████▋| 2886/2975 [1:57:23<03:25,  2.31s/it]
 97%|█████████▋| 2887/2975 [1:57:26<03:22,  2.31s/it]
 97%|█████████▋| 2888/2975 [1:57:28<03:20,  2.30s/it]
 97%|█████████▋| 2889/2975 [1:57:30<03:17,  2.30s/it]
 97%|█████████▋| 2890/2975 [1:57:33<03:15,  2.30s/it]
                                                     

 97%|█████████▋| 2890/2975 [1:57:33<03:15,  2.30s/it]
 97%|█████████▋| 2891/2975 [1:57:35<03:12,  2.29s/it]
 97%|█████████▋| 2892/2975 [1:57:37<03:11,  2.31s/it]
 97%|█████████▋| 2893/2975 [1:57:39<03:08,  2.30s/it]
 97%|█████████▋| 2894/2975 [1:57:42<03:07,  2.31s/it]
 97%|█████████▋| 2895/2975 [1:57:44<03:04,  2.30s/it]
 97%|█████████�
0: {'loss': 0.4108, 'grad_norm': 0.42510146439702673, 'learning_rate': 7.054544903069566e-07, 'epoch': 0.97}
0: ��| 2896/2975 [1:57:46<03:01,  2.30s/it]
 97%|█████████▋| 2897/2975 [1:57:49<03:00,  2.31s/it]
 97%|█████████▋| 2898/2975 [1:57:51<02:56,  2.30s/it]
 97%|█████████▋| 2899/2975 [1:57:53<02:56,  2.32s/it]
 97%|█████████▋| 2900/2975 [1:57:56<02:54,  2.32s/it]
                                                     

 97%|█████████▋| 2900/2975 [1:57:56<02:54,  2.32s/it]
 98%|█████████▊| 2901/2975 [1:57:58<03:01,  2.46s/it]
 98%|█████████▊| 2902/2975 [1:58:01<02:55,  2.40s/it]
 98%|█████████▊| 2903/2975 [1:58:03<02:50,  2.37s/it]
 98%|█████████▊| 2904/2975 [1:58:05<02:45,  2.33s/it]
 98%|█████████▊| 2905/2975 [1:58:08<02:50,  2.44s/it]
 98%|█████████▊| 2906/2975 [1:58:10<02:45,  2.39s/it]
 98%|█████████▊| 2907/2975 [1:58:13<02:40,  2.36s/it]
 98%|█████████▊| 290
0: {'loss': 0.4213, 'grad_norm': 0.462542311179908, 'learning_rate': 6.098071222526847e-07, 'epoch': 0.98}
0: 8/2975 [1:58:15<02:36,  2.34s/it]
 98%|█████████▊| 2909/2975 [1:58:17<02:33,  2.32s/it]
 98%|█████████▊| 2910/2975 [1:58:19<02:30,  2.31s/it]
                                                     

 98%|█████████▊| 2910/2975 [1:58:19<02:30,  2.31s/it]
 98%|█████████▊| 2911/2975 [1:58:22<02:27,  2.30s/it]
 98%|█████████▊| 2912/2975 [1:58:24<02:24,  2.30s/it]
 98%|█████████▊| 2913/2975 [1:58:26<02:21,  2.29s/it]
 98%|█████████▊| 2914/2975 [1:58:28<02:19,  2.28s/it]
 98%|█████████▊| 2915/2975 [1:58:31<02:18,  2.30s/it]
 98%|█████████▊| 2916/2975 [1:58:33<02:15,  2.30s/it]
 98%|█████████▊| 2917/2975 [1:58:35<02:12,  2.29s/it]
 98%|█████████▊| 2918/2975 [1:58:38<02:10,  2.29s/it]
 98%|█████████▊| 2919/2975 [1:58:40<02:08,  2.29s/it]
 98%|█████████▊| 2920/2975 
0: {'loss': 0.4024, 'grad_norm': 0.441015307443321, 'learning_rate': 5.255563250413658e-07, 'epoch': 0.98}
0: {'loss': 0.4188, 'grad_norm': 0.4546580627666499, 'learning_rate': 4.5362516803786016e-07, 'epoch': 0.98}
0: [1:58:42<02:06,  2.30s/it]
                                                     

 98%|█████████▊| 2920/2975 [1:58:42<02:06,  2.30s/it]
 98%|█████████▊| 2921/2975 [1:58:45<02:04,  2.30s/it]
 98%|█████████▊| 2922/2975 [1:58:47<02:01,  2.30s/it]
 98%|█████████▊| 2923/2975 [1:58:50<02:04,  2.40s/it]
 98%|█████████▊| 2924/2975 [1:58:52<02:00,  2.36s/it]
 98%|█████████▊| 2925/2975 [1:58:54<01:57,  2.34s/it]
 98%|█████████▊| 2926/2975 [1:58:56<01:53,  2.32s/it]
 98%|█████████▊| 2927/2975 [1:58:59<01:51,  2.32s/it]
 98%|█████████▊| 2928/2975 [1:59:01<01:48,  2.31s/it]
 98%|█████████▊| 2929/2975 [1:59:03<01:45,  2.30s/it]
 98%|█████████▊| 2930/2975 [1:59:06<01:43,  2.30s/it]
                                                     

 98%|█████████▊| 2930/2975 [1:59:06<01:43,  2.30s/it]
0: {'loss': 0.4077, 'grad_norm': 0.447404880960169, 'learning_rate': 3.948017440508607e-07, 'epoch': 0.99}
0: 
 99%|█████████▊| 2931/2975 [1:59:08<01:44,  2.37s/it]
 99%|█████████▊| 2932/2975 [1:59:11<01:42,  2.40s/it]
 99%|█████████▊| 2933/2975 [1:59:13<01:40,  2.39s/it]
 99%|█████████▊| 2934/2975 [1:59:15<01:38,  2.41s/it]
 99%|█████████▊| 2935/2975 [1:59:18<01:42,  2.55s/it]
 99%|█████████▊| 2936/2975 [1:59:21<01:36,  2.47s/it]
 99%|█████████▊| 2937/2975 [1:59:23<01:32,  2.44s/it]
 99%|█████████▉| 2938/2975 [1:59:25<01:28,  2.39s/it]
 99%|█████████▉| 2939/2975 [1:59:27<01:24,  2.36s/it]
 99%|█████████▉| 2940/2975 [1:59:30<01:21,  2.33s/it]
                                                     

 99%|█████████▉| 2940/2975 [1:59:30<01:21,  2.33s/it]
 99%|█████████▉| 2941/2975 [1:59:32<01:18,  2.31s/it]
 99%|█████████▉| 2942/2975 [1:59:34<01:15,  2.30s/it]
 99%|�
0: {'loss': 0.4229, 'grad_norm': 0.45813876446091023, 'learning_rate': 3.497305348231616e-07, 'epoch': 0.99}
0: ��████████▉| 2943/2975 [1:59:37<01:20,  2.52s/it]
 99%|█████████▉| 2944/2975 [1:59:40<01:17,  2.50s/it]
 99%|█████████▉| 2945/2975 [1:59:42<01:13,  2.45s/it]
 99%|█████████▉| 2946/2975 [1:59:44<01:09,  2.40s/it]
 99%|█████████▉| 2947/2975 [1:59:47<01:05,  2.36s/it]
 99%|█████████▉| 2948/2975 [1:59:49<01:03,  2.34s/it]
 99%|█████████▉| 2949/2975 [1:59:51<01:00,  2.33s/it]
 99%|█████████▉| 2950/2975 [1:59:54<00:57,  2.32s/it]
                                                     

 99%|█████████▉| 2950/2975 [1:59:54<00:57,  2.32s/it]
 99%|█████████▉| 2951/2975 [1:59:56<00:55,  2.31s/it]
 99%|█████████▉| 2952/2975 [1:59:58<00:52,  2.30s/it]
 99%|█████████▉| 2953/2975 [2:00:00<00:50,  2.29s/it]
 99%|█████████▉| 2954/2975 [2:00:03<00:48,  2.29s/it]
 99%|██�
0: {'loss': 0.4158, 'grad_norm': 0.45429090307434694, 'learning_rate': 3.189053499548184e-07, 'epoch': 0.99}
0: �██████▉| 2955/2975 [2:00:05<00:45,  2.30s/it]
 99%|█████████▉| 2956/2975 [2:00:07<00:43,  2.29s/it]
 99%|█████████▉| 2957/2975 [2:00:10<00:41,  2.29s/it]
 99%|█████████▉| 2958/2975 [2:00:12<00:39,  2.30s/it]
 99%|█████████▉| 2959/2975 [2:00:14<00:36,  2.30s/it]
 99%|█████████▉| 2960/2975 [2:00:16<00:34,  2.29s/it]
                                                     

 99%|█████████▉| 2960/2975 [2:00:16<00:34,  2.29s/it]
100%|█████████▉| 2961/2975 [2:00:19<00:32,  2.29s/it]
100%|█████████▉| 2962/2975 [2:00:21<00:30,  2.32s/it]
100%|█████████▉| 2963/2975 [2:00:23<00:27,  2.31s/it]
100%|█████████▉| 2964/2975 [2:00:26<00:25,  2.30s/it]
100%|█████████▉| 2965/2975 [2:00:28<00:22,  2.30s/it]
100%|█████████▉| 2966/2975 [2:00:30<00:20,  2.29s/it]
100%|█████
0: {'loss': 0.4135, 'grad_norm': 0.4473567583054121, 'learning_rate': 3.026639166218334e-07, 'epoch': 1.0}
0: {'train_runtime': 7272.845, 'train_samples_per_second': 6.545, 'train_steps_per_second': 0.409, 'train_loss': 0.43875975352375446, 'epoch': 1.0}
0: ████▉| 2967/2975 [2:00:32<00:18,  2.28s/it]
100%|█████████▉| 2968/2975 [2:00:35<00:15,  2.28s/it]
100%|█████████▉| 2969/2975 [2:00:37<00:13,  2.27s/it]
100%|█████████▉| 2970/2975 [2:00:39<00:11,  2.29s/it]
                                                     

100%|█████████▉| 2970/2975 [2:00:39<00:11,  2.29s/it]
100%|█████████▉| 2971/2975 [2:00:42<00:09,  2.29s/it]
100%|█████████▉| 2972/2975 [2:00:44<00:06,  2.29s/it]
100%|█████████▉| 2973/2975 [2:00:46<00:04,  2.30s/it]
100%|█████████▉| 2974/2975 [2:00:49<00:02,  2.35s/it]
100%|██████████| 2975/2975 [2:00:51<00:00,  2.33s/it]
                                                     

100%|██████████| 2975/2975 [2:01:12<00:00,  2.33s/it]
100%|██████████| 2975/2975 [2:01:12<00:00,  2.44s/it]
0: [2025-08-20 19:20:24,652] [INFO] [axolotl.train.save_trained_model:246] [PID:3534656] [RANK:0] Training completed! Saving trained model to /lustre/fswork/projects/rech/dgo/udv55np/ift/Qwen3-235B-A22B/Qwen2.5-7B/0.
0: [2025-08-20 19:20:39,852] [INFO] [axolotl.train.save_trained_model:331] [PID:3534656] [RANK:0] Model successfully saved to /lustre/fswork/projects/rech/dgo/udv55np/ift/Qwen3-235B-A22B/Qwen2.5-7B/0
2: W0820 19:20:43.678000 366223 torch/distributed/elastic/rendezvous/dynamic_rendezvous.py:1333] The node 'jzxh163.hpc.idris.fr_366223_0' has failed to send a keep-alive heartbeat to the rendezvous '417575' due to an error of type RendezvousTimeoutError.
1: W0820 19:20:43.678000 1669907 torch/distributed/elastic/rendezvous/dynamic_rendezvous.py:1333] The node 'jzxh132.hpc.idris.fr_1669907_0' has failed to send a keep-alive heartbeat to the rendezvous '417575' due to an error of type RendezvousTimeoutError.