Hugging Face's logo

jbenbudd
/
ADPrLlama

Text Generation
Transformers
Safetensors
PEFT
llama
protein
ptm
adp-ribosylation
lora
conversational
text-generation-inference
Model card Files
xet
 Community
Ben Buddendorff commited on
Commit
61d2404
·
1 Parent(s): 35bb4b1

train_1_epoch_test

Browse files
Files changed (8) hide show
  1. README.md +5 -5
  2. all_results.json +11 -11
  3. model.safetensors.index.json +298 -0
  4. trainer_log.jsonl +47 -185
  5. trainer_state.json +217 -1591
  6. training_args.bin +1 -1
  7. training_eval_loss.png +2 -2
  8. training_loss.png +2 -2
README.md CHANGED
@@ -7,19 +7,19 @@ tags:
7
  - lora
8
  - generated_from_trainer
9
  model-index:
10
- - name: train_202507141200
11
  results: []
12
  ---
13
 
14
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
  should probably proofread and complete it, then remove this comment. -->
16
 
17
- # train_202507141200
18
 
19
  This model is a fine-tuned version of [GreatCaptainNemo/ProLLaMA_Stage_1](https://huggingface.co/GreatCaptainNemo/ProLLaMA_Stage_1) on the adpr_train dataset.
20
  It achieves the following results on the evaluation set:
21
- - Loss: 0.2193
22
- - Num Input Tokens Seen: 10772736
23
 
24
  ## Model description
25
 
@@ -47,7 +47,7 @@ The following hyperparameters were used during training:
47
  - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
48
  - lr_scheduler_type: cosine
49
  - lr_scheduler_warmup_steps: 40
50
- - num_epochs: 4.0
51
 
52
  ### Training results
53
 
 
7
  - lora
8
  - generated_from_trainer
9
  model-index:
10
+ - name: train_1_epoch_test
11
  results: []
12
  ---
13
 
14
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
  should probably proofread and complete it, then remove this comment. -->
16
 
17
+ # train_1_epoch_test
18
 
19
  This model is a fine-tuned version of [GreatCaptainNemo/ProLLaMA_Stage_1](https://huggingface.co/GreatCaptainNemo/ProLLaMA_Stage_1) on the adpr_train dataset.
20
  It achieves the following results on the evaluation set:
21
+ - Loss: 0.3777
22
+ - Num Input Tokens Seen: 2691984
23
 
24
  ## Model description
25
 
 
47
  - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
48
  - lr_scheduler_type: cosine
49
  - lr_scheduler_warmup_steps: 40
50
+ - num_epochs: 1.0
51
 
52
  ### Training results
53
 
all_results.json CHANGED
@@ -1,13 +1,13 @@
1
  {
2
- "epoch": 4.0,
3
- "eval_loss": 0.21931865811347961,
4
- "eval_runtime": 32.2123,
5
- "eval_samples_per_second": 96.795,
6
- "eval_steps_per_second": 6.054,
7
- "num_input_tokens_seen": 10772736,
8
- "total_flos": 4.374108802866217e+17,
9
- "train_loss": 0.3781262902373617,
10
- "train_runtime": 4269.5185,
11
- "train_samples_per_second": 26.287,
12
- "train_steps_per_second": 0.206
13
  }
 
1
  {
2
+ "epoch": 1.0,
3
+ "eval_loss": 0.37766581773757935,
4
+ "eval_runtime": 32.5859,
5
+ "eval_samples_per_second": 95.686,
6
+ "eval_steps_per_second": 5.984,
7
+ "num_input_tokens_seen": 2691984,
8
+ "total_flos": 1.0930399586117222e+17,
9
+ "train_loss": 0.8674792235547846,
10
+ "train_runtime": 1074.777,
11
+ "train_samples_per_second": 26.106,
12
+ "train_steps_per_second": 0.205
13
  }
model.safetensors.index.json ADDED
@@ -0,0 +1,298 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "metadata": {
3
+ "total_size": 13476831232
4
+ },
5
+ "weight_map": {
6
+ "lm_head.weight": "model-00004-of-00004.safetensors",
7
+ "model.embed_tokens.weight": "model-00001-of-00004.safetensors",
8
+ "model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
9
+ "model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
10
+ "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
11
+ "model.layers.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
12
+ "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
13
+ "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
14
+ "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
15
+ "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
16
+ "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
17
+ "model.layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors",
18
+ "model.layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
19
+ "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
20
+ "model.layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
21
+ "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
22
+ "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
23
+ "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
24
+ "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
25
+ "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
26
+ "model.layers.10.input_layernorm.weight": "model-00002-of-00004.safetensors",
27
+ "model.layers.10.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
28
+ "model.layers.10.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
29
+ "model.layers.10.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
30
+ "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
31
+ "model.layers.10.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
32
+ "model.layers.10.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
33
+ "model.layers.10.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
34
+ "model.layers.10.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
35
+ "model.layers.11.input_layernorm.weight": "model-00002-of-00004.safetensors",
36
+ "model.layers.11.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
37
+ "model.layers.11.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
38
+ "model.layers.11.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
39
+ "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
40
+ "model.layers.11.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
41
+ "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
42
+ "model.layers.11.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
43
+ "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
44
+ "model.layers.12.input_layernorm.weight": "model-00002-of-00004.safetensors",
45
+ "model.layers.12.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
46
+ "model.layers.12.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
47
+ "model.layers.12.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
48
+ "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
49
+ "model.layers.12.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
50
+ "model.layers.12.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
51
+ "model.layers.12.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
52
+ "model.layers.12.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
53
+ "model.layers.13.input_layernorm.weight": "model-00002-of-00004.safetensors",
54
+ "model.layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
55
+ "model.layers.13.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
56
+ "model.layers.13.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
57
+ "model.layers.13.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
58
+ "model.layers.13.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
59
+ "model.layers.13.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
60
+ "model.layers.13.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
61
+ "model.layers.13.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
62
+ "model.layers.14.input_layernorm.weight": "model-00002-of-00004.safetensors",
63
+ "model.layers.14.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
64
+ "model.layers.14.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
65
+ "model.layers.14.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
66
+ "model.layers.14.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
67
+ "model.layers.14.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
68
+ "model.layers.14.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
69
+ "model.layers.14.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
70
+ "model.layers.14.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
71
+ "model.layers.15.input_layernorm.weight": "model-00002-of-00004.safetensors",
72
+ "model.layers.15.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
73
+ "model.layers.15.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
74
+ "model.layers.15.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
75
+ "model.layers.15.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
76
+ "model.layers.15.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
77
+ "model.layers.15.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
78
+ "model.layers.15.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
79
+ "model.layers.15.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
80
+ "model.layers.16.input_layernorm.weight": "model-00002-of-00004.safetensors",
81
+ "model.layers.16.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
82
+ "model.layers.16.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
83
+ "model.layers.16.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
84
+ "model.layers.16.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
85
+ "model.layers.16.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
86
+ "model.layers.16.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
87
+ "model.layers.16.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
88
+ "model.layers.16.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
89
+ "model.layers.17.input_layernorm.weight": "model-00002-of-00004.safetensors",
90
+ "model.layers.17.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
91
+ "model.layers.17.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
92
+ "model.layers.17.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
93
+ "model.layers.17.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
94
+ "model.layers.17.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
95
+ "model.layers.17.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
96
+ "model.layers.17.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
97
+ "model.layers.17.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
98
+ "model.layers.18.input_layernorm.weight": "model-00002-of-00004.safetensors",
99
+ "model.layers.18.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
100
+ "model.layers.18.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
101
+ "model.layers.18.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
102
+ "model.layers.18.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
103
+ "model.layers.18.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
104
+ "model.layers.18.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
105
+ "model.layers.18.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
106
+ "model.layers.18.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
107
+ "model.layers.19.input_layernorm.weight": "model-00003-of-00004.safetensors",
108
+ "model.layers.19.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
109
+ "model.layers.19.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
110
+ "model.layers.19.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
111
+ "model.layers.19.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
112
+ "model.layers.19.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
113
+ "model.layers.19.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
114
+ "model.layers.19.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
115
+ "model.layers.19.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
116
+ "model.layers.2.input_layernorm.weight": "model-00001-of-00004.safetensors",
117
+ "model.layers.2.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
118
+ "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
119
+ "model.layers.2.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
120
+ "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
121
+ "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
122
+ "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
123
+ "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
124
+ "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
125
+ "model.layers.20.input_layernorm.weight": "model-00003-of-00004.safetensors",
126
+ "model.layers.20.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
127
+ "model.layers.20.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
128
+ "model.layers.20.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
129
+ "model.layers.20.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
130
+ "model.layers.20.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
131
+ "model.layers.20.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
132
+ "model.layers.20.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
133
+ "model.layers.20.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
134
+ "model.layers.21.input_layernorm.weight": "model-00003-of-00004.safetensors",
135
+ "model.layers.21.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
136
+ "model.layers.21.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
137
+ "model.layers.21.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
138
+ "model.layers.21.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
139
+ "model.layers.21.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
140
+ "model.layers.21.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
141
+ "model.layers.21.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
142
+ "model.layers.21.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
143
+ "model.layers.22.input_layernorm.weight": "model-00003-of-00004.safetensors",
144
+ "model.layers.22.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
145
+ "model.layers.22.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
146
+ "model.layers.22.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
147
+ "model.layers.22.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
148
+ "model.layers.22.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
149
+ "model.layers.22.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
150
+ "model.layers.22.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
151
+ "model.layers.22.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
152
+ "model.layers.23.input_layernorm.weight": "model-00003-of-00004.safetensors",
153
+ "model.layers.23.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
154
+ "model.layers.23.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
155
+ "model.layers.23.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
156
+ "model.layers.23.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
157
+ "model.layers.23.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
158
+ "model.layers.23.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
159
+ "model.layers.23.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
160
+ "model.layers.23.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
161
+ "model.layers.24.input_layernorm.weight": "model-00003-of-00004.safetensors",
162
+ "model.layers.24.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
163
+ "model.layers.24.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
164
+ "model.layers.24.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
165
+ "model.layers.24.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
166
+ "model.layers.24.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
167
+ "model.layers.24.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
168
+ "model.layers.24.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
169
+ "model.layers.24.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
170
+ "model.layers.25.input_layernorm.weight": "model-00003-of-00004.safetensors",
171
+ "model.layers.25.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
172
+ "model.layers.25.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
173
+ "model.layers.25.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
174
+ "model.layers.25.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
175
+ "model.layers.25.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
176
+ "model.layers.25.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
177
+ "model.layers.25.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
178
+ "model.layers.25.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
179
+ "model.layers.26.input_layernorm.weight": "model-00003-of-00004.safetensors",
180
+ "model.layers.26.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
181
+ "model.layers.26.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
182
+ "model.layers.26.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
183
+ "model.layers.26.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
184
+ "model.layers.26.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
185
+ "model.layers.26.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
186
+ "model.layers.26.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
187
+ "model.layers.26.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
188
+ "model.layers.27.input_layernorm.weight": "model-00003-of-00004.safetensors",
189
+ "model.layers.27.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
190
+ "model.layers.27.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
191
+ "model.layers.27.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
192
+ "model.layers.27.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
193
+ "model.layers.27.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
194
+ "model.layers.27.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
195
+ "model.layers.27.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
196
+ "model.layers.27.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
197
+ "model.layers.28.input_layernorm.weight": "model-00004-of-00004.safetensors",
198
+ "model.layers.28.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
199
+ "model.layers.28.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
200
+ "model.layers.28.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
201
+ "model.layers.28.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
202
+ "model.layers.28.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
203
+ "model.layers.28.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
204
+ "model.layers.28.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
205
+ "model.layers.28.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
206
+ "model.layers.29.input_layernorm.weight": "model-00004-of-00004.safetensors",
207
+ "model.layers.29.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
208
+ "model.layers.29.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
209
+ "model.layers.29.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
210
+ "model.layers.29.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
211
+ "model.layers.29.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
212
+ "model.layers.29.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
213
+ "model.layers.29.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
214
+ "model.layers.29.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
215
+ "model.layers.3.input_layernorm.weight": "model-00001-of-00004.safetensors",
216
+ "model.layers.3.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
217
+ "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
218
+ "model.layers.3.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
219
+ "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
220
+ "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
221
+ "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
222
+ "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
223
+ "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
224
+ "model.layers.30.input_layernorm.weight": "model-00004-of-00004.safetensors",
225
+ "model.layers.30.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
226
+ "model.layers.30.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
227
+ "model.layers.30.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
228
+ "model.layers.30.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
229
+ "model.layers.30.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
230
+ "model.layers.30.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
231
+ "model.layers.30.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
232
+ "model.layers.30.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
233
+ "model.layers.31.input_layernorm.weight": "model-00004-of-00004.safetensors",
234
+ "model.layers.31.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
235
+ "model.layers.31.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
236
+ "model.layers.31.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
237
+ "model.layers.31.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
238
+ "model.layers.31.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
239
+ "model.layers.31.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
240
+ "model.layers.31.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
241
+ "model.layers.31.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
242
+ "model.layers.4.input_layernorm.weight": "model-00001-of-00004.safetensors",
243
+ "model.layers.4.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
244
+ "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
245
+ "model.layers.4.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
246
+ "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
247
+ "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
248
+ "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
249
+ "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
250
+ "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
251
+ "model.layers.5.input_layernorm.weight": "model-00001-of-00004.safetensors",
252
+ "model.layers.5.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
253
+ "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
254
+ "model.layers.5.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
255
+ "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
256
+ "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
257
+ "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
258
+ "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
259
+ "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
260
+ "model.layers.6.input_layernorm.weight": "model-00001-of-00004.safetensors",
261
+ "model.layers.6.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
262
+ "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
263
+ "model.layers.6.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
264
+ "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
265
+ "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
266
+ "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
267
+ "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
268
+ "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
269
+ "model.layers.7.input_layernorm.weight": "model-00001-of-00004.safetensors",
270
+ "model.layers.7.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
271
+ "model.layers.7.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
272
+ "model.layers.7.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
273
+ "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
274
+ "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
275
+ "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
276
+ "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
277
+ "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
278
+ "model.layers.8.input_layernorm.weight": "model-00001-of-00004.safetensors",
279
+ "model.layers.8.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
280
+ "model.layers.8.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
281
+ "model.layers.8.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
282
+ "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
283
+ "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
284
+ "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
285
+ "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
286
+ "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
287
+ "model.layers.9.input_layernorm.weight": "model-00002-of-00004.safetensors",
288
+ "model.layers.9.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
289
+ "model.layers.9.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
290
+ "model.layers.9.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
291
+ "model.layers.9.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
292
+ "model.layers.9.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
293
+ "model.layers.9.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
294
+ "model.layers.9.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
295
+ "model.layers.9.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
296
+ "model.norm.weight": "model-00004-of-00004.safetensors"
297
+ }
298
+ }
trainer_log.jsonl CHANGED
@@ -1,185 +1,47 @@
1
- {"current_steps": 5, "total_steps": 880, "loss": 12.6644, "lr": 2.9999999999999997e-05, "epoch": 0.02280501710376283, "percentage": 0.57, "elapsed_time": "0:00:24", "remaining_time": "1:10:49", "throughput": 2540.89, "total_tokens": 61696}
2
- {"current_steps": 10, "total_steps": 880, "loss": 5.0641, "lr": 6.75e-05, "epoch": 0.04561003420752566, "percentage": 1.14, "elapsed_time": "0:00:47", "remaining_time": "1:08:11", "throughput": 2618.17, "total_tokens": 123136}
3
- {"current_steps": 15, "total_steps": 880, "loss": 1.5053, "lr": 0.00010499999999999999, "epoch": 0.06841505131128849, "percentage": 1.7, "elapsed_time": "0:01:09", "remaining_time": "1:07:12", "throughput": 2642.91, "total_tokens": 184832}
4
- {"current_steps": 20, "total_steps": 880, "loss": 0.6917, "lr": 0.0001425, "epoch": 0.09122006841505131, "percentage": 2.27, "elapsed_time": "0:01:33", "remaining_time": "1:06:42", "throughput": 2659.34, "total_tokens": 247552}
5
- {"current_steps": 25, "total_steps": 880, "loss": 0.5721, "lr": 0.00017999999999999998, "epoch": 0.11402508551881414, "percentage": 2.84, "elapsed_time": "0:01:55", "remaining_time": "1:05:59", "throughput": 2668.83, "total_tokens": 308992}
6
- {"current_steps": 30, "total_steps": 880, "loss": 0.6254, "lr": 0.00021749999999999997, "epoch": 0.13683010262257697, "percentage": 3.41, "elapsed_time": "0:02:18", "remaining_time": "1:05:24", "throughput": 2672.47, "total_tokens": 370176}
7
- {"current_steps": 35, "total_steps": 880, "loss": 0.6389, "lr": 0.00025499999999999996, "epoch": 0.15963511972633979, "percentage": 3.98, "elapsed_time": "0:02:41", "remaining_time": "1:04:47", "throughput": 2674.69, "total_tokens": 430720}
8
- {"current_steps": 40, "total_steps": 880, "loss": 0.5258, "lr": 0.00029249999999999995, "epoch": 0.18244013683010263, "percentage": 4.55, "elapsed_time": "0:03:04", "remaining_time": "1:04:29", "throughput": 2674.34, "total_tokens": 492800}
9
- {"current_steps": 45, "total_steps": 880, "loss": 0.5039, "lr": 0.00029998321527154097, "epoch": 0.20524515393386544, "percentage": 5.11, "elapsed_time": "0:03:27", "remaining_time": "1:04:01", "throughput": 2675.62, "total_tokens": 553984}
10
- {"current_steps": 50, "total_steps": 880, "loss": 0.4873, "lr": 0.00029991503375003, "epoch": 0.22805017103762829, "percentage": 5.68, "elapsed_time": "0:03:49", "remaining_time": "1:03:34", "throughput": 2676.78, "total_tokens": 615040}
11
- {"current_steps": 55, "total_steps": 880, "loss": 0.4971, "lr": 0.000299794430213186, "epoch": 0.2508551881413911, "percentage": 6.25, "elapsed_time": "0:04:12", "remaining_time": "1:03:08", "throughput": 2679.17, "total_tokens": 676736}
12
- {"current_steps": 60, "total_steps": 880, "loss": 0.4761, "lr": 0.0002996214468334339, "epoch": 0.27366020524515394, "percentage": 6.82, "elapsed_time": "0:04:35", "remaining_time": "1:02:44", "throughput": 2680.2, "total_tokens": 738176}
13
- {"current_steps": 65, "total_steps": 880, "loss": 0.4849, "lr": 0.00029939614409928584, "epoch": 0.29646522234891676, "percentage": 7.39, "elapsed_time": "0:04:58", "remaining_time": "1:02:18", "throughput": 2681.16, "total_tokens": 799488}
14
- {"current_steps": 70, "total_steps": 880, "loss": 0.4819, "lr": 0.00029911860079419073, "epoch": 0.31927023945267957, "percentage": 7.95, "elapsed_time": "0:05:21", "remaining_time": "1:01:55", "throughput": 2682.95, "total_tokens": 861568}
15
- {"current_steps": 75, "total_steps": 880, "loss": 0.4812, "lr": 0.00029878891396898494, "epoch": 0.34207525655644244, "percentage": 8.52, "elapsed_time": "0:05:44", "remaining_time": "1:01:38", "throughput": 2683.42, "total_tokens": 924544}
16
- {"current_steps": 80, "total_steps": 880, "loss": 0.4691, "lr": 0.00029840719890795547, "epoch": 0.36488027366020526, "percentage": 9.09, "elapsed_time": "0:06:07", "remaining_time": "1:01:11", "throughput": 2684.07, "total_tokens": 985472}
17
- {"current_steps": 85, "total_steps": 880, "loss": 0.4845, "lr": 0.00029797358908852816, "epoch": 0.38768529076396807, "percentage": 9.66, "elapsed_time": "0:06:30", "remaining_time": "1:00:47", "throughput": 2684.34, "total_tokens": 1046912}
18
- {"current_steps": 90, "total_steps": 880, "loss": 0.482, "lr": 0.00029748823613459316, "epoch": 0.4104903078677309, "percentage": 10.23, "elapsed_time": "0:06:52", "remaining_time": "1:00:24", "throughput": 2685.47, "total_tokens": 1108992}
19
- {"current_steps": 95, "total_steps": 880, "loss": 0.4548, "lr": 0.0002969513097634853, "epoch": 0.43329532497149376, "percentage": 10.8, "elapsed_time": "0:07:15", "remaining_time": "1:00:00", "throughput": 2685.51, "total_tokens": 1170048}
20
- {"current_steps": 100, "total_steps": 880, "loss": 0.4571, "lr": 0.00029636299772663795, "epoch": 0.45610034207525657, "percentage": 11.36, "elapsed_time": "0:07:38", "remaining_time": "0:59:33", "throughput": 2685.69, "total_tokens": 1230464}
21
- {"current_steps": 100, "total_steps": 880, "eval_loss": 0.462704062461853, "epoch": 0.45610034207525657, "percentage": 11.36, "elapsed_time": "0:08:10", "remaining_time": "1:03:45", "throughput": 2509.15, "total_tokens": 1230464}
22
- {"current_steps": 105, "total_steps": 880, "loss": 0.4633, "lr": 0.0002957235057439301, "epoch": 0.4789053591790194, "percentage": 11.93, "elapsed_time": "0:08:33", "remaining_time": "1:03:08", "throughput": 2517.6, "total_tokens": 1292288}
23
- {"current_steps": 110, "total_steps": 880, "loss": 0.474, "lr": 0.0002950330574317509, "epoch": 0.5017103762827823, "percentage": 12.5, "elapsed_time": "0:08:55", "remaining_time": "1:02:31", "throughput": 2525.01, "total_tokens": 1353344}
24
- {"current_steps": 115, "total_steps": 880, "loss": 0.4709, "lr": 0.0002942918942248059, "epoch": 0.5245153933865451, "percentage": 13.07, "elapsed_time": "0:09:18", "remaining_time": "1:01:56", "throughput": 2532.46, "total_tokens": 1415040}
25
- {"current_steps": 120, "total_steps": 880, "loss": 0.4581, "lr": 0.0002935002752916921, "epoch": 0.5473204104903079, "percentage": 13.64, "elapsed_time": "0:09:41", "remaining_time": "1:01:23", "throughput": 2538.96, "total_tokens": 1476480}
26
- {"current_steps": 125, "total_steps": 880, "loss": 0.4428, "lr": 0.00029265847744427303, "epoch": 0.5701254275940707, "percentage": 14.2, "elapsed_time": "0:10:04", "remaining_time": "1:00:50", "throughput": 2544.32, "total_tokens": 1537664}
27
- {"current_steps": 130, "total_steps": 880, "loss": 0.441, "lr": 0.0002917667950408832, "epoch": 0.5929304446978335, "percentage": 14.77, "elapsed_time": "0:10:27", "remaining_time": "1:00:18", "throughput": 2549.51, "total_tokens": 1598848}
28
- {"current_steps": 135, "total_steps": 880, "loss": 0.4484, "lr": 0.0002908255398833979, "epoch": 0.6157354618015963, "percentage": 15.34, "elapsed_time": "0:10:49", "remaining_time": "0:59:47", "throughput": 2555.09, "total_tokens": 1660800}
29
- {"current_steps": 140, "total_steps": 880, "loss": 0.4312, "lr": 0.0002898350411082021, "epoch": 0.6385404789053591, "percentage": 15.91, "elapsed_time": "0:11:12", "remaining_time": "0:59:15", "throughput": 2559.58, "total_tokens": 1721600}
30
- {"current_steps": 145, "total_steps": 880, "loss": 0.4374, "lr": 0.00028879564507109946, "epoch": 0.661345496009122, "percentage": 16.48, "elapsed_time": "0:11:35", "remaining_time": "0:58:45", "throughput": 2564.01, "total_tokens": 1783168}
31
- {"current_steps": 150, "total_steps": 880, "loss": 0.4241, "lr": 0.00028770771522619885, "epoch": 0.6841505131128849, "percentage": 17.05, "elapsed_time": "0:11:58", "remaining_time": "0:58:15", "throughput": 2568.43, "total_tokens": 1844992}
32
- {"current_steps": 155, "total_steps": 880, "loss": 0.4048, "lr": 0.00028657163199882237, "epoch": 0.7069555302166477, "percentage": 17.61, "elapsed_time": "0:12:21", "remaining_time": "0:57:46", "throughput": 2572.06, "total_tokens": 1906048}
33
- {"current_steps": 160, "total_steps": 880, "loss": 0.4175, "lr": 0.0002853877926524791, "epoch": 0.7297605473204105, "percentage": 18.18, "elapsed_time": "0:12:43", "remaining_time": "0:57:17", "throughput": 2575.87, "total_tokens": 1967744}
34
- {"current_steps": 165, "total_steps": 880, "loss": 0.405, "lr": 0.0002841566111499505, "epoch": 0.7525655644241733, "percentage": 18.75, "elapsed_time": "0:13:06", "remaining_time": "0:56:47", "throughput": 2578.85, "total_tokens": 2028032}
35
- {"current_steps": 170, "total_steps": 880, "loss": 0.4023, "lr": 0.00028287851800853697, "epoch": 0.7753705815279361, "percentage": 19.32, "elapsed_time": "0:13:28", "remaining_time": "0:56:18", "throughput": 2581.88, "total_tokens": 2088448}
36
- {"current_steps": 175, "total_steps": 880, "loss": 0.4013, "lr": 0.00028155396014951494, "epoch": 0.798175598631699, "percentage": 19.89, "elapsed_time": "0:13:51", "remaining_time": "0:55:50", "throughput": 2585.24, "total_tokens": 2150144}
37
- {"current_steps": 180, "total_steps": 880, "loss": 0.4065, "lr": 0.0002801834007418587, "epoch": 0.8209806157354618, "percentage": 20.45, "elapsed_time": "0:14:14", "remaining_time": "0:55:23", "throughput": 2588.01, "total_tokens": 2211584}
38
- {"current_steps": 185, "total_steps": 880, "loss": 0.3784, "lr": 0.0002787673190402799, "epoch": 0.8437856328392246, "percentage": 21.02, "elapsed_time": "0:14:37", "remaining_time": "0:54:54", "throughput": 2590.73, "total_tokens": 2272256}
39
- {"current_steps": 190, "total_steps": 880, "loss": 0.3882, "lr": 0.0002773062102176429, "epoch": 0.8665906499429875, "percentage": 21.59, "elapsed_time": "0:14:59", "remaining_time": "0:54:27", "throughput": 2593.36, "total_tokens": 2333696}
40
- {"current_steps": 195, "total_steps": 880, "loss": 0.3953, "lr": 0.0002758005851918136, "epoch": 0.8893956670467503, "percentage": 22.16, "elapsed_time": "0:15:22", "remaining_time": "0:54:00", "throughput": 2595.95, "total_tokens": 2394880}
41
- {"current_steps": 200, "total_steps": 880, "loss": 0.3676, "lr": 0.00027425097044700246, "epoch": 0.9122006841505131, "percentage": 22.73, "elapsed_time": "0:15:45", "remaining_time": "0:53:33", "throughput": 2598.2, "total_tokens": 2455680}
42
- {"current_steps": 200, "total_steps": 880, "eval_loss": 0.3956287205219269, "epoch": 0.9122006841505131, "percentage": 22.73, "elapsed_time": "0:16:17", "remaining_time": "0:55:23", "throughput": 2512.32, "total_tokens": 2455680}
43
- {"current_steps": 205, "total_steps": 880, "loss": 0.3821, "lr": 0.0002726579078496647, "epoch": 0.935005701254276, "percentage": 23.3, "elapsed_time": "0:16:40", "remaining_time": "0:54:53", "throughput": 2516.49, "total_tokens": 2517376}
44
- {"current_steps": 210, "total_steps": 880, "loss": 0.3828, "lr": 0.0002710219544590212, "epoch": 0.9578107183580388, "percentage": 23.86, "elapsed_time": "0:17:03", "remaining_time": "0:54:24", "throughput": 2520.61, "total_tokens": 2578816}
45
- {"current_steps": 215, "total_steps": 880, "loss": 0.3569, "lr": 0.00026934368233226714, "epoch": 0.9806157354618016, "percentage": 24.43, "elapsed_time": "0:17:25", "remaining_time": "0:53:54", "throughput": 2524.31, "total_tokens": 2640128}
46
- {"current_steps": 220, "total_steps": 880, "loss": 0.3597, "lr": 0.00026762367832453617, "epoch": 1.0, "percentage": 25.0, "elapsed_time": "0:17:45", "remaining_time": "0:53:15", "throughput": 2527.29, "total_tokens": 2691984}
47
- {"current_steps": 225, "total_steps": 880, "loss": 0.3621, "lr": 0.0002658625438836899, "epoch": 1.0228050171037628, "percentage": 25.57, "elapsed_time": "0:18:08", "remaining_time": "0:52:47", "throughput": 2530.83, "total_tokens": 2753808}
48
- {"current_steps": 230, "total_steps": 880, "loss": 0.3623, "lr": 0.00026406089484000466, "epoch": 1.0456100342075256, "percentage": 26.14, "elapsed_time": "0:18:30", "remaining_time": "0:52:19", "throughput": 2534.3, "total_tokens": 2815248}
49
- {"current_steps": 235, "total_steps": 880, "loss": 0.3866, "lr": 0.00026221936119082904, "epoch": 1.0684150513112884, "percentage": 26.7, "elapsed_time": "0:18:53", "remaining_time": "0:51:52", "throughput": 2537.46, "total_tokens": 2877456}
50
- {"current_steps": 240, "total_steps": 880, "loss": 0.3598, "lr": 0.00026033858688028784, "epoch": 1.0912200684150513, "percentage": 27.27, "elapsed_time": "0:19:16", "remaining_time": "0:51:25", "throughput": 2540.5, "total_tokens": 2939280}
51
- {"current_steps": 245, "total_steps": 880, "loss": 0.3488, "lr": 0.0002584192295741087, "epoch": 1.114025085518814, "percentage": 27.84, "elapsed_time": "0:19:39", "remaining_time": "0:50:57", "throughput": 2543.27, "total_tokens": 3000336}
52
- {"current_steps": 250, "total_steps": 880, "loss": 0.3516, "lr": 0.0002564619604296506, "epoch": 1.1368301026225769, "percentage": 28.41, "elapsed_time": "0:20:02", "remaining_time": "0:50:30", "throughput": 2546.18, "total_tokens": 3062416}
53
- {"current_steps": 255, "total_steps": 880, "loss": 0.34, "lr": 0.00025446746386121504, "epoch": 1.1596351197263397, "percentage": 28.98, "elapsed_time": "0:20:25", "remaining_time": "0:50:04", "throughput": 2548.93, "total_tokens": 3124368}
54
- {"current_steps": 260, "total_steps": 880, "loss": 0.3361, "lr": 0.000252436437300721, "epoch": 1.1824401368301025, "percentage": 29.55, "elapsed_time": "0:20:48", "remaining_time": "0:49:37", "throughput": 2551.45, "total_tokens": 3185424}
55
- {"current_steps": 265, "total_steps": 880, "loss": 0.3172, "lr": 0.0002503695909538287, "epoch": 1.2052451539338653, "percentage": 30.11, "elapsed_time": "0:21:11", "remaining_time": "0:49:09", "throughput": 2553.91, "total_tokens": 3246096}
56
- {"current_steps": 270, "total_steps": 880, "loss": 0.3387, "lr": 0.00024826764755159566, "epoch": 1.2280501710376284, "percentage": 30.68, "elapsed_time": "0:21:33", "remaining_time": "0:48:43", "throughput": 2556.34, "total_tokens": 3307536}
57
- {"current_steps": 275, "total_steps": 880, "loss": 0.3112, "lr": 0.0002461313420977536, "epoch": 1.2508551881413912, "percentage": 31.25, "elapsed_time": "0:21:56", "remaining_time": "0:48:16", "throughput": 2558.63, "total_tokens": 3368976}
58
- {"current_steps": 280, "total_steps": 880, "loss": 0.3223, "lr": 0.0002439614216116939, "epoch": 1.273660205245154, "percentage": 31.82, "elapsed_time": "0:22:19", "remaining_time": "0:47:49", "throughput": 2560.82, "total_tokens": 3429520}
59
- {"current_steps": 285, "total_steps": 880, "loss": 0.312, "lr": 0.00024175864486725092, "epoch": 1.2964652223489168, "percentage": 32.39, "elapsed_time": "0:22:41", "remaining_time": "0:47:23", "throughput": 2562.88, "total_tokens": 3490448}
60
- {"current_steps": 290, "total_steps": 880, "loss": 0.3096, "lr": 0.00023952378212737554, "epoch": 1.3192702394526796, "percentage": 32.95, "elapsed_time": "0:23:04", "remaining_time": "0:46:56", "throughput": 2564.82, "total_tokens": 3551120}
61
- {"current_steps": 295, "total_steps": 880, "loss": 0.3109, "lr": 0.00023725761487479086, "epoch": 1.3420752565564424, "percentage": 33.52, "elapsed_time": "0:23:27", "remaining_time": "0:46:30", "throughput": 2566.78, "total_tokens": 3611920}
62
- {"current_steps": 300, "total_steps": 880, "loss": 0.3092, "lr": 0.0002349609355387249, "epoch": 1.3648802736602053, "percentage": 34.09, "elapsed_time": "0:23:50", "remaining_time": "0:46:04", "throughput": 2568.98, "total_tokens": 3673744}
63
- {"current_steps": 300, "total_steps": 880, "eval_loss": 0.31190308928489685, "epoch": 1.3648802736602053, "percentage": 34.09, "elapsed_time": "0:24:22", "remaining_time": "0:47:07", "throughput": 2512.21, "total_tokens": 3673744}
64
- {"current_steps": 305, "total_steps": 880, "loss": 0.308, "lr": 0.00023263454721781537, "epoch": 1.387685290763968, "percentage": 34.66, "elapsed_time": "0:24:45", "remaining_time": "0:46:40", "throughput": 2515.09, "total_tokens": 3735568}
65
- {"current_steps": 310, "total_steps": 880, "loss": 0.3089, "lr": 0.0002302792633992835, "epoch": 1.4104903078677309, "percentage": 35.23, "elapsed_time": "0:25:08", "remaining_time": "0:46:13", "throughput": 2517.8, "total_tokens": 3798160}
66
- {"current_steps": 315, "total_steps": 880, "loss": 0.3028, "lr": 0.00022789590767447533, "epoch": 1.4332953249714937, "percentage": 35.8, "elapsed_time": "0:25:31", "remaining_time": "0:45:46", "throughput": 2520.33, "total_tokens": 3859600}
67
- {"current_steps": 320, "total_steps": 880, "loss": 0.2925, "lr": 0.00022548531345087, "epoch": 1.4561003420752565, "percentage": 36.36, "elapsed_time": "0:25:53", "remaining_time": "0:45:19", "throughput": 2522.88, "total_tokens": 3920528}
68
- {"current_steps": 325, "total_steps": 880, "loss": 0.285, "lr": 0.00022304832366065505, "epoch": 1.4789053591790193, "percentage": 36.93, "elapsed_time": "0:26:16", "remaining_time": "0:44:52", "throughput": 2525.37, "total_tokens": 3982096}
69
- {"current_steps": 330, "total_steps": 880, "loss": 0.304, "lr": 0.0002205857904659717, "epoch": 1.5017103762827824, "percentage": 37.5, "elapsed_time": "0:26:39", "remaining_time": "0:44:26", "throughput": 2527.73, "total_tokens": 4044048}
70
- {"current_steps": 335, "total_steps": 880, "loss": 0.2892, "lr": 0.00021809857496093199, "epoch": 1.5245153933865452, "percentage": 38.07, "elapsed_time": "0:27:02", "remaining_time": "0:43:59", "throughput": 2530.08, "total_tokens": 4105104}
71
- {"current_steps": 340, "total_steps": 880, "loss": 0.2849, "lr": 0.00021558754687051344, "epoch": 1.547320410490308, "percentage": 38.64, "elapsed_time": "0:27:25", "remaining_time": "0:43:32", "throughput": 2532.44, "total_tokens": 4165904}
72
- {"current_steps": 345, "total_steps": 880, "loss": 0.2975, "lr": 0.0002130535842464348, "epoch": 1.5701254275940708, "percentage": 39.2, "elapsed_time": "0:27:47", "remaining_time": "0:43:06", "throughput": 2534.53, "total_tokens": 4226704}
73
- {"current_steps": 350, "total_steps": 880, "loss": 0.2845, "lr": 0.00021049757316012077, "epoch": 1.5929304446978336, "percentage": 39.77, "elapsed_time": "0:28:10", "remaining_time": "0:42:39", "throughput": 2536.78, "total_tokens": 4288400}
74
- {"current_steps": 355, "total_steps": 880, "loss": 0.2869, "lr": 0.0002079204073928619, "epoch": 1.6157354618015964, "percentage": 40.34, "elapsed_time": "0:28:33", "remaining_time": "0:42:13", "throughput": 2538.9, "total_tokens": 4349712}
75
- {"current_steps": 360, "total_steps": 880, "loss": 0.2601, "lr": 0.0002053229881232793, "epoch": 1.6385404789053593, "percentage": 40.91, "elapsed_time": "0:28:55", "remaining_time": "0:41:47", "throughput": 2540.88, "total_tokens": 4410256}
76
- {"current_steps": 365, "total_steps": 880, "loss": 0.2826, "lr": 0.0002027062236122014, "epoch": 1.661345496009122, "percentage": 41.48, "elapsed_time": "0:29:18", "remaining_time": "0:41:20", "throughput": 2542.97, "total_tokens": 4471440}
77
- {"current_steps": 370, "total_steps": 880, "loss": 0.2729, "lr": 0.0002000710288850656, "epoch": 1.6841505131128849, "percentage": 42.05, "elapsed_time": "0:29:41", "remaining_time": "0:40:55", "throughput": 2544.98, "total_tokens": 4533008}
78
- {"current_steps": 375, "total_steps": 880, "loss": 0.2772, "lr": 0.00019741832541195366, "epoch": 1.7069555302166477, "percentage": 42.61, "elapsed_time": "0:30:03", "remaining_time": "0:40:29", "throughput": 2546.98, "total_tokens": 4594320}
79
- {"current_steps": 380, "total_steps": 880, "loss": 0.2789, "lr": 0.0001947490407853734, "epoch": 1.7297605473204105, "percentage": 43.18, "elapsed_time": "0:30:26", "remaining_time": "0:40:03", "throughput": 2548.91, "total_tokens": 4655760}
80
- {"current_steps": 385, "total_steps": 880, "loss": 0.2743, "lr": 0.0001920641083959004, "epoch": 1.7525655644241733, "percentage": 43.75, "elapsed_time": "0:30:49", "remaining_time": "0:39:37", "throughput": 2550.81, "total_tokens": 4717328}
81
- {"current_steps": 390, "total_steps": 880, "loss": 0.2657, "lr": 0.00018936446710579122, "epoch": 1.7753705815279361, "percentage": 44.32, "elapsed_time": "0:31:12", "remaining_time": "0:39:12", "throughput": 2552.74, "total_tokens": 4779152}
82
- {"current_steps": 395, "total_steps": 880, "loss": 0.2847, "lr": 0.00018665106092068408, "epoch": 1.798175598631699, "percentage": 44.89, "elapsed_time": "0:31:34", "remaining_time": "0:38:46", "throughput": 2554.53, "total_tokens": 4840336}
83
- {"current_steps": 400, "total_steps": 880, "loss": 0.2796, "lr": 0.00018392483865950046, "epoch": 1.8209806157354618, "percentage": 45.45, "elapsed_time": "0:31:57", "remaining_time": "0:38:20", "throughput": 2556.32, "total_tokens": 4901648}
84
- {"current_steps": 400, "total_steps": 880, "eval_loss": 0.2729249596595764, "epoch": 1.8209806157354618, "percentage": 45.45, "elapsed_time": "0:32:29", "remaining_time": "0:38:59", "throughput": 2514.03, "total_tokens": 4901648}
85
- {"current_steps": 405, "total_steps": 880, "loss": 0.2725, "lr": 0.00018118675362266385, "epoch": 1.8437856328392246, "percentage": 46.02, "elapsed_time": "0:32:52", "remaining_time": "0:38:33", "throughput": 2516.19, "total_tokens": 4963216}
86
- {"current_steps": 410, "total_steps": 880, "loss": 0.2714, "lr": 0.00017843776325875176, "epoch": 1.8665906499429874, "percentage": 46.59, "elapsed_time": "0:33:15", "remaining_time": "0:38:07", "throughput": 2518.33, "total_tokens": 5024912}
87
- {"current_steps": 415, "total_steps": 880, "loss": 0.2547, "lr": 0.00017567882882969628, "epoch": 1.8893956670467502, "percentage": 47.16, "elapsed_time": "0:33:38", "remaining_time": "0:37:41", "throughput": 2520.24, "total_tokens": 5086224}
88
- {"current_steps": 420, "total_steps": 880, "loss": 0.2687, "lr": 0.0001729109150746521, "epoch": 1.912200684150513, "percentage": 47.73, "elapsed_time": "0:34:01", "remaining_time": "0:37:15", "throughput": 2522.2, "total_tokens": 5148048}
89
- {"current_steps": 425, "total_steps": 880, "loss": 0.2644, "lr": 0.00017013498987264832, "epoch": 1.9350057012542758, "percentage": 48.3, "elapsed_time": "0:34:23", "remaining_time": "0:36:49", "throughput": 2524.15, "total_tokens": 5209104}
90
- {"current_steps": 430, "total_steps": 880, "loss": 0.2696, "lr": 0.00016735202390414215, "epoch": 1.9578107183580387, "percentage": 48.86, "elapsed_time": "0:34:46", "remaining_time": "0:36:23", "throughput": 2526.03, "total_tokens": 5270544}
91
- {"current_steps": 435, "total_steps": 880, "loss": 0.2694, "lr": 0.00016456299031159322, "epoch": 1.9806157354618015, "percentage": 49.43, "elapsed_time": "0:35:09", "remaining_time": "0:35:57", "throughput": 2527.81, "total_tokens": 5331472}
92
- {"current_steps": 440, "total_steps": 880, "loss": 0.2495, "lr": 0.00016176886435917675, "epoch": 2.0, "percentage": 50.0, "elapsed_time": "0:35:28", "remaining_time": "0:35:28", "throughput": 2529.09, "total_tokens": 5382864}
93
- {"current_steps": 445, "total_steps": 880, "loss": 0.2453, "lr": 0.00015897062309175512, "epoch": 2.022805017103763, "percentage": 50.57, "elapsed_time": "0:35:51", "remaining_time": "0:35:02", "throughput": 2530.83, "total_tokens": 5444176}
94
- {"current_steps": 450, "total_steps": 880, "loss": 0.2299, "lr": 0.0001561692449932266, "epoch": 2.0456100342075256, "percentage": 51.14, "elapsed_time": "0:36:13", "remaining_time": "0:34:37", "throughput": 2532.6, "total_tokens": 5505872}
95
- {"current_steps": 455, "total_steps": 880, "loss": 0.2301, "lr": 0.00015336570964437075, "epoch": 2.0684150513112884, "percentage": 51.7, "elapsed_time": "0:36:37", "remaining_time": "0:34:12", "throughput": 2534.34, "total_tokens": 5568208}
96
- {"current_steps": 460, "total_steps": 880, "loss": 0.2376, "lr": 0.0001505609973803105, "epoch": 2.0912200684150513, "percentage": 52.27, "elapsed_time": "0:36:59", "remaining_time": "0:33:46", "throughput": 2535.86, "total_tokens": 5628880}
97
- {"current_steps": 465, "total_steps": 880, "loss": 0.2199, "lr": 0.00014775608894771047, "epoch": 2.114025085518814, "percentage": 52.84, "elapsed_time": "0:37:22", "remaining_time": "0:33:21", "throughput": 2537.51, "total_tokens": 5690320}
98
- {"current_steps": 470, "total_steps": 880, "loss": 0.2237, "lr": 0.00014495196516183095, "epoch": 2.136830102622577, "percentage": 53.41, "elapsed_time": "0:37:45", "remaining_time": "0:32:56", "throughput": 2539.15, "total_tokens": 5752912}
99
- {"current_steps": 475, "total_steps": 880, "loss": 0.2294, "lr": 0.0001421496065635584, "epoch": 2.1596351197263397, "percentage": 53.98, "elapsed_time": "0:38:08", "remaining_time": "0:32:31", "throughput": 2540.66, "total_tokens": 5813968}
100
- {"current_steps": 480, "total_steps": 880, "loss": 0.231, "lr": 0.00013934999307653193, "epoch": 2.1824401368301025, "percentage": 54.55, "elapsed_time": "0:38:30", "remaining_time": "0:32:05", "throughput": 2542.21, "total_tokens": 5874896}
101
- {"current_steps": 485, "total_steps": 880, "loss": 0.2252, "lr": 0.00013655410366448498, "epoch": 2.2052451539338653, "percentage": 55.11, "elapsed_time": "0:38:53", "remaining_time": "0:31:40", "throughput": 2543.74, "total_tokens": 5935952}
102
- {"current_steps": 490, "total_steps": 880, "loss": 0.2151, "lr": 0.0001337629159889232, "epoch": 2.228050171037628, "percentage": 55.68, "elapsed_time": "0:39:16", "remaining_time": "0:31:15", "throughput": 2545.24, "total_tokens": 5997008}
103
- {"current_steps": 495, "total_steps": 880, "loss": 0.2228, "lr": 0.00013097740606725776, "epoch": 2.250855188141391, "percentage": 56.25, "elapsed_time": "0:39:39", "remaining_time": "0:30:50", "throughput": 2546.67, "total_tokens": 6058704}
104
- {"current_steps": 500, "total_steps": 880, "loss": 0.2265, "lr": 0.00012819854793151312, "epoch": 2.2736602052451538, "percentage": 56.82, "elapsed_time": "0:40:01", "remaining_time": "0:30:25", "throughput": 2548.01, "total_tokens": 6120272}
105
- {"current_steps": 500, "total_steps": 880, "eval_loss": 0.24168384075164795, "epoch": 2.2736602052451538, "percentage": 56.82, "elapsed_time": "0:40:34", "remaining_time": "0:30:50", "throughput": 2514.23, "total_tokens": 6120272}
106
- {"current_steps": 505, "total_steps": 880, "loss": 0.2126, "lr": 0.00012542731328772934, "epoch": 2.2964652223489166, "percentage": 57.39, "elapsed_time": "0:41:02", "remaining_time": "0:30:28", "throughput": 2509.48, "total_tokens": 6180816}
107
- {"current_steps": 510, "total_steps": 880, "loss": 0.2202, "lr": 0.00012266467117617787, "epoch": 2.3192702394526794, "percentage": 57.95, "elapsed_time": "0:41:25", "remaining_time": "0:30:03", "throughput": 2511.22, "total_tokens": 6242128}
108
- {"current_steps": 515, "total_steps": 880, "loss": 0.2203, "lr": 0.00011991158763250908, "epoch": 2.342075256556442, "percentage": 58.52, "elapsed_time": "0:41:48", "remaining_time": "0:29:37", "throughput": 2512.89, "total_tokens": 6303312}
109
- {"current_steps": 520, "total_steps": 880, "loss": 0.223, "lr": 0.00011716902534995079, "epoch": 2.364880273660205, "percentage": 59.09, "elapsed_time": "0:42:11", "remaining_time": "0:29:12", "throughput": 2514.53, "total_tokens": 6364624}
110
- {"current_steps": 525, "total_steps": 880, "loss": 0.2218, "lr": 0.00011443794334267538, "epoch": 2.387685290763968, "percentage": 59.66, "elapsed_time": "0:42:33", "remaining_time": "0:28:46", "throughput": 2516.13, "total_tokens": 6425040}
111
- {"current_steps": 530, "total_steps": 880, "loss": 0.2156, "lr": 0.0001117192966104536, "epoch": 2.4104903078677307, "percentage": 60.23, "elapsed_time": "0:42:56", "remaining_time": "0:28:21", "throughput": 2517.81, "total_tokens": 6486736}
112
- {"current_steps": 535, "total_steps": 880, "loss": 0.2161, "lr": 0.000109014035804712, "epoch": 2.433295324971494, "percentage": 60.8, "elapsed_time": "0:43:19", "remaining_time": "0:27:56", "throughput": 2519.4, "total_tokens": 6548304}
113
- {"current_steps": 540, "total_steps": 880, "loss": 0.2045, "lr": 0.0001063231068961106, "epoch": 2.4561003420752567, "percentage": 61.36, "elapsed_time": "0:43:42", "remaining_time": "0:27:30", "throughput": 2520.94, "total_tokens": 6610000}
114
- {"current_steps": 545, "total_steps": 880, "loss": 0.2073, "lr": 0.0001036474508437579, "epoch": 2.4789053591790196, "percentage": 61.93, "elapsed_time": "0:44:04", "remaining_time": "0:27:05", "throughput": 2522.38, "total_tokens": 6670672}
115
- {"current_steps": 550, "total_steps": 880, "loss": 0.2018, "lr": 0.00010098800326617791, "epoch": 2.5017103762827824, "percentage": 62.5, "elapsed_time": "0:44:27", "remaining_time": "0:26:40", "throughput": 2523.89, "total_tokens": 6732112}
116
- {"current_steps": 555, "total_steps": 880, "loss": 0.2048, "lr": 9.834569411414471e-05, "epoch": 2.524515393386545, "percentage": 63.07, "elapsed_time": "0:44:50", "remaining_time": "0:26:15", "throughput": 2525.4, "total_tokens": 6793424}
117
- {"current_steps": 560, "total_steps": 880, "loss": 0.2062, "lr": 9.572144734549919e-05, "epoch": 2.547320410490308, "percentage": 63.64, "elapsed_time": "0:45:13", "remaining_time": "0:25:50", "throughput": 2526.96, "total_tokens": 6855760}
118
- {"current_steps": 565, "total_steps": 880, "loss": 0.216, "lr": 9.311618060206074e-05, "epoch": 2.570125427594071, "percentage": 64.2, "elapsed_time": "0:45:35", "remaining_time": "0:25:25", "throughput": 2528.28, "total_tokens": 6916688}
119
- {"current_steps": 570, "total_steps": 880, "loss": 0.1953, "lr": 9.053080488874836e-05, "epoch": 2.5929304446978336, "percentage": 64.77, "elapsed_time": "0:45:58", "remaining_time": "0:25:00", "throughput": 2529.73, "total_tokens": 6978512}
120
- {"current_steps": 575, "total_steps": 880, "loss": 0.2147, "lr": 8.796622425502192e-05, "epoch": 2.6157354618015964, "percentage": 65.34, "elapsed_time": "0:46:21", "remaining_time": "0:24:35", "throughput": 2531.03, "total_tokens": 7040208}
121
- {"current_steps": 580, "total_steps": 880, "loss": 0.209, "lr": 8.54233354787557e-05, "epoch": 2.6385404789053593, "percentage": 65.91, "elapsed_time": "0:46:44", "remaining_time": "0:24:10", "throughput": 2532.36, "total_tokens": 7101264}
122
- {"current_steps": 585, "total_steps": 880, "loss": 0.2077, "lr": 8.290302775265509e-05, "epoch": 2.661345496009122, "percentage": 66.48, "elapsed_time": "0:47:07", "remaining_time": "0:23:45", "throughput": 2533.77, "total_tokens": 7163600}
123
- {"current_steps": 590, "total_steps": 880, "loss": 0.2058, "lr": 8.04061823733249e-05, "epoch": 2.684150513112885, "percentage": 67.05, "elapsed_time": "0:47:30", "remaining_time": "0:23:20", "throughput": 2535.21, "total_tokens": 7225936}
124
- {"current_steps": 595, "total_steps": 880, "loss": 0.1961, "lr": 7.79336724330994e-05, "epoch": 2.7069555302166477, "percentage": 67.61, "elapsed_time": "0:47:53", "remaining_time": "0:22:56", "throughput": 2536.44, "total_tokens": 7287760}
125
- {"current_steps": 600, "total_steps": 880, "loss": 0.2003, "lr": 7.548636251474091e-05, "epoch": 2.7297605473204105, "percentage": 68.18, "elapsed_time": "0:48:15", "remaining_time": "0:22:31", "throughput": 2537.64, "total_tokens": 7348944}
126
- {"current_steps": 600, "total_steps": 880, "eval_loss": 0.22281652688980103, "epoch": 2.7297605473204105, "percentage": 68.18, "elapsed_time": "0:48:48", "remaining_time": "0:22:46", "throughput": 2509.68, "total_tokens": 7348944}
127
- {"current_steps": 605, "total_steps": 880, "loss": 0.2058, "lr": 7.30651083891141e-05, "epoch": 2.7525655644241733, "percentage": 68.75, "elapsed_time": "0:49:11", "remaining_time": "0:22:21", "throughput": 2511.21, "total_tokens": 7410640}
128
- {"current_steps": 610, "total_steps": 880, "loss": 0.1909, "lr": 7.067075671594168e-05, "epoch": 2.775370581527936, "percentage": 69.32, "elapsed_time": "0:49:33", "remaining_time": "0:21:56", "throughput": 2512.59, "total_tokens": 7471184}
129
- {"current_steps": 615, "total_steps": 880, "loss": 0.2035, "lr": 6.830414474774594e-05, "epoch": 2.798175598631699, "percentage": 69.89, "elapsed_time": "0:49:56", "remaining_time": "0:21:31", "throughput": 2514.06, "total_tokens": 7533392}
130
- {"current_steps": 620, "total_steps": 880, "loss": 0.1963, "lr": 6.596610003707958e-05, "epoch": 2.8209806157354618, "percentage": 70.45, "elapsed_time": "0:50:19", "remaining_time": "0:21:06", "throughput": 2515.46, "total_tokens": 7594832}
131
- {"current_steps": 625, "total_steps": 880, "loss": 0.1951, "lr": 6.365744014714898e-05, "epoch": 2.8437856328392246, "percentage": 71.02, "elapsed_time": "0:50:42", "remaining_time": "0:20:41", "throughput": 2516.85, "total_tokens": 7656656}
132
- {"current_steps": 630, "total_steps": 880, "loss": 0.193, "lr": 6.137897236593002e-05, "epoch": 2.8665906499429874, "percentage": 71.59, "elapsed_time": "0:51:04", "remaining_time": "0:20:16", "throughput": 2518.16, "total_tokens": 7717712}
133
- {"current_steps": 635, "total_steps": 880, "loss": 0.1925, "lr": 5.913149342387703e-05, "epoch": 2.88939566704675, "percentage": 72.16, "elapsed_time": "0:51:27", "remaining_time": "0:19:51", "throughput": 2519.53, "total_tokens": 7779920}
134
- {"current_steps": 640, "total_steps": 880, "loss": 0.1995, "lr": 5.6915789215323817e-05, "epoch": 2.912200684150513, "percentage": 72.73, "elapsed_time": "0:51:50", "remaining_time": "0:19:26", "throughput": 2520.84, "total_tokens": 7840976}
135
- {"current_steps": 645, "total_steps": 880, "loss": 0.1852, "lr": 5.473263452367318e-05, "epoch": 2.935005701254276, "percentage": 73.3, "elapsed_time": "0:52:13", "remaining_time": "0:19:01", "throughput": 2522.07, "total_tokens": 7902160}
136
- {"current_steps": 650, "total_steps": 880, "loss": 0.1878, "lr": 5.2582792750472464e-05, "epoch": 2.9578107183580387, "percentage": 73.86, "elapsed_time": "0:52:35", "remaining_time": "0:18:36", "throughput": 2523.41, "total_tokens": 7963856}
137
- {"current_steps": 655, "total_steps": 880, "loss": 0.1945, "lr": 5.046701564846834e-05, "epoch": 2.9806157354618015, "percentage": 74.43, "elapsed_time": "0:52:59", "remaining_time": "0:18:12", "throughput": 2524.69, "total_tokens": 8026448}
138
- {"current_steps": 660, "total_steps": 880, "loss": 0.1645, "lr": 4.8386043058735465e-05, "epoch": 3.0, "percentage": 75.0, "elapsed_time": "0:53:18", "remaining_time": "0:17:46", "throughput": 2525.72, "total_tokens": 8078224}
139
- {"current_steps": 665, "total_steps": 880, "loss": 0.1556, "lr": 4.63406026519703e-05, "epoch": 3.022805017103763, "percentage": 75.57, "elapsed_time": "0:53:41", "remaining_time": "0:17:21", "throughput": 2527.0, "total_tokens": 8140176}
140
- {"current_steps": 670, "total_steps": 880, "loss": 0.1397, "lr": 4.4331409674040324e-05, "epoch": 3.0456100342075256, "percentage": 76.14, "elapsed_time": "0:54:04", "remaining_time": "0:16:56", "throughput": 2528.18, "total_tokens": 8201616}
141
- {"current_steps": 675, "total_steps": 880, "loss": 0.1478, "lr": 4.23591666958782e-05, "epoch": 3.0684150513112884, "percentage": 76.7, "elapsed_time": "0:54:26", "remaining_time": "0:16:32", "throughput": 2529.34, "total_tokens": 8262928}
142
- {"current_steps": 680, "total_steps": 880, "loss": 0.1413, "lr": 4.042456336780837e-05, "epoch": 3.0912200684150513, "percentage": 77.27, "elapsed_time": "0:54:49", "remaining_time": "0:16:07", "throughput": 2530.57, "total_tokens": 8324496}
143
- {"current_steps": 685, "total_steps": 880, "loss": 0.1494, "lr": 3.852827617839084e-05, "epoch": 3.114025085518814, "percentage": 77.84, "elapsed_time": "0:55:12", "remaining_time": "0:15:42", "throughput": 2531.65, "total_tokens": 8385680}
144
- {"current_steps": 690, "total_steps": 880, "loss": 0.1426, "lr": 3.667096821786847e-05, "epoch": 3.136830102622577, "percentage": 78.41, "elapsed_time": "0:55:35", "remaining_time": "0:15:18", "throughput": 2532.72, "total_tokens": 8447632}
145
- {"current_steps": 695, "total_steps": 880, "loss": 0.1413, "lr": 3.485328894629833e-05, "epoch": 3.1596351197263397, "percentage": 78.98, "elapsed_time": "0:55:57", "remaining_time": "0:14:53", "throughput": 2533.81, "total_tokens": 8508432}
146
- {"current_steps": 700, "total_steps": 880, "loss": 0.1439, "lr": 3.307587396644996e-05, "epoch": 3.1824401368301025, "percentage": 79.55, "elapsed_time": "0:56:20", "remaining_time": "0:14:29", "throughput": 2534.97, "total_tokens": 8570128}
147
- {"current_steps": 700, "total_steps": 880, "eval_loss": 0.23035594820976257, "epoch": 3.1824401368301025, "percentage": 79.55, "elapsed_time": "0:56:53", "remaining_time": "0:14:37", "throughput": 2511.01, "total_tokens": 8570128}
148
- {"current_steps": 705, "total_steps": 880, "loss": 0.1474, "lr": 3.133934480154885e-05, "epoch": 3.2052451539338653, "percentage": 80.11, "elapsed_time": "0:57:15", "remaining_time": "0:14:12", "throughput": 2512.29, "total_tokens": 8631952}
149
- {"current_steps": 710, "total_steps": 880, "loss": 0.1453, "lr": 2.9644308677943313e-05, "epoch": 3.228050171037628, "percentage": 80.68, "elapsed_time": "0:57:38", "remaining_time": "0:13:48", "throughput": 2513.6, "total_tokens": 8694032}
150
- {"current_steps": 715, "total_steps": 880, "loss": 0.1379, "lr": 2.7991358312770805e-05, "epoch": 3.250855188141391, "percentage": 81.25, "elapsed_time": "0:58:01", "remaining_time": "0:13:23", "throughput": 2514.74, "total_tokens": 8754576}
151
- {"current_steps": 720, "total_steps": 880, "loss": 0.1441, "lr": 2.6381071706697644e-05, "epoch": 3.2736602052451538, "percentage": 81.82, "elapsed_time": "0:58:24", "remaining_time": "0:12:58", "throughput": 2515.91, "total_tokens": 8816528}
152
- {"current_steps": 725, "total_steps": 880, "loss": 0.1442, "lr": 2.48140119418046e-05, "epoch": 3.2964652223489166, "percentage": 82.39, "elapsed_time": "0:58:46", "remaining_time": "0:12:34", "throughput": 2517.04, "total_tokens": 8877200}
153
- {"current_steps": 730, "total_steps": 880, "loss": 0.139, "lr": 2.329072698468963e-05, "epoch": 3.3192702394526794, "percentage": 82.95, "elapsed_time": "0:59:09", "remaining_time": "0:12:09", "throughput": 2518.23, "total_tokens": 8938512}
154
- {"current_steps": 735, "total_steps": 880, "loss": 0.1429, "lr": 2.1811749494855778e-05, "epoch": 3.342075256556442, "percentage": 83.52, "elapsed_time": "0:59:32", "remaining_time": "0:11:44", "throughput": 2519.37, "total_tokens": 9000464}
155
- {"current_steps": 740, "total_steps": 880, "loss": 0.1452, "lr": 2.037759663845181e-05, "epoch": 3.364880273660205, "percentage": 84.09, "elapsed_time": "0:59:55", "remaining_time": "0:11:20", "throughput": 2520.49, "total_tokens": 9061648}
156
- {"current_steps": 745, "total_steps": 880, "loss": 0.1392, "lr": 1.8988769907430552e-05, "epoch": 3.387685290763968, "percentage": 84.66, "elapsed_time": "1:00:17", "remaining_time": "0:10:55", "throughput": 2521.6, "total_tokens": 9122704}
157
- {"current_steps": 750, "total_steps": 880, "loss": 0.1408, "lr": 1.7645754944187983e-05, "epoch": 3.4104903078677307, "percentage": 85.23, "elapsed_time": "1:00:40", "remaining_time": "0:10:30", "throughput": 2522.68, "total_tokens": 9183504}
158
- {"current_steps": 755, "total_steps": 880, "loss": 0.1406, "lr": 1.634902137174483e-05, "epoch": 3.433295324971494, "percentage": 85.8, "elapsed_time": "1:01:03", "remaining_time": "0:10:06", "throughput": 2523.73, "total_tokens": 9245072}
159
- {"current_steps": 760, "total_steps": 880, "loss": 0.1395, "lr": 1.5099022629529495e-05, "epoch": 3.4561003420752567, "percentage": 86.36, "elapsed_time": "1:01:26", "remaining_time": "0:09:42", "throughput": 2524.84, "total_tokens": 9307152}
160
- {"current_steps": 765, "total_steps": 880, "loss": 0.137, "lr": 1.3896195814820266e-05, "epoch": 3.4789053591790196, "percentage": 86.93, "elapsed_time": "1:01:49", "remaining_time": "0:09:17", "throughput": 2526.02, "total_tokens": 9369616}
161
- {"current_steps": 770, "total_steps": 880, "loss": 0.1337, "lr": 1.2740961529902027e-05, "epoch": 3.5017103762827824, "percentage": 87.5, "elapsed_time": "1:02:12", "remaining_time": "0:08:53", "throughput": 2527.07, "total_tokens": 9431568}
162
- {"current_steps": 775, "total_steps": 880, "loss": 0.1384, "lr": 1.1633723734990624e-05, "epoch": 3.524515393386545, "percentage": 88.07, "elapsed_time": "1:02:34", "remaining_time": "0:08:28", "throughput": 2528.19, "total_tokens": 9493264}
163
- {"current_steps": 780, "total_steps": 880, "loss": 0.1345, "lr": 1.0574869606976888e-05, "epoch": 3.547320410490308, "percentage": 88.64, "elapsed_time": "1:02:57", "remaining_time": "0:08:04", "throughput": 2529.21, "total_tokens": 9554448}
164
- {"current_steps": 785, "total_steps": 880, "loss": 0.1368, "lr": 9.564769404039419e-06, "epoch": 3.570125427594071, "percentage": 89.2, "elapsed_time": "1:03:20", "remaining_time": "0:07:39", "throughput": 2530.21, "total_tokens": 9615760}
165
- {"current_steps": 790, "total_steps": 880, "loss": 0.1452, "lr": 8.603776336173235e-06, "epoch": 3.5929304446978336, "percentage": 89.77, "elapsed_time": "1:03:43", "remaining_time": "0:07:15", "throughput": 2531.34, "total_tokens": 9678608}
166
- {"current_steps": 795, "total_steps": 880, "loss": 0.1413, "lr": 7.692226441680038e-06, "epoch": 3.6157354618015964, "percentage": 90.34, "elapsed_time": "1:04:06", "remaining_time": "0:06:51", "throughput": 2532.36, "total_tokens": 9740304}
167
- {"current_steps": 800, "total_steps": 880, "loss": 0.1356, "lr": 6.830438469662891e-06, "epoch": 3.6385404789053593, "percentage": 90.91, "elapsed_time": "1:04:29", "remaining_time": "0:06:26", "throughput": 2533.37, "total_tokens": 9801744}
168
- {"current_steps": 800, "total_steps": 880, "eval_loss": 0.22199535369873047, "epoch": 3.6385404789053593, "percentage": 90.91, "elapsed_time": "1:05:01", "remaining_time": "0:06:30", "throughput": 2512.41, "total_tokens": 9801744}
169
- {"current_steps": 805, "total_steps": 880, "loss": 0.1355, "lr": 6.018713768566658e-06, "epoch": 3.661345496009122, "percentage": 91.48, "elapsed_time": "1:05:24", "remaining_time": "0:06:05", "throughput": 2513.43, "total_tokens": 9862928}
170
- {"current_steps": 810, "total_steps": 880, "loss": 0.1376, "lr": 5.257336180803029e-06, "epoch": 3.684150513112885, "percentage": 92.05, "elapsed_time": "1:05:46", "remaining_time": "0:05:41", "throughput": 2514.47, "total_tokens": 9924112}
171
- {"current_steps": 815, "total_steps": 880, "loss": 0.1303, "lr": 4.546571943496968e-06, "epoch": 3.7069555302166477, "percentage": 92.61, "elapsed_time": "1:06:09", "remaining_time": "0:05:16", "throughput": 2515.44, "total_tokens": 9984784}
172
- {"current_steps": 820, "total_steps": 880, "loss": 0.1405, "lr": 3.88666959538953e-06, "epoch": 3.7297605473204105, "percentage": 93.18, "elapsed_time": "1:06:32", "remaining_time": "0:04:52", "throughput": 2516.55, "total_tokens": 10046736}
173
- {"current_steps": 825, "total_steps": 880, "loss": 0.132, "lr": 3.2778598899291465e-06, "epoch": 3.7525655644241733, "percentage": 93.75, "elapsed_time": "1:06:55", "remaining_time": "0:04:27", "throughput": 2517.59, "total_tokens": 10108176}
174
- {"current_steps": 830, "total_steps": 880, "loss": 0.1326, "lr": 2.7203557145822398e-06, "epoch": 3.775370581527936, "percentage": 94.32, "elapsed_time": "1:07:17", "remaining_time": "0:04:03", "throughput": 2518.54, "total_tokens": 10169744}
175
- {"current_steps": 835, "total_steps": 880, "loss": 0.1268, "lr": 2.2143520163911187e-06, "epoch": 3.798175598631699, "percentage": 94.89, "elapsed_time": "1:07:40", "remaining_time": "0:03:38", "throughput": 2519.61, "total_tokens": 10231696}
176
- {"current_steps": 840, "total_steps": 880, "loss": 0.1341, "lr": 1.7600257338052593e-06, "epoch": 3.8209806157354618, "percentage": 95.45, "elapsed_time": "1:08:03", "remaining_time": "0:03:14", "throughput": 2520.55, "total_tokens": 10292624}
177
- {"current_steps": 845, "total_steps": 880, "loss": 0.1366, "lr": 1.3575357348097948e-06, "epoch": 3.8437856328392246, "percentage": 96.02, "elapsed_time": "1:08:25", "remaining_time": "0:02:50", "throughput": 2521.47, "total_tokens": 10353040}
178
- {"current_steps": 850, "total_steps": 880, "loss": 0.1317, "lr": 1.0070227613728221e-06, "epoch": 3.8665906499429874, "percentage": 96.59, "elapsed_time": "1:08:48", "remaining_time": "0:02:25", "throughput": 2522.44, "total_tokens": 10414480}
179
- {"current_steps": 855, "total_steps": 880, "loss": 0.1312, "lr": 7.086093802310344e-07, "epoch": 3.88939566704675, "percentage": 97.16, "elapsed_time": "1:09:11", "remaining_time": "0:02:01", "throughput": 2523.32, "total_tokens": 10476432}
180
- {"current_steps": 860, "total_steps": 880, "loss": 0.1286, "lr": 4.623999400308054e-07, "epoch": 3.912200684150513, "percentage": 97.73, "elapsed_time": "1:09:34", "remaining_time": "0:01:37", "throughput": 2524.29, "total_tokens": 10538128}
181
- {"current_steps": 865, "total_steps": 880, "loss": 0.1279, "lr": 2.6848053483972677e-07, "epoch": 3.935005701254276, "percentage": 98.3, "elapsed_time": "1:09:57", "remaining_time": "0:01:12", "throughput": 2525.29, "total_tokens": 10600080}
182
- {"current_steps": 870, "total_steps": 880, "loss": 0.1374, "lr": 1.2691897404140671e-07, "epoch": 3.9578107183580387, "percentage": 98.86, "elapsed_time": "1:10:20", "remaining_time": "0:00:48", "throughput": 2526.16, "total_tokens": 10660752}
183
- {"current_steps": 875, "total_steps": 880, "loss": 0.1347, "lr": 3.77647586240204e-08, "epoch": 3.9806157354618015, "percentage": 99.43, "elapsed_time": "1:10:42", "remaining_time": "0:00:24", "throughput": 2527.09, "total_tokens": 10721680}
184
- {"current_steps": 880, "total_steps": 880, "loss": 0.1323, "lr": 1.0490638708393616e-09, "epoch": 4.0, "percentage": 100.0, "elapsed_time": "1:11:01", "remaining_time": "0:00:00", "throughput": 2527.8, "total_tokens": 10772736}
185
- {"current_steps": 880, "total_steps": 880, "epoch": 4.0, "percentage": 100.0, "elapsed_time": "1:11:09", "remaining_time": "0:00:00", "throughput": 2523.18, "total_tokens": 10772736}
 
1
+ {"current_steps": 5, "total_steps": 220, "loss": 12.6644, "lr": 2.9999999999999997e-05, "epoch": 0.02280501710376283, "percentage": 2.27, "elapsed_time": "0:00:24", "remaining_time": "0:17:28", "throughput": 2531.08, "total_tokens": 61696}
2
+ {"current_steps": 10, "total_steps": 220, "loss": 5.0641, "lr": 6.75e-05, "epoch": 0.04561003420752566, "percentage": 4.55, "elapsed_time": "0:00:47", "remaining_time": "0:16:31", "throughput": 2607.85, "total_tokens": 123136}
3
+ {"current_steps": 15, "total_steps": 220, "loss": 1.5053, "lr": 0.00010499999999999999, "epoch": 0.06841505131128849, "percentage": 6.82, "elapsed_time": "0:01:10", "remaining_time": "0:15:59", "throughput": 2632.04, "total_tokens": 184832}
4
+ {"current_steps": 20, "total_steps": 220, "loss": 0.6917, "lr": 0.0001425, "epoch": 0.09122006841505131, "percentage": 9.09, "elapsed_time": "0:01:33", "remaining_time": "0:15:34", "throughput": 2647.7, "total_tokens": 247552}
5
+ {"current_steps": 25, "total_steps": 220, "loss": 0.5721, "lr": 0.00017999999999999998, "epoch": 0.11402508551881414, "percentage": 11.36, "elapsed_time": "0:01:56", "remaining_time": "0:15:07", "throughput": 2656.08, "total_tokens": 308992}
6
+ {"current_steps": 30, "total_steps": 220, "loss": 0.6254, "lr": 0.00021749999999999997, "epoch": 0.13683010262257697, "percentage": 13.64, "elapsed_time": "0:02:19", "remaining_time": "0:14:41", "throughput": 2659.03, "total_tokens": 370176}
7
+ {"current_steps": 35, "total_steps": 220, "loss": 0.6389, "lr": 0.00025499999999999996, "epoch": 0.15963511972633979, "percentage": 15.91, "elapsed_time": "0:02:41", "remaining_time": "0:14:15", "throughput": 2660.78, "total_tokens": 430720}
8
+ {"current_steps": 40, "total_steps": 220, "loss": 0.5258, "lr": 0.00029249999999999995, "epoch": 0.18244013683010263, "percentage": 18.18, "elapsed_time": "0:03:05", "remaining_time": "0:13:53", "throughput": 2659.66, "total_tokens": 492800}
9
+ {"current_steps": 45, "total_steps": 220, "loss": 0.5041, "lr": 0.0002996346075389736, "epoch": 0.20524515393386544, "percentage": 20.45, "elapsed_time": "0:03:28", "remaining_time": "0:13:29", "throughput": 2660.7, "total_tokens": 553984}
10
+ {"current_steps": 50, "total_steps": 220, "loss": 0.4831, "lr": 0.00029815325108927063, "epoch": 0.22805017103762829, "percentage": 22.73, "elapsed_time": "0:03:51", "remaining_time": "0:13:05", "throughput": 2661.54, "total_tokens": 615040}
11
+ {"current_steps": 55, "total_steps": 220, "loss": 0.4957, "lr": 0.0002955443589413994, "epoch": 0.2508551881413911, "percentage": 25.0, "elapsed_time": "0:04:14", "remaining_time": "0:12:42", "throughput": 2663.54, "total_tokens": 676736}
12
+ {"current_steps": 60, "total_steps": 220, "loss": 0.4764, "lr": 0.00029182778633989753, "epoch": 0.27366020524515394, "percentage": 27.27, "elapsed_time": "0:04:37", "remaining_time": "0:12:18", "throughput": 2664.2, "total_tokens": 738176}
13
+ {"current_steps": 65, "total_steps": 220, "loss": 0.4829, "lr": 0.0002870318186463901, "epoch": 0.29646522234891676, "percentage": 29.55, "elapsed_time": "0:05:00", "remaining_time": "0:11:55", "throughput": 2664.82, "total_tokens": 799488}
14
+ {"current_steps": 70, "total_steps": 220, "loss": 0.478, "lr": 0.00028119295607090933, "epoch": 0.31927023945267957, "percentage": 31.82, "elapsed_time": "0:05:23", "remaining_time": "0:11:32", "throughput": 2666.34, "total_tokens": 861568}
15
+ {"current_steps": 75, "total_steps": 220, "loss": 0.4771, "lr": 0.0002743556358832562, "epoch": 0.34207525655644244, "percentage": 34.09, "elapsed_time": "0:05:46", "remaining_time": "0:11:10", "throughput": 2666.42, "total_tokens": 924544}
16
+ {"current_steps": 80, "total_steps": 220, "loss": 0.4657, "lr": 0.0002665718942185456, "epoch": 0.36488027366020526, "percentage": 36.36, "elapsed_time": "0:06:09", "remaining_time": "0:10:46", "throughput": 2666.94, "total_tokens": 985472}
17
+ {"current_steps": 85, "total_steps": 220, "loss": 0.4831, "lr": 0.00025790097005079764, "epoch": 0.38768529076396807, "percentage": 38.64, "elapsed_time": "0:06:32", "remaining_time": "0:10:23", "throughput": 2667.09, "total_tokens": 1046912}
18
+ {"current_steps": 90, "total_steps": 220, "loss": 0.4778, "lr": 0.0002484088543485761, "epoch": 0.4104903078677309, "percentage": 40.91, "elapsed_time": "0:06:55", "remaining_time": "0:10:00", "throughput": 2668.18, "total_tokens": 1108992}
19
+ {"current_steps": 95, "total_steps": 220, "loss": 0.455, "lr": 0.00023816778784387094, "epoch": 0.43329532497149376, "percentage": 43.18, "elapsed_time": "0:07:18", "remaining_time": "0:09:36", "throughput": 2668.23, "total_tokens": 1170048}
20
+ {"current_steps": 100, "total_steps": 220, "loss": 0.4571, "lr": 0.00022725571123650813, "epoch": 0.45610034207525657, "percentage": 45.45, "elapsed_time": "0:07:41", "remaining_time": "0:09:13", "throughput": 2668.44, "total_tokens": 1230464}
21
+ {"current_steps": 100, "total_steps": 220, "eval_loss": 0.4646710157394409, "epoch": 0.45610034207525657, "percentage": 45.45, "elapsed_time": "0:08:13", "remaining_time": "0:09:52", "throughput": 2491.96, "total_tokens": 1230464}
22
+ {"current_steps": 105, "total_steps": 220, "loss": 0.4667, "lr": 0.0002157556720183616, "epoch": 0.4789053591790194, "percentage": 47.73, "elapsed_time": "0:08:36", "remaining_time": "0:09:26", "throughput": 2500.4, "total_tokens": 1292288}
23
+ {"current_steps": 110, "total_steps": 220, "loss": 0.4758, "lr": 0.000203755192431795, "epoch": 0.5017103762827823, "percentage": 50.0, "elapsed_time": "0:08:59", "remaining_time": "0:08:59", "throughput": 2507.87, "total_tokens": 1353344}
24
+ {"current_steps": 115, "total_steps": 220, "loss": 0.4783, "lr": 0.00019134560337254986, "epoch": 0.5245153933865451, "percentage": 52.27, "elapsed_time": "0:09:22", "remaining_time": "0:08:33", "throughput": 2515.39, "total_tokens": 1415040}
25
+ {"current_steps": 120, "total_steps": 220, "loss": 0.4643, "lr": 0.0001786213493064817, "epoch": 0.5473204104903079, "percentage": 54.55, "elapsed_time": "0:09:45", "remaining_time": "0:08:07", "throughput": 2521.98, "total_tokens": 1476480}
26
+ {"current_steps": 125, "total_steps": 220, "loss": 0.4542, "lr": 0.000165679269490148, "epoch": 0.5701254275940707, "percentage": 56.82, "elapsed_time": "0:10:08", "remaining_time": "0:07:42", "throughput": 2527.64, "total_tokens": 1537664}
27
+ {"current_steps": 130, "total_steps": 220, "loss": 0.4539, "lr": 0.00015261786096559254, "epoch": 0.5929304446978335, "percentage": 59.09, "elapsed_time": "0:10:31", "remaining_time": "0:07:16", "throughput": 2533.11, "total_tokens": 1598848}
28
+ {"current_steps": 135, "total_steps": 220, "loss": 0.4563, "lr": 0.00013953652893838119, "epoch": 0.6157354618015963, "percentage": 61.36, "elapsed_time": "0:10:54", "remaining_time": "0:06:51", "throughput": 2538.83, "total_tokens": 1660800}
29
+ {"current_steps": 140, "total_steps": 220, "loss": 0.4434, "lr": 0.00012653483024396533, "epoch": 0.6385404789053591, "percentage": 63.64, "elapsed_time": "0:11:16", "remaining_time": "0:06:26", "throughput": 2543.39, "total_tokens": 1721600}
30
+ {"current_steps": 145, "total_steps": 220, "loss": 0.4484, "lr": 0.00011371171566004985, "epoch": 0.661345496009122, "percentage": 65.91, "elapsed_time": "0:11:39", "remaining_time": "0:06:01", "throughput": 2547.95, "total_tokens": 1783168}
31
+ {"current_steps": 150, "total_steps": 220, "loss": 0.4314, "lr": 0.00010116477683142652, "epoch": 0.6841505131128849, "percentage": 68.18, "elapsed_time": "0:12:02", "remaining_time": "0:05:37", "throughput": 2552.51, "total_tokens": 1844992}
32
+ {"current_steps": 155, "total_steps": 220, "loss": 0.4211, "lr": 8.898950353862998e-05, "epoch": 0.7069555302166477, "percentage": 70.45, "elapsed_time": "0:12:25", "remaining_time": "0:05:12", "throughput": 2556.34, "total_tokens": 1906048}
33
+ {"current_steps": 160, "total_steps": 220, "loss": 0.4324, "lr": 7.727855696304944e-05, "epoch": 0.7297605473204105, "percentage": 72.73, "elapsed_time": "0:12:48", "remaining_time": "0:04:48", "throughput": 2560.29, "total_tokens": 1967744}
34
+ {"current_steps": 165, "total_steps": 220, "loss": 0.4093, "lr": 6.612106447938799e-05, "epoch": 0.7525655644241733, "percentage": 75.0, "elapsed_time": "0:13:11", "remaining_time": "0:04:23", "throughput": 2563.4, "total_tokens": 2028032}
35
+ {"current_steps": 170, "total_steps": 220, "loss": 0.4113, "lr": 5.56019413425244e-05, "epoch": 0.7753705815279361, "percentage": 77.27, "elapsed_time": "0:13:33", "remaining_time": "0:03:59", "throughput": 2566.5, "total_tokens": 2088448}
36
+ {"current_steps": 175, "total_steps": 220, "loss": 0.4142, "lr": 4.5801244431150394e-05, "epoch": 0.798175598631699, "percentage": 79.55, "elapsed_time": "0:13:56", "remaining_time": "0:03:35", "throughput": 2569.89, "total_tokens": 2150144}
37
+ {"current_steps": 180, "total_steps": 220, "loss": 0.407, "lr": 3.6793562966584196e-05, "epoch": 0.8209806157354618, "percentage": 81.82, "elapsed_time": "0:14:19", "remaining_time": "0:03:11", "throughput": 2572.74, "total_tokens": 2211584}
38
+ {"current_steps": 185, "total_steps": 220, "loss": 0.3836, "lr": 2.8647450843757897e-05, "epoch": 0.8437856328392246, "percentage": 84.09, "elapsed_time": "0:14:42", "remaining_time": "0:02:46", "throughput": 2575.48, "total_tokens": 2272256}
39
+ {"current_steps": 190, "total_steps": 220, "loss": 0.3904, "lr": 2.1424904894683165e-05, "epoch": 0.8665906499429875, "percentage": 86.36, "elapsed_time": "0:15:05", "remaining_time": "0:02:22", "throughput": 2578.09, "total_tokens": 2333696}
40
+ {"current_steps": 195, "total_steps": 220, "loss": 0.4011, "lr": 1.5180893055124977e-05, "epoch": 0.8893956670467503, "percentage": 88.64, "elapsed_time": "0:15:28", "remaining_time": "0:01:58", "throughput": 2580.67, "total_tokens": 2394880}
41
+ {"current_steps": 200, "total_steps": 220, "loss": 0.3809, "lr": 9.962936025419754e-06, "epoch": 0.9122006841505131, "percentage": 90.91, "elapsed_time": "0:15:50", "remaining_time": "0:01:35", "throughput": 2582.96, "total_tokens": 2455680}
42
+ {"current_steps": 200, "total_steps": 220, "eval_loss": 0.3808976411819458, "epoch": 0.9122006841505131, "percentage": 90.91, "elapsed_time": "0:16:23", "remaining_time": "0:01:38", "throughput": 2497.07, "total_tokens": 2455680}
43
+ {"current_steps": 205, "total_steps": 220, "loss": 0.3799, "lr": 5.810745609252165e-06, "epoch": 0.935005701254276, "percentage": 93.18, "elapsed_time": "0:16:46", "remaining_time": "0:01:13", "throughput": 2501.27, "total_tokens": 2517376}
44
+ {"current_steps": 210, "total_steps": 220, "loss": 0.3944, "lr": 2.7559224828504035e-06, "epoch": 0.9578107183580388, "percentage": 95.45, "elapsed_time": "0:17:09", "remaining_time": "0:00:49", "throughput": 2505.48, "total_tokens": 2578816}
45
+ {"current_steps": 215, "total_steps": 220, "loss": 0.3721, "lr": 8.217156947590064e-07, "epoch": 0.9806157354618016, "percentage": 97.73, "elapsed_time": "0:17:32", "remaining_time": "0:00:24", "throughput": 2509.24, "total_tokens": 2640128}
46
+ {"current_steps": 220, "total_steps": 220, "loss": 0.3749, "lr": 2.284572654130956e-08, "epoch": 1.0, "percentage": 100.0, "elapsed_time": "0:17:51", "remaining_time": "0:00:00", "throughput": 2512.25, "total_tokens": 2691984}
47
+ {"current_steps": 220, "total_steps": 220, "epoch": 1.0, "percentage": 100.0, "elapsed_time": "0:17:54", "remaining_time": "0:00:00", "throughput": 2504.7, "total_tokens": 2691984}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
trainer_state.json CHANGED
@@ -2,9 +2,9 @@
2
  "best_global_step": null,
3
  "best_metric": null,
4
  "best_model_checkpoint": null,
5
- "epoch": 4.0,
6
  "eval_steps": 100,
7
- "global_step": 880,
8
  "is_hyper_param_search": false,
9
  "is_local_process_zero": true,
10
  "is_world_process_zero": true,
@@ -16,8 +16,8 @@
16
  "loss": 12.6644,
17
  "num_input_tokens_seen": 61696,
18
  "step": 5,
19
- "train_runtime": 24.2855,
20
- "train_tokens_per_second": 2540.442
21
  },
22
  {
23
  "epoch": 0.04561003420752566,
@@ -26,8 +26,8 @@
26
  "loss": 5.0641,
27
  "num_input_tokens_seen": 123136,
28
  "step": 10,
29
- "train_runtime": 47.0357,
30
- "train_tokens_per_second": 2617.929
31
  },
32
  {
33
  "epoch": 0.06841505131128849,
@@ -36,8 +36,8 @@
36
  "loss": 1.5053,
37
  "num_input_tokens_seen": 184832,
38
  "step": 15,
39
- "train_runtime": 69.9392,
40
- "train_tokens_per_second": 2642.751
41
  },
42
  {
43
  "epoch": 0.09122006841505131,
@@ -46,8 +46,8 @@
46
  "loss": 0.6917,
47
  "num_input_tokens_seen": 247552,
48
  "step": 20,
49
- "train_runtime": 93.0922,
50
- "train_tokens_per_second": 2659.212
51
  },
52
  {
53
  "epoch": 0.11402508551881414,
@@ -56,8 +56,8 @@
56
  "loss": 0.5721,
57
  "num_input_tokens_seen": 308992,
58
  "step": 25,
59
- "train_runtime": 115.7825,
60
- "train_tokens_per_second": 2668.728
61
  },
62
  {
63
  "epoch": 0.13683010262257697,
@@ -66,8 +66,8 @@
66
  "loss": 0.6254,
67
  "num_input_tokens_seen": 370176,
68
  "step": 30,
69
- "train_runtime": 138.519,
70
- "train_tokens_per_second": 2672.384
71
  },
72
  {
73
  "epoch": 0.15963511972633979,
@@ -76,8 +76,8 @@
76
  "loss": 0.6389,
77
  "num_input_tokens_seen": 430720,
78
  "step": 35,
79
- "train_runtime": 161.0396,
80
- "train_tokens_per_second": 2674.622
81
  },
82
  {
83
  "epoch": 0.18244013683010263,
@@ -86,1776 +86,402 @@
86
  "loss": 0.5258,
87
  "num_input_tokens_seen": 492800,
88
  "step": 40,
89
- "train_runtime": 184.2743,
90
- "train_tokens_per_second": 2674.275
91
  },
92
  {
93
  "epoch": 0.20524515393386544,
94
- "grad_norm": 4.25206184387207,
95
- "learning_rate": 0.00029998321527154097,
96
- "loss": 0.5039,
97
  "num_input_tokens_seen": 553984,
98
  "step": 45,
99
- "train_runtime": 207.0534,
100
- "train_tokens_per_second": 2675.562
101
  },
102
  {
103
  "epoch": 0.22805017103762829,
104
- "grad_norm": 0.5203157663345337,
105
- "learning_rate": 0.00029991503375003,
106
- "loss": 0.4873,
107
  "num_input_tokens_seen": 615040,
108
  "step": 50,
109
- "train_runtime": 229.7726,
110
- "train_tokens_per_second": 2676.734
111
  },
112
  {
113
  "epoch": 0.2508551881413911,
114
- "grad_norm": 0.31907328963279724,
115
- "learning_rate": 0.000299794430213186,
116
- "loss": 0.4971,
117
  "num_input_tokens_seen": 676736,
118
  "step": 55,
119
- "train_runtime": 252.5957,
120
- "train_tokens_per_second": 2679.127
121
  },
122
  {
123
  "epoch": 0.27366020524515394,
124
- "grad_norm": 0.36265772581100464,
125
- "learning_rate": 0.0002996214468334339,
126
- "loss": 0.4761,
127
  "num_input_tokens_seen": 738176,
128
  "step": 60,
129
- "train_runtime": 275.4227,
130
- "train_tokens_per_second": 2680.157
131
  },
132
  {
133
  "epoch": 0.29646522234891676,
134
- "grad_norm": 0.2299686223268509,
135
- "learning_rate": 0.00029939614409928584,
136
- "loss": 0.4849,
137
  "num_input_tokens_seen": 799488,
138
  "step": 65,
139
- "train_runtime": 298.1913,
140
- "train_tokens_per_second": 2681.124
141
  },
142
  {
143
  "epoch": 0.31927023945267957,
144
- "grad_norm": 0.15398980677127838,
145
- "learning_rate": 0.00029911860079419073,
146
- "loss": 0.4819,
147
  "num_input_tokens_seen": 861568,
148
  "step": 70,
149
- "train_runtime": 321.1312,
150
- "train_tokens_per_second": 2682.916
151
  },
152
  {
153
  "epoch": 0.34207525655644244,
154
- "grad_norm": 0.21743282675743103,
155
- "learning_rate": 0.00029878891396898494,
156
- "loss": 0.4812,
157
  "num_input_tokens_seen": 924544,
158
  "step": 75,
159
- "train_runtime": 344.5442,
160
- "train_tokens_per_second": 2683.383
161
  },
162
  {
163
  "epoch": 0.36488027366020526,
164
- "grad_norm": 0.18951573967933655,
165
- "learning_rate": 0.00029840719890795547,
166
- "loss": 0.4691,
167
  "num_input_tokens_seen": 985472,
168
  "step": 80,
169
- "train_runtime": 367.16,
170
- "train_tokens_per_second": 2684.04
171
  },
172
  {
173
  "epoch": 0.38768529076396807,
174
- "grad_norm": 0.4572216272354126,
175
- "learning_rate": 0.00029797358908852816,
176
- "loss": 0.4845,
177
  "num_input_tokens_seen": 1046912,
178
  "step": 85,
179
- "train_runtime": 390.0118,
180
- "train_tokens_per_second": 2684.308
181
  },
182
  {
183
  "epoch": 0.4104903078677309,
184
- "grad_norm": 0.3125233054161072,
185
- "learning_rate": 0.00029748823613459316,
186
- "loss": 0.482,
187
  "num_input_tokens_seen": 1108992,
188
  "step": 90,
189
- "train_runtime": 412.9641,
190
- "train_tokens_per_second": 2685.444
191
  },
192
  {
193
  "epoch": 0.43329532497149376,
194
- "grad_norm": 0.2670783996582031,
195
- "learning_rate": 0.0002969513097634853,
196
- "loss": 0.4548,
197
  "num_input_tokens_seen": 1170048,
198
  "step": 95,
199
- "train_runtime": 435.6939,
200
- "train_tokens_per_second": 2685.482
201
  },
202
  {
203
  "epoch": 0.45610034207525657,
204
- "grad_norm": 0.1218935027718544,
205
- "learning_rate": 0.00029636299772663795,
206
  "loss": 0.4571,
207
  "num_input_tokens_seen": 1230464,
208
  "step": 100,
209
- "train_runtime": 458.1608,
210
- "train_tokens_per_second": 2685.66
211
  },
212
  {
213
  "epoch": 0.45610034207525657,
214
- "eval_loss": 0.462704062461853,
215
- "eval_runtime": 32.2331,
216
- "eval_samples_per_second": 96.733,
217
- "eval_steps_per_second": 6.05,
218
  "num_input_tokens_seen": 1230464,
219
  "step": 100
220
  },
221
  {
222
  "epoch": 0.4789053591790194,
223
- "grad_norm": 0.16193914413452148,
224
- "learning_rate": 0.0002957235057439301,
225
- "loss": 0.4633,
226
  "num_input_tokens_seen": 1292288,
227
  "step": 105,
228
- "train_runtime": 513.3064,
229
- "train_tokens_per_second": 2517.576
230
  },
231
  {
232
  "epoch": 0.5017103762827823,
233
- "grad_norm": 0.2220025360584259,
234
- "learning_rate": 0.0002950330574317509,
235
- "loss": 0.474,
236
  "num_input_tokens_seen": 1353344,
237
  "step": 110,
238
- "train_runtime": 535.981,
239
- "train_tokens_per_second": 2524.985
240
  },
241
  {
242
  "epoch": 0.5245153933865451,
243
- "grad_norm": 0.25165775418281555,
244
- "learning_rate": 0.0002942918942248059,
245
- "loss": 0.4709,
246
  "num_input_tokens_seen": 1415040,
247
  "step": 115,
248
- "train_runtime": 558.7646,
249
- "train_tokens_per_second": 2532.444
250
  },
251
  {
252
  "epoch": 0.5473204104903079,
253
- "grad_norm": 0.2007230967283249,
254
- "learning_rate": 0.0002935002752916921,
255
- "loss": 0.4581,
256
  "num_input_tokens_seen": 1476480,
257
  "step": 120,
258
- "train_runtime": 581.5341,
259
- "train_tokens_per_second": 2538.94
260
  },
261
  {
262
  "epoch": 0.5701254275940707,
263
- "grad_norm": 0.17744658887386322,
264
- "learning_rate": 0.00029265847744427303,
265
- "loss": 0.4428,
266
  "num_input_tokens_seen": 1537664,
267
  "step": 125,
268
- "train_runtime": 604.3564,
269
- "train_tokens_per_second": 2544.3
270
  },
271
  {
272
  "epoch": 0.5929304446978335,
273
- "grad_norm": 0.36229953169822693,
274
- "learning_rate": 0.0002917667950408832,
275
- "loss": 0.441,
276
  "num_input_tokens_seen": 1598848,
277
  "step": 130,
278
- "train_runtime": 627.1244,
279
- "train_tokens_per_second": 2549.491
280
  },
281
  {
282
  "epoch": 0.6157354618015963,
283
- "grad_norm": 0.20263080298900604,
284
- "learning_rate": 0.0002908255398833979,
285
- "loss": 0.4484,
286
  "num_input_tokens_seen": 1660800,
287
  "step": 135,
288
- "train_runtime": 650.0019,
289
- "train_tokens_per_second": 2555.069
290
  },
291
  {
292
  "epoch": 0.6385404789053591,
293
- "grad_norm": 0.41183140873908997,
294
- "learning_rate": 0.0002898350411082021,
295
- "loss": 0.4312,
296
  "num_input_tokens_seen": 1721600,
297
  "step": 140,
298
- "train_runtime": 672.6152,
299
- "train_tokens_per_second": 2559.561
300
  },
301
  {
302
  "epoch": 0.661345496009122,
303
- "grad_norm": 0.24306368827819824,
304
- "learning_rate": 0.00028879564507109946,
305
- "loss": 0.4374,
306
  "num_input_tokens_seen": 1783168,
307
  "step": 145,
308
- "train_runtime": 695.4639,
309
- "train_tokens_per_second": 2563.998
310
  },
311
  {
312
  "epoch": 0.6841505131128849,
313
- "grad_norm": 0.24856272339820862,
314
- "learning_rate": 0.00028770771522619885,
315
- "loss": 0.4241,
316
  "num_input_tokens_seen": 1844992,
317
  "step": 150,
318
- "train_runtime": 718.3395,
319
- "train_tokens_per_second": 2568.412
320
  },
321
  {
322
  "epoch": 0.7069555302166477,
323
- "grad_norm": 0.2576836943626404,
324
- "learning_rate": 0.00028657163199882237,
325
- "loss": 0.4048,
326
  "num_input_tokens_seen": 1906048,
327
  "step": 155,
328
- "train_runtime": 741.0646,
329
- "train_tokens_per_second": 2572.04
330
  },
331
  {
332
  "epoch": 0.7297605473204105,
333
- "grad_norm": 0.28877368569374084,
334
- "learning_rate": 0.0002853877926524791,
335
- "loss": 0.4175,
336
  "num_input_tokens_seen": 1967744,
337
  "step": 160,
338
- "train_runtime": 763.9181,
339
- "train_tokens_per_second": 2575.857
340
  },
341
  {
342
  "epoch": 0.7525655644241733,
343
- "grad_norm": 0.2601846754550934,
344
- "learning_rate": 0.0002841566111499505,
345
- "loss": 0.405,
346
  "num_input_tokens_seen": 2028032,
347
  "step": 165,
348
- "train_runtime": 786.4134,
349
- "train_tokens_per_second": 2578.837
350
  },
351
  {
352
  "epoch": 0.7753705815279361,
353
- "grad_norm": 0.2702248990535736,
354
- "learning_rate": 0.00028287851800853697,
355
- "loss": 0.4023,
356
  "num_input_tokens_seen": 2088448,
357
  "step": 170,
358
- "train_runtime": 808.8913,
359
- "train_tokens_per_second": 2581.865
360
  },
361
  {
362
  "epoch": 0.798175598631699,
363
- "grad_norm": 0.2885015308856964,
364
- "learning_rate": 0.00028155396014951494,
365
- "loss": 0.4013,
366
  "num_input_tokens_seen": 2150144,
367
  "step": 175,
368
- "train_runtime": 831.7045,
369
- "train_tokens_per_second": 2585.226
370
  },
371
  {
372
  "epoch": 0.8209806157354618,
373
- "grad_norm": 0.4831572473049164,
374
- "learning_rate": 0.0002801834007418587,
375
- "loss": 0.4065,
376
  "num_input_tokens_seen": 2211584,
377
  "step": 180,
378
- "train_runtime": 854.5546,
379
- "train_tokens_per_second": 2587.996
380
  },
381
  {
382
  "epoch": 0.8437856328392246,
383
- "grad_norm": 0.39929574728012085,
384
- "learning_rate": 0.0002787673190402799,
385
- "loss": 0.3784,
386
  "num_input_tokens_seen": 2272256,
387
  "step": 185,
388
- "train_runtime": 877.0766,
389
- "train_tokens_per_second": 2590.716
390
  },
391
  {
392
  "epoch": 0.8665906499429875,
393
- "grad_norm": 0.42691075801849365,
394
- "learning_rate": 0.0002773062102176429,
395
- "loss": 0.3882,
396
  "num_input_tokens_seen": 2333696,
397
  "step": 190,
398
- "train_runtime": 899.8781,
399
- "train_tokens_per_second": 2593.347
400
  },
401
  {
402
  "epoch": 0.8893956670467503,
403
- "grad_norm": 0.4322110116481781,
404
- "learning_rate": 0.0002758005851918136,
405
- "loss": 0.3953,
406
  "num_input_tokens_seen": 2394880,
407
  "step": 195,
408
- "train_runtime": 922.5484,
409
- "train_tokens_per_second": 2595.94
410
  },
411
  {
412
  "epoch": 0.9122006841505131,
413
- "grad_norm": 0.27470600605010986,
414
- "learning_rate": 0.00027425097044700246,
415
- "loss": 0.3676,
416
  "num_input_tokens_seen": 2455680,
417
  "step": 200,
418
- "train_runtime": 945.1495,
419
- "train_tokens_per_second": 2598.192
420
  },
421
  {
422
  "epoch": 0.9122006841505131,
423
- "eval_loss": 0.3956287205219269,
424
- "eval_runtime": 32.3098,
425
- "eval_samples_per_second": 96.503,
426
- "eval_steps_per_second": 6.035,
427
  "num_input_tokens_seen": 2455680,
428
  "step": 200
429
  },
430
  {
431
  "epoch": 0.935005701254276,
432
- "grad_norm": 0.350835919380188,
433
- "learning_rate": 0.0002726579078496647,
434
- "loss": 0.3821,
435
  "num_input_tokens_seen": 2517376,
436
  "step": 205,
437
- "train_runtime": 1000.3566,
438
- "train_tokens_per_second": 2516.479
439
  },
440
  {
441
  "epoch": 0.9578107183580388,
442
- "grad_norm": 0.4473128914833069,
443
- "learning_rate": 0.0002710219544590212,
444
- "loss": 0.3828,
445
  "num_input_tokens_seen": 2578816,
446
  "step": 210,
447
- "train_runtime": 1023.095,
448
- "train_tokens_per_second": 2520.603
449
  },
450
  {
451
  "epoch": 0.9806157354618016,
452
- "grad_norm": 0.3217677175998688,
453
- "learning_rate": 0.00026934368233226714,
454
- "loss": 0.3569,
455
  "num_input_tokens_seen": 2640128,
456
  "step": 215,
457
- "train_runtime": 1045.8834,
458
- "train_tokens_per_second": 2524.304
459
  },
460
  {
461
  "epoch": 1.0,
462
- "grad_norm": 0.5540800094604492,
463
- "learning_rate": 0.00026762367832453617,
464
- "loss": 0.3597,
465
  "num_input_tokens_seen": 2691984,
466
  "step": 220,
467
- "train_runtime": 1065.1708,
468
- "train_tokens_per_second": 2527.279
469
- },
470
- {
471
- "epoch": 1.0228050171037628,
472
- "grad_norm": 0.352602481842041,
473
- "learning_rate": 0.0002658625438836899,
474
- "loss": 0.3621,
475
- "num_input_tokens_seen": 2753808,
476
- "step": 225,
477
- "train_runtime": 1088.108,
478
- "train_tokens_per_second": 2530.822
479
- },
480
- {
481
- "epoch": 1.0456100342075256,
482
- "grad_norm": 0.45127010345458984,
483
- "learning_rate": 0.00026406089484000466,
484
- "loss": 0.3623,
485
- "num_input_tokens_seen": 2815248,
486
- "step": 230,
487
- "train_runtime": 1110.8639,
488
- "train_tokens_per_second": 2534.287
489
- },
490
- {
491
- "epoch": 1.0684150513112884,
492
- "grad_norm": 0.34542417526245117,
493
- "learning_rate": 0.00026221936119082904,
494
- "loss": 0.3866,
495
- "num_input_tokens_seen": 2877456,
496
- "step": 235,
497
- "train_runtime": 1133.9972,
498
- "train_tokens_per_second": 2537.445
499
- },
500
- {
501
- "epoch": 1.0912200684150513,
502
- "grad_norm": 0.41213762760162354,
503
- "learning_rate": 0.00026033858688028784,
504
- "loss": 0.3598,
505
- "num_input_tokens_seen": 2939280,
506
- "step": 240,
507
- "train_runtime": 1156.9737,
508
- "train_tokens_per_second": 2540.49
509
- },
510
- {
511
- "epoch": 1.114025085518814,
512
- "grad_norm": 0.34747591614723206,
513
- "learning_rate": 0.0002584192295741087,
514
- "loss": 0.3488,
515
- "num_input_tokens_seen": 3000336,
516
- "step": 245,
517
- "train_runtime": 1179.7195,
518
- "train_tokens_per_second": 2543.262
519
- },
520
- {
521
- "epoch": 1.1368301026225769,
522
- "grad_norm": 0.2995326817035675,
523
- "learning_rate": 0.0002564619604296506,
524
- "loss": 0.3516,
525
- "num_input_tokens_seen": 3062416,
526
- "step": 250,
527
- "train_runtime": 1202.7529,
528
- "train_tokens_per_second": 2546.172
529
- },
530
- {
531
- "epoch": 1.1596351197263397,
532
- "grad_norm": 0.3599797785282135,
533
- "learning_rate": 0.00025446746386121504,
534
- "loss": 0.34,
535
- "num_input_tokens_seen": 3124368,
536
- "step": 255,
537
- "train_runtime": 1225.7586,
538
- "train_tokens_per_second": 2548.926
539
- },
540
- {
541
- "epoch": 1.1824401368301025,
542
- "grad_norm": 0.3630430996417999,
543
- "learning_rate": 0.000252436437300721,
544
- "loss": 0.3361,
545
- "num_input_tokens_seen": 3185424,
546
- "step": 260,
547
- "train_runtime": 1248.4807,
548
- "train_tokens_per_second": 2551.44
549
- },
550
- {
551
- "epoch": 1.2052451539338653,
552
- "grad_norm": 0.2464369684457779,
553
- "learning_rate": 0.0002503695909538287,
554
- "loss": 0.3172,
555
- "num_input_tokens_seen": 3246096,
556
- "step": 265,
557
- "train_runtime": 1271.0323,
558
- "train_tokens_per_second": 2553.905
559
- },
560
- {
561
- "epoch": 1.2280501710376284,
562
- "grad_norm": 0.41870465874671936,
563
- "learning_rate": 0.00024826764755159566,
564
- "loss": 0.3387,
565
- "num_input_tokens_seen": 3307536,
566
- "step": 270,
567
- "train_runtime": 1293.8594,
568
- "train_tokens_per_second": 2556.333
569
- },
570
- {
571
- "epoch": 1.2508551881413912,
572
- "grad_norm": 0.6106472611427307,
573
- "learning_rate": 0.0002461313420977536,
574
- "loss": 0.3112,
575
- "num_input_tokens_seen": 3368976,
576
- "step": 275,
577
- "train_runtime": 1316.714,
578
- "train_tokens_per_second": 2558.624
579
- },
580
- {
581
- "epoch": 1.273660205245154,
582
- "grad_norm": 0.5107876658439636,
583
- "learning_rate": 0.0002439614216116939,
584
- "loss": 0.3223,
585
- "num_input_tokens_seen": 3429520,
586
- "step": 280,
587
- "train_runtime": 1339.2313,
588
- "train_tokens_per_second": 2560.812
589
- },
590
- {
591
- "epoch": 1.2964652223489168,
592
- "grad_norm": 0.3615415096282959,
593
- "learning_rate": 0.00024175864486725092,
594
- "loss": 0.312,
595
- "num_input_tokens_seen": 3490448,
596
- "step": 285,
597
- "train_runtime": 1361.9259,
598
- "train_tokens_per_second": 2562.877
599
- },
600
- {
601
- "epoch": 1.3192702394526796,
602
- "grad_norm": 0.3464173674583435,
603
- "learning_rate": 0.00023952378212737554,
604
- "loss": 0.3096,
605
- "num_input_tokens_seen": 3551120,
606
- "step": 290,
607
- "train_runtime": 1384.5561,
608
- "train_tokens_per_second": 2564.808
609
- },
610
- {
611
- "epoch": 1.3420752565564424,
612
- "grad_norm": 0.24868755042552948,
613
- "learning_rate": 0.00023725761487479086,
614
- "loss": 0.3109,
615
- "num_input_tokens_seen": 3611920,
616
- "step": 295,
617
- "train_runtime": 1407.1864,
618
- "train_tokens_per_second": 2566.767
619
- },
620
- {
621
- "epoch": 1.3648802736602053,
622
- "grad_norm": 0.252758651971817,
623
- "learning_rate": 0.0002349609355387249,
624
- "loss": 0.3092,
625
- "num_input_tokens_seen": 3673744,
626
- "step": 300,
627
- "train_runtime": 1430.0422,
628
- "train_tokens_per_second": 2568.976
629
- },
630
- {
631
- "epoch": 1.3648802736602053,
632
- "eval_loss": 0.31190308928489685,
633
- "eval_runtime": 32.3176,
634
- "eval_samples_per_second": 96.48,
635
- "eval_steps_per_second": 6.034,
636
- "num_input_tokens_seen": 3673744,
637
- "step": 300
638
  },
639
  {
640
- "epoch": 1.387685290763968,
641
- "grad_norm": 0.3992477059364319,
642
- "learning_rate": 0.00023263454721781537,
643
- "loss": 0.308,
644
- "num_input_tokens_seen": 3735568,
645
- "step": 305,
646
- "train_runtime": 1485.2691,
647
- "train_tokens_per_second": 2515.078
648
- },
649
- {
650
- "epoch": 1.4104903078677309,
651
- "grad_norm": 0.32043761014938354,
652
- "learning_rate": 0.0002302792633992835,
653
- "loss": 0.3089,
654
- "num_input_tokens_seen": 3798160,
655
- "step": 310,
656
- "train_runtime": 1508.5292,
657
- "train_tokens_per_second": 2517.79
658
- },
659
- {
660
- "epoch": 1.4332953249714937,
661
- "grad_norm": 0.28808364272117615,
662
- "learning_rate": 0.00022789590767447533,
663
- "loss": 0.3028,
664
- "num_input_tokens_seen": 3859600,
665
- "step": 315,
666
- "train_runtime": 1531.393,
667
- "train_tokens_per_second": 2520.32
668
- },
669
- {
670
- "epoch": 1.4561003420752565,
671
- "grad_norm": 0.31557497382164,
672
- "learning_rate": 0.00022548531345087,
673
- "loss": 0.2925,
674
- "num_input_tokens_seen": 3920528,
675
- "step": 320,
676
- "train_runtime": 1553.9921,
677
- "train_tokens_per_second": 2522.875
678
- },
679
- {
680
- "epoch": 1.4789053591790193,
681
- "grad_norm": 0.28433188796043396,
682
- "learning_rate": 0.00022304832366065505,
683
- "loss": 0.285,
684
- "num_input_tokens_seen": 3982096,
685
- "step": 325,
686
- "train_runtime": 1576.8412,
687
- "train_tokens_per_second": 2525.363
688
- },
689
- {
690
- "epoch": 1.5017103762827824,
691
- "grad_norm": 0.2831820547580719,
692
- "learning_rate": 0.0002205857904659717,
693
- "loss": 0.304,
694
- "num_input_tokens_seen": 4044048,
695
- "step": 330,
696
- "train_runtime": 1599.876,
697
- "train_tokens_per_second": 2527.726
698
- },
699
- {
700
- "epoch": 1.5245153933865452,
701
- "grad_norm": 0.5233074426651001,
702
- "learning_rate": 0.00021809857496093199,
703
- "loss": 0.2892,
704
- "num_input_tokens_seen": 4105104,
705
- "step": 335,
706
- "train_runtime": 1622.5215,
707
- "train_tokens_per_second": 2530.077
708
- },
709
- {
710
- "epoch": 1.547320410490308,
711
- "grad_norm": 0.326762318611145,
712
- "learning_rate": 0.00021558754687051344,
713
- "loss": 0.2849,
714
- "num_input_tokens_seen": 4165904,
715
- "step": 340,
716
- "train_runtime": 1645.0233,
717
- "train_tokens_per_second": 2532.429
718
- },
719
- {
720
- "epoch": 1.5701254275940708,
721
- "grad_norm": 0.35533398389816284,
722
- "learning_rate": 0.0002130535842464348,
723
- "loss": 0.2975,
724
- "num_input_tokens_seen": 4226704,
725
- "step": 345,
726
- "train_runtime": 1667.6513,
727
- "train_tokens_per_second": 2534.525
728
- },
729
- {
730
- "epoch": 1.5929304446978336,
731
- "grad_norm": 0.2767379879951477,
732
- "learning_rate": 0.00021049757316012077,
733
- "loss": 0.2845,
734
- "num_input_tokens_seen": 4288400,
735
- "step": 350,
736
- "train_runtime": 1690.4931,
737
- "train_tokens_per_second": 2536.775
738
- },
739
- {
740
- "epoch": 1.6157354618015964,
741
- "grad_norm": 0.35976049304008484,
742
- "learning_rate": 0.0002079204073928619,
743
- "loss": 0.2869,
744
- "num_input_tokens_seen": 4349712,
745
- "step": 355,
746
- "train_runtime": 1713.2321,
747
- "train_tokens_per_second": 2538.892
748
- },
749
- {
750
- "epoch": 1.6385404789053593,
751
- "grad_norm": 0.23936209082603455,
752
- "learning_rate": 0.0002053229881232793,
753
- "loss": 0.2601,
754
- "num_input_tokens_seen": 4410256,
755
- "step": 360,
756
- "train_runtime": 1735.7264,
757
- "train_tokens_per_second": 2540.87
758
- },
759
- {
760
- "epoch": 1.661345496009122,
761
- "grad_norm": 0.2862788140773773,
762
- "learning_rate": 0.0002027062236122014,
763
- "loss": 0.2826,
764
- "num_input_tokens_seen": 4471440,
765
- "step": 365,
766
- "train_runtime": 1758.3589,
767
- "train_tokens_per_second": 2542.962
768
- },
769
- {
770
- "epoch": 1.6841505131128849,
771
- "grad_norm": 0.3150525391101837,
772
- "learning_rate": 0.0002000710288850656,
773
- "loss": 0.2729,
774
- "num_input_tokens_seen": 4533008,
775
- "step": 370,
776
- "train_runtime": 1781.1643,
777
- "train_tokens_per_second": 2544.969
778
- },
779
- {
780
- "epoch": 1.7069555302166477,
781
- "grad_norm": 0.31325292587280273,
782
- "learning_rate": 0.00019741832541195366,
783
- "loss": 0.2772,
784
- "num_input_tokens_seen": 4594320,
785
- "step": 375,
786
- "train_runtime": 1803.8324,
787
- "train_tokens_per_second": 2546.977
788
- },
789
- {
790
- "epoch": 1.7297605473204105,
791
- "grad_norm": 0.8041843175888062,
792
- "learning_rate": 0.0001947490407853734,
793
- "loss": 0.2789,
794
- "num_input_tokens_seen": 4655760,
795
- "step": 380,
796
- "train_runtime": 1826.573,
797
- "train_tokens_per_second": 2548.904
798
- },
799
- {
800
- "epoch": 1.7525655644241733,
801
- "grad_norm": 0.23466791212558746,
802
- "learning_rate": 0.0001920641083959004,
803
- "loss": 0.2743,
804
- "num_input_tokens_seen": 4717328,
805
- "step": 385,
806
- "train_runtime": 1849.3505,
807
- "train_tokens_per_second": 2550.803
808
- },
809
- {
810
- "epoch": 1.7753705815279361,
811
- "grad_norm": 0.29242169857025146,
812
- "learning_rate": 0.00018936446710579122,
813
- "loss": 0.2657,
814
- "num_input_tokens_seen": 4779152,
815
- "step": 390,
816
- "train_runtime": 1872.1695,
817
- "train_tokens_per_second": 2552.735
818
- },
819
- {
820
- "epoch": 1.798175598631699,
821
- "grad_norm": 0.3305339217185974,
822
- "learning_rate": 0.00018665106092068408,
823
- "loss": 0.2847,
824
- "num_input_tokens_seen": 4840336,
825
- "step": 395,
826
- "train_runtime": 1894.8106,
827
- "train_tokens_per_second": 2554.522
828
- },
829
- {
830
- "epoch": 1.8209806157354618,
831
- "grad_norm": 0.3732515871524811,
832
- "learning_rate": 0.00018392483865950046,
833
- "loss": 0.2796,
834
- "num_input_tokens_seen": 4901648,
835
- "step": 400,
836
- "train_runtime": 1917.4674,
837
- "train_tokens_per_second": 2556.314
838
- },
839
- {
840
- "epoch": 1.8209806157354618,
841
- "eval_loss": 0.2729249596595764,
842
- "eval_runtime": 32.2506,
843
- "eval_samples_per_second": 96.68,
844
- "eval_steps_per_second": 6.046,
845
- "num_input_tokens_seen": 4901648,
846
- "step": 400
847
- },
848
- {
849
- "epoch": 1.8437856328392246,
850
- "grad_norm": 0.30687257647514343,
851
- "learning_rate": 0.00018118675362266385,
852
- "loss": 0.2725,
853
- "num_input_tokens_seen": 4963216,
854
- "step": 405,
855
- "train_runtime": 1972.5152,
856
- "train_tokens_per_second": 2516.186
857
- },
858
- {
859
- "epoch": 1.8665906499429874,
860
- "grad_norm": 0.2922170162200928,
861
- "learning_rate": 0.00017843776325875176,
862
- "loss": 0.2714,
863
- "num_input_tokens_seen": 5024912,
864
- "step": 410,
865
- "train_runtime": 1995.3388,
866
- "train_tokens_per_second": 2518.325
867
- },
868
- {
869
- "epoch": 1.8893956670467502,
870
- "grad_norm": 0.3486429452896118,
871
- "learning_rate": 0.00017567882882969628,
872
- "loss": 0.2547,
873
- "num_input_tokens_seen": 5086224,
874
- "step": 415,
875
- "train_runtime": 2018.156,
876
- "train_tokens_per_second": 2520.233
877
- },
878
- {
879
- "epoch": 1.912200684150513,
880
- "grad_norm": 0.4654463827610016,
881
- "learning_rate": 0.0001729109150746521,
882
- "loss": 0.2687,
883
- "num_input_tokens_seen": 5148048,
884
- "step": 420,
885
- "train_runtime": 2041.096,
886
- "train_tokens_per_second": 2522.198
887
- },
888
- {
889
- "epoch": 1.9350057012542758,
890
- "grad_norm": 0.3267694413661957,
891
- "learning_rate": 0.00017013498987264832,
892
- "loss": 0.2644,
893
- "num_input_tokens_seen": 5209104,
894
- "step": 425,
895
- "train_runtime": 2063.7125,
896
- "train_tokens_per_second": 2524.142
897
- },
898
- {
899
- "epoch": 1.9578107183580387,
900
- "grad_norm": 0.6316472291946411,
901
- "learning_rate": 0.00016735202390414215,
902
- "loss": 0.2696,
903
- "num_input_tokens_seen": 5270544,
904
- "step": 430,
905
- "train_runtime": 2086.4995,
906
- "train_tokens_per_second": 2526.022
907
- },
908
- {
909
- "epoch": 1.9806157354618015,
910
- "grad_norm": 0.23907926678657532,
911
- "learning_rate": 0.00016456299031159322,
912
- "loss": 0.2694,
913
- "num_input_tokens_seen": 5331472,
914
- "step": 435,
915
- "train_runtime": 2109.1335,
916
- "train_tokens_per_second": 2527.802
917
- },
918
- {
919
- "epoch": 2.0,
920
- "grad_norm": 0.6817493438720703,
921
- "learning_rate": 0.00016176886435917675,
922
- "loss": 0.2495,
923
- "num_input_tokens_seen": 5382864,
924
- "step": 440,
925
- "train_runtime": 2128.3818,
926
- "train_tokens_per_second": 2529.088
927
- },
928
- {
929
- "epoch": 2.022805017103763,
930
- "grad_norm": 0.5987955927848816,
931
- "learning_rate": 0.00015897062309175512,
932
- "loss": 0.2453,
933
- "num_input_tokens_seen": 5444176,
934
- "step": 445,
935
- "train_runtime": 2151.1431,
936
- "train_tokens_per_second": 2530.829
937
- },
938
- {
939
- "epoch": 2.0456100342075256,
940
- "grad_norm": 0.3288906216621399,
941
- "learning_rate": 0.0001561692449932266,
942
- "loss": 0.2299,
943
- "num_input_tokens_seen": 5505872,
944
- "step": 450,
945
- "train_runtime": 2174.0024,
946
- "train_tokens_per_second": 2532.597
947
- },
948
- {
949
- "epoch": 2.0684150513112884,
950
- "grad_norm": 0.31571510434150696,
951
- "learning_rate": 0.00015336570964437075,
952
- "loss": 0.2301,
953
- "num_input_tokens_seen": 5568208,
954
- "step": 455,
955
- "train_runtime": 2197.104,
956
- "train_tokens_per_second": 2534.34
957
- },
958
- {
959
- "epoch": 2.0912200684150513,
960
- "grad_norm": 0.3670252859592438,
961
- "learning_rate": 0.0001505609973803105,
962
- "loss": 0.2376,
963
- "num_input_tokens_seen": 5628880,
964
- "step": 460,
965
- "train_runtime": 2219.7182,
966
- "train_tokens_per_second": 2535.853
967
- },
968
- {
969
- "epoch": 2.114025085518814,
970
- "grad_norm": 0.27864688634872437,
971
- "learning_rate": 0.00014775608894771047,
972
- "loss": 0.2199,
973
- "num_input_tokens_seen": 5690320,
974
- "step": 465,
975
- "train_runtime": 2242.4827,
976
- "train_tokens_per_second": 2537.509
977
- },
978
- {
979
- "epoch": 2.136830102622577,
980
- "grad_norm": 0.30822446942329407,
981
- "learning_rate": 0.00014495196516183095,
982
- "loss": 0.2237,
983
- "num_input_tokens_seen": 5752912,
984
- "step": 470,
985
- "train_runtime": 2265.6875,
986
- "train_tokens_per_second": 2539.146
987
- },
988
- {
989
- "epoch": 2.1596351197263397,
990
- "grad_norm": 0.34737417101860046,
991
- "learning_rate": 0.0001421496065635584,
992
- "loss": 0.2294,
993
- "num_input_tokens_seen": 5813968,
994
- "step": 475,
995
- "train_runtime": 2288.3705,
996
- "train_tokens_per_second": 2540.659
997
- },
998
- {
999
- "epoch": 2.1824401368301025,
1000
- "grad_norm": 0.2497541755437851,
1001
- "learning_rate": 0.00013934999307653193,
1002
- "loss": 0.231,
1003
- "num_input_tokens_seen": 5874896,
1004
- "step": 480,
1005
- "train_runtime": 2310.9431,
1006
- "train_tokens_per_second": 2542.207
1007
- },
1008
- {
1009
- "epoch": 2.2052451539338653,
1010
- "grad_norm": 0.30357566475868225,
1011
- "learning_rate": 0.00013655410366448498,
1012
- "loss": 0.2252,
1013
- "num_input_tokens_seen": 5935952,
1014
- "step": 485,
1015
- "train_runtime": 2333.5618,
1016
- "train_tokens_per_second": 2543.73
1017
- },
1018
- {
1019
- "epoch": 2.228050171037628,
1020
- "grad_norm": 0.3196811378002167,
1021
- "learning_rate": 0.0001337629159889232,
1022
- "loss": 0.2151,
1023
- "num_input_tokens_seen": 5997008,
1024
- "step": 490,
1025
- "train_runtime": 2356.1683,
1026
- "train_tokens_per_second": 2545.237
1027
- },
1028
- {
1029
- "epoch": 2.250855188141391,
1030
- "grad_norm": 0.464616984128952,
1031
- "learning_rate": 0.00013097740606725776,
1032
- "loss": 0.2228,
1033
- "num_input_tokens_seen": 6058704,
1034
- "step": 495,
1035
- "train_runtime": 2379.0776,
1036
- "train_tokens_per_second": 2546.661
1037
- },
1038
- {
1039
- "epoch": 2.2736602052451538,
1040
- "grad_norm": 0.31572940945625305,
1041
- "learning_rate": 0.00012819854793151312,
1042
- "loss": 0.2265,
1043
- "num_input_tokens_seen": 6120272,
1044
- "step": 500,
1045
- "train_runtime": 2401.9854,
1046
- "train_tokens_per_second": 2548.006
1047
- },
1048
- {
1049
- "epoch": 2.2736602052451538,
1050
- "eval_loss": 0.24168384075164795,
1051
- "eval_runtime": 32.2697,
1052
- "eval_samples_per_second": 96.623,
1053
- "eval_steps_per_second": 6.043,
1054
- "num_input_tokens_seen": 6120272,
1055
- "step": 500
1056
- },
1057
- {
1058
- "epoch": 2.2964652223489166,
1059
- "grad_norm": 0.3663758933544159,
1060
- "learning_rate": 0.00012542731328772934,
1061
- "loss": 0.2126,
1062
- "num_input_tokens_seen": 6180816,
1063
- "step": 505,
1064
- "train_runtime": 2462.9918,
1065
- "train_tokens_per_second": 2509.475
1066
- },
1067
- {
1068
- "epoch": 2.3192702394526794,
1069
- "grad_norm": 0.4008311629295349,
1070
- "learning_rate": 0.00012266467117617787,
1071
- "loss": 0.2202,
1072
- "num_input_tokens_seen": 6242128,
1073
- "step": 510,
1074
- "train_runtime": 2485.6998,
1075
- "train_tokens_per_second": 2511.216
1076
- },
1077
- {
1078
- "epoch": 2.342075256556442,
1079
- "grad_norm": 0.29326286911964417,
1080
- "learning_rate": 0.00011991158763250908,
1081
- "loss": 0.2203,
1082
- "num_input_tokens_seen": 6303312,
1083
- "step": 515,
1084
- "train_runtime": 2508.4008,
1085
- "train_tokens_per_second": 2512.881
1086
- },
1087
- {
1088
- "epoch": 2.364880273660205,
1089
- "grad_norm": 0.2958817780017853,
1090
- "learning_rate": 0.00011716902534995079,
1091
- "loss": 0.223,
1092
- "num_input_tokens_seen": 6364624,
1093
- "step": 520,
1094
- "train_runtime": 2531.1448,
1095
- "train_tokens_per_second": 2514.524
1096
- },
1097
- {
1098
- "epoch": 2.387685290763968,
1099
- "grad_norm": 0.42678675055503845,
1100
- "learning_rate": 0.00011443794334267538,
1101
- "loss": 0.2218,
1102
- "num_input_tokens_seen": 6425040,
1103
- "step": 525,
1104
- "train_runtime": 2553.5434,
1105
- "train_tokens_per_second": 2516.127
1106
- },
1107
- {
1108
- "epoch": 2.4104903078677307,
1109
- "grad_norm": 0.3538350760936737,
1110
- "learning_rate": 0.0001117192966104536,
1111
- "loss": 0.2156,
1112
- "num_input_tokens_seen": 6486736,
1113
- "step": 530,
1114
- "train_runtime": 2576.3405,
1115
- "train_tokens_per_second": 2517.81
1116
- },
1117
- {
1118
- "epoch": 2.433295324971494,
1119
- "grad_norm": 0.31797412037849426,
1120
- "learning_rate": 0.000109014035804712,
1121
- "loss": 0.2161,
1122
- "num_input_tokens_seen": 6548304,
1123
- "step": 535,
1124
- "train_runtime": 2599.1554,
1125
- "train_tokens_per_second": 2519.397
1126
- },
1127
- {
1128
- "epoch": 2.4561003420752567,
1129
- "grad_norm": 0.3164632320404053,
1130
- "learning_rate": 0.0001063231068961106,
1131
- "loss": 0.2045,
1132
- "num_input_tokens_seen": 6610000,
1133
- "step": 540,
1134
- "train_runtime": 2622.0446,
1135
- "train_tokens_per_second": 2520.933
1136
- },
1137
- {
1138
- "epoch": 2.4789053591790196,
1139
- "grad_norm": 0.2635822296142578,
1140
- "learning_rate": 0.0001036474508437579,
1141
- "loss": 0.2073,
1142
- "num_input_tokens_seen": 6670672,
1143
- "step": 545,
1144
- "train_runtime": 2644.5946,
1145
- "train_tokens_per_second": 2522.38
1146
- },
1147
- {
1148
- "epoch": 2.5017103762827824,
1149
- "grad_norm": 0.2644634246826172,
1150
- "learning_rate": 0.00010098800326617791,
1151
- "loss": 0.2018,
1152
- "num_input_tokens_seen": 6732112,
1153
- "step": 550,
1154
- "train_runtime": 2667.3609,
1155
- "train_tokens_per_second": 2523.885
1156
- },
1157
- {
1158
- "epoch": 2.524515393386545,
1159
- "grad_norm": 0.3596520721912384,
1160
- "learning_rate": 9.834569411414471e-05,
1161
- "loss": 0.2048,
1162
- "num_input_tokens_seen": 6793424,
1163
- "step": 555,
1164
- "train_runtime": 2690.0439,
1165
- "train_tokens_per_second": 2525.395
1166
- },
1167
- {
1168
- "epoch": 2.547320410490308,
1169
- "grad_norm": 0.3009176254272461,
1170
- "learning_rate": 9.572144734549919e-05,
1171
- "loss": 0.2062,
1172
- "num_input_tokens_seen": 6855760,
1173
- "step": 560,
1174
- "train_runtime": 2713.0534,
1175
- "train_tokens_per_second": 2526.954
1176
- },
1177
- {
1178
- "epoch": 2.570125427594071,
1179
- "grad_norm": 0.3456870913505554,
1180
- "learning_rate": 9.311618060206074e-05,
1181
- "loss": 0.216,
1182
- "num_input_tokens_seen": 6916688,
1183
- "step": 565,
1184
- "train_runtime": 2735.7383,
1185
- "train_tokens_per_second": 2528.271
1186
- },
1187
- {
1188
- "epoch": 2.5929304446978336,
1189
- "grad_norm": 0.27823150157928467,
1190
- "learning_rate": 9.053080488874836e-05,
1191
- "loss": 0.1953,
1192
- "num_input_tokens_seen": 6978512,
1193
- "step": 570,
1194
- "train_runtime": 2758.6045,
1195
- "train_tokens_per_second": 2529.725
1196
- },
1197
- {
1198
- "epoch": 2.6157354618015964,
1199
- "grad_norm": 0.31827694177627563,
1200
- "learning_rate": 8.796622425502192e-05,
1201
- "loss": 0.2147,
1202
- "num_input_tokens_seen": 7040208,
1203
- "step": 575,
1204
- "train_runtime": 2781.5669,
1205
- "train_tokens_per_second": 2531.022
1206
- },
1207
- {
1208
- "epoch": 2.6385404789053593,
1209
- "grad_norm": 0.3293188512325287,
1210
- "learning_rate": 8.54233354787557e-05,
1211
- "loss": 0.209,
1212
- "num_input_tokens_seen": 7101264,
1213
- "step": 580,
1214
- "train_runtime": 2804.2079,
1215
- "train_tokens_per_second": 2532.36
1216
- },
1217
- {
1218
- "epoch": 2.661345496009122,
1219
- "grad_norm": 0.26295793056488037,
1220
- "learning_rate": 8.290302775265509e-05,
1221
- "loss": 0.2077,
1222
- "num_input_tokens_seen": 7163600,
1223
- "step": 585,
1224
- "train_runtime": 2827.2586,
1225
- "train_tokens_per_second": 2533.762
1226
- },
1227
- {
1228
- "epoch": 2.684150513112885,
1229
- "grad_norm": 0.2915421426296234,
1230
- "learning_rate": 8.04061823733249e-05,
1231
- "loss": 0.2058,
1232
- "num_input_tokens_seen": 7225936,
1233
- "step": 590,
1234
- "train_runtime": 2850.236,
1235
- "train_tokens_per_second": 2535.206
1236
- },
1237
- {
1238
- "epoch": 2.7069555302166477,
1239
- "grad_norm": 0.3178254961967468,
1240
- "learning_rate": 7.79336724330994e-05,
1241
- "loss": 0.1961,
1242
- "num_input_tokens_seen": 7287760,
1243
- "step": 595,
1244
- "train_runtime": 2873.2284,
1245
- "train_tokens_per_second": 2536.436
1246
- },
1247
- {
1248
- "epoch": 2.7297605473204105,
1249
- "grad_norm": 0.2937912344932556,
1250
- "learning_rate": 7.548636251474091e-05,
1251
- "loss": 0.2003,
1252
- "num_input_tokens_seen": 7348944,
1253
- "step": 600,
1254
- "train_runtime": 2895.9783,
1255
- "train_tokens_per_second": 2537.638
1256
- },
1257
- {
1258
- "epoch": 2.7297605473204105,
1259
- "eval_loss": 0.22281652688980103,
1260
- "eval_runtime": 32.2665,
1261
- "eval_samples_per_second": 96.633,
1262
- "eval_steps_per_second": 6.043,
1263
- "num_input_tokens_seen": 7348944,
1264
- "step": 600
1265
- },
1266
- {
1267
- "epoch": 2.7525655644241733,
1268
- "grad_norm": 0.30021318793296814,
1269
- "learning_rate": 7.30651083891141e-05,
1270
- "loss": 0.2058,
1271
- "num_input_tokens_seen": 7410640,
1272
- "step": 605,
1273
- "train_runtime": 2951.0324,
1274
- "train_tokens_per_second": 2511.203
1275
- },
1276
- {
1277
- "epoch": 2.775370581527936,
1278
- "grad_norm": 0.3852008581161499,
1279
- "learning_rate": 7.067075671594168e-05,
1280
- "loss": 0.1909,
1281
- "num_input_tokens_seen": 7471184,
1282
- "step": 610,
1283
- "train_runtime": 2973.5012,
1284
- "train_tokens_per_second": 2512.588
1285
- },
1286
- {
1287
- "epoch": 2.798175598631699,
1288
- "grad_norm": 0.3185110092163086,
1289
- "learning_rate": 6.830414474774594e-05,
1290
- "loss": 0.2035,
1291
- "num_input_tokens_seen": 7533392,
1292
- "step": 615,
1293
- "train_runtime": 2996.5054,
1294
- "train_tokens_per_second": 2514.059
1295
- },
1296
- {
1297
- "epoch": 2.8209806157354618,
1298
- "grad_norm": 0.33980560302734375,
1299
- "learning_rate": 6.596610003707958e-05,
1300
- "loss": 0.1963,
1301
- "num_input_tokens_seen": 7594832,
1302
- "step": 620,
1303
- "train_runtime": 3019.2698,
1304
- "train_tokens_per_second": 2515.453
1305
- },
1306
- {
1307
- "epoch": 2.8437856328392246,
1308
- "grad_norm": 0.26889362931251526,
1309
- "learning_rate": 6.365744014714898e-05,
1310
- "loss": 0.1951,
1311
- "num_input_tokens_seen": 7656656,
1312
- "step": 625,
1313
- "train_runtime": 3042.165,
1314
- "train_tokens_per_second": 2516.844
1315
- },
1316
- {
1317
- "epoch": 2.8665906499429874,
1318
- "grad_norm": 0.2750576436519623,
1319
- "learning_rate": 6.137897236593002e-05,
1320
- "loss": 0.193,
1321
- "num_input_tokens_seen": 7717712,
1322
- "step": 630,
1323
- "train_runtime": 3064.8231,
1324
- "train_tokens_per_second": 2518.159
1325
- },
1326
- {
1327
- "epoch": 2.88939566704675,
1328
- "grad_norm": 0.32393938302993774,
1329
- "learning_rate": 5.913149342387703e-05,
1330
- "loss": 0.1925,
1331
- "num_input_tokens_seen": 7779920,
1332
- "step": 635,
1333
- "train_runtime": 3087.8496,
1334
- "train_tokens_per_second": 2519.527
1335
- },
1336
- {
1337
- "epoch": 2.912200684150513,
1338
- "grad_norm": 0.33339279890060425,
1339
- "learning_rate": 5.6915789215323817e-05,
1340
- "loss": 0.1995,
1341
- "num_input_tokens_seen": 7840976,
1342
- "step": 640,
1343
- "train_runtime": 3110.4659,
1344
- "train_tokens_per_second": 2520.836
1345
- },
1346
- {
1347
- "epoch": 2.935005701254276,
1348
- "grad_norm": 0.26764461398124695,
1349
- "learning_rate": 5.473263452367318e-05,
1350
- "loss": 0.1852,
1351
- "num_input_tokens_seen": 7902160,
1352
- "step": 645,
1353
- "train_runtime": 3133.2052,
1354
- "train_tokens_per_second": 2522.069
1355
- },
1356
- {
1357
- "epoch": 2.9578107183580387,
1358
- "grad_norm": 0.28049248456954956,
1359
- "learning_rate": 5.2582792750472464e-05,
1360
- "loss": 0.1878,
1361
- "num_input_tokens_seen": 7963856,
1362
- "step": 650,
1363
- "train_runtime": 3155.996,
1364
- "train_tokens_per_second": 2523.405
1365
- },
1366
- {
1367
- "epoch": 2.9806157354618015,
1368
- "grad_norm": 0.3994698226451874,
1369
- "learning_rate": 5.046701564846834e-05,
1370
- "loss": 0.1945,
1371
- "num_input_tokens_seen": 8026448,
1372
- "step": 655,
1373
- "train_runtime": 3179.1862,
1374
- "train_tokens_per_second": 2524.686
1375
- },
1376
- {
1377
- "epoch": 3.0,
1378
- "grad_norm": 0.43112221360206604,
1379
- "learning_rate": 4.8386043058735465e-05,
1380
- "loss": 0.1645,
1381
- "num_input_tokens_seen": 8078224,
1382
- "step": 660,
1383
- "train_runtime": 3198.3837,
1384
- "train_tokens_per_second": 2525.721
1385
- },
1386
- {
1387
- "epoch": 3.022805017103763,
1388
- "grad_norm": 0.38409358263015747,
1389
- "learning_rate": 4.63406026519703e-05,
1390
- "loss": 0.1556,
1391
- "num_input_tokens_seen": 8140176,
1392
- "step": 665,
1393
- "train_runtime": 3221.2903,
1394
- "train_tokens_per_second": 2526.992
1395
- },
1396
- {
1397
- "epoch": 3.0456100342075256,
1398
- "grad_norm": 0.2992386221885681,
1399
- "learning_rate": 4.4331409674040324e-05,
1400
- "loss": 0.1397,
1401
- "num_input_tokens_seen": 8201616,
1402
- "step": 670,
1403
- "train_runtime": 3244.0794,
1404
- "train_tokens_per_second": 2528.18
1405
- },
1406
- {
1407
- "epoch": 3.0684150513112884,
1408
- "grad_norm": 0.37943175435066223,
1409
- "learning_rate": 4.23591666958782e-05,
1410
- "loss": 0.1478,
1411
- "num_input_tokens_seen": 8262928,
1412
- "step": 675,
1413
- "train_runtime": 3266.8349,
1414
- "train_tokens_per_second": 2529.338
1415
- },
1416
- {
1417
- "epoch": 3.0912200684150513,
1418
- "grad_norm": 0.31016451120376587,
1419
- "learning_rate": 4.042456336780837e-05,
1420
- "loss": 0.1413,
1421
- "num_input_tokens_seen": 8324496,
1422
- "step": 680,
1423
- "train_runtime": 3289.573,
1424
- "train_tokens_per_second": 2530.57
1425
- },
1426
- {
1427
- "epoch": 3.114025085518814,
1428
- "grad_norm": 0.3854951858520508,
1429
- "learning_rate": 3.852827617839084e-05,
1430
- "loss": 0.1494,
1431
- "num_input_tokens_seen": 8385680,
1432
- "step": 685,
1433
- "train_runtime": 3312.3405,
1434
- "train_tokens_per_second": 2531.648
1435
- },
1436
- {
1437
- "epoch": 3.136830102622577,
1438
- "grad_norm": 0.36697664856910706,
1439
- "learning_rate": 3.667096821786847e-05,
1440
- "loss": 0.1426,
1441
- "num_input_tokens_seen": 8447632,
1442
- "step": 690,
1443
- "train_runtime": 3335.3977,
1444
- "train_tokens_per_second": 2532.721
1445
- },
1446
- {
1447
- "epoch": 3.1596351197263397,
1448
- "grad_norm": 0.2914251983165741,
1449
- "learning_rate": 3.485328894629833e-05,
1450
- "loss": 0.1413,
1451
- "num_input_tokens_seen": 8508432,
1452
- "step": 695,
1453
- "train_runtime": 3357.9702,
1454
- "train_tokens_per_second": 2533.802
1455
- },
1456
- {
1457
- "epoch": 3.1824401368301025,
1458
- "grad_norm": 0.3913465440273285,
1459
- "learning_rate": 3.307587396644996e-05,
1460
- "loss": 0.1439,
1461
- "num_input_tokens_seen": 8570128,
1462
- "step": 700,
1463
- "train_runtime": 3380.7711,
1464
- "train_tokens_per_second": 2534.963
1465
- },
1466
- {
1467
- "epoch": 3.1824401368301025,
1468
- "eval_loss": 0.23035594820976257,
1469
- "eval_runtime": 32.2504,
1470
- "eval_samples_per_second": 96.681,
1471
- "eval_steps_per_second": 6.046,
1472
- "num_input_tokens_seen": 8570128,
1473
- "step": 700
1474
- },
1475
- {
1476
- "epoch": 3.2052451539338653,
1477
- "grad_norm": 0.3519875705242157,
1478
- "learning_rate": 3.133934480154885e-05,
1479
- "loss": 0.1474,
1480
- "num_input_tokens_seen": 8631952,
1481
- "step": 705,
1482
- "train_runtime": 3435.8884,
1483
- "train_tokens_per_second": 2512.291
1484
- },
1485
- {
1486
- "epoch": 3.228050171037628,
1487
- "grad_norm": 0.2907137870788574,
1488
- "learning_rate": 2.9644308677943313e-05,
1489
- "loss": 0.1453,
1490
- "num_input_tokens_seen": 8694032,
1491
- "step": 710,
1492
- "train_runtime": 3458.7963,
1493
- "train_tokens_per_second": 2513.6
1494
- },
1495
- {
1496
- "epoch": 3.250855188141391,
1497
- "grad_norm": 0.4542185962200165,
1498
- "learning_rate": 2.7991358312770805e-05,
1499
- "loss": 0.1379,
1500
- "num_input_tokens_seen": 8754576,
1501
- "step": 715,
1502
- "train_runtime": 3481.3102,
1503
- "train_tokens_per_second": 2514.736
1504
- },
1505
- {
1506
- "epoch": 3.2736602052451538,
1507
- "grad_norm": 0.3293727934360504,
1508
- "learning_rate": 2.6381071706697644e-05,
1509
- "loss": 0.1441,
1510
- "num_input_tokens_seen": 8816528,
1511
- "step": 720,
1512
- "train_runtime": 3504.3181,
1513
- "train_tokens_per_second": 2515.904
1514
- },
1515
- {
1516
- "epoch": 3.2964652223489166,
1517
- "grad_norm": 0.3290797173976898,
1518
- "learning_rate": 2.48140119418046e-05,
1519
- "loss": 0.1442,
1520
- "num_input_tokens_seen": 8877200,
1521
- "step": 725,
1522
- "train_runtime": 3526.8441,
1523
- "train_tokens_per_second": 2517.038
1524
- },
1525
- {
1526
- "epoch": 3.3192702394526794,
1527
- "grad_norm": 0.313092976808548,
1528
- "learning_rate": 2.329072698468963e-05,
1529
- "loss": 0.139,
1530
- "num_input_tokens_seen": 8938512,
1531
- "step": 730,
1532
- "train_runtime": 3549.5232,
1533
- "train_tokens_per_second": 2518.229
1534
- },
1535
- {
1536
- "epoch": 3.342075256556442,
1537
- "grad_norm": 0.28016939759254456,
1538
- "learning_rate": 2.1811749494855778e-05,
1539
- "loss": 0.1429,
1540
- "num_input_tokens_seen": 9000464,
1541
- "step": 735,
1542
- "train_runtime": 3572.5169,
1543
- "train_tokens_per_second": 2519.362
1544
- },
1545
- {
1546
- "epoch": 3.364880273660205,
1547
- "grad_norm": 0.3928128480911255,
1548
- "learning_rate": 2.037759663845181e-05,
1549
- "loss": 0.1452,
1550
- "num_input_tokens_seen": 9061648,
1551
- "step": 740,
1552
- "train_runtime": 3595.1996,
1553
- "train_tokens_per_second": 2520.485
1554
- },
1555
- {
1556
- "epoch": 3.387685290763968,
1557
- "grad_norm": 0.3684266209602356,
1558
- "learning_rate": 1.8988769907430552e-05,
1559
- "loss": 0.1392,
1560
- "num_input_tokens_seen": 9122704,
1561
- "step": 745,
1562
- "train_runtime": 3617.8228,
1563
- "train_tokens_per_second": 2521.601
1564
- },
1565
- {
1566
- "epoch": 3.4104903078677307,
1567
- "grad_norm": 0.35439157485961914,
1568
- "learning_rate": 1.7645754944187983e-05,
1569
- "loss": 0.1408,
1570
- "num_input_tokens_seen": 9183504,
1571
- "step": 750,
1572
- "train_runtime": 3640.3859,
1573
- "train_tokens_per_second": 2522.673
1574
- },
1575
- {
1576
- "epoch": 3.433295324971494,
1577
- "grad_norm": 0.3279169797897339,
1578
- "learning_rate": 1.634902137174483e-05,
1579
- "loss": 0.1406,
1580
- "num_input_tokens_seen": 9245072,
1581
- "step": 755,
1582
- "train_runtime": 3663.2554,
1583
- "train_tokens_per_second": 2523.731
1584
- },
1585
- {
1586
- "epoch": 3.4561003420752567,
1587
- "grad_norm": 0.35586997866630554,
1588
- "learning_rate": 1.5099022629529495e-05,
1589
- "loss": 0.1395,
1590
- "num_input_tokens_seen": 9307152,
1591
- "step": 760,
1592
- "train_runtime": 3686.2342,
1593
- "train_tokens_per_second": 2524.84
1594
- },
1595
- {
1596
- "epoch": 3.4789053591790196,
1597
- "grad_norm": 0.35383179783821106,
1598
- "learning_rate": 1.3896195814820266e-05,
1599
- "loss": 0.137,
1600
- "num_input_tokens_seen": 9369616,
1601
- "step": 765,
1602
- "train_runtime": 3709.241,
1603
- "train_tokens_per_second": 2526.02
1604
- },
1605
- {
1606
- "epoch": 3.5017103762827824,
1607
- "grad_norm": 0.32018619775772095,
1608
- "learning_rate": 1.2740961529902027e-05,
1609
- "loss": 0.1337,
1610
- "num_input_tokens_seen": 9431568,
1611
- "step": 770,
1612
- "train_runtime": 3732.2242,
1613
- "train_tokens_per_second": 2527.064
1614
- },
1615
- {
1616
- "epoch": 3.524515393386545,
1617
- "grad_norm": 0.3486245274543762,
1618
- "learning_rate": 1.1633723734990624e-05,
1619
- "loss": 0.1384,
1620
- "num_input_tokens_seen": 9493264,
1621
- "step": 775,
1622
- "train_runtime": 3754.9747,
1623
- "train_tokens_per_second": 2528.183
1624
- },
1625
- {
1626
- "epoch": 3.547320410490308,
1627
- "grad_norm": 0.3063414394855499,
1628
- "learning_rate": 1.0574869606976888e-05,
1629
- "loss": 0.1345,
1630
- "num_input_tokens_seen": 9554448,
1631
- "step": 780,
1632
- "train_runtime": 3777.6485,
1633
- "train_tokens_per_second": 2529.205
1634
- },
1635
- {
1636
- "epoch": 3.570125427594071,
1637
- "grad_norm": 0.38662609457969666,
1638
- "learning_rate": 9.564769404039419e-06,
1639
- "loss": 0.1368,
1640
- "num_input_tokens_seen": 9615760,
1641
- "step": 785,
1642
- "train_runtime": 3800.381,
1643
- "train_tokens_per_second": 2530.209
1644
- },
1645
- {
1646
- "epoch": 3.5929304446978336,
1647
- "grad_norm": 0.3517778813838959,
1648
- "learning_rate": 8.603776336173235e-06,
1649
- "loss": 0.1452,
1650
- "num_input_tokens_seen": 9678608,
1651
- "step": 790,
1652
- "train_runtime": 3823.5086,
1653
- "train_tokens_per_second": 2531.342
1654
- },
1655
- {
1656
- "epoch": 3.6157354618015964,
1657
- "grad_norm": 0.40929150581359863,
1658
- "learning_rate": 7.692226441680038e-06,
1659
- "loss": 0.1413,
1660
- "num_input_tokens_seen": 9740304,
1661
- "step": 795,
1662
- "train_runtime": 3846.3369,
1663
- "train_tokens_per_second": 2532.359
1664
- },
1665
- {
1666
- "epoch": 3.6385404789053593,
1667
- "grad_norm": 0.4051925837993622,
1668
- "learning_rate": 6.830438469662891e-06,
1669
- "loss": 0.1356,
1670
- "num_input_tokens_seen": 9801744,
1671
- "step": 800,
1672
- "train_runtime": 3869.0646,
1673
- "train_tokens_per_second": 2533.363
1674
- },
1675
- {
1676
- "epoch": 3.6385404789053593,
1677
- "eval_loss": 0.22199535369873047,
1678
- "eval_runtime": 32.2759,
1679
- "eval_samples_per_second": 96.604,
1680
- "eval_steps_per_second": 6.042,
1681
- "num_input_tokens_seen": 9801744,
1682
- "step": 800
1683
- },
1684
- {
1685
- "epoch": 3.661345496009122,
1686
- "grad_norm": 0.4790637195110321,
1687
- "learning_rate": 6.018713768566658e-06,
1688
- "loss": 0.1355,
1689
- "num_input_tokens_seen": 9862928,
1690
- "step": 805,
1691
- "train_runtime": 3924.1012,
1692
- "train_tokens_per_second": 2513.423
1693
- },
1694
- {
1695
- "epoch": 3.684150513112885,
1696
- "grad_norm": 0.3185409903526306,
1697
- "learning_rate": 5.257336180803029e-06,
1698
- "loss": 0.1376,
1699
- "num_input_tokens_seen": 9924112,
1700
- "step": 810,
1701
- "train_runtime": 3946.8024,
1702
- "train_tokens_per_second": 2514.469
1703
- },
1704
- {
1705
- "epoch": 3.7069555302166477,
1706
- "grad_norm": 0.36550357937812805,
1707
- "learning_rate": 4.546571943496968e-06,
1708
- "loss": 0.1303,
1709
- "num_input_tokens_seen": 9984784,
1710
- "step": 815,
1711
- "train_runtime": 3969.4002,
1712
- "train_tokens_per_second": 2515.439
1713
- },
1714
- {
1715
- "epoch": 3.7297605473204105,
1716
- "grad_norm": 0.3187613785266876,
1717
- "learning_rate": 3.88666959538953e-06,
1718
- "loss": 0.1405,
1719
- "num_input_tokens_seen": 10046736,
1720
- "step": 820,
1721
- "train_runtime": 3992.2692,
1722
- "train_tokens_per_second": 2516.548
1723
- },
1724
- {
1725
- "epoch": 3.7525655644241733,
1726
- "grad_norm": 0.3294983208179474,
1727
- "learning_rate": 3.2778598899291465e-06,
1728
- "loss": 0.132,
1729
- "num_input_tokens_seen": 10108176,
1730
- "step": 825,
1731
- "train_runtime": 4015.0234,
1732
- "train_tokens_per_second": 2517.588
1733
- },
1734
- {
1735
- "epoch": 3.775370581527936,
1736
- "grad_norm": 0.32192060351371765,
1737
- "learning_rate": 2.7203557145822398e-06,
1738
- "loss": 0.1326,
1739
- "num_input_tokens_seen": 10169744,
1740
- "step": 830,
1741
- "train_runtime": 4037.9551,
1742
- "train_tokens_per_second": 2518.538
1743
- },
1744
- {
1745
- "epoch": 3.798175598631699,
1746
- "grad_norm": 0.35039815306663513,
1747
- "learning_rate": 2.2143520163911187e-06,
1748
- "loss": 0.1268,
1749
- "num_input_tokens_seen": 10231696,
1750
- "step": 835,
1751
- "train_runtime": 4060.8327,
1752
- "train_tokens_per_second": 2519.605
1753
- },
1754
- {
1755
- "epoch": 3.8209806157354618,
1756
- "grad_norm": 0.3158867359161377,
1757
- "learning_rate": 1.7600257338052593e-06,
1758
- "loss": 0.1341,
1759
- "num_input_tokens_seen": 10292624,
1760
- "step": 840,
1761
- "train_runtime": 4083.4905,
1762
- "train_tokens_per_second": 2520.546
1763
- },
1764
- {
1765
- "epoch": 3.8437856328392246,
1766
- "grad_norm": 0.3523620367050171,
1767
- "learning_rate": 1.3575357348097948e-06,
1768
- "loss": 0.1366,
1769
- "num_input_tokens_seen": 10353040,
1770
- "step": 845,
1771
- "train_runtime": 4105.9585,
1772
- "train_tokens_per_second": 2521.467
1773
- },
1774
- {
1775
- "epoch": 3.8665906499429874,
1776
- "grad_norm": 0.34996289014816284,
1777
- "learning_rate": 1.0070227613728221e-06,
1778
- "loss": 0.1317,
1779
- "num_input_tokens_seen": 10414480,
1780
- "step": 850,
1781
- "train_runtime": 4128.7438,
1782
- "train_tokens_per_second": 2522.433
1783
- },
1784
- {
1785
- "epoch": 3.88939566704675,
1786
- "grad_norm": 0.36095747351646423,
1787
- "learning_rate": 7.086093802310344e-07,
1788
- "loss": 0.1312,
1789
- "num_input_tokens_seen": 10476432,
1790
- "step": 855,
1791
- "train_runtime": 4151.8485,
1792
- "train_tokens_per_second": 2523.317
1793
- },
1794
- {
1795
- "epoch": 3.912200684150513,
1796
- "grad_norm": 0.32904523611068726,
1797
- "learning_rate": 4.623999400308054e-07,
1798
- "loss": 0.1286,
1799
- "num_input_tokens_seen": 10538128,
1800
- "step": 860,
1801
- "train_runtime": 4174.6954,
1802
- "train_tokens_per_second": 2524.287
1803
- },
1804
- {
1805
- "epoch": 3.935005701254276,
1806
- "grad_norm": 0.31018680334091187,
1807
- "learning_rate": 2.6848053483972677e-07,
1808
- "loss": 0.1279,
1809
- "num_input_tokens_seen": 10600080,
1810
- "step": 865,
1811
- "train_runtime": 4197.5751,
1812
- "train_tokens_per_second": 2525.287
1813
- },
1814
- {
1815
- "epoch": 3.9578107183580387,
1816
- "grad_norm": 0.360738605260849,
1817
- "learning_rate": 1.2691897404140671e-07,
1818
- "loss": 0.1374,
1819
- "num_input_tokens_seen": 10660752,
1820
- "step": 870,
1821
- "train_runtime": 4220.1428,
1822
- "train_tokens_per_second": 2526.159
1823
- },
1824
- {
1825
- "epoch": 3.9806157354618015,
1826
- "grad_norm": 0.38618308305740356,
1827
- "learning_rate": 3.77647586240204e-08,
1828
- "loss": 0.1347,
1829
- "num_input_tokens_seen": 10721680,
1830
- "step": 875,
1831
- "train_runtime": 4242.7093,
1832
- "train_tokens_per_second": 2527.083
1833
- },
1834
- {
1835
- "epoch": 4.0,
1836
- "grad_norm": 0.6333906650543213,
1837
- "learning_rate": 1.0490638708393616e-09,
1838
- "loss": 0.1323,
1839
- "num_input_tokens_seen": 10772736,
1840
- "step": 880,
1841
- "train_runtime": 4261.7116,
1842
- "train_tokens_per_second": 2527.796
1843
- },
1844
- {
1845
- "epoch": 4.0,
1846
- "num_input_tokens_seen": 10772736,
1847
- "step": 880,
1848
- "total_flos": 4.374108802866217e+17,
1849
- "train_loss": 0.3781262902373617,
1850
- "train_runtime": 4269.5185,
1851
- "train_samples_per_second": 26.287,
1852
- "train_steps_per_second": 0.206
1853
  }
1854
  ],
1855
  "logging_steps": 5,
1856
- "max_steps": 880,
1857
- "num_input_tokens_seen": 10772736,
1858
- "num_train_epochs": 4,
1859
  "save_steps": 500,
1860
  "stateful_callbacks": {
1861
  "TrainerControl": {
@@ -1869,7 +495,7 @@
1869
  "attributes": {}
1870
  }
1871
  },
1872
- "total_flos": 4.374108802866217e+17,
1873
  "train_batch_size": 16,
1874
  "trial_name": null,
1875
  "trial_params": null
 
2
  "best_global_step": null,
3
  "best_metric": null,
4
  "best_model_checkpoint": null,
5
+ "epoch": 1.0,
6
  "eval_steps": 100,
7
+ "global_step": 220,
8
  "is_hyper_param_search": false,
9
  "is_local_process_zero": true,
10
  "is_world_process_zero": true,
 
16
  "loss": 12.6644,
17
  "num_input_tokens_seen": 61696,
18
  "step": 5,
19
+ "train_runtime": 24.3804,
20
+ "train_tokens_per_second": 2530.557
21
  },
22
  {
23
  "epoch": 0.04561003420752566,
 
26
  "loss": 5.0641,
27
  "num_input_tokens_seen": 123136,
28
  "step": 10,
29
+ "train_runtime": 47.2224,
30
+ "train_tokens_per_second": 2607.578
31
  },
32
  {
33
  "epoch": 0.06841505131128849,
 
36
  "loss": 1.5053,
37
  "num_input_tokens_seen": 184832,
38
  "step": 15,
39
+ "train_runtime": 70.2288,
40
+ "train_tokens_per_second": 2631.856
41
  },
42
  {
43
  "epoch": 0.09122006841505131,
 
46
  "loss": 0.6917,
47
  "num_input_tokens_seen": 247552,
48
  "step": 20,
49
+ "train_runtime": 93.5022,
50
+ "train_tokens_per_second": 2647.553
51
  },
52
  {
53
  "epoch": 0.11402508551881414,
 
56
  "loss": 0.5721,
57
  "num_input_tokens_seen": 308992,
58
  "step": 25,
59
+ "train_runtime": 116.3387,
60
+ "train_tokens_per_second": 2655.97
61
  },
62
  {
63
  "epoch": 0.13683010262257697,
 
66
  "loss": 0.6254,
67
  "num_input_tokens_seen": 370176,
68
  "step": 30,
69
+ "train_runtime": 139.2197,
70
+ "train_tokens_per_second": 2658.935
71
  },
72
  {
73
  "epoch": 0.15963511972633979,
 
76
  "loss": 0.6389,
77
  "num_input_tokens_seen": 430720,
78
  "step": 35,
79
+ "train_runtime": 161.8821,
80
+ "train_tokens_per_second": 2660.702
81
  },
82
  {
83
  "epoch": 0.18244013683010263,
 
86
  "loss": 0.5258,
87
  "num_input_tokens_seen": 492800,
88
  "step": 40,
89
+ "train_runtime": 185.2921,
90
+ "train_tokens_per_second": 2659.584
91
  },
92
  {
93
  "epoch": 0.20524515393386544,
94
+ "grad_norm": 6.443251132965088,
95
+ "learning_rate": 0.0002996346075389736,
96
+ "loss": 0.5041,
97
  "num_input_tokens_seen": 553984,
98
  "step": 45,
99
+ "train_runtime": 208.2146,
100
+ "train_tokens_per_second": 2660.639
101
  },
102
  {
103
  "epoch": 0.22805017103762829,
104
+ "grad_norm": 0.32593730092048645,
105
+ "learning_rate": 0.00029815325108927063,
106
+ "loss": 0.4831,
107
  "num_input_tokens_seen": 615040,
108
  "step": 50,
109
+ "train_runtime": 231.0893,
110
+ "train_tokens_per_second": 2661.482
111
  },
112
  {
113
  "epoch": 0.2508551881413911,
114
+ "grad_norm": 0.46452033519744873,
115
+ "learning_rate": 0.0002955443589413994,
116
+ "loss": 0.4957,
117
  "num_input_tokens_seen": 676736,
118
  "step": 55,
119
+ "train_runtime": 254.0788,
120
+ "train_tokens_per_second": 2663.488
121
  },
122
  {
123
  "epoch": 0.27366020524515394,
124
+ "grad_norm": 0.26457586884498596,
125
+ "learning_rate": 0.00029182778633989753,
126
+ "loss": 0.4764,
127
  "num_input_tokens_seen": 738176,
128
  "step": 60,
129
+ "train_runtime": 277.0778,
130
+ "train_tokens_per_second": 2664.147
131
  },
132
  {
133
  "epoch": 0.29646522234891676,
134
+ "grad_norm": 0.14807738363742828,
135
+ "learning_rate": 0.0002870318186463901,
136
+ "loss": 0.4829,
137
  "num_input_tokens_seen": 799488,
138
  "step": 65,
139
+ "train_runtime": 300.021,
140
+ "train_tokens_per_second": 2664.773
141
  },
142
  {
143
  "epoch": 0.31927023945267957,
144
+ "grad_norm": 0.3857417404651642,
145
+ "learning_rate": 0.00028119295607090933,
146
+ "loss": 0.478,
147
  "num_input_tokens_seen": 861568,
148
  "step": 70,
149
+ "train_runtime": 323.133,
150
+ "train_tokens_per_second": 2666.296
151
  },
152
  {
153
  "epoch": 0.34207525655644244,
154
+ "grad_norm": 0.22548428177833557,
155
+ "learning_rate": 0.0002743556358832562,
156
+ "loss": 0.4771,
157
  "num_input_tokens_seen": 924544,
158
  "step": 75,
159
+ "train_runtime": 346.7417,
160
+ "train_tokens_per_second": 2666.376
161
  },
162
  {
163
  "epoch": 0.36488027366020526,
164
+ "grad_norm": 0.22088338434696198,
165
+ "learning_rate": 0.0002665718942185456,
166
+ "loss": 0.4657,
167
  "num_input_tokens_seen": 985472,
168
  "step": 80,
169
+ "train_runtime": 369.5185,
170
+ "train_tokens_per_second": 2666.908
171
  },
172
  {
173
  "epoch": 0.38768529076396807,
174
+ "grad_norm": 0.3554978668689728,
175
+ "learning_rate": 0.00025790097005079764,
176
+ "loss": 0.4831,
177
  "num_input_tokens_seen": 1046912,
178
  "step": 85,
179
+ "train_runtime": 392.5346,
180
+ "train_tokens_per_second": 2667.056
181
  },
182
  {
183
  "epoch": 0.4104903078677309,
184
+ "grad_norm": 0.131392702460289,
185
+ "learning_rate": 0.0002484088543485761,
186
+ "loss": 0.4778,
187
  "num_input_tokens_seen": 1108992,
188
  "step": 90,
189
+ "train_runtime": 415.6419,
190
+ "train_tokens_per_second": 2668.143
191
  },
192
  {
193
  "epoch": 0.43329532497149376,
194
+ "grad_norm": 0.30744704604148865,
195
+ "learning_rate": 0.00023816778784387094,
196
+ "loss": 0.455,
197
  "num_input_tokens_seen": 1170048,
198
  "step": 95,
199
+ "train_runtime": 438.5167,
200
+ "train_tokens_per_second": 2668.195
201
  },
202
  {
203
  "epoch": 0.45610034207525657,
204
+ "grad_norm": 0.18866093456745148,
205
+ "learning_rate": 0.00022725571123650813,
206
  "loss": 0.4571,
207
  "num_input_tokens_seen": 1230464,
208
  "step": 100,
209
+ "train_runtime": 461.123,
210
+ "train_tokens_per_second": 2668.407
211
  },
212
  {
213
  "epoch": 0.45610034207525657,
214
+ "eval_loss": 0.4646710157394409,
215
+ "eval_runtime": 32.6532,
216
+ "eval_samples_per_second": 95.488,
217
+ "eval_steps_per_second": 5.972,
218
  "num_input_tokens_seen": 1230464,
219
  "step": 100
220
  },
221
  {
222
  "epoch": 0.4789053591790194,
223
+ "grad_norm": 0.23965908586978912,
224
+ "learning_rate": 0.0002157556720183616,
225
+ "loss": 0.4667,
226
  "num_input_tokens_seen": 1292288,
227
  "step": 105,
228
+ "train_runtime": 516.8377,
229
+ "train_tokens_per_second": 2500.375
230
  },
231
  {
232
  "epoch": 0.5017103762827823,
233
+ "grad_norm": 0.25870010256767273,
234
+ "learning_rate": 0.000203755192431795,
235
+ "loss": 0.4758,
236
  "num_input_tokens_seen": 1353344,
237
  "step": 110,
238
+ "train_runtime": 539.6435,
239
+ "train_tokens_per_second": 2507.848
240
  },
241
  {
242
  "epoch": 0.5245153933865451,
243
+ "grad_norm": 0.26426610350608826,
244
+ "learning_rate": 0.00019134560337254986,
245
+ "loss": 0.4783,
246
  "num_input_tokens_seen": 1415040,
247
  "step": 115,
248
+ "train_runtime": 562.5569,
249
+ "train_tokens_per_second": 2515.372
250
  },
251
  {
252
  "epoch": 0.5473204104903079,
253
+ "grad_norm": 0.22881975769996643,
254
+ "learning_rate": 0.0001786213493064817,
255
+ "loss": 0.4643,
256
  "num_input_tokens_seen": 1476480,
257
  "step": 120,
258
+ "train_runtime": 585.45,
259
+ "train_tokens_per_second": 2521.958
260
  },
261
  {
262
  "epoch": 0.5701254275940707,
263
+ "grad_norm": 0.2016284018754959,
264
+ "learning_rate": 0.000165679269490148,
265
+ "loss": 0.4542,
266
  "num_input_tokens_seen": 1537664,
267
  "step": 125,
268
+ "train_runtime": 608.3455,
269
+ "train_tokens_per_second": 2527.616
270
  },
271
  {
272
  "epoch": 0.5929304446978335,
273
+ "grad_norm": 0.1549897938966751,
274
+ "learning_rate": 0.00015261786096559254,
275
+ "loss": 0.4539,
276
  "num_input_tokens_seen": 1598848,
277
  "step": 130,
278
+ "train_runtime": 631.1849,
279
+ "train_tokens_per_second": 2533.09
280
  },
281
  {
282
  "epoch": 0.6157354618015963,
283
+ "grad_norm": 0.4272724986076355,
284
+ "learning_rate": 0.00013953652893838119,
285
+ "loss": 0.4563,
286
  "num_input_tokens_seen": 1660800,
287
  "step": 135,
288
+ "train_runtime": 654.1659,
289
+ "train_tokens_per_second": 2538.806
290
  },
291
  {
292
  "epoch": 0.6385404789053591,
293
+ "grad_norm": 0.2291804850101471,
294
+ "learning_rate": 0.00012653483024396533,
295
+ "loss": 0.4434,
296
  "num_input_tokens_seen": 1721600,
297
  "step": 140,
298
+ "train_runtime": 676.8974,
299
+ "train_tokens_per_second": 2543.369
300
  },
301
  {
302
  "epoch": 0.661345496009122,
303
+ "grad_norm": 0.3113957345485687,
304
+ "learning_rate": 0.00011371171566004985,
305
+ "loss": 0.4484,
306
  "num_input_tokens_seen": 1783168,
307
  "step": 145,
308
+ "train_runtime": 699.8482,
309
+ "train_tokens_per_second": 2547.935
310
  },
311
  {
312
  "epoch": 0.6841505131128849,
313
+ "grad_norm": 0.25940707325935364,
314
+ "learning_rate": 0.00010116477683142652,
315
+ "loss": 0.4314,
316
  "num_input_tokens_seen": 1844992,
317
  "step": 150,
318
+ "train_runtime": 722.8201,
319
+ "train_tokens_per_second": 2552.491
320
  },
321
  {
322
  "epoch": 0.7069555302166477,
323
+ "grad_norm": 0.28187158703804016,
324
+ "learning_rate": 8.898950353862998e-05,
325
+ "loss": 0.4211,
326
  "num_input_tokens_seen": 1906048,
327
  "step": 155,
328
+ "train_runtime": 745.621,
329
+ "train_tokens_per_second": 2556.323
330
  },
331
  {
332
  "epoch": 0.7297605473204105,
333
+ "grad_norm": 0.317065954208374,
334
+ "learning_rate": 7.727855696304944e-05,
335
+ "loss": 0.4324,
336
  "num_input_tokens_seen": 1967744,
337
  "step": 160,
338
+ "train_runtime": 768.5694,
339
+ "train_tokens_per_second": 2560.268
340
  },
341
  {
342
  "epoch": 0.7525655644241733,
343
+ "grad_norm": 0.2838670313358307,
344
+ "learning_rate": 6.612106447938799e-05,
345
+ "loss": 0.4093,
346
  "num_input_tokens_seen": 2028032,
347
  "step": 165,
348
+ "train_runtime": 791.1533,
349
+ "train_tokens_per_second": 2563.387
350
  },
351
  {
352
  "epoch": 0.7753705815279361,
353
+ "grad_norm": 0.29432976245880127,
354
+ "learning_rate": 5.56019413425244e-05,
355
+ "loss": 0.4113,
356
  "num_input_tokens_seen": 2088448,
357
  "step": 170,
358
+ "train_runtime": 813.7398,
359
+ "train_tokens_per_second": 2566.481
360
  },
361
  {
362
  "epoch": 0.798175598631699,
363
+ "grad_norm": 0.2523755133152008,
364
+ "learning_rate": 4.5801244431150394e-05,
365
+ "loss": 0.4142,
366
  "num_input_tokens_seen": 2150144,
367
  "step": 175,
368
+ "train_runtime": 836.6712,
369
+ "train_tokens_per_second": 2569.879
370
  },
371
  {
372
  "epoch": 0.8209806157354618,
373
+ "grad_norm": 0.25406619906425476,
374
+ "learning_rate": 3.6793562966584196e-05,
375
+ "loss": 0.407,
376
  "num_input_tokens_seen": 2211584,
377
  "step": 180,
378
+ "train_runtime": 859.6283,
379
+ "train_tokens_per_second": 2572.721
380
  },
381
  {
382
  "epoch": 0.8437856328392246,
383
+ "grad_norm": 0.35335877537727356,
384
+ "learning_rate": 2.8647450843757897e-05,
385
+ "loss": 0.3836,
386
  "num_input_tokens_seen": 2272256,
387
  "step": 185,
388
+ "train_runtime": 882.269,
389
+ "train_tokens_per_second": 2575.469
390
  },
391
  {
392
  "epoch": 0.8665906499429875,
393
+ "grad_norm": 0.3066641688346863,
394
+ "learning_rate": 2.1424904894683165e-05,
395
+ "loss": 0.3904,
396
  "num_input_tokens_seen": 2333696,
397
  "step": 190,
398
+ "train_runtime": 905.2082,
399
+ "train_tokens_per_second": 2578.076
400
  },
401
  {
402
  "epoch": 0.8893956670467503,
403
+ "grad_norm": 0.3389071524143219,
404
+ "learning_rate": 1.5180893055124977e-05,
405
+ "loss": 0.4011,
406
  "num_input_tokens_seen": 2394880,
407
  "step": 195,
408
+ "train_runtime": 928.0104,
409
+ "train_tokens_per_second": 2580.661
410
  },
411
  {
412
  "epoch": 0.9122006841505131,
413
+ "grad_norm": 0.3012617826461792,
414
+ "learning_rate": 9.962936025419754e-06,
415
+ "loss": 0.3809,
416
  "num_input_tokens_seen": 2455680,
417
  "step": 200,
418
+ "train_runtime": 950.7291,
419
+ "train_tokens_per_second": 2582.944
420
  },
421
  {
422
  "epoch": 0.9122006841505131,
423
+ "eval_loss": 0.3808976411819458,
424
+ "eval_runtime": 32.6975,
425
+ "eval_samples_per_second": 95.359,
426
+ "eval_steps_per_second": 5.964,
427
  "num_input_tokens_seen": 2455680,
428
  "step": 200
429
  },
430
  {
431
  "epoch": 0.935005701254276,
432
+ "grad_norm": 0.2728487253189087,
433
+ "learning_rate": 5.810745609252165e-06,
434
+ "loss": 0.3799,
435
  "num_input_tokens_seen": 2517376,
436
  "step": 205,
437
+ "train_runtime": 1006.4425,
438
+ "train_tokens_per_second": 2501.262
439
  },
440
  {
441
  "epoch": 0.9578107183580388,
442
+ "grad_norm": 0.29773786664009094,
443
+ "learning_rate": 2.7559224828504035e-06,
444
+ "loss": 0.3944,
445
  "num_input_tokens_seen": 2578816,
446
  "step": 210,
447
+ "train_runtime": 1029.2761,
448
+ "train_tokens_per_second": 2505.466
449
  },
450
  {
451
  "epoch": 0.9806157354618016,
452
+ "grad_norm": 0.3076622486114502,
453
+ "learning_rate": 8.217156947590064e-07,
454
+ "loss": 0.3721,
455
  "num_input_tokens_seen": 2640128,
456
  "step": 215,
457
+ "train_runtime": 1052.1692,
458
+ "train_tokens_per_second": 2509.224
459
  },
460
  {
461
  "epoch": 1.0,
462
+ "grad_norm": 0.5157559514045715,
463
+ "learning_rate": 2.284572654130956e-08,
464
+ "loss": 0.3749,
465
  "num_input_tokens_seen": 2691984,
466
  "step": 220,
467
+ "train_runtime": 1071.5462,
468
+ "train_tokens_per_second": 2512.243
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
469
  },
470
  {
471
+ "epoch": 1.0,
472
+ "num_input_tokens_seen": 2691984,
473
+ "step": 220,
474
+ "total_flos": 1.0930399586117222e+17,
475
+ "train_loss": 0.8674792235547846,
476
+ "train_runtime": 1074.777,
477
+ "train_samples_per_second": 26.106,
478
+ "train_steps_per_second": 0.205
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
479
  }
480
  ],
481
  "logging_steps": 5,
482
+ "max_steps": 220,
483
+ "num_input_tokens_seen": 2691984,
484
+ "num_train_epochs": 1,
485
  "save_steps": 500,
486
  "stateful_callbacks": {
487
  "TrainerControl": {
 
495
  "attributes": {}
496
  }
497
  },
498
+ "total_flos": 1.0930399586117222e+17,
499
  "train_batch_size": 16,
500
  "trial_name": null,
501
  "trial_params": null
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0425f47b47b9ab105b7c8824afc016e2251bd80d7cab8bc3f99c37fc0ad556d0
3
  size 5688
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0191e759e04c2dae0eb6abca0935a54ed2dcd54ff24d48ddcbe50dceeb2725a2
3
  size 5688
training_eval_loss.png CHANGED

Git LFS Details

  • SHA256: 80f97bd76df5ee2b07a41405fa7be0f48f8dd5421f21746a52632ce74929e2de
  • Pointer size: 130 Bytes
  • Size of remote file: 37 kB

Git LFS Details

  • SHA256: fc6aa75b6f165957ebf79c2338fdf1d1111365abedda188251d1734ed7d100df
  • Pointer size: 130 Bytes
  • Size of remote file: 38.1 kB
training_loss.png CHANGED

Git LFS Details

  • SHA256: 20968a56271c76b9926a5f212d3232998076ce779d8dfd1960758c9335851d9f
  • Pointer size: 130 Bytes
  • Size of remote file: 28.7 kB

Git LFS Details

  • SHA256: 621b0538a55776a0655b3c33f071cc312c3c35d096359dde615417cc34c11f6f
  • Pointer size: 130 Bytes
  • Size of remote file: 28 kB