Hugging Face's logo

ErrorAI
/
7ce82059-66f7-4ccd-bc95-18dea2313e1f

PEFT
Safetensors
gpt_neox
axolotl
Generated from Trainer
Model card Files
xet
 Community
ErrorAI commited on
Commit
1f8200b
·
verified ·
1 Parent(s): 265a1d0

Training in progress, step 381, checkpoint

Browse files
Files changed (11) hide show
  1. last-checkpoint/README.md +202 -0
  2. last-checkpoint/adapter_config.json +31 -0
  3. last-checkpoint/adapter_model.safetensors +3 -0
  4. last-checkpoint/optimizer.pt +3 -0
  5. last-checkpoint/rng_state.pth +3 -0
  6. last-checkpoint/scheduler.pt +3 -0
  7. last-checkpoint/special_tokens_map.json +30 -0
  8. last-checkpoint/tokenizer.json +0 -0
  9. last-checkpoint/tokenizer_config.json +223 -0
  10. last-checkpoint/trainer_state.json +2716 -0
  11. last-checkpoint/training_args.bin +3 -0
last-checkpoint/README.md ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: EleutherAI/pythia-160m
3
+ library_name: peft
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
200
+ ### Framework versions
201
+
202
+ - PEFT 0.13.2
last-checkpoint/adapter_config.json ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "EleutherAI/pythia-160m",
5
+ "bias": "none",
6
+ "fan_in_fan_out": null,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layer_replication": null,
10
+ "layers_pattern": null,
11
+ "layers_to_transform": null,
12
+ "loftq_config": {},
13
+ "lora_alpha": 16,
14
+ "lora_dropout": 0.05,
15
+ "megatron_config": null,
16
+ "megatron_core": "megatron.core",
17
+ "modules_to_save": null,
18
+ "peft_type": "LORA",
19
+ "r": 8,
20
+ "rank_pattern": {},
21
+ "revision": null,
22
+ "target_modules": [
23
+ "dense_h_to_4h",
24
+ "query_key_value",
25
+ "dense_4h_to_h",
26
+ "dense"
27
+ ],
28
+ "task_type": "CAUSAL_LM",
29
+ "use_dora": false,
30
+ "use_rslora": false
31
+ }
last-checkpoint/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f5ae3eda2d55a6422b7c2c8605e25f7d9e097c4ae0eda7c88668f66b13b0dd4
3
+ size 4731640
last-checkpoint/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a7f0a2e425aaa2feafa6f489a398fbb98b5ec54c5415ab5448e58b394efce029
3
+ size 2505850
last-checkpoint/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d4ee5943d1e441480ca88ae632db7b263f10c34365676ed955a61f8d8e5c08df
3
+ size 14244
last-checkpoint/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:73e0e7a6b479ed5f5f10bf81713902edc810415550be81bd7dca31e35c6bfa33
3
+ size 1064
last-checkpoint/special_tokens_map.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<|endoftext|>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "<|endoftext|>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "<|endoftext|>",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "unk_token": {
24
+ "content": "<|endoftext|>",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ }
30
+ }
last-checkpoint/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
last-checkpoint/tokenizer_config.json ADDED
@@ -0,0 +1,223 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": false,
3
+ "add_eos_token": false,
4
+ "add_prefix_space": false,
5
+ "added_tokens_decoder": {
6
+ "0": {
7
+ "content": "<|endoftext|>",
8
+ "lstrip": false,
9
+ "normalized": false,
10
+ "rstrip": false,
11
+ "single_word": false,
12
+ "special": true
13
+ },
14
+ "1": {
15
+ "content": "<|padding|>",
16
+ "lstrip": false,
17
+ "normalized": false,
18
+ "rstrip": false,
19
+ "single_word": false,
20
+ "special": true
21
+ },
22
+ "50254": {
23
+ "content": " ",
24
+ "lstrip": false,
25
+ "normalized": true,
26
+ "rstrip": false,
27
+ "single_word": false,
28
+ "special": false
29
+ },
30
+ "50255": {
31
+ "content": " ",
32
+ "lstrip": false,
33
+ "normalized": true,
34
+ "rstrip": false,
35
+ "single_word": false,
36
+ "special": false
37
+ },
38
+ "50256": {
39
+ "content": " ",
40
+ "lstrip": false,
41
+ "normalized": true,
42
+ "rstrip": false,
43
+ "single_word": false,
44
+ "special": false
45
+ },
46
+ "50257": {
47
+ "content": " ",
48
+ "lstrip": false,
49
+ "normalized": true,
50
+ "rstrip": false,
51
+ "single_word": false,
52
+ "special": false
53
+ },
54
+ "50258": {
55
+ "content": " ",
56
+ "lstrip": false,
57
+ "normalized": true,
58
+ "rstrip": false,
59
+ "single_word": false,
60
+ "special": false
61
+ },
62
+ "50259": {
63
+ "content": " ",
64
+ "lstrip": false,
65
+ "normalized": true,
66
+ "rstrip": false,
67
+ "single_word": false,
68
+ "special": false
69
+ },
70
+ "50260": {
71
+ "content": " ",
72
+ "lstrip": false,
73
+ "normalized": true,
74
+ "rstrip": false,
75
+ "single_word": false,
76
+ "special": false
77
+ },
78
+ "50261": {
79
+ "content": " ",
80
+ "lstrip": false,
81
+ "normalized": true,
82
+ "rstrip": false,
83
+ "single_word": false,
84
+ "special": false
85
+ },
86
+ "50262": {
87
+ "content": " ",
88
+ "lstrip": false,
89
+ "normalized": true,
90
+ "rstrip": false,
91
+ "single_word": false,
92
+ "special": false
93
+ },
94
+ "50263": {
95
+ "content": " ",
96
+ "lstrip": false,
97
+ "normalized": true,
98
+ "rstrip": false,
99
+ "single_word": false,
100
+ "special": false
101
+ },
102
+ "50264": {
103
+ "content": " ",
104
+ "lstrip": false,
105
+ "normalized": true,
106
+ "rstrip": false,
107
+ "single_word": false,
108
+ "special": false
109
+ },
110
+ "50265": {
111
+ "content": " ",
112
+ "lstrip": false,
113
+ "normalized": true,
114
+ "rstrip": false,
115
+ "single_word": false,
116
+ "special": false
117
+ },
118
+ "50266": {
119
+ "content": " ",
120
+ "lstrip": false,
121
+ "normalized": true,
122
+ "rstrip": false,
123
+ "single_word": false,
124
+ "special": false
125
+ },
126
+ "50267": {
127
+ "content": " ",
128
+ "lstrip": false,
129
+ "normalized": true,
130
+ "rstrip": false,
131
+ "single_word": false,
132
+ "special": false
133
+ },
134
+ "50268": {
135
+ "content": " ",
136
+ "lstrip": false,
137
+ "normalized": true,
138
+ "rstrip": false,
139
+ "single_word": false,
140
+ "special": false
141
+ },
142
+ "50269": {
143
+ "content": " ",
144
+ "lstrip": false,
145
+ "normalized": true,
146
+ "rstrip": false,
147
+ "single_word": false,
148
+ "special": false
149
+ },
150
+ "50270": {
151
+ "content": " ",
152
+ "lstrip": false,
153
+ "normalized": true,
154
+ "rstrip": false,
155
+ "single_word": false,
156
+ "special": false
157
+ },
158
+ "50271": {
159
+ "content": " ",
160
+ "lstrip": false,
161
+ "normalized": true,
162
+ "rstrip": false,
163
+ "single_word": false,
164
+ "special": false
165
+ },
166
+ "50272": {
167
+ "content": " ",
168
+ "lstrip": false,
169
+ "normalized": true,
170
+ "rstrip": false,
171
+ "single_word": false,
172
+ "special": false
173
+ },
174
+ "50273": {
175
+ "content": " ",
176
+ "lstrip": false,
177
+ "normalized": true,
178
+ "rstrip": false,
179
+ "single_word": false,
180
+ "special": false
181
+ },
182
+ "50274": {
183
+ "content": " ",
184
+ "lstrip": false,
185
+ "normalized": true,
186
+ "rstrip": false,
187
+ "single_word": false,
188
+ "special": false
189
+ },
190
+ "50275": {
191
+ "content": " ",
192
+ "lstrip": false,
193
+ "normalized": true,
194
+ "rstrip": false,
195
+ "single_word": false,
196
+ "special": false
197
+ },
198
+ "50276": {
199
+ "content": " ",
200
+ "lstrip": false,
201
+ "normalized": true,
202
+ "rstrip": false,
203
+ "single_word": false,
204
+ "special": false
205
+ },
206
+ "50277": {
207
+ "content": "[PAD]",
208
+ "lstrip": false,
209
+ "normalized": false,
210
+ "rstrip": false,
211
+ "single_word": false,
212
+ "special": true
213
+ }
214
+ },
215
+ "bos_token": "<|endoftext|>",
216
+ "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
217
+ "clean_up_tokenization_spaces": false,
218
+ "eos_token": "<|endoftext|>",
219
+ "model_max_length": 1000000000000000019884624838656,
220
+ "pad_token": "<|endoftext|>",
221
+ "tokenizer_class": "GPTNeoXTokenizer",
222
+ "unk_token": "<|endoftext|>"
223
+ }
last-checkpoint/trainer_state.json ADDED
@@ -0,0 +1,2716 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 0.21656956089242574,
5
+ "eval_steps": 381,
6
+ "global_step": 381,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.0005684240443370755,
13
+ "grad_norm": 715.1505126953125,
14
+ "learning_rate": 2e-05,
15
+ "loss": 11.566,
16
+ "step": 1
17
+ },
18
+ {
19
+ "epoch": 0.0005684240443370755,
20
+ "eval_loss": 2.825512170791626,
21
+ "eval_runtime": 7.1017,
22
+ "eval_samples_per_second": 104.341,
23
+ "eval_steps_per_second": 52.241,
24
+ "step": 1
25
+ },
26
+ {
27
+ "epoch": 0.001136848088674151,
28
+ "grad_norm": 2585.229248046875,
29
+ "learning_rate": 4e-05,
30
+ "loss": 10.0057,
31
+ "step": 2
32
+ },
33
+ {
34
+ "epoch": 0.0017052721330112265,
35
+ "grad_norm": 20126.2734375,
36
+ "learning_rate": 6e-05,
37
+ "loss": 12.3752,
38
+ "step": 3
39
+ },
40
+ {
41
+ "epoch": 0.002273696177348302,
42
+ "grad_norm": 1895.610595703125,
43
+ "learning_rate": 8e-05,
44
+ "loss": 12.4936,
45
+ "step": 4
46
+ },
47
+ {
48
+ "epoch": 0.002842120221685377,
49
+ "grad_norm": 984.6260986328125,
50
+ "learning_rate": 0.0001,
51
+ "loss": 11.7755,
52
+ "step": 5
53
+ },
54
+ {
55
+ "epoch": 0.003410544266022453,
56
+ "grad_norm": 615.008544921875,
57
+ "learning_rate": 0.00012,
58
+ "loss": 12.9264,
59
+ "step": 6
60
+ },
61
+ {
62
+ "epoch": 0.003978968310359528,
63
+ "grad_norm": 996.290771484375,
64
+ "learning_rate": 0.00014,
65
+ "loss": 10.8169,
66
+ "step": 7
67
+ },
68
+ {
69
+ "epoch": 0.004547392354696604,
70
+ "grad_norm": 2017.6121826171875,
71
+ "learning_rate": 0.00016,
72
+ "loss": 10.3199,
73
+ "step": 8
74
+ },
75
+ {
76
+ "epoch": 0.005115816399033679,
77
+ "grad_norm": 1687.360107421875,
78
+ "learning_rate": 0.00018,
79
+ "loss": 9.0321,
80
+ "step": 9
81
+ },
82
+ {
83
+ "epoch": 0.005684240443370754,
84
+ "grad_norm": 2396.41357421875,
85
+ "learning_rate": 0.0002,
86
+ "loss": 11.7856,
87
+ "step": 10
88
+ },
89
+ {
90
+ "epoch": 0.00625266448770783,
91
+ "grad_norm": 7873.9150390625,
92
+ "learning_rate": 0.00019999978385725403,
93
+ "loss": 12.1593,
94
+ "step": 11
95
+ },
96
+ {
97
+ "epoch": 0.006821088532044906,
98
+ "grad_norm": 1229.39208984375,
99
+ "learning_rate": 0.00019999913542995056,
100
+ "loss": 10.1226,
101
+ "step": 12
102
+ },
103
+ {
104
+ "epoch": 0.007389512576381981,
105
+ "grad_norm": 1588.6375732421875,
106
+ "learning_rate": 0.00019999805472089254,
107
+ "loss": 9.9923,
108
+ "step": 13
109
+ },
110
+ {
111
+ "epoch": 0.007957936620719057,
112
+ "grad_norm": 1996.9405517578125,
113
+ "learning_rate": 0.00019999654173475179,
114
+ "loss": 12.7867,
115
+ "step": 14
116
+ },
117
+ {
118
+ "epoch": 0.008526360665056132,
119
+ "grad_norm": 4578.20703125,
120
+ "learning_rate": 0.00019999459647806873,
121
+ "loss": 9.8249,
122
+ "step": 15
123
+ },
124
+ {
125
+ "epoch": 0.009094784709393207,
126
+ "grad_norm": 1579.720703125,
127
+ "learning_rate": 0.0001999922189592524,
128
+ "loss": 10.6916,
129
+ "step": 16
130
+ },
131
+ {
132
+ "epoch": 0.009663208753730283,
133
+ "grad_norm": 2316.38330078125,
134
+ "learning_rate": 0.00019998940918858042,
135
+ "loss": 10.0,
136
+ "step": 17
137
+ },
138
+ {
139
+ "epoch": 0.010231632798067358,
140
+ "grad_norm": 5936.943359375,
141
+ "learning_rate": 0.0001999861671781991,
142
+ "loss": 8.6157,
143
+ "step": 18
144
+ },
145
+ {
146
+ "epoch": 0.010800056842404433,
147
+ "grad_norm": 605.010986328125,
148
+ "learning_rate": 0.00019998249294212318,
149
+ "loss": 10.2147,
150
+ "step": 19
151
+ },
152
+ {
153
+ "epoch": 0.011368480886741509,
154
+ "grad_norm": 542.8692626953125,
155
+ "learning_rate": 0.0001999783864962358,
156
+ "loss": 11.1432,
157
+ "step": 20
158
+ },
159
+ {
160
+ "epoch": 0.011936904931078584,
161
+ "grad_norm": 1332.5885009765625,
162
+ "learning_rate": 0.0001999738478582886,
163
+ "loss": 10.7122,
164
+ "step": 21
165
+ },
166
+ {
167
+ "epoch": 0.01250532897541566,
168
+ "grad_norm": 1285.6971435546875,
169
+ "learning_rate": 0.00019996887704790134,
170
+ "loss": 10.9664,
171
+ "step": 22
172
+ },
173
+ {
174
+ "epoch": 0.013073753019752736,
175
+ "grad_norm": 2288.237548828125,
176
+ "learning_rate": 0.00019996347408656224,
177
+ "loss": 8.5736,
178
+ "step": 23
179
+ },
180
+ {
181
+ "epoch": 0.013642177064089812,
182
+ "grad_norm": 1243.52392578125,
183
+ "learning_rate": 0.00019995763899762742,
184
+ "loss": 11.2758,
185
+ "step": 24
186
+ },
187
+ {
188
+ "epoch": 0.014210601108426887,
189
+ "grad_norm": 2815.61962890625,
190
+ "learning_rate": 0.0001999513718063212,
191
+ "loss": 10.4599,
192
+ "step": 25
193
+ },
194
+ {
195
+ "epoch": 0.014779025152763962,
196
+ "grad_norm": 1203.433837890625,
197
+ "learning_rate": 0.00019994467253973562,
198
+ "loss": 10.6303,
199
+ "step": 26
200
+ },
201
+ {
202
+ "epoch": 0.015347449197101038,
203
+ "grad_norm": 1196.458740234375,
204
+ "learning_rate": 0.0001999375412268308,
205
+ "loss": 10.9198,
206
+ "step": 27
207
+ },
208
+ {
209
+ "epoch": 0.015915873241438113,
210
+ "grad_norm": 5211.41064453125,
211
+ "learning_rate": 0.00019992997789843427,
212
+ "loss": 11.6272,
213
+ "step": 28
214
+ },
215
+ {
216
+ "epoch": 0.016484297285775187,
217
+ "grad_norm": 1361.6744384765625,
218
+ "learning_rate": 0.00019992198258724118,
219
+ "loss": 11.7488,
220
+ "step": 29
221
+ },
222
+ {
223
+ "epoch": 0.017052721330112264,
224
+ "grad_norm": 2589.90283203125,
225
+ "learning_rate": 0.00019991355532781414,
226
+ "loss": 10.2517,
227
+ "step": 30
228
+ },
229
+ {
230
+ "epoch": 0.01762114537444934,
231
+ "grad_norm": 1039.8492431640625,
232
+ "learning_rate": 0.000199904696156583,
233
+ "loss": 10.7175,
234
+ "step": 31
235
+ },
236
+ {
237
+ "epoch": 0.018189569418786414,
238
+ "grad_norm": 1555.1090087890625,
239
+ "learning_rate": 0.00019989540511184466,
240
+ "loss": 10.5819,
241
+ "step": 32
242
+ },
243
+ {
244
+ "epoch": 0.01875799346312349,
245
+ "grad_norm": 542.172119140625,
246
+ "learning_rate": 0.0001998856822337629,
247
+ "loss": 13.806,
248
+ "step": 33
249
+ },
250
+ {
251
+ "epoch": 0.019326417507460565,
252
+ "grad_norm": 1158.800048828125,
253
+ "learning_rate": 0.00019987552756436835,
254
+ "loss": 10.5162,
255
+ "step": 34
256
+ },
257
+ {
258
+ "epoch": 0.019894841551797642,
259
+ "grad_norm": 1740.593994140625,
260
+ "learning_rate": 0.0001998649411475582,
261
+ "loss": 8.8364,
262
+ "step": 35
263
+ },
264
+ {
265
+ "epoch": 0.020463265596134716,
266
+ "grad_norm": 1305.1563720703125,
267
+ "learning_rate": 0.00019985392302909597,
268
+ "loss": 9.098,
269
+ "step": 36
270
+ },
271
+ {
272
+ "epoch": 0.021031689640471793,
273
+ "grad_norm": 2509.5439453125,
274
+ "learning_rate": 0.00019984247325661135,
275
+ "loss": 11.789,
276
+ "step": 37
277
+ },
278
+ {
279
+ "epoch": 0.021600113684808867,
280
+ "grad_norm": 587.509521484375,
281
+ "learning_rate": 0.00019983059187960012,
282
+ "loss": 11.38,
283
+ "step": 38
284
+ },
285
+ {
286
+ "epoch": 0.022168537729145944,
287
+ "grad_norm": 2486.3291015625,
288
+ "learning_rate": 0.0001998182789494237,
289
+ "loss": 10.0442,
290
+ "step": 39
291
+ },
292
+ {
293
+ "epoch": 0.022736961773483017,
294
+ "grad_norm": 2356.46826171875,
295
+ "learning_rate": 0.0001998055345193091,
296
+ "loss": 11.6395,
297
+ "step": 40
298
+ },
299
+ {
300
+ "epoch": 0.023305385817820094,
301
+ "grad_norm": 1564.4888916015625,
302
+ "learning_rate": 0.00019979235864434864,
303
+ "loss": 11.1636,
304
+ "step": 41
305
+ },
306
+ {
307
+ "epoch": 0.023873809862157168,
308
+ "grad_norm": 1384.5040283203125,
309
+ "learning_rate": 0.00019977875138149975,
310
+ "loss": 9.923,
311
+ "step": 42
312
+ },
313
+ {
314
+ "epoch": 0.024442233906494245,
315
+ "grad_norm": 1887.17578125,
316
+ "learning_rate": 0.0001997647127895846,
317
+ "loss": 9.3615,
318
+ "step": 43
319
+ },
320
+ {
321
+ "epoch": 0.02501065795083132,
322
+ "grad_norm": 728.4740600585938,
323
+ "learning_rate": 0.00019975024292929008,
324
+ "loss": 10.7779,
325
+ "step": 44
326
+ },
327
+ {
328
+ "epoch": 0.025579081995168396,
329
+ "grad_norm": 2763.714111328125,
330
+ "learning_rate": 0.00019973534186316718,
331
+ "loss": 11.3021,
332
+ "step": 45
333
+ },
334
+ {
335
+ "epoch": 0.026147506039505473,
336
+ "grad_norm": 737.8002319335938,
337
+ "learning_rate": 0.00019972000965563113,
338
+ "loss": 10.3222,
339
+ "step": 46
340
+ },
341
+ {
342
+ "epoch": 0.026715930083842546,
343
+ "grad_norm": 1477.6663818359375,
344
+ "learning_rate": 0.00019970424637296078,
345
+ "loss": 10.4799,
346
+ "step": 47
347
+ },
348
+ {
349
+ "epoch": 0.027284354128179623,
350
+ "grad_norm": 1223.7069091796875,
351
+ "learning_rate": 0.00019968805208329856,
352
+ "loss": 9.9983,
353
+ "step": 48
354
+ },
355
+ {
356
+ "epoch": 0.027852778172516697,
357
+ "grad_norm": 3341.6689453125,
358
+ "learning_rate": 0.00019967142685665002,
359
+ "loss": 11.832,
360
+ "step": 49
361
+ },
362
+ {
363
+ "epoch": 0.028421202216853774,
364
+ "grad_norm": 1535.1341552734375,
365
+ "learning_rate": 0.00019965437076488362,
366
+ "loss": 10.3633,
367
+ "step": 50
368
+ },
369
+ {
370
+ "epoch": 0.028989626261190848,
371
+ "grad_norm": 2500.49462890625,
372
+ "learning_rate": 0.00019963688388173034,
373
+ "loss": 9.8745,
374
+ "step": 51
375
+ },
376
+ {
377
+ "epoch": 0.029558050305527925,
378
+ "grad_norm": 2795.143798828125,
379
+ "learning_rate": 0.00019961896628278345,
380
+ "loss": 10.067,
381
+ "step": 52
382
+ },
383
+ {
384
+ "epoch": 0.030126474349865,
385
+ "grad_norm": 4155.7490234375,
386
+ "learning_rate": 0.00019960061804549813,
387
+ "loss": 12.0709,
388
+ "step": 53
389
+ },
390
+ {
391
+ "epoch": 0.030694898394202075,
392
+ "grad_norm": 943.591064453125,
393
+ "learning_rate": 0.00019958183924919114,
394
+ "loss": 11.3457,
395
+ "step": 54
396
+ },
397
+ {
398
+ "epoch": 0.03126332243853915,
399
+ "grad_norm": 1366.348388671875,
400
+ "learning_rate": 0.00019956262997504053,
401
+ "loss": 10.475,
402
+ "step": 55
403
+ },
404
+ {
405
+ "epoch": 0.031831746482876226,
406
+ "grad_norm": 1189.337158203125,
407
+ "learning_rate": 0.00019954299030608514,
408
+ "loss": 11.5766,
409
+ "step": 56
410
+ },
411
+ {
412
+ "epoch": 0.0324001705272133,
413
+ "grad_norm": 1499.7227783203125,
414
+ "learning_rate": 0.00019952292032722447,
415
+ "loss": 9.5228,
416
+ "step": 57
417
+ },
418
+ {
419
+ "epoch": 0.03296859457155037,
420
+ "grad_norm": 969.8496704101562,
421
+ "learning_rate": 0.0001995024201252181,
422
+ "loss": 9.0326,
423
+ "step": 58
424
+ },
425
+ {
426
+ "epoch": 0.033537018615887454,
427
+ "grad_norm": 1969.1591796875,
428
+ "learning_rate": 0.0001994814897886854,
429
+ "loss": 11.9541,
430
+ "step": 59
431
+ },
432
+ {
433
+ "epoch": 0.03410544266022453,
434
+ "grad_norm": 920.8987426757812,
435
+ "learning_rate": 0.0001994601294081053,
436
+ "loss": 11.2321,
437
+ "step": 60
438
+ },
439
+ {
440
+ "epoch": 0.0346738667045616,
441
+ "grad_norm": 820.4376220703125,
442
+ "learning_rate": 0.00019943833907581544,
443
+ "loss": 12.3152,
444
+ "step": 61
445
+ },
446
+ {
447
+ "epoch": 0.03524229074889868,
448
+ "grad_norm": 1614.395751953125,
449
+ "learning_rate": 0.00019941611888601242,
450
+ "loss": 10.611,
451
+ "step": 62
452
+ },
453
+ {
454
+ "epoch": 0.035810714793235755,
455
+ "grad_norm": 1378.6353759765625,
456
+ "learning_rate": 0.0001993934689347508,
457
+ "loss": 11.6645,
458
+ "step": 63
459
+ },
460
+ {
461
+ "epoch": 0.03637913883757283,
462
+ "grad_norm": 1604.013427734375,
463
+ "learning_rate": 0.0001993703893199431,
464
+ "loss": 11.3859,
465
+ "step": 64
466
+ },
467
+ {
468
+ "epoch": 0.0369475628819099,
469
+ "grad_norm": 2112.920654296875,
470
+ "learning_rate": 0.00019934688014135908,
471
+ "loss": 11.2036,
472
+ "step": 65
473
+ },
474
+ {
475
+ "epoch": 0.03751598692624698,
476
+ "grad_norm": 2538.67431640625,
477
+ "learning_rate": 0.0001993229415006256,
478
+ "loss": 11.3158,
479
+ "step": 66
480
+ },
481
+ {
482
+ "epoch": 0.03808441097058406,
483
+ "grad_norm": 2191.82666015625,
484
+ "learning_rate": 0.00019929857350122583,
485
+ "loss": 14.4237,
486
+ "step": 67
487
+ },
488
+ {
489
+ "epoch": 0.03865283501492113,
490
+ "grad_norm": 2429.986572265625,
491
+ "learning_rate": 0.00019927377624849916,
492
+ "loss": 11.1028,
493
+ "step": 68
494
+ },
495
+ {
496
+ "epoch": 0.039221259059258204,
497
+ "grad_norm": 1850.013671875,
498
+ "learning_rate": 0.00019924854984964052,
499
+ "loss": 9.2762,
500
+ "step": 69
501
+ },
502
+ {
503
+ "epoch": 0.039789683103595284,
504
+ "grad_norm": 6456.251953125,
505
+ "learning_rate": 0.00019922289441369993,
506
+ "loss": 10.7683,
507
+ "step": 70
508
+ },
509
+ {
510
+ "epoch": 0.04035810714793236,
511
+ "grad_norm": 986.1968994140625,
512
+ "learning_rate": 0.00019919681005158216,
513
+ "loss": 10.8592,
514
+ "step": 71
515
+ },
516
+ {
517
+ "epoch": 0.04092653119226943,
518
+ "grad_norm": 2998.34423828125,
519
+ "learning_rate": 0.0001991702968760461,
520
+ "loss": 11.0712,
521
+ "step": 72
522
+ },
523
+ {
524
+ "epoch": 0.041494955236606505,
525
+ "grad_norm": 2790.07958984375,
526
+ "learning_rate": 0.00019914335500170434,
527
+ "loss": 12.567,
528
+ "step": 73
529
+ },
530
+ {
531
+ "epoch": 0.042063379280943586,
532
+ "grad_norm": 2455.44140625,
533
+ "learning_rate": 0.00019911598454502275,
534
+ "loss": 12.3398,
535
+ "step": 74
536
+ },
537
+ {
538
+ "epoch": 0.04263180332528066,
539
+ "grad_norm": 6441.51171875,
540
+ "learning_rate": 0.00019908818562431978,
541
+ "loss": 11.666,
542
+ "step": 75
543
+ },
544
+ {
545
+ "epoch": 0.04320022736961773,
546
+ "grad_norm": 3151.3193359375,
547
+ "learning_rate": 0.00019905995835976616,
548
+ "loss": 10.709,
549
+ "step": 76
550
+ },
551
+ {
552
+ "epoch": 0.043768651413954814,
553
+ "grad_norm": 1786.642333984375,
554
+ "learning_rate": 0.00019903130287338426,
555
+ "loss": 13.0289,
556
+ "step": 77
557
+ },
558
+ {
559
+ "epoch": 0.04433707545829189,
560
+ "grad_norm": 2310.83544921875,
561
+ "learning_rate": 0.00019900221928904763,
562
+ "loss": 8.9232,
563
+ "step": 78
564
+ },
565
+ {
566
+ "epoch": 0.04490549950262896,
567
+ "grad_norm": 693.9208374023438,
568
+ "learning_rate": 0.00019897270773248032,
569
+ "loss": 11.5207,
570
+ "step": 79
571
+ },
572
+ {
573
+ "epoch": 0.045473923546966034,
574
+ "grad_norm": 1073.7001953125,
575
+ "learning_rate": 0.00019894276833125652,
576
+ "loss": 12.2091,
577
+ "step": 80
578
+ },
579
+ {
580
+ "epoch": 0.046042347591303115,
581
+ "grad_norm": 1177.6282958984375,
582
+ "learning_rate": 0.00019891240121479997,
583
+ "loss": 11.0831,
584
+ "step": 81
585
+ },
586
+ {
587
+ "epoch": 0.04661077163564019,
588
+ "grad_norm": 1403.3370361328125,
589
+ "learning_rate": 0.00019888160651438327,
590
+ "loss": 10.284,
591
+ "step": 82
592
+ },
593
+ {
594
+ "epoch": 0.04717919567997726,
595
+ "grad_norm": 1599.4073486328125,
596
+ "learning_rate": 0.0001988503843631274,
597
+ "loss": 12.6646,
598
+ "step": 83
599
+ },
600
+ {
601
+ "epoch": 0.047747619724314336,
602
+ "grad_norm": 1667.661865234375,
603
+ "learning_rate": 0.00019881873489600128,
604
+ "loss": 9.1977,
605
+ "step": 84
606
+ },
607
+ {
608
+ "epoch": 0.048316043768651416,
609
+ "grad_norm": 1429.27685546875,
610
+ "learning_rate": 0.00019878665824982087,
611
+ "loss": 10.502,
612
+ "step": 85
613
+ },
614
+ {
615
+ "epoch": 0.04888446781298849,
616
+ "grad_norm": 2474.26513671875,
617
+ "learning_rate": 0.00019875415456324896,
618
+ "loss": 10.6261,
619
+ "step": 86
620
+ },
621
+ {
622
+ "epoch": 0.049452891857325564,
623
+ "grad_norm": 1770.6605224609375,
624
+ "learning_rate": 0.0001987212239767942,
625
+ "loss": 11.8647,
626
+ "step": 87
627
+ },
628
+ {
629
+ "epoch": 0.05002131590166264,
630
+ "grad_norm": 2947.497802734375,
631
+ "learning_rate": 0.00019868786663281072,
632
+ "loss": 12.1916,
633
+ "step": 88
634
+ },
635
+ {
636
+ "epoch": 0.05058973994599972,
637
+ "grad_norm": 1825.549560546875,
638
+ "learning_rate": 0.00019865408267549754,
639
+ "loss": 10.1334,
640
+ "step": 89
641
+ },
642
+ {
643
+ "epoch": 0.05115816399033679,
644
+ "grad_norm": 13938.9267578125,
645
+ "learning_rate": 0.0001986198722508978,
646
+ "loss": 13.4323,
647
+ "step": 90
648
+ },
649
+ {
650
+ "epoch": 0.051726588034673865,
651
+ "grad_norm": 1956.8978271484375,
652
+ "learning_rate": 0.00019858523550689817,
653
+ "loss": 10.9729,
654
+ "step": 91
655
+ },
656
+ {
657
+ "epoch": 0.052295012079010945,
658
+ "grad_norm": 4884.1005859375,
659
+ "learning_rate": 0.00019855017259322826,
660
+ "loss": 11.324,
661
+ "step": 92
662
+ },
663
+ {
664
+ "epoch": 0.05286343612334802,
665
+ "grad_norm": 2630.69775390625,
666
+ "learning_rate": 0.00019851468366146002,
667
+ "loss": 10.6297,
668
+ "step": 93
669
+ },
670
+ {
671
+ "epoch": 0.05343186016768509,
672
+ "grad_norm": 3588.2236328125,
673
+ "learning_rate": 0.0001984787688650069,
674
+ "loss": 12.1773,
675
+ "step": 94
676
+ },
677
+ {
678
+ "epoch": 0.054000284212022166,
679
+ "grad_norm": 1561.0384521484375,
680
+ "learning_rate": 0.00019844242835912335,
681
+ "loss": 14.0535,
682
+ "step": 95
683
+ },
684
+ {
685
+ "epoch": 0.05456870825635925,
686
+ "grad_norm": 2454.306396484375,
687
+ "learning_rate": 0.0001984056623009041,
688
+ "loss": 12.48,
689
+ "step": 96
690
+ },
691
+ {
692
+ "epoch": 0.05513713230069632,
693
+ "grad_norm": 2470.94140625,
694
+ "learning_rate": 0.00019836847084928354,
695
+ "loss": 11.1904,
696
+ "step": 97
697
+ },
698
+ {
699
+ "epoch": 0.055705556345033394,
700
+ "grad_norm": 3758.377685546875,
701
+ "learning_rate": 0.0001983308541650349,
702
+ "loss": 12.9302,
703
+ "step": 98
704
+ },
705
+ {
706
+ "epoch": 0.05627398038937047,
707
+ "grad_norm": 2643.673095703125,
708
+ "learning_rate": 0.0001982928124107696,
709
+ "loss": 11.1229,
710
+ "step": 99
711
+ },
712
+ {
713
+ "epoch": 0.05684240443370755,
714
+ "grad_norm": 1595.1947021484375,
715
+ "learning_rate": 0.0001982543457509367,
716
+ "loss": 10.5307,
717
+ "step": 100
718
+ },
719
+ {
720
+ "epoch": 0.05741082847804462,
721
+ "grad_norm": 4985.3349609375,
722
+ "learning_rate": 0.00019821545435182194,
723
+ "loss": 11.1607,
724
+ "step": 101
725
+ },
726
+ {
727
+ "epoch": 0.057979252522381695,
728
+ "grad_norm": 4426.3662109375,
729
+ "learning_rate": 0.0001981761383815472,
730
+ "loss": 10.5471,
731
+ "step": 102
732
+ },
733
+ {
734
+ "epoch": 0.05854767656671877,
735
+ "grad_norm": 7010.75,
736
+ "learning_rate": 0.00019813639801006971,
737
+ "loss": 12.8988,
738
+ "step": 103
739
+ },
740
+ {
741
+ "epoch": 0.05911610061105585,
742
+ "grad_norm": 4185.978515625,
743
+ "learning_rate": 0.00019809623340918134,
744
+ "loss": 13.8598,
745
+ "step": 104
746
+ },
747
+ {
748
+ "epoch": 0.05968452465539292,
749
+ "grad_norm": 2997.8544921875,
750
+ "learning_rate": 0.00019805564475250785,
751
+ "loss": 12.9365,
752
+ "step": 105
753
+ },
754
+ {
755
+ "epoch": 0.06025294869973,
756
+ "grad_norm": 3403.239990234375,
757
+ "learning_rate": 0.0001980146322155081,
758
+ "loss": 10.4629,
759
+ "step": 106
760
+ },
761
+ {
762
+ "epoch": 0.06082137274406708,
763
+ "grad_norm": 4110.40185546875,
764
+ "learning_rate": 0.00019797319597547334,
765
+ "loss": 10.9108,
766
+ "step": 107
767
+ },
768
+ {
769
+ "epoch": 0.06138979678840415,
770
+ "grad_norm": 3244.458984375,
771
+ "learning_rate": 0.0001979313362115264,
772
+ "loss": 12.3091,
773
+ "step": 108
774
+ },
775
+ {
776
+ "epoch": 0.061958220832741225,
777
+ "grad_norm": 2387.46826171875,
778
+ "learning_rate": 0.00019788905310462102,
779
+ "loss": 11.3503,
780
+ "step": 109
781
+ },
782
+ {
783
+ "epoch": 0.0625266448770783,
784
+ "grad_norm": 3462.8232421875,
785
+ "learning_rate": 0.0001978463468375409,
786
+ "loss": 11.8997,
787
+ "step": 110
788
+ },
789
+ {
790
+ "epoch": 0.06309506892141538,
791
+ "grad_norm": 1626.2708740234375,
792
+ "learning_rate": 0.00019780321759489903,
793
+ "loss": 11.3754,
794
+ "step": 111
795
+ },
796
+ {
797
+ "epoch": 0.06366349296575245,
798
+ "grad_norm": 2304.5087890625,
799
+ "learning_rate": 0.00019775966556313685,
800
+ "loss": 11.9003,
801
+ "step": 112
802
+ },
803
+ {
804
+ "epoch": 0.06423191701008953,
805
+ "grad_norm": 2844.057861328125,
806
+ "learning_rate": 0.00019771569093052354,
807
+ "loss": 11.8291,
808
+ "step": 113
809
+ },
810
+ {
811
+ "epoch": 0.0648003410544266,
812
+ "grad_norm": 2847.722412109375,
813
+ "learning_rate": 0.000197671293887155,
814
+ "loss": 12.119,
815
+ "step": 114
816
+ },
817
+ {
818
+ "epoch": 0.06536876509876367,
819
+ "grad_norm": 7991.02197265625,
820
+ "learning_rate": 0.00019762647462495325,
821
+ "loss": 13.5387,
822
+ "step": 115
823
+ },
824
+ {
825
+ "epoch": 0.06593718914310075,
826
+ "grad_norm": 1973.8907470703125,
827
+ "learning_rate": 0.00019758123333766544,
828
+ "loss": 12.6017,
829
+ "step": 116
830
+ },
831
+ {
832
+ "epoch": 0.06650561318743783,
833
+ "grad_norm": 1864.024658203125,
834
+ "learning_rate": 0.00019753557022086305,
835
+ "loss": 9.0838,
836
+ "step": 117
837
+ },
838
+ {
839
+ "epoch": 0.06707403723177491,
840
+ "grad_norm": 3261.030517578125,
841
+ "learning_rate": 0.00019748948547194118,
842
+ "loss": 13.5888,
843
+ "step": 118
844
+ },
845
+ {
846
+ "epoch": 0.06764246127611198,
847
+ "grad_norm": 3405.32421875,
848
+ "learning_rate": 0.00019744297929011743,
849
+ "loss": 13.0024,
850
+ "step": 119
851
+ },
852
+ {
853
+ "epoch": 0.06821088532044906,
854
+ "grad_norm": 8975.3642578125,
855
+ "learning_rate": 0.00019739605187643137,
856
+ "loss": 9.2995,
857
+ "step": 120
858
+ },
859
+ {
860
+ "epoch": 0.06877930936478613,
861
+ "grad_norm": 1806.4010009765625,
862
+ "learning_rate": 0.00019734870343374335,
863
+ "loss": 10.3708,
864
+ "step": 121
865
+ },
866
+ {
867
+ "epoch": 0.0693477334091232,
868
+ "grad_norm": 1527.980712890625,
869
+ "learning_rate": 0.00019730093416673382,
870
+ "loss": 10.9931,
871
+ "step": 122
872
+ },
873
+ {
874
+ "epoch": 0.06991615745346028,
875
+ "grad_norm": 1843.93115234375,
876
+ "learning_rate": 0.00019725274428190241,
877
+ "loss": 12.0665,
878
+ "step": 123
879
+ },
880
+ {
881
+ "epoch": 0.07048458149779736,
882
+ "grad_norm": 1749.8299560546875,
883
+ "learning_rate": 0.00019720413398756698,
884
+ "loss": 10.4359,
885
+ "step": 124
886
+ },
887
+ {
888
+ "epoch": 0.07105300554213444,
889
+ "grad_norm": 4265.53369140625,
890
+ "learning_rate": 0.0001971551034938628,
891
+ "loss": 13.3879,
892
+ "step": 125
893
+ },
894
+ {
895
+ "epoch": 0.07162142958647151,
896
+ "grad_norm": 2279.584228515625,
897
+ "learning_rate": 0.00019710565301274158,
898
+ "loss": 12.6601,
899
+ "step": 126
900
+ },
901
+ {
902
+ "epoch": 0.07218985363080858,
903
+ "grad_norm": 3952.930908203125,
904
+ "learning_rate": 0.00019705578275797052,
905
+ "loss": 11.6864,
906
+ "step": 127
907
+ },
908
+ {
909
+ "epoch": 0.07275827767514566,
910
+ "grad_norm": 4334.033203125,
911
+ "learning_rate": 0.0001970054929451316,
912
+ "loss": 12.4551,
913
+ "step": 128
914
+ },
915
+ {
916
+ "epoch": 0.07332670171948273,
917
+ "grad_norm": 3982.023681640625,
918
+ "learning_rate": 0.00019695478379162026,
919
+ "loss": 12.2981,
920
+ "step": 129
921
+ },
922
+ {
923
+ "epoch": 0.0738951257638198,
924
+ "grad_norm": 11521.0478515625,
925
+ "learning_rate": 0.00019690365551664488,
926
+ "loss": 9.7671,
927
+ "step": 130
928
+ },
929
+ {
930
+ "epoch": 0.07446354980815688,
931
+ "grad_norm": 10996.0869140625,
932
+ "learning_rate": 0.00019685210834122563,
933
+ "loss": 11.1024,
934
+ "step": 131
935
+ },
936
+ {
937
+ "epoch": 0.07503197385249397,
938
+ "grad_norm": 2001.555419921875,
939
+ "learning_rate": 0.00019680014248819338,
940
+ "loss": 12.6802,
941
+ "step": 132
942
+ },
943
+ {
944
+ "epoch": 0.07560039789683104,
945
+ "grad_norm": 2967.9482421875,
946
+ "learning_rate": 0.00019674775818218903,
947
+ "loss": 11.3207,
948
+ "step": 133
949
+ },
950
+ {
951
+ "epoch": 0.07616882194116811,
952
+ "grad_norm": 2594.755859375,
953
+ "learning_rate": 0.00019669495564966227,
954
+ "loss": 9.5385,
955
+ "step": 134
956
+ },
957
+ {
958
+ "epoch": 0.07673724598550519,
959
+ "grad_norm": 1451.6942138671875,
960
+ "learning_rate": 0.00019664173511887086,
961
+ "loss": 10.3379,
962
+ "step": 135
963
+ },
964
+ {
965
+ "epoch": 0.07730567002984226,
966
+ "grad_norm": 5727.142578125,
967
+ "learning_rate": 0.0001965880968198794,
968
+ "loss": 13.5146,
969
+ "step": 136
970
+ },
971
+ {
972
+ "epoch": 0.07787409407417933,
973
+ "grad_norm": 3225.37744140625,
974
+ "learning_rate": 0.0001965340409845585,
975
+ "loss": 10.1241,
976
+ "step": 137
977
+ },
978
+ {
979
+ "epoch": 0.07844251811851641,
980
+ "grad_norm": 6057.52392578125,
981
+ "learning_rate": 0.00019647956784658367,
982
+ "loss": 10.274,
983
+ "step": 138
984
+ },
985
+ {
986
+ "epoch": 0.0790109421628535,
987
+ "grad_norm": 7593.2138671875,
988
+ "learning_rate": 0.00019642467764143434,
989
+ "loss": 11.1785,
990
+ "step": 139
991
+ },
992
+ {
993
+ "epoch": 0.07957936620719057,
994
+ "grad_norm": 3727.695556640625,
995
+ "learning_rate": 0.000196369370606393,
996
+ "loss": 11.094,
997
+ "step": 140
998
+ },
999
+ {
1000
+ "epoch": 0.08014779025152764,
1001
+ "grad_norm": 4268.923828125,
1002
+ "learning_rate": 0.00019631364698054386,
1003
+ "loss": 10.9563,
1004
+ "step": 141
1005
+ },
1006
+ {
1007
+ "epoch": 0.08071621429586472,
1008
+ "grad_norm": 6838.72265625,
1009
+ "learning_rate": 0.00019625750700477207,
1010
+ "loss": 10.3983,
1011
+ "step": 142
1012
+ },
1013
+ {
1014
+ "epoch": 0.08128463834020179,
1015
+ "grad_norm": 3911.334228515625,
1016
+ "learning_rate": 0.00019620095092176266,
1017
+ "loss": 10.1044,
1018
+ "step": 143
1019
+ },
1020
+ {
1021
+ "epoch": 0.08185306238453886,
1022
+ "grad_norm": 15097.9462890625,
1023
+ "learning_rate": 0.00019614397897599936,
1024
+ "loss": 11.4009,
1025
+ "step": 144
1026
+ },
1027
+ {
1028
+ "epoch": 0.08242148642887594,
1029
+ "grad_norm": 1902.943359375,
1030
+ "learning_rate": 0.00019608659141376355,
1031
+ "loss": 11.3478,
1032
+ "step": 145
1033
+ },
1034
+ {
1035
+ "epoch": 0.08298991047321301,
1036
+ "grad_norm": 2225.294921875,
1037
+ "learning_rate": 0.00019602878848313344,
1038
+ "loss": 9.8102,
1039
+ "step": 146
1040
+ },
1041
+ {
1042
+ "epoch": 0.0835583345175501,
1043
+ "grad_norm": 1152.1954345703125,
1044
+ "learning_rate": 0.00019597057043398264,
1045
+ "loss": 9.0274,
1046
+ "step": 147
1047
+ },
1048
+ {
1049
+ "epoch": 0.08412675856188717,
1050
+ "grad_norm": 2429.097900390625,
1051
+ "learning_rate": 0.00019591193751797935,
1052
+ "loss": 11.1794,
1053
+ "step": 148
1054
+ },
1055
+ {
1056
+ "epoch": 0.08469518260622425,
1057
+ "grad_norm": 3074.980712890625,
1058
+ "learning_rate": 0.00019585288998858518,
1059
+ "loss": 12.4276,
1060
+ "step": 149
1061
+ },
1062
+ {
1063
+ "epoch": 0.08526360665056132,
1064
+ "grad_norm": 1387.078857421875,
1065
+ "learning_rate": 0.00019579342810105402,
1066
+ "loss": 9.9927,
1067
+ "step": 150
1068
+ },
1069
+ {
1070
+ "epoch": 0.08583203069489839,
1071
+ "grad_norm": 4883.7138671875,
1072
+ "learning_rate": 0.00019573355211243093,
1073
+ "loss": 9.9918,
1074
+ "step": 151
1075
+ },
1076
+ {
1077
+ "epoch": 0.08640045473923547,
1078
+ "grad_norm": 3293.441162109375,
1079
+ "learning_rate": 0.00019567326228155124,
1080
+ "loss": 11.9814,
1081
+ "step": 152
1082
+ },
1083
+ {
1084
+ "epoch": 0.08696887878357254,
1085
+ "grad_norm": 2784.984130859375,
1086
+ "learning_rate": 0.000195612558869039,
1087
+ "loss": 10.6432,
1088
+ "step": 153
1089
+ },
1090
+ {
1091
+ "epoch": 0.08753730282790963,
1092
+ "grad_norm": 2429.671875,
1093
+ "learning_rate": 0.00019555144213730637,
1094
+ "loss": 10.9787,
1095
+ "step": 154
1096
+ },
1097
+ {
1098
+ "epoch": 0.0881057268722467,
1099
+ "grad_norm": 1944.4161376953125,
1100
+ "learning_rate": 0.00019548991235055207,
1101
+ "loss": 10.524,
1102
+ "step": 155
1103
+ },
1104
+ {
1105
+ "epoch": 0.08867415091658377,
1106
+ "grad_norm": 2096.28076171875,
1107
+ "learning_rate": 0.00019542796977476044,
1108
+ "loss": 10.0407,
1109
+ "step": 156
1110
+ },
1111
+ {
1112
+ "epoch": 0.08924257496092085,
1113
+ "grad_norm": 7156.40185546875,
1114
+ "learning_rate": 0.00019536561467770028,
1115
+ "loss": 10.0375,
1116
+ "step": 157
1117
+ },
1118
+ {
1119
+ "epoch": 0.08981099900525792,
1120
+ "grad_norm": 6097.43017578125,
1121
+ "learning_rate": 0.00019530284732892356,
1122
+ "loss": 10.4774,
1123
+ "step": 158
1124
+ },
1125
+ {
1126
+ "epoch": 0.090379423049595,
1127
+ "grad_norm": 6408.6826171875,
1128
+ "learning_rate": 0.00019523966799976446,
1129
+ "loss": 10.3953,
1130
+ "step": 159
1131
+ },
1132
+ {
1133
+ "epoch": 0.09094784709393207,
1134
+ "grad_norm": 2025.72216796875,
1135
+ "learning_rate": 0.00019517607696333808,
1136
+ "loss": 12.0608,
1137
+ "step": 160
1138
+ },
1139
+ {
1140
+ "epoch": 0.09151627113826914,
1141
+ "grad_norm": 2097.453857421875,
1142
+ "learning_rate": 0.0001951120744945392,
1143
+ "loss": 10.432,
1144
+ "step": 161
1145
+ },
1146
+ {
1147
+ "epoch": 0.09208469518260623,
1148
+ "grad_norm": 2883.83984375,
1149
+ "learning_rate": 0.00019504766087004126,
1150
+ "loss": 11.6631,
1151
+ "step": 162
1152
+ },
1153
+ {
1154
+ "epoch": 0.0926531192269433,
1155
+ "grad_norm": 9866.7177734375,
1156
+ "learning_rate": 0.00019498283636829497,
1157
+ "loss": 11.6008,
1158
+ "step": 163
1159
+ },
1160
+ {
1161
+ "epoch": 0.09322154327128038,
1162
+ "grad_norm": 1902.9012451171875,
1163
+ "learning_rate": 0.00019491760126952724,
1164
+ "loss": 12.5862,
1165
+ "step": 164
1166
+ },
1167
+ {
1168
+ "epoch": 0.09378996731561745,
1169
+ "grad_norm": 2026.79541015625,
1170
+ "learning_rate": 0.00019485195585573997,
1171
+ "loss": 14.339,
1172
+ "step": 165
1173
+ },
1174
+ {
1175
+ "epoch": 0.09435839135995452,
1176
+ "grad_norm": 7162.0654296875,
1177
+ "learning_rate": 0.00019478590041070875,
1178
+ "loss": 13.1482,
1179
+ "step": 166
1180
+ },
1181
+ {
1182
+ "epoch": 0.0949268154042916,
1183
+ "grad_norm": 2979.805419921875,
1184
+ "learning_rate": 0.00019471943521998165,
1185
+ "loss": 9.2892,
1186
+ "step": 167
1187
+ },
1188
+ {
1189
+ "epoch": 0.09549523944862867,
1190
+ "grad_norm": 4213.1591796875,
1191
+ "learning_rate": 0.00019465256057087812,
1192
+ "loss": 10.9719,
1193
+ "step": 168
1194
+ },
1195
+ {
1196
+ "epoch": 0.09606366349296576,
1197
+ "grad_norm": 6381.18310546875,
1198
+ "learning_rate": 0.0001945852767524875,
1199
+ "loss": 11.0731,
1200
+ "step": 169
1201
+ },
1202
+ {
1203
+ "epoch": 0.09663208753730283,
1204
+ "grad_norm": 5097.34912109375,
1205
+ "learning_rate": 0.00019451758405566801,
1206
+ "loss": 12.5336,
1207
+ "step": 170
1208
+ },
1209
+ {
1210
+ "epoch": 0.0972005115816399,
1211
+ "grad_norm": 11202.513671875,
1212
+ "learning_rate": 0.00019444948277304533,
1213
+ "loss": 10.7456,
1214
+ "step": 171
1215
+ },
1216
+ {
1217
+ "epoch": 0.09776893562597698,
1218
+ "grad_norm": 9468.578125,
1219
+ "learning_rate": 0.00019438097319901147,
1220
+ "loss": 11.3035,
1221
+ "step": 172
1222
+ },
1223
+ {
1224
+ "epoch": 0.09833735967031405,
1225
+ "grad_norm": 8223.53125,
1226
+ "learning_rate": 0.00019431205562972333,
1227
+ "loss": 11.0613,
1228
+ "step": 173
1229
+ },
1230
+ {
1231
+ "epoch": 0.09890578371465113,
1232
+ "grad_norm": 4253.04150390625,
1233
+ "learning_rate": 0.00019424273036310158,
1234
+ "loss": 9.7612,
1235
+ "step": 174
1236
+ },
1237
+ {
1238
+ "epoch": 0.0994742077589882,
1239
+ "grad_norm": 4771.42138671875,
1240
+ "learning_rate": 0.0001941729976988293,
1241
+ "loss": 8.2577,
1242
+ "step": 175
1243
+ },
1244
+ {
1245
+ "epoch": 0.10004263180332527,
1246
+ "grad_norm": 3440.725830078125,
1247
+ "learning_rate": 0.00019410285793835069,
1248
+ "loss": 12.5277,
1249
+ "step": 176
1250
+ },
1251
+ {
1252
+ "epoch": 0.10061105584766236,
1253
+ "grad_norm": 2819.26220703125,
1254
+ "learning_rate": 0.0001940323113848697,
1255
+ "loss": 10.7385,
1256
+ "step": 177
1257
+ },
1258
+ {
1259
+ "epoch": 0.10117947989199944,
1260
+ "grad_norm": 4552.572265625,
1261
+ "learning_rate": 0.00019396135834334893,
1262
+ "loss": 12.0743,
1263
+ "step": 178
1264
+ },
1265
+ {
1266
+ "epoch": 0.10174790393633651,
1267
+ "grad_norm": 7676.693359375,
1268
+ "learning_rate": 0.000193889999120508,
1269
+ "loss": 9.8995,
1270
+ "step": 179
1271
+ },
1272
+ {
1273
+ "epoch": 0.10231632798067358,
1274
+ "grad_norm": 3235.019287109375,
1275
+ "learning_rate": 0.00019381823402482256,
1276
+ "loss": 11.0758,
1277
+ "step": 180
1278
+ },
1279
+ {
1280
+ "epoch": 0.10288475202501066,
1281
+ "grad_norm": 2716.653076171875,
1282
+ "learning_rate": 0.00019374606336652265,
1283
+ "loss": 10.4924,
1284
+ "step": 181
1285
+ },
1286
+ {
1287
+ "epoch": 0.10345317606934773,
1288
+ "grad_norm": 2976.852294921875,
1289
+ "learning_rate": 0.00019367348745759157,
1290
+ "loss": 12.2543,
1291
+ "step": 182
1292
+ },
1293
+ {
1294
+ "epoch": 0.1040216001136848,
1295
+ "grad_norm": 5885.666015625,
1296
+ "learning_rate": 0.00019360050661176446,
1297
+ "loss": 9.8339,
1298
+ "step": 183
1299
+ },
1300
+ {
1301
+ "epoch": 0.10459002415802189,
1302
+ "grad_norm": 3451.98046875,
1303
+ "learning_rate": 0.00019352712114452692,
1304
+ "loss": 9.919,
1305
+ "step": 184
1306
+ },
1307
+ {
1308
+ "epoch": 0.10515844820235896,
1309
+ "grad_norm": 2482.541748046875,
1310
+ "learning_rate": 0.0001934533313731137,
1311
+ "loss": 11.3988,
1312
+ "step": 185
1313
+ },
1314
+ {
1315
+ "epoch": 0.10572687224669604,
1316
+ "grad_norm": 5550.8798828125,
1317
+ "learning_rate": 0.0001933791376165072,
1318
+ "loss": 13.1169,
1319
+ "step": 186
1320
+ },
1321
+ {
1322
+ "epoch": 0.10629529629103311,
1323
+ "grad_norm": 3188.782958984375,
1324
+ "learning_rate": 0.0001933045401954363,
1325
+ "loss": 13.2072,
1326
+ "step": 187
1327
+ },
1328
+ {
1329
+ "epoch": 0.10686372033537019,
1330
+ "grad_norm": 4378.90478515625,
1331
+ "learning_rate": 0.00019322953943237488,
1332
+ "loss": 11.1468,
1333
+ "step": 188
1334
+ },
1335
+ {
1336
+ "epoch": 0.10743214437970726,
1337
+ "grad_norm": 2656.765625,
1338
+ "learning_rate": 0.00019315413565154034,
1339
+ "loss": 12.1954,
1340
+ "step": 189
1341
+ },
1342
+ {
1343
+ "epoch": 0.10800056842404433,
1344
+ "grad_norm": 1895.977294921875,
1345
+ "learning_rate": 0.00019307832917889224,
1346
+ "loss": 11.9331,
1347
+ "step": 190
1348
+ },
1349
+ {
1350
+ "epoch": 0.1085689924683814,
1351
+ "grad_norm": 4219.88916015625,
1352
+ "learning_rate": 0.00019300212034213096,
1353
+ "loss": 12.6816,
1354
+ "step": 191
1355
+ },
1356
+ {
1357
+ "epoch": 0.1091374165127185,
1358
+ "grad_norm": 5541.60791015625,
1359
+ "learning_rate": 0.00019292550947069633,
1360
+ "loss": 13.0854,
1361
+ "step": 192
1362
+ },
1363
+ {
1364
+ "epoch": 0.10970584055705557,
1365
+ "grad_norm": 8763.7861328125,
1366
+ "learning_rate": 0.00019284849689576596,
1367
+ "loss": 12.3964,
1368
+ "step": 193
1369
+ },
1370
+ {
1371
+ "epoch": 0.11027426460139264,
1372
+ "grad_norm": 1885.9329833984375,
1373
+ "learning_rate": 0.00019277108295025403,
1374
+ "loss": 12.9015,
1375
+ "step": 194
1376
+ },
1377
+ {
1378
+ "epoch": 0.11084268864572971,
1379
+ "grad_norm": 6760.47509765625,
1380
+ "learning_rate": 0.00019269326796880987,
1381
+ "loss": 13.4623,
1382
+ "step": 195
1383
+ },
1384
+ {
1385
+ "epoch": 0.11141111269006679,
1386
+ "grad_norm": 1420.9971923828125,
1387
+ "learning_rate": 0.00019261505228781626,
1388
+ "loss": 13.1383,
1389
+ "step": 196
1390
+ },
1391
+ {
1392
+ "epoch": 0.11197953673440386,
1393
+ "grad_norm": 1789.84765625,
1394
+ "learning_rate": 0.0001925364362453883,
1395
+ "loss": 12.9684,
1396
+ "step": 197
1397
+ },
1398
+ {
1399
+ "epoch": 0.11254796077874094,
1400
+ "grad_norm": 2940.865478515625,
1401
+ "learning_rate": 0.00019245742018137176,
1402
+ "loss": 12.1597,
1403
+ "step": 198
1404
+ },
1405
+ {
1406
+ "epoch": 0.11311638482307802,
1407
+ "grad_norm": 3828.867919921875,
1408
+ "learning_rate": 0.00019237800443734155,
1409
+ "loss": 11.8966,
1410
+ "step": 199
1411
+ },
1412
+ {
1413
+ "epoch": 0.1136848088674151,
1414
+ "grad_norm": 1485.7850341796875,
1415
+ "learning_rate": 0.00019229818935660045,
1416
+ "loss": 12.0223,
1417
+ "step": 200
1418
+ },
1419
+ {
1420
+ "epoch": 0.11425323291175217,
1421
+ "grad_norm": 2678.6611328125,
1422
+ "learning_rate": 0.00019221797528417745,
1423
+ "loss": 11.5316,
1424
+ "step": 201
1425
+ },
1426
+ {
1427
+ "epoch": 0.11482165695608924,
1428
+ "grad_norm": 4726.84521484375,
1429
+ "learning_rate": 0.0001921373625668264,
1430
+ "loss": 10.06,
1431
+ "step": 202
1432
+ },
1433
+ {
1434
+ "epoch": 0.11539008100042632,
1435
+ "grad_norm": 2610.596923828125,
1436
+ "learning_rate": 0.00019205635155302434,
1437
+ "loss": 13.1575,
1438
+ "step": 203
1439
+ },
1440
+ {
1441
+ "epoch": 0.11595850504476339,
1442
+ "grad_norm": 4310.37158203125,
1443
+ "learning_rate": 0.0001919749425929701,
1444
+ "loss": 12.6187,
1445
+ "step": 204
1446
+ },
1447
+ {
1448
+ "epoch": 0.11652692908910046,
1449
+ "grad_norm": 3431.9853515625,
1450
+ "learning_rate": 0.0001918931360385829,
1451
+ "loss": 10.649,
1452
+ "step": 205
1453
+ },
1454
+ {
1455
+ "epoch": 0.11709535313343754,
1456
+ "grad_norm": 2197.531005859375,
1457
+ "learning_rate": 0.0001918109322435005,
1458
+ "loss": 12.1493,
1459
+ "step": 206
1460
+ },
1461
+ {
1462
+ "epoch": 0.11766377717777463,
1463
+ "grad_norm": 9167.5810546875,
1464
+ "learning_rate": 0.00019172833156307805,
1465
+ "loss": 16.0639,
1466
+ "step": 207
1467
+ },
1468
+ {
1469
+ "epoch": 0.1182322012221117,
1470
+ "grad_norm": 3232.08935546875,
1471
+ "learning_rate": 0.00019164533435438627,
1472
+ "loss": 13.2877,
1473
+ "step": 208
1474
+ },
1475
+ {
1476
+ "epoch": 0.11880062526644877,
1477
+ "grad_norm": 3288.341796875,
1478
+ "learning_rate": 0.00019156194097621007,
1479
+ "loss": 12.2261,
1480
+ "step": 209
1481
+ },
1482
+ {
1483
+ "epoch": 0.11936904931078585,
1484
+ "grad_norm": 9568.271484375,
1485
+ "learning_rate": 0.00019147815178904693,
1486
+ "loss": 12.0787,
1487
+ "step": 210
1488
+ },
1489
+ {
1490
+ "epoch": 0.11993747335512292,
1491
+ "grad_norm": 1965.083740234375,
1492
+ "learning_rate": 0.00019139396715510534,
1493
+ "loss": 11.956,
1494
+ "step": 211
1495
+ },
1496
+ {
1497
+ "epoch": 0.12050589739946,
1498
+ "grad_norm": 2447.73193359375,
1499
+ "learning_rate": 0.00019130938743830328,
1500
+ "loss": 13.7234,
1501
+ "step": 212
1502
+ },
1503
+ {
1504
+ "epoch": 0.12107432144379707,
1505
+ "grad_norm": 5586.15869140625,
1506
+ "learning_rate": 0.0001912244130042665,
1507
+ "loss": 11.81,
1508
+ "step": 213
1509
+ },
1510
+ {
1511
+ "epoch": 0.12164274548813415,
1512
+ "grad_norm": 2780.562255859375,
1513
+ "learning_rate": 0.00019113904422032732,
1514
+ "loss": 12.7053,
1515
+ "step": 214
1516
+ },
1517
+ {
1518
+ "epoch": 0.12221116953247123,
1519
+ "grad_norm": 1958.6663818359375,
1520
+ "learning_rate": 0.00019105328145552246,
1521
+ "loss": 12.8653,
1522
+ "step": 215
1523
+ },
1524
+ {
1525
+ "epoch": 0.1227795935768083,
1526
+ "grad_norm": 2051.577880859375,
1527
+ "learning_rate": 0.00019096712508059196,
1528
+ "loss": 13.4584,
1529
+ "step": 216
1530
+ },
1531
+ {
1532
+ "epoch": 0.12334801762114538,
1533
+ "grad_norm": 4086.435302734375,
1534
+ "learning_rate": 0.00019088057546797735,
1535
+ "loss": 13.8054,
1536
+ "step": 217
1537
+ },
1538
+ {
1539
+ "epoch": 0.12391644166548245,
1540
+ "grad_norm": 4879.5966796875,
1541
+ "learning_rate": 0.00019079363299182004,
1542
+ "loss": 12.3353,
1543
+ "step": 218
1544
+ },
1545
+ {
1546
+ "epoch": 0.12448486570981952,
1547
+ "grad_norm": 12456.7314453125,
1548
+ "learning_rate": 0.0001907062980279597,
1549
+ "loss": 11.2014,
1550
+ "step": 219
1551
+ },
1552
+ {
1553
+ "epoch": 0.1250532897541566,
1554
+ "grad_norm": 4802.9072265625,
1555
+ "learning_rate": 0.00019061857095393277,
1556
+ "loss": 13.5862,
1557
+ "step": 220
1558
+ },
1559
+ {
1560
+ "epoch": 0.12562171379849368,
1561
+ "grad_norm": 2849.338623046875,
1562
+ "learning_rate": 0.0001905304521489706,
1563
+ "loss": 12.072,
1564
+ "step": 221
1565
+ },
1566
+ {
1567
+ "epoch": 0.12619013784283076,
1568
+ "grad_norm": 4029.34375,
1569
+ "learning_rate": 0.00019044194199399807,
1570
+ "loss": 11.8001,
1571
+ "step": 222
1572
+ },
1573
+ {
1574
+ "epoch": 0.12675856188716783,
1575
+ "grad_norm": 5020.30615234375,
1576
+ "learning_rate": 0.00019035304087163167,
1577
+ "loss": 10.9671,
1578
+ "step": 223
1579
+ },
1580
+ {
1581
+ "epoch": 0.1273269859315049,
1582
+ "grad_norm": 9186.296875,
1583
+ "learning_rate": 0.00019026374916617807,
1584
+ "loss": 13.6867,
1585
+ "step": 224
1586
+ },
1587
+ {
1588
+ "epoch": 0.12789540997584198,
1589
+ "grad_norm": 3389.864990234375,
1590
+ "learning_rate": 0.00019017406726363242,
1591
+ "loss": 12.7294,
1592
+ "step": 225
1593
+ },
1594
+ {
1595
+ "epoch": 0.12846383402017905,
1596
+ "grad_norm": 6309.7822265625,
1597
+ "learning_rate": 0.0001900839955516765,
1598
+ "loss": 13.0971,
1599
+ "step": 226
1600
+ },
1601
+ {
1602
+ "epoch": 0.12903225806451613,
1603
+ "grad_norm": 2293.2294921875,
1604
+ "learning_rate": 0.00018999353441967728,
1605
+ "loss": 12.332,
1606
+ "step": 227
1607
+ },
1608
+ {
1609
+ "epoch": 0.1296006821088532,
1610
+ "grad_norm": 5950.0126953125,
1611
+ "learning_rate": 0.00018990268425868513,
1612
+ "loss": 11.8517,
1613
+ "step": 228
1614
+ },
1615
+ {
1616
+ "epoch": 0.13016910615319027,
1617
+ "grad_norm": 6341.48193359375,
1618
+ "learning_rate": 0.00018981144546143205,
1619
+ "loss": 15.1847,
1620
+ "step": 229
1621
+ },
1622
+ {
1623
+ "epoch": 0.13073753019752735,
1624
+ "grad_norm": 1336.18896484375,
1625
+ "learning_rate": 0.0001897198184223302,
1626
+ "loss": 12.4568,
1627
+ "step": 230
1628
+ },
1629
+ {
1630
+ "epoch": 0.13130595424186442,
1631
+ "grad_norm": 3987.446044921875,
1632
+ "learning_rate": 0.00018962780353746995,
1633
+ "loss": 12.7261,
1634
+ "step": 231
1635
+ },
1636
+ {
1637
+ "epoch": 0.1318743782862015,
1638
+ "grad_norm": 2917.445556640625,
1639
+ "learning_rate": 0.0001895354012046183,
1640
+ "loss": 12.623,
1641
+ "step": 232
1642
+ },
1643
+ {
1644
+ "epoch": 0.1324428023305386,
1645
+ "grad_norm": 4980.18994140625,
1646
+ "learning_rate": 0.0001894426118232171,
1647
+ "loss": 13.2703,
1648
+ "step": 233
1649
+ },
1650
+ {
1651
+ "epoch": 0.13301122637487567,
1652
+ "grad_norm": 7874.12255859375,
1653
+ "learning_rate": 0.00018934943579438144,
1654
+ "loss": 12.3513,
1655
+ "step": 234
1656
+ },
1657
+ {
1658
+ "epoch": 0.13357965041921274,
1659
+ "grad_norm": 1594.990478515625,
1660
+ "learning_rate": 0.0001892558735208977,
1661
+ "loss": 13.0442,
1662
+ "step": 235
1663
+ },
1664
+ {
1665
+ "epoch": 0.13414807446354982,
1666
+ "grad_norm": 3103.23974609375,
1667
+ "learning_rate": 0.00018916192540722207,
1668
+ "loss": 14.6219,
1669
+ "step": 236
1670
+ },
1671
+ {
1672
+ "epoch": 0.1347164985078869,
1673
+ "grad_norm": 1444.2427978515625,
1674
+ "learning_rate": 0.00018906759185947857,
1675
+ "loss": 11.6902,
1676
+ "step": 237
1677
+ },
1678
+ {
1679
+ "epoch": 0.13528492255222396,
1680
+ "grad_norm": 3407.40087890625,
1681
+ "learning_rate": 0.00018897287328545752,
1682
+ "loss": 11.0158,
1683
+ "step": 238
1684
+ },
1685
+ {
1686
+ "epoch": 0.13585334659656104,
1687
+ "grad_norm": 1644.69873046875,
1688
+ "learning_rate": 0.00018887777009461346,
1689
+ "loss": 13.374,
1690
+ "step": 239
1691
+ },
1692
+ {
1693
+ "epoch": 0.1364217706408981,
1694
+ "grad_norm": 3099.279541015625,
1695
+ "learning_rate": 0.00018878228269806382,
1696
+ "loss": 13.3341,
1697
+ "step": 240
1698
+ },
1699
+ {
1700
+ "epoch": 0.13699019468523518,
1701
+ "grad_norm": 5028.576171875,
1702
+ "learning_rate": 0.00018868641150858668,
1703
+ "loss": 14.1646,
1704
+ "step": 241
1705
+ },
1706
+ {
1707
+ "epoch": 0.13755861872957226,
1708
+ "grad_norm": 1891.924072265625,
1709
+ "learning_rate": 0.00018859015694061925,
1710
+ "loss": 10.3512,
1711
+ "step": 242
1712
+ },
1713
+ {
1714
+ "epoch": 0.13812704277390933,
1715
+ "grad_norm": 1996.42578125,
1716
+ "learning_rate": 0.00018849351941025615,
1717
+ "loss": 11.7482,
1718
+ "step": 243
1719
+ },
1720
+ {
1721
+ "epoch": 0.1386954668182464,
1722
+ "grad_norm": 1995.262939453125,
1723
+ "learning_rate": 0.00018839649933524735,
1724
+ "loss": 14.3018,
1725
+ "step": 244
1726
+ },
1727
+ {
1728
+ "epoch": 0.13926389086258348,
1729
+ "grad_norm": 3902.045166015625,
1730
+ "learning_rate": 0.00018829909713499656,
1731
+ "loss": 11.7036,
1732
+ "step": 245
1733
+ },
1734
+ {
1735
+ "epoch": 0.13983231490692055,
1736
+ "grad_norm": 1602.975341796875,
1737
+ "learning_rate": 0.0001882013132305594,
1738
+ "loss": 11.7867,
1739
+ "step": 246
1740
+ },
1741
+ {
1742
+ "epoch": 0.14040073895125763,
1743
+ "grad_norm": 7255.88818359375,
1744
+ "learning_rate": 0.00018810314804464144,
1745
+ "loss": 11.4703,
1746
+ "step": 247
1747
+ },
1748
+ {
1749
+ "epoch": 0.14096916299559473,
1750
+ "grad_norm": 5502.51416015625,
1751
+ "learning_rate": 0.00018800460200159657,
1752
+ "loss": 13.6289,
1753
+ "step": 248
1754
+ },
1755
+ {
1756
+ "epoch": 0.1415375870399318,
1757
+ "grad_norm": 4792.44384765625,
1758
+ "learning_rate": 0.00018790567552742505,
1759
+ "loss": 14.5212,
1760
+ "step": 249
1761
+ },
1762
+ {
1763
+ "epoch": 0.14210601108426887,
1764
+ "grad_norm": 3758.884521484375,
1765
+ "learning_rate": 0.00018780636904977167,
1766
+ "loss": 13.2351,
1767
+ "step": 250
1768
+ },
1769
+ {
1770
+ "epoch": 0.14267443512860595,
1771
+ "grad_norm": 3761.82080078125,
1772
+ "learning_rate": 0.0001877066829979239,
1773
+ "loss": 11.1298,
1774
+ "step": 251
1775
+ },
1776
+ {
1777
+ "epoch": 0.14324285917294302,
1778
+ "grad_norm": 4422.80224609375,
1779
+ "learning_rate": 0.0001876066178028101,
1780
+ "loss": 12.2986,
1781
+ "step": 252
1782
+ },
1783
+ {
1784
+ "epoch": 0.1438112832172801,
1785
+ "grad_norm": 1997.0972900390625,
1786
+ "learning_rate": 0.0001875061738969976,
1787
+ "loss": 11.6043,
1788
+ "step": 253
1789
+ },
1790
+ {
1791
+ "epoch": 0.14437970726161717,
1792
+ "grad_norm": 2278.685791015625,
1793
+ "learning_rate": 0.0001874053517146908,
1794
+ "loss": 11.22,
1795
+ "step": 254
1796
+ },
1797
+ {
1798
+ "epoch": 0.14494813130595424,
1799
+ "grad_norm": 8272.763671875,
1800
+ "learning_rate": 0.00018730415169172942,
1801
+ "loss": 10.7463,
1802
+ "step": 255
1803
+ },
1804
+ {
1805
+ "epoch": 0.14551655535029132,
1806
+ "grad_norm": 2491.65576171875,
1807
+ "learning_rate": 0.00018720257426558642,
1808
+ "loss": 13.1333,
1809
+ "step": 256
1810
+ },
1811
+ {
1812
+ "epoch": 0.1460849793946284,
1813
+ "grad_norm": 1537.728271484375,
1814
+ "learning_rate": 0.00018710061987536628,
1815
+ "loss": 11.1176,
1816
+ "step": 257
1817
+ },
1818
+ {
1819
+ "epoch": 0.14665340343896546,
1820
+ "grad_norm": 1369.56298828125,
1821
+ "learning_rate": 0.00018699828896180312,
1822
+ "loss": 12.4531,
1823
+ "step": 258
1824
+ },
1825
+ {
1826
+ "epoch": 0.14722182748330254,
1827
+ "grad_norm": 1322.3084716796875,
1828
+ "learning_rate": 0.00018689558196725853,
1829
+ "loss": 12.325,
1830
+ "step": 259
1831
+ },
1832
+ {
1833
+ "epoch": 0.1477902515276396,
1834
+ "grad_norm": 10544.2890625,
1835
+ "learning_rate": 0.00018679249933571999,
1836
+ "loss": 12.0443,
1837
+ "step": 260
1838
+ },
1839
+ {
1840
+ "epoch": 0.14835867557197668,
1841
+ "grad_norm": 2657.380615234375,
1842
+ "learning_rate": 0.00018668904151279876,
1843
+ "loss": 10.444,
1844
+ "step": 261
1845
+ },
1846
+ {
1847
+ "epoch": 0.14892709961631376,
1848
+ "grad_norm": 1990.4234619140625,
1849
+ "learning_rate": 0.00018658520894572798,
1850
+ "loss": 12.5582,
1851
+ "step": 262
1852
+ },
1853
+ {
1854
+ "epoch": 0.14949552366065086,
1855
+ "grad_norm": 4418.38525390625,
1856
+ "learning_rate": 0.0001864810020833608,
1857
+ "loss": 11.4966,
1858
+ "step": 263
1859
+ },
1860
+ {
1861
+ "epoch": 0.15006394770498793,
1862
+ "grad_norm": 2691.17822265625,
1863
+ "learning_rate": 0.00018637642137616837,
1864
+ "loss": 10.963,
1865
+ "step": 264
1866
+ },
1867
+ {
1868
+ "epoch": 0.150632371749325,
1869
+ "grad_norm": 1920.5191650390625,
1870
+ "learning_rate": 0.0001862714672762379,
1871
+ "loss": 12.9823,
1872
+ "step": 265
1873
+ },
1874
+ {
1875
+ "epoch": 0.15120079579366208,
1876
+ "grad_norm": 3860.548828125,
1877
+ "learning_rate": 0.00018616614023727075,
1878
+ "loss": 12.5424,
1879
+ "step": 266
1880
+ },
1881
+ {
1882
+ "epoch": 0.15176921983799915,
1883
+ "grad_norm": 6393.43994140625,
1884
+ "learning_rate": 0.00018606044071458038,
1885
+ "loss": 11.2672,
1886
+ "step": 267
1887
+ },
1888
+ {
1889
+ "epoch": 0.15233764388233623,
1890
+ "grad_norm": 4462.0322265625,
1891
+ "learning_rate": 0.00018595436916509056,
1892
+ "loss": 16.8448,
1893
+ "step": 268
1894
+ },
1895
+ {
1896
+ "epoch": 0.1529060679266733,
1897
+ "grad_norm": 1656.4317626953125,
1898
+ "learning_rate": 0.0001858479260473332,
1899
+ "loss": 10.8226,
1900
+ "step": 269
1901
+ },
1902
+ {
1903
+ "epoch": 0.15347449197101037,
1904
+ "grad_norm": 2701.659912109375,
1905
+ "learning_rate": 0.00018574111182144637,
1906
+ "loss": 11.9604,
1907
+ "step": 270
1908
+ },
1909
+ {
1910
+ "epoch": 0.15404291601534745,
1911
+ "grad_norm": 6866.8134765625,
1912
+ "learning_rate": 0.0001856339269491726,
1913
+ "loss": 11.4421,
1914
+ "step": 271
1915
+ },
1916
+ {
1917
+ "epoch": 0.15461134005968452,
1918
+ "grad_norm": 1585.0523681640625,
1919
+ "learning_rate": 0.00018552637189385643,
1920
+ "loss": 11.2402,
1921
+ "step": 272
1922
+ },
1923
+ {
1924
+ "epoch": 0.1551797641040216,
1925
+ "grad_norm": 2695.2822265625,
1926
+ "learning_rate": 0.00018541844712044285,
1927
+ "loss": 12.7832,
1928
+ "step": 273
1929
+ },
1930
+ {
1931
+ "epoch": 0.15574818814835867,
1932
+ "grad_norm": 1258.8656005859375,
1933
+ "learning_rate": 0.00018531015309547494,
1934
+ "loss": 13.7431,
1935
+ "step": 274
1936
+ },
1937
+ {
1938
+ "epoch": 0.15631661219269574,
1939
+ "grad_norm": 949.9840087890625,
1940
+ "learning_rate": 0.00018520149028709206,
1941
+ "loss": 11.9473,
1942
+ "step": 275
1943
+ },
1944
+ {
1945
+ "epoch": 0.15688503623703282,
1946
+ "grad_norm": 1541.031005859375,
1947
+ "learning_rate": 0.00018509245916502779,
1948
+ "loss": 8.9786,
1949
+ "step": 276
1950
+ },
1951
+ {
1952
+ "epoch": 0.1574534602813699,
1953
+ "grad_norm": 1811.456298828125,
1954
+ "learning_rate": 0.00018498306020060782,
1955
+ "loss": 11.1169,
1956
+ "step": 277
1957
+ },
1958
+ {
1959
+ "epoch": 0.158021884325707,
1960
+ "grad_norm": 1015.3645629882812,
1961
+ "learning_rate": 0.00018487329386674808,
1962
+ "loss": 10.9756,
1963
+ "step": 278
1964
+ },
1965
+ {
1966
+ "epoch": 0.15859030837004406,
1967
+ "grad_norm": 1195.05712890625,
1968
+ "learning_rate": 0.0001847631606379524,
1969
+ "loss": 12.434,
1970
+ "step": 279
1971
+ },
1972
+ {
1973
+ "epoch": 0.15915873241438114,
1974
+ "grad_norm": 1368.8919677734375,
1975
+ "learning_rate": 0.00018465266099031082,
1976
+ "loss": 11.0154,
1977
+ "step": 280
1978
+ },
1979
+ {
1980
+ "epoch": 0.1597271564587182,
1981
+ "grad_norm": 2175.390380859375,
1982
+ "learning_rate": 0.00018454179540149728,
1983
+ "loss": 11.712,
1984
+ "step": 281
1985
+ },
1986
+ {
1987
+ "epoch": 0.16029558050305528,
1988
+ "grad_norm": 1087.5726318359375,
1989
+ "learning_rate": 0.0001844305643507676,
1990
+ "loss": 10.6293,
1991
+ "step": 282
1992
+ },
1993
+ {
1994
+ "epoch": 0.16086400454739236,
1995
+ "grad_norm": 1140.515380859375,
1996
+ "learning_rate": 0.0001843189683189575,
1997
+ "loss": 11.0227,
1998
+ "step": 283
1999
+ },
2000
+ {
2001
+ "epoch": 0.16143242859172943,
2002
+ "grad_norm": 1044.4326171875,
2003
+ "learning_rate": 0.00018420700778848043,
2004
+ "loss": 9.3956,
2005
+ "step": 284
2006
+ },
2007
+ {
2008
+ "epoch": 0.1620008526360665,
2009
+ "grad_norm": 2006.2518310546875,
2010
+ "learning_rate": 0.0001840946832433255,
2011
+ "loss": 11.541,
2012
+ "step": 285
2013
+ },
2014
+ {
2015
+ "epoch": 0.16256927668040358,
2016
+ "grad_norm": 5861.81298828125,
2017
+ "learning_rate": 0.00018398199516905548,
2018
+ "loss": 13.8056,
2019
+ "step": 286
2020
+ },
2021
+ {
2022
+ "epoch": 0.16313770072474065,
2023
+ "grad_norm": 1458.2388916015625,
2024
+ "learning_rate": 0.00018386894405280453,
2025
+ "loss": 12.7987,
2026
+ "step": 287
2027
+ },
2028
+ {
2029
+ "epoch": 0.16370612476907773,
2030
+ "grad_norm": 2750.82373046875,
2031
+ "learning_rate": 0.00018375553038327622,
2032
+ "loss": 13.4016,
2033
+ "step": 288
2034
+ },
2035
+ {
2036
+ "epoch": 0.1642745488134148,
2037
+ "grad_norm": 1254.9464111328125,
2038
+ "learning_rate": 0.00018364175465074135,
2039
+ "loss": 12.5969,
2040
+ "step": 289
2041
+ },
2042
+ {
2043
+ "epoch": 0.16484297285775187,
2044
+ "grad_norm": 19019.25390625,
2045
+ "learning_rate": 0.00018352761734703602,
2046
+ "loss": 11.2982,
2047
+ "step": 290
2048
+ },
2049
+ {
2050
+ "epoch": 0.16541139690208895,
2051
+ "grad_norm": 2205.30908203125,
2052
+ "learning_rate": 0.0001834131189655591,
2053
+ "loss": 12.3371,
2054
+ "step": 291
2055
+ },
2056
+ {
2057
+ "epoch": 0.16597982094642602,
2058
+ "grad_norm": 3541.462646484375,
2059
+ "learning_rate": 0.00018329826000127057,
2060
+ "loss": 11.5133,
2061
+ "step": 292
2062
+ },
2063
+ {
2064
+ "epoch": 0.16654824499076312,
2065
+ "grad_norm": 10169.9482421875,
2066
+ "learning_rate": 0.00018318304095068909,
2067
+ "loss": 13.7055,
2068
+ "step": 293
2069
+ },
2070
+ {
2071
+ "epoch": 0.1671166690351002,
2072
+ "grad_norm": 2107.942626953125,
2073
+ "learning_rate": 0.00018306746231188978,
2074
+ "loss": 12.0856,
2075
+ "step": 294
2076
+ },
2077
+ {
2078
+ "epoch": 0.16768509307943727,
2079
+ "grad_norm": 3659.464599609375,
2080
+ "learning_rate": 0.00018295152458450243,
2081
+ "loss": 10.785,
2082
+ "step": 295
2083
+ },
2084
+ {
2085
+ "epoch": 0.16825351712377434,
2086
+ "grad_norm": 2204.259521484375,
2087
+ "learning_rate": 0.000182835228269709,
2088
+ "loss": 10.9013,
2089
+ "step": 296
2090
+ },
2091
+ {
2092
+ "epoch": 0.16882194116811142,
2093
+ "grad_norm": 1345.0950927734375,
2094
+ "learning_rate": 0.00018271857387024154,
2095
+ "loss": 10.9842,
2096
+ "step": 297
2097
+ },
2098
+ {
2099
+ "epoch": 0.1693903652124485,
2100
+ "grad_norm": 1188.415771484375,
2101
+ "learning_rate": 0.00018260156189038015,
2102
+ "loss": 11.9986,
2103
+ "step": 298
2104
+ },
2105
+ {
2106
+ "epoch": 0.16995878925678556,
2107
+ "grad_norm": 852.0604248046875,
2108
+ "learning_rate": 0.0001824841928359506,
2109
+ "loss": 10.5311,
2110
+ "step": 299
2111
+ },
2112
+ {
2113
+ "epoch": 0.17052721330112264,
2114
+ "grad_norm": 4719.693359375,
2115
+ "learning_rate": 0.00018236646721432235,
2116
+ "loss": 8.7847,
2117
+ "step": 300
2118
+ },
2119
+ {
2120
+ "epoch": 0.1710956373454597,
2121
+ "grad_norm": 2755.71923828125,
2122
+ "learning_rate": 0.00018224838553440606,
2123
+ "loss": 13.4404,
2124
+ "step": 301
2125
+ },
2126
+ {
2127
+ "epoch": 0.17166406138979678,
2128
+ "grad_norm": 898.6322021484375,
2129
+ "learning_rate": 0.00018212994830665182,
2130
+ "loss": 10.0005,
2131
+ "step": 302
2132
+ },
2133
+ {
2134
+ "epoch": 0.17223248543413386,
2135
+ "grad_norm": 4063.603515625,
2136
+ "learning_rate": 0.00018201115604304656,
2137
+ "loss": 12.3396,
2138
+ "step": 303
2139
+ },
2140
+ {
2141
+ "epoch": 0.17280090947847093,
2142
+ "grad_norm": 1588.1654052734375,
2143
+ "learning_rate": 0.00018189200925711196,
2144
+ "loss": 12.1632,
2145
+ "step": 304
2146
+ },
2147
+ {
2148
+ "epoch": 0.173369333522808,
2149
+ "grad_norm": 3080.884033203125,
2150
+ "learning_rate": 0.0001817725084639023,
2151
+ "loss": 12.9716,
2152
+ "step": 305
2153
+ },
2154
+ {
2155
+ "epoch": 0.17393775756714508,
2156
+ "grad_norm": 1664.848388671875,
2157
+ "learning_rate": 0.0001816526541800022,
2158
+ "loss": 10.1787,
2159
+ "step": 306
2160
+ },
2161
+ {
2162
+ "epoch": 0.17450618161148215,
2163
+ "grad_norm": 14394.3740234375,
2164
+ "learning_rate": 0.00018153244692352432,
2165
+ "loss": 11.9161,
2166
+ "step": 307
2167
+ },
2168
+ {
2169
+ "epoch": 0.17507460565581925,
2170
+ "grad_norm": 2257.2421875,
2171
+ "learning_rate": 0.00018141188721410717,
2172
+ "loss": 11.7464,
2173
+ "step": 308
2174
+ },
2175
+ {
2176
+ "epoch": 0.17564302970015633,
2177
+ "grad_norm": 1703.25927734375,
2178
+ "learning_rate": 0.00018129097557291292,
2179
+ "loss": 14.1072,
2180
+ "step": 309
2181
+ },
2182
+ {
2183
+ "epoch": 0.1762114537444934,
2184
+ "grad_norm": 1446.6278076171875,
2185
+ "learning_rate": 0.00018116971252262503,
2186
+ "loss": 12.2933,
2187
+ "step": 310
2188
+ },
2189
+ {
2190
+ "epoch": 0.17677987778883048,
2191
+ "grad_norm": 3190.185791015625,
2192
+ "learning_rate": 0.0001810480985874461,
2193
+ "loss": 13.9696,
2194
+ "step": 311
2195
+ },
2196
+ {
2197
+ "epoch": 0.17734830183316755,
2198
+ "grad_norm": 21618.0078125,
2199
+ "learning_rate": 0.0001809261342930955,
2200
+ "loss": 15.1179,
2201
+ "step": 312
2202
+ },
2203
+ {
2204
+ "epoch": 0.17791672587750462,
2205
+ "grad_norm": 1274.9013671875,
2206
+ "learning_rate": 0.0001808038201668072,
2207
+ "loss": 13.7353,
2208
+ "step": 313
2209
+ },
2210
+ {
2211
+ "epoch": 0.1784851499218417,
2212
+ "grad_norm": 7062.59375,
2213
+ "learning_rate": 0.0001806811567373274,
2214
+ "loss": 12.3906,
2215
+ "step": 314
2216
+ },
2217
+ {
2218
+ "epoch": 0.17905357396617877,
2219
+ "grad_norm": 1014.2584838867188,
2220
+ "learning_rate": 0.00018055814453491232,
2221
+ "loss": 10.0779,
2222
+ "step": 315
2223
+ },
2224
+ {
2225
+ "epoch": 0.17962199801051584,
2226
+ "grad_norm": 2294.896728515625,
2227
+ "learning_rate": 0.0001804347840913259,
2228
+ "loss": 13.85,
2229
+ "step": 316
2230
+ },
2231
+ {
2232
+ "epoch": 0.18019042205485292,
2233
+ "grad_norm": 1380.632080078125,
2234
+ "learning_rate": 0.00018031107593983737,
2235
+ "loss": 12.4406,
2236
+ "step": 317
2237
+ },
2238
+ {
2239
+ "epoch": 0.18075884609919,
2240
+ "grad_norm": 2982.55419921875,
2241
+ "learning_rate": 0.00018018702061521917,
2242
+ "loss": 11.1636,
2243
+ "step": 318
2244
+ },
2245
+ {
2246
+ "epoch": 0.18132727014352706,
2247
+ "grad_norm": 9898.26953125,
2248
+ "learning_rate": 0.00018006261865374446,
2249
+ "loss": 12.7255,
2250
+ "step": 319
2251
+ },
2252
+ {
2253
+ "epoch": 0.18189569418786414,
2254
+ "grad_norm": 7027.0048828125,
2255
+ "learning_rate": 0.00017993787059318486,
2256
+ "loss": 13.5697,
2257
+ "step": 320
2258
+ },
2259
+ {
2260
+ "epoch": 0.1824641182322012,
2261
+ "grad_norm": 2703.15673828125,
2262
+ "learning_rate": 0.00017981277697280815,
2263
+ "loss": 10.6677,
2264
+ "step": 321
2265
+ },
2266
+ {
2267
+ "epoch": 0.18303254227653828,
2268
+ "grad_norm": 5552.56689453125,
2269
+ "learning_rate": 0.0001796873383333759,
2270
+ "loss": 13.6973,
2271
+ "step": 322
2272
+ },
2273
+ {
2274
+ "epoch": 0.1836009663208754,
2275
+ "grad_norm": 1792.861572265625,
2276
+ "learning_rate": 0.00017956155521714115,
2277
+ "loss": 11.7813,
2278
+ "step": 323
2279
+ },
2280
+ {
2281
+ "epoch": 0.18416939036521246,
2282
+ "grad_norm": 1677.70703125,
2283
+ "learning_rate": 0.00017943542816784605,
2284
+ "loss": 11.7493,
2285
+ "step": 324
2286
+ },
2287
+ {
2288
+ "epoch": 0.18473781440954953,
2289
+ "grad_norm": 3065.983642578125,
2290
+ "learning_rate": 0.00017930895773071956,
2291
+ "loss": 14.5555,
2292
+ "step": 325
2293
+ },
2294
+ {
2295
+ "epoch": 0.1853062384538866,
2296
+ "grad_norm": 2134.20947265625,
2297
+ "learning_rate": 0.000179182144452475,
2298
+ "loss": 12.2791,
2299
+ "step": 326
2300
+ },
2301
+ {
2302
+ "epoch": 0.18587466249822368,
2303
+ "grad_norm": 1505.5361328125,
2304
+ "learning_rate": 0.0001790549888813078,
2305
+ "loss": 10.6904,
2306
+ "step": 327
2307
+ },
2308
+ {
2309
+ "epoch": 0.18644308654256075,
2310
+ "grad_norm": 1918.1795654296875,
2311
+ "learning_rate": 0.00017892749156689307,
2312
+ "loss": 10.8969,
2313
+ "step": 328
2314
+ },
2315
+ {
2316
+ "epoch": 0.18701151058689783,
2317
+ "grad_norm": 1818.625,
2318
+ "learning_rate": 0.00017879965306038314,
2319
+ "loss": 13.2012,
2320
+ "step": 329
2321
+ },
2322
+ {
2323
+ "epoch": 0.1875799346312349,
2324
+ "grad_norm": 1994.570556640625,
2325
+ "learning_rate": 0.00017867147391440534,
2326
+ "loss": 10.5649,
2327
+ "step": 330
2328
+ },
2329
+ {
2330
+ "epoch": 0.18814835867557198,
2331
+ "grad_norm": 1111.2037353515625,
2332
+ "learning_rate": 0.00017854295468305956,
2333
+ "loss": 13.3289,
2334
+ "step": 331
2335
+ },
2336
+ {
2337
+ "epoch": 0.18871678271990905,
2338
+ "grad_norm": 1132.3131103515625,
2339
+ "learning_rate": 0.00017841409592191576,
2340
+ "loss": 11.2649,
2341
+ "step": 332
2342
+ },
2343
+ {
2344
+ "epoch": 0.18928520676424612,
2345
+ "grad_norm": 4223.74951171875,
2346
+ "learning_rate": 0.0001782848981880117,
2347
+ "loss": 11.3041,
2348
+ "step": 333
2349
+ },
2350
+ {
2351
+ "epoch": 0.1898536308085832,
2352
+ "grad_norm": 4478.46826171875,
2353
+ "learning_rate": 0.0001781553620398504,
2354
+ "loss": 12.6508,
2355
+ "step": 334
2356
+ },
2357
+ {
2358
+ "epoch": 0.19042205485292027,
2359
+ "grad_norm": 2291.978515625,
2360
+ "learning_rate": 0.00017802548803739785,
2361
+ "loss": 13.0596,
2362
+ "step": 335
2363
+ },
2364
+ {
2365
+ "epoch": 0.19099047889725734,
2366
+ "grad_norm": 2809.587158203125,
2367
+ "learning_rate": 0.00017789527674208052,
2368
+ "loss": 12.392,
2369
+ "step": 336
2370
+ },
2371
+ {
2372
+ "epoch": 0.19155890294159442,
2373
+ "grad_norm": 1859.555908203125,
2374
+ "learning_rate": 0.00017776472871678295,
2375
+ "loss": 12.668,
2376
+ "step": 337
2377
+ },
2378
+ {
2379
+ "epoch": 0.19212732698593152,
2380
+ "grad_norm": 2988.171142578125,
2381
+ "learning_rate": 0.00017763384452584537,
2382
+ "loss": 11.2274,
2383
+ "step": 338
2384
+ },
2385
+ {
2386
+ "epoch": 0.1926957510302686,
2387
+ "grad_norm": 884.398193359375,
2388
+ "learning_rate": 0.00017750262473506103,
2389
+ "loss": 13.544,
2390
+ "step": 339
2391
+ },
2392
+ {
2393
+ "epoch": 0.19326417507460567,
2394
+ "grad_norm": 6331.970703125,
2395
+ "learning_rate": 0.00017737106991167414,
2396
+ "loss": 10.6381,
2397
+ "step": 340
2398
+ },
2399
+ {
2400
+ "epoch": 0.19383259911894274,
2401
+ "grad_norm": 4517.8837890625,
2402
+ "learning_rate": 0.0001772391806243771,
2403
+ "loss": 12.4022,
2404
+ "step": 341
2405
+ },
2406
+ {
2407
+ "epoch": 0.1944010231632798,
2408
+ "grad_norm": 5656.2353515625,
2409
+ "learning_rate": 0.00017710695744330813,
2410
+ "loss": 11.8059,
2411
+ "step": 342
2412
+ },
2413
+ {
2414
+ "epoch": 0.19496944720761689,
2415
+ "grad_norm": 2568.832275390625,
2416
+ "learning_rate": 0.00017697440094004892,
2417
+ "loss": 10.8074,
2418
+ "step": 343
2419
+ },
2420
+ {
2421
+ "epoch": 0.19553787125195396,
2422
+ "grad_norm": 9009.8232421875,
2423
+ "learning_rate": 0.00017684151168762192,
2424
+ "loss": 10.8937,
2425
+ "step": 344
2426
+ },
2427
+ {
2428
+ "epoch": 0.19610629529629103,
2429
+ "grad_norm": 3261.870849609375,
2430
+ "learning_rate": 0.00017670829026048817,
2431
+ "loss": 10.4608,
2432
+ "step": 345
2433
+ },
2434
+ {
2435
+ "epoch": 0.1966747193406281,
2436
+ "grad_norm": 2830.861572265625,
2437
+ "learning_rate": 0.0001765747372345445,
2438
+ "loss": 14.387,
2439
+ "step": 346
2440
+ },
2441
+ {
2442
+ "epoch": 0.19724314338496518,
2443
+ "grad_norm": 902.2265625,
2444
+ "learning_rate": 0.0001764408531871213,
2445
+ "loss": 13.6699,
2446
+ "step": 347
2447
+ },
2448
+ {
2449
+ "epoch": 0.19781156742930225,
2450
+ "grad_norm": 4266.66650390625,
2451
+ "learning_rate": 0.00017630663869697992,
2452
+ "loss": 12.4768,
2453
+ "step": 348
2454
+ },
2455
+ {
2456
+ "epoch": 0.19837999147363933,
2457
+ "grad_norm": 2162.527587890625,
2458
+ "learning_rate": 0.00017617209434431005,
2459
+ "loss": 12.7676,
2460
+ "step": 349
2461
+ },
2462
+ {
2463
+ "epoch": 0.1989484155179764,
2464
+ "grad_norm": 2415.94091796875,
2465
+ "learning_rate": 0.00017603722071072748,
2466
+ "loss": 11.1853,
2467
+ "step": 350
2468
+ },
2469
+ {
2470
+ "epoch": 0.19951683956231347,
2471
+ "grad_norm": 2408.1376953125,
2472
+ "learning_rate": 0.00017590201837927133,
2473
+ "loss": 11.8255,
2474
+ "step": 351
2475
+ },
2476
+ {
2477
+ "epoch": 0.20008526360665055,
2478
+ "grad_norm": 2698.591552734375,
2479
+ "learning_rate": 0.0001757664879344016,
2480
+ "loss": 11.0812,
2481
+ "step": 352
2482
+ },
2483
+ {
2484
+ "epoch": 0.20065368765098765,
2485
+ "grad_norm": 2491.31787109375,
2486
+ "learning_rate": 0.00017563062996199687,
2487
+ "loss": 11.8187,
2488
+ "step": 353
2489
+ },
2490
+ {
2491
+ "epoch": 0.20122211169532472,
2492
+ "grad_norm": 9000.8408203125,
2493
+ "learning_rate": 0.00017549444504935132,
2494
+ "loss": 12.9253,
2495
+ "step": 354
2496
+ },
2497
+ {
2498
+ "epoch": 0.2017905357396618,
2499
+ "grad_norm": 2151.921875,
2500
+ "learning_rate": 0.0001753579337851727,
2501
+ "loss": 12.3966,
2502
+ "step": 355
2503
+ },
2504
+ {
2505
+ "epoch": 0.20235895978399887,
2506
+ "grad_norm": 5135.994140625,
2507
+ "learning_rate": 0.00017522109675957928,
2508
+ "loss": 11.432,
2509
+ "step": 356
2510
+ },
2511
+ {
2512
+ "epoch": 0.20292738382833594,
2513
+ "grad_norm": 2409.67626953125,
2514
+ "learning_rate": 0.00017508393456409772,
2515
+ "loss": 11.6589,
2516
+ "step": 357
2517
+ },
2518
+ {
2519
+ "epoch": 0.20349580787267302,
2520
+ "grad_norm": 1465.730224609375,
2521
+ "learning_rate": 0.00017494644779166027,
2522
+ "loss": 11.9227,
2523
+ "step": 358
2524
+ },
2525
+ {
2526
+ "epoch": 0.2040642319170101,
2527
+ "grad_norm": 1712.6947021484375,
2528
+ "learning_rate": 0.00017480863703660235,
2529
+ "loss": 11.9249,
2530
+ "step": 359
2531
+ },
2532
+ {
2533
+ "epoch": 0.20463265596134717,
2534
+ "grad_norm": 20784.3359375,
2535
+ "learning_rate": 0.00017467050289465981,
2536
+ "loss": 11.027,
2537
+ "step": 360
2538
+ },
2539
+ {
2540
+ "epoch": 0.20520108000568424,
2541
+ "grad_norm": 2024.4476318359375,
2542
+ "learning_rate": 0.0001745320459629665,
2543
+ "loss": 11.4299,
2544
+ "step": 361
2545
+ },
2546
+ {
2547
+ "epoch": 0.2057695040500213,
2548
+ "grad_norm": 6059.2177734375,
2549
+ "learning_rate": 0.0001743932668400517,
2550
+ "loss": 10.421,
2551
+ "step": 362
2552
+ },
2553
+ {
2554
+ "epoch": 0.20633792809435839,
2555
+ "grad_norm": 1232.2406005859375,
2556
+ "learning_rate": 0.0001742541661258374,
2557
+ "loss": 11.6251,
2558
+ "step": 363
2559
+ },
2560
+ {
2561
+ "epoch": 0.20690635213869546,
2562
+ "grad_norm": 2170.210205078125,
2563
+ "learning_rate": 0.00017411474442163578,
2564
+ "loss": 14.4437,
2565
+ "step": 364
2566
+ },
2567
+ {
2568
+ "epoch": 0.20747477618303253,
2569
+ "grad_norm": 1405.988037109375,
2570
+ "learning_rate": 0.00017397500233014667,
2571
+ "loss": 10.8243,
2572
+ "step": 365
2573
+ },
2574
+ {
2575
+ "epoch": 0.2080432002273696,
2576
+ "grad_norm": 2395.445556640625,
2577
+ "learning_rate": 0.00017383494045545485,
2578
+ "loss": 11.6437,
2579
+ "step": 366
2580
+ },
2581
+ {
2582
+ "epoch": 0.20861162427170668,
2583
+ "grad_norm": 10106.9404296875,
2584
+ "learning_rate": 0.0001736945594030275,
2585
+ "loss": 12.8797,
2586
+ "step": 367
2587
+ },
2588
+ {
2589
+ "epoch": 0.20918004831604378,
2590
+ "grad_norm": 5351.23779296875,
2591
+ "learning_rate": 0.0001735538597797115,
2592
+ "loss": 12.555,
2593
+ "step": 368
2594
+ },
2595
+ {
2596
+ "epoch": 0.20974847236038086,
2597
+ "grad_norm": 8931.0205078125,
2598
+ "learning_rate": 0.00017341284219373092,
2599
+ "loss": 11.1019,
2600
+ "step": 369
2601
+ },
2602
+ {
2603
+ "epoch": 0.21031689640471793,
2604
+ "grad_norm": 3570.2431640625,
2605
+ "learning_rate": 0.00017327150725468437,
2606
+ "loss": 13.7804,
2607
+ "step": 370
2608
+ },
2609
+ {
2610
+ "epoch": 0.210885320449055,
2611
+ "grad_norm": 9746.34375,
2612
+ "learning_rate": 0.00017312985557354225,
2613
+ "loss": 11.3145,
2614
+ "step": 371
2615
+ },
2616
+ {
2617
+ "epoch": 0.21145374449339208,
2618
+ "grad_norm": 2833.336669921875,
2619
+ "learning_rate": 0.00017298788776264423,
2620
+ "loss": 14.7045,
2621
+ "step": 372
2622
+ },
2623
+ {
2624
+ "epoch": 0.21202216853772915,
2625
+ "grad_norm": 3378.20654296875,
2626
+ "learning_rate": 0.00017284560443569657,
2627
+ "loss": 9.795,
2628
+ "step": 373
2629
+ },
2630
+ {
2631
+ "epoch": 0.21259059258206622,
2632
+ "grad_norm": 5840.08447265625,
2633
+ "learning_rate": 0.00017270300620776942,
2634
+ "loss": 10.8172,
2635
+ "step": 374
2636
+ },
2637
+ {
2638
+ "epoch": 0.2131590166264033,
2639
+ "grad_norm": 3910.785888671875,
2640
+ "learning_rate": 0.0001725600936952943,
2641
+ "loss": 11.8837,
2642
+ "step": 375
2643
+ },
2644
+ {
2645
+ "epoch": 0.21372744067074037,
2646
+ "grad_norm": 2122.12158203125,
2647
+ "learning_rate": 0.00017241686751606115,
2648
+ "loss": 12.598,
2649
+ "step": 376
2650
+ },
2651
+ {
2652
+ "epoch": 0.21429586471507744,
2653
+ "grad_norm": 2105.682373046875,
2654
+ "learning_rate": 0.00017227332828921612,
2655
+ "loss": 12.0783,
2656
+ "step": 377
2657
+ },
2658
+ {
2659
+ "epoch": 0.21486428875941452,
2660
+ "grad_norm": 3060.28369140625,
2661
+ "learning_rate": 0.0001721294766352583,
2662
+ "loss": 10.7985,
2663
+ "step": 378
2664
+ },
2665
+ {
2666
+ "epoch": 0.2154327128037516,
2667
+ "grad_norm": 2431.94775390625,
2668
+ "learning_rate": 0.00017198531317603766,
2669
+ "loss": 11.4005,
2670
+ "step": 379
2671
+ },
2672
+ {
2673
+ "epoch": 0.21600113684808867,
2674
+ "grad_norm": 1950.9508056640625,
2675
+ "learning_rate": 0.00017184083853475182,
2676
+ "loss": 12.1844,
2677
+ "step": 380
2678
+ },
2679
+ {
2680
+ "epoch": 0.21656956089242574,
2681
+ "grad_norm": 7467.8984375,
2682
+ "learning_rate": 0.00017169605333594375,
2683
+ "loss": 12.6121,
2684
+ "step": 381
2685
+ },
2686
+ {
2687
+ "epoch": 0.21656956089242574,
2688
+ "eval_loss": 2.956733465194702,
2689
+ "eval_runtime": 5.4757,
2690
+ "eval_samples_per_second": 135.326,
2691
+ "eval_steps_per_second": 67.754,
2692
+ "step": 381
2693
+ }
2694
+ ],
2695
+ "logging_steps": 1,
2696
+ "max_steps": 1521,
2697
+ "num_input_tokens_seen": 0,
2698
+ "num_train_epochs": 1,
2699
+ "save_steps": 381,
2700
+ "stateful_callbacks": {
2701
+ "TrainerControl": {
2702
+ "args": {
2703
+ "should_epoch_stop": false,
2704
+ "should_evaluate": false,
2705
+ "should_log": false,
2706
+ "should_save": true,
2707
+ "should_training_stop": false
2708
+ },
2709
+ "attributes": {}
2710
+ }
2711
+ },
2712
+ "total_flos": 1594233024675840.0,
2713
+ "train_batch_size": 2,
2714
+ "trial_name": null,
2715
+ "trial_params": null
2716
+ }
last-checkpoint/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ebc51a4f0a402f36df55f990394a1b283af0ce612bd14acc31af5eff9a32d946
3
+ size 6776