Hugging Face's logo

rmdhirr
/
kenanga-lora-b-2-2400

Text Generation
PEFT
Safetensors
Transformers
lora
sft
trl
unsloth
conversational
Model card Files
xet
 Community
rmdhirr commited on
Commit
da82642
·
verified ·
1 Parent(s): b9abc86

Upload folder using huggingface_hub

Browse files
Files changed (17) hide show
  1. .gitattributes +1 -0
  2. README.md +210 -0
  3. adapter_config.json +43 -0
  4. adapter_model.safetensors +3 -0
  5. added_tokens.json +3 -0
  6. chat_template.jinja +47 -0
  7. optimizer.pt +3 -0
  8. preprocessor_config.json +29 -0
  9. processor_config.json +4 -0
  10. rng_state.pth +3 -0
  11. scheduler.pt +3 -0
  12. special_tokens_map.json +33 -0
  13. tokenizer.json +3 -0
  14. tokenizer.model +3 -0
  15. tokenizer_config.json +0 -0
  16. trainer_state.json +3658 -0
  17. training_args.bin +3 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,210 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: unsloth/gemma-3-12b-it-unsloth-bnb-4bit
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:unsloth/gemma-3-12b-it-unsloth-bnb-4bit
7
+ - lora
8
+ - sft
9
+ - transformers
10
+ - trl
11
+ - unsloth
12
+ ---
13
+
14
+ # Model Card for Model ID
15
+
16
+ <!-- Provide a quick summary of what the model is/does. -->
17
+
18
+
19
+
20
+ ## Model Details
21
+
22
+ ### Model Description
23
+
24
+ <!-- Provide a longer summary of what this model is. -->
25
+
26
+
27
+
28
+ - **Developed by:** [More Information Needed]
29
+ - **Funded by [optional]:** [More Information Needed]
30
+ - **Shared by [optional]:** [More Information Needed]
31
+ - **Model type:** [More Information Needed]
32
+ - **Language(s) (NLP):** [More Information Needed]
33
+ - **License:** [More Information Needed]
34
+ - **Finetuned from model [optional]:** [More Information Needed]
35
+
36
+ ### Model Sources [optional]
37
+
38
+ <!-- Provide the basic links for the model. -->
39
+
40
+ - **Repository:** [More Information Needed]
41
+ - **Paper [optional]:** [More Information Needed]
42
+ - **Demo [optional]:** [More Information Needed]
43
+
44
+ ## Uses
45
+
46
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
47
+
48
+ ### Direct Use
49
+
50
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
51
+
52
+ [More Information Needed]
53
+
54
+ ### Downstream Use [optional]
55
+
56
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
57
+
58
+ [More Information Needed]
59
+
60
+ ### Out-of-Scope Use
61
+
62
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
63
+
64
+ [More Information Needed]
65
+
66
+ ## Bias, Risks, and Limitations
67
+
68
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
69
+
70
+ [More Information Needed]
71
+
72
+ ### Recommendations
73
+
74
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
75
+
76
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
77
+
78
+ ## How to Get Started with the Model
79
+
80
+ Use the code below to get started with the model.
81
+
82
+ [More Information Needed]
83
+
84
+ ## Training Details
85
+
86
+ ### Training Data
87
+
88
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
89
+
90
+ [More Information Needed]
91
+
92
+ ### Training Procedure
93
+
94
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
95
+
96
+ #### Preprocessing [optional]
97
+
98
+ [More Information Needed]
99
+
100
+
101
+ #### Training Hyperparameters
102
+
103
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
104
+
105
+ #### Speeds, Sizes, Times [optional]
106
+
107
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
108
+
109
+ [More Information Needed]
110
+
111
+ ## Evaluation
112
+
113
+ <!-- This section describes the evaluation protocols and provides the results. -->
114
+
115
+ ### Testing Data, Factors & Metrics
116
+
117
+ #### Testing Data
118
+
119
+ <!-- This should link to a Dataset Card if possible. -->
120
+
121
+ [More Information Needed]
122
+
123
+ #### Factors
124
+
125
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
126
+
127
+ [More Information Needed]
128
+
129
+ #### Metrics
130
+
131
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
132
+
133
+ [More Information Needed]
134
+
135
+ ### Results
136
+
137
+ [More Information Needed]
138
+
139
+ #### Summary
140
+
141
+
142
+
143
+ ## Model Examination [optional]
144
+
145
+ <!-- Relevant interpretability work for the model goes here -->
146
+
147
+ [More Information Needed]
148
+
149
+ ## Environmental Impact
150
+
151
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
152
+
153
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
154
+
155
+ - **Hardware Type:** [More Information Needed]
156
+ - **Hours used:** [More Information Needed]
157
+ - **Cloud Provider:** [More Information Needed]
158
+ - **Compute Region:** [More Information Needed]
159
+ - **Carbon Emitted:** [More Information Needed]
160
+
161
+ ## Technical Specifications [optional]
162
+
163
+ ### Model Architecture and Objective
164
+
165
+ [More Information Needed]
166
+
167
+ ### Compute Infrastructure
168
+
169
+ [More Information Needed]
170
+
171
+ #### Hardware
172
+
173
+ [More Information Needed]
174
+
175
+ #### Software
176
+
177
+ [More Information Needed]
178
+
179
+ ## Citation [optional]
180
+
181
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
182
+
183
+ **BibTeX:**
184
+
185
+ [More Information Needed]
186
+
187
+ **APA:**
188
+
189
+ [More Information Needed]
190
+
191
+ ## Glossary [optional]
192
+
193
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
194
+
195
+ [More Information Needed]
196
+
197
+ ## More Information [optional]
198
+
199
+ [More Information Needed]
200
+
201
+ ## Model Card Authors [optional]
202
+
203
+ [More Information Needed]
204
+
205
+ ## Model Card Contact
206
+
207
+ [More Information Needed]
208
+ ### Framework versions
209
+
210
+ - PEFT 0.16.0
adapter_config.json ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "unsloth/gemma-3-12b-it-unsloth-bnb-4bit",
5
+ "bias": "none",
6
+ "corda_config": null,
7
+ "eva_config": null,
8
+ "exclude_modules": null,
9
+ "fan_in_fan_out": false,
10
+ "inference_mode": true,
11
+ "init_lora_weights": true,
12
+ "layer_replication": null,
13
+ "layers_pattern": null,
14
+ "layers_to_transform": null,
15
+ "loftq_config": {},
16
+ "lora_alpha": 64,
17
+ "lora_bias": false,
18
+ "lora_dropout": 0.05,
19
+ "megatron_config": null,
20
+ "megatron_core": "megatron.core",
21
+ "modules_to_save": null,
22
+ "peft_type": "LORA",
23
+ "qalora_group_size": 16,
24
+ "r": 64,
25
+ "rank_pattern": {},
26
+ "revision": null,
27
+ "target_modules": [
28
+ "down_proj",
29
+ "gate_proj",
30
+ "o_proj",
31
+ "embed_tokens",
32
+ "lm_head",
33
+ "q_proj",
34
+ "v_proj",
35
+ "k_proj",
36
+ "up_proj"
37
+ ],
38
+ "task_type": "CAUSAL_LM",
39
+ "trainable_token_indices": null,
40
+ "use_dora": false,
41
+ "use_qalora": false,
42
+ "use_rslora": false
43
+ }
adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5b59b4ad57f88276ac307919b9d65ea736538ccf578a6634d63067e2ed5c3458
3
+ size 5259164736
added_tokens.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "<image_soft_token>": 262144
3
+ }
chat_template.jinja ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {{ bos_token }}
2
+ {%- if messages[0]['role'] == 'system' -%}
3
+ {%- if messages[0]['content'] is string -%}
4
+ {%- set first_user_prefix = messages[0]['content'] + '
5
+
6
+ ' -%}
7
+ {%- else -%}
8
+ {%- set first_user_prefix = messages[0]['content'][0]['text'] + '
9
+
10
+ ' -%}
11
+ {%- endif -%}
12
+ {%- set loop_messages = messages[1:] -%}
13
+ {%- else -%}
14
+ {%- set first_user_prefix = "" -%}
15
+ {%- set loop_messages = messages -%}
16
+ {%- endif -%}
17
+ {%- for message in loop_messages -%}
18
+ {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
19
+ {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
20
+ {%- endif -%}
21
+ {%- if (message['role'] == 'assistant') -%}
22
+ {%- set role = "model" -%}
23
+ {%- else -%}
24
+ {%- set role = message['role'] -%}
25
+ {%- endif -%}
26
+ {{ '<start_of_turn>' + role + '
27
+ ' + (first_user_prefix if loop.first else "") }}
28
+ {%- if message['content'] is string -%}
29
+ {{ message['content'] | trim }}
30
+ {%- elif message['content'] is iterable -%}
31
+ {%- for item in message['content'] -%}
32
+ {%- if item['type'] == 'image' -%}
33
+ {{ '<start_of_image>' }}
34
+ {%- elif item['type'] == 'text' -%}
35
+ {{ item['text'] | trim }}
36
+ {%- endif -%}
37
+ {%- endfor -%}
38
+ {%- else -%}
39
+ {{ raise_exception("Invalid content type") }}
40
+ {%- endif -%}
41
+ {{ '<end_of_turn>
42
+ ' }}
43
+ {%- endfor -%}
44
+ {%- if add_generation_prompt -%}
45
+ {{'<start_of_turn>model
46
+ '}}
47
+ {%- endif -%}
optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8501eee9adfc44ef49550de16020462e39bde58d84f5e577576a7234f3b3954
3
+ size 569083247
preprocessor_config.json ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_convert_rgb": null,
3
+ "do_normalize": true,
4
+ "do_pan_and_scan": null,
5
+ "do_rescale": true,
6
+ "do_resize": true,
7
+ "image_mean": [
8
+ 0.5,
9
+ 0.5,
10
+ 0.5
11
+ ],
12
+ "image_processor_type": "Gemma3ImageProcessor",
13
+ "image_seq_length": 256,
14
+ "image_std": [
15
+ 0.5,
16
+ 0.5,
17
+ 0.5
18
+ ],
19
+ "pan_and_scan_max_num_crops": null,
20
+ "pan_and_scan_min_crop_size": null,
21
+ "pan_and_scan_min_ratio_to_activate": null,
22
+ "processor_class": "Gemma3Processor",
23
+ "resample": 2,
24
+ "rescale_factor": 0.00392156862745098,
25
+ "size": {
26
+ "height": 896,
27
+ "width": 896
28
+ }
29
+ }
processor_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "image_seq_length": 256,
3
+ "processor_class": "Gemma3Processor"
4
+ }
rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:74b5c5db9ea92cd2690f112d8ebd77fc693c067fb9027aed57a650c17b6f8e9d
3
+ size 14645
scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b411fbc03a3342ee9c30051850b5847d90196c02e4f3f9b241cfe652461be98a
3
+ size 1465
special_tokens_map.json ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "boi_token": "<start_of_image>",
3
+ "bos_token": {
4
+ "content": "<bos>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false
9
+ },
10
+ "eoi_token": "<end_of_image>",
11
+ "eos_token": {
12
+ "content": "<end_of_turn>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false
17
+ },
18
+ "image_token": "<image_soft_token>",
19
+ "pad_token": {
20
+ "content": "<pad>",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false
25
+ },
26
+ "unk_token": {
27
+ "content": "<unk>",
28
+ "lstrip": false,
29
+ "normalized": false,
30
+ "rstrip": false,
31
+ "single_word": false
32
+ }
33
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4667f2089529e8e7657cfb6d1c19910ae71ff5f28aa7ab2ff2763330affad795
3
+ size 33384568
tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
3
+ size 4689074
tokenizer_config.json ADDED
The diff for this file is too large to render. See raw diff
 
trainer_state.json ADDED
@@ -0,0 +1,3658 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": 2400,
3
+ "best_metric": 1.1524348258972168,
4
+ "best_model_checkpoint": "outputs/checkpoint-2400",
5
+ "epoch": 0.37644106344600425,
6
+ "eval_steps": 300,
7
+ "global_step": 2400,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.0009614460148062686,
14
+ "grad_norm": 5.263479232788086,
15
+ "learning_rate": 2.3032629558541267e-07,
16
+ "loss": 2.2998,
17
+ "step": 5
18
+ },
19
+ {
20
+ "epoch": 0.0019228920296125373,
21
+ "grad_norm": 4.919901371002197,
22
+ "learning_rate": 5.182341650671785e-07,
23
+ "loss": 2.2468,
24
+ "step": 10
25
+ },
26
+ {
27
+ "epoch": 0.002884338044418806,
28
+ "grad_norm": 3.7949042320251465,
29
+ "learning_rate": 8.061420345489444e-07,
30
+ "loss": 2.2097,
31
+ "step": 15
32
+ },
33
+ {
34
+ "epoch": 0.0038457840592250746,
35
+ "grad_norm": 3.5610361099243164,
36
+ "learning_rate": 1.0940499040307102e-06,
37
+ "loss": 2.083,
38
+ "step": 20
39
+ },
40
+ {
41
+ "epoch": 0.0048072300740313436,
42
+ "grad_norm": 3.32427978515625,
43
+ "learning_rate": 1.381957773512476e-06,
44
+ "loss": 2.0052,
45
+ "step": 25
46
+ },
47
+ {
48
+ "epoch": 0.005768676088837612,
49
+ "grad_norm": 3.5147716999053955,
50
+ "learning_rate": 1.6698656429942417e-06,
51
+ "loss": 2.043,
52
+ "step": 30
53
+ },
54
+ {
55
+ "epoch": 0.006730122103643881,
56
+ "grad_norm": 2.091193437576294,
57
+ "learning_rate": 1.9577735124760077e-06,
58
+ "loss": 2.0161,
59
+ "step": 35
60
+ },
61
+ {
62
+ "epoch": 0.007691568118450149,
63
+ "grad_norm": 2.3086185455322266,
64
+ "learning_rate": 2.2456813819577732e-06,
65
+ "loss": 1.9567,
66
+ "step": 40
67
+ },
68
+ {
69
+ "epoch": 0.008653014133256417,
70
+ "grad_norm": 1.6346129179000854,
71
+ "learning_rate": 2.5335892514395392e-06,
72
+ "loss": 1.8985,
73
+ "step": 45
74
+ },
75
+ {
76
+ "epoch": 0.009614460148062687,
77
+ "grad_norm": 1.260735034942627,
78
+ "learning_rate": 2.821497120921305e-06,
79
+ "loss": 1.8132,
80
+ "step": 50
81
+ },
82
+ {
83
+ "epoch": 0.010575906162868956,
84
+ "grad_norm": 1.5411990880966187,
85
+ "learning_rate": 3.109404990403071e-06,
86
+ "loss": 1.6944,
87
+ "step": 55
88
+ },
89
+ {
90
+ "epoch": 0.011537352177675224,
91
+ "grad_norm": 1.319563388824463,
92
+ "learning_rate": 3.397312859884837e-06,
93
+ "loss": 1.7036,
94
+ "step": 60
95
+ },
96
+ {
97
+ "epoch": 0.012498798192481493,
98
+ "grad_norm": 1.0878187417984009,
99
+ "learning_rate": 3.6852207293666026e-06,
100
+ "loss": 1.7662,
101
+ "step": 65
102
+ },
103
+ {
104
+ "epoch": 0.013460244207287761,
105
+ "grad_norm": 1.1386423110961914,
106
+ "learning_rate": 3.973128598848369e-06,
107
+ "loss": 1.6783,
108
+ "step": 70
109
+ },
110
+ {
111
+ "epoch": 0.01442169022209403,
112
+ "grad_norm": 1.2391140460968018,
113
+ "learning_rate": 4.2610364683301346e-06,
114
+ "loss": 1.5672,
115
+ "step": 75
116
+ },
117
+ {
118
+ "epoch": 0.015383136236900298,
119
+ "grad_norm": 1.0566521883010864,
120
+ "learning_rate": 4.5489443378119e-06,
121
+ "loss": 1.6949,
122
+ "step": 80
123
+ },
124
+ {
125
+ "epoch": 0.016344582251706565,
126
+ "grad_norm": 1.0937573909759521,
127
+ "learning_rate": 4.8368522072936665e-06,
128
+ "loss": 1.5464,
129
+ "step": 85
130
+ },
131
+ {
132
+ "epoch": 0.017306028266512834,
133
+ "grad_norm": 1.0400891304016113,
134
+ "learning_rate": 5.124760076775432e-06,
135
+ "loss": 1.5824,
136
+ "step": 90
137
+ },
138
+ {
139
+ "epoch": 0.018267474281319102,
140
+ "grad_norm": 1.1960211992263794,
141
+ "learning_rate": 5.4126679462571984e-06,
142
+ "loss": 1.6269,
143
+ "step": 95
144
+ },
145
+ {
146
+ "epoch": 0.019228920296125374,
147
+ "grad_norm": 1.197791576385498,
148
+ "learning_rate": 5.7005758157389635e-06,
149
+ "loss": 1.5177,
150
+ "step": 100
151
+ },
152
+ {
153
+ "epoch": 0.020190366310931643,
154
+ "grad_norm": 0.9536961913108826,
155
+ "learning_rate": 5.9884836852207295e-06,
156
+ "loss": 1.6222,
157
+ "step": 105
158
+ },
159
+ {
160
+ "epoch": 0.02115181232573791,
161
+ "grad_norm": 0.9854758977890015,
162
+ "learning_rate": 6.2763915547024955e-06,
163
+ "loss": 1.5503,
164
+ "step": 110
165
+ },
166
+ {
167
+ "epoch": 0.02211325834054418,
168
+ "grad_norm": 0.8579792976379395,
169
+ "learning_rate": 6.5642994241842614e-06,
170
+ "loss": 1.5246,
171
+ "step": 115
172
+ },
173
+ {
174
+ "epoch": 0.02307470435535045,
175
+ "grad_norm": 0.8161287307739258,
176
+ "learning_rate": 6.852207293666027e-06,
177
+ "loss": 1.5207,
178
+ "step": 120
179
+ },
180
+ {
181
+ "epoch": 0.024036150370156717,
182
+ "grad_norm": 1.0825903415679932,
183
+ "learning_rate": 7.1401151631477925e-06,
184
+ "loss": 1.4963,
185
+ "step": 125
186
+ },
187
+ {
188
+ "epoch": 0.024997596384962985,
189
+ "grad_norm": 0.9904459118843079,
190
+ "learning_rate": 7.4280230326295585e-06,
191
+ "loss": 1.5351,
192
+ "step": 130
193
+ },
194
+ {
195
+ "epoch": 0.025959042399769254,
196
+ "grad_norm": 1.0706708431243896,
197
+ "learning_rate": 7.715930902111324e-06,
198
+ "loss": 1.477,
199
+ "step": 135
200
+ },
201
+ {
202
+ "epoch": 0.026920488414575523,
203
+ "grad_norm": 0.910792887210846,
204
+ "learning_rate": 8.003838771593091e-06,
205
+ "loss": 1.563,
206
+ "step": 140
207
+ },
208
+ {
209
+ "epoch": 0.02788193442938179,
210
+ "grad_norm": 0.8813047409057617,
211
+ "learning_rate": 8.291746641074856e-06,
212
+ "loss": 1.5286,
213
+ "step": 145
214
+ },
215
+ {
216
+ "epoch": 0.02884338044418806,
217
+ "grad_norm": 1.135494589805603,
218
+ "learning_rate": 8.579654510556621e-06,
219
+ "loss": 1.4987,
220
+ "step": 150
221
+ },
222
+ {
223
+ "epoch": 0.029804826458994328,
224
+ "grad_norm": 1.4200812578201294,
225
+ "learning_rate": 8.867562380038388e-06,
226
+ "loss": 1.5176,
227
+ "step": 155
228
+ },
229
+ {
230
+ "epoch": 0.030766272473800597,
231
+ "grad_norm": 0.9361958503723145,
232
+ "learning_rate": 9.155470249520153e-06,
233
+ "loss": 1.5375,
234
+ "step": 160
235
+ },
236
+ {
237
+ "epoch": 0.031727718488606865,
238
+ "grad_norm": 0.7996511459350586,
239
+ "learning_rate": 9.44337811900192e-06,
240
+ "loss": 1.504,
241
+ "step": 165
242
+ },
243
+ {
244
+ "epoch": 0.03268916450341313,
245
+ "grad_norm": 0.8341512680053711,
246
+ "learning_rate": 9.731285988483685e-06,
247
+ "loss": 1.5099,
248
+ "step": 170
249
+ },
250
+ {
251
+ "epoch": 0.0336506105182194,
252
+ "grad_norm": 0.8165861368179321,
253
+ "learning_rate": 1.001919385796545e-05,
254
+ "loss": 1.4714,
255
+ "step": 175
256
+ },
257
+ {
258
+ "epoch": 0.03461205653302567,
259
+ "grad_norm": 1.0153369903564453,
260
+ "learning_rate": 1.0307101727447217e-05,
261
+ "loss": 1.4675,
262
+ "step": 180
263
+ },
264
+ {
265
+ "epoch": 0.03557350254783194,
266
+ "grad_norm": 0.8753921985626221,
267
+ "learning_rate": 1.0595009596928982e-05,
268
+ "loss": 1.478,
269
+ "step": 185
270
+ },
271
+ {
272
+ "epoch": 0.036534948562638205,
273
+ "grad_norm": 0.8925352096557617,
274
+ "learning_rate": 1.088291746641075e-05,
275
+ "loss": 1.4238,
276
+ "step": 190
277
+ },
278
+ {
279
+ "epoch": 0.037496394577444477,
280
+ "grad_norm": 0.8326230049133301,
281
+ "learning_rate": 1.1170825335892514e-05,
282
+ "loss": 1.4648,
283
+ "step": 195
284
+ },
285
+ {
286
+ "epoch": 0.03845784059225075,
287
+ "grad_norm": 1.0663974285125732,
288
+ "learning_rate": 1.1458733205374281e-05,
289
+ "loss": 1.5123,
290
+ "step": 200
291
+ },
292
+ {
293
+ "epoch": 0.039419286607057014,
294
+ "grad_norm": 1.0833837985992432,
295
+ "learning_rate": 1.1746641074856046e-05,
296
+ "loss": 1.5031,
297
+ "step": 205
298
+ },
299
+ {
300
+ "epoch": 0.040380732621863286,
301
+ "grad_norm": 0.8054072260856628,
302
+ "learning_rate": 1.2034548944337811e-05,
303
+ "loss": 1.4416,
304
+ "step": 210
305
+ },
306
+ {
307
+ "epoch": 0.04134217863666955,
308
+ "grad_norm": 0.8595219850540161,
309
+ "learning_rate": 1.2322456813819578e-05,
310
+ "loss": 1.4417,
311
+ "step": 215
312
+ },
313
+ {
314
+ "epoch": 0.04230362465147582,
315
+ "grad_norm": 0.8635595440864563,
316
+ "learning_rate": 1.2610364683301345e-05,
317
+ "loss": 1.4175,
318
+ "step": 220
319
+ },
320
+ {
321
+ "epoch": 0.04326507066628209,
322
+ "grad_norm": 0.8640530705451965,
323
+ "learning_rate": 1.289827255278311e-05,
324
+ "loss": 1.4442,
325
+ "step": 225
326
+ },
327
+ {
328
+ "epoch": 0.04422651668108836,
329
+ "grad_norm": 0.8653771281242371,
330
+ "learning_rate": 1.3186180422264875e-05,
331
+ "loss": 1.4193,
332
+ "step": 230
333
+ },
334
+ {
335
+ "epoch": 0.045187962695894625,
336
+ "grad_norm": 0.7947669625282288,
337
+ "learning_rate": 1.347408829174664e-05,
338
+ "loss": 1.4092,
339
+ "step": 235
340
+ },
341
+ {
342
+ "epoch": 0.0461494087107009,
343
+ "grad_norm": 0.8416963815689087,
344
+ "learning_rate": 1.3761996161228407e-05,
345
+ "loss": 1.5512,
346
+ "step": 240
347
+ },
348
+ {
349
+ "epoch": 0.04711085472550716,
350
+ "grad_norm": 0.9726578593254089,
351
+ "learning_rate": 1.4049904030710174e-05,
352
+ "loss": 1.4525,
353
+ "step": 245
354
+ },
355
+ {
356
+ "epoch": 0.048072300740313434,
357
+ "grad_norm": 1.0061571598052979,
358
+ "learning_rate": 1.4337811900191939e-05,
359
+ "loss": 1.4552,
360
+ "step": 250
361
+ },
362
+ {
363
+ "epoch": 0.0490337467551197,
364
+ "grad_norm": 0.8465285897254944,
365
+ "learning_rate": 1.4625719769673704e-05,
366
+ "loss": 1.3991,
367
+ "step": 255
368
+ },
369
+ {
370
+ "epoch": 0.04999519276992597,
371
+ "grad_norm": 0.9697036147117615,
372
+ "learning_rate": 1.491362763915547e-05,
373
+ "loss": 1.4067,
374
+ "step": 260
375
+ },
376
+ {
377
+ "epoch": 0.050956638784732236,
378
+ "grad_norm": 0.9910932183265686,
379
+ "learning_rate": 1.5201535508637236e-05,
380
+ "loss": 1.4834,
381
+ "step": 265
382
+ },
383
+ {
384
+ "epoch": 0.05191808479953851,
385
+ "grad_norm": 0.937353789806366,
386
+ "learning_rate": 1.5489443378119003e-05,
387
+ "loss": 1.4806,
388
+ "step": 270
389
+ },
390
+ {
391
+ "epoch": 0.05287953081434477,
392
+ "grad_norm": 0.873612105846405,
393
+ "learning_rate": 1.577735124760077e-05,
394
+ "loss": 1.4694,
395
+ "step": 275
396
+ },
397
+ {
398
+ "epoch": 0.053840976829151045,
399
+ "grad_norm": 0.7373448610305786,
400
+ "learning_rate": 1.6065259117082533e-05,
401
+ "loss": 1.4042,
402
+ "step": 280
403
+ },
404
+ {
405
+ "epoch": 0.05480242284395731,
406
+ "grad_norm": 0.771385908126831,
407
+ "learning_rate": 1.63531669865643e-05,
408
+ "loss": 1.418,
409
+ "step": 285
410
+ },
411
+ {
412
+ "epoch": 0.05576386885876358,
413
+ "grad_norm": 0.8392884731292725,
414
+ "learning_rate": 1.6641074856046063e-05,
415
+ "loss": 1.3782,
416
+ "step": 290
417
+ },
418
+ {
419
+ "epoch": 0.05672531487356985,
420
+ "grad_norm": 0.8971521258354187,
421
+ "learning_rate": 1.692898272552783e-05,
422
+ "loss": 1.503,
423
+ "step": 295
424
+ },
425
+ {
426
+ "epoch": 0.05768676088837612,
427
+ "grad_norm": 0.938372790813446,
428
+ "learning_rate": 1.72168905950096e-05,
429
+ "loss": 1.4005,
430
+ "step": 300
431
+ },
432
+ {
433
+ "epoch": 0.05768676088837612,
434
+ "eval_loss": 1.4069678783416748,
435
+ "eval_runtime": 1829.2346,
436
+ "eval_samples_per_second": 11.372,
437
+ "eval_steps_per_second": 2.843,
438
+ "step": 300
439
+ },
440
+ {
441
+ "epoch": 0.058648206903182384,
442
+ "grad_norm": 0.8293926119804382,
443
+ "learning_rate": 1.7504798464491364e-05,
444
+ "loss": 1.3648,
445
+ "step": 305
446
+ },
447
+ {
448
+ "epoch": 0.059609652917988656,
449
+ "grad_norm": 0.9733208417892456,
450
+ "learning_rate": 1.779270633397313e-05,
451
+ "loss": 1.4087,
452
+ "step": 310
453
+ },
454
+ {
455
+ "epoch": 0.06057109893279492,
456
+ "grad_norm": 0.9246333241462708,
457
+ "learning_rate": 1.8080614203454894e-05,
458
+ "loss": 1.4551,
459
+ "step": 315
460
+ },
461
+ {
462
+ "epoch": 0.061532544947601193,
463
+ "grad_norm": 0.7493950724601746,
464
+ "learning_rate": 1.836852207293666e-05,
465
+ "loss": 1.4551,
466
+ "step": 320
467
+ },
468
+ {
469
+ "epoch": 0.06249399096240746,
470
+ "grad_norm": 1.0027576684951782,
471
+ "learning_rate": 1.8656429942418428e-05,
472
+ "loss": 1.3908,
473
+ "step": 325
474
+ },
475
+ {
476
+ "epoch": 0.06345543697721373,
477
+ "grad_norm": 0.8040695190429688,
478
+ "learning_rate": 1.894433781190019e-05,
479
+ "loss": 1.424,
480
+ "step": 330
481
+ },
482
+ {
483
+ "epoch": 0.06441688299202,
484
+ "grad_norm": 0.8406041264533997,
485
+ "learning_rate": 1.9232245681381958e-05,
486
+ "loss": 1.353,
487
+ "step": 335
488
+ },
489
+ {
490
+ "epoch": 0.06537832900682626,
491
+ "grad_norm": 0.9334944486618042,
492
+ "learning_rate": 1.952015355086372e-05,
493
+ "loss": 1.431,
494
+ "step": 340
495
+ },
496
+ {
497
+ "epoch": 0.06633977502163253,
498
+ "grad_norm": 0.7912152409553528,
499
+ "learning_rate": 1.980806142034549e-05,
500
+ "loss": 1.4298,
501
+ "step": 345
502
+ },
503
+ {
504
+ "epoch": 0.0673012210364388,
505
+ "grad_norm": 0.9743331670761108,
506
+ "learning_rate": 2.009596928982726e-05,
507
+ "loss": 1.357,
508
+ "step": 350
509
+ },
510
+ {
511
+ "epoch": 0.06826266705124508,
512
+ "grad_norm": 0.9452155232429504,
513
+ "learning_rate": 2.0383877159309022e-05,
514
+ "loss": 1.4023,
515
+ "step": 355
516
+ },
517
+ {
518
+ "epoch": 0.06922411306605133,
519
+ "grad_norm": 0.8321967124938965,
520
+ "learning_rate": 2.067178502879079e-05,
521
+ "loss": 1.4188,
522
+ "step": 360
523
+ },
524
+ {
525
+ "epoch": 0.0701855590808576,
526
+ "grad_norm": 0.8946035504341125,
527
+ "learning_rate": 2.0959692898272552e-05,
528
+ "loss": 1.3417,
529
+ "step": 365
530
+ },
531
+ {
532
+ "epoch": 0.07114700509566388,
533
+ "grad_norm": 0.8722978234291077,
534
+ "learning_rate": 2.124760076775432e-05,
535
+ "loss": 1.4045,
536
+ "step": 370
537
+ },
538
+ {
539
+ "epoch": 0.07210845111047015,
540
+ "grad_norm": 1.0040546655654907,
541
+ "learning_rate": 2.1535508637236086e-05,
542
+ "loss": 1.4069,
543
+ "step": 375
544
+ },
545
+ {
546
+ "epoch": 0.07306989712527641,
547
+ "grad_norm": 0.8072942495346069,
548
+ "learning_rate": 2.182341650671785e-05,
549
+ "loss": 1.3406,
550
+ "step": 380
551
+ },
552
+ {
553
+ "epoch": 0.07403134314008268,
554
+ "grad_norm": 0.9878551363945007,
555
+ "learning_rate": 2.2111324376199616e-05,
556
+ "loss": 1.4355,
557
+ "step": 385
558
+ },
559
+ {
560
+ "epoch": 0.07499278915488895,
561
+ "grad_norm": 1.0963833332061768,
562
+ "learning_rate": 2.2399232245681383e-05,
563
+ "loss": 1.4208,
564
+ "step": 390
565
+ },
566
+ {
567
+ "epoch": 0.07595423516969523,
568
+ "grad_norm": 0.9907721281051636,
569
+ "learning_rate": 2.268714011516315e-05,
570
+ "loss": 1.3403,
571
+ "step": 395
572
+ },
573
+ {
574
+ "epoch": 0.0769156811845015,
575
+ "grad_norm": 0.9433814287185669,
576
+ "learning_rate": 2.2975047984644916e-05,
577
+ "loss": 1.3567,
578
+ "step": 400
579
+ },
580
+ {
581
+ "epoch": 0.07787712719930776,
582
+ "grad_norm": 0.9044313430786133,
583
+ "learning_rate": 2.326295585412668e-05,
584
+ "loss": 1.4668,
585
+ "step": 405
586
+ },
587
+ {
588
+ "epoch": 0.07883857321411403,
589
+ "grad_norm": 0.9079594612121582,
590
+ "learning_rate": 2.3550863723608447e-05,
591
+ "loss": 1.3796,
592
+ "step": 410
593
+ },
594
+ {
595
+ "epoch": 0.0798000192289203,
596
+ "grad_norm": 0.9176523685455322,
597
+ "learning_rate": 2.383877159309021e-05,
598
+ "loss": 1.41,
599
+ "step": 415
600
+ },
601
+ {
602
+ "epoch": 0.08076146524372657,
603
+ "grad_norm": 0.9850307106971741,
604
+ "learning_rate": 2.4126679462571977e-05,
605
+ "loss": 1.4052,
606
+ "step": 420
607
+ },
608
+ {
609
+ "epoch": 0.08172291125853283,
610
+ "grad_norm": 1.0259281396865845,
611
+ "learning_rate": 2.4414587332053744e-05,
612
+ "loss": 1.4044,
613
+ "step": 425
614
+ },
615
+ {
616
+ "epoch": 0.0826843572733391,
617
+ "grad_norm": 0.9912247061729431,
618
+ "learning_rate": 2.470249520153551e-05,
619
+ "loss": 1.3343,
620
+ "step": 430
621
+ },
622
+ {
623
+ "epoch": 0.08364580328814537,
624
+ "grad_norm": 0.9430367946624756,
625
+ "learning_rate": 2.4990403071017277e-05,
626
+ "loss": 1.4015,
627
+ "step": 435
628
+ },
629
+ {
630
+ "epoch": 0.08460724930295165,
631
+ "grad_norm": 0.9094961285591125,
632
+ "learning_rate": 2.527831094049904e-05,
633
+ "loss": 1.469,
634
+ "step": 440
635
+ },
636
+ {
637
+ "epoch": 0.0855686953177579,
638
+ "grad_norm": 0.8672477006912231,
639
+ "learning_rate": 2.5566218809980807e-05,
640
+ "loss": 1.3456,
641
+ "step": 445
642
+ },
643
+ {
644
+ "epoch": 0.08653014133256418,
645
+ "grad_norm": 0.9972670078277588,
646
+ "learning_rate": 2.5854126679462574e-05,
647
+ "loss": 1.2994,
648
+ "step": 450
649
+ },
650
+ {
651
+ "epoch": 0.08749158734737045,
652
+ "grad_norm": 0.9733589887619019,
653
+ "learning_rate": 2.6142034548944338e-05,
654
+ "loss": 1.3375,
655
+ "step": 455
656
+ },
657
+ {
658
+ "epoch": 0.08845303336217672,
659
+ "grad_norm": 0.9801984429359436,
660
+ "learning_rate": 2.6429942418426104e-05,
661
+ "loss": 1.3271,
662
+ "step": 460
663
+ },
664
+ {
665
+ "epoch": 0.08941447937698298,
666
+ "grad_norm": 0.8056566715240479,
667
+ "learning_rate": 2.6717850287907868e-05,
668
+ "loss": 1.3696,
669
+ "step": 465
670
+ },
671
+ {
672
+ "epoch": 0.09037592539178925,
673
+ "grad_norm": 0.9498573541641235,
674
+ "learning_rate": 2.7005758157389635e-05,
675
+ "loss": 1.3724,
676
+ "step": 470
677
+ },
678
+ {
679
+ "epoch": 0.09133737140659552,
680
+ "grad_norm": 0.9388139247894287,
681
+ "learning_rate": 2.7293666026871405e-05,
682
+ "loss": 1.3394,
683
+ "step": 475
684
+ },
685
+ {
686
+ "epoch": 0.0922988174214018,
687
+ "grad_norm": 0.9051626324653625,
688
+ "learning_rate": 2.758157389635317e-05,
689
+ "loss": 1.3465,
690
+ "step": 480
691
+ },
692
+ {
693
+ "epoch": 0.09326026343620805,
694
+ "grad_norm": 1.0678194761276245,
695
+ "learning_rate": 2.7869481765834935e-05,
696
+ "loss": 1.3369,
697
+ "step": 485
698
+ },
699
+ {
700
+ "epoch": 0.09422170945101432,
701
+ "grad_norm": 1.0179715156555176,
702
+ "learning_rate": 2.81573896353167e-05,
703
+ "loss": 1.2868,
704
+ "step": 490
705
+ },
706
+ {
707
+ "epoch": 0.0951831554658206,
708
+ "grad_norm": 1.025252103805542,
709
+ "learning_rate": 2.8445297504798465e-05,
710
+ "loss": 1.4128,
711
+ "step": 495
712
+ },
713
+ {
714
+ "epoch": 0.09614460148062687,
715
+ "grad_norm": 0.9994450211524963,
716
+ "learning_rate": 2.8733205374280232e-05,
717
+ "loss": 1.3105,
718
+ "step": 500
719
+ },
720
+ {
721
+ "epoch": 0.09710604749543313,
722
+ "grad_norm": 1.0843088626861572,
723
+ "learning_rate": 2.9021113243761996e-05,
724
+ "loss": 1.4405,
725
+ "step": 505
726
+ },
727
+ {
728
+ "epoch": 0.0980674935102394,
729
+ "grad_norm": 0.8648557662963867,
730
+ "learning_rate": 2.9309021113243762e-05,
731
+ "loss": 1.3738,
732
+ "step": 510
733
+ },
734
+ {
735
+ "epoch": 0.09902893952504567,
736
+ "grad_norm": 1.0407732725143433,
737
+ "learning_rate": 2.9596928982725526e-05,
738
+ "loss": 1.4331,
739
+ "step": 515
740
+ },
741
+ {
742
+ "epoch": 0.09999038553985194,
743
+ "grad_norm": 1.8220276832580566,
744
+ "learning_rate": 2.9884836852207296e-05,
745
+ "loss": 1.3347,
746
+ "step": 520
747
+ },
748
+ {
749
+ "epoch": 0.1009518315546582,
750
+ "grad_norm": 0.8917143940925598,
751
+ "learning_rate": 2.9990891610160915e-05,
752
+ "loss": 1.3576,
753
+ "step": 525
754
+ },
755
+ {
756
+ "epoch": 0.10191327756946447,
757
+ "grad_norm": 0.9336162209510803,
758
+ "learning_rate": 2.9975710960429106e-05,
759
+ "loss": 1.3353,
760
+ "step": 530
761
+ },
762
+ {
763
+ "epoch": 0.10287472358427074,
764
+ "grad_norm": 0.9028390049934387,
765
+ "learning_rate": 2.9960530310697297e-05,
766
+ "loss": 1.4185,
767
+ "step": 535
768
+ },
769
+ {
770
+ "epoch": 0.10383616959907702,
771
+ "grad_norm": 0.9123654365539551,
772
+ "learning_rate": 2.9945349660965488e-05,
773
+ "loss": 1.3747,
774
+ "step": 540
775
+ },
776
+ {
777
+ "epoch": 0.10479761561388327,
778
+ "grad_norm": 0.9282749891281128,
779
+ "learning_rate": 2.993016901123368e-05,
780
+ "loss": 1.3323,
781
+ "step": 545
782
+ },
783
+ {
784
+ "epoch": 0.10575906162868955,
785
+ "grad_norm": 1.06288743019104,
786
+ "learning_rate": 2.9914988361501872e-05,
787
+ "loss": 1.3638,
788
+ "step": 550
789
+ },
790
+ {
791
+ "epoch": 0.10672050764349582,
792
+ "grad_norm": 0.9988644123077393,
793
+ "learning_rate": 2.9899807711770063e-05,
794
+ "loss": 1.3527,
795
+ "step": 555
796
+ },
797
+ {
798
+ "epoch": 0.10768195365830209,
799
+ "grad_norm": 0.963537871837616,
800
+ "learning_rate": 2.9884627062038254e-05,
801
+ "loss": 1.3287,
802
+ "step": 560
803
+ },
804
+ {
805
+ "epoch": 0.10864339967310835,
806
+ "grad_norm": 1.1065447330474854,
807
+ "learning_rate": 2.9869446412306445e-05,
808
+ "loss": 1.3152,
809
+ "step": 565
810
+ },
811
+ {
812
+ "epoch": 0.10960484568791462,
813
+ "grad_norm": 0.9934567213058472,
814
+ "learning_rate": 2.985426576257464e-05,
815
+ "loss": 1.285,
816
+ "step": 570
817
+ },
818
+ {
819
+ "epoch": 0.11056629170272089,
820
+ "grad_norm": 0.9684432148933411,
821
+ "learning_rate": 2.983908511284283e-05,
822
+ "loss": 1.4081,
823
+ "step": 575
824
+ },
825
+ {
826
+ "epoch": 0.11152773771752716,
827
+ "grad_norm": 1.044866919517517,
828
+ "learning_rate": 2.982390446311102e-05,
829
+ "loss": 1.3684,
830
+ "step": 580
831
+ },
832
+ {
833
+ "epoch": 0.11248918373233342,
834
+ "grad_norm": 0.9235266447067261,
835
+ "learning_rate": 2.9808723813379215e-05,
836
+ "loss": 1.2453,
837
+ "step": 585
838
+ },
839
+ {
840
+ "epoch": 0.1134506297471397,
841
+ "grad_norm": 1.0257271528244019,
842
+ "learning_rate": 2.9793543163647405e-05,
843
+ "loss": 1.3664,
844
+ "step": 590
845
+ },
846
+ {
847
+ "epoch": 0.11441207576194597,
848
+ "grad_norm": 1.1091939210891724,
849
+ "learning_rate": 2.97783625139156e-05,
850
+ "loss": 1.2905,
851
+ "step": 595
852
+ },
853
+ {
854
+ "epoch": 0.11537352177675224,
855
+ "grad_norm": 0.9549099802970886,
856
+ "learning_rate": 2.976318186418379e-05,
857
+ "loss": 1.4162,
858
+ "step": 600
859
+ },
860
+ {
861
+ "epoch": 0.11537352177675224,
862
+ "eval_loss": 1.3265293836593628,
863
+ "eval_runtime": 1813.3194,
864
+ "eval_samples_per_second": 11.472,
865
+ "eval_steps_per_second": 2.868,
866
+ "step": 600
867
+ },
868
+ {
869
+ "epoch": 0.11633496779155851,
870
+ "grad_norm": 1.1611406803131104,
871
+ "learning_rate": 2.974800121445198e-05,
872
+ "loss": 1.2258,
873
+ "step": 605
874
+ },
875
+ {
876
+ "epoch": 0.11729641380636477,
877
+ "grad_norm": 1.0069003105163574,
878
+ "learning_rate": 2.9732820564720172e-05,
879
+ "loss": 1.2956,
880
+ "step": 610
881
+ },
882
+ {
883
+ "epoch": 0.11825785982117104,
884
+ "grad_norm": 0.8448907732963562,
885
+ "learning_rate": 2.9717639914988363e-05,
886
+ "loss": 1.35,
887
+ "step": 615
888
+ },
889
+ {
890
+ "epoch": 0.11921930583597731,
891
+ "grad_norm": 0.8714330792427063,
892
+ "learning_rate": 2.9702459265256557e-05,
893
+ "loss": 1.2409,
894
+ "step": 620
895
+ },
896
+ {
897
+ "epoch": 0.12018075185078358,
898
+ "grad_norm": 0.8822401762008667,
899
+ "learning_rate": 2.9687278615524747e-05,
900
+ "loss": 1.3148,
901
+ "step": 625
902
+ },
903
+ {
904
+ "epoch": 0.12114219786558984,
905
+ "grad_norm": 1.030093789100647,
906
+ "learning_rate": 2.9672097965792938e-05,
907
+ "loss": 1.3713,
908
+ "step": 630
909
+ },
910
+ {
911
+ "epoch": 0.12210364388039611,
912
+ "grad_norm": 1.0077892541885376,
913
+ "learning_rate": 2.965691731606113e-05,
914
+ "loss": 1.354,
915
+ "step": 635
916
+ },
917
+ {
918
+ "epoch": 0.12306508989520239,
919
+ "grad_norm": 1.0031296014785767,
920
+ "learning_rate": 2.964173666632932e-05,
921
+ "loss": 1.3483,
922
+ "step": 640
923
+ },
924
+ {
925
+ "epoch": 0.12402653591000866,
926
+ "grad_norm": 0.9184396266937256,
927
+ "learning_rate": 2.9626556016597514e-05,
928
+ "loss": 1.2783,
929
+ "step": 645
930
+ },
931
+ {
932
+ "epoch": 0.12498798192481492,
933
+ "grad_norm": 0.9425734281539917,
934
+ "learning_rate": 2.9611375366865705e-05,
935
+ "loss": 1.2896,
936
+ "step": 650
937
+ },
938
+ {
939
+ "epoch": 0.1259494279396212,
940
+ "grad_norm": 0.8437754511833191,
941
+ "learning_rate": 2.9596194717133895e-05,
942
+ "loss": 1.3343,
943
+ "step": 655
944
+ },
945
+ {
946
+ "epoch": 0.12691087395442746,
947
+ "grad_norm": 0.9617599844932556,
948
+ "learning_rate": 2.9581014067402086e-05,
949
+ "loss": 1.2472,
950
+ "step": 660
951
+ },
952
+ {
953
+ "epoch": 0.12787231996923373,
954
+ "grad_norm": 0.9239601492881775,
955
+ "learning_rate": 2.9565833417670277e-05,
956
+ "loss": 1.3209,
957
+ "step": 665
958
+ },
959
+ {
960
+ "epoch": 0.12883376598404,
961
+ "grad_norm": 0.9953628182411194,
962
+ "learning_rate": 2.955065276793847e-05,
963
+ "loss": 1.3152,
964
+ "step": 670
965
+ },
966
+ {
967
+ "epoch": 0.12979521199884628,
968
+ "grad_norm": 0.8559119701385498,
969
+ "learning_rate": 2.9535472118206662e-05,
970
+ "loss": 1.3301,
971
+ "step": 675
972
+ },
973
+ {
974
+ "epoch": 0.13075665801365252,
975
+ "grad_norm": 0.984180212020874,
976
+ "learning_rate": 2.9520291468474853e-05,
977
+ "loss": 1.3678,
978
+ "step": 680
979
+ },
980
+ {
981
+ "epoch": 0.1317181040284588,
982
+ "grad_norm": 0.9163925647735596,
983
+ "learning_rate": 2.9505110818743043e-05,
984
+ "loss": 1.3398,
985
+ "step": 685
986
+ },
987
+ {
988
+ "epoch": 0.13267955004326507,
989
+ "grad_norm": 1.1147078275680542,
990
+ "learning_rate": 2.9489930169011234e-05,
991
+ "loss": 1.2758,
992
+ "step": 690
993
+ },
994
+ {
995
+ "epoch": 0.13364099605807134,
996
+ "grad_norm": 0.980420708656311,
997
+ "learning_rate": 2.947474951927943e-05,
998
+ "loss": 1.3137,
999
+ "step": 695
1000
+ },
1001
+ {
1002
+ "epoch": 0.1346024420728776,
1003
+ "grad_norm": 1.0157663822174072,
1004
+ "learning_rate": 2.945956886954762e-05,
1005
+ "loss": 1.2144,
1006
+ "step": 700
1007
+ },
1008
+ {
1009
+ "epoch": 0.13556388808768388,
1010
+ "grad_norm": 0.8547846078872681,
1011
+ "learning_rate": 2.944438821981581e-05,
1012
+ "loss": 1.2879,
1013
+ "step": 705
1014
+ },
1015
+ {
1016
+ "epoch": 0.13652533410249015,
1017
+ "grad_norm": 1.018324851989746,
1018
+ "learning_rate": 2.9429207570084e-05,
1019
+ "loss": 1.267,
1020
+ "step": 710
1021
+ },
1022
+ {
1023
+ "epoch": 0.13748678011729643,
1024
+ "grad_norm": 0.832151472568512,
1025
+ "learning_rate": 2.941402692035219e-05,
1026
+ "loss": 1.3185,
1027
+ "step": 715
1028
+ },
1029
+ {
1030
+ "epoch": 0.13844822613210267,
1031
+ "grad_norm": 1.0583784580230713,
1032
+ "learning_rate": 2.9398846270620386e-05,
1033
+ "loss": 1.2489,
1034
+ "step": 720
1035
+ },
1036
+ {
1037
+ "epoch": 0.13940967214690894,
1038
+ "grad_norm": 1.0460695028305054,
1039
+ "learning_rate": 2.9383665620888576e-05,
1040
+ "loss": 1.3299,
1041
+ "step": 725
1042
+ },
1043
+ {
1044
+ "epoch": 0.1403711181617152,
1045
+ "grad_norm": 0.8800564408302307,
1046
+ "learning_rate": 2.9368484971156767e-05,
1047
+ "loss": 1.3494,
1048
+ "step": 730
1049
+ },
1050
+ {
1051
+ "epoch": 0.14133256417652149,
1052
+ "grad_norm": 0.9371108412742615,
1053
+ "learning_rate": 2.9353304321424958e-05,
1054
+ "loss": 1.3129,
1055
+ "step": 735
1056
+ },
1057
+ {
1058
+ "epoch": 0.14229401019132776,
1059
+ "grad_norm": 0.9252986907958984,
1060
+ "learning_rate": 2.933812367169315e-05,
1061
+ "loss": 1.3028,
1062
+ "step": 740
1063
+ },
1064
+ {
1065
+ "epoch": 0.14325545620613403,
1066
+ "grad_norm": 0.9625592827796936,
1067
+ "learning_rate": 2.9322943021961343e-05,
1068
+ "loss": 1.366,
1069
+ "step": 745
1070
+ },
1071
+ {
1072
+ "epoch": 0.1442169022209403,
1073
+ "grad_norm": 0.9188767671585083,
1074
+ "learning_rate": 2.9307762372229534e-05,
1075
+ "loss": 1.3584,
1076
+ "step": 750
1077
+ },
1078
+ {
1079
+ "epoch": 0.14517834823574657,
1080
+ "grad_norm": 0.8308439254760742,
1081
+ "learning_rate": 2.9292581722497724e-05,
1082
+ "loss": 1.2567,
1083
+ "step": 755
1084
+ },
1085
+ {
1086
+ "epoch": 0.14613979425055282,
1087
+ "grad_norm": 0.9764241576194763,
1088
+ "learning_rate": 2.9277401072765915e-05,
1089
+ "loss": 1.299,
1090
+ "step": 760
1091
+ },
1092
+ {
1093
+ "epoch": 0.1471012402653591,
1094
+ "grad_norm": 0.854916512966156,
1095
+ "learning_rate": 2.9262220423034106e-05,
1096
+ "loss": 1.2568,
1097
+ "step": 765
1098
+ },
1099
+ {
1100
+ "epoch": 0.14806268628016536,
1101
+ "grad_norm": 0.9988142251968384,
1102
+ "learning_rate": 2.92470397733023e-05,
1103
+ "loss": 1.2633,
1104
+ "step": 770
1105
+ },
1106
+ {
1107
+ "epoch": 0.14902413229497163,
1108
+ "grad_norm": 1.032973051071167,
1109
+ "learning_rate": 2.923185912357049e-05,
1110
+ "loss": 1.3592,
1111
+ "step": 775
1112
+ },
1113
+ {
1114
+ "epoch": 0.1499855783097779,
1115
+ "grad_norm": 1.057490587234497,
1116
+ "learning_rate": 2.921667847383868e-05,
1117
+ "loss": 1.3316,
1118
+ "step": 780
1119
+ },
1120
+ {
1121
+ "epoch": 0.15094702432458418,
1122
+ "grad_norm": 0.9987697005271912,
1123
+ "learning_rate": 2.9201497824106872e-05,
1124
+ "loss": 1.2957,
1125
+ "step": 785
1126
+ },
1127
+ {
1128
+ "epoch": 0.15190847033939045,
1129
+ "grad_norm": 1.0518527030944824,
1130
+ "learning_rate": 2.9186317174375063e-05,
1131
+ "loss": 1.3373,
1132
+ "step": 790
1133
+ },
1134
+ {
1135
+ "epoch": 0.15286991635419672,
1136
+ "grad_norm": 0.9941504001617432,
1137
+ "learning_rate": 2.9171136524643257e-05,
1138
+ "loss": 1.3207,
1139
+ "step": 795
1140
+ },
1141
+ {
1142
+ "epoch": 0.153831362369003,
1143
+ "grad_norm": 0.9565390348434448,
1144
+ "learning_rate": 2.9155955874911448e-05,
1145
+ "loss": 1.3035,
1146
+ "step": 800
1147
+ },
1148
+ {
1149
+ "epoch": 0.15479280838380924,
1150
+ "grad_norm": 0.9072451591491699,
1151
+ "learning_rate": 2.914077522517964e-05,
1152
+ "loss": 1.3077,
1153
+ "step": 805
1154
+ },
1155
+ {
1156
+ "epoch": 0.1557542543986155,
1157
+ "grad_norm": 0.8639485239982605,
1158
+ "learning_rate": 2.912559457544783e-05,
1159
+ "loss": 1.2388,
1160
+ "step": 810
1161
+ },
1162
+ {
1163
+ "epoch": 0.15671570041342178,
1164
+ "grad_norm": 0.8549638390541077,
1165
+ "learning_rate": 2.911041392571602e-05,
1166
+ "loss": 1.2112,
1167
+ "step": 815
1168
+ },
1169
+ {
1170
+ "epoch": 0.15767714642822805,
1171
+ "grad_norm": 0.9586772322654724,
1172
+ "learning_rate": 2.9095233275984214e-05,
1173
+ "loss": 1.3981,
1174
+ "step": 820
1175
+ },
1176
+ {
1177
+ "epoch": 0.15863859244303433,
1178
+ "grad_norm": 1.0374456644058228,
1179
+ "learning_rate": 2.9080052626252405e-05,
1180
+ "loss": 1.2159,
1181
+ "step": 825
1182
+ },
1183
+ {
1184
+ "epoch": 0.1596000384578406,
1185
+ "grad_norm": 0.9278589487075806,
1186
+ "learning_rate": 2.9064871976520596e-05,
1187
+ "loss": 1.2849,
1188
+ "step": 830
1189
+ },
1190
+ {
1191
+ "epoch": 0.16056148447264687,
1192
+ "grad_norm": 0.8326074481010437,
1193
+ "learning_rate": 2.9049691326788787e-05,
1194
+ "loss": 1.3178,
1195
+ "step": 835
1196
+ },
1197
+ {
1198
+ "epoch": 0.16152293048745314,
1199
+ "grad_norm": 1.0694983005523682,
1200
+ "learning_rate": 2.9034510677056977e-05,
1201
+ "loss": 1.2796,
1202
+ "step": 840
1203
+ },
1204
+ {
1205
+ "epoch": 0.1624843765022594,
1206
+ "grad_norm": 1.1848194599151611,
1207
+ "learning_rate": 2.901933002732517e-05,
1208
+ "loss": 1.3145,
1209
+ "step": 845
1210
+ },
1211
+ {
1212
+ "epoch": 0.16344582251706566,
1213
+ "grad_norm": 1.1083351373672485,
1214
+ "learning_rate": 2.9004149377593362e-05,
1215
+ "loss": 1.3274,
1216
+ "step": 850
1217
+ },
1218
+ {
1219
+ "epoch": 0.16440726853187193,
1220
+ "grad_norm": 0.8984082937240601,
1221
+ "learning_rate": 2.8988968727861553e-05,
1222
+ "loss": 1.3023,
1223
+ "step": 855
1224
+ },
1225
+ {
1226
+ "epoch": 0.1653687145466782,
1227
+ "grad_norm": 1.0397671461105347,
1228
+ "learning_rate": 2.8973788078129744e-05,
1229
+ "loss": 1.2713,
1230
+ "step": 860
1231
+ },
1232
+ {
1233
+ "epoch": 0.16633016056148447,
1234
+ "grad_norm": 0.9931337237358093,
1235
+ "learning_rate": 2.8958607428397935e-05,
1236
+ "loss": 1.26,
1237
+ "step": 865
1238
+ },
1239
+ {
1240
+ "epoch": 0.16729160657629075,
1241
+ "grad_norm": 1.0527026653289795,
1242
+ "learning_rate": 2.894342677866613e-05,
1243
+ "loss": 1.2944,
1244
+ "step": 870
1245
+ },
1246
+ {
1247
+ "epoch": 0.16825305259109702,
1248
+ "grad_norm": 1.153441309928894,
1249
+ "learning_rate": 2.892824612893432e-05,
1250
+ "loss": 1.2475,
1251
+ "step": 875
1252
+ },
1253
+ {
1254
+ "epoch": 0.1692144986059033,
1255
+ "grad_norm": 0.8572908639907837,
1256
+ "learning_rate": 2.891306547920251e-05,
1257
+ "loss": 1.242,
1258
+ "step": 880
1259
+ },
1260
+ {
1261
+ "epoch": 0.17017594462070953,
1262
+ "grad_norm": 1.0271085500717163,
1263
+ "learning_rate": 2.88978848294707e-05,
1264
+ "loss": 1.3052,
1265
+ "step": 885
1266
+ },
1267
+ {
1268
+ "epoch": 0.1711373906355158,
1269
+ "grad_norm": 1.0767736434936523,
1270
+ "learning_rate": 2.8882704179738892e-05,
1271
+ "loss": 1.2851,
1272
+ "step": 890
1273
+ },
1274
+ {
1275
+ "epoch": 0.17209883665032208,
1276
+ "grad_norm": 0.9310041069984436,
1277
+ "learning_rate": 2.8867523530007086e-05,
1278
+ "loss": 1.367,
1279
+ "step": 895
1280
+ },
1281
+ {
1282
+ "epoch": 0.17306028266512835,
1283
+ "grad_norm": 0.8906955122947693,
1284
+ "learning_rate": 2.8852342880275277e-05,
1285
+ "loss": 1.2853,
1286
+ "step": 900
1287
+ },
1288
+ {
1289
+ "epoch": 0.17306028266512835,
1290
+ "eval_loss": 1.2756867408752441,
1291
+ "eval_runtime": 1880.9614,
1292
+ "eval_samples_per_second": 11.059,
1293
+ "eval_steps_per_second": 2.765,
1294
+ "step": 900
1295
+ },
1296
+ {
1297
+ "epoch": 0.17402172867993462,
1298
+ "grad_norm": 0.8874945640563965,
1299
+ "learning_rate": 2.8837162230543468e-05,
1300
+ "loss": 1.252,
1301
+ "step": 905
1302
+ },
1303
+ {
1304
+ "epoch": 0.1749831746947409,
1305
+ "grad_norm": 0.992651641368866,
1306
+ "learning_rate": 2.882198158081166e-05,
1307
+ "loss": 1.3451,
1308
+ "step": 910
1309
+ },
1310
+ {
1311
+ "epoch": 0.17594462070954717,
1312
+ "grad_norm": 1.1066889762878418,
1313
+ "learning_rate": 2.8806800931079852e-05,
1314
+ "loss": 1.2555,
1315
+ "step": 915
1316
+ },
1317
+ {
1318
+ "epoch": 0.17690606672435344,
1319
+ "grad_norm": 1.0017170906066895,
1320
+ "learning_rate": 2.8791620281348043e-05,
1321
+ "loss": 1.3645,
1322
+ "step": 920
1323
+ },
1324
+ {
1325
+ "epoch": 0.17786751273915968,
1326
+ "grad_norm": 1.146898627281189,
1327
+ "learning_rate": 2.8776439631616234e-05,
1328
+ "loss": 1.3557,
1329
+ "step": 925
1330
+ },
1331
+ {
1332
+ "epoch": 0.17882895875396596,
1333
+ "grad_norm": 0.9736440777778625,
1334
+ "learning_rate": 2.8761258981884425e-05,
1335
+ "loss": 1.2755,
1336
+ "step": 930
1337
+ },
1338
+ {
1339
+ "epoch": 0.17979040476877223,
1340
+ "grad_norm": 0.9591579437255859,
1341
+ "learning_rate": 2.8746078332152616e-05,
1342
+ "loss": 1.2618,
1343
+ "step": 935
1344
+ },
1345
+ {
1346
+ "epoch": 0.1807518507835785,
1347
+ "grad_norm": 1.3127062320709229,
1348
+ "learning_rate": 2.873089768242081e-05,
1349
+ "loss": 1.2494,
1350
+ "step": 940
1351
+ },
1352
+ {
1353
+ "epoch": 0.18171329679838477,
1354
+ "grad_norm": 1.2093756198883057,
1355
+ "learning_rate": 2.8715717032689e-05,
1356
+ "loss": 1.3055,
1357
+ "step": 945
1358
+ },
1359
+ {
1360
+ "epoch": 0.18267474281319104,
1361
+ "grad_norm": 1.2279831171035767,
1362
+ "learning_rate": 2.870053638295719e-05,
1363
+ "loss": 1.2797,
1364
+ "step": 950
1365
+ },
1366
+ {
1367
+ "epoch": 0.18363618882799732,
1368
+ "grad_norm": 0.9829315543174744,
1369
+ "learning_rate": 2.8685355733225382e-05,
1370
+ "loss": 1.252,
1371
+ "step": 955
1372
+ },
1373
+ {
1374
+ "epoch": 0.1845976348428036,
1375
+ "grad_norm": 1.0075795650482178,
1376
+ "learning_rate": 2.8670175083493573e-05,
1377
+ "loss": 1.2432,
1378
+ "step": 960
1379
+ },
1380
+ {
1381
+ "epoch": 0.18555908085760983,
1382
+ "grad_norm": 0.9815109968185425,
1383
+ "learning_rate": 2.8654994433761767e-05,
1384
+ "loss": 1.3011,
1385
+ "step": 965
1386
+ },
1387
+ {
1388
+ "epoch": 0.1865205268724161,
1389
+ "grad_norm": 0.9473528265953064,
1390
+ "learning_rate": 2.8639813784029958e-05,
1391
+ "loss": 1.328,
1392
+ "step": 970
1393
+ },
1394
+ {
1395
+ "epoch": 0.18748197288722238,
1396
+ "grad_norm": 1.0563870668411255,
1397
+ "learning_rate": 2.862463313429815e-05,
1398
+ "loss": 1.4197,
1399
+ "step": 975
1400
+ },
1401
+ {
1402
+ "epoch": 0.18844341890202865,
1403
+ "grad_norm": 0.999548614025116,
1404
+ "learning_rate": 2.860945248456634e-05,
1405
+ "loss": 1.2636,
1406
+ "step": 980
1407
+ },
1408
+ {
1409
+ "epoch": 0.18940486491683492,
1410
+ "grad_norm": 1.0543882846832275,
1411
+ "learning_rate": 2.859427183483453e-05,
1412
+ "loss": 1.2293,
1413
+ "step": 985
1414
+ },
1415
+ {
1416
+ "epoch": 0.1903663109316412,
1417
+ "grad_norm": 1.0018796920776367,
1418
+ "learning_rate": 2.8579091185102724e-05,
1419
+ "loss": 1.262,
1420
+ "step": 990
1421
+ },
1422
+ {
1423
+ "epoch": 0.19132775694644746,
1424
+ "grad_norm": 1.0547089576721191,
1425
+ "learning_rate": 2.8563910535370915e-05,
1426
+ "loss": 1.2329,
1427
+ "step": 995
1428
+ },
1429
+ {
1430
+ "epoch": 0.19228920296125374,
1431
+ "grad_norm": 0.9648796916007996,
1432
+ "learning_rate": 2.8548729885639106e-05,
1433
+ "loss": 1.327,
1434
+ "step": 1000
1435
+ },
1436
+ {
1437
+ "epoch": 0.19325064897606,
1438
+ "grad_norm": 0.964148998260498,
1439
+ "learning_rate": 2.8533549235907296e-05,
1440
+ "loss": 1.2733,
1441
+ "step": 1005
1442
+ },
1443
+ {
1444
+ "epoch": 0.19421209499086625,
1445
+ "grad_norm": 1.0552051067352295,
1446
+ "learning_rate": 2.8518368586175487e-05,
1447
+ "loss": 1.2462,
1448
+ "step": 1010
1449
+ },
1450
+ {
1451
+ "epoch": 0.19517354100567252,
1452
+ "grad_norm": 1.1029739379882812,
1453
+ "learning_rate": 2.850318793644368e-05,
1454
+ "loss": 1.2478,
1455
+ "step": 1015
1456
+ },
1457
+ {
1458
+ "epoch": 0.1961349870204788,
1459
+ "grad_norm": 1.1883385181427002,
1460
+ "learning_rate": 2.8488007286711872e-05,
1461
+ "loss": 1.1937,
1462
+ "step": 1020
1463
+ },
1464
+ {
1465
+ "epoch": 0.19709643303528507,
1466
+ "grad_norm": 0.9168397784233093,
1467
+ "learning_rate": 2.8472826636980063e-05,
1468
+ "loss": 1.3682,
1469
+ "step": 1025
1470
+ },
1471
+ {
1472
+ "epoch": 0.19805787905009134,
1473
+ "grad_norm": 0.9574885964393616,
1474
+ "learning_rate": 2.8457645987248254e-05,
1475
+ "loss": 1.2358,
1476
+ "step": 1030
1477
+ },
1478
+ {
1479
+ "epoch": 0.1990193250648976,
1480
+ "grad_norm": 1.0165746212005615,
1481
+ "learning_rate": 2.8442465337516444e-05,
1482
+ "loss": 1.2765,
1483
+ "step": 1035
1484
+ },
1485
+ {
1486
+ "epoch": 0.19998077107970388,
1487
+ "grad_norm": 0.8473765254020691,
1488
+ "learning_rate": 2.842728468778464e-05,
1489
+ "loss": 1.1938,
1490
+ "step": 1040
1491
+ },
1492
+ {
1493
+ "epoch": 0.20094221709451016,
1494
+ "grad_norm": 1.000200629234314,
1495
+ "learning_rate": 2.841210403805283e-05,
1496
+ "loss": 1.2242,
1497
+ "step": 1045
1498
+ },
1499
+ {
1500
+ "epoch": 0.2019036631093164,
1501
+ "grad_norm": 1.1640363931655884,
1502
+ "learning_rate": 2.839692338832102e-05,
1503
+ "loss": 1.3152,
1504
+ "step": 1050
1505
+ },
1506
+ {
1507
+ "epoch": 0.20286510912412267,
1508
+ "grad_norm": 1.0977058410644531,
1509
+ "learning_rate": 2.838174273858921e-05,
1510
+ "loss": 1.2639,
1511
+ "step": 1055
1512
+ },
1513
+ {
1514
+ "epoch": 0.20382655513892894,
1515
+ "grad_norm": 1.0105798244476318,
1516
+ "learning_rate": 2.83665620888574e-05,
1517
+ "loss": 1.2381,
1518
+ "step": 1060
1519
+ },
1520
+ {
1521
+ "epoch": 0.20478800115373522,
1522
+ "grad_norm": 0.9707632064819336,
1523
+ "learning_rate": 2.8351381439125596e-05,
1524
+ "loss": 1.2691,
1525
+ "step": 1065
1526
+ },
1527
+ {
1528
+ "epoch": 0.2057494471685415,
1529
+ "grad_norm": 0.8312526345252991,
1530
+ "learning_rate": 2.8336200789393787e-05,
1531
+ "loss": 1.2571,
1532
+ "step": 1070
1533
+ },
1534
+ {
1535
+ "epoch": 0.20671089318334776,
1536
+ "grad_norm": 0.8867915868759155,
1537
+ "learning_rate": 2.8321020139661977e-05,
1538
+ "loss": 1.3126,
1539
+ "step": 1075
1540
+ },
1541
+ {
1542
+ "epoch": 0.20767233919815403,
1543
+ "grad_norm": 0.9358422160148621,
1544
+ "learning_rate": 2.8305839489930168e-05,
1545
+ "loss": 1.3131,
1546
+ "step": 1080
1547
+ },
1548
+ {
1549
+ "epoch": 0.2086337852129603,
1550
+ "grad_norm": 1.0027930736541748,
1551
+ "learning_rate": 2.829065884019836e-05,
1552
+ "loss": 1.3345,
1553
+ "step": 1085
1554
+ },
1555
+ {
1556
+ "epoch": 0.20959523122776655,
1557
+ "grad_norm": 0.9601215720176697,
1558
+ "learning_rate": 2.8275478190466553e-05,
1559
+ "loss": 1.2801,
1560
+ "step": 1090
1561
+ },
1562
+ {
1563
+ "epoch": 0.21055667724257282,
1564
+ "grad_norm": 0.8792690634727478,
1565
+ "learning_rate": 2.8260297540734744e-05,
1566
+ "loss": 1.2366,
1567
+ "step": 1095
1568
+ },
1569
+ {
1570
+ "epoch": 0.2115181232573791,
1571
+ "grad_norm": 1.0058884620666504,
1572
+ "learning_rate": 2.8245116891002935e-05,
1573
+ "loss": 1.192,
1574
+ "step": 1100
1575
+ },
1576
+ {
1577
+ "epoch": 0.21247956927218536,
1578
+ "grad_norm": 1.0079177618026733,
1579
+ "learning_rate": 2.8229936241271125e-05,
1580
+ "loss": 1.2698,
1581
+ "step": 1105
1582
+ },
1583
+ {
1584
+ "epoch": 0.21344101528699164,
1585
+ "grad_norm": 1.0556343793869019,
1586
+ "learning_rate": 2.8214755591539316e-05,
1587
+ "loss": 1.2477,
1588
+ "step": 1110
1589
+ },
1590
+ {
1591
+ "epoch": 0.2144024613017979,
1592
+ "grad_norm": 0.9617932438850403,
1593
+ "learning_rate": 2.819957494180751e-05,
1594
+ "loss": 1.2387,
1595
+ "step": 1115
1596
+ },
1597
+ {
1598
+ "epoch": 0.21536390731660418,
1599
+ "grad_norm": 0.9451227188110352,
1600
+ "learning_rate": 2.81843942920757e-05,
1601
+ "loss": 1.276,
1602
+ "step": 1120
1603
+ },
1604
+ {
1605
+ "epoch": 0.21632535333141045,
1606
+ "grad_norm": 0.9511146545410156,
1607
+ "learning_rate": 2.8169213642343892e-05,
1608
+ "loss": 1.2211,
1609
+ "step": 1125
1610
+ },
1611
+ {
1612
+ "epoch": 0.2172867993462167,
1613
+ "grad_norm": 0.9459360241889954,
1614
+ "learning_rate": 2.8154032992612082e-05,
1615
+ "loss": 1.1895,
1616
+ "step": 1130
1617
+ },
1618
+ {
1619
+ "epoch": 0.21824824536102297,
1620
+ "grad_norm": 0.9601720571517944,
1621
+ "learning_rate": 2.8138852342880273e-05,
1622
+ "loss": 1.2573,
1623
+ "step": 1135
1624
+ },
1625
+ {
1626
+ "epoch": 0.21920969137582924,
1627
+ "grad_norm": 0.9400343894958496,
1628
+ "learning_rate": 2.8123671693148467e-05,
1629
+ "loss": 1.2944,
1630
+ "step": 1140
1631
+ },
1632
+ {
1633
+ "epoch": 0.2201711373906355,
1634
+ "grad_norm": 1.0376919507980347,
1635
+ "learning_rate": 2.8108491043416658e-05,
1636
+ "loss": 1.2634,
1637
+ "step": 1145
1638
+ },
1639
+ {
1640
+ "epoch": 0.22113258340544178,
1641
+ "grad_norm": 0.9042131304740906,
1642
+ "learning_rate": 2.809331039368485e-05,
1643
+ "loss": 1.2978,
1644
+ "step": 1150
1645
+ },
1646
+ {
1647
+ "epoch": 0.22209402942024806,
1648
+ "grad_norm": 0.8955277800559998,
1649
+ "learning_rate": 2.807812974395304e-05,
1650
+ "loss": 1.314,
1651
+ "step": 1155
1652
+ },
1653
+ {
1654
+ "epoch": 0.22305547543505433,
1655
+ "grad_norm": 0.9650217890739441,
1656
+ "learning_rate": 2.806294909422123e-05,
1657
+ "loss": 1.3183,
1658
+ "step": 1160
1659
+ },
1660
+ {
1661
+ "epoch": 0.2240169214498606,
1662
+ "grad_norm": 0.9972891211509705,
1663
+ "learning_rate": 2.8047768444489425e-05,
1664
+ "loss": 1.2108,
1665
+ "step": 1165
1666
+ },
1667
+ {
1668
+ "epoch": 0.22497836746466685,
1669
+ "grad_norm": 0.841851532459259,
1670
+ "learning_rate": 2.8032587794757615e-05,
1671
+ "loss": 1.2214,
1672
+ "step": 1170
1673
+ },
1674
+ {
1675
+ "epoch": 0.22593981347947312,
1676
+ "grad_norm": 1.0023276805877686,
1677
+ "learning_rate": 2.8017407145025806e-05,
1678
+ "loss": 1.3031,
1679
+ "step": 1175
1680
+ },
1681
+ {
1682
+ "epoch": 0.2269012594942794,
1683
+ "grad_norm": 0.9950117468833923,
1684
+ "learning_rate": 2.8002226495293997e-05,
1685
+ "loss": 1.2046,
1686
+ "step": 1180
1687
+ },
1688
+ {
1689
+ "epoch": 0.22786270550908566,
1690
+ "grad_norm": 0.9552078247070312,
1691
+ "learning_rate": 2.7987045845562188e-05,
1692
+ "loss": 1.2556,
1693
+ "step": 1185
1694
+ },
1695
+ {
1696
+ "epoch": 0.22882415152389193,
1697
+ "grad_norm": 0.9237502813339233,
1698
+ "learning_rate": 2.7971865195830382e-05,
1699
+ "loss": 1.3269,
1700
+ "step": 1190
1701
+ },
1702
+ {
1703
+ "epoch": 0.2297855975386982,
1704
+ "grad_norm": 0.8797445893287659,
1705
+ "learning_rate": 2.7956684546098576e-05,
1706
+ "loss": 1.2626,
1707
+ "step": 1195
1708
+ },
1709
+ {
1710
+ "epoch": 0.23074704355350448,
1711
+ "grad_norm": 1.0588504076004028,
1712
+ "learning_rate": 2.7941503896366767e-05,
1713
+ "loss": 1.2855,
1714
+ "step": 1200
1715
+ },
1716
+ {
1717
+ "epoch": 0.23074704355350448,
1718
+ "eval_loss": 1.2411783933639526,
1719
+ "eval_runtime": 1809.2447,
1720
+ "eval_samples_per_second": 11.498,
1721
+ "eval_steps_per_second": 2.875,
1722
+ "step": 1200
1723
+ },
1724
+ {
1725
+ "epoch": 0.23170848956831075,
1726
+ "grad_norm": 0.9612005352973938,
1727
+ "learning_rate": 2.7926323246634958e-05,
1728
+ "loss": 1.2355,
1729
+ "step": 1205
1730
+ },
1731
+ {
1732
+ "epoch": 0.23266993558311702,
1733
+ "grad_norm": 1.047343373298645,
1734
+ "learning_rate": 2.7911142596903148e-05,
1735
+ "loss": 1.3218,
1736
+ "step": 1210
1737
+ },
1738
+ {
1739
+ "epoch": 0.23363138159792327,
1740
+ "grad_norm": 1.1566919088363647,
1741
+ "learning_rate": 2.7895961947171342e-05,
1742
+ "loss": 1.235,
1743
+ "step": 1215
1744
+ },
1745
+ {
1746
+ "epoch": 0.23459282761272954,
1747
+ "grad_norm": 0.9153907895088196,
1748
+ "learning_rate": 2.7880781297439533e-05,
1749
+ "loss": 1.2727,
1750
+ "step": 1220
1751
+ },
1752
+ {
1753
+ "epoch": 0.2355542736275358,
1754
+ "grad_norm": 0.9632519483566284,
1755
+ "learning_rate": 2.7865600647707724e-05,
1756
+ "loss": 1.258,
1757
+ "step": 1225
1758
+ },
1759
+ {
1760
+ "epoch": 0.23651571964234208,
1761
+ "grad_norm": 0.9103459119796753,
1762
+ "learning_rate": 2.7850419997975915e-05,
1763
+ "loss": 1.2255,
1764
+ "step": 1230
1765
+ },
1766
+ {
1767
+ "epoch": 0.23747716565714835,
1768
+ "grad_norm": 0.9904808402061462,
1769
+ "learning_rate": 2.7835239348244105e-05,
1770
+ "loss": 1.1586,
1771
+ "step": 1235
1772
+ },
1773
+ {
1774
+ "epoch": 0.23843861167195463,
1775
+ "grad_norm": 0.9824590086936951,
1776
+ "learning_rate": 2.78200586985123e-05,
1777
+ "loss": 1.343,
1778
+ "step": 1240
1779
+ },
1780
+ {
1781
+ "epoch": 0.2394000576867609,
1782
+ "grad_norm": 1.057007074356079,
1783
+ "learning_rate": 2.780487804878049e-05,
1784
+ "loss": 1.2222,
1785
+ "step": 1245
1786
+ },
1787
+ {
1788
+ "epoch": 0.24036150370156717,
1789
+ "grad_norm": 1.0756044387817383,
1790
+ "learning_rate": 2.778969739904868e-05,
1791
+ "loss": 1.2955,
1792
+ "step": 1250
1793
+ },
1794
+ {
1795
+ "epoch": 0.2413229497163734,
1796
+ "grad_norm": 0.9179213643074036,
1797
+ "learning_rate": 2.7774516749316872e-05,
1798
+ "loss": 1.3747,
1799
+ "step": 1255
1800
+ },
1801
+ {
1802
+ "epoch": 0.24228439573117969,
1803
+ "grad_norm": 0.8226995468139648,
1804
+ "learning_rate": 2.7759336099585063e-05,
1805
+ "loss": 1.1311,
1806
+ "step": 1260
1807
+ },
1808
+ {
1809
+ "epoch": 0.24324584174598596,
1810
+ "grad_norm": 0.933071494102478,
1811
+ "learning_rate": 2.7744155449853257e-05,
1812
+ "loss": 1.2981,
1813
+ "step": 1265
1814
+ },
1815
+ {
1816
+ "epoch": 0.24420728776079223,
1817
+ "grad_norm": 1.0771886110305786,
1818
+ "learning_rate": 2.7728974800121448e-05,
1819
+ "loss": 1.2566,
1820
+ "step": 1270
1821
+ },
1822
+ {
1823
+ "epoch": 0.2451687337755985,
1824
+ "grad_norm": 1.0421266555786133,
1825
+ "learning_rate": 2.771379415038964e-05,
1826
+ "loss": 1.2516,
1827
+ "step": 1275
1828
+ },
1829
+ {
1830
+ "epoch": 0.24613017979040477,
1831
+ "grad_norm": 1.0798468589782715,
1832
+ "learning_rate": 2.769861350065783e-05,
1833
+ "loss": 1.2286,
1834
+ "step": 1280
1835
+ },
1836
+ {
1837
+ "epoch": 0.24709162580521105,
1838
+ "grad_norm": 1.0607397556304932,
1839
+ "learning_rate": 2.768343285092602e-05,
1840
+ "loss": 1.23,
1841
+ "step": 1285
1842
+ },
1843
+ {
1844
+ "epoch": 0.24805307182001732,
1845
+ "grad_norm": 1.0066001415252686,
1846
+ "learning_rate": 2.7668252201194214e-05,
1847
+ "loss": 1.1863,
1848
+ "step": 1290
1849
+ },
1850
+ {
1851
+ "epoch": 0.24901451783482356,
1852
+ "grad_norm": 0.9080028533935547,
1853
+ "learning_rate": 2.7653071551462405e-05,
1854
+ "loss": 1.2056,
1855
+ "step": 1295
1856
+ },
1857
+ {
1858
+ "epoch": 0.24997596384962983,
1859
+ "grad_norm": 1.03952157497406,
1860
+ "learning_rate": 2.7637890901730596e-05,
1861
+ "loss": 1.2561,
1862
+ "step": 1300
1863
+ },
1864
+ {
1865
+ "epoch": 0.25093740986443613,
1866
+ "grad_norm": 0.9551082849502563,
1867
+ "learning_rate": 2.7622710251998786e-05,
1868
+ "loss": 1.2647,
1869
+ "step": 1305
1870
+ },
1871
+ {
1872
+ "epoch": 0.2518988558792424,
1873
+ "grad_norm": 1.0255504846572876,
1874
+ "learning_rate": 2.7607529602266977e-05,
1875
+ "loss": 1.2722,
1876
+ "step": 1310
1877
+ },
1878
+ {
1879
+ "epoch": 0.2528603018940486,
1880
+ "grad_norm": 0.976844310760498,
1881
+ "learning_rate": 2.759234895253517e-05,
1882
+ "loss": 1.2058,
1883
+ "step": 1315
1884
+ },
1885
+ {
1886
+ "epoch": 0.2538217479088549,
1887
+ "grad_norm": 0.9272964000701904,
1888
+ "learning_rate": 2.7577168302803362e-05,
1889
+ "loss": 1.2655,
1890
+ "step": 1320
1891
+ },
1892
+ {
1893
+ "epoch": 0.25478319392366117,
1894
+ "grad_norm": 0.9797655344009399,
1895
+ "learning_rate": 2.7561987653071553e-05,
1896
+ "loss": 1.278,
1897
+ "step": 1325
1898
+ },
1899
+ {
1900
+ "epoch": 0.25574463993846747,
1901
+ "grad_norm": 0.9461880326271057,
1902
+ "learning_rate": 2.7546807003339744e-05,
1903
+ "loss": 1.2449,
1904
+ "step": 1330
1905
+ },
1906
+ {
1907
+ "epoch": 0.2567060859532737,
1908
+ "grad_norm": 0.9833115339279175,
1909
+ "learning_rate": 2.7531626353607934e-05,
1910
+ "loss": 1.1786,
1911
+ "step": 1335
1912
+ },
1913
+ {
1914
+ "epoch": 0.25766753196808,
1915
+ "grad_norm": 1.1373913288116455,
1916
+ "learning_rate": 2.751644570387613e-05,
1917
+ "loss": 1.2068,
1918
+ "step": 1340
1919
+ },
1920
+ {
1921
+ "epoch": 0.25862897798288625,
1922
+ "grad_norm": 0.9782432317733765,
1923
+ "learning_rate": 2.750126505414432e-05,
1924
+ "loss": 1.2333,
1925
+ "step": 1345
1926
+ },
1927
+ {
1928
+ "epoch": 0.25959042399769255,
1929
+ "grad_norm": 1.0395206212997437,
1930
+ "learning_rate": 2.748608440441251e-05,
1931
+ "loss": 1.1949,
1932
+ "step": 1350
1933
+ },
1934
+ {
1935
+ "epoch": 0.2605518700124988,
1936
+ "grad_norm": 0.9710389971733093,
1937
+ "learning_rate": 2.74709037546807e-05,
1938
+ "loss": 1.2321,
1939
+ "step": 1355
1940
+ },
1941
+ {
1942
+ "epoch": 0.26151331602730504,
1943
+ "grad_norm": 0.9782922267913818,
1944
+ "learning_rate": 2.7455723104948895e-05,
1945
+ "loss": 1.2246,
1946
+ "step": 1360
1947
+ },
1948
+ {
1949
+ "epoch": 0.26247476204211134,
1950
+ "grad_norm": 1.0231250524520874,
1951
+ "learning_rate": 2.7440542455217086e-05,
1952
+ "loss": 1.2761,
1953
+ "step": 1365
1954
+ },
1955
+ {
1956
+ "epoch": 0.2634362080569176,
1957
+ "grad_norm": 0.9569921493530273,
1958
+ "learning_rate": 2.7425361805485276e-05,
1959
+ "loss": 1.1911,
1960
+ "step": 1370
1961
+ },
1962
+ {
1963
+ "epoch": 0.2643976540717239,
1964
+ "grad_norm": 1.121079921722412,
1965
+ "learning_rate": 2.7410181155753467e-05,
1966
+ "loss": 1.2505,
1967
+ "step": 1375
1968
+ },
1969
+ {
1970
+ "epoch": 0.26535910008653013,
1971
+ "grad_norm": 0.9387443661689758,
1972
+ "learning_rate": 2.7395000506021658e-05,
1973
+ "loss": 1.2585,
1974
+ "step": 1380
1975
+ },
1976
+ {
1977
+ "epoch": 0.26632054610133643,
1978
+ "grad_norm": 1.0291073322296143,
1979
+ "learning_rate": 2.7379819856289852e-05,
1980
+ "loss": 1.1902,
1981
+ "step": 1385
1982
+ },
1983
+ {
1984
+ "epoch": 0.2672819921161427,
1985
+ "grad_norm": 1.0154651403427124,
1986
+ "learning_rate": 2.7364639206558043e-05,
1987
+ "loss": 1.1511,
1988
+ "step": 1390
1989
+ },
1990
+ {
1991
+ "epoch": 0.268243438130949,
1992
+ "grad_norm": 1.0184061527252197,
1993
+ "learning_rate": 2.7349458556826234e-05,
1994
+ "loss": 1.2372,
1995
+ "step": 1395
1996
+ },
1997
+ {
1998
+ "epoch": 0.2692048841457552,
1999
+ "grad_norm": 1.0806524753570557,
2000
+ "learning_rate": 2.7334277907094424e-05,
2001
+ "loss": 1.2685,
2002
+ "step": 1400
2003
+ },
2004
+ {
2005
+ "epoch": 0.27016633016056146,
2006
+ "grad_norm": 1.0703365802764893,
2007
+ "learning_rate": 2.7319097257362615e-05,
2008
+ "loss": 1.2318,
2009
+ "step": 1405
2010
+ },
2011
+ {
2012
+ "epoch": 0.27112777617536776,
2013
+ "grad_norm": 1.0059064626693726,
2014
+ "learning_rate": 2.730391660763081e-05,
2015
+ "loss": 1.227,
2016
+ "step": 1410
2017
+ },
2018
+ {
2019
+ "epoch": 0.272089222190174,
2020
+ "grad_norm": 1.0088706016540527,
2021
+ "learning_rate": 2.7288735957899e-05,
2022
+ "loss": 1.2514,
2023
+ "step": 1415
2024
+ },
2025
+ {
2026
+ "epoch": 0.2730506682049803,
2027
+ "grad_norm": 0.8656735420227051,
2028
+ "learning_rate": 2.727355530816719e-05,
2029
+ "loss": 1.1894,
2030
+ "step": 1420
2031
+ },
2032
+ {
2033
+ "epoch": 0.27401211421978655,
2034
+ "grad_norm": 0.9790570139884949,
2035
+ "learning_rate": 2.725837465843538e-05,
2036
+ "loss": 1.1832,
2037
+ "step": 1425
2038
+ },
2039
+ {
2040
+ "epoch": 0.27497356023459285,
2041
+ "grad_norm": 1.1856906414031982,
2042
+ "learning_rate": 2.7243194008703572e-05,
2043
+ "loss": 1.2734,
2044
+ "step": 1430
2045
+ },
2046
+ {
2047
+ "epoch": 0.2759350062493991,
2048
+ "grad_norm": 0.9872468113899231,
2049
+ "learning_rate": 2.7228013358971767e-05,
2050
+ "loss": 1.26,
2051
+ "step": 1435
2052
+ },
2053
+ {
2054
+ "epoch": 0.27689645226420534,
2055
+ "grad_norm": 0.8397803902626038,
2056
+ "learning_rate": 2.7212832709239957e-05,
2057
+ "loss": 1.239,
2058
+ "step": 1440
2059
+ },
2060
+ {
2061
+ "epoch": 0.27785789827901164,
2062
+ "grad_norm": 0.871387243270874,
2063
+ "learning_rate": 2.7197652059508148e-05,
2064
+ "loss": 1.1995,
2065
+ "step": 1445
2066
+ },
2067
+ {
2068
+ "epoch": 0.2788193442938179,
2069
+ "grad_norm": 0.9028294682502747,
2070
+ "learning_rate": 2.718247140977634e-05,
2071
+ "loss": 1.2067,
2072
+ "step": 1450
2073
+ },
2074
+ {
2075
+ "epoch": 0.2797807903086242,
2076
+ "grad_norm": 1.0373823642730713,
2077
+ "learning_rate": 2.716729076004453e-05,
2078
+ "loss": 1.2968,
2079
+ "step": 1455
2080
+ },
2081
+ {
2082
+ "epoch": 0.2807422363234304,
2083
+ "grad_norm": 0.8917548656463623,
2084
+ "learning_rate": 2.7152110110312724e-05,
2085
+ "loss": 1.2429,
2086
+ "step": 1460
2087
+ },
2088
+ {
2089
+ "epoch": 0.2817036823382367,
2090
+ "grad_norm": 1.0244743824005127,
2091
+ "learning_rate": 2.7136929460580915e-05,
2092
+ "loss": 1.2312,
2093
+ "step": 1465
2094
+ },
2095
+ {
2096
+ "epoch": 0.28266512835304297,
2097
+ "grad_norm": 1.1365673542022705,
2098
+ "learning_rate": 2.7121748810849105e-05,
2099
+ "loss": 1.2658,
2100
+ "step": 1470
2101
+ },
2102
+ {
2103
+ "epoch": 0.28362657436784927,
2104
+ "grad_norm": 1.018379807472229,
2105
+ "learning_rate": 2.7106568161117296e-05,
2106
+ "loss": 1.2666,
2107
+ "step": 1475
2108
+ },
2109
+ {
2110
+ "epoch": 0.2845880203826555,
2111
+ "grad_norm": 1.031159520149231,
2112
+ "learning_rate": 2.7091387511385487e-05,
2113
+ "loss": 1.2293,
2114
+ "step": 1480
2115
+ },
2116
+ {
2117
+ "epoch": 0.28554946639746176,
2118
+ "grad_norm": 0.9636439681053162,
2119
+ "learning_rate": 2.707620686165368e-05,
2120
+ "loss": 1.2292,
2121
+ "step": 1485
2122
+ },
2123
+ {
2124
+ "epoch": 0.28651091241226806,
2125
+ "grad_norm": 0.9116979837417603,
2126
+ "learning_rate": 2.7061026211921872e-05,
2127
+ "loss": 1.2468,
2128
+ "step": 1490
2129
+ },
2130
+ {
2131
+ "epoch": 0.2874723584270743,
2132
+ "grad_norm": 1.0782617330551147,
2133
+ "learning_rate": 2.7045845562190063e-05,
2134
+ "loss": 1.2427,
2135
+ "step": 1495
2136
+ },
2137
+ {
2138
+ "epoch": 0.2884338044418806,
2139
+ "grad_norm": 0.8577907085418701,
2140
+ "learning_rate": 2.7030664912458253e-05,
2141
+ "loss": 1.1824,
2142
+ "step": 1500
2143
+ },
2144
+ {
2145
+ "epoch": 0.2884338044418806,
2146
+ "eval_loss": 1.2134031057357788,
2147
+ "eval_runtime": 1809.8508,
2148
+ "eval_samples_per_second": 11.494,
2149
+ "eval_steps_per_second": 2.874,
2150
+ "step": 1500
2151
+ },
2152
+ {
2153
+ "epoch": 0.23605991686926517,
2154
+ "grad_norm": 0.9484060406684875,
2155
+ "learning_rate": 2.7855373947498763e-05,
2156
+ "loss": 1.2617,
2157
+ "step": 1505
2158
+ },
2159
+ {
2160
+ "epoch": 0.23684416908477768,
2161
+ "grad_norm": 1.177593469619751,
2162
+ "learning_rate": 2.7842991579990096e-05,
2163
+ "loss": 1.2651,
2164
+ "step": 1510
2165
+ },
2166
+ {
2167
+ "epoch": 0.23762842130029016,
2168
+ "grad_norm": 1.026624083518982,
2169
+ "learning_rate": 2.783060921248143e-05,
2170
+ "loss": 1.2084,
2171
+ "step": 1515
2172
+ },
2173
+ {
2174
+ "epoch": 0.23841267351580267,
2175
+ "grad_norm": 0.9503470659255981,
2176
+ "learning_rate": 2.7818226844972762e-05,
2177
+ "loss": 1.1794,
2178
+ "step": 1520
2179
+ },
2180
+ {
2181
+ "epoch": 0.23919692573131518,
2182
+ "grad_norm": 1.0257048606872559,
2183
+ "learning_rate": 2.7805844477464095e-05,
2184
+ "loss": 1.2519,
2185
+ "step": 1525
2186
+ },
2187
+ {
2188
+ "epoch": 0.2399811779468277,
2189
+ "grad_norm": 0.9679185152053833,
2190
+ "learning_rate": 2.7793462109955424e-05,
2191
+ "loss": 1.2237,
2192
+ "step": 1530
2193
+ },
2194
+ {
2195
+ "epoch": 0.2407654301623402,
2196
+ "grad_norm": 0.9033888578414917,
2197
+ "learning_rate": 2.7781079742446757e-05,
2198
+ "loss": 1.2419,
2199
+ "step": 1535
2200
+ },
2201
+ {
2202
+ "epoch": 0.2415496823778527,
2203
+ "grad_norm": 1.0314648151397705,
2204
+ "learning_rate": 2.776869737493809e-05,
2205
+ "loss": 1.1416,
2206
+ "step": 1540
2207
+ },
2208
+ {
2209
+ "epoch": 0.24233393459336522,
2210
+ "grad_norm": 1.0082125663757324,
2211
+ "learning_rate": 2.7756315007429422e-05,
2212
+ "loss": 1.21,
2213
+ "step": 1545
2214
+ },
2215
+ {
2216
+ "epoch": 0.24311818680887773,
2217
+ "grad_norm": 1.155539631843567,
2218
+ "learning_rate": 2.7743932639920755e-05,
2219
+ "loss": 1.2061,
2220
+ "step": 1550
2221
+ },
2222
+ {
2223
+ "epoch": 0.24390243902439024,
2224
+ "grad_norm": 0.9028305411338806,
2225
+ "learning_rate": 2.7731550272412088e-05,
2226
+ "loss": 1.2128,
2227
+ "step": 1555
2228
+ },
2229
+ {
2230
+ "epoch": 0.24468669123990275,
2231
+ "grad_norm": 0.9822825193405151,
2232
+ "learning_rate": 2.771916790490342e-05,
2233
+ "loss": 1.1644,
2234
+ "step": 1560
2235
+ },
2236
+ {
2237
+ "epoch": 0.24547094345541526,
2238
+ "grad_norm": 1.0660860538482666,
2239
+ "learning_rate": 2.7706785537394754e-05,
2240
+ "loss": 1.1766,
2241
+ "step": 1565
2242
+ },
2243
+ {
2244
+ "epoch": 0.24625519567092777,
2245
+ "grad_norm": 1.035258173942566,
2246
+ "learning_rate": 2.7694403169886083e-05,
2247
+ "loss": 1.2011,
2248
+ "step": 1570
2249
+ },
2250
+ {
2251
+ "epoch": 0.24703944788644028,
2252
+ "grad_norm": 1.058321475982666,
2253
+ "learning_rate": 2.7682020802377416e-05,
2254
+ "loss": 1.2155,
2255
+ "step": 1575
2256
+ },
2257
+ {
2258
+ "epoch": 0.2478237001019528,
2259
+ "grad_norm": 0.8912761807441711,
2260
+ "learning_rate": 2.766963843486875e-05,
2261
+ "loss": 1.2136,
2262
+ "step": 1580
2263
+ },
2264
+ {
2265
+ "epoch": 0.2486079523174653,
2266
+ "grad_norm": 1.1558911800384521,
2267
+ "learning_rate": 2.765725606736008e-05,
2268
+ "loss": 1.1699,
2269
+ "step": 1585
2270
+ },
2271
+ {
2272
+ "epoch": 0.2493922045329778,
2273
+ "grad_norm": 1.0567753314971924,
2274
+ "learning_rate": 2.764487369985141e-05,
2275
+ "loss": 1.1509,
2276
+ "step": 1590
2277
+ },
2278
+ {
2279
+ "epoch": 0.2501764567484903,
2280
+ "grad_norm": 1.043428659439087,
2281
+ "learning_rate": 2.7632491332342744e-05,
2282
+ "loss": 1.1715,
2283
+ "step": 1595
2284
+ },
2285
+ {
2286
+ "epoch": 0.2509607089640028,
2287
+ "grad_norm": 1.096039056777954,
2288
+ "learning_rate": 2.7620108964834077e-05,
2289
+ "loss": 1.23,
2290
+ "step": 1600
2291
+ },
2292
+ {
2293
+ "epoch": 0.25174496117951534,
2294
+ "grad_norm": 1.0177409648895264,
2295
+ "learning_rate": 2.760772659732541e-05,
2296
+ "loss": 1.1831,
2297
+ "step": 1605
2298
+ },
2299
+ {
2300
+ "epoch": 0.2525292133950278,
2301
+ "grad_norm": 0.9627098441123962,
2302
+ "learning_rate": 2.759534422981674e-05,
2303
+ "loss": 1.1984,
2304
+ "step": 1610
2305
+ },
2306
+ {
2307
+ "epoch": 0.25331346561054036,
2308
+ "grad_norm": 0.9108409285545349,
2309
+ "learning_rate": 2.7582961862308072e-05,
2310
+ "loss": 1.2579,
2311
+ "step": 1615
2312
+ },
2313
+ {
2314
+ "epoch": 0.25409771782605284,
2315
+ "grad_norm": 1.1243115663528442,
2316
+ "learning_rate": 2.7570579494799405e-05,
2317
+ "loss": 1.1519,
2318
+ "step": 1620
2319
+ },
2320
+ {
2321
+ "epoch": 0.2548819700415654,
2322
+ "grad_norm": 0.9947587847709656,
2323
+ "learning_rate": 2.7558197127290737e-05,
2324
+ "loss": 1.2171,
2325
+ "step": 1625
2326
+ },
2327
+ {
2328
+ "epoch": 0.25566622225707786,
2329
+ "grad_norm": 0.9268325567245483,
2330
+ "learning_rate": 2.754581475978207e-05,
2331
+ "loss": 1.1465,
2332
+ "step": 1630
2333
+ },
2334
+ {
2335
+ "epoch": 0.2564504744725904,
2336
+ "grad_norm": 0.9441059231758118,
2337
+ "learning_rate": 2.7533432392273403e-05,
2338
+ "loss": 1.1612,
2339
+ "step": 1635
2340
+ },
2341
+ {
2342
+ "epoch": 0.2572347266881029,
2343
+ "grad_norm": 0.9630559682846069,
2344
+ "learning_rate": 2.7521050024764736e-05,
2345
+ "loss": 1.2094,
2346
+ "step": 1640
2347
+ },
2348
+ {
2349
+ "epoch": 0.2580189789036154,
2350
+ "grad_norm": 0.8531478047370911,
2351
+ "learning_rate": 2.7508667657256065e-05,
2352
+ "loss": 1.1541,
2353
+ "step": 1645
2354
+ },
2355
+ {
2356
+ "epoch": 0.2588032311191279,
2357
+ "grad_norm": 0.8630156517028809,
2358
+ "learning_rate": 2.7496285289747398e-05,
2359
+ "loss": 1.1737,
2360
+ "step": 1650
2361
+ },
2362
+ {
2363
+ "epoch": 0.25958748333464043,
2364
+ "grad_norm": 1.0434770584106445,
2365
+ "learning_rate": 2.748390292223873e-05,
2366
+ "loss": 1.1819,
2367
+ "step": 1655
2368
+ },
2369
+ {
2370
+ "epoch": 0.2603717355501529,
2371
+ "grad_norm": 1.0269476175308228,
2372
+ "learning_rate": 2.7471520554730064e-05,
2373
+ "loss": 1.2117,
2374
+ "step": 1660
2375
+ },
2376
+ {
2377
+ "epoch": 0.26115598776566545,
2378
+ "grad_norm": 1.110245704650879,
2379
+ "learning_rate": 2.7459138187221397e-05,
2380
+ "loss": 1.226,
2381
+ "step": 1665
2382
+ },
2383
+ {
2384
+ "epoch": 0.26194023998117794,
2385
+ "grad_norm": 1.0157169103622437,
2386
+ "learning_rate": 2.744675581971273e-05,
2387
+ "loss": 1.1693,
2388
+ "step": 1670
2389
+ },
2390
+ {
2391
+ "epoch": 0.2627244921966905,
2392
+ "grad_norm": 1.019687533378601,
2393
+ "learning_rate": 2.7434373452204062e-05,
2394
+ "loss": 1.1955,
2395
+ "step": 1675
2396
+ },
2397
+ {
2398
+ "epoch": 0.26350874441220296,
2399
+ "grad_norm": 1.6299337148666382,
2400
+ "learning_rate": 2.7421991084695395e-05,
2401
+ "loss": 1.2044,
2402
+ "step": 1680
2403
+ },
2404
+ {
2405
+ "epoch": 0.2642929966277155,
2406
+ "grad_norm": 1.0400195121765137,
2407
+ "learning_rate": 2.7409608717186724e-05,
2408
+ "loss": 1.2014,
2409
+ "step": 1685
2410
+ },
2411
+ {
2412
+ "epoch": 0.265077248843228,
2413
+ "grad_norm": 0.9269382357597351,
2414
+ "learning_rate": 2.7397226349678057e-05,
2415
+ "loss": 1.1195,
2416
+ "step": 1690
2417
+ },
2418
+ {
2419
+ "epoch": 0.2658615010587405,
2420
+ "grad_norm": 0.8400747179985046,
2421
+ "learning_rate": 2.738484398216939e-05,
2422
+ "loss": 1.2513,
2423
+ "step": 1695
2424
+ },
2425
+ {
2426
+ "epoch": 0.266645753274253,
2427
+ "grad_norm": 1.018113136291504,
2428
+ "learning_rate": 2.7372461614660723e-05,
2429
+ "loss": 1.201,
2430
+ "step": 1700
2431
+ },
2432
+ {
2433
+ "epoch": 0.26743000548976553,
2434
+ "grad_norm": 0.9843900203704834,
2435
+ "learning_rate": 2.7360079247152056e-05,
2436
+ "loss": 1.115,
2437
+ "step": 1705
2438
+ },
2439
+ {
2440
+ "epoch": 0.268214257705278,
2441
+ "grad_norm": 0.9973382353782654,
2442
+ "learning_rate": 2.734769687964339e-05,
2443
+ "loss": 1.1881,
2444
+ "step": 1710
2445
+ },
2446
+ {
2447
+ "epoch": 0.26899850992079055,
2448
+ "grad_norm": 0.9756709337234497,
2449
+ "learning_rate": 2.733531451213472e-05,
2450
+ "loss": 1.1922,
2451
+ "step": 1715
2452
+ },
2453
+ {
2454
+ "epoch": 0.26978276213630303,
2455
+ "grad_norm": 0.9447654485702515,
2456
+ "learning_rate": 2.7322932144626054e-05,
2457
+ "loss": 1.1474,
2458
+ "step": 1720
2459
+ },
2460
+ {
2461
+ "epoch": 0.27056701435181557,
2462
+ "grad_norm": 1.0497111082077026,
2463
+ "learning_rate": 2.7310549777117384e-05,
2464
+ "loss": 1.2021,
2465
+ "step": 1725
2466
+ },
2467
+ {
2468
+ "epoch": 0.27135126656732805,
2469
+ "grad_norm": 1.121382236480713,
2470
+ "learning_rate": 2.7298167409608716e-05,
2471
+ "loss": 1.2463,
2472
+ "step": 1730
2473
+ },
2474
+ {
2475
+ "epoch": 0.27213551878284054,
2476
+ "grad_norm": 0.9564398527145386,
2477
+ "learning_rate": 2.728578504210005e-05,
2478
+ "loss": 1.1741,
2479
+ "step": 1735
2480
+ },
2481
+ {
2482
+ "epoch": 0.2729197709983531,
2483
+ "grad_norm": 0.993923544883728,
2484
+ "learning_rate": 2.7273402674591382e-05,
2485
+ "loss": 1.1696,
2486
+ "step": 1740
2487
+ },
2488
+ {
2489
+ "epoch": 0.27370402321386555,
2490
+ "grad_norm": 1.1519091129302979,
2491
+ "learning_rate": 2.7261020307082715e-05,
2492
+ "loss": 1.2981,
2493
+ "step": 1745
2494
+ },
2495
+ {
2496
+ "epoch": 0.2744882754293781,
2497
+ "grad_norm": 1.1992340087890625,
2498
+ "learning_rate": 2.7248637939574048e-05,
2499
+ "loss": 1.2348,
2500
+ "step": 1750
2501
+ },
2502
+ {
2503
+ "epoch": 0.2752725276448906,
2504
+ "grad_norm": 0.954759418964386,
2505
+ "learning_rate": 2.723625557206538e-05,
2506
+ "loss": 1.0986,
2507
+ "step": 1755
2508
+ },
2509
+ {
2510
+ "epoch": 0.2760567798604031,
2511
+ "grad_norm": 0.9630308151245117,
2512
+ "learning_rate": 2.7223873204556713e-05,
2513
+ "loss": 1.1829,
2514
+ "step": 1760
2515
+ },
2516
+ {
2517
+ "epoch": 0.2768410320759156,
2518
+ "grad_norm": 0.8537359833717346,
2519
+ "learning_rate": 2.7211490837048043e-05,
2520
+ "loss": 1.1762,
2521
+ "step": 1765
2522
+ },
2523
+ {
2524
+ "epoch": 0.27762528429142813,
2525
+ "grad_norm": 0.8548294305801392,
2526
+ "learning_rate": 2.7199108469539376e-05,
2527
+ "loss": 1.1497,
2528
+ "step": 1770
2529
+ },
2530
+ {
2531
+ "epoch": 0.2784095365069406,
2532
+ "grad_norm": 1.1810269355773926,
2533
+ "learning_rate": 2.718672610203071e-05,
2534
+ "loss": 1.0642,
2535
+ "step": 1775
2536
+ },
2537
+ {
2538
+ "epoch": 0.27919378872245315,
2539
+ "grad_norm": 1.244332194328308,
2540
+ "learning_rate": 2.717434373452204e-05,
2541
+ "loss": 1.1635,
2542
+ "step": 1780
2543
+ },
2544
+ {
2545
+ "epoch": 0.27997804093796563,
2546
+ "grad_norm": 0.9151502251625061,
2547
+ "learning_rate": 2.7161961367013374e-05,
2548
+ "loss": 1.2191,
2549
+ "step": 1785
2550
+ },
2551
+ {
2552
+ "epoch": 0.28076229315347817,
2553
+ "grad_norm": 0.9919447898864746,
2554
+ "learning_rate": 2.7149578999504707e-05,
2555
+ "loss": 1.1431,
2556
+ "step": 1790
2557
+ },
2558
+ {
2559
+ "epoch": 0.28154654536899065,
2560
+ "grad_norm": 1.0805011987686157,
2561
+ "learning_rate": 2.713719663199604e-05,
2562
+ "loss": 1.1191,
2563
+ "step": 1795
2564
+ },
2565
+ {
2566
+ "epoch": 0.2823307975845032,
2567
+ "grad_norm": 1.0525835752487183,
2568
+ "learning_rate": 2.7124814264487373e-05,
2569
+ "loss": 1.1335,
2570
+ "step": 1800
2571
+ },
2572
+ {
2573
+ "epoch": 0.2823307975845032,
2574
+ "eval_loss": 1.1871418952941895,
2575
+ "eval_runtime": 180.9498,
2576
+ "eval_samples_per_second": 11.053,
2577
+ "eval_steps_per_second": 2.763,
2578
+ "step": 1800
2579
+ },
2580
+ {
2581
+ "epoch": 0.2831150498000157,
2582
+ "grad_norm": 0.9144219756126404,
2583
+ "learning_rate": 2.7112431896978702e-05,
2584
+ "loss": 1.2166,
2585
+ "step": 1805
2586
+ },
2587
+ {
2588
+ "epoch": 0.2838993020155282,
2589
+ "grad_norm": 1.0782276391983032,
2590
+ "learning_rate": 2.7100049529470035e-05,
2591
+ "loss": 1.165,
2592
+ "step": 1810
2593
+ },
2594
+ {
2595
+ "epoch": 0.2846835542310407,
2596
+ "grad_norm": 1.0270676612854004,
2597
+ "learning_rate": 2.7087667161961368e-05,
2598
+ "loss": 1.2225,
2599
+ "step": 1815
2600
+ },
2601
+ {
2602
+ "epoch": 0.28546780644655323,
2603
+ "grad_norm": 1.1217344999313354,
2604
+ "learning_rate": 2.70752847944527e-05,
2605
+ "loss": 1.1336,
2606
+ "step": 1820
2607
+ },
2608
+ {
2609
+ "epoch": 0.2862520586620657,
2610
+ "grad_norm": 1.1019388437271118,
2611
+ "learning_rate": 2.7062902426944033e-05,
2612
+ "loss": 1.1739,
2613
+ "step": 1825
2614
+ },
2615
+ {
2616
+ "epoch": 0.28703631087757825,
2617
+ "grad_norm": 1.0203742980957031,
2618
+ "learning_rate": 2.7050520059435366e-05,
2619
+ "loss": 1.1209,
2620
+ "step": 1830
2621
+ },
2622
+ {
2623
+ "epoch": 0.28782056309309073,
2624
+ "grad_norm": 0.9274325370788574,
2625
+ "learning_rate": 2.70381376919267e-05,
2626
+ "loss": 1.164,
2627
+ "step": 1835
2628
+ },
2629
+ {
2630
+ "epoch": 0.28860481530860327,
2631
+ "grad_norm": 1.072279453277588,
2632
+ "learning_rate": 2.702575532441803e-05,
2633
+ "loss": 1.1874,
2634
+ "step": 1840
2635
+ },
2636
+ {
2637
+ "epoch": 0.28938906752411575,
2638
+ "grad_norm": 1.0064656734466553,
2639
+ "learning_rate": 2.701337295690936e-05,
2640
+ "loss": 1.2334,
2641
+ "step": 1845
2642
+ },
2643
+ {
2644
+ "epoch": 0.2901733197396283,
2645
+ "grad_norm": 1.0672551393508911,
2646
+ "learning_rate": 2.7000990589400694e-05,
2647
+ "loss": 1.1567,
2648
+ "step": 1850
2649
+ },
2650
+ {
2651
+ "epoch": 0.29095757195514077,
2652
+ "grad_norm": 1.044426441192627,
2653
+ "learning_rate": 2.6988608221892027e-05,
2654
+ "loss": 1.1743,
2655
+ "step": 1855
2656
+ },
2657
+ {
2658
+ "epoch": 0.2917418241706533,
2659
+ "grad_norm": 0.9725475907325745,
2660
+ "learning_rate": 2.697622585438336e-05,
2661
+ "loss": 1.1858,
2662
+ "step": 1860
2663
+ },
2664
+ {
2665
+ "epoch": 0.2925260763861658,
2666
+ "grad_norm": 1.3251436948776245,
2667
+ "learning_rate": 2.6963843486874692e-05,
2668
+ "loss": 1.1101,
2669
+ "step": 1865
2670
+ },
2671
+ {
2672
+ "epoch": 0.2933103286016783,
2673
+ "grad_norm": 0.9774639010429382,
2674
+ "learning_rate": 2.6951461119366025e-05,
2675
+ "loss": 1.14,
2676
+ "step": 1870
2677
+ },
2678
+ {
2679
+ "epoch": 0.2940945808171908,
2680
+ "grad_norm": 0.8695831894874573,
2681
+ "learning_rate": 2.6939078751857358e-05,
2682
+ "loss": 1.2076,
2683
+ "step": 1875
2684
+ },
2685
+ {
2686
+ "epoch": 0.2948788330327033,
2687
+ "grad_norm": 1.1581186056137085,
2688
+ "learning_rate": 2.6926696384348688e-05,
2689
+ "loss": 1.1829,
2690
+ "step": 1880
2691
+ },
2692
+ {
2693
+ "epoch": 0.29566308524821583,
2694
+ "grad_norm": 0.9855690002441406,
2695
+ "learning_rate": 2.691431401684002e-05,
2696
+ "loss": 1.0984,
2697
+ "step": 1885
2698
+ },
2699
+ {
2700
+ "epoch": 0.2964473374637283,
2701
+ "grad_norm": 0.9336042404174805,
2702
+ "learning_rate": 2.6901931649331353e-05,
2703
+ "loss": 1.194,
2704
+ "step": 1890
2705
+ },
2706
+ {
2707
+ "epoch": 0.29723158967924085,
2708
+ "grad_norm": 0.9858350157737732,
2709
+ "learning_rate": 2.6889549281822686e-05,
2710
+ "loss": 1.2448,
2711
+ "step": 1895
2712
+ },
2713
+ {
2714
+ "epoch": 0.29801584189475333,
2715
+ "grad_norm": 1.166487693786621,
2716
+ "learning_rate": 2.687716691431402e-05,
2717
+ "loss": 1.2255,
2718
+ "step": 1900
2719
+ },
2720
+ {
2721
+ "epoch": 0.29880009411026587,
2722
+ "grad_norm": 1.1159576177597046,
2723
+ "learning_rate": 2.686478454680535e-05,
2724
+ "loss": 1.2223,
2725
+ "step": 1905
2726
+ },
2727
+ {
2728
+ "epoch": 0.29958434632577835,
2729
+ "grad_norm": 0.9471052289009094,
2730
+ "learning_rate": 2.6852402179296684e-05,
2731
+ "loss": 1.1046,
2732
+ "step": 1910
2733
+ },
2734
+ {
2735
+ "epoch": 0.3003685985412909,
2736
+ "grad_norm": 1.2097647190093994,
2737
+ "learning_rate": 2.6840019811788017e-05,
2738
+ "loss": 1.2032,
2739
+ "step": 1915
2740
+ },
2741
+ {
2742
+ "epoch": 0.30115285075680337,
2743
+ "grad_norm": 0.9261752367019653,
2744
+ "learning_rate": 2.6827637444279347e-05,
2745
+ "loss": 1.2118,
2746
+ "step": 1920
2747
+ },
2748
+ {
2749
+ "epoch": 0.3019371029723159,
2750
+ "grad_norm": 1.0517253875732422,
2751
+ "learning_rate": 2.681525507677068e-05,
2752
+ "loss": 1.1608,
2753
+ "step": 1925
2754
+ },
2755
+ {
2756
+ "epoch": 0.3027213551878284,
2757
+ "grad_norm": 1.0006099939346313,
2758
+ "learning_rate": 2.6802872709262012e-05,
2759
+ "loss": 1.2163,
2760
+ "step": 1930
2761
+ },
2762
+ {
2763
+ "epoch": 0.3035056074033409,
2764
+ "grad_norm": 1.055713176727295,
2765
+ "learning_rate": 2.6790490341753345e-05,
2766
+ "loss": 1.0843,
2767
+ "step": 1935
2768
+ },
2769
+ {
2770
+ "epoch": 0.3042898596188534,
2771
+ "grad_norm": 1.0398874282836914,
2772
+ "learning_rate": 2.6778107974244678e-05,
2773
+ "loss": 1.1755,
2774
+ "step": 1940
2775
+ },
2776
+ {
2777
+ "epoch": 0.30507411183436595,
2778
+ "grad_norm": 0.94144606590271,
2779
+ "learning_rate": 2.676572560673601e-05,
2780
+ "loss": 1.186,
2781
+ "step": 1945
2782
+ },
2783
+ {
2784
+ "epoch": 0.30585836404987843,
2785
+ "grad_norm": 0.9723899364471436,
2786
+ "learning_rate": 2.6753343239227344e-05,
2787
+ "loss": 1.1243,
2788
+ "step": 1950
2789
+ },
2790
+ {
2791
+ "epoch": 0.30664261626539097,
2792
+ "grad_norm": 1.2370935678482056,
2793
+ "learning_rate": 2.6740960871718676e-05,
2794
+ "loss": 1.2102,
2795
+ "step": 1955
2796
+ },
2797
+ {
2798
+ "epoch": 0.30742686848090345,
2799
+ "grad_norm": 1.2505570650100708,
2800
+ "learning_rate": 2.6728578504210006e-05,
2801
+ "loss": 1.1864,
2802
+ "step": 1960
2803
+ },
2804
+ {
2805
+ "epoch": 0.308211120696416,
2806
+ "grad_norm": 1.3583003282546997,
2807
+ "learning_rate": 2.6716196136701335e-05,
2808
+ "loss": 1.1566,
2809
+ "step": 1965
2810
+ },
2811
+ {
2812
+ "epoch": 0.30899537291192847,
2813
+ "grad_norm": 0.9221394658088684,
2814
+ "learning_rate": 2.6703813769192668e-05,
2815
+ "loss": 1.2274,
2816
+ "step": 1970
2817
+ },
2818
+ {
2819
+ "epoch": 0.309779625127441,
2820
+ "grad_norm": 1.09176504611969,
2821
+ "learning_rate": 2.6691431401684e-05,
2822
+ "loss": 1.2366,
2823
+ "step": 1975
2824
+ },
2825
+ {
2826
+ "epoch": 0.3105638773429535,
2827
+ "grad_norm": 1.0992720127105713,
2828
+ "learning_rate": 2.6679049034175334e-05,
2829
+ "loss": 1.2034,
2830
+ "step": 1980
2831
+ },
2832
+ {
2833
+ "epoch": 0.311348129558466,
2834
+ "grad_norm": 0.9352986216545105,
2835
+ "learning_rate": 2.6666666666666667e-05,
2836
+ "loss": 1.2083,
2837
+ "step": 1985
2838
+ },
2839
+ {
2840
+ "epoch": 0.3121323817739785,
2841
+ "grad_norm": 0.9142504334449768,
2842
+ "learning_rate": 2.6654284299158e-05,
2843
+ "loss": 1.1766,
2844
+ "step": 1990
2845
+ },
2846
+ {
2847
+ "epoch": 0.31291663398949104,
2848
+ "grad_norm": 0.9476444721221924,
2849
+ "learning_rate": 2.6641901931649332e-05,
2850
+ "loss": 1.1211,
2851
+ "step": 1995
2852
+ },
2853
+ {
2854
+ "epoch": 0.3137008862050035,
2855
+ "grad_norm": 1.0240795612335205,
2856
+ "learning_rate": 2.662951956414066e-05,
2857
+ "loss": 1.1624,
2858
+ "step": 2000
2859
+ },
2860
+ {
2861
+ "epoch": 0.31448513842051606,
2862
+ "grad_norm": 0.9615570902824402,
2863
+ "learning_rate": 2.6617137196631994e-05,
2864
+ "loss": 1.1661,
2865
+ "step": 2005
2866
+ },
2867
+ {
2868
+ "epoch": 0.31526939063602855,
2869
+ "grad_norm": 1.0321637392044067,
2870
+ "learning_rate": 2.6604754829123327e-05,
2871
+ "loss": 1.1595,
2872
+ "step": 2010
2873
+ },
2874
+ {
2875
+ "epoch": 0.31605364285154103,
2876
+ "grad_norm": 1.2370376586914062,
2877
+ "learning_rate": 2.659237246161466e-05,
2878
+ "loss": 1.218,
2879
+ "step": 2015
2880
+ },
2881
+ {
2882
+ "epoch": 0.31683789506705357,
2883
+ "grad_norm": 1.012313723564148,
2884
+ "learning_rate": 2.6579990094105993e-05,
2885
+ "loss": 1.1164,
2886
+ "step": 2020
2887
+ },
2888
+ {
2889
+ "epoch": 0.31762214728256605,
2890
+ "grad_norm": 1.0664358139038086,
2891
+ "learning_rate": 2.6567607726597326e-05,
2892
+ "loss": 1.1516,
2893
+ "step": 2025
2894
+ },
2895
+ {
2896
+ "epoch": 0.3184063994980786,
2897
+ "grad_norm": 1.0644055604934692,
2898
+ "learning_rate": 2.655522535908866e-05,
2899
+ "loss": 1.2166,
2900
+ "step": 2030
2901
+ },
2902
+ {
2903
+ "epoch": 0.31919065171359107,
2904
+ "grad_norm": 0.9738732576370239,
2905
+ "learning_rate": 2.654284299157999e-05,
2906
+ "loss": 1.144,
2907
+ "step": 2035
2908
+ },
2909
+ {
2910
+ "epoch": 0.3199749039291036,
2911
+ "grad_norm": 1.080315351486206,
2912
+ "learning_rate": 2.653046062407132e-05,
2913
+ "loss": 1.1568,
2914
+ "step": 2040
2915
+ },
2916
+ {
2917
+ "epoch": 0.3207591561446161,
2918
+ "grad_norm": 1.4979441165924072,
2919
+ "learning_rate": 2.6518078256562654e-05,
2920
+ "loss": 1.2082,
2921
+ "step": 2045
2922
+ },
2923
+ {
2924
+ "epoch": 0.3215434083601286,
2925
+ "grad_norm": 1.1411837339401245,
2926
+ "learning_rate": 2.6505695889053986e-05,
2927
+ "loss": 1.1533,
2928
+ "step": 2050
2929
+ },
2930
+ {
2931
+ "epoch": 0.3223276605756411,
2932
+ "grad_norm": 1.117168664932251,
2933
+ "learning_rate": 2.649331352154532e-05,
2934
+ "loss": 1.2033,
2935
+ "step": 2055
2936
+ },
2937
+ {
2938
+ "epoch": 0.32311191279115364,
2939
+ "grad_norm": 1.1053581237792969,
2940
+ "learning_rate": 2.6480931154036652e-05,
2941
+ "loss": 1.1623,
2942
+ "step": 2060
2943
+ },
2944
+ {
2945
+ "epoch": 0.3238961650066661,
2946
+ "grad_norm": 0.9980604648590088,
2947
+ "learning_rate": 2.6468548786527985e-05,
2948
+ "loss": 1.1331,
2949
+ "step": 2065
2950
+ },
2951
+ {
2952
+ "epoch": 0.32468041722217866,
2953
+ "grad_norm": 1.0467514991760254,
2954
+ "learning_rate": 2.6456166419019318e-05,
2955
+ "loss": 1.0776,
2956
+ "step": 2070
2957
+ },
2958
+ {
2959
+ "epoch": 0.32546466943769115,
2960
+ "grad_norm": 0.9543486833572388,
2961
+ "learning_rate": 2.6443784051510647e-05,
2962
+ "loss": 1.1842,
2963
+ "step": 2075
2964
+ },
2965
+ {
2966
+ "epoch": 0.3262489216532037,
2967
+ "grad_norm": 1.0781240463256836,
2968
+ "learning_rate": 2.643140168400198e-05,
2969
+ "loss": 1.2276,
2970
+ "step": 2080
2971
+ },
2972
+ {
2973
+ "epoch": 0.32703317386871616,
2974
+ "grad_norm": 0.9011218547821045,
2975
+ "learning_rate": 2.6419019316493313e-05,
2976
+ "loss": 1.1861,
2977
+ "step": 2085
2978
+ },
2979
+ {
2980
+ "epoch": 0.3278174260842287,
2981
+ "grad_norm": 0.9879097938537598,
2982
+ "learning_rate": 2.6406636948984646e-05,
2983
+ "loss": 1.1592,
2984
+ "step": 2090
2985
+ },
2986
+ {
2987
+ "epoch": 0.3286016782997412,
2988
+ "grad_norm": 0.9515993595123291,
2989
+ "learning_rate": 2.639425458147598e-05,
2990
+ "loss": 1.1446,
2991
+ "step": 2095
2992
+ },
2993
+ {
2994
+ "epoch": 0.3293859305152537,
2995
+ "grad_norm": 0.8950434327125549,
2996
+ "learning_rate": 2.638187221396731e-05,
2997
+ "loss": 1.1898,
2998
+ "step": 2100
2999
+ },
3000
+ {
3001
+ "epoch": 0.3293859305152537,
3002
+ "eval_loss": 1.1681427955627441,
3003
+ "eval_runtime": 179.1096,
3004
+ "eval_samples_per_second": 11.166,
3005
+ "eval_steps_per_second": 2.792,
3006
+ "step": 2100
3007
+ },
3008
+ {
3009
+ "epoch": 0.3301701827307662,
3010
+ "grad_norm": 0.9106898307800293,
3011
+ "learning_rate": 2.6369489846458644e-05,
3012
+ "loss": 1.1754,
3013
+ "step": 2105
3014
+ },
3015
+ {
3016
+ "epoch": 0.33095443494627874,
3017
+ "grad_norm": 1.0223875045776367,
3018
+ "learning_rate": 2.6357107478949977e-05,
3019
+ "loss": 1.1653,
3020
+ "step": 2110
3021
+ },
3022
+ {
3023
+ "epoch": 0.3317386871617912,
3024
+ "grad_norm": 1.2506979703903198,
3025
+ "learning_rate": 2.6344725111441306e-05,
3026
+ "loss": 1.1476,
3027
+ "step": 2115
3028
+ },
3029
+ {
3030
+ "epoch": 0.33252293937730376,
3031
+ "grad_norm": 1.1079163551330566,
3032
+ "learning_rate": 2.633234274393264e-05,
3033
+ "loss": 1.1618,
3034
+ "step": 2120
3035
+ },
3036
+ {
3037
+ "epoch": 0.33330719159281624,
3038
+ "grad_norm": 1.060522198677063,
3039
+ "learning_rate": 2.6319960376423972e-05,
3040
+ "loss": 1.2003,
3041
+ "step": 2125
3042
+ },
3043
+ {
3044
+ "epoch": 0.3340914438083288,
3045
+ "grad_norm": 1.1587939262390137,
3046
+ "learning_rate": 2.6307578008915305e-05,
3047
+ "loss": 1.0764,
3048
+ "step": 2130
3049
+ },
3050
+ {
3051
+ "epoch": 0.33487569602384126,
3052
+ "grad_norm": 1.041363000869751,
3053
+ "learning_rate": 2.6295195641406638e-05,
3054
+ "loss": 1.1807,
3055
+ "step": 2135
3056
+ },
3057
+ {
3058
+ "epoch": 0.3356599482393538,
3059
+ "grad_norm": 1.0703729391098022,
3060
+ "learning_rate": 2.628281327389797e-05,
3061
+ "loss": 1.1269,
3062
+ "step": 2140
3063
+ },
3064
+ {
3065
+ "epoch": 0.3364442004548663,
3066
+ "grad_norm": 1.1608914136886597,
3067
+ "learning_rate": 2.6270430906389303e-05,
3068
+ "loss": 1.1839,
3069
+ "step": 2145
3070
+ },
3071
+ {
3072
+ "epoch": 0.3372284526703788,
3073
+ "grad_norm": 1.1163076162338257,
3074
+ "learning_rate": 2.6258048538880636e-05,
3075
+ "loss": 1.129,
3076
+ "step": 2150
3077
+ },
3078
+ {
3079
+ "epoch": 0.3380127048858913,
3080
+ "grad_norm": 1.1649436950683594,
3081
+ "learning_rate": 2.6245666171371966e-05,
3082
+ "loss": 1.1632,
3083
+ "step": 2155
3084
+ },
3085
+ {
3086
+ "epoch": 0.3387969571014038,
3087
+ "grad_norm": 0.9917750954627991,
3088
+ "learning_rate": 2.62332838038633e-05,
3089
+ "loss": 1.2069,
3090
+ "step": 2160
3091
+ },
3092
+ {
3093
+ "epoch": 0.3395812093169163,
3094
+ "grad_norm": 1.0782402753829956,
3095
+ "learning_rate": 2.622090143635463e-05,
3096
+ "loss": 1.1399,
3097
+ "step": 2165
3098
+ },
3099
+ {
3100
+ "epoch": 0.3403654615324288,
3101
+ "grad_norm": 1.0078198909759521,
3102
+ "learning_rate": 2.6208519068845964e-05,
3103
+ "loss": 1.1705,
3104
+ "step": 2170
3105
+ },
3106
+ {
3107
+ "epoch": 0.34114971374794134,
3108
+ "grad_norm": 0.9952524304389954,
3109
+ "learning_rate": 2.6196136701337297e-05,
3110
+ "loss": 1.1471,
3111
+ "step": 2175
3112
+ },
3113
+ {
3114
+ "epoch": 0.3419339659634538,
3115
+ "grad_norm": 1.0381946563720703,
3116
+ "learning_rate": 2.618375433382863e-05,
3117
+ "loss": 1.1568,
3118
+ "step": 2180
3119
+ },
3120
+ {
3121
+ "epoch": 0.34271821817896636,
3122
+ "grad_norm": 1.0317137241363525,
3123
+ "learning_rate": 2.6171371966319962e-05,
3124
+ "loss": 1.1823,
3125
+ "step": 2185
3126
+ },
3127
+ {
3128
+ "epoch": 0.34350247039447884,
3129
+ "grad_norm": 0.9337929487228394,
3130
+ "learning_rate": 2.6158989598811295e-05,
3131
+ "loss": 1.1432,
3132
+ "step": 2190
3133
+ },
3134
+ {
3135
+ "epoch": 0.3442867226099914,
3136
+ "grad_norm": 0.9774899482727051,
3137
+ "learning_rate": 2.6146607231302625e-05,
3138
+ "loss": 1.062,
3139
+ "step": 2195
3140
+ },
3141
+ {
3142
+ "epoch": 0.34507097482550386,
3143
+ "grad_norm": 1.2686885595321655,
3144
+ "learning_rate": 2.6134224863793958e-05,
3145
+ "loss": 1.1447,
3146
+ "step": 2200
3147
+ },
3148
+ {
3149
+ "epoch": 0.3458552270410164,
3150
+ "grad_norm": 1.043630599975586,
3151
+ "learning_rate": 2.612184249628529e-05,
3152
+ "loss": 1.1469,
3153
+ "step": 2205
3154
+ },
3155
+ {
3156
+ "epoch": 0.3466394792565289,
3157
+ "grad_norm": 0.9587001800537109,
3158
+ "learning_rate": 2.6109460128776623e-05,
3159
+ "loss": 1.1216,
3160
+ "step": 2210
3161
+ },
3162
+ {
3163
+ "epoch": 0.3474237314720414,
3164
+ "grad_norm": 0.9291378855705261,
3165
+ "learning_rate": 2.6097077761267956e-05,
3166
+ "loss": 1.1394,
3167
+ "step": 2215
3168
+ },
3169
+ {
3170
+ "epoch": 0.3482079836875539,
3171
+ "grad_norm": 1.1724514961242676,
3172
+ "learning_rate": 2.608469539375929e-05,
3173
+ "loss": 1.0988,
3174
+ "step": 2220
3175
+ },
3176
+ {
3177
+ "epoch": 0.34899223590306644,
3178
+ "grad_norm": 1.0215635299682617,
3179
+ "learning_rate": 2.607231302625062e-05,
3180
+ "loss": 1.1086,
3181
+ "step": 2225
3182
+ },
3183
+ {
3184
+ "epoch": 0.3497764881185789,
3185
+ "grad_norm": 1.0366919040679932,
3186
+ "learning_rate": 2.6059930658741954e-05,
3187
+ "loss": 1.1486,
3188
+ "step": 2230
3189
+ },
3190
+ {
3191
+ "epoch": 0.35056074033409146,
3192
+ "grad_norm": 1.0955818891525269,
3193
+ "learning_rate": 2.6047548291233284e-05,
3194
+ "loss": 1.114,
3195
+ "step": 2235
3196
+ },
3197
+ {
3198
+ "epoch": 0.35134499254960394,
3199
+ "grad_norm": 0.9413565993309021,
3200
+ "learning_rate": 2.6035165923724617e-05,
3201
+ "loss": 1.2112,
3202
+ "step": 2240
3203
+ },
3204
+ {
3205
+ "epoch": 0.3521292447651165,
3206
+ "grad_norm": 0.9644827842712402,
3207
+ "learning_rate": 2.602278355621595e-05,
3208
+ "loss": 1.1204,
3209
+ "step": 2245
3210
+ },
3211
+ {
3212
+ "epoch": 0.35291349698062896,
3213
+ "grad_norm": 0.9966211915016174,
3214
+ "learning_rate": 2.6010401188707282e-05,
3215
+ "loss": 1.1577,
3216
+ "step": 2250
3217
+ },
3218
+ {
3219
+ "epoch": 0.3536977491961415,
3220
+ "grad_norm": 0.9792212247848511,
3221
+ "learning_rate": 2.5998018821198615e-05,
3222
+ "loss": 1.1731,
3223
+ "step": 2255
3224
+ },
3225
+ {
3226
+ "epoch": 0.354482001411654,
3227
+ "grad_norm": 1.0859739780426025,
3228
+ "learning_rate": 2.5985636453689948e-05,
3229
+ "loss": 1.0999,
3230
+ "step": 2260
3231
+ },
3232
+ {
3233
+ "epoch": 0.3552662536271665,
3234
+ "grad_norm": 1.0696343183517456,
3235
+ "learning_rate": 2.597325408618128e-05,
3236
+ "loss": 1.1291,
3237
+ "step": 2265
3238
+ },
3239
+ {
3240
+ "epoch": 0.356050505842679,
3241
+ "grad_norm": 0.8906198740005493,
3242
+ "learning_rate": 2.596087171867261e-05,
3243
+ "loss": 1.1939,
3244
+ "step": 2270
3245
+ },
3246
+ {
3247
+ "epoch": 0.35683475805819154,
3248
+ "grad_norm": 0.9733899235725403,
3249
+ "learning_rate": 2.5948489351163943e-05,
3250
+ "loss": 1.0984,
3251
+ "step": 2275
3252
+ },
3253
+ {
3254
+ "epoch": 0.357619010273704,
3255
+ "grad_norm": 1.212216854095459,
3256
+ "learning_rate": 2.5936106983655276e-05,
3257
+ "loss": 1.1115,
3258
+ "step": 2280
3259
+ },
3260
+ {
3261
+ "epoch": 0.35840326248921656,
3262
+ "grad_norm": 0.895348072052002,
3263
+ "learning_rate": 2.592372461614661e-05,
3264
+ "loss": 1.0905,
3265
+ "step": 2285
3266
+ },
3267
+ {
3268
+ "epoch": 0.35918751470472904,
3269
+ "grad_norm": 0.8904415965080261,
3270
+ "learning_rate": 2.591134224863794e-05,
3271
+ "loss": 1.2289,
3272
+ "step": 2290
3273
+ },
3274
+ {
3275
+ "epoch": 0.3599717669202416,
3276
+ "grad_norm": 0.9107303023338318,
3277
+ "learning_rate": 2.5898959881129274e-05,
3278
+ "loss": 1.1023,
3279
+ "step": 2295
3280
+ },
3281
+ {
3282
+ "epoch": 0.36075601913575406,
3283
+ "grad_norm": 1.031053066253662,
3284
+ "learning_rate": 2.5886577513620607e-05,
3285
+ "loss": 1.0644,
3286
+ "step": 2300
3287
+ },
3288
+ {
3289
+ "epoch": 0.36154027135126654,
3290
+ "grad_norm": 0.9950774312019348,
3291
+ "learning_rate": 2.587419514611194e-05,
3292
+ "loss": 1.1475,
3293
+ "step": 2305
3294
+ },
3295
+ {
3296
+ "epoch": 0.3623245235667791,
3297
+ "grad_norm": 1.0093636512756348,
3298
+ "learning_rate": 2.586181277860327e-05,
3299
+ "loss": 1.1459,
3300
+ "step": 2310
3301
+ },
3302
+ {
3303
+ "epoch": 0.36310877578229156,
3304
+ "grad_norm": 1.3535128831863403,
3305
+ "learning_rate": 2.5849430411094602e-05,
3306
+ "loss": 1.1934,
3307
+ "step": 2315
3308
+ },
3309
+ {
3310
+ "epoch": 0.3638930279978041,
3311
+ "grad_norm": 0.996488094329834,
3312
+ "learning_rate": 2.5837048043585935e-05,
3313
+ "loss": 1.1603,
3314
+ "step": 2320
3315
+ },
3316
+ {
3317
+ "epoch": 0.3646772802133166,
3318
+ "grad_norm": 1.5754882097244263,
3319
+ "learning_rate": 2.5824665676077268e-05,
3320
+ "loss": 1.0841,
3321
+ "step": 2325
3322
+ },
3323
+ {
3324
+ "epoch": 0.3654615324288291,
3325
+ "grad_norm": 1.000685214996338,
3326
+ "learning_rate": 2.58122833085686e-05,
3327
+ "loss": 1.2038,
3328
+ "step": 2330
3329
+ },
3330
+ {
3331
+ "epoch": 0.3662457846443416,
3332
+ "grad_norm": 1.0758228302001953,
3333
+ "learning_rate": 2.5799900941059934e-05,
3334
+ "loss": 1.1153,
3335
+ "step": 2335
3336
+ },
3337
+ {
3338
+ "epoch": 0.36703003685985414,
3339
+ "grad_norm": 1.0312342643737793,
3340
+ "learning_rate": 2.5787518573551266e-05,
3341
+ "loss": 1.1022,
3342
+ "step": 2340
3343
+ },
3344
+ {
3345
+ "epoch": 0.3678142890753666,
3346
+ "grad_norm": 0.98909991979599,
3347
+ "learning_rate": 2.5775136206042596e-05,
3348
+ "loss": 1.1798,
3349
+ "step": 2345
3350
+ },
3351
+ {
3352
+ "epoch": 0.36859854129087916,
3353
+ "grad_norm": 1.0817371606826782,
3354
+ "learning_rate": 2.5762753838533925e-05,
3355
+ "loss": 1.1854,
3356
+ "step": 2350
3357
+ },
3358
+ {
3359
+ "epoch": 0.36938279350639164,
3360
+ "grad_norm": 1.0057563781738281,
3361
+ "learning_rate": 2.5750371471025258e-05,
3362
+ "loss": 1.1812,
3363
+ "step": 2355
3364
+ },
3365
+ {
3366
+ "epoch": 0.3701670457219042,
3367
+ "grad_norm": 1.2944605350494385,
3368
+ "learning_rate": 2.573798910351659e-05,
3369
+ "loss": 1.077,
3370
+ "step": 2360
3371
+ },
3372
+ {
3373
+ "epoch": 0.37095129793741666,
3374
+ "grad_norm": 1.098381519317627,
3375
+ "learning_rate": 2.5725606736007924e-05,
3376
+ "loss": 1.1499,
3377
+ "step": 2365
3378
+ },
3379
+ {
3380
+ "epoch": 0.3717355501529292,
3381
+ "grad_norm": 0.9754157066345215,
3382
+ "learning_rate": 2.5713224368499256e-05,
3383
+ "loss": 1.134,
3384
+ "step": 2370
3385
+ },
3386
+ {
3387
+ "epoch": 0.3725198023684417,
3388
+ "grad_norm": 1.0134391784667969,
3389
+ "learning_rate": 2.570084200099059e-05,
3390
+ "loss": 1.133,
3391
+ "step": 2375
3392
+ },
3393
+ {
3394
+ "epoch": 0.3733040545839542,
3395
+ "grad_norm": 1.020948886871338,
3396
+ "learning_rate": 2.5688459633481922e-05,
3397
+ "loss": 1.1475,
3398
+ "step": 2380
3399
+ },
3400
+ {
3401
+ "epoch": 0.3740883067994667,
3402
+ "grad_norm": 1.0865211486816406,
3403
+ "learning_rate": 2.5676077265973255e-05,
3404
+ "loss": 1.163,
3405
+ "step": 2385
3406
+ },
3407
+ {
3408
+ "epoch": 0.37487255901497923,
3409
+ "grad_norm": 1.0372055768966675,
3410
+ "learning_rate": 2.5663694898464584e-05,
3411
+ "loss": 1.1459,
3412
+ "step": 2390
3413
+ },
3414
+ {
3415
+ "epoch": 0.3756568112304917,
3416
+ "grad_norm": 0.95745849609375,
3417
+ "learning_rate": 2.5651312530955917e-05,
3418
+ "loss": 1.1553,
3419
+ "step": 2395
3420
+ },
3421
+ {
3422
+ "epoch": 0.37644106344600425,
3423
+ "grad_norm": 1.1391905546188354,
3424
+ "learning_rate": 2.563893016344725e-05,
3425
+ "loss": 1.0905,
3426
+ "step": 2400
3427
+ },
3428
+ {
3429
+ "epoch": 0.37644106344600425,
3430
+ "eval_loss": 1.1524348258972168,
3431
+ "eval_runtime": 178.5756,
3432
+ "eval_samples_per_second": 11.2,
3433
+ "eval_steps_per_second": 2.8,
3434
+ "step": 2400
3435
+ }
3436
+ ],
3437
+ "logging_steps": 5,
3438
+ "max_steps": 12752,
3439
+ "num_input_tokens_seen": 0,
3440
+ "num_train_epochs": 2,
3441
+ "save_steps": 300,
3442
+ "stateful_callbacks": {
3443
+ "EarlyStoppingCallback": [
3444
+ {
3445
+ "args": {
3446
+ "early_stopping_patience": 8,
3447
+ "early_stopping_threshold": 0.01
3448
+ },
3449
+ "attributes": {
3450
+ "early_stopping_patience_counter": 0
3451
+ }
3452
+ },
3453
+ {
3454
+ "args": {
3455
+ "early_stopping_patience": 8,
3456
+ "early_stopping_threshold": 0.01
3457
+ },
3458
+ "attributes": {
3459
+ "early_stopping_patience_counter": 0
3460
+ }
3461
+ },
3462
+ {
3463
+ "args": {
3464
+ "early_stopping_patience": 8,
3465
+ "early_stopping_threshold": 0.01
3466
+ },
3467
+ "attributes": {
3468
+ "early_stopping_patience_counter": 0
3469
+ }
3470
+ },
3471
+ {
3472
+ "args": {
3473
+ "early_stopping_patience": 8,
3474
+ "early_stopping_threshold": 0.01
3475
+ },
3476
+ "attributes": {
3477
+ "early_stopping_patience_counter": 0
3478
+ }
3479
+ },
3480
+ {
3481
+ "args": {
3482
+ "early_stopping_patience": 8,
3483
+ "early_stopping_threshold": 0.01
3484
+ },
3485
+ "attributes": {
3486
+ "early_stopping_patience_counter": 0
3487
+ }
3488
+ },
3489
+ {
3490
+ "args": {
3491
+ "early_stopping_patience": 8,
3492
+ "early_stopping_threshold": 0.01
3493
+ },
3494
+ "attributes": {
3495
+ "early_stopping_patience_counter": 0
3496
+ }
3497
+ },
3498
+ {
3499
+ "args": {
3500
+ "early_stopping_patience": 8,
3501
+ "early_stopping_threshold": 0.01
3502
+ },
3503
+ "attributes": {
3504
+ "early_stopping_patience_counter": 0
3505
+ }
3506
+ },
3507
+ {
3508
+ "args": {
3509
+ "early_stopping_patience": 8,
3510
+ "early_stopping_threshold": 0.01
3511
+ },
3512
+ "attributes": {
3513
+ "early_stopping_patience_counter": 0
3514
+ }
3515
+ },
3516
+ {
3517
+ "args": {
3518
+ "early_stopping_patience": 8,
3519
+ "early_stopping_threshold": 0.01
3520
+ },
3521
+ "attributes": {
3522
+ "early_stopping_patience_counter": 0
3523
+ }
3524
+ },
3525
+ {
3526
+ "args": {
3527
+ "early_stopping_patience": 8,
3528
+ "early_stopping_threshold": 0.01
3529
+ },
3530
+ "attributes": {
3531
+ "early_stopping_patience_counter": 0
3532
+ }
3533
+ },
3534
+ {
3535
+ "args": {
3536
+ "early_stopping_patience": 8,
3537
+ "early_stopping_threshold": 0.01
3538
+ },
3539
+ "attributes": {
3540
+ "early_stopping_patience_counter": 0
3541
+ }
3542
+ },
3543
+ {
3544
+ "args": {
3545
+ "early_stopping_patience": 8,
3546
+ "early_stopping_threshold": 0.01
3547
+ },
3548
+ "attributes": {
3549
+ "early_stopping_patience_counter": 0
3550
+ }
3551
+ },
3552
+ {
3553
+ "args": {
3554
+ "early_stopping_patience": 8,
3555
+ "early_stopping_threshold": 0.01
3556
+ },
3557
+ "attributes": {
3558
+ "early_stopping_patience_counter": 0
3559
+ }
3560
+ },
3561
+ {
3562
+ "args": {
3563
+ "early_stopping_patience": 8,
3564
+ "early_stopping_threshold": 0.01
3565
+ },
3566
+ "attributes": {
3567
+ "early_stopping_patience_counter": 0
3568
+ }
3569
+ },
3570
+ {
3571
+ "args": {
3572
+ "early_stopping_patience": 8,
3573
+ "early_stopping_threshold": 0.01
3574
+ },
3575
+ "attributes": {
3576
+ "early_stopping_patience_counter": 0
3577
+ }
3578
+ },
3579
+ {
3580
+ "args": {
3581
+ "early_stopping_patience": 8,
3582
+ "early_stopping_threshold": 0.01
3583
+ },
3584
+ "attributes": {
3585
+ "early_stopping_patience_counter": 0
3586
+ }
3587
+ },
3588
+ {
3589
+ "args": {
3590
+ "early_stopping_patience": 8,
3591
+ "early_stopping_threshold": 0.01
3592
+ },
3593
+ "attributes": {
3594
+ "early_stopping_patience_counter": 0
3595
+ }
3596
+ },
3597
+ {
3598
+ "args": {
3599
+ "early_stopping_patience": 8,
3600
+ "early_stopping_threshold": 0.01
3601
+ },
3602
+ "attributes": {
3603
+ "early_stopping_patience_counter": 0
3604
+ }
3605
+ },
3606
+ {
3607
+ "args": {
3608
+ "early_stopping_patience": 8,
3609
+ "early_stopping_threshold": 0.01
3610
+ },
3611
+ "attributes": {
3612
+ "early_stopping_patience_counter": 0
3613
+ }
3614
+ },
3615
+ {
3616
+ "args": {
3617
+ "early_stopping_patience": 8,
3618
+ "early_stopping_threshold": 0.01
3619
+ },
3620
+ "attributes": {
3621
+ "early_stopping_patience_counter": 0
3622
+ }
3623
+ },
3624
+ {
3625
+ "args": {
3626
+ "early_stopping_patience": 8,
3627
+ "early_stopping_threshold": 0.01
3628
+ },
3629
+ "attributes": {
3630
+ "early_stopping_patience_counter": 0
3631
+ }
3632
+ },
3633
+ {
3634
+ "args": {
3635
+ "early_stopping_patience": 8,
3636
+ "early_stopping_threshold": 0.01
3637
+ },
3638
+ "attributes": {
3639
+ "early_stopping_patience_counter": 0
3640
+ }
3641
+ }
3642
+ ],
3643
+ "TrainerControl": {
3644
+ "args": {
3645
+ "should_epoch_stop": false,
3646
+ "should_evaluate": false,
3647
+ "should_log": false,
3648
+ "should_save": true,
3649
+ "should_training_stop": false
3650
+ },
3651
+ "attributes": {}
3652
+ }
3653
+ },
3654
+ "total_flos": 1.4247328016884516e+18,
3655
+ "train_batch_size": 8,
3656
+ "trial_name": null,
3657
+ "trial_params": null
3658
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6116de18fab8d9016737084423d5513355a4dcf1904f43f48b0cf855f13e9ed7
3
+ size 6161