Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,210 @@
+---
+base_model: unsloth/gemma-3-12b-it-unsloth-bnb-4bit
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/gemma-3-12b-it-unsloth-bnb-4bit
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.16.0

adapter_config.json

@@ -0,0 +1,43 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "unsloth/gemma-3-12b-it-unsloth-bnb-4bit",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "o_proj",
+    "k_proj",
+    "embed_tokens",
+    "q_proj",
+    "v_proj",
+    "lm_head",
+    "up_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bdc08d632c0b5d1cfa5eae09c45926e0b909c5515bc838803120a5043ff3dc7b
+size 5259164736

added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "<image_soft_token>": 262144
+}

chat_template.jinja

@@ -0,0 +1,47 @@
+{{ bos_token }}
+{%- if messages[0]['role'] == 'system' -%}
+    {%- if messages[0]['content'] is string -%}
+        {%- set first_user_prefix = messages[0]['content'] + '
+
+' -%}
+    {%- else -%}
+        {%- set first_user_prefix = messages[0]['content'][0]['text'] + '
+
+' -%}
+    {%- endif -%}
+    {%- set loop_messages = messages[1:] -%}
+{%- else -%}
+    {%- set first_user_prefix = "" -%}
+    {%- set loop_messages = messages -%}
+{%- endif -%}
+{%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
+    {%- if (message['role'] == 'assistant') -%}
+        {%- set role = "model" -%}
+    {%- else -%}
+        {%- set role = message['role'] -%}
+    {%- endif -%}
+    {{ '<start_of_turn>' + role + '
+' + (first_user_prefix if loop.first else "") }}
+    {%- if message['content'] is string -%}
+        {{ message['content'] | trim }}
+    {%- elif message['content'] is iterable -%}
+        {%- for item in message['content'] -%}
+            {%- if item['type'] == 'image' -%}
+                {{ '<start_of_image>' }}
+            {%- elif item['type'] == 'text' -%}
+                {{ item['text'] | trim }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{ raise_exception("Invalid content type") }}
+    {%- endif -%}
+    {{ '<end_of_turn>
+' }}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{'<start_of_turn>model
+'}}
+{%- endif -%}

optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa9f80553517c2f7981df1cd4a9f0551b495e6bf9c47d27e6ee13a1b12ed0e42
+size 359807025

preprocessor_config.json

@@ -0,0 +1,29 @@
+{
+  "do_convert_rgb": null,
+  "do_normalize": true,
+  "do_pan_and_scan": null,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_processor_type": "Gemma3ImageProcessor",
+  "image_seq_length": 256,
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "pan_and_scan_max_num_crops": null,
+  "pan_and_scan_min_crop_size": null,
+  "pan_and_scan_min_ratio_to_activate": null,
+  "processor_class": "Gemma3Processor",
+  "resample": 2,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 896,
+    "width": 896
+  }
+}

processor_config.json

@@ -0,0 +1,4 @@
+{
+  "image_seq_length": 256,
+  "processor_class": "Gemma3Processor"
+}

rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd66943074862a902069570042b244bc349cb5c55a3ac57ed4bc154b5c1c0cfa
+size 14645

scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e5d82d6c02c6f882a9f9d5f32cfd5152ddeee3a90fd413f4bd5b668feaf4783
+size 1465

special_tokens_map.json

@@ -0,0 +1,33 @@
+{
+  "boi_token": "<start_of_image>",
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eoi_token": "<end_of_image>",
+  "eos_token": {
+    "content": "<end_of_turn>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<image_soft_token>",
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4667f2089529e8e7657cfb6d1c19910ae71ff5f28aa7ab2ff2763330affad795
+size 33384568

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
+size 4689074

trainer_state.json

@@ -0,0 +1,1838 @@
+{
+  "best_global_step": 1200,
+  "best_metric": 1.0104354619979858,
+  "best_model_checkpoint": "outputs/checkpoint-1200",
+  "epoch": 0.6555585905490303,
+  "eval_steps": 300,
+  "global_step": 1200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0027314941272876263,
+      "grad_norm": 4.851426601409912,
+      "learning_rate": 6.521739130434783e-07,
+      "loss": 2.9623,
+      "step": 5
+    },
+    {
+      "epoch": 0.005462988254575253,
+      "grad_norm": 5.322866439819336,
+      "learning_rate": 1.4673913043478262e-06,
+      "loss": 2.8968,
+      "step": 10
+    },
+    {
+      "epoch": 0.00819448238186288,
+      "grad_norm": 4.726293563842773,
+      "learning_rate": 2.282608695652174e-06,
+      "loss": 3.0188,
+      "step": 15
+    },
+    {
+      "epoch": 0.010925976509150505,
+      "grad_norm": 4.048459529876709,
+      "learning_rate": 3.097826086956522e-06,
+      "loss": 2.8014,
+      "step": 20
+    },
+    {
+      "epoch": 0.013657470636438131,
+      "grad_norm": 4.118284225463867,
+      "learning_rate": 3.913043478260869e-06,
+      "loss": 2.7867,
+      "step": 25
+    },
+    {
+      "epoch": 0.01638896476372576,
+      "grad_norm": 4.78012752532959,
+      "learning_rate": 4.728260869565218e-06,
+      "loss": 2.7244,
+      "step": 30
+    },
+    {
+      "epoch": 0.019120458891013385,
+      "grad_norm": 3.870762586593628,
+      "learning_rate": 5.543478260869565e-06,
+      "loss": 2.7813,
+      "step": 35
+    },
+    {
+      "epoch": 0.02185195301830101,
+      "grad_norm": 3.015080451965332,
+      "learning_rate": 6.358695652173913e-06,
+      "loss": 2.4723,
+      "step": 40
+    },
+    {
+      "epoch": 0.024583447145588636,
+      "grad_norm": 2.9946231842041016,
+      "learning_rate": 7.173913043478261e-06,
+      "loss": 2.462,
+      "step": 45
+    },
+    {
+      "epoch": 0.027314941272876262,
+      "grad_norm": 2.191171884536743,
+      "learning_rate": 7.98913043478261e-06,
+      "loss": 2.3881,
+      "step": 50
+    },
+    {
+      "epoch": 0.030046435400163888,
+      "grad_norm": 2.354739189147949,
+      "learning_rate": 8.804347826086957e-06,
+      "loss": 2.1182,
+      "step": 55
+    },
+    {
+      "epoch": 0.03277792952745152,
+      "grad_norm": 1.6525721549987793,
+      "learning_rate": 9.619565217391304e-06,
+      "loss": 1.9386,
+      "step": 60
+    },
+    {
+      "epoch": 0.03550942365473914,
+      "grad_norm": 1.3781287670135498,
+      "learning_rate": 1.0434782608695653e-05,
+      "loss": 2.0235,
+      "step": 65
+    },
+    {
+      "epoch": 0.03824091778202677,
+      "grad_norm": 1.2135268449783325,
+      "learning_rate": 1.125e-05,
+      "loss": 1.8313,
+      "step": 70
+    },
+    {
+      "epoch": 0.040972411909314395,
+      "grad_norm": 0.9717159271240234,
+      "learning_rate": 1.2065217391304348e-05,
+      "loss": 1.9393,
+      "step": 75
+    },
+    {
+      "epoch": 0.04370390603660202,
+      "grad_norm": 1.1752846240997314,
+      "learning_rate": 1.2880434782608696e-05,
+      "loss": 1.9298,
+      "step": 80
+    },
+    {
+      "epoch": 0.04643540016388965,
+      "grad_norm": 0.9449806213378906,
+      "learning_rate": 1.3695652173913042e-05,
+      "loss": 1.823,
+      "step": 85
+    },
+    {
+      "epoch": 0.04916689429117727,
+      "grad_norm": 1.2805055379867554,
+      "learning_rate": 1.4510869565217392e-05,
+      "loss": 1.8492,
+      "step": 90
+    },
+    {
+      "epoch": 0.0518983884184649,
+      "grad_norm": 0.9438149333000183,
+      "learning_rate": 1.532608695652174e-05,
+      "loss": 1.663,
+      "step": 95
+    },
+    {
+      "epoch": 0.054629882545752524,
+      "grad_norm": 0.9939911961555481,
+      "learning_rate": 1.6141304347826086e-05,
+      "loss": 1.7269,
+      "step": 100
+    },
+    {
+      "epoch": 0.05736137667304015,
+      "grad_norm": 0.8572550415992737,
+      "learning_rate": 1.6956521739130433e-05,
+      "loss": 1.6393,
+      "step": 105
+    },
+    {
+      "epoch": 0.060092870800327776,
+      "grad_norm": 0.9384661912918091,
+      "learning_rate": 1.777173913043478e-05,
+      "loss": 1.6523,
+      "step": 110
+    },
+    {
+      "epoch": 0.06282436492761541,
+      "grad_norm": 0.8437032103538513,
+      "learning_rate": 1.858695652173913e-05,
+      "loss": 1.6622,
+      "step": 115
+    },
+    {
+      "epoch": 0.06555585905490303,
+      "grad_norm": 0.8985423445701599,
+      "learning_rate": 1.940217391304348e-05,
+      "loss": 1.5545,
+      "step": 120
+    },
+    {
+      "epoch": 0.06828735318219066,
+      "grad_norm": 0.9783689975738525,
+      "learning_rate": 2.0217391304347828e-05,
+      "loss": 1.5837,
+      "step": 125
+    },
+    {
+      "epoch": 0.07101884730947829,
+      "grad_norm": 0.77556312084198,
+      "learning_rate": 2.1032608695652175e-05,
+      "loss": 1.5758,
+      "step": 130
+    },
+    {
+      "epoch": 0.07375034143676591,
+      "grad_norm": 0.8036277294158936,
+      "learning_rate": 2.1847826086956523e-05,
+      "loss": 1.5933,
+      "step": 135
+    },
+    {
+      "epoch": 0.07648183556405354,
+      "grad_norm": 0.8047032952308655,
+      "learning_rate": 2.266304347826087e-05,
+      "loss": 1.4694,
+      "step": 140
+    },
+    {
+      "epoch": 0.07921332969134116,
+      "grad_norm": 0.8819392323493958,
+      "learning_rate": 2.347826086956522e-05,
+      "loss": 1.4516,
+      "step": 145
+    },
+    {
+      "epoch": 0.08194482381862879,
+      "grad_norm": 0.7637084722518921,
+      "learning_rate": 2.4293478260869566e-05,
+      "loss": 1.3716,
+      "step": 150
+    },
+    {
+      "epoch": 0.08467631794591642,
+      "grad_norm": 0.8421655893325806,
+      "learning_rate": 2.5108695652173914e-05,
+      "loss": 1.4497,
+      "step": 155
+    },
+    {
+      "epoch": 0.08740781207320404,
+      "grad_norm": 0.7615973949432373,
+      "learning_rate": 2.592391304347826e-05,
+      "loss": 1.4217,
+      "step": 160
+    },
+    {
+      "epoch": 0.09013930620049167,
+      "grad_norm": 0.8904805779457092,
+      "learning_rate": 2.673913043478261e-05,
+      "loss": 1.4529,
+      "step": 165
+    },
+    {
+      "epoch": 0.0928708003277793,
+      "grad_norm": 0.7236261367797852,
+      "learning_rate": 2.7554347826086957e-05,
+      "loss": 1.369,
+      "step": 170
+    },
+    {
+      "epoch": 0.09560229445506692,
+      "grad_norm": 0.7706332802772522,
+      "learning_rate": 2.8369565217391305e-05,
+      "loss": 1.3706,
+      "step": 175
+    },
+    {
+      "epoch": 0.09833378858235454,
+      "grad_norm": 0.7844731211662292,
+      "learning_rate": 2.9184782608695653e-05,
+      "loss": 1.4144,
+      "step": 180
+    },
+    {
+      "epoch": 0.10106528270964217,
+      "grad_norm": 0.872904360294342,
+      "learning_rate": 3e-05,
+      "loss": 1.2978,
+      "step": 185
+    },
+    {
+      "epoch": 0.1037967768369298,
+      "grad_norm": 0.7814828753471375,
+      "learning_rate": 2.9956871765382405e-05,
+      "loss": 1.3356,
+      "step": 190
+    },
+    {
+      "epoch": 0.10652827096421742,
+      "grad_norm": 0.9173154830932617,
+      "learning_rate": 2.9913743530764806e-05,
+      "loss": 1.3388,
+      "step": 195
+    },
+    {
+      "epoch": 0.10925976509150505,
+      "grad_norm": 0.8572229743003845,
+      "learning_rate": 2.987061529614721e-05,
+      "loss": 1.353,
+      "step": 200
+    },
+    {
+      "epoch": 0.11199125921879267,
+      "grad_norm": 0.7920759320259094,
+      "learning_rate": 2.9827487061529615e-05,
+      "loss": 1.305,
+      "step": 205
+    },
+    {
+      "epoch": 0.1147227533460803,
+      "grad_norm": 0.8120269775390625,
+      "learning_rate": 2.978435882691202e-05,
+      "loss": 1.2763,
+      "step": 210
+    },
+    {
+      "epoch": 0.11745424747336793,
+      "grad_norm": 0.7644383907318115,
+      "learning_rate": 2.974123059229442e-05,
+      "loss": 1.3108,
+      "step": 215
+    },
+    {
+      "epoch": 0.12018574160065555,
+      "grad_norm": 0.7263749241828918,
+      "learning_rate": 2.9698102357676825e-05,
+      "loss": 1.3364,
+      "step": 220
+    },
+    {
+      "epoch": 0.12291723572794319,
+      "grad_norm": 0.7868081331253052,
+      "learning_rate": 2.965497412305923e-05,
+      "loss": 1.2965,
+      "step": 225
+    },
+    {
+      "epoch": 0.12564872985523082,
+      "grad_norm": 0.7651657462120056,
+      "learning_rate": 2.9611845888441637e-05,
+      "loss": 1.3027,
+      "step": 230
+    },
+    {
+      "epoch": 0.12838022398251844,
+      "grad_norm": 0.7855558395385742,
+      "learning_rate": 2.9568717653824038e-05,
+      "loss": 1.3203,
+      "step": 235
+    },
+    {
+      "epoch": 0.13111171810980607,
+      "grad_norm": 0.7353278398513794,
+      "learning_rate": 2.9525589419206442e-05,
+      "loss": 1.1794,
+      "step": 240
+    },
+    {
+      "epoch": 0.1338432122370937,
+      "grad_norm": 0.9344907999038696,
+      "learning_rate": 2.9482461184588846e-05,
+      "loss": 1.2336,
+      "step": 245
+    },
+    {
+      "epoch": 0.13657470636438132,
+      "grad_norm": 0.7568232417106628,
+      "learning_rate": 2.943933294997125e-05,
+      "loss": 1.3051,
+      "step": 250
+    },
+    {
+      "epoch": 0.13930620049166895,
+      "grad_norm": 0.8257033228874207,
+      "learning_rate": 2.939620471535365e-05,
+      "loss": 1.2554,
+      "step": 255
+    },
+    {
+      "epoch": 0.14203769461895657,
+      "grad_norm": 0.7484506368637085,
+      "learning_rate": 2.9353076480736056e-05,
+      "loss": 1.2549,
+      "step": 260
+    },
+    {
+      "epoch": 0.1447691887462442,
+      "grad_norm": 0.7581287622451782,
+      "learning_rate": 2.930994824611846e-05,
+      "loss": 1.1332,
+      "step": 265
+    },
+    {
+      "epoch": 0.14750068287353182,
+      "grad_norm": 0.7606173753738403,
+      "learning_rate": 2.9266820011500865e-05,
+      "loss": 1.1799,
+      "step": 270
+    },
+    {
+      "epoch": 0.15023217700081945,
+      "grad_norm": 0.8215767741203308,
+      "learning_rate": 2.9223691776883266e-05,
+      "loss": 1.1879,
+      "step": 275
+    },
+    {
+      "epoch": 0.15296367112810708,
+      "grad_norm": 0.9438003897666931,
+      "learning_rate": 2.918056354226567e-05,
+      "loss": 1.1292,
+      "step": 280
+    },
+    {
+      "epoch": 0.1556951652553947,
+      "grad_norm": 0.6569656133651733,
+      "learning_rate": 2.9137435307648074e-05,
+      "loss": 1.1767,
+      "step": 285
+    },
+    {
+      "epoch": 0.15842665938268233,
+      "grad_norm": 0.8340663909912109,
+      "learning_rate": 2.909430707303048e-05,
+      "loss": 1.2064,
+      "step": 290
+    },
+    {
+      "epoch": 0.16115815350996995,
+      "grad_norm": 0.6959854364395142,
+      "learning_rate": 2.905117883841288e-05,
+      "loss": 1.1955,
+      "step": 295
+    },
+    {
+      "epoch": 0.16388964763725758,
+      "grad_norm": 0.7269774675369263,
+      "learning_rate": 2.9008050603795284e-05,
+      "loss": 1.1903,
+      "step": 300
+    },
+    {
+      "epoch": 0.16388964763725758,
+      "eval_loss": 1.2255185842514038,
+      "eval_runtime": 328.4188,
+      "eval_samples_per_second": 10.721,
+      "eval_steps_per_second": 2.683,
+      "step": 300
+    },
+    {
+      "epoch": 0.1666211417645452,
+      "grad_norm": 0.724984347820282,
+      "learning_rate": 2.896492236917769e-05,
+      "loss": 1.103,
+      "step": 305
+    },
+    {
+      "epoch": 0.16935263589183283,
+      "grad_norm": 0.7624502182006836,
+      "learning_rate": 2.8921794134560093e-05,
+      "loss": 1.1216,
+      "step": 310
+    },
+    {
+      "epoch": 0.17208413001912046,
+      "grad_norm": 0.6501736640930176,
+      "learning_rate": 2.8878665899942497e-05,
+      "loss": 1.0736,
+      "step": 315
+    },
+    {
+      "epoch": 0.17481562414640808,
+      "grad_norm": 0.7219209671020508,
+      "learning_rate": 2.88355376653249e-05,
+      "loss": 1.1018,
+      "step": 320
+    },
+    {
+      "epoch": 0.1775471182736957,
+      "grad_norm": 0.6269301176071167,
+      "learning_rate": 2.8792409430707306e-05,
+      "loss": 1.1242,
+      "step": 325
+    },
+    {
+      "epoch": 0.18027861240098333,
+      "grad_norm": 0.785416305065155,
+      "learning_rate": 2.8749281196089707e-05,
+      "loss": 1.0851,
+      "step": 330
+    },
+    {
+      "epoch": 0.18301010652827096,
+      "grad_norm": 0.7034997344017029,
+      "learning_rate": 2.870615296147211e-05,
+      "loss": 1.1384,
+      "step": 335
+    },
+    {
+      "epoch": 0.1857416006555586,
+      "grad_norm": 0.8468294739723206,
+      "learning_rate": 2.8663024726854516e-05,
+      "loss": 1.1302,
+      "step": 340
+    },
+    {
+      "epoch": 0.1884730947828462,
+      "grad_norm": 0.7820891737937927,
+      "learning_rate": 2.861989649223692e-05,
+      "loss": 1.0532,
+      "step": 345
+    },
+    {
+      "epoch": 0.19120458891013384,
+      "grad_norm": 0.8448447585105896,
+      "learning_rate": 2.857676825761932e-05,
+      "loss": 1.2118,
+      "step": 350
+    },
+    {
+      "epoch": 0.19393608303742146,
+      "grad_norm": 0.6977779269218445,
+      "learning_rate": 2.8533640023001725e-05,
+      "loss": 1.1829,
+      "step": 355
+    },
+    {
+      "epoch": 0.1966675771647091,
+      "grad_norm": 0.6859474778175354,
+      "learning_rate": 2.849051178838413e-05,
+      "loss": 0.9995,
+      "step": 360
+    },
+    {
+      "epoch": 0.19939907129199672,
+      "grad_norm": 0.7476114630699158,
+      "learning_rate": 2.8447383553766534e-05,
+      "loss": 1.0597,
+      "step": 365
+    },
+    {
+      "epoch": 0.20213056541928434,
+      "grad_norm": 0.7813637852668762,
+      "learning_rate": 2.8404255319148935e-05,
+      "loss": 1.1079,
+      "step": 370
+    },
+    {
+      "epoch": 0.20486205954657197,
+      "grad_norm": 0.8648521304130554,
+      "learning_rate": 2.836112708453134e-05,
+      "loss": 1.1522,
+      "step": 375
+    },
+    {
+      "epoch": 0.2075935536738596,
+      "grad_norm": 0.7016454339027405,
+      "learning_rate": 2.8317998849913744e-05,
+      "loss": 1.0281,
+      "step": 380
+    },
+    {
+      "epoch": 0.21032504780114722,
+      "grad_norm": 0.6951316595077515,
+      "learning_rate": 2.8274870615296148e-05,
+      "loss": 1.1146,
+      "step": 385
+    },
+    {
+      "epoch": 0.21305654192843484,
+      "grad_norm": 0.7843300700187683,
+      "learning_rate": 2.823174238067855e-05,
+      "loss": 1.1078,
+      "step": 390
+    },
+    {
+      "epoch": 0.21578803605572247,
+      "grad_norm": 0.7950036525726318,
+      "learning_rate": 2.8188614146060953e-05,
+      "loss": 1.0833,
+      "step": 395
+    },
+    {
+      "epoch": 0.2185195301830101,
+      "grad_norm": 0.7733429074287415,
+      "learning_rate": 2.814548591144336e-05,
+      "loss": 1.1077,
+      "step": 400
+    },
+    {
+      "epoch": 0.22125102431029772,
+      "grad_norm": 0.813205897808075,
+      "learning_rate": 2.8102357676825766e-05,
+      "loss": 1.1326,
+      "step": 405
+    },
+    {
+      "epoch": 0.22398251843758535,
+      "grad_norm": 0.7515183687210083,
+      "learning_rate": 2.8059229442208166e-05,
+      "loss": 1.0858,
+      "step": 410
+    },
+    {
+      "epoch": 0.22671401256487297,
+      "grad_norm": 0.775006890296936,
+      "learning_rate": 2.801610120759057e-05,
+      "loss": 1.1325,
+      "step": 415
+    },
+    {
+      "epoch": 0.2294455066921606,
+      "grad_norm": 0.7226970195770264,
+      "learning_rate": 2.7972972972972975e-05,
+      "loss": 1.113,
+      "step": 420
+    },
+    {
+      "epoch": 0.23217700081944823,
+      "grad_norm": 0.803119421005249,
+      "learning_rate": 2.792984473835538e-05,
+      "loss": 1.0701,
+      "step": 425
+    },
+    {
+      "epoch": 0.23490849494673585,
+      "grad_norm": 0.7977150678634644,
+      "learning_rate": 2.788671650373778e-05,
+      "loss": 1.1111,
+      "step": 430
+    },
+    {
+      "epoch": 0.23763998907402348,
+      "grad_norm": 0.7857878804206848,
+      "learning_rate": 2.7843588269120185e-05,
+      "loss": 1.0683,
+      "step": 435
+    },
+    {
+      "epoch": 0.2403714832013111,
+      "grad_norm": 0.7843796610832214,
+      "learning_rate": 2.780046003450259e-05,
+      "loss": 1.122,
+      "step": 440
+    },
+    {
+      "epoch": 0.24310297732859876,
+      "grad_norm": 0.7460730075836182,
+      "learning_rate": 2.775733179988499e-05,
+      "loss": 1.0579,
+      "step": 445
+    },
+    {
+      "epoch": 0.24583447145588638,
+      "grad_norm": 0.6824869513511658,
+      "learning_rate": 2.7714203565267395e-05,
+      "loss": 1.0787,
+      "step": 450
+    },
+    {
+      "epoch": 0.248565965583174,
+      "grad_norm": 0.6470029950141907,
+      "learning_rate": 2.76710753306498e-05,
+      "loss": 1.0975,
+      "step": 455
+    },
+    {
+      "epoch": 0.25129745971046163,
+      "grad_norm": 0.7347692847251892,
+      "learning_rate": 2.7627947096032203e-05,
+      "loss": 1.1854,
+      "step": 460
+    },
+    {
+      "epoch": 0.25402895383774926,
+      "grad_norm": 0.8426661491394043,
+      "learning_rate": 2.7584818861414604e-05,
+      "loss": 1.0489,
+      "step": 465
+    },
+    {
+      "epoch": 0.2567604479650369,
+      "grad_norm": 0.7579281330108643,
+      "learning_rate": 2.754169062679701e-05,
+      "loss": 1.0804,
+      "step": 470
+    },
+    {
+      "epoch": 0.2594919420923245,
+      "grad_norm": 0.8320958018302917,
+      "learning_rate": 2.7498562392179413e-05,
+      "loss": 1.1259,
+      "step": 475
+    },
+    {
+      "epoch": 0.26222343621961214,
+      "grad_norm": 0.8479028940200806,
+      "learning_rate": 2.7455434157561817e-05,
+      "loss": 1.0846,
+      "step": 480
+    },
+    {
+      "epoch": 0.26495493034689976,
+      "grad_norm": 0.7769616842269897,
+      "learning_rate": 2.7412305922944222e-05,
+      "loss": 1.0907,
+      "step": 485
+    },
+    {
+      "epoch": 0.2676864244741874,
+      "grad_norm": 0.6947383880615234,
+      "learning_rate": 2.7369177688326626e-05,
+      "loss": 1.0606,
+      "step": 490
+    },
+    {
+      "epoch": 0.270417918601475,
+      "grad_norm": 0.6838494539260864,
+      "learning_rate": 2.732604945370903e-05,
+      "loss": 1.0027,
+      "step": 495
+    },
+    {
+      "epoch": 0.27314941272876264,
+      "grad_norm": 0.7338347434997559,
+      "learning_rate": 2.7282921219091435e-05,
+      "loss": 1.0859,
+      "step": 500
+    },
+    {
+      "epoch": 0.27588090685605027,
+      "grad_norm": 0.7138042449951172,
+      "learning_rate": 2.7239792984473836e-05,
+      "loss": 1.0208,
+      "step": 505
+    },
+    {
+      "epoch": 0.2786124009833379,
+      "grad_norm": 0.8377683162689209,
+      "learning_rate": 2.719666474985624e-05,
+      "loss": 1.1176,
+      "step": 510
+    },
+    {
+      "epoch": 0.2813438951106255,
+      "grad_norm": 0.817639172077179,
+      "learning_rate": 2.7153536515238644e-05,
+      "loss": 1.1092,
+      "step": 515
+    },
+    {
+      "epoch": 0.28407538923791315,
+      "grad_norm": 0.7524867057800293,
+      "learning_rate": 2.711040828062105e-05,
+      "loss": 1.07,
+      "step": 520
+    },
+    {
+      "epoch": 0.28680688336520077,
+      "grad_norm": 0.7723881006240845,
+      "learning_rate": 2.706728004600345e-05,
+      "loss": 1.0319,
+      "step": 525
+    },
+    {
+      "epoch": 0.2895383774924884,
+      "grad_norm": 0.688042163848877,
+      "learning_rate": 2.7024151811385854e-05,
+      "loss": 1.0426,
+      "step": 530
+    },
+    {
+      "epoch": 0.292269871619776,
+      "grad_norm": 0.7268008589744568,
+      "learning_rate": 2.698102357676826e-05,
+      "loss": 1.0404,
+      "step": 535
+    },
+    {
+      "epoch": 0.29500136574706365,
+      "grad_norm": 0.6592392325401306,
+      "learning_rate": 2.6937895342150663e-05,
+      "loss": 1.0363,
+      "step": 540
+    },
+    {
+      "epoch": 0.2977328598743513,
+      "grad_norm": 0.694995641708374,
+      "learning_rate": 2.6894767107533064e-05,
+      "loss": 1.1075,
+      "step": 545
+    },
+    {
+      "epoch": 0.3004643540016389,
+      "grad_norm": 0.8423897624015808,
+      "learning_rate": 2.6851638872915468e-05,
+      "loss": 1.0985,
+      "step": 550
+    },
+    {
+      "epoch": 0.3031958481289265,
+      "grad_norm": 0.6766981482505798,
+      "learning_rate": 2.6808510638297873e-05,
+      "loss": 1.0216,
+      "step": 555
+    },
+    {
+      "epoch": 0.30592734225621415,
+      "grad_norm": 0.7330387830734253,
+      "learning_rate": 2.6765382403680274e-05,
+      "loss": 1.0111,
+      "step": 560
+    },
+    {
+      "epoch": 0.3086588363835018,
+      "grad_norm": 0.7680959105491638,
+      "learning_rate": 2.672225416906268e-05,
+      "loss": 1.0391,
+      "step": 565
+    },
+    {
+      "epoch": 0.3113903305107894,
+      "grad_norm": 0.8362131118774414,
+      "learning_rate": 2.6679125934445086e-05,
+      "loss": 1.0963,
+      "step": 570
+    },
+    {
+      "epoch": 0.31412182463807703,
+      "grad_norm": 0.9484680891036987,
+      "learning_rate": 2.663599769982749e-05,
+      "loss": 1.0853,
+      "step": 575
+    },
+    {
+      "epoch": 0.31685331876536466,
+      "grad_norm": 0.7578262090682983,
+      "learning_rate": 2.659286946520989e-05,
+      "loss": 1.0243,
+      "step": 580
+    },
+    {
+      "epoch": 0.3195848128926523,
+      "grad_norm": 0.7981600165367126,
+      "learning_rate": 2.6549741230592295e-05,
+      "loss": 1.0074,
+      "step": 585
+    },
+    {
+      "epoch": 0.3223163070199399,
+      "grad_norm": 0.7634698748588562,
+      "learning_rate": 2.65066129959747e-05,
+      "loss": 1.0407,
+      "step": 590
+    },
+    {
+      "epoch": 0.32504780114722753,
+      "grad_norm": 0.7340299487113953,
+      "learning_rate": 2.6463484761357104e-05,
+      "loss": 1.0205,
+      "step": 595
+    },
+    {
+      "epoch": 0.32777929527451516,
+      "grad_norm": 0.7297033667564392,
+      "learning_rate": 2.6420356526739505e-05,
+      "loss": 1.1023,
+      "step": 600
+    },
+    {
+      "epoch": 0.32777929527451516,
+      "eval_loss": 1.0993021726608276,
+      "eval_runtime": 306.2283,
+      "eval_samples_per_second": 11.498,
+      "eval_steps_per_second": 2.877,
+      "step": 600
+    },
+    {
+      "epoch": 0.3305107894018028,
+      "grad_norm": 0.645929217338562,
+      "learning_rate": 2.637722829212191e-05,
+      "loss": 0.9895,
+      "step": 605
+    },
+    {
+      "epoch": 0.3332422835290904,
+      "grad_norm": 0.7104042768478394,
+      "learning_rate": 2.6334100057504314e-05,
+      "loss": 1.0063,
+      "step": 610
+    },
+    {
+      "epoch": 0.33597377765637804,
+      "grad_norm": 0.7610509395599365,
+      "learning_rate": 2.6290971822886718e-05,
+      "loss": 0.9546,
+      "step": 615
+    },
+    {
+      "epoch": 0.33870527178366566,
+      "grad_norm": 0.7508594989776611,
+      "learning_rate": 2.624784358826912e-05,
+      "loss": 0.9908,
+      "step": 620
+    },
+    {
+      "epoch": 0.3414367659109533,
+      "grad_norm": 0.7074321508407593,
+      "learning_rate": 2.6204715353651523e-05,
+      "loss": 1.0301,
+      "step": 625
+    },
+    {
+      "epoch": 0.3441682600382409,
+      "grad_norm": 0.7267704010009766,
+      "learning_rate": 2.6161587119033928e-05,
+      "loss": 1.0927,
+      "step": 630
+    },
+    {
+      "epoch": 0.34689975416552854,
+      "grad_norm": 0.6761760711669922,
+      "learning_rate": 2.6118458884416332e-05,
+      "loss": 0.9735,
+      "step": 635
+    },
+    {
+      "epoch": 0.34963124829281617,
+      "grad_norm": 0.7219847440719604,
+      "learning_rate": 2.6075330649798733e-05,
+      "loss": 1.0233,
+      "step": 640
+    },
+    {
+      "epoch": 0.3523627424201038,
+      "grad_norm": 0.753565788269043,
+      "learning_rate": 2.6032202415181138e-05,
+      "loss": 1.0658,
+      "step": 645
+    },
+    {
+      "epoch": 0.3550942365473914,
+      "grad_norm": 0.7490048408508301,
+      "learning_rate": 2.5989074180563545e-05,
+      "loss": 1.0668,
+      "step": 650
+    },
+    {
+      "epoch": 0.35782573067467904,
+      "grad_norm": 0.7493592500686646,
+      "learning_rate": 2.594594594594595e-05,
+      "loss": 0.9936,
+      "step": 655
+    },
+    {
+      "epoch": 0.36055722480196667,
+      "grad_norm": 0.7734257578849792,
+      "learning_rate": 2.590281771132835e-05,
+      "loss": 0.9868,
+      "step": 660
+    },
+    {
+      "epoch": 0.3632887189292543,
+      "grad_norm": 0.7188918590545654,
+      "learning_rate": 2.5859689476710755e-05,
+      "loss": 0.9771,
+      "step": 665
+    },
+    {
+      "epoch": 0.3660202130565419,
+      "grad_norm": 0.7556160688400269,
+      "learning_rate": 2.581656124209316e-05,
+      "loss": 1.0392,
+      "step": 670
+    },
+    {
+      "epoch": 0.36875170718382955,
+      "grad_norm": 0.5352792739868164,
+      "learning_rate": 2.577343300747556e-05,
+      "loss": 0.941,
+      "step": 675
+    },
+    {
+      "epoch": 0.3714832013111172,
+      "grad_norm": 0.6794752478599548,
+      "learning_rate": 2.5730304772857965e-05,
+      "loss": 1.078,
+      "step": 680
+    },
+    {
+      "epoch": 0.3742146954384048,
+      "grad_norm": 0.7562423348426819,
+      "learning_rate": 2.568717653824037e-05,
+      "loss": 0.9774,
+      "step": 685
+    },
+    {
+      "epoch": 0.3769461895656924,
+      "grad_norm": 0.7022319436073303,
+      "learning_rate": 2.5644048303622773e-05,
+      "loss": 0.9832,
+      "step": 690
+    },
+    {
+      "epoch": 0.37967768369298005,
+      "grad_norm": 0.771272599697113,
+      "learning_rate": 2.5600920069005174e-05,
+      "loss": 1.0179,
+      "step": 695
+    },
+    {
+      "epoch": 0.3824091778202677,
+      "grad_norm": 0.6900454163551331,
+      "learning_rate": 2.555779183438758e-05,
+      "loss": 0.9452,
+      "step": 700
+    },
+    {
+      "epoch": 0.3851406719475553,
+      "grad_norm": 0.7504772543907166,
+      "learning_rate": 2.5514663599769983e-05,
+      "loss": 0.9733,
+      "step": 705
+    },
+    {
+      "epoch": 0.38787216607484293,
+      "grad_norm": 0.8242804408073425,
+      "learning_rate": 2.5471535365152387e-05,
+      "loss": 1.0468,
+      "step": 710
+    },
+    {
+      "epoch": 0.39060366020213055,
+      "grad_norm": 0.7705220580101013,
+      "learning_rate": 2.542840713053479e-05,
+      "loss": 1.0101,
+      "step": 715
+    },
+    {
+      "epoch": 0.3933351543294182,
+      "grad_norm": 0.7174106240272522,
+      "learning_rate": 2.5385278895917193e-05,
+      "loss": 0.9818,
+      "step": 720
+    },
+    {
+      "epoch": 0.3960666484567058,
+      "grad_norm": 0.7383962273597717,
+      "learning_rate": 2.5342150661299597e-05,
+      "loss": 1.0289,
+      "step": 725
+    },
+    {
+      "epoch": 0.39879814258399343,
+      "grad_norm": 0.6323188543319702,
+      "learning_rate": 2.5299022426682e-05,
+      "loss": 0.9597,
+      "step": 730
+    },
+    {
+      "epoch": 0.40152963671128106,
+      "grad_norm": 0.825609028339386,
+      "learning_rate": 2.5255894192064406e-05,
+      "loss": 0.978,
+      "step": 735
+    },
+    {
+      "epoch": 0.4042611308385687,
+      "grad_norm": 0.7169507741928101,
+      "learning_rate": 2.521276595744681e-05,
+      "loss": 0.9965,
+      "step": 740
+    },
+    {
+      "epoch": 0.4069926249658563,
+      "grad_norm": 0.7201241254806519,
+      "learning_rate": 2.5169637722829215e-05,
+      "loss": 1.0158,
+      "step": 745
+    },
+    {
+      "epoch": 0.40972411909314393,
+      "grad_norm": 0.7305246591567993,
+      "learning_rate": 2.512650948821162e-05,
+      "loss": 1.0034,
+      "step": 750
+    },
+    {
+      "epoch": 0.41245561322043156,
+      "grad_norm": 0.66969895362854,
+      "learning_rate": 2.508338125359402e-05,
+      "loss": 1.0505,
+      "step": 755
+    },
+    {
+      "epoch": 0.4151871073477192,
+      "grad_norm": 0.6087076663970947,
+      "learning_rate": 2.5040253018976424e-05,
+      "loss": 0.9876,
+      "step": 760
+    },
+    {
+      "epoch": 0.4179186014750068,
+      "grad_norm": 0.740016520023346,
+      "learning_rate": 2.499712478435883e-05,
+      "loss": 1.0002,
+      "step": 765
+    },
+    {
+      "epoch": 0.42065009560229444,
+      "grad_norm": 0.6993871331214905,
+      "learning_rate": 2.4953996549741233e-05,
+      "loss": 0.9828,
+      "step": 770
+    },
+    {
+      "epoch": 0.42338158972958206,
+      "grad_norm": 0.6651398539543152,
+      "learning_rate": 2.4910868315123634e-05,
+      "loss": 1.0651,
+      "step": 775
+    },
+    {
+      "epoch": 0.4261130838568697,
+      "grad_norm": 0.7585716247558594,
+      "learning_rate": 2.4867740080506038e-05,
+      "loss": 1.0542,
+      "step": 780
+    },
+    {
+      "epoch": 0.4288445779841573,
+      "grad_norm": 0.8134998679161072,
+      "learning_rate": 2.4824611845888443e-05,
+      "loss": 0.9729,
+      "step": 785
+    },
+    {
+      "epoch": 0.43157607211144494,
+      "grad_norm": 0.784631073474884,
+      "learning_rate": 2.4781483611270847e-05,
+      "loss": 0.9971,
+      "step": 790
+    },
+    {
+      "epoch": 0.43430756623873257,
+      "grad_norm": 0.8132376670837402,
+      "learning_rate": 2.4738355376653248e-05,
+      "loss": 0.9859,
+      "step": 795
+    },
+    {
+      "epoch": 0.4370390603660202,
+      "grad_norm": 0.6785438656806946,
+      "learning_rate": 2.4695227142035652e-05,
+      "loss": 0.9933,
+      "step": 800
+    },
+    {
+      "epoch": 0.4397705544933078,
+      "grad_norm": 0.7485066056251526,
+      "learning_rate": 2.4652098907418057e-05,
+      "loss": 1.0549,
+      "step": 805
+    },
+    {
+      "epoch": 0.44250204862059545,
+      "grad_norm": 0.626559853553772,
+      "learning_rate": 2.4608970672800458e-05,
+      "loss": 0.8987,
+      "step": 810
+    },
+    {
+      "epoch": 0.44523354274788307,
+      "grad_norm": 0.6525225639343262,
+      "learning_rate": 2.4565842438182862e-05,
+      "loss": 1.0055,
+      "step": 815
+    },
+    {
+      "epoch": 0.4479650368751707,
+      "grad_norm": 0.723646879196167,
+      "learning_rate": 2.452271420356527e-05,
+      "loss": 0.9561,
+      "step": 820
+    },
+    {
+      "epoch": 0.4506965310024583,
+      "grad_norm": 0.6696566343307495,
+      "learning_rate": 2.4479585968947674e-05,
+      "loss": 0.9626,
+      "step": 825
+    },
+    {
+      "epoch": 0.45342802512974595,
+      "grad_norm": 0.6937575936317444,
+      "learning_rate": 2.4436457734330075e-05,
+      "loss": 0.9679,
+      "step": 830
+    },
+    {
+      "epoch": 0.4561595192570336,
+      "grad_norm": 0.7571921944618225,
+      "learning_rate": 2.439332949971248e-05,
+      "loss": 1.0327,
+      "step": 835
+    },
+    {
+      "epoch": 0.4588910133843212,
+      "grad_norm": 0.7212653160095215,
+      "learning_rate": 2.4350201265094884e-05,
+      "loss": 0.9714,
+      "step": 840
+    },
+    {
+      "epoch": 0.4616225075116088,
+      "grad_norm": 0.6409651041030884,
+      "learning_rate": 2.4307073030477288e-05,
+      "loss": 1.0007,
+      "step": 845
+    },
+    {
+      "epoch": 0.46435400163889645,
+      "grad_norm": 0.7958700656890869,
+      "learning_rate": 2.426394479585969e-05,
+      "loss": 1.0335,
+      "step": 850
+    },
+    {
+      "epoch": 0.4670854957661841,
+      "grad_norm": 0.7226616740226746,
+      "learning_rate": 2.4220816561242094e-05,
+      "loss": 0.982,
+      "step": 855
+    },
+    {
+      "epoch": 0.4698169898934717,
+      "grad_norm": 0.6726953387260437,
+      "learning_rate": 2.4177688326624498e-05,
+      "loss": 0.9823,
+      "step": 860
+    },
+    {
+      "epoch": 0.47254848402075933,
+      "grad_norm": 0.7555478811264038,
+      "learning_rate": 2.4134560092006902e-05,
+      "loss": 0.9672,
+      "step": 865
+    },
+    {
+      "epoch": 0.47527997814804696,
+      "grad_norm": 0.8764699697494507,
+      "learning_rate": 2.4091431857389303e-05,
+      "loss": 0.9728,
+      "step": 870
+    },
+    {
+      "epoch": 0.4780114722753346,
+      "grad_norm": 0.8067215085029602,
+      "learning_rate": 2.4048303622771708e-05,
+      "loss": 1.0206,
+      "step": 875
+    },
+    {
+      "epoch": 0.4807429664026222,
+      "grad_norm": 0.7664445638656616,
+      "learning_rate": 2.4005175388154112e-05,
+      "loss": 0.9477,
+      "step": 880
+    },
+    {
+      "epoch": 0.48347446052990983,
+      "grad_norm": 0.7092654705047607,
+      "learning_rate": 2.3962047153536516e-05,
+      "loss": 1.0208,
+      "step": 885
+    },
+    {
+      "epoch": 0.4862059546571975,
+      "grad_norm": 0.6885917782783508,
+      "learning_rate": 2.3918918918918917e-05,
+      "loss": 1.0035,
+      "step": 890
+    },
+    {
+      "epoch": 0.48893744878448514,
+      "grad_norm": 0.7531325221061707,
+      "learning_rate": 2.387579068430132e-05,
+      "loss": 0.9716,
+      "step": 895
+    },
+    {
+      "epoch": 0.49166894291177277,
+      "grad_norm": 0.7733385562896729,
+      "learning_rate": 2.3832662449683726e-05,
+      "loss": 1.0056,
+      "step": 900
+    },
+    {
+      "epoch": 0.49166894291177277,
+      "eval_loss": 1.0448055267333984,
+      "eval_runtime": 307.1158,
+      "eval_samples_per_second": 11.465,
+      "eval_steps_per_second": 2.869,
+      "step": 900
+    },
+    {
+      "epoch": 0.4944004370390604,
+      "grad_norm": 0.8963056802749634,
+      "learning_rate": 2.3789534215066134e-05,
+      "loss": 0.9844,
+      "step": 905
+    },
+    {
+      "epoch": 0.497131931166348,
+      "grad_norm": 0.820163905620575,
+      "learning_rate": 2.3746405980448535e-05,
+      "loss": 1.0478,
+      "step": 910
+    },
+    {
+      "epoch": 0.49986342529363564,
+      "grad_norm": 0.6968187689781189,
+      "learning_rate": 2.370327774583094e-05,
+      "loss": 0.9435,
+      "step": 915
+    },
+    {
+      "epoch": 0.5025949194209233,
+      "grad_norm": 0.7683607339859009,
+      "learning_rate": 2.3660149511213343e-05,
+      "loss": 1.0467,
+      "step": 920
+    },
+    {
+      "epoch": 0.5053264135482108,
+      "grad_norm": 0.8514032959938049,
+      "learning_rate": 2.3617021276595744e-05,
+      "loss": 1.0109,
+      "step": 925
+    },
+    {
+      "epoch": 0.5080579076754985,
+      "grad_norm": 0.7920552492141724,
+      "learning_rate": 2.357389304197815e-05,
+      "loss": 0.9999,
+      "step": 930
+    },
+    {
+      "epoch": 0.5107894018027861,
+      "grad_norm": 0.6363257765769958,
+      "learning_rate": 2.3530764807360553e-05,
+      "loss": 0.963,
+      "step": 935
+    },
+    {
+      "epoch": 0.5135208959300738,
+      "grad_norm": 0.7315125465393066,
+      "learning_rate": 2.3487636572742957e-05,
+      "loss": 0.9841,
+      "step": 940
+    },
+    {
+      "epoch": 0.5162523900573613,
+      "grad_norm": 0.756065845489502,
+      "learning_rate": 2.344450833812536e-05,
+      "loss": 0.9945,
+      "step": 945
+    },
+    {
+      "epoch": 0.518983884184649,
+      "grad_norm": 0.7601234912872314,
+      "learning_rate": 2.3401380103507763e-05,
+      "loss": 0.9914,
+      "step": 950
+    },
+    {
+      "epoch": 0.5217153783119366,
+      "grad_norm": 0.7508507370948792,
+      "learning_rate": 2.3358251868890167e-05,
+      "loss": 0.9712,
+      "step": 955
+    },
+    {
+      "epoch": 0.5244468724392243,
+      "grad_norm": 0.768269419670105,
+      "learning_rate": 2.331512363427257e-05,
+      "loss": 1.0563,
+      "step": 960
+    },
+    {
+      "epoch": 0.5271783665665118,
+      "grad_norm": 0.6745092868804932,
+      "learning_rate": 2.3271995399654973e-05,
+      "loss": 0.9691,
+      "step": 965
+    },
+    {
+      "epoch": 0.5299098606937995,
+      "grad_norm": 0.7175255417823792,
+      "learning_rate": 2.3228867165037377e-05,
+      "loss": 0.9945,
+      "step": 970
+    },
+    {
+      "epoch": 0.5326413548210871,
+      "grad_norm": 0.7807250022888184,
+      "learning_rate": 2.318573893041978e-05,
+      "loss": 0.9538,
+      "step": 975
+    },
+    {
+      "epoch": 0.5353728489483748,
+      "grad_norm": 0.6616771221160889,
+      "learning_rate": 2.3142610695802186e-05,
+      "loss": 0.9973,
+      "step": 980
+    },
+    {
+      "epoch": 0.5381043430756624,
+      "grad_norm": 0.7407675981521606,
+      "learning_rate": 2.3099482461184587e-05,
+      "loss": 1.0041,
+      "step": 985
+    },
+    {
+      "epoch": 0.54083583720295,
+      "grad_norm": 0.7497495412826538,
+      "learning_rate": 2.3056354226566994e-05,
+      "loss": 0.9286,
+      "step": 990
+    },
+    {
+      "epoch": 0.5435673313302376,
+      "grad_norm": 0.6467530727386475,
+      "learning_rate": 2.30132259919494e-05,
+      "loss": 0.9433,
+      "step": 995
+    },
+    {
+      "epoch": 0.5462988254575253,
+      "grad_norm": 0.6888414621353149,
+      "learning_rate": 2.2970097757331803e-05,
+      "loss": 0.9507,
+      "step": 1000
+    },
+    {
+      "epoch": 0.5490303195848129,
+      "grad_norm": 0.8949408531188965,
+      "learning_rate": 2.2926969522714204e-05,
+      "loss": 0.9643,
+      "step": 1005
+    },
+    {
+      "epoch": 0.5517618137121005,
+      "grad_norm": 0.7237836122512817,
+      "learning_rate": 2.288384128809661e-05,
+      "loss": 0.917,
+      "step": 1010
+    },
+    {
+      "epoch": 0.5544933078393881,
+      "grad_norm": 0.6691592931747437,
+      "learning_rate": 2.2840713053479013e-05,
+      "loss": 0.999,
+      "step": 1015
+    },
+    {
+      "epoch": 0.5572248019666758,
+      "grad_norm": 0.7884166836738586,
+      "learning_rate": 2.2797584818861417e-05,
+      "loss": 0.9466,
+      "step": 1020
+    },
+    {
+      "epoch": 0.5599562960939634,
+      "grad_norm": 0.7341962456703186,
+      "learning_rate": 2.2754456584243818e-05,
+      "loss": 0.9567,
+      "step": 1025
+    },
+    {
+      "epoch": 0.562687790221251,
+      "grad_norm": 0.6674230098724365,
+      "learning_rate": 2.2711328349626222e-05,
+      "loss": 0.9487,
+      "step": 1030
+    },
+    {
+      "epoch": 0.5654192843485386,
+      "grad_norm": 0.6951179504394531,
+      "learning_rate": 2.2668200115008627e-05,
+      "loss": 0.9885,
+      "step": 1035
+    },
+    {
+      "epoch": 0.5681507784758263,
+      "grad_norm": 0.7464187741279602,
+      "learning_rate": 2.262507188039103e-05,
+      "loss": 0.9503,
+      "step": 1040
+    },
+    {
+      "epoch": 0.5708822726031139,
+      "grad_norm": 0.69831383228302,
+      "learning_rate": 2.2581943645773432e-05,
+      "loss": 0.9567,
+      "step": 1045
+    },
+    {
+      "epoch": 0.5736137667304015,
+      "grad_norm": 0.6780604720115662,
+      "learning_rate": 2.2538815411155836e-05,
+      "loss": 0.975,
+      "step": 1050
+    },
+    {
+      "epoch": 0.5763452608576891,
+      "grad_norm": 0.7216975688934326,
+      "learning_rate": 2.249568717653824e-05,
+      "loss": 0.9948,
+      "step": 1055
+    },
+    {
+      "epoch": 0.5790767549849768,
+      "grad_norm": 0.7621845006942749,
+      "learning_rate": 2.2452558941920642e-05,
+      "loss": 0.9995,
+      "step": 1060
+    },
+    {
+      "epoch": 0.5818082491122644,
+      "grad_norm": 0.7196091413497925,
+      "learning_rate": 2.2409430707303046e-05,
+      "loss": 1.0007,
+      "step": 1065
+    },
+    {
+      "epoch": 0.584539743239552,
+      "grad_norm": 0.6992865204811096,
+      "learning_rate": 2.236630247268545e-05,
+      "loss": 0.9363,
+      "step": 1070
+    },
+    {
+      "epoch": 0.5872712373668396,
+      "grad_norm": 0.6687476634979248,
+      "learning_rate": 2.2323174238067858e-05,
+      "loss": 1.0072,
+      "step": 1075
+    },
+    {
+      "epoch": 0.5900027314941273,
+      "grad_norm": 0.7828387022018433,
+      "learning_rate": 2.228004600345026e-05,
+      "loss": 0.994,
+      "step": 1080
+    },
+    {
+      "epoch": 0.5927342256214149,
+      "grad_norm": 0.7133594751358032,
+      "learning_rate": 2.2236917768832664e-05,
+      "loss": 0.8923,
+      "step": 1085
+    },
+    {
+      "epoch": 0.5954657197487025,
+      "grad_norm": 0.7610213160514832,
+      "learning_rate": 2.2193789534215068e-05,
+      "loss": 0.9855,
+      "step": 1090
+    },
+    {
+      "epoch": 0.5981972138759901,
+      "grad_norm": 0.8042429089546204,
+      "learning_rate": 2.2150661299597472e-05,
+      "loss": 0.8981,
+      "step": 1095
+    },
+    {
+      "epoch": 0.6009287080032778,
+      "grad_norm": 0.7705995440483093,
+      "learning_rate": 2.2107533064979873e-05,
+      "loss": 1.0007,
+      "step": 1100
+    },
+    {
+      "epoch": 0.6036602021305654,
+      "grad_norm": 0.8342954516410828,
+      "learning_rate": 2.2064404830362278e-05,
+      "loss": 0.9614,
+      "step": 1105
+    },
+    {
+      "epoch": 0.606391696257853,
+      "grad_norm": 0.7102698087692261,
+      "learning_rate": 2.2021276595744682e-05,
+      "loss": 0.9482,
+      "step": 1110
+    },
+    {
+      "epoch": 0.6091231903851406,
+      "grad_norm": 0.686964750289917,
+      "learning_rate": 2.1978148361127086e-05,
+      "loss": 0.9045,
+      "step": 1115
+    },
+    {
+      "epoch": 0.6118546845124283,
+      "grad_norm": 0.8660001158714294,
+      "learning_rate": 2.1935020126509487e-05,
+      "loss": 0.9551,
+      "step": 1120
+    },
+    {
+      "epoch": 0.6145861786397159,
+      "grad_norm": 0.689612090587616,
+      "learning_rate": 2.1891891891891892e-05,
+      "loss": 0.9795,
+      "step": 1125
+    },
+    {
+      "epoch": 0.6173176727670036,
+      "grad_norm": 0.7324284315109253,
+      "learning_rate": 2.1848763657274296e-05,
+      "loss": 0.9707,
+      "step": 1130
+    },
+    {
+      "epoch": 0.6200491668942911,
+      "grad_norm": 0.7044214010238647,
+      "learning_rate": 2.18056354226567e-05,
+      "loss": 0.9864,
+      "step": 1135
+    },
+    {
+      "epoch": 0.6227806610215788,
+      "grad_norm": 0.6462258696556091,
+      "learning_rate": 2.17625071880391e-05,
+      "loss": 0.9469,
+      "step": 1140
+    },
+    {
+      "epoch": 0.6255121551488664,
+      "grad_norm": 0.7109295129776001,
+      "learning_rate": 2.1719378953421506e-05,
+      "loss": 0.8984,
+      "step": 1145
+    },
+    {
+      "epoch": 0.6282436492761541,
+      "grad_norm": 0.5951446890830994,
+      "learning_rate": 2.167625071880391e-05,
+      "loss": 0.9383,
+      "step": 1150
+    },
+    {
+      "epoch": 0.6309751434034416,
+      "grad_norm": 0.7271871566772461,
+      "learning_rate": 2.1633122484186314e-05,
+      "loss": 0.9713,
+      "step": 1155
+    },
+    {
+      "epoch": 0.6337066375307293,
+      "grad_norm": 0.6672769784927368,
+      "learning_rate": 2.158999424956872e-05,
+      "loss": 0.9567,
+      "step": 1160
+    },
+    {
+      "epoch": 0.6364381316580169,
+      "grad_norm": 0.6343885064125061,
+      "learning_rate": 2.1546866014951123e-05,
+      "loss": 0.9575,
+      "step": 1165
+    },
+    {
+      "epoch": 0.6391696257853046,
+      "grad_norm": 0.6709259748458862,
+      "learning_rate": 2.1503737780333528e-05,
+      "loss": 0.9778,
+      "step": 1170
+    },
+    {
+      "epoch": 0.6419011199125921,
+      "grad_norm": 0.7463548183441162,
+      "learning_rate": 2.146060954571593e-05,
+      "loss": 0.9363,
+      "step": 1175
+    },
+    {
+      "epoch": 0.6446326140398798,
+      "grad_norm": 0.7336212396621704,
+      "learning_rate": 2.1417481311098333e-05,
+      "loss": 0.9984,
+      "step": 1180
+    },
+    {
+      "epoch": 0.6473641081671675,
+      "grad_norm": 0.7141072154045105,
+      "learning_rate": 2.1374353076480737e-05,
+      "loss": 0.959,
+      "step": 1185
+    },
+    {
+      "epoch": 0.6500956022944551,
+      "grad_norm": 0.7275980710983276,
+      "learning_rate": 2.133122484186314e-05,
+      "loss": 1.0157,
+      "step": 1190
+    },
+    {
+      "epoch": 0.6528270964217427,
+      "grad_norm": 0.7292263507843018,
+      "learning_rate": 2.1288096607245543e-05,
+      "loss": 0.9455,
+      "step": 1195
+    },
+    {
+      "epoch": 0.6555585905490303,
+      "grad_norm": 0.7778198719024658,
+      "learning_rate": 2.1244968372627947e-05,
+      "loss": 0.9972,
+      "step": 1200
+    },
+    {
+      "epoch": 0.6555585905490303,
+      "eval_loss": 1.0104354619979858,
+      "eval_runtime": 306.0261,
+      "eval_samples_per_second": 11.506,
+      "eval_steps_per_second": 2.879,
+      "step": 1200
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 3662,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 300,
+  "stateful_callbacks": {
+    "EarlyStoppingCallback": [
+      {
+        "args": {
+          "early_stopping_patience": 8,
+          "early_stopping_threshold": 0.01
+        },
+        "attributes": {
+          "early_stopping_patience_counter": 0
+        }
+      },
+      {
+        "args": {
+          "early_stopping_patience": 8,
+          "early_stopping_threshold": 0.01
+        },
+        "attributes": {
+          "early_stopping_patience_counter": 0
+        }
+      },
+      {
+        "args": {
+          "early_stopping_patience": 8,
+          "early_stopping_threshold": 0.01
+        },
+        "attributes": {
+          "early_stopping_patience_counter": 0
+        }
+      },
+      {
+        "args": {
+          "early_stopping_patience": 8,
+          "early_stopping_threshold": 0.01
+        },
+        "attributes": {
+          "early_stopping_patience_counter": 0
+        }
+      },
+      {
+        "args": {
+          "early_stopping_patience": 8,
+          "early_stopping_threshold": 0.01
+        },
+        "attributes": {
+          "early_stopping_patience_counter": 0
+        }
+      },
+      {
+        "args": {
+          "early_stopping_patience": 8,
+          "early_stopping_threshold": 0.01
+        },
+        "attributes": {
+          "early_stopping_patience_counter": 0
+        }
+      },
+      {
+        "args": {
+          "early_stopping_patience": 8,
+          "early_stopping_threshold": 0.01
+        },
+        "attributes": {
+          "early_stopping_patience_counter": 0
+        }
+      },
+      {
+        "args": {
+          "early_stopping_patience": 8,
+          "early_stopping_threshold": 0.01
+        },
+        "attributes": {
+          "early_stopping_patience_counter": 0
+        }
+      },
+      {
+        "args": {
+          "early_stopping_patience": 8,
+          "early_stopping_threshold": 0.01
+        },
+        "attributes": {
+          "early_stopping_patience_counter": 0
+        }
+      },
+      {
+        "args": {
+          "early_stopping_patience": 8,
+          "early_stopping_threshold": 0.01
+        },
+        "attributes": {
+          "early_stopping_patience_counter": 0
+        }
+      }
+    ],
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.922653197624351e+17,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0392540a753fe2703a66afdb90bd7f667f52b4c4990847d8cfa6d1bb9eb8d6e1
+size 6161