BowLingual 12B v2.1 raw training output

.gitattributes

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+best-lora/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-1400/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-1425/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-950/tokenizer.json filter=lfs diff=lfs merge=lfs -text

DATASET_SHA256.txt

@@ -0,0 +1 @@
+154ccb8fb7abe1ffd8494a083dbe1e77c8ca410878e2ad01daf1f7e99b8cf01e  /app/data/dataset.jsonl

RC_SELECTION.tsv

@@ -0,0 +1,3 @@
+1	950	0.5508978963	2.0000000000	/output/checkpoint-950
+2	1400	0.6027438641	2.9473684211	/output/checkpoint-1400
+3	1425	0.6031771302	3.0000000000	/output/checkpoint-1425

README.md

@@ -0,0 +1,59 @@
+---
+base_model: unsloth/gemma-4-12b-it
+library_name: transformers
+model_name: output
+tags:
+- generated_from_trainer
+- sft
+- trl
+- unsloth
+licence: license
+---
+
+# Model Card for output
+
+This model is a fine-tuned version of [unsloth/gemma-4-12b-it](https://huggingface.co/unsloth/gemma-4-12b-it).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+ 
+
+
+This model was trained with SFT.
+
+### Framework versions
+
+- TRL: 0.24.0
+- Transformers: 5.10.2
+- Pytorch: 2.11.0+cu130
+- Datasets: 4.3.0
+- Tokenizers: 0.22.2
+
+## Citations
+
+
+
+Cite TRL as:
+    
+```bibtex
+@misc{vonwerra2022trl,
+	title        = {{TRL: Transformer Reinforcement Learning}},
+	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
+	year         = 2020,
+	journal      = {GitHub repository},
+	publisher    = {GitHub},
+	howpublished = {\url{https://github.com/huggingface/trl}}
+}
+```

best-lora/README.md

@@ -0,0 +1,210 @@
+---
+base_model: unsloth/gemma-4-12b-it
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/gemma-4-12b-it
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

best-lora/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Gemma4UnifiedForConditionalGeneration",
+    "parent_library": "transformers.models.gemma4_unified.modeling_gemma4_unified",
+    "unsloth_fixed": true
+  },
+  "base_model_name_or_path": "unsloth/gemma-4-12b-it",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": "(?:.*?(?:language|text).*?(?:self_attn|attention|attn|mixer|mlp|feed_forward|ffn|dense|mixer).*?(?:q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj|embedding_projection))|(?:\\bmodel\\.layers\\.[\\d]{1,}\\.(?:self_attn|attention|attn|mixer|mlp|feed_forward|ffn|dense|mixer)\\.(?:(?:q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj|embedding_projection)))",
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

best-lora/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f05b624df3aa0d19a9c5bb1f9413057987f10a140fd822057666c011c43bdc6
+size 262373216

best-lora/chat_template.jinja

@@ -0,0 +1,70 @@
+{{ bos_token }}{%- macro strip_thinking(text) -%}
+    {%- set ns = namespace(result='') -%}
+    {%- for part in text.split('<channel|>') -%}
+        {%- if '<|channel>' in part -%}
+            {%- set ns.result = ns.result + part.split('<|channel>')[0] -%}
+        {%- else -%}
+            {%- set ns.result = ns.result + part -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- ns.result | trim -}}
+{%- endmacro -%}
+{%- set thinking = enable_thinking is defined and enable_thinking -%}
+{%- set loop_messages = messages -%}
+{%- if messages[0]['role'] in ['system', 'developer'] or thinking -%}
+    {{ '<|turn>system
+' }}
+    {%- if thinking -%}
+        {{ '<|think|>
+' }}
+    {%- endif -%}
+    {%- if messages[0]['role'] in ['system', 'developer'] -%}
+        {{ messages[0]['content'] | trim }}
+        {%- set loop_messages = messages[1:] -%}
+    {%- endif -%}
+    {{ '<turn|>
+' }}
+{%- endif -%}
+{%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
+    {%- if (message['role'] == 'assistant') -%}
+        {%- set role = "model" -%}
+    {%- else -%}
+        {%- set role = message['role'] -%}
+    {%- endif -%}
+    {{ '<|turn>' + role + '
+' }}
+    {%- if message['content'] is string -%}
+        {%- if role == "model" -%}
+            {{ strip_thinking(message['content']) }}
+        {%- else -%}
+            {{ message['content'] | trim }}
+        {%- endif -%}
+    {%- elif message['content'] is iterable -%}
+        {%- for item in message['content'] -%}
+            {%- if item['type'] == 'audio' -%}
+                {{ '<|audio|>' }}
+            {%- elif item['type'] == 'image' -%}
+                {{ '<|image|>' }}
+            {%- elif item['type'] == 'video' -%}
+                {{ '<|video|>' }}
+            {%- elif item['type'] == 'text' -%}
+                {%- if role == "model" -%}
+                    {{ strip_thinking(item['text']) }}
+                {%- else -%}
+                    {{ item['text'] | trim }}
+                {%- endif -%}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{ raise_exception("Invalid content type") }}
+    {%- endif -%}
+    {{ '<turn|>
+' }}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{'<|turn>model
+'}}
+{%- endif -%}

best-lora/processor_config.json

@@ -0,0 +1,62 @@
+{
+  "audio_ms_per_token": 40,
+  "audio_seq_length": 750,
+  "feature_extractor": {
+    "audio_samples_per_token": 640,
+    "feature_extractor_type": "Gemma4UnifiedAudioFeatureExtractor",
+    "feature_size": 640,
+    "padding_side": "right",
+    "padding_value": 0.0,
+    "return_attention_mask": true,
+    "sampling_rate": 16000
+  },
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": false,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_processor_type": "Gemma4UnifiedImageProcessor",
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 280,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098
+  },
+  "image_seq_length": 280,
+  "processor_class": "Gemma4UnifiedProcessor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 70,
+    "num_frames": 32,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "video_processor_type": "Gemma4UnifiedVideoProcessor"
+  }
+}

best-lora/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc8d3a0ce36466ccc1278bf987df5f71db1719b9ca6b4118264f45cb627bfe0f
+size 32169626

best-lora/tokenizer_config.json

@@ -0,0 +1,290 @@
+{
+  "audio_token": "<|audio|>",
+  "backend": "tokenizers",
+  "boa_token": "<|audio>",
+  "boi_token": "<|image>",
+  "bos_token": "<bos>",
+  "eoa_token": "<audio|>",
+  "eoc_token": "<channel|>",
+  "eoi_token": "<image|>",
+  "eos_token": "<turn|>",
+  "eot_token": "<turn|>",
+  "escape_token": "<|\"|>",
+  "etc_token": "<tool_call|>",
+  "etd_token": "<tool|>",
+  "etr_token": "<tool_response|>",
+  "extra_special_tokens": [
+    "<|video|>"
+  ],
+  "image_token": "<|image|>",
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "<mask>",
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_token": "<|audio|>",
+    "boa_token": "<|audio>",
+    "boi_token": "<|image>",
+    "eoa_token": "<audio|>",
+    "eoc_token": "<channel|>",
+    "eoi_token": "<image|>",
+    "eot_token": "<turn|>",
+    "escape_token": "<|\"|>",
+    "etc_token": "<tool_call|>",
+    "etd_token": "<tool|>",
+    "etr_token": "<tool_response|>",
+    "image_token": "<|image|>",
+    "soc_token": "<|channel>",
+    "sot_token": "<|turn>",
+    "stc_token": "<|tool_call>",
+    "std_token": "<|tool>",
+    "str_token": "<|tool_response>",
+    "think_token": "<|think|>"
+  },
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "processor_class": "Gemma4UnifiedProcessor",
+  "response_schema": {
+    "properties": {
+      "content": {
+        "type": "string"
+      },
+      "role": {
+        "const": "assistant"
+      },
+      "thinking": {
+        "type": "string"
+      },
+      "tool_calls": {
+        "items": {
+          "properties": {
+            "function": {
+              "properties": {
+                "arguments": {
+                  "additionalProperties": {},
+                  "type": "object",
+                  "x-parser": "gemma4-tool-call"
+                },
+                "name": {
+                  "type": "string"
+                }
+              },
+              "type": "object",
+              "x-regex": "call\\:(?P<name>\\w+)(?P<arguments>\\{.*\\})"
+            },
+            "type": {
+              "const": "function"
+            }
+          },
+          "type": "object"
+        },
+        "type": "array",
+        "x-regex-iterator": "<\\|tool_call>(.*?)<tool_call\\|>"
+      }
+    },
+    "type": "object",
+    "x-regex": "(\\<\\|channel\\>thought\\n(?P<thinking>.*?)\\<channel\\|\\>)?(?P<tool_calls>\\<\\|tool_call\\>.*\\<tool_call\\|\\>)?(?P<content>(?:(?!\\<turn\\|\\>)(?!\\<\\|tool_response\\>).)+)?(?:\\<turn\\|\\>|\\<\\|tool_response\\>)?"
+  },
+  "soc_token": "<|channel>",
+  "sot_token": "<|turn>",
+  "stc_token": "<|tool_call>",
+  "std_token": "<|tool>",
+  "str_token": "<|tool_response>",
+  "think_token": "<|think|>",
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "1": {
+      "content": "<eos>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "2": {
+      "content": "<bos>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "46": {
+      "content": "<|tool>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "47": {
+      "content": "<tool|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "48": {
+      "content": "<|tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "49": {
+      "content": "<tool_call|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "50": {
+      "content": "<|tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "51": {
+      "content": "<tool_response|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "52": {
+      "content": "<|\"|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "98": {
+      "content": "<|think|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "100": {
+      "content": "<|channel>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "101": {
+      "content": "<channel|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "105": {
+      "content": "<|turn>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "106": {
+      "content": "<turn|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "255999": {
+      "content": "<|image>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "256000": {
+      "content": "<|audio>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258880": {
+      "content": "<|image|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258881": {
+      "content": "<|audio|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258882": {
+      "content": "<image|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258883": {
+      "content": "<audio|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258884": {
+      "content": "<|video|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  }
+}

best-lora/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:757b7d78e07bd589f7cd9db1a1a3019f58127f01f91cba75e4a6cbb7ea4b4a6c
+size 5713

best_checkpoint.txt

@@ -0,0 +1,2 @@
+/output/checkpoint-950
+eval_loss=0.5508978962898254

checkpoint-1400/README.md

@@ -0,0 +1,210 @@
+---
+base_model: unsloth/gemma-4-12b-it
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/gemma-4-12b-it
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

checkpoint-1400/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Gemma4UnifiedForConditionalGeneration",
+    "parent_library": "transformers.models.gemma4_unified.modeling_gemma4_unified",
+    "unsloth_fixed": true
+  },
+  "base_model_name_or_path": "unsloth/gemma-4-12b-it",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": "(?:.*?(?:language|text).*?(?:self_attn|attention|attn|mixer|mlp|feed_forward|ffn|dense|mixer).*?(?:q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj|embedding_projection))|(?:\\bmodel\\.layers\\.[\\d]{1,}\\.(?:self_attn|attention|attn|mixer|mlp|feed_forward|ffn|dense|mixer)\\.(?:(?:q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj|embedding_projection)))",
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

checkpoint-1400/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8367b932d2ccf89169b4104de5476941c459df42a6f25a98264614c3a2a7a7b2
+size 262373216

checkpoint-1400/chat_template.jinja

@@ -0,0 +1,70 @@
+{{ bos_token }}{%- macro strip_thinking(text) -%}
+    {%- set ns = namespace(result='') -%}
+    {%- for part in text.split('<channel|>') -%}
+        {%- if '<|channel>' in part -%}
+            {%- set ns.result = ns.result + part.split('<|channel>')[0] -%}
+        {%- else -%}
+            {%- set ns.result = ns.result + part -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- ns.result | trim -}}
+{%- endmacro -%}
+{%- set thinking = enable_thinking is defined and enable_thinking -%}
+{%- set loop_messages = messages -%}
+{%- if messages[0]['role'] in ['system', 'developer'] or thinking -%}
+    {{ '<|turn>system
+' }}
+    {%- if thinking -%}
+        {{ '<|think|>
+' }}
+    {%- endif -%}
+    {%- if messages[0]['role'] in ['system', 'developer'] -%}
+        {{ messages[0]['content'] | trim }}
+        {%- set loop_messages = messages[1:] -%}
+    {%- endif -%}
+    {{ '<turn|>
+' }}
+{%- endif -%}
+{%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
+    {%- if (message['role'] == 'assistant') -%}
+        {%- set role = "model" -%}
+    {%- else -%}
+        {%- set role = message['role'] -%}
+    {%- endif -%}
+    {{ '<|turn>' + role + '
+' }}
+    {%- if message['content'] is string -%}
+        {%- if role == "model" -%}
+            {{ strip_thinking(message['content']) }}
+        {%- else -%}
+            {{ message['content'] | trim }}
+        {%- endif -%}
+    {%- elif message['content'] is iterable -%}
+        {%- for item in message['content'] -%}
+            {%- if item['type'] == 'audio' -%}
+                {{ '<|audio|>' }}
+            {%- elif item['type'] == 'image' -%}
+                {{ '<|image|>' }}
+            {%- elif item['type'] == 'video' -%}
+                {{ '<|video|>' }}
+            {%- elif item['type'] == 'text' -%}
+                {%- if role == "model" -%}
+                    {{ strip_thinking(item['text']) }}
+                {%- else -%}
+                    {{ item['text'] | trim }}
+                {%- endif -%}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{ raise_exception("Invalid content type") }}
+    {%- endif -%}
+    {{ '<turn|>
+' }}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{'<|turn>model
+'}}
+{%- endif -%}

checkpoint-1400/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48f51510b2400a3bf83dcaaabddc310952c77edbd884b6cf45200ff58faa72f8
+size 133962277

checkpoint-1400/processor_config.json

@@ -0,0 +1,62 @@
+{
+  "audio_ms_per_token": 40,
+  "audio_seq_length": 750,
+  "feature_extractor": {
+    "audio_samples_per_token": 640,
+    "feature_extractor_type": "Gemma4UnifiedAudioFeatureExtractor",
+    "feature_size": 640,
+    "padding_side": "right",
+    "padding_value": 0.0,
+    "return_attention_mask": true,
+    "sampling_rate": 16000
+  },
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": false,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_processor_type": "Gemma4UnifiedImageProcessor",
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 280,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098
+  },
+  "image_seq_length": 280,
+  "processor_class": "Gemma4UnifiedProcessor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 70,
+    "num_frames": 32,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "video_processor_type": "Gemma4UnifiedVideoProcessor"
+  }
+}

checkpoint-1400/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:081b902a1cc1d7c04ce34bfbbb677ba64777c8e7d0d114a6cad2aaf4dff7c81d
+size 14645

checkpoint-1400/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b86947f72fdf30d840c361f579ca6179be7e2263d649cc31b242e447e5611163
+size 1465

checkpoint-1400/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc8d3a0ce36466ccc1278bf987df5f71db1719b9ca6b4118264f45cb627bfe0f
+size 32169626

checkpoint-1400/tokenizer_config.json

@@ -0,0 +1,290 @@
+{
+  "audio_token": "<|audio|>",
+  "backend": "tokenizers",
+  "boa_token": "<|audio>",
+  "boi_token": "<|image>",
+  "bos_token": "<bos>",
+  "eoa_token": "<audio|>",
+  "eoc_token": "<channel|>",
+  "eoi_token": "<image|>",
+  "eos_token": "<turn|>",
+  "eot_token": "<turn|>",
+  "escape_token": "<|\"|>",
+  "etc_token": "<tool_call|>",
+  "etd_token": "<tool|>",
+  "etr_token": "<tool_response|>",
+  "extra_special_tokens": [
+    "<|video|>"
+  ],
+  "image_token": "<|image|>",
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "<mask>",
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_token": "<|audio|>",
+    "boa_token": "<|audio>",
+    "boi_token": "<|image>",
+    "eoa_token": "<audio|>",
+    "eoc_token": "<channel|>",
+    "eoi_token": "<image|>",
+    "eot_token": "<turn|>",
+    "escape_token": "<|\"|>",
+    "etc_token": "<tool_call|>",
+    "etd_token": "<tool|>",
+    "etr_token": "<tool_response|>",
+    "image_token": "<|image|>",
+    "soc_token": "<|channel>",
+    "sot_token": "<|turn>",
+    "stc_token": "<|tool_call>",
+    "std_token": "<|tool>",
+    "str_token": "<|tool_response>",
+    "think_token": "<|think|>"
+  },
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "processor_class": "Gemma4UnifiedProcessor",
+  "response_schema": {
+    "properties": {
+      "content": {
+        "type": "string"
+      },
+      "role": {
+        "const": "assistant"
+      },
+      "thinking": {
+        "type": "string"
+      },
+      "tool_calls": {
+        "items": {
+          "properties": {
+            "function": {
+              "properties": {
+                "arguments": {
+                  "additionalProperties": {},
+                  "type": "object",
+                  "x-parser": "gemma4-tool-call"
+                },
+                "name": {
+                  "type": "string"
+                }
+              },
+              "type": "object",
+              "x-regex": "call\\:(?P<name>\\w+)(?P<arguments>\\{.*\\})"
+            },
+            "type": {
+              "const": "function"
+            }
+          },
+          "type": "object"
+        },
+        "type": "array",
+        "x-regex-iterator": "<\\|tool_call>(.*?)<tool_call\\|>"
+      }
+    },
+    "type": "object",
+    "x-regex": "(\\<\\|channel\\>thought\\n(?P<thinking>.*?)\\<channel\\|\\>)?(?P<tool_calls>\\<\\|tool_call\\>.*\\<tool_call\\|\\>)?(?P<content>(?:(?!\\<turn\\|\\>)(?!\\<\\|tool_response\\>).)+)?(?:\\<turn\\|\\>|\\<\\|tool_response\\>)?"
+  },
+  "soc_token": "<|channel>",
+  "sot_token": "<|turn>",
+  "stc_token": "<|tool_call>",
+  "std_token": "<|tool>",
+  "str_token": "<|tool_response>",
+  "think_token": "<|think|>",
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "1": {
+      "content": "<eos>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "2": {
+      "content": "<bos>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "46": {
+      "content": "<|tool>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "47": {
+      "content": "<tool|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "48": {
+      "content": "<|tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "49": {
+      "content": "<tool_call|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "50": {
+      "content": "<|tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "51": {
+      "content": "<tool_response|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "52": {
+      "content": "<|\"|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "98": {
+      "content": "<|think|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "100": {
+      "content": "<|channel>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "101": {
+      "content": "<channel|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "105": {
+      "content": "<|turn>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "106": {
+      "content": "<turn|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "255999": {
+      "content": "<|image>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "256000": {
+      "content": "<|audio>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258880": {
+      "content": "<|image|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258881": {
+      "content": "<|audio|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258882": {
+      "content": "<image|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258883": {
+      "content": "<audio|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258884": {
+      "content": "<|video|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  }
+}

checkpoint-1400/trainer_state.json

@@ -0,0 +1,2218 @@
+{
+  "best_global_step": 950,
+  "best_metric": 0.5508978962898254,
+  "best_model_checkpoint": "/output/checkpoint-950",
+  "epoch": 2.9473684210526314,
+  "eval_steps": 50,
+  "global_step": 1400,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.010526315789473684,
+      "grad_norm": 2.0839247703552246,
+      "learning_rate": 9.302325581395349e-06,
+      "loss": 2.646485137939453,
+      "step": 5
+    },
+    {
+      "epoch": 0.021052631578947368,
+      "grad_norm": 3.253295660018921,
+      "learning_rate": 2.0930232558139536e-05,
+      "loss": 2.9888063430786134,
+      "step": 10
+    },
+    {
+      "epoch": 0.031578947368421054,
+      "grad_norm": 2.4486842155456543,
+      "learning_rate": 3.2558139534883724e-05,
+      "loss": 2.2748624801635744,
+      "step": 15
+    },
+    {
+      "epoch": 0.042105263157894736,
+      "grad_norm": 2.5299599170684814,
+      "learning_rate": 4.418604651162791e-05,
+      "loss": 1.9410686492919922,
+      "step": 20
+    },
+    {
+      "epoch": 0.05263157894736842,
+      "grad_norm": 1.6780576705932617,
+      "learning_rate": 5.5813953488372095e-05,
+      "loss": 1.3348378181457519,
+      "step": 25
+    },
+    {
+      "epoch": 0.06315789473684211,
+      "grad_norm": 0.7812225222587585,
+      "learning_rate": 6.744186046511628e-05,
+      "loss": 0.8430736541748047,
+      "step": 30
+    },
+    {
+      "epoch": 0.07368421052631578,
+      "grad_norm": 0.8205987215042114,
+      "learning_rate": 7.906976744186047e-05,
+      "loss": 0.8522604942321778,
+      "step": 35
+    },
+    {
+      "epoch": 0.08421052631578947,
+      "grad_norm": 0.6925882697105408,
+      "learning_rate": 9.069767441860465e-05,
+      "loss": 0.6957129955291748,
+      "step": 40
+    },
+    {
+      "epoch": 0.09473684210526316,
+      "grad_norm": 0.7553223371505737,
+      "learning_rate": 9.999987081161148e-05,
+      "loss": 0.8302087783813477,
+      "step": 45
+    },
+    {
+      "epoch": 0.10526315789473684,
+      "grad_norm": 0.6056613922119141,
+      "learning_rate": 9.999534928810904e-05,
+      "loss": 0.6982179641723633,
+      "step": 50
+    },
+    {
+      "epoch": 0.10526315789473684,
+      "eval_loss": 0.7021176218986511,
+      "eval_runtime": 27.8753,
+      "eval_samples_per_second": 14.35,
+      "eval_steps_per_second": 14.35,
+      "step": 50
+    },
+    {
+      "epoch": 0.11578947368421053,
+      "grad_norm": 1.4581637382507324,
+      "learning_rate": 9.998436901275022e-05,
+      "loss": 0.7925346374511719,
+      "step": 55
+    },
+    {
+      "epoch": 0.12631578947368421,
+      "grad_norm": 0.6449484825134277,
+      "learning_rate": 9.99669314040444e-05,
+      "loss": 0.7095213413238526,
+      "step": 60
+    },
+    {
+      "epoch": 0.1368421052631579,
+      "grad_norm": 0.6936064958572388,
+      "learning_rate": 9.994303871470489e-05,
+      "loss": 0.6608465194702149,
+      "step": 65
+    },
+    {
+      "epoch": 0.14736842105263157,
+      "grad_norm": 0.5944413542747498,
+      "learning_rate": 9.991269403135783e-05,
+      "loss": 0.6789331436157227,
+      "step": 70
+    },
+    {
+      "epoch": 0.15789473684210525,
+      "grad_norm": 0.7518417835235596,
+      "learning_rate": 9.987590127414344e-05,
+      "loss": 0.669992208480835,
+      "step": 75
+    },
+    {
+      "epoch": 0.16842105263157894,
+      "grad_norm": 0.7772454619407654,
+      "learning_rate": 9.98326651962096e-05,
+      "loss": 0.6882061958312988,
+      "step": 80
+    },
+    {
+      "epoch": 0.17894736842105263,
+      "grad_norm": 0.7076404690742493,
+      "learning_rate": 9.978299138309781e-05,
+      "loss": 0.6435281276702881,
+      "step": 85
+    },
+    {
+      "epoch": 0.18947368421052632,
+      "grad_norm": 0.6353256702423096,
+      "learning_rate": 9.972688625202164e-05,
+      "loss": 0.6035995960235596,
+      "step": 90
+    },
+    {
+      "epoch": 0.2,
+      "grad_norm": 0.6928842663764954,
+      "learning_rate": 9.966435705103765e-05,
+      "loss": 0.6339406967163086,
+      "step": 95
+    },
+    {
+      "epoch": 0.21052631578947367,
+      "grad_norm": 1.1047526597976685,
+      "learning_rate": 9.959541185810906e-05,
+      "loss": 0.5951476097106934,
+      "step": 100
+    },
+    {
+      "epoch": 0.21052631578947367,
+      "eval_loss": 0.6416817307472229,
+      "eval_runtime": 27.0456,
+      "eval_samples_per_second": 14.79,
+      "eval_steps_per_second": 14.79,
+      "step": 100
+    },
+    {
+      "epoch": 0.22105263157894736,
+      "grad_norm": 0.6272938847541809,
+      "learning_rate": 9.952005958006217e-05,
+      "loss": 0.6947292804718017,
+      "step": 105
+    },
+    {
+      "epoch": 0.23157894736842105,
+      "grad_norm": 0.6077613234519958,
+      "learning_rate": 9.943830995143577e-05,
+      "loss": 0.7127768993377686,
+      "step": 110
+    },
+    {
+      "epoch": 0.24210526315789474,
+      "grad_norm": 0.6263899803161621,
+      "learning_rate": 9.935017353322347e-05,
+      "loss": 0.643881893157959,
+      "step": 115
+    },
+    {
+      "epoch": 0.25263157894736843,
+      "grad_norm": 0.6517646908760071,
+      "learning_rate": 9.925566171150945e-05,
+      "loss": 0.6408101081848144,
+      "step": 120
+    },
+    {
+      "epoch": 0.2631578947368421,
+      "grad_norm": 0.5079100728034973,
+      "learning_rate": 9.915478669599747e-05,
+      "loss": 0.5704734802246094,
+      "step": 125
+    },
+    {
+      "epoch": 0.2736842105263158,
+      "grad_norm": 0.6810751557350159,
+      "learning_rate": 9.904756151843353e-05,
+      "loss": 0.7103314876556397,
+      "step": 130
+    },
+    {
+      "epoch": 0.28421052631578947,
+      "grad_norm": 0.6985617876052856,
+      "learning_rate": 9.893400003092237e-05,
+      "loss": 0.7399785995483399,
+      "step": 135
+    },
+    {
+      "epoch": 0.29473684210526313,
+      "grad_norm": 0.611039936542511,
+      "learning_rate": 9.881411690413796e-05,
+      "loss": 0.6211562633514405,
+      "step": 140
+    },
+    {
+      "epoch": 0.30526315789473685,
+      "grad_norm": 0.6540597677230835,
+      "learning_rate": 9.868792762542814e-05,
+      "loss": 0.6841907501220703,
+      "step": 145
+    },
+    {
+      "epoch": 0.3157894736842105,
+      "grad_norm": 0.6657130122184753,
+      "learning_rate": 9.855544849681404e-05,
+      "loss": 0.6384834289550781,
+      "step": 150
+    },
+    {
+      "epoch": 0.3157894736842105,
+      "eval_loss": 0.6154849529266357,
+      "eval_runtime": 28.9994,
+      "eval_samples_per_second": 13.793,
+      "eval_steps_per_second": 13.793,
+      "step": 150
+    },
+    {
+      "epoch": 0.3263157894736842,
+      "grad_norm": 0.6323176622390747,
+      "learning_rate": 9.841669663288391e-05,
+      "loss": 0.6482361793518067,
+      "step": 155
+    },
+    {
+      "epoch": 0.3368421052631579,
+      "grad_norm": 0.6461365818977356,
+      "learning_rate": 9.82716899585822e-05,
+      "loss": 0.6088940620422363,
+      "step": 160
+    },
+    {
+      "epoch": 0.3473684210526316,
+      "grad_norm": 0.518051028251648,
+      "learning_rate": 9.812044720689387e-05,
+      "loss": 0.6474239826202393,
+      "step": 165
+    },
+    {
+      "epoch": 0.35789473684210527,
+      "grad_norm": 0.6599591374397278,
+      "learning_rate": 9.796298791642435e-05,
+      "loss": 0.639539098739624,
+      "step": 170
+    },
+    {
+      "epoch": 0.3684210526315789,
+      "grad_norm": 0.6555272340774536,
+      "learning_rate": 9.779933242887542e-05,
+      "loss": 0.6650691986083984,
+      "step": 175
+    },
+    {
+      "epoch": 0.37894736842105264,
+      "grad_norm": 0.5666244626045227,
+      "learning_rate": 9.762950188641728e-05,
+      "loss": 0.6851089000701904,
+      "step": 180
+    },
+    {
+      "epoch": 0.3894736842105263,
+      "grad_norm": 0.7169855833053589,
+      "learning_rate": 9.745351822895727e-05,
+      "loss": 0.5861614227294922,
+      "step": 185
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 0.5155227184295654,
+      "learning_rate": 9.727140419130553e-05,
+      "loss": 0.6075415134429931,
+      "step": 190
+    },
+    {
+      "epoch": 0.4105263157894737,
+      "grad_norm": 0.8202889561653137,
+      "learning_rate": 9.708318330023798e-05,
+      "loss": 0.6071089267730713,
+      "step": 195
+    },
+    {
+      "epoch": 0.42105263157894735,
+      "grad_norm": 0.4622310996055603,
+      "learning_rate": 9.688887987145691e-05,
+      "loss": 0.5945972919464111,
+      "step": 200
+    },
+    {
+      "epoch": 0.42105263157894735,
+      "eval_loss": 0.6035570502281189,
+      "eval_runtime": 27.8485,
+      "eval_samples_per_second": 14.363,
+      "eval_steps_per_second": 14.363,
+      "step": 200
+    },
+    {
+      "epoch": 0.43157894736842106,
+      "grad_norm": 0.6781761050224304,
+      "learning_rate": 9.668851900644975e-05,
+      "loss": 0.570584774017334,
+      "step": 205
+    },
+    {
+      "epoch": 0.4421052631578947,
+      "grad_norm": 0.5482363104820251,
+      "learning_rate": 9.648212658924625e-05,
+      "loss": 0.6329983234405517,
+      "step": 210
+    },
+    {
+      "epoch": 0.45263157894736844,
+      "grad_norm": 0.6346020698547363,
+      "learning_rate": 9.62697292830746e-05,
+      "loss": 0.6306288242340088,
+      "step": 215
+    },
+    {
+      "epoch": 0.4631578947368421,
+      "grad_norm": 0.7032224535942078,
+      "learning_rate": 9.6051354526917e-05,
+      "loss": 0.6967363357543945,
+      "step": 220
+    },
+    {
+      "epoch": 0.47368421052631576,
+      "grad_norm": 0.4979580044746399,
+      "learning_rate": 9.582703053196464e-05,
+      "loss": 0.5608481884002685,
+      "step": 225
+    },
+    {
+      "epoch": 0.4842105263157895,
+      "grad_norm": 0.7327916622161865,
+      "learning_rate": 9.55967862779735e-05,
+      "loss": 0.584914255142212,
+      "step": 230
+    },
+    {
+      "epoch": 0.49473684210526314,
+      "grad_norm": 0.7519303560256958,
+      "learning_rate": 9.536065150952025e-05,
+      "loss": 0.6214882850646972,
+      "step": 235
+    },
+    {
+      "epoch": 0.5052631578947369,
+      "grad_norm": 0.7328101396560669,
+      "learning_rate": 9.511865673215986e-05,
+      "loss": 0.6112482070922851,
+      "step": 240
+    },
+    {
+      "epoch": 0.5157894736842106,
+      "grad_norm": 0.5348976254463196,
+      "learning_rate": 9.487083320848454e-05,
+      "loss": 0.6417016983032227,
+      "step": 245
+    },
+    {
+      "epoch": 0.5263157894736842,
+      "grad_norm": 0.5690417885780334,
+      "learning_rate": 9.461721295408505e-05,
+      "loss": 0.5277723789215087,
+      "step": 250
+    },
+    {
+      "epoch": 0.5263157894736842,
+      "eval_loss": 0.5920669436454773,
+      "eval_runtime": 27.6132,
+      "eval_samples_per_second": 14.486,
+      "eval_steps_per_second": 14.486,
+      "step": 250
+    },
+    {
+      "epoch": 0.5368421052631579,
+      "grad_norm": 0.5612736344337463,
+      "learning_rate": 9.435782873341474e-05,
+      "loss": 0.5604990959167481,
+      "step": 255
+    },
+    {
+      "epoch": 0.5473684210526316,
+      "grad_norm": 0.6084933876991272,
+      "learning_rate": 9.409271405555677e-05,
+      "loss": 0.5558807373046875,
+      "step": 260
+    },
+    {
+      "epoch": 0.5578947368421052,
+      "grad_norm": 0.6289730668067932,
+      "learning_rate": 9.382190316989518e-05,
+      "loss": 0.5646376609802246,
+      "step": 265
+    },
+    {
+      "epoch": 0.5684210526315789,
+      "grad_norm": 0.6343289613723755,
+      "learning_rate": 9.354543106169029e-05,
+      "loss": 0.6692690372467041,
+      "step": 270
+    },
+    {
+      "epoch": 0.5789473684210527,
+      "grad_norm": 1.0037399530410767,
+      "learning_rate": 9.326333344755912e-05,
+      "loss": 0.6066013813018799,
+      "step": 275
+    },
+    {
+      "epoch": 0.5894736842105263,
+      "grad_norm": 0.8138774633407593,
+      "learning_rate": 9.297564677086118e-05,
+      "loss": 0.6074648857116699,
+      "step": 280
+    },
+    {
+      "epoch": 0.6,
+      "grad_norm": 1.0000449419021606,
+      "learning_rate": 9.268240819699054e-05,
+      "loss": 0.5891916275024414,
+      "step": 285
+    },
+    {
+      "epoch": 0.6105263157894737,
+      "grad_norm": 0.7454043030738831,
+      "learning_rate": 9.238365560857447e-05,
+      "loss": 0.6280234336853028,
+      "step": 290
+    },
+    {
+      "epoch": 0.6210526315789474,
+      "grad_norm": 0.756981611251831,
+      "learning_rate": 9.207942760057958e-05,
+      "loss": 0.6314973831176758,
+      "step": 295
+    },
+    {
+      "epoch": 0.631578947368421,
+      "grad_norm": 0.7583857178688049,
+      "learning_rate": 9.176976347532575e-05,
+      "loss": 0.6410815715789795,
+      "step": 300
+    },
+    {
+      "epoch": 0.631578947368421,
+      "eval_loss": 0.5863876342773438,
+      "eval_runtime": 27.6386,
+      "eval_samples_per_second": 14.472,
+      "eval_steps_per_second": 14.472,
+      "step": 300
+    },
+    {
+      "epoch": 0.6421052631578947,
+      "grad_norm": 0.791587769985199,
+      "learning_rate": 9.145470323740885e-05,
+      "loss": 0.5841981410980225,
+      "step": 305
+    },
+    {
+      "epoch": 0.6526315789473685,
+      "grad_norm": 1.1765379905700684,
+      "learning_rate": 9.113428758853268e-05,
+      "loss": 0.5464617729187011,
+      "step": 310
+    },
+    {
+      "epoch": 0.6631578947368421,
+      "grad_norm": 3.7840137481689453,
+      "learning_rate": 9.080855792225076e-05,
+      "loss": 0.6217821121215821,
+      "step": 315
+    },
+    {
+      "epoch": 0.6736842105263158,
+      "grad_norm": 1.2177997827529907,
+      "learning_rate": 9.047755631861884e-05,
+      "loss": 0.6125752449035644,
+      "step": 320
+    },
+    {
+      "epoch": 0.6842105263157895,
+      "grad_norm": 0.6323914527893066,
+      "learning_rate": 9.014132553875878e-05,
+      "loss": 0.5617509365081788,
+      "step": 325
+    },
+    {
+      "epoch": 0.6947368421052632,
+      "grad_norm": 0.9683524370193481,
+      "learning_rate": 8.979990901933428e-05,
+      "loss": 0.6395817756652832,
+      "step": 330
+    },
+    {
+      "epoch": 0.7052631578947368,
+      "grad_norm": 0.6486535668373108,
+      "learning_rate": 8.945335086693942e-05,
+      "loss": 0.6556002616882324,
+      "step": 335
+    },
+    {
+      "epoch": 0.7157894736842105,
+      "grad_norm": 1.642112374305725,
+      "learning_rate": 8.910169585240078e-05,
+      "loss": 0.588761568069458,
+      "step": 340
+    },
+    {
+      "epoch": 0.7263157894736842,
+      "grad_norm": 1.007238507270813,
+      "learning_rate": 8.874498940499346e-05,
+      "loss": 0.5744077682495117,
+      "step": 345
+    },
+    {
+      "epoch": 0.7368421052631579,
+      "grad_norm": 0.4945877194404602,
+      "learning_rate": 8.83832776065723e-05,
+      "loss": 0.5382843494415284,
+      "step": 350
+    },
+    {
+      "epoch": 0.7368421052631579,
+      "eval_loss": 0.5782474279403687,
+      "eval_runtime": 27.574,
+      "eval_samples_per_second": 14.506,
+      "eval_steps_per_second": 14.506,
+      "step": 350
+    },
+    {
+      "epoch": 0.7473684210526316,
+      "grad_norm": 0.7869954109191895,
+      "learning_rate": 8.801660718561875e-05,
+      "loss": 0.5697366237640381,
+      "step": 355
+    },
+    {
+      "epoch": 0.7578947368421053,
+      "grad_norm": 0.48311182856559753,
+      "learning_rate": 8.7645025511204e-05,
+      "loss": 0.6168550491333008,
+      "step": 360
+    },
+    {
+      "epoch": 0.7684210526315789,
+      "grad_norm": 0.5001690983772278,
+      "learning_rate": 8.726858058686968e-05,
+      "loss": 0.5362951278686523,
+      "step": 365
+    },
+    {
+      "epoch": 0.7789473684210526,
+      "grad_norm": 0.5915603041648865,
+      "learning_rate": 8.688732104442632e-05,
+      "loss": 0.6041481018066406,
+      "step": 370
+    },
+    {
+      "epoch": 0.7894736842105263,
+      "grad_norm": 0.5077712535858154,
+      "learning_rate": 8.650129613767075e-05,
+      "loss": 0.5257850646972656,
+      "step": 375
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.7458930015563965,
+      "learning_rate": 8.611055573602323e-05,
+      "loss": 0.6048945903778076,
+      "step": 380
+    },
+    {
+      "epoch": 0.8105263157894737,
+      "grad_norm": 0.6072155237197876,
+      "learning_rate": 8.571515031808484e-05,
+      "loss": 0.5505197048187256,
+      "step": 385
+    },
+    {
+      "epoch": 0.8210526315789474,
+      "grad_norm": 0.5989276170730591,
+      "learning_rate": 8.531513096511646e-05,
+      "loss": 0.528736162185669,
+      "step": 390
+    },
+    {
+      "epoch": 0.8315789473684211,
+      "grad_norm": 0.47586777806282043,
+      "learning_rate": 8.491054935443954e-05,
+      "loss": 0.5494171142578125,
+      "step": 395
+    },
+    {
+      "epoch": 0.8421052631578947,
+      "grad_norm": 0.6332405209541321,
+      "learning_rate": 8.450145775276024e-05,
+      "loss": 0.605548906326294,
+      "step": 400
+    },
+    {
+      "epoch": 0.8421052631578947,
+      "eval_loss": 0.5717179775238037,
+      "eval_runtime": 27.9225,
+      "eval_samples_per_second": 14.325,
+      "eval_steps_per_second": 14.325,
+      "step": 400
+    },
+    {
+      "epoch": 0.8526315789473684,
+      "grad_norm": 0.873306155204773,
+      "learning_rate": 8.40879090094171e-05,
+      "loss": 0.6015737056732178,
+      "step": 405
+    },
+    {
+      "epoch": 0.8631578947368421,
+      "grad_norm": 0.6360308527946472,
+      "learning_rate": 8.366995654955375e-05,
+      "loss": 0.5689235687255859,
+      "step": 410
+    },
+    {
+      "epoch": 0.8736842105263158,
+      "grad_norm": 0.7359456419944763,
+      "learning_rate": 8.324765436721688e-05,
+      "loss": 0.6225551128387451,
+      "step": 415
+    },
+    {
+      "epoch": 0.8842105263157894,
+      "grad_norm": 0.6135861873626709,
+      "learning_rate": 8.282105701838106e-05,
+      "loss": 0.567096471786499,
+      "step": 420
+    },
+    {
+      "epoch": 0.8947368421052632,
+      "grad_norm": 0.5699036717414856,
+      "learning_rate": 8.239021961390078e-05,
+      "loss": 0.5435199737548828,
+      "step": 425
+    },
+    {
+      "epoch": 0.9052631578947369,
+      "grad_norm": 0.6312369704246521,
+      "learning_rate": 8.195519781239079e-05,
+      "loss": 0.6320878028869629,
+      "step": 430
+    },
+    {
+      "epoch": 0.9157894736842105,
+      "grad_norm": 0.7718231678009033,
+      "learning_rate": 8.151604781303577e-05,
+      "loss": 0.5115500450134277,
+      "step": 435
+    },
+    {
+      "epoch": 0.9263157894736842,
+      "grad_norm": 0.7022733688354492,
+      "learning_rate": 8.107282634833015e-05,
+      "loss": 0.5616052627563477,
+      "step": 440
+    },
+    {
+      "epoch": 0.9368421052631579,
+      "grad_norm": 0.6376118659973145,
+      "learning_rate": 8.06255906767489e-05,
+      "loss": 0.6315895080566406,
+      "step": 445
+    },
+    {
+      "epoch": 0.9473684210526315,
+      "grad_norm": 0.5550912618637085,
+      "learning_rate": 8.01743985753506e-05,
+      "loss": 0.597140884399414,
+      "step": 450
+    },
+    {
+      "epoch": 0.9473684210526315,
+      "eval_loss": 0.5748050212860107,
+      "eval_runtime": 27.2437,
+      "eval_samples_per_second": 14.682,
+      "eval_steps_per_second": 14.682,
+      "step": 450
+    },
+    {
+      "epoch": 0.9578947368421052,
+      "grad_norm": 0.5438539385795593,
+      "learning_rate": 7.971930833231338e-05,
+      "loss": 0.5860416889190674,
+      "step": 455
+    },
+    {
+      "epoch": 0.968421052631579,
+      "grad_norm": 0.7530905604362488,
+      "learning_rate": 7.926037873940469e-05,
+      "loss": 0.5889331340789795,
+      "step": 460
+    },
+    {
+      "epoch": 0.9789473684210527,
+      "grad_norm": 0.6347785592079163,
+      "learning_rate": 7.879766908438638e-05,
+      "loss": 0.5784619808197021,
+      "step": 465
+    },
+    {
+      "epoch": 0.9894736842105263,
+      "grad_norm": 0.6019025444984436,
+      "learning_rate": 7.83312391433553e-05,
+      "loss": 0.5674707412719726,
+      "step": 470
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.5169152617454529,
+      "learning_rate": 7.786114917302118e-05,
+      "loss": 0.6072670936584472,
+      "step": 475
+    },
+    {
+      "epoch": 1.0105263157894737,
+      "grad_norm": 0.5598861575126648,
+      "learning_rate": 7.738745990292208e-05,
+      "loss": 0.5027533531188965,
+      "step": 480
+    },
+    {
+      "epoch": 1.0210526315789474,
+      "grad_norm": 0.5044442415237427,
+      "learning_rate": 7.691023252757901e-05,
+      "loss": 0.50599045753479,
+      "step": 485
+    },
+    {
+      "epoch": 1.0315789473684212,
+      "grad_norm": 0.5246402025222778,
+      "learning_rate": 7.64295286985904e-05,
+      "loss": 0.4706981658935547,
+      "step": 490
+    },
+    {
+      "epoch": 1.0421052631578946,
+      "grad_norm": 0.6998376846313477,
+      "learning_rate": 7.594541051666742e-05,
+      "loss": 0.48616642951965333,
+      "step": 495
+    },
+    {
+      "epoch": 1.0526315789473684,
+      "grad_norm": 0.5945510864257812,
+      "learning_rate": 7.545794052361149e-05,
+      "loss": 0.46446027755737307,
+      "step": 500
+    },
+    {
+      "epoch": 1.0526315789473684,
+      "eval_loss": 0.5771171450614929,
+      "eval_runtime": 27.1019,
+      "eval_samples_per_second": 14.759,
+      "eval_steps_per_second": 14.759,
+      "step": 500
+    },
+    {
+      "epoch": 1.063157894736842,
+      "grad_norm": 12.692610740661621,
+      "learning_rate": 7.496718169423462e-05,
+      "loss": 0.4399024486541748,
+      "step": 505
+    },
+    {
+      "epoch": 1.0736842105263158,
+      "grad_norm": 0.6038990020751953,
+      "learning_rate": 7.447319742822392e-05,
+      "loss": 0.48290238380432127,
+      "step": 510
+    },
+    {
+      "epoch": 1.0842105263157895,
+      "grad_norm": 0.6743545532226562,
+      "learning_rate": 7.397605154195106e-05,
+      "loss": 0.4822847366333008,
+      "step": 515
+    },
+    {
+      "epoch": 1.0947368421052632,
+      "grad_norm": 0.6746461391448975,
+      "learning_rate": 7.347580826022821e-05,
+      "loss": 0.47967071533203126,
+      "step": 520
+    },
+    {
+      "epoch": 1.1052631578947367,
+      "grad_norm": 0.7964761853218079,
+      "learning_rate": 7.29725322080109e-05,
+      "loss": 0.5180202484130859,
+      "step": 525
+    },
+    {
+      "epoch": 1.1157894736842104,
+      "grad_norm": 0.6772136688232422,
+      "learning_rate": 7.246628840204935e-05,
+      "loss": 0.46326284408569335,
+      "step": 530
+    },
+    {
+      "epoch": 1.1263157894736842,
+      "grad_norm": 0.7225517630577087,
+      "learning_rate": 7.195714224248912e-05,
+      "loss": 0.5262459278106689,
+      "step": 535
+    },
+    {
+      "epoch": 1.1368421052631579,
+      "grad_norm": 0.8155159950256348,
+      "learning_rate": 7.144515950442232e-05,
+      "loss": 0.5381614685058593,
+      "step": 540
+    },
+    {
+      "epoch": 1.1473684210526316,
+      "grad_norm": 0.7686448097229004,
+      "learning_rate": 7.093040632939023e-05,
+      "loss": 0.48169612884521484,
+      "step": 545
+    },
+    {
+      "epoch": 1.1578947368421053,
+      "grad_norm": 0.798007071018219,
+      "learning_rate": 7.041294921683876e-05,
+      "loss": 0.571818494796753,
+      "step": 550
+    },
+    {
+      "epoch": 1.1578947368421053,
+      "eval_loss": 0.5729017853736877,
+      "eval_runtime": 27.3923,
+      "eval_samples_per_second": 14.603,
+      "eval_steps_per_second": 14.603,
+      "step": 550
+    },
+    {
+      "epoch": 1.168421052631579,
+      "grad_norm": 0.9871010184288025,
+      "learning_rate": 6.989285501552751e-05,
+      "loss": 0.5277251243591309,
+      "step": 555
+    },
+    {
+      "epoch": 1.1789473684210527,
+      "grad_norm": 0.6931464076042175,
+      "learning_rate": 6.93701909148938e-05,
+      "loss": 0.4839730739593506,
+      "step": 560
+    },
+    {
+      "epoch": 1.1894736842105262,
+      "grad_norm": 0.6945406198501587,
+      "learning_rate": 6.884502443637273e-05,
+      "loss": 0.4632833480834961,
+      "step": 565
+    },
+    {
+      "epoch": 1.2,
+      "grad_norm": 0.8374159336090088,
+      "learning_rate": 6.831742342467418e-05,
+      "loss": 0.46635966300964354,
+      "step": 570
+    },
+    {
+      "epoch": 1.2105263157894737,
+      "grad_norm": 0.8007093667984009,
+      "learning_rate": 6.778745603901817e-05,
+      "loss": 0.4775552272796631,
+      "step": 575
+    },
+    {
+      "epoch": 1.2210526315789474,
+      "grad_norm": 0.922927975654602,
+      "learning_rate": 6.725519074432965e-05,
+      "loss": 0.41661796569824217,
+      "step": 580
+    },
+    {
+      "epoch": 1.231578947368421,
+      "grad_norm": 0.8915149569511414,
+      "learning_rate": 6.672069630239366e-05,
+      "loss": 0.46227364540100097,
+      "step": 585
+    },
+    {
+      "epoch": 1.2421052631578948,
+      "grad_norm": 0.7948114275932312,
+      "learning_rate": 6.618404176297217e-05,
+      "loss": 0.5104813575744629,
+      "step": 590
+    },
+    {
+      "epoch": 1.2526315789473683,
+      "grad_norm": 0.9380368590354919,
+      "learning_rate": 6.564529645488383e-05,
+      "loss": 0.4952188014984131,
+      "step": 595
+    },
+    {
+      "epoch": 1.263157894736842,
+      "grad_norm": 0.8004136085510254,
+      "learning_rate": 6.510452997704748e-05,
+      "loss": 0.4533547878265381,
+      "step": 600
+    },
+    {
+      "epoch": 1.263157894736842,
+      "eval_loss": 0.5720676779747009,
+      "eval_runtime": 27.2065,
+      "eval_samples_per_second": 14.702,
+      "eval_steps_per_second": 14.702,
+      "step": 600
+    },
+    {
+      "epoch": 1.2736842105263158,
+      "grad_norm": 0.8849948644638062,
+      "learning_rate": 6.456181218949096e-05,
+      "loss": 0.5185356140136719,
+      "step": 605
+    },
+    {
+      "epoch": 1.2842105263157895,
+      "grad_norm": 0.8258388638496399,
+      "learning_rate": 6.401721320432604e-05,
+      "loss": 0.45459442138671874,
+      "step": 610
+    },
+    {
+      "epoch": 1.2947368421052632,
+      "grad_norm": 0.7006433606147766,
+      "learning_rate": 6.34708033766909e-05,
+      "loss": 0.42599120140075686,
+      "step": 615
+    },
+    {
+      "epoch": 1.305263157894737,
+      "grad_norm": 0.8149921298027039,
+      "learning_rate": 6.292265329566108e-05,
+      "loss": 0.47135143280029296,
+      "step": 620
+    },
+    {
+      "epoch": 1.3157894736842106,
+      "grad_norm": 0.8304453492164612,
+      "learning_rate": 6.237283377513036e-05,
+      "loss": 0.543507719039917,
+      "step": 625
+    },
+    {
+      "epoch": 1.3263157894736843,
+      "grad_norm": 2.146064281463623,
+      "learning_rate": 6.182141584466247e-05,
+      "loss": 0.525644063949585,
+      "step": 630
+    },
+    {
+      "epoch": 1.3368421052631578,
+      "grad_norm": 0.8405390381813049,
+      "learning_rate": 6.126847074031507e-05,
+      "loss": 0.4994760513305664,
+      "step": 635
+    },
+    {
+      "epoch": 1.3473684210526315,
+      "grad_norm": 0.7558398842811584,
+      "learning_rate": 6.071406989543678e-05,
+      "loss": 0.45131878852844237,
+      "step": 640
+    },
+    {
+      "epoch": 1.3578947368421053,
+      "grad_norm": 1.3861230611801147,
+      "learning_rate": 6.0158284931439177e-05,
+      "loss": 0.45543656349182127,
+      "step": 645
+    },
+    {
+      "epoch": 1.368421052631579,
+      "grad_norm": 0.7160976529121399,
+      "learning_rate": 5.9601187648544056e-05,
+      "loss": 0.4914837837219238,
+      "step": 650
+    },
+    {
+      "epoch": 1.368421052631579,
+      "eval_loss": 0.566149890422821,
+      "eval_runtime": 27.1418,
+      "eval_samples_per_second": 14.737,
+      "eval_steps_per_second": 14.737,
+      "step": 650
+    },
+    {
+      "epoch": 1.3789473684210527,
+      "grad_norm": 0.7861437201499939,
+      "learning_rate": 5.904285001650783e-05,
+      "loss": 0.497259521484375,
+      "step": 655
+    },
+    {
+      "epoch": 1.3894736842105262,
+      "grad_norm": 0.7229910492897034,
+      "learning_rate": 5.8483344165323975e-05,
+      "loss": 0.5121944904327392,
+      "step": 660
+    },
+    {
+      "epoch": 1.4,
+      "grad_norm": 0.8752608299255371,
+      "learning_rate": 5.792274237590471e-05,
+      "loss": 0.5024206161499023,
+      "step": 665
+    },
+    {
+      "epoch": 1.4105263157894736,
+      "grad_norm": 0.8390534520149231,
+      "learning_rate": 5.7361117070743374e-05,
+      "loss": 0.48681044578552246,
+      "step": 670
+    },
+    {
+      "epoch": 1.4210526315789473,
+      "grad_norm": 0.8038208484649658,
+      "learning_rate": 5.679854080455821e-05,
+      "loss": 0.4971346378326416,
+      "step": 675
+    },
+    {
+      "epoch": 1.431578947368421,
+      "grad_norm": 0.7638700008392334,
+      "learning_rate": 5.6235086254919324e-05,
+      "loss": 0.42120847702026365,
+      "step": 680
+    },
+    {
+      "epoch": 1.4421052631578948,
+      "grad_norm": 0.7635045051574707,
+      "learning_rate": 5.567082621285969e-05,
+      "loss": 0.47087574005126953,
+      "step": 685
+    },
+    {
+      "epoch": 1.4526315789473685,
+      "grad_norm": 0.77474045753479,
+      "learning_rate": 5.510583357347149e-05,
+      "loss": 0.48073482513427734,
+      "step": 690
+    },
+    {
+      "epoch": 1.4631578947368422,
+      "grad_norm": 0.8228810429573059,
+      "learning_rate": 5.454018132648897e-05,
+      "loss": 0.46302366256713867,
+      "step": 695
+    },
+    {
+      "epoch": 1.4736842105263157,
+      "grad_norm": 0.8553083539009094,
+      "learning_rate": 5.3973942546859145e-05,
+      "loss": 0.47696647644042967,
+      "step": 700
+    },
+    {
+      "epoch": 1.4736842105263157,
+      "eval_loss": 0.5677127242088318,
+      "eval_runtime": 27.2564,
+      "eval_samples_per_second": 14.675,
+      "eval_steps_per_second": 14.675,
+      "step": 700
+    },
+    {
+      "epoch": 1.4842105263157894,
+      "grad_norm": 0.885428249835968,
+      "learning_rate": 5.3407190385301456e-05,
+      "loss": 0.426132345199585,
+      "step": 705
+    },
+    {
+      "epoch": 1.4947368421052631,
+      "grad_norm": 0.7517284154891968,
+      "learning_rate": 5.283999805885764e-05,
+      "loss": 0.5038762092590332,
+      "step": 710
+    },
+    {
+      "epoch": 1.5052631578947369,
+      "grad_norm": 0.6059720516204834,
+      "learning_rate": 5.227243884143306e-05,
+      "loss": 0.40855984687805175,
+      "step": 715
+    },
+    {
+      "epoch": 1.5157894736842106,
+      "grad_norm": 0.883471667766571,
+      "learning_rate": 5.170458605433059e-05,
+      "loss": 0.48758525848388673,
+      "step": 720
+    },
+    {
+      "epoch": 1.526315789473684,
+      "grad_norm": 0.8542289733886719,
+      "learning_rate": 5.113651305677856e-05,
+      "loss": 0.4801379680633545,
+      "step": 725
+    },
+    {
+      "epoch": 1.5368421052631578,
+      "grad_norm": 0.7095292210578918,
+      "learning_rate": 5.0568293236453614e-05,
+      "loss": 0.4451273441314697,
+      "step": 730
+    },
+    {
+      "epoch": 1.5473684210526315,
+      "grad_norm": 0.7466174960136414,
+      "learning_rate": 5e-05,
+      "loss": 0.4858893871307373,
+      "step": 735
+    },
+    {
+      "epoch": 1.5578947368421052,
+      "grad_norm": 0.8140954971313477,
+      "learning_rate": 4.94317067635464e-05,
+      "loss": 0.5117893695831299,
+      "step": 740
+    },
+    {
+      "epoch": 1.568421052631579,
+      "grad_norm": 0.8628697991371155,
+      "learning_rate": 4.886348694322145e-05,
+      "loss": 0.48410491943359374,
+      "step": 745
+    },
+    {
+      "epoch": 1.5789473684210527,
+      "grad_norm": 0.7672389149665833,
+      "learning_rate": 4.829541394566942e-05,
+      "loss": 0.4889042854309082,
+      "step": 750
+    },
+    {
+      "epoch": 1.5789473684210527,
+      "eval_loss": 0.5602394938468933,
+      "eval_runtime": 27.2938,
+      "eval_samples_per_second": 14.655,
+      "eval_steps_per_second": 14.655,
+      "step": 750
+    },
+    {
+      "epoch": 1.5894736842105264,
+      "grad_norm": 0.8665842413902283,
+      "learning_rate": 4.772756115856695e-05,
+      "loss": 0.4589890480041504,
+      "step": 755
+    },
+    {
+      "epoch": 1.6,
+      "grad_norm": 0.948796808719635,
+      "learning_rate": 4.7160001941142365e-05,
+      "loss": 0.4775981426239014,
+      "step": 760
+    },
+    {
+      "epoch": 1.6105263157894738,
+      "grad_norm": 0.8106200695037842,
+      "learning_rate": 4.6592809614698556e-05,
+      "loss": 0.5602635860443115,
+      "step": 765
+    },
+    {
+      "epoch": 1.6210526315789475,
+      "grad_norm": 1.2499663829803467,
+      "learning_rate": 4.602605745314087e-05,
+      "loss": 0.5022463321685791,
+      "step": 770
+    },
+    {
+      "epoch": 1.631578947368421,
+      "grad_norm": 0.8327662944793701,
+      "learning_rate": 4.545981867351104e-05,
+      "loss": 0.4951312065124512,
+      "step": 775
+    },
+    {
+      "epoch": 1.6421052631578947,
+      "grad_norm": 0.7749590277671814,
+      "learning_rate": 4.4894166426528524e-05,
+      "loss": 0.4875455379486084,
+      "step": 780
+    },
+    {
+      "epoch": 1.6526315789473685,
+      "grad_norm": 0.764395534992218,
+      "learning_rate": 4.432917378714032e-05,
+      "loss": 0.47612872123718264,
+      "step": 785
+    },
+    {
+      "epoch": 1.663157894736842,
+      "grad_norm": 0.8650004863739014,
+      "learning_rate": 4.3764913745080695e-05,
+      "loss": 0.4810147285461426,
+      "step": 790
+    },
+    {
+      "epoch": 1.6736842105263157,
+      "grad_norm": 0.852967381477356,
+      "learning_rate": 4.32014591954418e-05,
+      "loss": 0.46249094009399416,
+      "step": 795
+    },
+    {
+      "epoch": 1.6842105263157894,
+      "grad_norm": 0.752549946308136,
+      "learning_rate": 4.263888292925664e-05,
+      "loss": 0.463092565536499,
+      "step": 800
+    },
+    {
+      "epoch": 1.6842105263157894,
+      "eval_loss": 0.557573139667511,
+      "eval_runtime": 27.1956,
+      "eval_samples_per_second": 14.708,
+      "eval_steps_per_second": 14.708,
+      "step": 800
+    },
+    {
+      "epoch": 1.694736842105263,
+      "grad_norm": 0.7666327953338623,
+      "learning_rate": 4.207725762409529e-05,
+      "loss": 0.549990177154541,
+      "step": 805
+    },
+    {
+      "epoch": 1.7052631578947368,
+      "grad_norm": 1.2041642665863037,
+      "learning_rate": 4.151665583467604e-05,
+      "loss": 0.473134708404541,
+      "step": 810
+    },
+    {
+      "epoch": 1.7157894736842105,
+      "grad_norm": 0.9291930794715881,
+      "learning_rate": 4.095714998349218e-05,
+      "loss": 0.43207721710205077,
+      "step": 815
+    },
+    {
+      "epoch": 1.7263157894736842,
+      "grad_norm": 0.8421230316162109,
+      "learning_rate": 4.0398812351455955e-05,
+      "loss": 0.42284450531005857,
+      "step": 820
+    },
+    {
+      "epoch": 1.736842105263158,
+      "grad_norm": 0.7550533413887024,
+      "learning_rate": 3.9841715068560835e-05,
+      "loss": 0.4533982276916504,
+      "step": 825
+    },
+    {
+      "epoch": 1.7473684210526317,
+      "grad_norm": 0.7380033731460571,
+      "learning_rate": 3.9285930104563234e-05,
+      "loss": 0.47853550910949705,
+      "step": 830
+    },
+    {
+      "epoch": 1.7578947368421054,
+      "grad_norm": 1.0466768741607666,
+      "learning_rate": 3.873152925968495e-05,
+      "loss": 0.5334872245788574,
+      "step": 835
+    },
+    {
+      "epoch": 1.768421052631579,
+      "grad_norm": 0.8049618601799011,
+      "learning_rate": 3.8178584155337525e-05,
+      "loss": 0.4399109363555908,
+      "step": 840
+    },
+    {
+      "epoch": 1.7789473684210526,
+      "grad_norm": 0.7510071992874146,
+      "learning_rate": 3.762716622486965e-05,
+      "loss": 0.521062707901001,
+      "step": 845
+    },
+    {
+      "epoch": 1.7894736842105263,
+      "grad_norm": 1.6552070379257202,
+      "learning_rate": 3.7077346704338935e-05,
+      "loss": 0.45749530792236326,
+      "step": 850
+    },
+    {
+      "epoch": 1.7894736842105263,
+      "eval_loss": 0.552564799785614,
+      "eval_runtime": 27.3375,
+      "eval_samples_per_second": 14.632,
+      "eval_steps_per_second": 14.632,
+      "step": 850
+    },
+    {
+      "epoch": 1.8,
+      "grad_norm": 0.7396476864814758,
+      "learning_rate": 3.6529196623309115e-05,
+      "loss": 0.38660905361175535,
+      "step": 855
+    },
+    {
+      "epoch": 1.8105263157894735,
+      "grad_norm": 0.8277924656867981,
+      "learning_rate": 3.598278679567397e-05,
+      "loss": 0.5064235210418702,
+      "step": 860
+    },
+    {
+      "epoch": 1.8210526315789473,
+      "grad_norm": 0.9554834365844727,
+      "learning_rate": 3.543818781050906e-05,
+      "loss": 0.4140141010284424,
+      "step": 865
+    },
+    {
+      "epoch": 1.831578947368421,
+      "grad_norm": 0.7180286049842834,
+      "learning_rate": 3.4895470022952536e-05,
+      "loss": 0.5401397705078125,
+      "step": 870
+    },
+    {
+      "epoch": 1.8421052631578947,
+      "grad_norm": 0.972764790058136,
+      "learning_rate": 3.4354703545116185e-05,
+      "loss": 0.504509973526001,
+      "step": 875
+    },
+    {
+      "epoch": 1.8526315789473684,
+      "grad_norm": 0.7626535892486572,
+      "learning_rate": 3.381595823702784e-05,
+      "loss": 0.46036224365234374,
+      "step": 880
+    },
+    {
+      "epoch": 1.8631578947368421,
+      "grad_norm": 0.9077037572860718,
+      "learning_rate": 3.3279303697606354e-05,
+      "loss": 0.4127952098846436,
+      "step": 885
+    },
+    {
+      "epoch": 1.8736842105263158,
+      "grad_norm": 0.764255702495575,
+      "learning_rate": 3.274480925567036e-05,
+      "loss": 0.4129067897796631,
+      "step": 890
+    },
+    {
+      "epoch": 1.8842105263157896,
+      "grad_norm": 0.9645035266876221,
+      "learning_rate": 3.2212543960981845e-05,
+      "loss": 0.486614990234375,
+      "step": 895
+    },
+    {
+      "epoch": 1.8947368421052633,
+      "grad_norm": 0.7822158932685852,
+      "learning_rate": 3.168257657532584e-05,
+      "loss": 0.43874049186706543,
+      "step": 900
+    },
+    {
+      "epoch": 1.8947368421052633,
+      "eval_loss": 0.5511002540588379,
+      "eval_runtime": 27.8563,
+      "eval_samples_per_second": 14.359,
+      "eval_steps_per_second": 14.359,
+      "step": 900
+    },
+    {
+      "epoch": 1.905263157894737,
+      "grad_norm": 0.823441207408905,
+      "learning_rate": 3.115497556362727e-05,
+      "loss": 0.4594260692596436,
+      "step": 905
+    },
+    {
+      "epoch": 1.9157894736842105,
+      "grad_norm": 0.821818470954895,
+      "learning_rate": 3.06298090851062e-05,
+      "loss": 0.4467416763305664,
+      "step": 910
+    },
+    {
+      "epoch": 1.9263157894736842,
+      "grad_norm": 0.8790761232376099,
+      "learning_rate": 3.0107144984472502e-05,
+      "loss": 0.4823316097259521,
+      "step": 915
+    },
+    {
+      "epoch": 1.936842105263158,
+      "grad_norm": 0.7914945483207703,
+      "learning_rate": 2.9587050783161252e-05,
+      "loss": 0.4314936637878418,
+      "step": 920
+    },
+    {
+      "epoch": 1.9473684210526314,
+      "grad_norm": 0.805029571056366,
+      "learning_rate": 2.9069593670609775e-05,
+      "loss": 0.4740307331085205,
+      "step": 925
+    },
+    {
+      "epoch": 1.9578947368421051,
+      "grad_norm": 0.7928342223167419,
+      "learning_rate": 2.8554840495577682e-05,
+      "loss": 0.4155329704284668,
+      "step": 930
+    },
+    {
+      "epoch": 1.9684210526315788,
+      "grad_norm": 0.8041934370994568,
+      "learning_rate": 2.8042857757510877e-05,
+      "loss": 0.5049230098724365,
+      "step": 935
+    },
+    {
+      "epoch": 1.9789473684210526,
+      "grad_norm": 0.8169903755187988,
+      "learning_rate": 2.753371159795065e-05,
+      "loss": 0.3966445684432983,
+      "step": 940
+    },
+    {
+      "epoch": 1.9894736842105263,
+      "grad_norm": 0.86372309923172,
+      "learning_rate": 2.702746779198912e-05,
+      "loss": 0.5153264999389648,
+      "step": 945
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.97676020860672,
+      "learning_rate": 2.6524191739771815e-05,
+      "loss": 0.4318437099456787,
+      "step": 950
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.5508978962898254,
+      "eval_runtime": 27.8163,
+      "eval_samples_per_second": 14.38,
+      "eval_steps_per_second": 14.38,
+      "step": 950
+    },
+    {
+      "epoch": 2.0105263157894737,
+      "grad_norm": 0.7161779999732971,
+      "learning_rate": 2.6023948458048965e-05,
+      "loss": 0.3977535247802734,
+      "step": 955
+    },
+    {
+      "epoch": 2.0210526315789474,
+      "grad_norm": 0.8996826410293579,
+      "learning_rate": 2.552680257177611e-05,
+      "loss": 0.38445310592651366,
+      "step": 960
+    },
+    {
+      "epoch": 2.031578947368421,
+      "grad_norm": 0.7812126278877258,
+      "learning_rate": 2.5032818305765383e-05,
+      "loss": 0.37283446788787844,
+      "step": 965
+    },
+    {
+      "epoch": 2.042105263157895,
+      "grad_norm": 0.7872561812400818,
+      "learning_rate": 2.454205947638852e-05,
+      "loss": 0.3594684600830078,
+      "step": 970
+    },
+    {
+      "epoch": 2.0526315789473686,
+      "grad_norm": 0.8357042670249939,
+      "learning_rate": 2.4054589483332597e-05,
+      "loss": 0.30683672428131104,
+      "step": 975
+    },
+    {
+      "epoch": 2.0631578947368423,
+      "grad_norm": 0.7905634045600891,
+      "learning_rate": 2.3570471301409618e-05,
+      "loss": 0.32751510143280027,
+      "step": 980
+    },
+    {
+      "epoch": 2.0736842105263156,
+      "grad_norm": 1.3322068452835083,
+      "learning_rate": 2.3089767472421e-05,
+      "loss": 0.3961126089096069,
+      "step": 985
+    },
+    {
+      "epoch": 2.0842105263157893,
+      "grad_norm": 0.9144253730773926,
+      "learning_rate": 2.2612540097077935e-05,
+      "loss": 0.3600940704345703,
+      "step": 990
+    },
+    {
+      "epoch": 2.094736842105263,
+      "grad_norm": 0.8574570417404175,
+      "learning_rate": 2.213885082697883e-05,
+      "loss": 0.3191706895828247,
+      "step": 995
+    },
+    {
+      "epoch": 2.1052631578947367,
+      "grad_norm": 0.9719475507736206,
+      "learning_rate": 2.1668760856644703e-05,
+      "loss": 0.39908566474914553,
+      "step": 1000
+    },
+    {
+      "epoch": 2.1052631578947367,
+      "eval_loss": 0.5967375040054321,
+      "eval_runtime": 28.5798,
+      "eval_samples_per_second": 13.996,
+      "eval_steps_per_second": 13.996,
+      "step": 1000
+    },
+    {
+      "epoch": 2.1157894736842104,
+      "grad_norm": 1.0026988983154297,
+      "learning_rate": 2.1202330915613638e-05,
+      "loss": 0.324949836730957,
+      "step": 1005
+    },
+    {
+      "epoch": 2.126315789473684,
+      "grad_norm": 0.9327012896537781,
+      "learning_rate": 2.0739621260595315e-05,
+      "loss": 0.31343927383422854,
+      "step": 1010
+    },
+    {
+      "epoch": 2.136842105263158,
+      "grad_norm": 0.8879086375236511,
+      "learning_rate": 2.028069166768663e-05,
+      "loss": 0.35412182807922366,
+      "step": 1015
+    },
+    {
+      "epoch": 2.1473684210526316,
+      "grad_norm": 1.014451265335083,
+      "learning_rate": 1.982560142464939e-05,
+      "loss": 0.3389745235443115,
+      "step": 1020
+    },
+    {
+      "epoch": 2.1578947368421053,
+      "grad_norm": 1.067197561264038,
+      "learning_rate": 1.937440932325112e-05,
+      "loss": 0.30197973251342775,
+      "step": 1025
+    },
+    {
+      "epoch": 2.168421052631579,
+      "grad_norm": 0.9498446583747864,
+      "learning_rate": 1.8927173651669877e-05,
+      "loss": 0.3828472375869751,
+      "step": 1030
+    },
+    {
+      "epoch": 2.1789473684210527,
+      "grad_norm": 1.0303031206130981,
+      "learning_rate": 1.8483952186964237e-05,
+      "loss": 0.4124176025390625,
+      "step": 1035
+    },
+    {
+      "epoch": 2.1894736842105265,
+      "grad_norm": 0.958342969417572,
+      "learning_rate": 1.804480218760922e-05,
+      "loss": 0.3103924751281738,
+      "step": 1040
+    },
+    {
+      "epoch": 2.2,
+      "grad_norm": 1.0009726285934448,
+      "learning_rate": 1.7609780386099234e-05,
+      "loss": 0.33572826385498045,
+      "step": 1045
+    },
+    {
+      "epoch": 2.2105263157894735,
+      "grad_norm": 1.0434421300888062,
+      "learning_rate": 1.7178942981618945e-05,
+      "loss": 0.3350696086883545,
+      "step": 1050
+    },
+    {
+      "epoch": 2.2105263157894735,
+      "eval_loss": 0.6022932529449463,
+      "eval_runtime": 27.6209,
+      "eval_samples_per_second": 14.482,
+      "eval_steps_per_second": 14.482,
+      "step": 1050
+    },
+    {
+      "epoch": 2.221052631578947,
+      "grad_norm": 1.1546400785446167,
+      "learning_rate": 1.6752345632783135e-05,
+      "loss": 0.34174375534057616,
+      "step": 1055
+    },
+    {
+      "epoch": 2.231578947368421,
+      "grad_norm": 1.0252087116241455,
+      "learning_rate": 1.6330043450446265e-05,
+      "loss": 0.3031753063201904,
+      "step": 1060
+    },
+    {
+      "epoch": 2.2421052631578946,
+      "grad_norm": 1.0577940940856934,
+      "learning_rate": 1.59120909905829e-05,
+      "loss": 0.35698933601379396,
+      "step": 1065
+    },
+    {
+      "epoch": 2.2526315789473683,
+      "grad_norm": 1.175123929977417,
+      "learning_rate": 1.549854224723978e-05,
+      "loss": 0.2965430736541748,
+      "step": 1070
+    },
+    {
+      "epoch": 2.263157894736842,
+      "grad_norm": 0.9692304730415344,
+      "learning_rate": 1.508945064556047e-05,
+      "loss": 0.3899634122848511,
+      "step": 1075
+    },
+    {
+      "epoch": 2.2736842105263158,
+      "grad_norm": 1.0569312572479248,
+      "learning_rate": 1.4684869034883554e-05,
+      "loss": 0.34727933406829836,
+      "step": 1080
+    },
+    {
+      "epoch": 2.2842105263157895,
+      "grad_norm": 0.8666245937347412,
+      "learning_rate": 1.4284849681915158e-05,
+      "loss": 0.3896636009216309,
+      "step": 1085
+    },
+    {
+      "epoch": 2.294736842105263,
+      "grad_norm": 1.0360913276672363,
+      "learning_rate": 1.3889444263976786e-05,
+      "loss": 0.31699607372283933,
+      "step": 1090
+    },
+    {
+      "epoch": 2.305263157894737,
+      "grad_norm": 1.2606230974197388,
+      "learning_rate": 1.3498703862329254e-05,
+      "loss": 0.33367414474487306,
+      "step": 1095
+    },
+    {
+      "epoch": 2.3157894736842106,
+      "grad_norm": 1.2789041996002197,
+      "learning_rate": 1.3112678955573693e-05,
+      "loss": 0.3069949150085449,
+      "step": 1100
+    },
+    {
+      "epoch": 2.3157894736842106,
+      "eval_loss": 0.6039758324623108,
+      "eval_runtime": 27.6614,
+      "eval_samples_per_second": 14.461,
+      "eval_steps_per_second": 14.461,
+      "step": 1100
+    },
+    {
+      "epoch": 2.3263157894736843,
+      "grad_norm": 1.1908059120178223,
+      "learning_rate": 1.2731419413130325e-05,
+      "loss": 0.38684911727905275,
+      "step": 1105
+    },
+    {
+      "epoch": 2.336842105263158,
+      "grad_norm": 1.262253999710083,
+      "learning_rate": 1.2354974488796017e-05,
+      "loss": 0.36271347999572756,
+      "step": 1110
+    },
+    {
+      "epoch": 2.3473684210526318,
+      "grad_norm": 0.9189227819442749,
+      "learning_rate": 1.1983392814381273e-05,
+      "loss": 0.3401113271713257,
+      "step": 1115
+    },
+    {
+      "epoch": 2.3578947368421055,
+      "grad_norm": 1.0726381540298462,
+      "learning_rate": 1.1616722393427704e-05,
+      "loss": 0.33876895904541016,
+      "step": 1120
+    },
+    {
+      "epoch": 2.3684210526315788,
+      "grad_norm": 0.9959712028503418,
+      "learning_rate": 1.125501059500656e-05,
+      "loss": 0.3982860803604126,
+      "step": 1125
+    },
+    {
+      "epoch": 2.3789473684210525,
+      "grad_norm": 0.9985882043838501,
+      "learning_rate": 1.0898304147599231e-05,
+      "loss": 0.3682390213012695,
+      "step": 1130
+    },
+    {
+      "epoch": 2.389473684210526,
+      "grad_norm": 0.9115525484085083,
+      "learning_rate": 1.0546649133060583e-05,
+      "loss": 0.3579556465148926,
+      "step": 1135
+    },
+    {
+      "epoch": 2.4,
+      "grad_norm": 1.1207247972488403,
+      "learning_rate": 1.0200090980665739e-05,
+      "loss": 0.42283267974853517,
+      "step": 1140
+    },
+    {
+      "epoch": 2.4105263157894736,
+      "grad_norm": 1.0433000326156616,
+      "learning_rate": 9.858674461241229e-06,
+      "loss": 0.362597393989563,
+      "step": 1145
+    },
+    {
+      "epoch": 2.4210526315789473,
+      "grad_norm": 0.9558874368667603,
+      "learning_rate": 9.522443681381172e-06,
+      "loss": 0.40636916160583497,
+      "step": 1150
+    },
+    {
+      "epoch": 2.4210526315789473,
+      "eval_loss": 0.6012060642242432,
+      "eval_runtime": 27.6853,
+      "eval_samples_per_second": 14.448,
+      "eval_steps_per_second": 14.448,
+      "step": 1150
+    },
+    {
+      "epoch": 2.431578947368421,
+      "grad_norm": 0.9411963224411011,
+      "learning_rate": 9.191442077749257e-06,
+      "loss": 0.32624173164367676,
+      "step": 1155
+    },
+    {
+      "epoch": 2.442105263157895,
+      "grad_norm": 1.1000866889953613,
+      "learning_rate": 8.86571241146732e-06,
+      "loss": 0.36433489322662355,
+      "step": 1160
+    },
+    {
+      "epoch": 2.4526315789473685,
+      "grad_norm": 0.8302079439163208,
+      "learning_rate": 8.545296762591144e-06,
+      "loss": 0.34126312732696534,
+      "step": 1165
+    },
+    {
+      "epoch": 2.463157894736842,
+      "grad_norm": 1.1295444965362549,
+      "learning_rate": 8.230236524674256e-06,
+      "loss": 0.34795560836791994,
+      "step": 1170
+    },
+    {
+      "epoch": 2.473684210526316,
+      "grad_norm": 1.0424487590789795,
+      "learning_rate": 7.920572399420428e-06,
+      "loss": 0.34096055030822753,
+      "step": 1175
+    },
+    {
+      "epoch": 2.4842105263157896,
+      "grad_norm": 1.1002000570297241,
+      "learning_rate": 7.616344391425534e-06,
+      "loss": 0.3707140922546387,
+      "step": 1180
+    },
+    {
+      "epoch": 2.4947368421052634,
+      "grad_norm": 0.946130096912384,
+      "learning_rate": 7.317591803009472e-06,
+      "loss": 0.3825195074081421,
+      "step": 1185
+    },
+    {
+      "epoch": 2.5052631578947366,
+      "grad_norm": 1.1903181076049805,
+      "learning_rate": 7.024353229138836e-06,
+      "loss": 0.3591669797897339,
+      "step": 1190
+    },
+    {
+      "epoch": 2.515789473684211,
+      "grad_norm": 0.9660072326660156,
+      "learning_rate": 6.736666552440896e-06,
+      "loss": 0.31880993843078614,
+      "step": 1195
+    },
+    {
+      "epoch": 2.526315789473684,
+      "grad_norm": 1.1138476133346558,
+      "learning_rate": 6.454568938309724e-06,
+      "loss": 0.33600804805755613,
+      "step": 1200
+    },
+    {
+      "epoch": 2.526315789473684,
+      "eval_loss": 0.6024701595306396,
+      "eval_runtime": 28.6271,
+      "eval_samples_per_second": 13.973,
+      "eval_steps_per_second": 13.973,
+      "step": 1200
+    },
+    {
+      "epoch": 2.536842105263158,
+      "grad_norm": 1.0256083011627197,
+      "learning_rate": 6.1780968301048406e-06,
+      "loss": 0.30231916904449463,
+      "step": 1205
+    },
+    {
+      "epoch": 2.5473684210526315,
+      "grad_norm": 1.3465616703033447,
+      "learning_rate": 5.907285944443241e-06,
+      "loss": 0.37215938568115237,
+      "step": 1210
+    },
+    {
+      "epoch": 2.557894736842105,
+      "grad_norm": 1.0157089233398438,
+      "learning_rate": 5.642171266585272e-06,
+      "loss": 0.38835783004760743,
+      "step": 1215
+    },
+    {
+      "epoch": 2.568421052631579,
+      "grad_norm": 1.0423297882080078,
+      "learning_rate": 5.3827870459149665e-06,
+      "loss": 0.3684116840362549,
+      "step": 1220
+    },
+    {
+      "epoch": 2.5789473684210527,
+      "grad_norm": 1.2063053846359253,
+      "learning_rate": 5.1291667915154774e-06,
+      "loss": 0.39838123321533203,
+      "step": 1225
+    },
+    {
+      "epoch": 2.5894736842105264,
+      "grad_norm": 1.045510172843933,
+      "learning_rate": 4.88134326784015e-06,
+      "loss": 0.3727046728134155,
+      "step": 1230
+    },
+    {
+      "epoch": 2.6,
+      "grad_norm": 1.079955816268921,
+      "learning_rate": 4.639348490479755e-06,
+      "loss": 0.3064917802810669,
+      "step": 1235
+    },
+    {
+      "epoch": 2.610526315789474,
+      "grad_norm": 1.029150128364563,
+      "learning_rate": 4.403213722026516e-06,
+      "loss": 0.3460820436477661,
+      "step": 1240
+    },
+    {
+      "epoch": 2.6210526315789475,
+      "grad_norm": 1.2160367965698242,
+      "learning_rate": 4.172969468035359e-06,
+      "loss": 0.3394759178161621,
+      "step": 1245
+    },
+    {
+      "epoch": 2.6315789473684212,
+      "grad_norm": 1.292370319366455,
+      "learning_rate": 3.948645473083018e-06,
+      "loss": 0.4295994281768799,
+      "step": 1250
+    },
+    {
+      "epoch": 2.6315789473684212,
+      "eval_loss": 0.6037428975105286,
+      "eval_runtime": 28.4938,
+      "eval_samples_per_second": 14.038,
+      "eval_steps_per_second": 14.038,
+      "step": 1250
+    },
+    {
+      "epoch": 2.6421052631578945,
+      "grad_norm": 0.9467762112617493,
+      "learning_rate": 3.730270716925394e-06,
+      "loss": 0.3685704946517944,
+      "step": 1255
+    },
+    {
+      "epoch": 2.6526315789473687,
+      "grad_norm": 0.9402472376823425,
+      "learning_rate": 3.5178734107537637e-06,
+      "loss": 0.38207206726074217,
+      "step": 1260
+    },
+    {
+      "epoch": 2.663157894736842,
+      "grad_norm": 0.9341251254081726,
+      "learning_rate": 3.311480993550259e-06,
+      "loss": 0.3355901002883911,
+      "step": 1265
+    },
+    {
+      "epoch": 2.6736842105263157,
+      "grad_norm": 1.3845702409744263,
+      "learning_rate": 3.111120128543088e-06,
+      "loss": 0.3235702276229858,
+      "step": 1270
+    },
+    {
+      "epoch": 2.6842105263157894,
+      "grad_norm": 0.9406099319458008,
+      "learning_rate": 2.9168166997620263e-06,
+      "loss": 0.33657886981964114,
+      "step": 1275
+    },
+    {
+      "epoch": 2.694736842105263,
+      "grad_norm": 1.0643633604049683,
+      "learning_rate": 2.7285958086944786e-06,
+      "loss": 0.35396461486816405,
+      "step": 1280
+    },
+    {
+      "epoch": 2.705263157894737,
+      "grad_norm": 0.8600782155990601,
+      "learning_rate": 2.5464817710427414e-06,
+      "loss": 0.3184598445892334,
+      "step": 1285
+    },
+    {
+      "epoch": 2.7157894736842105,
+      "grad_norm": 1.0098100900650024,
+      "learning_rate": 2.370498113582731e-06,
+      "loss": 0.3798511981964111,
+      "step": 1290
+    },
+    {
+      "epoch": 2.7263157894736842,
+      "grad_norm": 1.2423481941223145,
+      "learning_rate": 2.2006675711245818e-06,
+      "loss": 0.36703929901123045,
+      "step": 1295
+    },
+    {
+      "epoch": 2.736842105263158,
+      "grad_norm": 1.097049593925476,
+      "learning_rate": 2.0370120835756513e-06,
+      "loss": 0.2991799354553223,
+      "step": 1300
+    },
+    {
+      "epoch": 2.736842105263158,
+      "eval_loss": 0.6043052077293396,
+      "eval_runtime": 27.8515,
+      "eval_samples_per_second": 14.362,
+      "eval_steps_per_second": 14.362,
+      "step": 1300
+    },
+    {
+      "epoch": 2.7473684210526317,
+      "grad_norm": 1.128627061843872,
+      "learning_rate": 1.8795527931061374e-06,
+      "loss": 0.34659562110900877,
+      "step": 1305
+    },
+    {
+      "epoch": 2.7578947368421054,
+      "grad_norm": 0.958167314529419,
+      "learning_rate": 1.7283100414178078e-06,
+      "loss": 0.3479694128036499,
+      "step": 1310
+    },
+    {
+      "epoch": 2.768421052631579,
+      "grad_norm": 1.2653661966323853,
+      "learning_rate": 1.58330336711609e-06,
+      "loss": 0.3173972606658936,
+      "step": 1315
+    },
+    {
+      "epoch": 2.7789473684210524,
+      "grad_norm": 1.3858840465545654,
+      "learning_rate": 1.4445515031859591e-06,
+      "loss": 0.42389545440673826,
+      "step": 1320
+    },
+    {
+      "epoch": 2.7894736842105265,
+      "grad_norm": 1.142336368560791,
+      "learning_rate": 1.31207237457186e-06,
+      "loss": 0.3316075563430786,
+      "step": 1325
+    },
+    {
+      "epoch": 2.8,
+      "grad_norm": 1.341241478919983,
+      "learning_rate": 1.1858830958620559e-06,
+      "loss": 0.370093846321106,
+      "step": 1330
+    },
+    {
+      "epoch": 2.8105263157894735,
+      "grad_norm": 1.2752244472503662,
+      "learning_rate": 1.0659999690776302e-06,
+      "loss": 0.37927913665771484,
+      "step": 1335
+    },
+    {
+      "epoch": 2.8210526315789473,
+      "grad_norm": 1.2686620950698853,
+      "learning_rate": 9.524384815664699e-07,
+      "loss": 0.31010196208953855,
+      "step": 1340
+    },
+    {
+      "epoch": 2.831578947368421,
+      "grad_norm": 0.9856387376785278,
+      "learning_rate": 8.452133040025345e-07,
+      "loss": 0.37815587520599364,
+      "step": 1345
+    },
+    {
+      "epoch": 2.8421052631578947,
+      "grad_norm": 1.1066813468933105,
+      "learning_rate": 7.443382884905536e-07,
+      "loss": 0.3322570562362671,
+      "step": 1350
+    },
+    {
+      "epoch": 2.8421052631578947,
+      "eval_loss": 0.6025745272636414,
+      "eval_runtime": 27.5996,
+      "eval_samples_per_second": 14.493,
+      "eval_steps_per_second": 14.493,
+      "step": 1350
+    },
+    {
+      "epoch": 2.8526315789473684,
+      "grad_norm": 1.8500337600708008,
+      "learning_rate": 6.498264667765374e-07,
+      "loss": 0.362324595451355,
+      "step": 1355
+    },
+    {
+      "epoch": 2.863157894736842,
+      "grad_norm": 1.127455711364746,
+      "learning_rate": 5.616900485642506e-07,
+      "loss": 0.2964962005615234,
+      "step": 1360
+    },
+    {
+      "epoch": 2.873684210526316,
+      "grad_norm": 0.9335603713989258,
+      "learning_rate": 4.79940419937841e-07,
+      "loss": 0.33919923305511473,
+      "step": 1365
+    },
+    {
+      "epoch": 2.8842105263157896,
+      "grad_norm": 1.0695230960845947,
+      "learning_rate": 4.045881418909547e-07,
+      "loss": 0.33406035900115966,
+      "step": 1370
+    },
+    {
+      "epoch": 2.8947368421052633,
+      "grad_norm": 1.2964420318603516,
+      "learning_rate": 3.356429489623558e-07,
+      "loss": 0.3467694282531738,
+      "step": 1375
+    },
+    {
+      "epoch": 2.905263157894737,
+      "grad_norm": 0.9756618738174438,
+      "learning_rate": 2.7311374797835966e-07,
+      "loss": 0.27626986503601075,
+      "step": 1380
+    },
+    {
+      "epoch": 2.9157894736842103,
+      "grad_norm": 1.125270962715149,
+      "learning_rate": 2.170086169021923e-07,
+      "loss": 0.350561261177063,
+      "step": 1385
+    },
+    {
+      "epoch": 2.9263157894736844,
+      "grad_norm": 1.0733071565628052,
+      "learning_rate": 1.6733480379041434e-07,
+      "loss": 0.3240832328796387,
+      "step": 1390
+    },
+    {
+      "epoch": 2.9368421052631577,
+      "grad_norm": 1.8108630180358887,
+      "learning_rate": 1.240987258565751e-07,
+      "loss": 0.3057235240936279,
+      "step": 1395
+    },
+    {
+      "epoch": 2.9473684210526314,
+      "grad_norm": 1.3284285068511963,
+      "learning_rate": 8.730596864218177e-08,
+      "loss": 0.34192938804626466,
+      "step": 1400
+    },
+    {
+      "epoch": 2.9473684210526314,
+      "eval_loss": 0.6027438640594482,
+      "eval_runtime": 27.589,
+      "eval_samples_per_second": 14.499,
+      "eval_steps_per_second": 14.499,
+      "step": 1400
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 1425,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 50,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.3629504961622835e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1400/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:757b7d78e07bd589f7cd9db1a1a3019f58127f01f91cba75e4a6cbb7ea4b4a6c
+size 5713

checkpoint-1425/README.md

@@ -0,0 +1,210 @@
+---
+base_model: unsloth/gemma-4-12b-it
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/gemma-4-12b-it
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

checkpoint-1425/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Gemma4UnifiedForConditionalGeneration",
+    "parent_library": "transformers.models.gemma4_unified.modeling_gemma4_unified",
+    "unsloth_fixed": true
+  },
+  "base_model_name_or_path": "unsloth/gemma-4-12b-it",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": "(?:.*?(?:language|text).*?(?:self_attn|attention|attn|mixer|mlp|feed_forward|ffn|dense|mixer).*?(?:q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj|embedding_projection))|(?:\\bmodel\\.layers\\.[\\d]{1,}\\.(?:self_attn|attention|attn|mixer|mlp|feed_forward|ffn|dense|mixer)\\.(?:(?:q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj|embedding_projection)))",
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

checkpoint-1425/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4611adeed740ccdb4edf6676e3e71fc4a38013ed53f11c89d0419ce2950584f1
+size 262373216

checkpoint-1425/chat_template.jinja

@@ -0,0 +1,70 @@
+{{ bos_token }}{%- macro strip_thinking(text) -%}
+    {%- set ns = namespace(result='') -%}
+    {%- for part in text.split('<channel|>') -%}
+        {%- if '<|channel>' in part -%}
+            {%- set ns.result = ns.result + part.split('<|channel>')[0] -%}
+        {%- else -%}
+            {%- set ns.result = ns.result + part -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- ns.result | trim -}}
+{%- endmacro -%}
+{%- set thinking = enable_thinking is defined and enable_thinking -%}
+{%- set loop_messages = messages -%}
+{%- if messages[0]['role'] in ['system', 'developer'] or thinking -%}
+    {{ '<|turn>system
+' }}
+    {%- if thinking -%}
+        {{ '<|think|>
+' }}
+    {%- endif -%}
+    {%- if messages[0]['role'] in ['system', 'developer'] -%}
+        {{ messages[0]['content'] | trim }}
+        {%- set loop_messages = messages[1:] -%}
+    {%- endif -%}
+    {{ '<turn|>
+' }}
+{%- endif -%}
+{%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
+    {%- if (message['role'] == 'assistant') -%}
+        {%- set role = "model" -%}
+    {%- else -%}
+        {%- set role = message['role'] -%}
+    {%- endif -%}
+    {{ '<|turn>' + role + '
+' }}
+    {%- if message['content'] is string -%}
+        {%- if role == "model" -%}
+            {{ strip_thinking(message['content']) }}
+        {%- else -%}
+            {{ message['content'] | trim }}
+        {%- endif -%}
+    {%- elif message['content'] is iterable -%}
+        {%- for item in message['content'] -%}
+            {%- if item['type'] == 'audio' -%}
+                {{ '<|audio|>' }}
+            {%- elif item['type'] == 'image' -%}
+                {{ '<|image|>' }}
+            {%- elif item['type'] == 'video' -%}
+                {{ '<|video|>' }}
+            {%- elif item['type'] == 'text' -%}
+                {%- if role == "model" -%}
+                    {{ strip_thinking(item['text']) }}
+                {%- else -%}
+                    {{ item['text'] | trim }}
+                {%- endif -%}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{ raise_exception("Invalid content type") }}
+    {%- endif -%}
+    {{ '<turn|>
+' }}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{'<|turn>model
+'}}
+{%- endif -%}

checkpoint-1425/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b4731990d49a99f765469cd819af2eb409a4d999213888f5ff8934b62ec3dca
+size 133962277

checkpoint-1425/processor_config.json

@@ -0,0 +1,62 @@
+{
+  "audio_ms_per_token": 40,
+  "audio_seq_length": 750,
+  "feature_extractor": {
+    "audio_samples_per_token": 640,
+    "feature_extractor_type": "Gemma4UnifiedAudioFeatureExtractor",
+    "feature_size": 640,
+    "padding_side": "right",
+    "padding_value": 0.0,
+    "return_attention_mask": true,
+    "sampling_rate": 16000
+  },
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": false,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_processor_type": "Gemma4UnifiedImageProcessor",
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 280,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098
+  },
+  "image_seq_length": 280,
+  "processor_class": "Gemma4UnifiedProcessor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 70,
+    "num_frames": 32,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "video_processor_type": "Gemma4UnifiedVideoProcessor"
+  }
+}

checkpoint-1425/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e79e6176793184a411bc49e872d78de6d894c1cb767ab56d61843c1a459e3493
+size 14645

checkpoint-1425/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:423ee3b87236ef862966fb5d2c5f0c4b6f2667cca9d2185c683c9dcb3becb6f2
+size 1465

checkpoint-1425/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc8d3a0ce36466ccc1278bf987df5f71db1719b9ca6b4118264f45cb627bfe0f
+size 32169626

checkpoint-1425/tokenizer_config.json

@@ -0,0 +1,290 @@
+{
+  "audio_token": "<|audio|>",
+  "backend": "tokenizers",
+  "boa_token": "<|audio>",
+  "boi_token": "<|image>",
+  "bos_token": "<bos>",
+  "eoa_token": "<audio|>",
+  "eoc_token": "<channel|>",
+  "eoi_token": "<image|>",
+  "eos_token": "<turn|>",
+  "eot_token": "<turn|>",
+  "escape_token": "<|\"|>",
+  "etc_token": "<tool_call|>",
+  "etd_token": "<tool|>",
+  "etr_token": "<tool_response|>",
+  "extra_special_tokens": [
+    "<|video|>"
+  ],
+  "image_token": "<|image|>",
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "<mask>",
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_token": "<|audio|>",
+    "boa_token": "<|audio>",
+    "boi_token": "<|image>",
+    "eoa_token": "<audio|>",
+    "eoc_token": "<channel|>",
+    "eoi_token": "<image|>",
+    "eot_token": "<turn|>",
+    "escape_token": "<|\"|>",
+    "etc_token": "<tool_call|>",
+    "etd_token": "<tool|>",
+    "etr_token": "<tool_response|>",
+    "image_token": "<|image|>",
+    "soc_token": "<|channel>",
+    "sot_token": "<|turn>",
+    "stc_token": "<|tool_call>",
+    "std_token": "<|tool>",
+    "str_token": "<|tool_response>",
+    "think_token": "<|think|>"
+  },
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "processor_class": "Gemma4UnifiedProcessor",
+  "response_schema": {
+    "properties": {
+      "content": {
+        "type": "string"
+      },
+      "role": {
+        "const": "assistant"
+      },
+      "thinking": {
+        "type": "string"
+      },
+      "tool_calls": {
+        "items": {
+          "properties": {
+            "function": {
+              "properties": {
+                "arguments": {
+                  "additionalProperties": {},
+                  "type": "object",
+                  "x-parser": "gemma4-tool-call"
+                },
+                "name": {
+                  "type": "string"
+                }
+              },
+              "type": "object",
+              "x-regex": "call\\:(?P<name>\\w+)(?P<arguments>\\{.*\\})"
+            },
+            "type": {
+              "const": "function"
+            }
+          },
+          "type": "object"
+        },
+        "type": "array",
+        "x-regex-iterator": "<\\|tool_call>(.*?)<tool_call\\|>"
+      }
+    },
+    "type": "object",
+    "x-regex": "(\\<\\|channel\\>thought\\n(?P<thinking>.*?)\\<channel\\|\\>)?(?P<tool_calls>\\<\\|tool_call\\>.*\\<tool_call\\|\\>)?(?P<content>(?:(?!\\<turn\\|\\>)(?!\\<\\|tool_response\\>).)+)?(?:\\<turn\\|\\>|\\<\\|tool_response\\>)?"
+  },
+  "soc_token": "<|channel>",
+  "sot_token": "<|turn>",
+  "stc_token": "<|tool_call>",
+  "std_token": "<|tool>",
+  "str_token": "<|tool_response>",
+  "think_token": "<|think|>",
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "1": {
+      "content": "<eos>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "2": {
+      "content": "<bos>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "46": {
+      "content": "<|tool>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "47": {
+      "content": "<tool|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "48": {
+      "content": "<|tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "49": {
+      "content": "<tool_call|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "50": {
+      "content": "<|tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "51": {
+      "content": "<tool_response|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "52": {
+      "content": "<|\"|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "98": {
+      "content": "<|think|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "100": {
+      "content": "<|channel>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "101": {
+      "content": "<channel|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "105": {
+      "content": "<|turn>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "106": {
+      "content": "<turn|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "255999": {
+      "content": "<|image>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "256000": {
+      "content": "<|audio>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258880": {
+      "content": "<|image|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258881": {
+      "content": "<|audio|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258882": {
+      "content": "<image|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258883": {
+      "content": "<audio|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258884": {
+      "content": "<|video|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  }
+}

checkpoint-1425/trainer_state.json

@@ -0,0 +1,2261 @@
+{
+  "best_global_step": 950,
+  "best_metric": 0.5508978962898254,
+  "best_model_checkpoint": "/output/checkpoint-950",
+  "epoch": 3.0,
+  "eval_steps": 50,
+  "global_step": 1425,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.010526315789473684,
+      "grad_norm": 2.0839247703552246,
+      "learning_rate": 9.302325581395349e-06,
+      "loss": 2.646485137939453,
+      "step": 5
+    },
+    {
+      "epoch": 0.021052631578947368,
+      "grad_norm": 3.253295660018921,
+      "learning_rate": 2.0930232558139536e-05,
+      "loss": 2.9888063430786134,
+      "step": 10
+    },
+    {
+      "epoch": 0.031578947368421054,
+      "grad_norm": 2.4486842155456543,
+      "learning_rate": 3.2558139534883724e-05,
+      "loss": 2.2748624801635744,
+      "step": 15
+    },
+    {
+      "epoch": 0.042105263157894736,
+      "grad_norm": 2.5299599170684814,
+      "learning_rate": 4.418604651162791e-05,
+      "loss": 1.9410686492919922,
+      "step": 20
+    },
+    {
+      "epoch": 0.05263157894736842,
+      "grad_norm": 1.6780576705932617,
+      "learning_rate": 5.5813953488372095e-05,
+      "loss": 1.3348378181457519,
+      "step": 25
+    },
+    {
+      "epoch": 0.06315789473684211,
+      "grad_norm": 0.7812225222587585,
+      "learning_rate": 6.744186046511628e-05,
+      "loss": 0.8430736541748047,
+      "step": 30
+    },
+    {
+      "epoch": 0.07368421052631578,
+      "grad_norm": 0.8205987215042114,
+      "learning_rate": 7.906976744186047e-05,
+      "loss": 0.8522604942321778,
+      "step": 35
+    },
+    {
+      "epoch": 0.08421052631578947,
+      "grad_norm": 0.6925882697105408,
+      "learning_rate": 9.069767441860465e-05,
+      "loss": 0.6957129955291748,
+      "step": 40
+    },
+    {
+      "epoch": 0.09473684210526316,
+      "grad_norm": 0.7553223371505737,
+      "learning_rate": 9.999987081161148e-05,
+      "loss": 0.8302087783813477,
+      "step": 45
+    },
+    {
+      "epoch": 0.10526315789473684,
+      "grad_norm": 0.6056613922119141,
+      "learning_rate": 9.999534928810904e-05,
+      "loss": 0.6982179641723633,
+      "step": 50
+    },
+    {
+      "epoch": 0.10526315789473684,
+      "eval_loss": 0.7021176218986511,
+      "eval_runtime": 27.8753,
+      "eval_samples_per_second": 14.35,
+      "eval_steps_per_second": 14.35,
+      "step": 50
+    },
+    {
+      "epoch": 0.11578947368421053,
+      "grad_norm": 1.4581637382507324,
+      "learning_rate": 9.998436901275022e-05,
+      "loss": 0.7925346374511719,
+      "step": 55
+    },
+    {
+      "epoch": 0.12631578947368421,
+      "grad_norm": 0.6449484825134277,
+      "learning_rate": 9.99669314040444e-05,
+      "loss": 0.7095213413238526,
+      "step": 60
+    },
+    {
+      "epoch": 0.1368421052631579,
+      "grad_norm": 0.6936064958572388,
+      "learning_rate": 9.994303871470489e-05,
+      "loss": 0.6608465194702149,
+      "step": 65
+    },
+    {
+      "epoch": 0.14736842105263157,
+      "grad_norm": 0.5944413542747498,
+      "learning_rate": 9.991269403135783e-05,
+      "loss": 0.6789331436157227,
+      "step": 70
+    },
+    {
+      "epoch": 0.15789473684210525,
+      "grad_norm": 0.7518417835235596,
+      "learning_rate": 9.987590127414344e-05,
+      "loss": 0.669992208480835,
+      "step": 75
+    },
+    {
+      "epoch": 0.16842105263157894,
+      "grad_norm": 0.7772454619407654,
+      "learning_rate": 9.98326651962096e-05,
+      "loss": 0.6882061958312988,
+      "step": 80
+    },
+    {
+      "epoch": 0.17894736842105263,
+      "grad_norm": 0.7076404690742493,
+      "learning_rate": 9.978299138309781e-05,
+      "loss": 0.6435281276702881,
+      "step": 85
+    },
+    {
+      "epoch": 0.18947368421052632,
+      "grad_norm": 0.6353256702423096,
+      "learning_rate": 9.972688625202164e-05,
+      "loss": 0.6035995960235596,
+      "step": 90
+    },
+    {
+      "epoch": 0.2,
+      "grad_norm": 0.6928842663764954,
+      "learning_rate": 9.966435705103765e-05,
+      "loss": 0.6339406967163086,
+      "step": 95
+    },
+    {
+      "epoch": 0.21052631578947367,
+      "grad_norm": 1.1047526597976685,
+      "learning_rate": 9.959541185810906e-05,
+      "loss": 0.5951476097106934,
+      "step": 100
+    },
+    {
+      "epoch": 0.21052631578947367,
+      "eval_loss": 0.6416817307472229,
+      "eval_runtime": 27.0456,
+      "eval_samples_per_second": 14.79,
+      "eval_steps_per_second": 14.79,
+      "step": 100
+    },
+    {
+      "epoch": 0.22105263157894736,
+      "grad_norm": 0.6272938847541809,
+      "learning_rate": 9.952005958006217e-05,
+      "loss": 0.6947292804718017,
+      "step": 105
+    },
+    {
+      "epoch": 0.23157894736842105,
+      "grad_norm": 0.6077613234519958,
+      "learning_rate": 9.943830995143577e-05,
+      "loss": 0.7127768993377686,
+      "step": 110
+    },
+    {
+      "epoch": 0.24210526315789474,
+      "grad_norm": 0.6263899803161621,
+      "learning_rate": 9.935017353322347e-05,
+      "loss": 0.643881893157959,
+      "step": 115
+    },
+    {
+      "epoch": 0.25263157894736843,
+      "grad_norm": 0.6517646908760071,
+      "learning_rate": 9.925566171150945e-05,
+      "loss": 0.6408101081848144,
+      "step": 120
+    },
+    {
+      "epoch": 0.2631578947368421,
+      "grad_norm": 0.5079100728034973,
+      "learning_rate": 9.915478669599747e-05,
+      "loss": 0.5704734802246094,
+      "step": 125
+    },
+    {
+      "epoch": 0.2736842105263158,
+      "grad_norm": 0.6810751557350159,
+      "learning_rate": 9.904756151843353e-05,
+      "loss": 0.7103314876556397,
+      "step": 130
+    },
+    {
+      "epoch": 0.28421052631578947,
+      "grad_norm": 0.6985617876052856,
+      "learning_rate": 9.893400003092237e-05,
+      "loss": 0.7399785995483399,
+      "step": 135
+    },
+    {
+      "epoch": 0.29473684210526313,
+      "grad_norm": 0.611039936542511,
+      "learning_rate": 9.881411690413796e-05,
+      "loss": 0.6211562633514405,
+      "step": 140
+    },
+    {
+      "epoch": 0.30526315789473685,
+      "grad_norm": 0.6540597677230835,
+      "learning_rate": 9.868792762542814e-05,
+      "loss": 0.6841907501220703,
+      "step": 145
+    },
+    {
+      "epoch": 0.3157894736842105,
+      "grad_norm": 0.6657130122184753,
+      "learning_rate": 9.855544849681404e-05,
+      "loss": 0.6384834289550781,
+      "step": 150
+    },
+    {
+      "epoch": 0.3157894736842105,
+      "eval_loss": 0.6154849529266357,
+      "eval_runtime": 28.9994,
+      "eval_samples_per_second": 13.793,
+      "eval_steps_per_second": 13.793,
+      "step": 150
+    },
+    {
+      "epoch": 0.3263157894736842,
+      "grad_norm": 0.6323176622390747,
+      "learning_rate": 9.841669663288391e-05,
+      "loss": 0.6482361793518067,
+      "step": 155
+    },
+    {
+      "epoch": 0.3368421052631579,
+      "grad_norm": 0.6461365818977356,
+      "learning_rate": 9.82716899585822e-05,
+      "loss": 0.6088940620422363,
+      "step": 160
+    },
+    {
+      "epoch": 0.3473684210526316,
+      "grad_norm": 0.518051028251648,
+      "learning_rate": 9.812044720689387e-05,
+      "loss": 0.6474239826202393,
+      "step": 165
+    },
+    {
+      "epoch": 0.35789473684210527,
+      "grad_norm": 0.6599591374397278,
+      "learning_rate": 9.796298791642435e-05,
+      "loss": 0.639539098739624,
+      "step": 170
+    },
+    {
+      "epoch": 0.3684210526315789,
+      "grad_norm": 0.6555272340774536,
+      "learning_rate": 9.779933242887542e-05,
+      "loss": 0.6650691986083984,
+      "step": 175
+    },
+    {
+      "epoch": 0.37894736842105264,
+      "grad_norm": 0.5666244626045227,
+      "learning_rate": 9.762950188641728e-05,
+      "loss": 0.6851089000701904,
+      "step": 180
+    },
+    {
+      "epoch": 0.3894736842105263,
+      "grad_norm": 0.7169855833053589,
+      "learning_rate": 9.745351822895727e-05,
+      "loss": 0.5861614227294922,
+      "step": 185
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 0.5155227184295654,
+      "learning_rate": 9.727140419130553e-05,
+      "loss": 0.6075415134429931,
+      "step": 190
+    },
+    {
+      "epoch": 0.4105263157894737,
+      "grad_norm": 0.8202889561653137,
+      "learning_rate": 9.708318330023798e-05,
+      "loss": 0.6071089267730713,
+      "step": 195
+    },
+    {
+      "epoch": 0.42105263157894735,
+      "grad_norm": 0.4622310996055603,
+      "learning_rate": 9.688887987145691e-05,
+      "loss": 0.5945972919464111,
+      "step": 200
+    },
+    {
+      "epoch": 0.42105263157894735,
+      "eval_loss": 0.6035570502281189,
+      "eval_runtime": 27.8485,
+      "eval_samples_per_second": 14.363,
+      "eval_steps_per_second": 14.363,
+      "step": 200
+    },
+    {
+      "epoch": 0.43157894736842106,
+      "grad_norm": 0.6781761050224304,
+      "learning_rate": 9.668851900644975e-05,
+      "loss": 0.570584774017334,
+      "step": 205
+    },
+    {
+      "epoch": 0.4421052631578947,
+      "grad_norm": 0.5482363104820251,
+      "learning_rate": 9.648212658924625e-05,
+      "loss": 0.6329983234405517,
+      "step": 210
+    },
+    {
+      "epoch": 0.45263157894736844,
+      "grad_norm": 0.6346020698547363,
+      "learning_rate": 9.62697292830746e-05,
+      "loss": 0.6306288242340088,
+      "step": 215
+    },
+    {
+      "epoch": 0.4631578947368421,
+      "grad_norm": 0.7032224535942078,
+      "learning_rate": 9.6051354526917e-05,
+      "loss": 0.6967363357543945,
+      "step": 220
+    },
+    {
+      "epoch": 0.47368421052631576,
+      "grad_norm": 0.4979580044746399,
+      "learning_rate": 9.582703053196464e-05,
+      "loss": 0.5608481884002685,
+      "step": 225
+    },
+    {
+      "epoch": 0.4842105263157895,
+      "grad_norm": 0.7327916622161865,
+      "learning_rate": 9.55967862779735e-05,
+      "loss": 0.584914255142212,
+      "step": 230
+    },
+    {
+      "epoch": 0.49473684210526314,
+      "grad_norm": 0.7519303560256958,
+      "learning_rate": 9.536065150952025e-05,
+      "loss": 0.6214882850646972,
+      "step": 235
+    },
+    {
+      "epoch": 0.5052631578947369,
+      "grad_norm": 0.7328101396560669,
+      "learning_rate": 9.511865673215986e-05,
+      "loss": 0.6112482070922851,
+      "step": 240
+    },
+    {
+      "epoch": 0.5157894736842106,
+      "grad_norm": 0.5348976254463196,
+      "learning_rate": 9.487083320848454e-05,
+      "loss": 0.6417016983032227,
+      "step": 245
+    },
+    {
+      "epoch": 0.5263157894736842,
+      "grad_norm": 0.5690417885780334,
+      "learning_rate": 9.461721295408505e-05,
+      "loss": 0.5277723789215087,
+      "step": 250
+    },
+    {
+      "epoch": 0.5263157894736842,
+      "eval_loss": 0.5920669436454773,
+      "eval_runtime": 27.6132,
+      "eval_samples_per_second": 14.486,
+      "eval_steps_per_second": 14.486,
+      "step": 250
+    },
+    {
+      "epoch": 0.5368421052631579,
+      "grad_norm": 0.5612736344337463,
+      "learning_rate": 9.435782873341474e-05,
+      "loss": 0.5604990959167481,
+      "step": 255
+    },
+    {
+      "epoch": 0.5473684210526316,
+      "grad_norm": 0.6084933876991272,
+      "learning_rate": 9.409271405555677e-05,
+      "loss": 0.5558807373046875,
+      "step": 260
+    },
+    {
+      "epoch": 0.5578947368421052,
+      "grad_norm": 0.6289730668067932,
+      "learning_rate": 9.382190316989518e-05,
+      "loss": 0.5646376609802246,
+      "step": 265
+    },
+    {
+      "epoch": 0.5684210526315789,
+      "grad_norm": 0.6343289613723755,
+      "learning_rate": 9.354543106169029e-05,
+      "loss": 0.6692690372467041,
+      "step": 270
+    },
+    {
+      "epoch": 0.5789473684210527,
+      "grad_norm": 1.0037399530410767,
+      "learning_rate": 9.326333344755912e-05,
+      "loss": 0.6066013813018799,
+      "step": 275
+    },
+    {
+      "epoch": 0.5894736842105263,
+      "grad_norm": 0.8138774633407593,
+      "learning_rate": 9.297564677086118e-05,
+      "loss": 0.6074648857116699,
+      "step": 280
+    },
+    {
+      "epoch": 0.6,
+      "grad_norm": 1.0000449419021606,
+      "learning_rate": 9.268240819699054e-05,
+      "loss": 0.5891916275024414,
+      "step": 285
+    },
+    {
+      "epoch": 0.6105263157894737,
+      "grad_norm": 0.7454043030738831,
+      "learning_rate": 9.238365560857447e-05,
+      "loss": 0.6280234336853028,
+      "step": 290
+    },
+    {
+      "epoch": 0.6210526315789474,
+      "grad_norm": 0.756981611251831,
+      "learning_rate": 9.207942760057958e-05,
+      "loss": 0.6314973831176758,
+      "step": 295
+    },
+    {
+      "epoch": 0.631578947368421,
+      "grad_norm": 0.7583857178688049,
+      "learning_rate": 9.176976347532575e-05,
+      "loss": 0.6410815715789795,
+      "step": 300
+    },
+    {
+      "epoch": 0.631578947368421,
+      "eval_loss": 0.5863876342773438,
+      "eval_runtime": 27.6386,
+      "eval_samples_per_second": 14.472,
+      "eval_steps_per_second": 14.472,
+      "step": 300
+    },
+    {
+      "epoch": 0.6421052631578947,
+      "grad_norm": 0.791587769985199,
+      "learning_rate": 9.145470323740885e-05,
+      "loss": 0.5841981410980225,
+      "step": 305
+    },
+    {
+      "epoch": 0.6526315789473685,
+      "grad_norm": 1.1765379905700684,
+      "learning_rate": 9.113428758853268e-05,
+      "loss": 0.5464617729187011,
+      "step": 310
+    },
+    {
+      "epoch": 0.6631578947368421,
+      "grad_norm": 3.7840137481689453,
+      "learning_rate": 9.080855792225076e-05,
+      "loss": 0.6217821121215821,
+      "step": 315
+    },
+    {
+      "epoch": 0.6736842105263158,
+      "grad_norm": 1.2177997827529907,
+      "learning_rate": 9.047755631861884e-05,
+      "loss": 0.6125752449035644,
+      "step": 320
+    },
+    {
+      "epoch": 0.6842105263157895,
+      "grad_norm": 0.6323914527893066,
+      "learning_rate": 9.014132553875878e-05,
+      "loss": 0.5617509365081788,
+      "step": 325
+    },
+    {
+      "epoch": 0.6947368421052632,
+      "grad_norm": 0.9683524370193481,
+      "learning_rate": 8.979990901933428e-05,
+      "loss": 0.6395817756652832,
+      "step": 330
+    },
+    {
+      "epoch": 0.7052631578947368,
+      "grad_norm": 0.6486535668373108,
+      "learning_rate": 8.945335086693942e-05,
+      "loss": 0.6556002616882324,
+      "step": 335
+    },
+    {
+      "epoch": 0.7157894736842105,
+      "grad_norm": 1.642112374305725,
+      "learning_rate": 8.910169585240078e-05,
+      "loss": 0.588761568069458,
+      "step": 340
+    },
+    {
+      "epoch": 0.7263157894736842,
+      "grad_norm": 1.007238507270813,
+      "learning_rate": 8.874498940499346e-05,
+      "loss": 0.5744077682495117,
+      "step": 345
+    },
+    {
+      "epoch": 0.7368421052631579,
+      "grad_norm": 0.4945877194404602,
+      "learning_rate": 8.83832776065723e-05,
+      "loss": 0.5382843494415284,
+      "step": 350
+    },
+    {
+      "epoch": 0.7368421052631579,
+      "eval_loss": 0.5782474279403687,
+      "eval_runtime": 27.574,
+      "eval_samples_per_second": 14.506,
+      "eval_steps_per_second": 14.506,
+      "step": 350
+    },
+    {
+      "epoch": 0.7473684210526316,
+      "grad_norm": 0.7869954109191895,
+      "learning_rate": 8.801660718561875e-05,
+      "loss": 0.5697366237640381,
+      "step": 355
+    },
+    {
+      "epoch": 0.7578947368421053,
+      "grad_norm": 0.48311182856559753,
+      "learning_rate": 8.7645025511204e-05,
+      "loss": 0.6168550491333008,
+      "step": 360
+    },
+    {
+      "epoch": 0.7684210526315789,
+      "grad_norm": 0.5001690983772278,
+      "learning_rate": 8.726858058686968e-05,
+      "loss": 0.5362951278686523,
+      "step": 365
+    },
+    {
+      "epoch": 0.7789473684210526,
+      "grad_norm": 0.5915603041648865,
+      "learning_rate": 8.688732104442632e-05,
+      "loss": 0.6041481018066406,
+      "step": 370
+    },
+    {
+      "epoch": 0.7894736842105263,
+      "grad_norm": 0.5077712535858154,
+      "learning_rate": 8.650129613767075e-05,
+      "loss": 0.5257850646972656,
+      "step": 375
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.7458930015563965,
+      "learning_rate": 8.611055573602323e-05,
+      "loss": 0.6048945903778076,
+      "step": 380
+    },
+    {
+      "epoch": 0.8105263157894737,
+      "grad_norm": 0.6072155237197876,
+      "learning_rate": 8.571515031808484e-05,
+      "loss": 0.5505197048187256,
+      "step": 385
+    },
+    {
+      "epoch": 0.8210526315789474,
+      "grad_norm": 0.5989276170730591,
+      "learning_rate": 8.531513096511646e-05,
+      "loss": 0.528736162185669,
+      "step": 390
+    },
+    {
+      "epoch": 0.8315789473684211,
+      "grad_norm": 0.47586777806282043,
+      "learning_rate": 8.491054935443954e-05,
+      "loss": 0.5494171142578125,
+      "step": 395
+    },
+    {
+      "epoch": 0.8421052631578947,
+      "grad_norm": 0.6332405209541321,
+      "learning_rate": 8.450145775276024e-05,
+      "loss": 0.605548906326294,
+      "step": 400
+    },
+    {
+      "epoch": 0.8421052631578947,
+      "eval_loss": 0.5717179775238037,
+      "eval_runtime": 27.9225,
+      "eval_samples_per_second": 14.325,
+      "eval_steps_per_second": 14.325,
+      "step": 400
+    },
+    {
+      "epoch": 0.8526315789473684,
+      "grad_norm": 0.873306155204773,
+      "learning_rate": 8.40879090094171e-05,
+      "loss": 0.6015737056732178,
+      "step": 405
+    },
+    {
+      "epoch": 0.8631578947368421,
+      "grad_norm": 0.6360308527946472,
+      "learning_rate": 8.366995654955375e-05,
+      "loss": 0.5689235687255859,
+      "step": 410
+    },
+    {
+      "epoch": 0.8736842105263158,
+      "grad_norm": 0.7359456419944763,
+      "learning_rate": 8.324765436721688e-05,
+      "loss": 0.6225551128387451,
+      "step": 415
+    },
+    {
+      "epoch": 0.8842105263157894,
+      "grad_norm": 0.6135861873626709,
+      "learning_rate": 8.282105701838106e-05,
+      "loss": 0.567096471786499,
+      "step": 420
+    },
+    {
+      "epoch": 0.8947368421052632,
+      "grad_norm": 0.5699036717414856,
+      "learning_rate": 8.239021961390078e-05,
+      "loss": 0.5435199737548828,
+      "step": 425
+    },
+    {
+      "epoch": 0.9052631578947369,
+      "grad_norm": 0.6312369704246521,
+      "learning_rate": 8.195519781239079e-05,
+      "loss": 0.6320878028869629,
+      "step": 430
+    },
+    {
+      "epoch": 0.9157894736842105,
+      "grad_norm": 0.7718231678009033,
+      "learning_rate": 8.151604781303577e-05,
+      "loss": 0.5115500450134277,
+      "step": 435
+    },
+    {
+      "epoch": 0.9263157894736842,
+      "grad_norm": 0.7022733688354492,
+      "learning_rate": 8.107282634833015e-05,
+      "loss": 0.5616052627563477,
+      "step": 440
+    },
+    {
+      "epoch": 0.9368421052631579,
+      "grad_norm": 0.6376118659973145,
+      "learning_rate": 8.06255906767489e-05,
+      "loss": 0.6315895080566406,
+      "step": 445
+    },
+    {
+      "epoch": 0.9473684210526315,
+      "grad_norm": 0.5550912618637085,
+      "learning_rate": 8.01743985753506e-05,
+      "loss": 0.597140884399414,
+      "step": 450
+    },
+    {
+      "epoch": 0.9473684210526315,
+      "eval_loss": 0.5748050212860107,
+      "eval_runtime": 27.2437,
+      "eval_samples_per_second": 14.682,
+      "eval_steps_per_second": 14.682,
+      "step": 450
+    },
+    {
+      "epoch": 0.9578947368421052,
+      "grad_norm": 0.5438539385795593,
+      "learning_rate": 7.971930833231338e-05,
+      "loss": 0.5860416889190674,
+      "step": 455
+    },
+    {
+      "epoch": 0.968421052631579,
+      "grad_norm": 0.7530905604362488,
+      "learning_rate": 7.926037873940469e-05,
+      "loss": 0.5889331340789795,
+      "step": 460
+    },
+    {
+      "epoch": 0.9789473684210527,
+      "grad_norm": 0.6347785592079163,
+      "learning_rate": 7.879766908438638e-05,
+      "loss": 0.5784619808197021,
+      "step": 465
+    },
+    {
+      "epoch": 0.9894736842105263,
+      "grad_norm": 0.6019025444984436,
+      "learning_rate": 7.83312391433553e-05,
+      "loss": 0.5674707412719726,
+      "step": 470
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.5169152617454529,
+      "learning_rate": 7.786114917302118e-05,
+      "loss": 0.6072670936584472,
+      "step": 475
+    },
+    {
+      "epoch": 1.0105263157894737,
+      "grad_norm": 0.5598861575126648,
+      "learning_rate": 7.738745990292208e-05,
+      "loss": 0.5027533531188965,
+      "step": 480
+    },
+    {
+      "epoch": 1.0210526315789474,
+      "grad_norm": 0.5044442415237427,
+      "learning_rate": 7.691023252757901e-05,
+      "loss": 0.50599045753479,
+      "step": 485
+    },
+    {
+      "epoch": 1.0315789473684212,
+      "grad_norm": 0.5246402025222778,
+      "learning_rate": 7.64295286985904e-05,
+      "loss": 0.4706981658935547,
+      "step": 490
+    },
+    {
+      "epoch": 1.0421052631578946,
+      "grad_norm": 0.6998376846313477,
+      "learning_rate": 7.594541051666742e-05,
+      "loss": 0.48616642951965333,
+      "step": 495
+    },
+    {
+      "epoch": 1.0526315789473684,
+      "grad_norm": 0.5945510864257812,
+      "learning_rate": 7.545794052361149e-05,
+      "loss": 0.46446027755737307,
+      "step": 500
+    },
+    {
+      "epoch": 1.0526315789473684,
+      "eval_loss": 0.5771171450614929,
+      "eval_runtime": 27.1019,
+      "eval_samples_per_second": 14.759,
+      "eval_steps_per_second": 14.759,
+      "step": 500
+    },
+    {
+      "epoch": 1.063157894736842,
+      "grad_norm": 12.692610740661621,
+      "learning_rate": 7.496718169423462e-05,
+      "loss": 0.4399024486541748,
+      "step": 505
+    },
+    {
+      "epoch": 1.0736842105263158,
+      "grad_norm": 0.6038990020751953,
+      "learning_rate": 7.447319742822392e-05,
+      "loss": 0.48290238380432127,
+      "step": 510
+    },
+    {
+      "epoch": 1.0842105263157895,
+      "grad_norm": 0.6743545532226562,
+      "learning_rate": 7.397605154195106e-05,
+      "loss": 0.4822847366333008,
+      "step": 515
+    },
+    {
+      "epoch": 1.0947368421052632,
+      "grad_norm": 0.6746461391448975,
+      "learning_rate": 7.347580826022821e-05,
+      "loss": 0.47967071533203126,
+      "step": 520
+    },
+    {
+      "epoch": 1.1052631578947367,
+      "grad_norm": 0.7964761853218079,
+      "learning_rate": 7.29725322080109e-05,
+      "loss": 0.5180202484130859,
+      "step": 525
+    },
+    {
+      "epoch": 1.1157894736842104,
+      "grad_norm": 0.6772136688232422,
+      "learning_rate": 7.246628840204935e-05,
+      "loss": 0.46326284408569335,
+      "step": 530
+    },
+    {
+      "epoch": 1.1263157894736842,
+      "grad_norm": 0.7225517630577087,
+      "learning_rate": 7.195714224248912e-05,
+      "loss": 0.5262459278106689,
+      "step": 535
+    },
+    {
+      "epoch": 1.1368421052631579,
+      "grad_norm": 0.8155159950256348,
+      "learning_rate": 7.144515950442232e-05,
+      "loss": 0.5381614685058593,
+      "step": 540
+    },
+    {
+      "epoch": 1.1473684210526316,
+      "grad_norm": 0.7686448097229004,
+      "learning_rate": 7.093040632939023e-05,
+      "loss": 0.48169612884521484,
+      "step": 545
+    },
+    {
+      "epoch": 1.1578947368421053,
+      "grad_norm": 0.798007071018219,
+      "learning_rate": 7.041294921683876e-05,
+      "loss": 0.571818494796753,
+      "step": 550
+    },
+    {
+      "epoch": 1.1578947368421053,
+      "eval_loss": 0.5729017853736877,
+      "eval_runtime": 27.3923,
+      "eval_samples_per_second": 14.603,
+      "eval_steps_per_second": 14.603,
+      "step": 550
+    },
+    {
+      "epoch": 1.168421052631579,
+      "grad_norm": 0.9871010184288025,
+      "learning_rate": 6.989285501552751e-05,
+      "loss": 0.5277251243591309,
+      "step": 555
+    },
+    {
+      "epoch": 1.1789473684210527,
+      "grad_norm": 0.6931464076042175,
+      "learning_rate": 6.93701909148938e-05,
+      "loss": 0.4839730739593506,
+      "step": 560
+    },
+    {
+      "epoch": 1.1894736842105262,
+      "grad_norm": 0.6945406198501587,
+      "learning_rate": 6.884502443637273e-05,
+      "loss": 0.4632833480834961,
+      "step": 565
+    },
+    {
+      "epoch": 1.2,
+      "grad_norm": 0.8374159336090088,
+      "learning_rate": 6.831742342467418e-05,
+      "loss": 0.46635966300964354,
+      "step": 570
+    },
+    {
+      "epoch": 1.2105263157894737,
+      "grad_norm": 0.8007093667984009,
+      "learning_rate": 6.778745603901817e-05,
+      "loss": 0.4775552272796631,
+      "step": 575
+    },
+    {
+      "epoch": 1.2210526315789474,
+      "grad_norm": 0.922927975654602,
+      "learning_rate": 6.725519074432965e-05,
+      "loss": 0.41661796569824217,
+      "step": 580
+    },
+    {
+      "epoch": 1.231578947368421,
+      "grad_norm": 0.8915149569511414,
+      "learning_rate": 6.672069630239366e-05,
+      "loss": 0.46227364540100097,
+      "step": 585
+    },
+    {
+      "epoch": 1.2421052631578948,
+      "grad_norm": 0.7948114275932312,
+      "learning_rate": 6.618404176297217e-05,
+      "loss": 0.5104813575744629,
+      "step": 590
+    },
+    {
+      "epoch": 1.2526315789473683,
+      "grad_norm": 0.9380368590354919,
+      "learning_rate": 6.564529645488383e-05,
+      "loss": 0.4952188014984131,
+      "step": 595
+    },
+    {
+      "epoch": 1.263157894736842,
+      "grad_norm": 0.8004136085510254,
+      "learning_rate": 6.510452997704748e-05,
+      "loss": 0.4533547878265381,
+      "step": 600
+    },
+    {
+      "epoch": 1.263157894736842,
+      "eval_loss": 0.5720676779747009,
+      "eval_runtime": 27.2065,
+      "eval_samples_per_second": 14.702,
+      "eval_steps_per_second": 14.702,
+      "step": 600
+    },
+    {
+      "epoch": 1.2736842105263158,
+      "grad_norm": 0.8849948644638062,
+      "learning_rate": 6.456181218949096e-05,
+      "loss": 0.5185356140136719,
+      "step": 605
+    },
+    {
+      "epoch": 1.2842105263157895,
+      "grad_norm": 0.8258388638496399,
+      "learning_rate": 6.401721320432604e-05,
+      "loss": 0.45459442138671874,
+      "step": 610
+    },
+    {
+      "epoch": 1.2947368421052632,
+      "grad_norm": 0.7006433606147766,
+      "learning_rate": 6.34708033766909e-05,
+      "loss": 0.42599120140075686,
+      "step": 615
+    },
+    {
+      "epoch": 1.305263157894737,
+      "grad_norm": 0.8149921298027039,
+      "learning_rate": 6.292265329566108e-05,
+      "loss": 0.47135143280029296,
+      "step": 620
+    },
+    {
+      "epoch": 1.3157894736842106,
+      "grad_norm": 0.8304453492164612,
+      "learning_rate": 6.237283377513036e-05,
+      "loss": 0.543507719039917,
+      "step": 625
+    },
+    {
+      "epoch": 1.3263157894736843,
+      "grad_norm": 2.146064281463623,
+      "learning_rate": 6.182141584466247e-05,
+      "loss": 0.525644063949585,
+      "step": 630
+    },
+    {
+      "epoch": 1.3368421052631578,
+      "grad_norm": 0.8405390381813049,
+      "learning_rate": 6.126847074031507e-05,
+      "loss": 0.4994760513305664,
+      "step": 635
+    },
+    {
+      "epoch": 1.3473684210526315,
+      "grad_norm": 0.7558398842811584,
+      "learning_rate": 6.071406989543678e-05,
+      "loss": 0.45131878852844237,
+      "step": 640
+    },
+    {
+      "epoch": 1.3578947368421053,
+      "grad_norm": 1.3861230611801147,
+      "learning_rate": 6.0158284931439177e-05,
+      "loss": 0.45543656349182127,
+      "step": 645
+    },
+    {
+      "epoch": 1.368421052631579,
+      "grad_norm": 0.7160976529121399,
+      "learning_rate": 5.9601187648544056e-05,
+      "loss": 0.4914837837219238,
+      "step": 650
+    },
+    {
+      "epoch": 1.368421052631579,
+      "eval_loss": 0.566149890422821,
+      "eval_runtime": 27.1418,
+      "eval_samples_per_second": 14.737,
+      "eval_steps_per_second": 14.737,
+      "step": 650
+    },
+    {
+      "epoch": 1.3789473684210527,
+      "grad_norm": 0.7861437201499939,
+      "learning_rate": 5.904285001650783e-05,
+      "loss": 0.497259521484375,
+      "step": 655
+    },
+    {
+      "epoch": 1.3894736842105262,
+      "grad_norm": 0.7229910492897034,
+      "learning_rate": 5.8483344165323975e-05,
+      "loss": 0.5121944904327392,
+      "step": 660
+    },
+    {
+      "epoch": 1.4,
+      "grad_norm": 0.8752608299255371,
+      "learning_rate": 5.792274237590471e-05,
+      "loss": 0.5024206161499023,
+      "step": 665
+    },
+    {
+      "epoch": 1.4105263157894736,
+      "grad_norm": 0.8390534520149231,
+      "learning_rate": 5.7361117070743374e-05,
+      "loss": 0.48681044578552246,
+      "step": 670
+    },
+    {
+      "epoch": 1.4210526315789473,
+      "grad_norm": 0.8038208484649658,
+      "learning_rate": 5.679854080455821e-05,
+      "loss": 0.4971346378326416,
+      "step": 675
+    },
+    {
+      "epoch": 1.431578947368421,
+      "grad_norm": 0.7638700008392334,
+      "learning_rate": 5.6235086254919324e-05,
+      "loss": 0.42120847702026365,
+      "step": 680
+    },
+    {
+      "epoch": 1.4421052631578948,
+      "grad_norm": 0.7635045051574707,
+      "learning_rate": 5.567082621285969e-05,
+      "loss": 0.47087574005126953,
+      "step": 685
+    },
+    {
+      "epoch": 1.4526315789473685,
+      "grad_norm": 0.77474045753479,
+      "learning_rate": 5.510583357347149e-05,
+      "loss": 0.48073482513427734,
+      "step": 690
+    },
+    {
+      "epoch": 1.4631578947368422,
+      "grad_norm": 0.8228810429573059,
+      "learning_rate": 5.454018132648897e-05,
+      "loss": 0.46302366256713867,
+      "step": 695
+    },
+    {
+      "epoch": 1.4736842105263157,
+      "grad_norm": 0.8553083539009094,
+      "learning_rate": 5.3973942546859145e-05,
+      "loss": 0.47696647644042967,
+      "step": 700
+    },
+    {
+      "epoch": 1.4736842105263157,
+      "eval_loss": 0.5677127242088318,
+      "eval_runtime": 27.2564,
+      "eval_samples_per_second": 14.675,
+      "eval_steps_per_second": 14.675,
+      "step": 700
+    },
+    {
+      "epoch": 1.4842105263157894,
+      "grad_norm": 0.885428249835968,
+      "learning_rate": 5.3407190385301456e-05,
+      "loss": 0.426132345199585,
+      "step": 705
+    },
+    {
+      "epoch": 1.4947368421052631,
+      "grad_norm": 0.7517284154891968,
+      "learning_rate": 5.283999805885764e-05,
+      "loss": 0.5038762092590332,
+      "step": 710
+    },
+    {
+      "epoch": 1.5052631578947369,
+      "grad_norm": 0.6059720516204834,
+      "learning_rate": 5.227243884143306e-05,
+      "loss": 0.40855984687805175,
+      "step": 715
+    },
+    {
+      "epoch": 1.5157894736842106,
+      "grad_norm": 0.883471667766571,
+      "learning_rate": 5.170458605433059e-05,
+      "loss": 0.48758525848388673,
+      "step": 720
+    },
+    {
+      "epoch": 1.526315789473684,
+      "grad_norm": 0.8542289733886719,
+      "learning_rate": 5.113651305677856e-05,
+      "loss": 0.4801379680633545,
+      "step": 725
+    },
+    {
+      "epoch": 1.5368421052631578,
+      "grad_norm": 0.7095292210578918,
+      "learning_rate": 5.0568293236453614e-05,
+      "loss": 0.4451273441314697,
+      "step": 730
+    },
+    {
+      "epoch": 1.5473684210526315,
+      "grad_norm": 0.7466174960136414,
+      "learning_rate": 5e-05,
+      "loss": 0.4858893871307373,
+      "step": 735
+    },
+    {
+      "epoch": 1.5578947368421052,
+      "grad_norm": 0.8140954971313477,
+      "learning_rate": 4.94317067635464e-05,
+      "loss": 0.5117893695831299,
+      "step": 740
+    },
+    {
+      "epoch": 1.568421052631579,
+      "grad_norm": 0.8628697991371155,
+      "learning_rate": 4.886348694322145e-05,
+      "loss": 0.48410491943359374,
+      "step": 745
+    },
+    {
+      "epoch": 1.5789473684210527,
+      "grad_norm": 0.7672389149665833,
+      "learning_rate": 4.829541394566942e-05,
+      "loss": 0.4889042854309082,
+      "step": 750
+    },
+    {
+      "epoch": 1.5789473684210527,
+      "eval_loss": 0.5602394938468933,
+      "eval_runtime": 27.2938,
+      "eval_samples_per_second": 14.655,
+      "eval_steps_per_second": 14.655,
+      "step": 750
+    },
+    {
+      "epoch": 1.5894736842105264,
+      "grad_norm": 0.8665842413902283,
+      "learning_rate": 4.772756115856695e-05,
+      "loss": 0.4589890480041504,
+      "step": 755
+    },
+    {
+      "epoch": 1.6,
+      "grad_norm": 0.948796808719635,
+      "learning_rate": 4.7160001941142365e-05,
+      "loss": 0.4775981426239014,
+      "step": 760
+    },
+    {
+      "epoch": 1.6105263157894738,
+      "grad_norm": 0.8106200695037842,
+      "learning_rate": 4.6592809614698556e-05,
+      "loss": 0.5602635860443115,
+      "step": 765
+    },
+    {
+      "epoch": 1.6210526315789475,
+      "grad_norm": 1.2499663829803467,
+      "learning_rate": 4.602605745314087e-05,
+      "loss": 0.5022463321685791,
+      "step": 770
+    },
+    {
+      "epoch": 1.631578947368421,
+      "grad_norm": 0.8327662944793701,
+      "learning_rate": 4.545981867351104e-05,
+      "loss": 0.4951312065124512,
+      "step": 775
+    },
+    {
+      "epoch": 1.6421052631578947,
+      "grad_norm": 0.7749590277671814,
+      "learning_rate": 4.4894166426528524e-05,
+      "loss": 0.4875455379486084,
+      "step": 780
+    },
+    {
+      "epoch": 1.6526315789473685,
+      "grad_norm": 0.764395534992218,
+      "learning_rate": 4.432917378714032e-05,
+      "loss": 0.47612872123718264,
+      "step": 785
+    },
+    {
+      "epoch": 1.663157894736842,
+      "grad_norm": 0.8650004863739014,
+      "learning_rate": 4.3764913745080695e-05,
+      "loss": 0.4810147285461426,
+      "step": 790
+    },
+    {
+      "epoch": 1.6736842105263157,
+      "grad_norm": 0.852967381477356,
+      "learning_rate": 4.32014591954418e-05,
+      "loss": 0.46249094009399416,
+      "step": 795
+    },
+    {
+      "epoch": 1.6842105263157894,
+      "grad_norm": 0.752549946308136,
+      "learning_rate": 4.263888292925664e-05,
+      "loss": 0.463092565536499,
+      "step": 800
+    },
+    {
+      "epoch": 1.6842105263157894,
+      "eval_loss": 0.557573139667511,
+      "eval_runtime": 27.1956,
+      "eval_samples_per_second": 14.708,
+      "eval_steps_per_second": 14.708,
+      "step": 800
+    },
+    {
+      "epoch": 1.694736842105263,
+      "grad_norm": 0.7666327953338623,
+      "learning_rate": 4.207725762409529e-05,
+      "loss": 0.549990177154541,
+      "step": 805
+    },
+    {
+      "epoch": 1.7052631578947368,
+      "grad_norm": 1.2041642665863037,
+      "learning_rate": 4.151665583467604e-05,
+      "loss": 0.473134708404541,
+      "step": 810
+    },
+    {
+      "epoch": 1.7157894736842105,
+      "grad_norm": 0.9291930794715881,
+      "learning_rate": 4.095714998349218e-05,
+      "loss": 0.43207721710205077,
+      "step": 815
+    },
+    {
+      "epoch": 1.7263157894736842,
+      "grad_norm": 0.8421230316162109,
+      "learning_rate": 4.0398812351455955e-05,
+      "loss": 0.42284450531005857,
+      "step": 820
+    },
+    {
+      "epoch": 1.736842105263158,
+      "grad_norm": 0.7550533413887024,
+      "learning_rate": 3.9841715068560835e-05,
+      "loss": 0.4533982276916504,
+      "step": 825
+    },
+    {
+      "epoch": 1.7473684210526317,
+      "grad_norm": 0.7380033731460571,
+      "learning_rate": 3.9285930104563234e-05,
+      "loss": 0.47853550910949705,
+      "step": 830
+    },
+    {
+      "epoch": 1.7578947368421054,
+      "grad_norm": 1.0466768741607666,
+      "learning_rate": 3.873152925968495e-05,
+      "loss": 0.5334872245788574,
+      "step": 835
+    },
+    {
+      "epoch": 1.768421052631579,
+      "grad_norm": 0.8049618601799011,
+      "learning_rate": 3.8178584155337525e-05,
+      "loss": 0.4399109363555908,
+      "step": 840
+    },
+    {
+      "epoch": 1.7789473684210526,
+      "grad_norm": 0.7510071992874146,
+      "learning_rate": 3.762716622486965e-05,
+      "loss": 0.521062707901001,
+      "step": 845
+    },
+    {
+      "epoch": 1.7894736842105263,
+      "grad_norm": 1.6552070379257202,
+      "learning_rate": 3.7077346704338935e-05,
+      "loss": 0.45749530792236326,
+      "step": 850
+    },
+    {
+      "epoch": 1.7894736842105263,
+      "eval_loss": 0.552564799785614,
+      "eval_runtime": 27.3375,
+      "eval_samples_per_second": 14.632,
+      "eval_steps_per_second": 14.632,
+      "step": 850
+    },
+    {
+      "epoch": 1.8,
+      "grad_norm": 0.7396476864814758,
+      "learning_rate": 3.6529196623309115e-05,
+      "loss": 0.38660905361175535,
+      "step": 855
+    },
+    {
+      "epoch": 1.8105263157894735,
+      "grad_norm": 0.8277924656867981,
+      "learning_rate": 3.598278679567397e-05,
+      "loss": 0.5064235210418702,
+      "step": 860
+    },
+    {
+      "epoch": 1.8210526315789473,
+      "grad_norm": 0.9554834365844727,
+      "learning_rate": 3.543818781050906e-05,
+      "loss": 0.4140141010284424,
+      "step": 865
+    },
+    {
+      "epoch": 1.831578947368421,
+      "grad_norm": 0.7180286049842834,
+      "learning_rate": 3.4895470022952536e-05,
+      "loss": 0.5401397705078125,
+      "step": 870
+    },
+    {
+      "epoch": 1.8421052631578947,
+      "grad_norm": 0.972764790058136,
+      "learning_rate": 3.4354703545116185e-05,
+      "loss": 0.504509973526001,
+      "step": 875
+    },
+    {
+      "epoch": 1.8526315789473684,
+      "grad_norm": 0.7626535892486572,
+      "learning_rate": 3.381595823702784e-05,
+      "loss": 0.46036224365234374,
+      "step": 880
+    },
+    {
+      "epoch": 1.8631578947368421,
+      "grad_norm": 0.9077037572860718,
+      "learning_rate": 3.3279303697606354e-05,
+      "loss": 0.4127952098846436,
+      "step": 885
+    },
+    {
+      "epoch": 1.8736842105263158,
+      "grad_norm": 0.764255702495575,
+      "learning_rate": 3.274480925567036e-05,
+      "loss": 0.4129067897796631,
+      "step": 890
+    },
+    {
+      "epoch": 1.8842105263157896,
+      "grad_norm": 0.9645035266876221,
+      "learning_rate": 3.2212543960981845e-05,
+      "loss": 0.486614990234375,
+      "step": 895
+    },
+    {
+      "epoch": 1.8947368421052633,
+      "grad_norm": 0.7822158932685852,
+      "learning_rate": 3.168257657532584e-05,
+      "loss": 0.43874049186706543,
+      "step": 900
+    },
+    {
+      "epoch": 1.8947368421052633,
+      "eval_loss": 0.5511002540588379,
+      "eval_runtime": 27.8563,
+      "eval_samples_per_second": 14.359,
+      "eval_steps_per_second": 14.359,
+      "step": 900
+    },
+    {
+      "epoch": 1.905263157894737,
+      "grad_norm": 0.823441207408905,
+      "learning_rate": 3.115497556362727e-05,
+      "loss": 0.4594260692596436,
+      "step": 905
+    },
+    {
+      "epoch": 1.9157894736842105,
+      "grad_norm": 0.821818470954895,
+      "learning_rate": 3.06298090851062e-05,
+      "loss": 0.4467416763305664,
+      "step": 910
+    },
+    {
+      "epoch": 1.9263157894736842,
+      "grad_norm": 0.8790761232376099,
+      "learning_rate": 3.0107144984472502e-05,
+      "loss": 0.4823316097259521,
+      "step": 915
+    },
+    {
+      "epoch": 1.936842105263158,
+      "grad_norm": 0.7914945483207703,
+      "learning_rate": 2.9587050783161252e-05,
+      "loss": 0.4314936637878418,
+      "step": 920
+    },
+    {
+      "epoch": 1.9473684210526314,
+      "grad_norm": 0.805029571056366,
+      "learning_rate": 2.9069593670609775e-05,
+      "loss": 0.4740307331085205,
+      "step": 925
+    },
+    {
+      "epoch": 1.9578947368421051,
+      "grad_norm": 0.7928342223167419,
+      "learning_rate": 2.8554840495577682e-05,
+      "loss": 0.4155329704284668,
+      "step": 930
+    },
+    {
+      "epoch": 1.9684210526315788,
+      "grad_norm": 0.8041934370994568,
+      "learning_rate": 2.8042857757510877e-05,
+      "loss": 0.5049230098724365,
+      "step": 935
+    },
+    {
+      "epoch": 1.9789473684210526,
+      "grad_norm": 0.8169903755187988,
+      "learning_rate": 2.753371159795065e-05,
+      "loss": 0.3966445684432983,
+      "step": 940
+    },
+    {
+      "epoch": 1.9894736842105263,
+      "grad_norm": 0.86372309923172,
+      "learning_rate": 2.702746779198912e-05,
+      "loss": 0.5153264999389648,
+      "step": 945
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.97676020860672,
+      "learning_rate": 2.6524191739771815e-05,
+      "loss": 0.4318437099456787,
+      "step": 950
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.5508978962898254,
+      "eval_runtime": 27.8163,
+      "eval_samples_per_second": 14.38,
+      "eval_steps_per_second": 14.38,
+      "step": 950
+    },
+    {
+      "epoch": 2.0105263157894737,
+      "grad_norm": 0.7161779999732971,
+      "learning_rate": 2.6023948458048965e-05,
+      "loss": 0.3977535247802734,
+      "step": 955
+    },
+    {
+      "epoch": 2.0210526315789474,
+      "grad_norm": 0.8996826410293579,
+      "learning_rate": 2.552680257177611e-05,
+      "loss": 0.38445310592651366,
+      "step": 960
+    },
+    {
+      "epoch": 2.031578947368421,
+      "grad_norm": 0.7812126278877258,
+      "learning_rate": 2.5032818305765383e-05,
+      "loss": 0.37283446788787844,
+      "step": 965
+    },
+    {
+      "epoch": 2.042105263157895,
+      "grad_norm": 0.7872561812400818,
+      "learning_rate": 2.454205947638852e-05,
+      "loss": 0.3594684600830078,
+      "step": 970
+    },
+    {
+      "epoch": 2.0526315789473686,
+      "grad_norm": 0.8357042670249939,
+      "learning_rate": 2.4054589483332597e-05,
+      "loss": 0.30683672428131104,
+      "step": 975
+    },
+    {
+      "epoch": 2.0631578947368423,
+      "grad_norm": 0.7905634045600891,
+      "learning_rate": 2.3570471301409618e-05,
+      "loss": 0.32751510143280027,
+      "step": 980
+    },
+    {
+      "epoch": 2.0736842105263156,
+      "grad_norm": 1.3322068452835083,
+      "learning_rate": 2.3089767472421e-05,
+      "loss": 0.3961126089096069,
+      "step": 985
+    },
+    {
+      "epoch": 2.0842105263157893,
+      "grad_norm": 0.9144253730773926,
+      "learning_rate": 2.2612540097077935e-05,
+      "loss": 0.3600940704345703,
+      "step": 990
+    },
+    {
+      "epoch": 2.094736842105263,
+      "grad_norm": 0.8574570417404175,
+      "learning_rate": 2.213885082697883e-05,
+      "loss": 0.3191706895828247,
+      "step": 995
+    },
+    {
+      "epoch": 2.1052631578947367,
+      "grad_norm": 0.9719475507736206,
+      "learning_rate": 2.1668760856644703e-05,
+      "loss": 0.39908566474914553,
+      "step": 1000
+    },
+    {
+      "epoch": 2.1052631578947367,
+      "eval_loss": 0.5967375040054321,
+      "eval_runtime": 28.5798,
+      "eval_samples_per_second": 13.996,
+      "eval_steps_per_second": 13.996,
+      "step": 1000
+    },
+    {
+      "epoch": 2.1157894736842104,
+      "grad_norm": 1.0026988983154297,
+      "learning_rate": 2.1202330915613638e-05,
+      "loss": 0.324949836730957,
+      "step": 1005
+    },
+    {
+      "epoch": 2.126315789473684,
+      "grad_norm": 0.9327012896537781,
+      "learning_rate": 2.0739621260595315e-05,
+      "loss": 0.31343927383422854,
+      "step": 1010
+    },
+    {
+      "epoch": 2.136842105263158,
+      "grad_norm": 0.8879086375236511,
+      "learning_rate": 2.028069166768663e-05,
+      "loss": 0.35412182807922366,
+      "step": 1015
+    },
+    {
+      "epoch": 2.1473684210526316,
+      "grad_norm": 1.014451265335083,
+      "learning_rate": 1.982560142464939e-05,
+      "loss": 0.3389745235443115,
+      "step": 1020
+    },
+    {
+      "epoch": 2.1578947368421053,
+      "grad_norm": 1.067197561264038,
+      "learning_rate": 1.937440932325112e-05,
+      "loss": 0.30197973251342775,
+      "step": 1025
+    },
+    {
+      "epoch": 2.168421052631579,
+      "grad_norm": 0.9498446583747864,
+      "learning_rate": 1.8927173651669877e-05,
+      "loss": 0.3828472375869751,
+      "step": 1030
+    },
+    {
+      "epoch": 2.1789473684210527,
+      "grad_norm": 1.0303031206130981,
+      "learning_rate": 1.8483952186964237e-05,
+      "loss": 0.4124176025390625,
+      "step": 1035
+    },
+    {
+      "epoch": 2.1894736842105265,
+      "grad_norm": 0.958342969417572,
+      "learning_rate": 1.804480218760922e-05,
+      "loss": 0.3103924751281738,
+      "step": 1040
+    },
+    {
+      "epoch": 2.2,
+      "grad_norm": 1.0009726285934448,
+      "learning_rate": 1.7609780386099234e-05,
+      "loss": 0.33572826385498045,
+      "step": 1045
+    },
+    {
+      "epoch": 2.2105263157894735,
+      "grad_norm": 1.0434421300888062,
+      "learning_rate": 1.7178942981618945e-05,
+      "loss": 0.3350696086883545,
+      "step": 1050
+    },
+    {
+      "epoch": 2.2105263157894735,
+      "eval_loss": 0.6022932529449463,
+      "eval_runtime": 27.6209,
+      "eval_samples_per_second": 14.482,
+      "eval_steps_per_second": 14.482,
+      "step": 1050
+    },
+    {
+      "epoch": 2.221052631578947,
+      "grad_norm": 1.1546400785446167,
+      "learning_rate": 1.6752345632783135e-05,
+      "loss": 0.34174375534057616,
+      "step": 1055
+    },
+    {
+      "epoch": 2.231578947368421,
+      "grad_norm": 1.0252087116241455,
+      "learning_rate": 1.6330043450446265e-05,
+      "loss": 0.3031753063201904,
+      "step": 1060
+    },
+    {
+      "epoch": 2.2421052631578946,
+      "grad_norm": 1.0577940940856934,
+      "learning_rate": 1.59120909905829e-05,
+      "loss": 0.35698933601379396,
+      "step": 1065
+    },
+    {
+      "epoch": 2.2526315789473683,
+      "grad_norm": 1.175123929977417,
+      "learning_rate": 1.549854224723978e-05,
+      "loss": 0.2965430736541748,
+      "step": 1070
+    },
+    {
+      "epoch": 2.263157894736842,
+      "grad_norm": 0.9692304730415344,
+      "learning_rate": 1.508945064556047e-05,
+      "loss": 0.3899634122848511,
+      "step": 1075
+    },
+    {
+      "epoch": 2.2736842105263158,
+      "grad_norm": 1.0569312572479248,
+      "learning_rate": 1.4684869034883554e-05,
+      "loss": 0.34727933406829836,
+      "step": 1080
+    },
+    {
+      "epoch": 2.2842105263157895,
+      "grad_norm": 0.8666245937347412,
+      "learning_rate": 1.4284849681915158e-05,
+      "loss": 0.3896636009216309,
+      "step": 1085
+    },
+    {
+      "epoch": 2.294736842105263,
+      "grad_norm": 1.0360913276672363,
+      "learning_rate": 1.3889444263976786e-05,
+      "loss": 0.31699607372283933,
+      "step": 1090
+    },
+    {
+      "epoch": 2.305263157894737,
+      "grad_norm": 1.2606230974197388,
+      "learning_rate": 1.3498703862329254e-05,
+      "loss": 0.33367414474487306,
+      "step": 1095
+    },
+    {
+      "epoch": 2.3157894736842106,
+      "grad_norm": 1.2789041996002197,
+      "learning_rate": 1.3112678955573693e-05,
+      "loss": 0.3069949150085449,
+      "step": 1100
+    },
+    {
+      "epoch": 2.3157894736842106,
+      "eval_loss": 0.6039758324623108,
+      "eval_runtime": 27.6614,
+      "eval_samples_per_second": 14.461,
+      "eval_steps_per_second": 14.461,
+      "step": 1100
+    },
+    {
+      "epoch": 2.3263157894736843,
+      "grad_norm": 1.1908059120178223,
+      "learning_rate": 1.2731419413130325e-05,
+      "loss": 0.38684911727905275,
+      "step": 1105
+    },
+    {
+      "epoch": 2.336842105263158,
+      "grad_norm": 1.262253999710083,
+      "learning_rate": 1.2354974488796017e-05,
+      "loss": 0.36271347999572756,
+      "step": 1110
+    },
+    {
+      "epoch": 2.3473684210526318,
+      "grad_norm": 0.9189227819442749,
+      "learning_rate": 1.1983392814381273e-05,
+      "loss": 0.3401113271713257,
+      "step": 1115
+    },
+    {
+      "epoch": 2.3578947368421055,
+      "grad_norm": 1.0726381540298462,
+      "learning_rate": 1.1616722393427704e-05,
+      "loss": 0.33876895904541016,
+      "step": 1120
+    },
+    {
+      "epoch": 2.3684210526315788,
+      "grad_norm": 0.9959712028503418,
+      "learning_rate": 1.125501059500656e-05,
+      "loss": 0.3982860803604126,
+      "step": 1125
+    },
+    {
+      "epoch": 2.3789473684210525,
+      "grad_norm": 0.9985882043838501,
+      "learning_rate": 1.0898304147599231e-05,
+      "loss": 0.3682390213012695,
+      "step": 1130
+    },
+    {
+      "epoch": 2.389473684210526,
+      "grad_norm": 0.9115525484085083,
+      "learning_rate": 1.0546649133060583e-05,
+      "loss": 0.3579556465148926,
+      "step": 1135
+    },
+    {
+      "epoch": 2.4,
+      "grad_norm": 1.1207247972488403,
+      "learning_rate": 1.0200090980665739e-05,
+      "loss": 0.42283267974853517,
+      "step": 1140
+    },
+    {
+      "epoch": 2.4105263157894736,
+      "grad_norm": 1.0433000326156616,
+      "learning_rate": 9.858674461241229e-06,
+      "loss": 0.362597393989563,
+      "step": 1145
+    },
+    {
+      "epoch": 2.4210526315789473,
+      "grad_norm": 0.9558874368667603,
+      "learning_rate": 9.522443681381172e-06,
+      "loss": 0.40636916160583497,
+      "step": 1150
+    },
+    {
+      "epoch": 2.4210526315789473,
+      "eval_loss": 0.6012060642242432,
+      "eval_runtime": 27.6853,
+      "eval_samples_per_second": 14.448,
+      "eval_steps_per_second": 14.448,
+      "step": 1150
+    },
+    {
+      "epoch": 2.431578947368421,
+      "grad_norm": 0.9411963224411011,
+      "learning_rate": 9.191442077749257e-06,
+      "loss": 0.32624173164367676,
+      "step": 1155
+    },
+    {
+      "epoch": 2.442105263157895,
+      "grad_norm": 1.1000866889953613,
+      "learning_rate": 8.86571241146732e-06,
+      "loss": 0.36433489322662355,
+      "step": 1160
+    },
+    {
+      "epoch": 2.4526315789473685,
+      "grad_norm": 0.8302079439163208,
+      "learning_rate": 8.545296762591144e-06,
+      "loss": 0.34126312732696534,
+      "step": 1165
+    },
+    {
+      "epoch": 2.463157894736842,
+      "grad_norm": 1.1295444965362549,
+      "learning_rate": 8.230236524674256e-06,
+      "loss": 0.34795560836791994,
+      "step": 1170
+    },
+    {
+      "epoch": 2.473684210526316,
+      "grad_norm": 1.0424487590789795,
+      "learning_rate": 7.920572399420428e-06,
+      "loss": 0.34096055030822753,
+      "step": 1175
+    },
+    {
+      "epoch": 2.4842105263157896,
+      "grad_norm": 1.1002000570297241,
+      "learning_rate": 7.616344391425534e-06,
+      "loss": 0.3707140922546387,
+      "step": 1180
+    },
+    {
+      "epoch": 2.4947368421052634,
+      "grad_norm": 0.946130096912384,
+      "learning_rate": 7.317591803009472e-06,
+      "loss": 0.3825195074081421,
+      "step": 1185
+    },
+    {
+      "epoch": 2.5052631578947366,
+      "grad_norm": 1.1903181076049805,
+      "learning_rate": 7.024353229138836e-06,
+      "loss": 0.3591669797897339,
+      "step": 1190
+    },
+    {
+      "epoch": 2.515789473684211,
+      "grad_norm": 0.9660072326660156,
+      "learning_rate": 6.736666552440896e-06,
+      "loss": 0.31880993843078614,
+      "step": 1195
+    },
+    {
+      "epoch": 2.526315789473684,
+      "grad_norm": 1.1138476133346558,
+      "learning_rate": 6.454568938309724e-06,
+      "loss": 0.33600804805755613,
+      "step": 1200
+    },
+    {
+      "epoch": 2.526315789473684,
+      "eval_loss": 0.6024701595306396,
+      "eval_runtime": 28.6271,
+      "eval_samples_per_second": 13.973,
+      "eval_steps_per_second": 13.973,
+      "step": 1200
+    },
+    {
+      "epoch": 2.536842105263158,
+      "grad_norm": 1.0256083011627197,
+      "learning_rate": 6.1780968301048406e-06,
+      "loss": 0.30231916904449463,
+      "step": 1205
+    },
+    {
+      "epoch": 2.5473684210526315,
+      "grad_norm": 1.3465616703033447,
+      "learning_rate": 5.907285944443241e-06,
+      "loss": 0.37215938568115237,
+      "step": 1210
+    },
+    {
+      "epoch": 2.557894736842105,
+      "grad_norm": 1.0157089233398438,
+      "learning_rate": 5.642171266585272e-06,
+      "loss": 0.38835783004760743,
+      "step": 1215
+    },
+    {
+      "epoch": 2.568421052631579,
+      "grad_norm": 1.0423297882080078,
+      "learning_rate": 5.3827870459149665e-06,
+      "loss": 0.3684116840362549,
+      "step": 1220
+    },
+    {
+      "epoch": 2.5789473684210527,
+      "grad_norm": 1.2063053846359253,
+      "learning_rate": 5.1291667915154774e-06,
+      "loss": 0.39838123321533203,
+      "step": 1225
+    },
+    {
+      "epoch": 2.5894736842105264,
+      "grad_norm": 1.045510172843933,
+      "learning_rate": 4.88134326784015e-06,
+      "loss": 0.3727046728134155,
+      "step": 1230
+    },
+    {
+      "epoch": 2.6,
+      "grad_norm": 1.079955816268921,
+      "learning_rate": 4.639348490479755e-06,
+      "loss": 0.3064917802810669,
+      "step": 1235
+    },
+    {
+      "epoch": 2.610526315789474,
+      "grad_norm": 1.029150128364563,
+      "learning_rate": 4.403213722026516e-06,
+      "loss": 0.3460820436477661,
+      "step": 1240
+    },
+    {
+      "epoch": 2.6210526315789475,
+      "grad_norm": 1.2160367965698242,
+      "learning_rate": 4.172969468035359e-06,
+      "loss": 0.3394759178161621,
+      "step": 1245
+    },
+    {
+      "epoch": 2.6315789473684212,
+      "grad_norm": 1.292370319366455,
+      "learning_rate": 3.948645473083018e-06,
+      "loss": 0.4295994281768799,
+      "step": 1250
+    },
+    {
+      "epoch": 2.6315789473684212,
+      "eval_loss": 0.6037428975105286,
+      "eval_runtime": 28.4938,
+      "eval_samples_per_second": 14.038,
+      "eval_steps_per_second": 14.038,
+      "step": 1250
+    },
+    {
+      "epoch": 2.6421052631578945,
+      "grad_norm": 0.9467762112617493,
+      "learning_rate": 3.730270716925394e-06,
+      "loss": 0.3685704946517944,
+      "step": 1255
+    },
+    {
+      "epoch": 2.6526315789473687,
+      "grad_norm": 0.9402472376823425,
+      "learning_rate": 3.5178734107537637e-06,
+      "loss": 0.38207206726074217,
+      "step": 1260
+    },
+    {
+      "epoch": 2.663157894736842,
+      "grad_norm": 0.9341251254081726,
+      "learning_rate": 3.311480993550259e-06,
+      "loss": 0.3355901002883911,
+      "step": 1265
+    },
+    {
+      "epoch": 2.6736842105263157,
+      "grad_norm": 1.3845702409744263,
+      "learning_rate": 3.111120128543088e-06,
+      "loss": 0.3235702276229858,
+      "step": 1270
+    },
+    {
+      "epoch": 2.6842105263157894,
+      "grad_norm": 0.9406099319458008,
+      "learning_rate": 2.9168166997620263e-06,
+      "loss": 0.33657886981964114,
+      "step": 1275
+    },
+    {
+      "epoch": 2.694736842105263,
+      "grad_norm": 1.0643633604049683,
+      "learning_rate": 2.7285958086944786e-06,
+      "loss": 0.35396461486816405,
+      "step": 1280
+    },
+    {
+      "epoch": 2.705263157894737,
+      "grad_norm": 0.8600782155990601,
+      "learning_rate": 2.5464817710427414e-06,
+      "loss": 0.3184598445892334,
+      "step": 1285
+    },
+    {
+      "epoch": 2.7157894736842105,
+      "grad_norm": 1.0098100900650024,
+      "learning_rate": 2.370498113582731e-06,
+      "loss": 0.3798511981964111,
+      "step": 1290
+    },
+    {
+      "epoch": 2.7263157894736842,
+      "grad_norm": 1.2423481941223145,
+      "learning_rate": 2.2006675711245818e-06,
+      "loss": 0.36703929901123045,
+      "step": 1295
+    },
+    {
+      "epoch": 2.736842105263158,
+      "grad_norm": 1.097049593925476,
+      "learning_rate": 2.0370120835756513e-06,
+      "loss": 0.2991799354553223,
+      "step": 1300
+    },
+    {
+      "epoch": 2.736842105263158,
+      "eval_loss": 0.6043052077293396,
+      "eval_runtime": 27.8515,
+      "eval_samples_per_second": 14.362,
+      "eval_steps_per_second": 14.362,
+      "step": 1300
+    },
+    {
+      "epoch": 2.7473684210526317,
+      "grad_norm": 1.128627061843872,
+      "learning_rate": 1.8795527931061374e-06,
+      "loss": 0.34659562110900877,
+      "step": 1305
+    },
+    {
+      "epoch": 2.7578947368421054,
+      "grad_norm": 0.958167314529419,
+      "learning_rate": 1.7283100414178078e-06,
+      "loss": 0.3479694128036499,
+      "step": 1310
+    },
+    {
+      "epoch": 2.768421052631579,
+      "grad_norm": 1.2653661966323853,
+      "learning_rate": 1.58330336711609e-06,
+      "loss": 0.3173972606658936,
+      "step": 1315
+    },
+    {
+      "epoch": 2.7789473684210524,
+      "grad_norm": 1.3858840465545654,
+      "learning_rate": 1.4445515031859591e-06,
+      "loss": 0.42389545440673826,
+      "step": 1320
+    },
+    {
+      "epoch": 2.7894736842105265,
+      "grad_norm": 1.142336368560791,
+      "learning_rate": 1.31207237457186e-06,
+      "loss": 0.3316075563430786,
+      "step": 1325
+    },
+    {
+      "epoch": 2.8,
+      "grad_norm": 1.341241478919983,
+      "learning_rate": 1.1858830958620559e-06,
+      "loss": 0.370093846321106,
+      "step": 1330
+    },
+    {
+      "epoch": 2.8105263157894735,
+      "grad_norm": 1.2752244472503662,
+      "learning_rate": 1.0659999690776302e-06,
+      "loss": 0.37927913665771484,
+      "step": 1335
+    },
+    {
+      "epoch": 2.8210526315789473,
+      "grad_norm": 1.2686620950698853,
+      "learning_rate": 9.524384815664699e-07,
+      "loss": 0.31010196208953855,
+      "step": 1340
+    },
+    {
+      "epoch": 2.831578947368421,
+      "grad_norm": 0.9856387376785278,
+      "learning_rate": 8.452133040025345e-07,
+      "loss": 0.37815587520599364,
+      "step": 1345
+    },
+    {
+      "epoch": 2.8421052631578947,
+      "grad_norm": 1.1066813468933105,
+      "learning_rate": 7.443382884905536e-07,
+      "loss": 0.3322570562362671,
+      "step": 1350
+    },
+    {
+      "epoch": 2.8421052631578947,
+      "eval_loss": 0.6025745272636414,
+      "eval_runtime": 27.5996,
+      "eval_samples_per_second": 14.493,
+      "eval_steps_per_second": 14.493,
+      "step": 1350
+    },
+    {
+      "epoch": 2.8526315789473684,
+      "grad_norm": 1.8500337600708008,
+      "learning_rate": 6.498264667765374e-07,
+      "loss": 0.362324595451355,
+      "step": 1355
+    },
+    {
+      "epoch": 2.863157894736842,
+      "grad_norm": 1.127455711364746,
+      "learning_rate": 5.616900485642506e-07,
+      "loss": 0.2964962005615234,
+      "step": 1360
+    },
+    {
+      "epoch": 2.873684210526316,
+      "grad_norm": 0.9335603713989258,
+      "learning_rate": 4.79940419937841e-07,
+      "loss": 0.33919923305511473,
+      "step": 1365
+    },
+    {
+      "epoch": 2.8842105263157896,
+      "grad_norm": 1.0695230960845947,
+      "learning_rate": 4.045881418909547e-07,
+      "loss": 0.33406035900115966,
+      "step": 1370
+    },
+    {
+      "epoch": 2.8947368421052633,
+      "grad_norm": 1.2964420318603516,
+      "learning_rate": 3.356429489623558e-07,
+      "loss": 0.3467694282531738,
+      "step": 1375
+    },
+    {
+      "epoch": 2.905263157894737,
+      "grad_norm": 0.9756618738174438,
+      "learning_rate": 2.7311374797835966e-07,
+      "loss": 0.27626986503601075,
+      "step": 1380
+    },
+    {
+      "epoch": 2.9157894736842103,
+      "grad_norm": 1.125270962715149,
+      "learning_rate": 2.170086169021923e-07,
+      "loss": 0.350561261177063,
+      "step": 1385
+    },
+    {
+      "epoch": 2.9263157894736844,
+      "grad_norm": 1.0733071565628052,
+      "learning_rate": 1.6733480379041434e-07,
+      "loss": 0.3240832328796387,
+      "step": 1390
+    },
+    {
+      "epoch": 2.9368421052631577,
+      "grad_norm": 1.8108630180358887,
+      "learning_rate": 1.240987258565751e-07,
+      "loss": 0.3057235240936279,
+      "step": 1395
+    },
+    {
+      "epoch": 2.9473684210526314,
+      "grad_norm": 1.3284285068511963,
+      "learning_rate": 8.730596864218177e-08,
+      "loss": 0.34192938804626466,
+      "step": 1400
+    },
+    {
+      "epoch": 2.9473684210526314,
+      "eval_loss": 0.6027438640594482,
+      "eval_runtime": 27.589,
+      "eval_samples_per_second": 14.499,
+      "eval_steps_per_second": 14.499,
+      "step": 1400
+    },
+    {
+      "epoch": 2.957894736842105,
+      "grad_norm": 1.0273772478103638,
+      "learning_rate": 5.696128529511513e-08,
+      "loss": 0.3200652599334717,
+      "step": 1405
+    },
+    {
+      "epoch": 2.968421052631579,
+      "grad_norm": 0.9442003965377808,
+      "learning_rate": 3.306859595560985e-08,
+      "loss": 0.3131251335144043,
+      "step": 1410
+    },
+    {
+      "epoch": 2.9789473684210526,
+      "grad_norm": 0.9659796357154846,
+      "learning_rate": 1.563098724979839e-08,
+      "loss": 0.3898029088973999,
+      "step": 1415
+    },
+    {
+      "epoch": 2.9894736842105263,
+      "grad_norm": 1.381455659866333,
+      "learning_rate": 4.650711890963333e-09,
+      "loss": 0.37259109020233155,
+      "step": 1420
+    },
+    {
+      "epoch": 3.0,
+      "grad_norm": 1.172485113143921,
+      "learning_rate": 1.2918838853126502e-10,
+      "loss": 0.397792387008667,
+      "step": 1425
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 0.6031771302223206,
+      "eval_runtime": 27.5452,
+      "eval_samples_per_second": 14.522,
+      "eval_steps_per_second": 14.522,
+      "step": 1425
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 1425,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 50,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.4041038913726054e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1425/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:757b7d78e07bd589f7cd9db1a1a3019f58127f01f91cba75e4a6cbb7ea4b4a6c
+size 5713

checkpoint-950/README.md

@@ -0,0 +1,210 @@
+---
+base_model: unsloth/gemma-4-12b-it
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/gemma-4-12b-it
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

checkpoint-950/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Gemma4UnifiedForConditionalGeneration",
+    "parent_library": "transformers.models.gemma4_unified.modeling_gemma4_unified",
+    "unsloth_fixed": true
+  },
+  "base_model_name_or_path": "unsloth/gemma-4-12b-it",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": "(?:.*?(?:language|text).*?(?:self_attn|attention|attn|mixer|mlp|feed_forward|ffn|dense|mixer).*?(?:q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj|embedding_projection))|(?:\\bmodel\\.layers\\.[\\d]{1,}\\.(?:self_attn|attention|attn|mixer|mlp|feed_forward|ffn|dense|mixer)\\.(?:(?:q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj|embedding_projection)))",
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

checkpoint-950/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f05b624df3aa0d19a9c5bb1f9413057987f10a140fd822057666c011c43bdc6
+size 262373216

checkpoint-950/chat_template.jinja

@@ -0,0 +1,70 @@
+{{ bos_token }}{%- macro strip_thinking(text) -%}
+    {%- set ns = namespace(result='') -%}
+    {%- for part in text.split('<channel|>') -%}
+        {%- if '<|channel>' in part -%}
+            {%- set ns.result = ns.result + part.split('<|channel>')[0] -%}
+        {%- else -%}
+            {%- set ns.result = ns.result + part -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- ns.result | trim -}}
+{%- endmacro -%}
+{%- set thinking = enable_thinking is defined and enable_thinking -%}
+{%- set loop_messages = messages -%}
+{%- if messages[0]['role'] in ['system', 'developer'] or thinking -%}
+    {{ '<|turn>system
+' }}
+    {%- if thinking -%}
+        {{ '<|think|>
+' }}
+    {%- endif -%}
+    {%- if messages[0]['role'] in ['system', 'developer'] -%}
+        {{ messages[0]['content'] | trim }}
+        {%- set loop_messages = messages[1:] -%}
+    {%- endif -%}
+    {{ '<turn|>
+' }}
+{%- endif -%}
+{%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
+    {%- if (message['role'] == 'assistant') -%}
+        {%- set role = "model" -%}
+    {%- else -%}
+        {%- set role = message['role'] -%}
+    {%- endif -%}
+    {{ '<|turn>' + role + '
+' }}
+    {%- if message['content'] is string -%}
+        {%- if role == "model" -%}
+            {{ strip_thinking(message['content']) }}
+        {%- else -%}
+            {{ message['content'] | trim }}
+        {%- endif -%}
+    {%- elif message['content'] is iterable -%}
+        {%- for item in message['content'] -%}
+            {%- if item['type'] == 'audio' -%}
+                {{ '<|audio|>' }}
+            {%- elif item['type'] == 'image' -%}
+                {{ '<|image|>' }}
+            {%- elif item['type'] == 'video' -%}
+                {{ '<|video|>' }}
+            {%- elif item['type'] == 'text' -%}
+                {%- if role == "model" -%}
+                    {{ strip_thinking(item['text']) }}
+                {%- else -%}
+                    {{ item['text'] | trim }}
+                {%- endif -%}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{ raise_exception("Invalid content type") }}
+    {%- endif -%}
+    {{ '<turn|>
+' }}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{'<|turn>model
+'}}
+{%- endif -%}

checkpoint-950/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f8cc9a8d00b3b47eb331ebe258e75b2407ce7d9c737e7f22e8240196766e76bc
+size 133962277

checkpoint-950/processor_config.json

@@ -0,0 +1,62 @@
+{
+  "audio_ms_per_token": 40,
+  "audio_seq_length": 750,
+  "feature_extractor": {
+    "audio_samples_per_token": 640,
+    "feature_extractor_type": "Gemma4UnifiedAudioFeatureExtractor",
+    "feature_size": 640,
+    "padding_side": "right",
+    "padding_value": 0.0,
+    "return_attention_mask": true,
+    "sampling_rate": 16000
+  },
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": false,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_processor_type": "Gemma4UnifiedImageProcessor",
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 280,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098
+  },
+  "image_seq_length": 280,
+  "processor_class": "Gemma4UnifiedProcessor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 70,
+    "num_frames": 32,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "video_processor_type": "Gemma4UnifiedVideoProcessor"
+  }
+}

checkpoint-950/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7687932e55f71d6428995bb2034f447743a80421cc89fa68cf6f3ec266a96caa
+size 14645

checkpoint-950/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5fe665b98fcabe2f1b3babcf39648ab6610ea0ebafc3d8d53ca089b983698f61
+size 1465

checkpoint-950/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc8d3a0ce36466ccc1278bf987df5f71db1719b9ca6b4118264f45cb627bfe0f
+size 32169626

checkpoint-950/tokenizer_config.json

@@ -0,0 +1,290 @@
+{
+  "audio_token": "<|audio|>",
+  "backend": "tokenizers",
+  "boa_token": "<|audio>",
+  "boi_token": "<|image>",
+  "bos_token": "<bos>",
+  "eoa_token": "<audio|>",
+  "eoc_token": "<channel|>",
+  "eoi_token": "<image|>",
+  "eos_token": "<turn|>",
+  "eot_token": "<turn|>",
+  "escape_token": "<|\"|>",
+  "etc_token": "<tool_call|>",
+  "etd_token": "<tool|>",
+  "etr_token": "<tool_response|>",
+  "extra_special_tokens": [
+    "<|video|>"
+  ],
+  "image_token": "<|image|>",
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "<mask>",
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_token": "<|audio|>",
+    "boa_token": "<|audio>",
+    "boi_token": "<|image>",
+    "eoa_token": "<audio|>",
+    "eoc_token": "<channel|>",
+    "eoi_token": "<image|>",
+    "eot_token": "<turn|>",
+    "escape_token": "<|\"|>",
+    "etc_token": "<tool_call|>",
+    "etd_token": "<tool|>",
+    "etr_token": "<tool_response|>",
+    "image_token": "<|image|>",
+    "soc_token": "<|channel>",
+    "sot_token": "<|turn>",
+    "stc_token": "<|tool_call>",
+    "std_token": "<|tool>",
+    "str_token": "<|tool_response>",
+    "think_token": "<|think|>"
+  },
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "processor_class": "Gemma4UnifiedProcessor",
+  "response_schema": {
+    "properties": {
+      "content": {
+        "type": "string"
+      },
+      "role": {
+        "const": "assistant"
+      },
+      "thinking": {
+        "type": "string"
+      },
+      "tool_calls": {
+        "items": {
+          "properties": {
+            "function": {
+              "properties": {
+                "arguments": {
+                  "additionalProperties": {},
+                  "type": "object",
+                  "x-parser": "gemma4-tool-call"
+                },
+                "name": {
+                  "type": "string"
+                }
+              },
+              "type": "object",
+              "x-regex": "call\\:(?P<name>\\w+)(?P<arguments>\\{.*\\})"
+            },
+            "type": {
+              "const": "function"
+            }
+          },
+          "type": "object"
+        },
+        "type": "array",
+        "x-regex-iterator": "<\\|tool_call>(.*?)<tool_call\\|>"
+      }
+    },
+    "type": "object",
+    "x-regex": "(\\<\\|channel\\>thought\\n(?P<thinking>.*?)\\<channel\\|\\>)?(?P<tool_calls>\\<\\|tool_call\\>.*\\<tool_call\\|\\>)?(?P<content>(?:(?!\\<turn\\|\\>)(?!\\<\\|tool_response\\>).)+)?(?:\\<turn\\|\\>|\\<\\|tool_response\\>)?"
+  },
+  "soc_token": "<|channel>",
+  "sot_token": "<|turn>",
+  "stc_token": "<|tool_call>",
+  "std_token": "<|tool>",
+  "str_token": "<|tool_response>",
+  "think_token": "<|think|>",
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "1": {
+      "content": "<eos>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "2": {
+      "content": "<bos>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "46": {
+      "content": "<|tool>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "47": {
+      "content": "<tool|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "48": {
+      "content": "<|tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "49": {
+      "content": "<tool_call|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "50": {
+      "content": "<|tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "51": {
+      "content": "<tool_response|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "52": {
+      "content": "<|\"|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "98": {
+      "content": "<|think|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "100": {
+      "content": "<|channel>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "101": {
+      "content": "<channel|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "105": {
+      "content": "<|turn>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "106": {
+      "content": "<turn|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "255999": {
+      "content": "<|image>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "256000": {
+      "content": "<|audio>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258880": {
+      "content": "<|image|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258881": {
+      "content": "<|audio|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258882": {
+      "content": "<image|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258883": {
+      "content": "<audio|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258884": {
+      "content": "<|video|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  }
+}

checkpoint-950/trainer_state.json

@@ -0,0 +1,1516 @@
+{
+  "best_global_step": 950,
+  "best_metric": 0.5508978962898254,
+  "best_model_checkpoint": "/output/checkpoint-950",
+  "epoch": 2.0,
+  "eval_steps": 50,
+  "global_step": 950,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.010526315789473684,
+      "grad_norm": 2.0839247703552246,
+      "learning_rate": 9.302325581395349e-06,
+      "loss": 2.646485137939453,
+      "step": 5
+    },
+    {
+      "epoch": 0.021052631578947368,
+      "grad_norm": 3.253295660018921,
+      "learning_rate": 2.0930232558139536e-05,
+      "loss": 2.9888063430786134,
+      "step": 10
+    },
+    {
+      "epoch": 0.031578947368421054,
+      "grad_norm": 2.4486842155456543,
+      "learning_rate": 3.2558139534883724e-05,
+      "loss": 2.2748624801635744,
+      "step": 15
+    },
+    {
+      "epoch": 0.042105263157894736,
+      "grad_norm": 2.5299599170684814,
+      "learning_rate": 4.418604651162791e-05,
+      "loss": 1.9410686492919922,
+      "step": 20
+    },
+    {
+      "epoch": 0.05263157894736842,
+      "grad_norm": 1.6780576705932617,
+      "learning_rate": 5.5813953488372095e-05,
+      "loss": 1.3348378181457519,
+      "step": 25
+    },
+    {
+      "epoch": 0.06315789473684211,
+      "grad_norm": 0.7812225222587585,
+      "learning_rate": 6.744186046511628e-05,
+      "loss": 0.8430736541748047,
+      "step": 30
+    },
+    {
+      "epoch": 0.07368421052631578,
+      "grad_norm": 0.8205987215042114,
+      "learning_rate": 7.906976744186047e-05,
+      "loss": 0.8522604942321778,
+      "step": 35
+    },
+    {
+      "epoch": 0.08421052631578947,
+      "grad_norm": 0.6925882697105408,
+      "learning_rate": 9.069767441860465e-05,
+      "loss": 0.6957129955291748,
+      "step": 40
+    },
+    {
+      "epoch": 0.09473684210526316,
+      "grad_norm": 0.7553223371505737,
+      "learning_rate": 9.999987081161148e-05,
+      "loss": 0.8302087783813477,
+      "step": 45
+    },
+    {
+      "epoch": 0.10526315789473684,
+      "grad_norm": 0.6056613922119141,
+      "learning_rate": 9.999534928810904e-05,
+      "loss": 0.6982179641723633,
+      "step": 50
+    },
+    {
+      "epoch": 0.10526315789473684,
+      "eval_loss": 0.7021176218986511,
+      "eval_runtime": 27.8753,
+      "eval_samples_per_second": 14.35,
+      "eval_steps_per_second": 14.35,
+      "step": 50
+    },
+    {
+      "epoch": 0.11578947368421053,
+      "grad_norm": 1.4581637382507324,
+      "learning_rate": 9.998436901275022e-05,
+      "loss": 0.7925346374511719,
+      "step": 55
+    },
+    {
+      "epoch": 0.12631578947368421,
+      "grad_norm": 0.6449484825134277,
+      "learning_rate": 9.99669314040444e-05,
+      "loss": 0.7095213413238526,
+      "step": 60
+    },
+    {
+      "epoch": 0.1368421052631579,
+      "grad_norm": 0.6936064958572388,
+      "learning_rate": 9.994303871470489e-05,
+      "loss": 0.6608465194702149,
+      "step": 65
+    },
+    {
+      "epoch": 0.14736842105263157,
+      "grad_norm": 0.5944413542747498,
+      "learning_rate": 9.991269403135783e-05,
+      "loss": 0.6789331436157227,
+      "step": 70
+    },
+    {
+      "epoch": 0.15789473684210525,
+      "grad_norm": 0.7518417835235596,
+      "learning_rate": 9.987590127414344e-05,
+      "loss": 0.669992208480835,
+      "step": 75
+    },
+    {
+      "epoch": 0.16842105263157894,
+      "grad_norm": 0.7772454619407654,
+      "learning_rate": 9.98326651962096e-05,
+      "loss": 0.6882061958312988,
+      "step": 80
+    },
+    {
+      "epoch": 0.17894736842105263,
+      "grad_norm": 0.7076404690742493,
+      "learning_rate": 9.978299138309781e-05,
+      "loss": 0.6435281276702881,
+      "step": 85
+    },
+    {
+      "epoch": 0.18947368421052632,
+      "grad_norm": 0.6353256702423096,
+      "learning_rate": 9.972688625202164e-05,
+      "loss": 0.6035995960235596,
+      "step": 90
+    },
+    {
+      "epoch": 0.2,
+      "grad_norm": 0.6928842663764954,
+      "learning_rate": 9.966435705103765e-05,
+      "loss": 0.6339406967163086,
+      "step": 95
+    },
+    {
+      "epoch": 0.21052631578947367,
+      "grad_norm": 1.1047526597976685,
+      "learning_rate": 9.959541185810906e-05,
+      "loss": 0.5951476097106934,
+      "step": 100
+    },
+    {
+      "epoch": 0.21052631578947367,
+      "eval_loss": 0.6416817307472229,
+      "eval_runtime": 27.0456,
+      "eval_samples_per_second": 14.79,
+      "eval_steps_per_second": 14.79,
+      "step": 100
+    },
+    {
+      "epoch": 0.22105263157894736,
+      "grad_norm": 0.6272938847541809,
+      "learning_rate": 9.952005958006217e-05,
+      "loss": 0.6947292804718017,
+      "step": 105
+    },
+    {
+      "epoch": 0.23157894736842105,
+      "grad_norm": 0.6077613234519958,
+      "learning_rate": 9.943830995143577e-05,
+      "loss": 0.7127768993377686,
+      "step": 110
+    },
+    {
+      "epoch": 0.24210526315789474,
+      "grad_norm": 0.6263899803161621,
+      "learning_rate": 9.935017353322347e-05,
+      "loss": 0.643881893157959,
+      "step": 115
+    },
+    {
+      "epoch": 0.25263157894736843,
+      "grad_norm": 0.6517646908760071,
+      "learning_rate": 9.925566171150945e-05,
+      "loss": 0.6408101081848144,
+      "step": 120
+    },
+    {
+      "epoch": 0.2631578947368421,
+      "grad_norm": 0.5079100728034973,
+      "learning_rate": 9.915478669599747e-05,
+      "loss": 0.5704734802246094,
+      "step": 125
+    },
+    {
+      "epoch": 0.2736842105263158,
+      "grad_norm": 0.6810751557350159,
+      "learning_rate": 9.904756151843353e-05,
+      "loss": 0.7103314876556397,
+      "step": 130
+    },
+    {
+      "epoch": 0.28421052631578947,
+      "grad_norm": 0.6985617876052856,
+      "learning_rate": 9.893400003092237e-05,
+      "loss": 0.7399785995483399,
+      "step": 135
+    },
+    {
+      "epoch": 0.29473684210526313,
+      "grad_norm": 0.611039936542511,
+      "learning_rate": 9.881411690413796e-05,
+      "loss": 0.6211562633514405,
+      "step": 140
+    },
+    {
+      "epoch": 0.30526315789473685,
+      "grad_norm": 0.6540597677230835,
+      "learning_rate": 9.868792762542814e-05,
+      "loss": 0.6841907501220703,
+      "step": 145
+    },
+    {
+      "epoch": 0.3157894736842105,
+      "grad_norm": 0.6657130122184753,
+      "learning_rate": 9.855544849681404e-05,
+      "loss": 0.6384834289550781,
+      "step": 150
+    },
+    {
+      "epoch": 0.3157894736842105,
+      "eval_loss": 0.6154849529266357,
+      "eval_runtime": 28.9994,
+      "eval_samples_per_second": 13.793,
+      "eval_steps_per_second": 13.793,
+      "step": 150
+    },
+    {
+      "epoch": 0.3263157894736842,
+      "grad_norm": 0.6323176622390747,
+      "learning_rate": 9.841669663288391e-05,
+      "loss": 0.6482361793518067,
+      "step": 155
+    },
+    {
+      "epoch": 0.3368421052631579,
+      "grad_norm": 0.6461365818977356,
+      "learning_rate": 9.82716899585822e-05,
+      "loss": 0.6088940620422363,
+      "step": 160
+    },
+    {
+      "epoch": 0.3473684210526316,
+      "grad_norm": 0.518051028251648,
+      "learning_rate": 9.812044720689387e-05,
+      "loss": 0.6474239826202393,
+      "step": 165
+    },
+    {
+      "epoch": 0.35789473684210527,
+      "grad_norm": 0.6599591374397278,
+      "learning_rate": 9.796298791642435e-05,
+      "loss": 0.639539098739624,
+      "step": 170
+    },
+    {
+      "epoch": 0.3684210526315789,
+      "grad_norm": 0.6555272340774536,
+      "learning_rate": 9.779933242887542e-05,
+      "loss": 0.6650691986083984,
+      "step": 175
+    },
+    {
+      "epoch": 0.37894736842105264,
+      "grad_norm": 0.5666244626045227,
+      "learning_rate": 9.762950188641728e-05,
+      "loss": 0.6851089000701904,
+      "step": 180
+    },
+    {
+      "epoch": 0.3894736842105263,
+      "grad_norm": 0.7169855833053589,
+      "learning_rate": 9.745351822895727e-05,
+      "loss": 0.5861614227294922,
+      "step": 185
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 0.5155227184295654,
+      "learning_rate": 9.727140419130553e-05,
+      "loss": 0.6075415134429931,
+      "step": 190
+    },
+    {
+      "epoch": 0.4105263157894737,
+      "grad_norm": 0.8202889561653137,
+      "learning_rate": 9.708318330023798e-05,
+      "loss": 0.6071089267730713,
+      "step": 195
+    },
+    {
+      "epoch": 0.42105263157894735,
+      "grad_norm": 0.4622310996055603,
+      "learning_rate": 9.688887987145691e-05,
+      "loss": 0.5945972919464111,
+      "step": 200
+    },
+    {
+      "epoch": 0.42105263157894735,
+      "eval_loss": 0.6035570502281189,
+      "eval_runtime": 27.8485,
+      "eval_samples_per_second": 14.363,
+      "eval_steps_per_second": 14.363,
+      "step": 200
+    },
+    {
+      "epoch": 0.43157894736842106,
+      "grad_norm": 0.6781761050224304,
+      "learning_rate": 9.668851900644975e-05,
+      "loss": 0.570584774017334,
+      "step": 205
+    },
+    {
+      "epoch": 0.4421052631578947,
+      "grad_norm": 0.5482363104820251,
+      "learning_rate": 9.648212658924625e-05,
+      "loss": 0.6329983234405517,
+      "step": 210
+    },
+    {
+      "epoch": 0.45263157894736844,
+      "grad_norm": 0.6346020698547363,
+      "learning_rate": 9.62697292830746e-05,
+      "loss": 0.6306288242340088,
+      "step": 215
+    },
+    {
+      "epoch": 0.4631578947368421,
+      "grad_norm": 0.7032224535942078,
+      "learning_rate": 9.6051354526917e-05,
+      "loss": 0.6967363357543945,
+      "step": 220
+    },
+    {
+      "epoch": 0.47368421052631576,
+      "grad_norm": 0.4979580044746399,
+      "learning_rate": 9.582703053196464e-05,
+      "loss": 0.5608481884002685,
+      "step": 225
+    },
+    {
+      "epoch": 0.4842105263157895,
+      "grad_norm": 0.7327916622161865,
+      "learning_rate": 9.55967862779735e-05,
+      "loss": 0.584914255142212,
+      "step": 230
+    },
+    {
+      "epoch": 0.49473684210526314,
+      "grad_norm": 0.7519303560256958,
+      "learning_rate": 9.536065150952025e-05,
+      "loss": 0.6214882850646972,
+      "step": 235
+    },
+    {
+      "epoch": 0.5052631578947369,
+      "grad_norm": 0.7328101396560669,
+      "learning_rate": 9.511865673215986e-05,
+      "loss": 0.6112482070922851,
+      "step": 240
+    },
+    {
+      "epoch": 0.5157894736842106,
+      "grad_norm": 0.5348976254463196,
+      "learning_rate": 9.487083320848454e-05,
+      "loss": 0.6417016983032227,
+      "step": 245
+    },
+    {
+      "epoch": 0.5263157894736842,
+      "grad_norm": 0.5690417885780334,
+      "learning_rate": 9.461721295408505e-05,
+      "loss": 0.5277723789215087,
+      "step": 250
+    },
+    {
+      "epoch": 0.5263157894736842,
+      "eval_loss": 0.5920669436454773,
+      "eval_runtime": 27.6132,
+      "eval_samples_per_second": 14.486,
+      "eval_steps_per_second": 14.486,
+      "step": 250
+    },
+    {
+      "epoch": 0.5368421052631579,
+      "grad_norm": 0.5612736344337463,
+      "learning_rate": 9.435782873341474e-05,
+      "loss": 0.5604990959167481,
+      "step": 255
+    },
+    {
+      "epoch": 0.5473684210526316,
+      "grad_norm": 0.6084933876991272,
+      "learning_rate": 9.409271405555677e-05,
+      "loss": 0.5558807373046875,
+      "step": 260
+    },
+    {
+      "epoch": 0.5578947368421052,
+      "grad_norm": 0.6289730668067932,
+      "learning_rate": 9.382190316989518e-05,
+      "loss": 0.5646376609802246,
+      "step": 265
+    },
+    {
+      "epoch": 0.5684210526315789,
+      "grad_norm": 0.6343289613723755,
+      "learning_rate": 9.354543106169029e-05,
+      "loss": 0.6692690372467041,
+      "step": 270
+    },
+    {
+      "epoch": 0.5789473684210527,
+      "grad_norm": 1.0037399530410767,
+      "learning_rate": 9.326333344755912e-05,
+      "loss": 0.6066013813018799,
+      "step": 275
+    },
+    {
+      "epoch": 0.5894736842105263,
+      "grad_norm": 0.8138774633407593,
+      "learning_rate": 9.297564677086118e-05,
+      "loss": 0.6074648857116699,
+      "step": 280
+    },
+    {
+      "epoch": 0.6,
+      "grad_norm": 1.0000449419021606,
+      "learning_rate": 9.268240819699054e-05,
+      "loss": 0.5891916275024414,
+      "step": 285
+    },
+    {
+      "epoch": 0.6105263157894737,
+      "grad_norm": 0.7454043030738831,
+      "learning_rate": 9.238365560857447e-05,
+      "loss": 0.6280234336853028,
+      "step": 290
+    },
+    {
+      "epoch": 0.6210526315789474,
+      "grad_norm": 0.756981611251831,
+      "learning_rate": 9.207942760057958e-05,
+      "loss": 0.6314973831176758,
+      "step": 295
+    },
+    {
+      "epoch": 0.631578947368421,
+      "grad_norm": 0.7583857178688049,
+      "learning_rate": 9.176976347532575e-05,
+      "loss": 0.6410815715789795,
+      "step": 300
+    },
+    {
+      "epoch": 0.631578947368421,
+      "eval_loss": 0.5863876342773438,
+      "eval_runtime": 27.6386,
+      "eval_samples_per_second": 14.472,
+      "eval_steps_per_second": 14.472,
+      "step": 300
+    },
+    {
+      "epoch": 0.6421052631578947,
+      "grad_norm": 0.791587769985199,
+      "learning_rate": 9.145470323740885e-05,
+      "loss": 0.5841981410980225,
+      "step": 305
+    },
+    {
+      "epoch": 0.6526315789473685,
+      "grad_norm": 1.1765379905700684,
+      "learning_rate": 9.113428758853268e-05,
+      "loss": 0.5464617729187011,
+      "step": 310
+    },
+    {
+      "epoch": 0.6631578947368421,
+      "grad_norm": 3.7840137481689453,
+      "learning_rate": 9.080855792225076e-05,
+      "loss": 0.6217821121215821,
+      "step": 315
+    },
+    {
+      "epoch": 0.6736842105263158,
+      "grad_norm": 1.2177997827529907,
+      "learning_rate": 9.047755631861884e-05,
+      "loss": 0.6125752449035644,
+      "step": 320
+    },
+    {
+      "epoch": 0.6842105263157895,
+      "grad_norm": 0.6323914527893066,
+      "learning_rate": 9.014132553875878e-05,
+      "loss": 0.5617509365081788,
+      "step": 325
+    },
+    {
+      "epoch": 0.6947368421052632,
+      "grad_norm": 0.9683524370193481,
+      "learning_rate": 8.979990901933428e-05,
+      "loss": 0.6395817756652832,
+      "step": 330
+    },
+    {
+      "epoch": 0.7052631578947368,
+      "grad_norm": 0.6486535668373108,
+      "learning_rate": 8.945335086693942e-05,
+      "loss": 0.6556002616882324,
+      "step": 335
+    },
+    {
+      "epoch": 0.7157894736842105,
+      "grad_norm": 1.642112374305725,
+      "learning_rate": 8.910169585240078e-05,
+      "loss": 0.588761568069458,
+      "step": 340
+    },
+    {
+      "epoch": 0.7263157894736842,
+      "grad_norm": 1.007238507270813,
+      "learning_rate": 8.874498940499346e-05,
+      "loss": 0.5744077682495117,
+      "step": 345
+    },
+    {
+      "epoch": 0.7368421052631579,
+      "grad_norm": 0.4945877194404602,
+      "learning_rate": 8.83832776065723e-05,
+      "loss": 0.5382843494415284,
+      "step": 350
+    },
+    {
+      "epoch": 0.7368421052631579,
+      "eval_loss": 0.5782474279403687,
+      "eval_runtime": 27.574,
+      "eval_samples_per_second": 14.506,
+      "eval_steps_per_second": 14.506,
+      "step": 350
+    },
+    {
+      "epoch": 0.7473684210526316,
+      "grad_norm": 0.7869954109191895,
+      "learning_rate": 8.801660718561875e-05,
+      "loss": 0.5697366237640381,
+      "step": 355
+    },
+    {
+      "epoch": 0.7578947368421053,
+      "grad_norm": 0.48311182856559753,
+      "learning_rate": 8.7645025511204e-05,
+      "loss": 0.6168550491333008,
+      "step": 360
+    },
+    {
+      "epoch": 0.7684210526315789,
+      "grad_norm": 0.5001690983772278,
+      "learning_rate": 8.726858058686968e-05,
+      "loss": 0.5362951278686523,
+      "step": 365
+    },
+    {
+      "epoch": 0.7789473684210526,
+      "grad_norm": 0.5915603041648865,
+      "learning_rate": 8.688732104442632e-05,
+      "loss": 0.6041481018066406,
+      "step": 370
+    },
+    {
+      "epoch": 0.7894736842105263,
+      "grad_norm": 0.5077712535858154,
+      "learning_rate": 8.650129613767075e-05,
+      "loss": 0.5257850646972656,
+      "step": 375
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.7458930015563965,
+      "learning_rate": 8.611055573602323e-05,
+      "loss": 0.6048945903778076,
+      "step": 380
+    },
+    {
+      "epoch": 0.8105263157894737,
+      "grad_norm": 0.6072155237197876,
+      "learning_rate": 8.571515031808484e-05,
+      "loss": 0.5505197048187256,
+      "step": 385
+    },
+    {
+      "epoch": 0.8210526315789474,
+      "grad_norm": 0.5989276170730591,
+      "learning_rate": 8.531513096511646e-05,
+      "loss": 0.528736162185669,
+      "step": 390
+    },
+    {
+      "epoch": 0.8315789473684211,
+      "grad_norm": 0.47586777806282043,
+      "learning_rate": 8.491054935443954e-05,
+      "loss": 0.5494171142578125,
+      "step": 395
+    },
+    {
+      "epoch": 0.8421052631578947,
+      "grad_norm": 0.6332405209541321,
+      "learning_rate": 8.450145775276024e-05,
+      "loss": 0.605548906326294,
+      "step": 400
+    },
+    {
+      "epoch": 0.8421052631578947,
+      "eval_loss": 0.5717179775238037,
+      "eval_runtime": 27.9225,
+      "eval_samples_per_second": 14.325,
+      "eval_steps_per_second": 14.325,
+      "step": 400
+    },
+    {
+      "epoch": 0.8526315789473684,
+      "grad_norm": 0.873306155204773,
+      "learning_rate": 8.40879090094171e-05,
+      "loss": 0.6015737056732178,
+      "step": 405
+    },
+    {
+      "epoch": 0.8631578947368421,
+      "grad_norm": 0.6360308527946472,
+      "learning_rate": 8.366995654955375e-05,
+      "loss": 0.5689235687255859,
+      "step": 410
+    },
+    {
+      "epoch": 0.8736842105263158,
+      "grad_norm": 0.7359456419944763,
+      "learning_rate": 8.324765436721688e-05,
+      "loss": 0.6225551128387451,
+      "step": 415
+    },
+    {
+      "epoch": 0.8842105263157894,
+      "grad_norm": 0.6135861873626709,
+      "learning_rate": 8.282105701838106e-05,
+      "loss": 0.567096471786499,
+      "step": 420
+    },
+    {
+      "epoch": 0.8947368421052632,
+      "grad_norm": 0.5699036717414856,
+      "learning_rate": 8.239021961390078e-05,
+      "loss": 0.5435199737548828,
+      "step": 425
+    },
+    {
+      "epoch": 0.9052631578947369,
+      "grad_norm": 0.6312369704246521,
+      "learning_rate": 8.195519781239079e-05,
+      "loss": 0.6320878028869629,
+      "step": 430
+    },
+    {
+      "epoch": 0.9157894736842105,
+      "grad_norm": 0.7718231678009033,
+      "learning_rate": 8.151604781303577e-05,
+      "loss": 0.5115500450134277,
+      "step": 435
+    },
+    {
+      "epoch": 0.9263157894736842,
+      "grad_norm": 0.7022733688354492,
+      "learning_rate": 8.107282634833015e-05,
+      "loss": 0.5616052627563477,
+      "step": 440
+    },
+    {
+      "epoch": 0.9368421052631579,
+      "grad_norm": 0.6376118659973145,
+      "learning_rate": 8.06255906767489e-05,
+      "loss": 0.6315895080566406,
+      "step": 445
+    },
+    {
+      "epoch": 0.9473684210526315,
+      "grad_norm": 0.5550912618637085,
+      "learning_rate": 8.01743985753506e-05,
+      "loss": 0.597140884399414,
+      "step": 450
+    },
+    {
+      "epoch": 0.9473684210526315,
+      "eval_loss": 0.5748050212860107,
+      "eval_runtime": 27.2437,
+      "eval_samples_per_second": 14.682,
+      "eval_steps_per_second": 14.682,
+      "step": 450
+    },
+    {
+      "epoch": 0.9578947368421052,
+      "grad_norm": 0.5438539385795593,
+      "learning_rate": 7.971930833231338e-05,
+      "loss": 0.5860416889190674,
+      "step": 455
+    },
+    {
+      "epoch": 0.968421052631579,
+      "grad_norm": 0.7530905604362488,
+      "learning_rate": 7.926037873940469e-05,
+      "loss": 0.5889331340789795,
+      "step": 460
+    },
+    {
+      "epoch": 0.9789473684210527,
+      "grad_norm": 0.6347785592079163,
+      "learning_rate": 7.879766908438638e-05,
+      "loss": 0.5784619808197021,
+      "step": 465
+    },
+    {
+      "epoch": 0.9894736842105263,
+      "grad_norm": 0.6019025444984436,
+      "learning_rate": 7.83312391433553e-05,
+      "loss": 0.5674707412719726,
+      "step": 470
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.5169152617454529,
+      "learning_rate": 7.786114917302118e-05,
+      "loss": 0.6072670936584472,
+      "step": 475
+    },
+    {
+      "epoch": 1.0105263157894737,
+      "grad_norm": 0.5598861575126648,
+      "learning_rate": 7.738745990292208e-05,
+      "loss": 0.5027533531188965,
+      "step": 480
+    },
+    {
+      "epoch": 1.0210526315789474,
+      "grad_norm": 0.5044442415237427,
+      "learning_rate": 7.691023252757901e-05,
+      "loss": 0.50599045753479,
+      "step": 485
+    },
+    {
+      "epoch": 1.0315789473684212,
+      "grad_norm": 0.5246402025222778,
+      "learning_rate": 7.64295286985904e-05,
+      "loss": 0.4706981658935547,
+      "step": 490
+    },
+    {
+      "epoch": 1.0421052631578946,
+      "grad_norm": 0.6998376846313477,
+      "learning_rate": 7.594541051666742e-05,
+      "loss": 0.48616642951965333,
+      "step": 495
+    },
+    {
+      "epoch": 1.0526315789473684,
+      "grad_norm": 0.5945510864257812,
+      "learning_rate": 7.545794052361149e-05,
+      "loss": 0.46446027755737307,
+      "step": 500
+    },
+    {
+      "epoch": 1.0526315789473684,
+      "eval_loss": 0.5771171450614929,
+      "eval_runtime": 27.1019,
+      "eval_samples_per_second": 14.759,
+      "eval_steps_per_second": 14.759,
+      "step": 500
+    },
+    {
+      "epoch": 1.063157894736842,
+      "grad_norm": 12.692610740661621,
+      "learning_rate": 7.496718169423462e-05,
+      "loss": 0.4399024486541748,
+      "step": 505
+    },
+    {
+      "epoch": 1.0736842105263158,
+      "grad_norm": 0.6038990020751953,
+      "learning_rate": 7.447319742822392e-05,
+      "loss": 0.48290238380432127,
+      "step": 510
+    },
+    {
+      "epoch": 1.0842105263157895,
+      "grad_norm": 0.6743545532226562,
+      "learning_rate": 7.397605154195106e-05,
+      "loss": 0.4822847366333008,
+      "step": 515
+    },
+    {
+      "epoch": 1.0947368421052632,
+      "grad_norm": 0.6746461391448975,
+      "learning_rate": 7.347580826022821e-05,
+      "loss": 0.47967071533203126,
+      "step": 520
+    },
+    {
+      "epoch": 1.1052631578947367,
+      "grad_norm": 0.7964761853218079,
+      "learning_rate": 7.29725322080109e-05,
+      "loss": 0.5180202484130859,
+      "step": 525
+    },
+    {
+      "epoch": 1.1157894736842104,
+      "grad_norm": 0.6772136688232422,
+      "learning_rate": 7.246628840204935e-05,
+      "loss": 0.46326284408569335,
+      "step": 530
+    },
+    {
+      "epoch": 1.1263157894736842,
+      "grad_norm": 0.7225517630577087,
+      "learning_rate": 7.195714224248912e-05,
+      "loss": 0.5262459278106689,
+      "step": 535
+    },
+    {
+      "epoch": 1.1368421052631579,
+      "grad_norm": 0.8155159950256348,
+      "learning_rate": 7.144515950442232e-05,
+      "loss": 0.5381614685058593,
+      "step": 540
+    },
+    {
+      "epoch": 1.1473684210526316,
+      "grad_norm": 0.7686448097229004,
+      "learning_rate": 7.093040632939023e-05,
+      "loss": 0.48169612884521484,
+      "step": 545
+    },
+    {
+      "epoch": 1.1578947368421053,
+      "grad_norm": 0.798007071018219,
+      "learning_rate": 7.041294921683876e-05,
+      "loss": 0.571818494796753,
+      "step": 550
+    },
+    {
+      "epoch": 1.1578947368421053,
+      "eval_loss": 0.5729017853736877,
+      "eval_runtime": 27.3923,
+      "eval_samples_per_second": 14.603,
+      "eval_steps_per_second": 14.603,
+      "step": 550
+    },
+    {
+      "epoch": 1.168421052631579,
+      "grad_norm": 0.9871010184288025,
+      "learning_rate": 6.989285501552751e-05,
+      "loss": 0.5277251243591309,
+      "step": 555
+    },
+    {
+      "epoch": 1.1789473684210527,
+      "grad_norm": 0.6931464076042175,
+      "learning_rate": 6.93701909148938e-05,
+      "loss": 0.4839730739593506,
+      "step": 560
+    },
+    {
+      "epoch": 1.1894736842105262,
+      "grad_norm": 0.6945406198501587,
+      "learning_rate": 6.884502443637273e-05,
+      "loss": 0.4632833480834961,
+      "step": 565
+    },
+    {
+      "epoch": 1.2,
+      "grad_norm": 0.8374159336090088,
+      "learning_rate": 6.831742342467418e-05,
+      "loss": 0.46635966300964354,
+      "step": 570
+    },
+    {
+      "epoch": 1.2105263157894737,
+      "grad_norm": 0.8007093667984009,
+      "learning_rate": 6.778745603901817e-05,
+      "loss": 0.4775552272796631,
+      "step": 575
+    },
+    {
+      "epoch": 1.2210526315789474,
+      "grad_norm": 0.922927975654602,
+      "learning_rate": 6.725519074432965e-05,
+      "loss": 0.41661796569824217,
+      "step": 580
+    },
+    {
+      "epoch": 1.231578947368421,
+      "grad_norm": 0.8915149569511414,
+      "learning_rate": 6.672069630239366e-05,
+      "loss": 0.46227364540100097,
+      "step": 585
+    },
+    {
+      "epoch": 1.2421052631578948,
+      "grad_norm": 0.7948114275932312,
+      "learning_rate": 6.618404176297217e-05,
+      "loss": 0.5104813575744629,
+      "step": 590
+    },
+    {
+      "epoch": 1.2526315789473683,
+      "grad_norm": 0.9380368590354919,
+      "learning_rate": 6.564529645488383e-05,
+      "loss": 0.4952188014984131,
+      "step": 595
+    },
+    {
+      "epoch": 1.263157894736842,
+      "grad_norm": 0.8004136085510254,
+      "learning_rate": 6.510452997704748e-05,
+      "loss": 0.4533547878265381,
+      "step": 600
+    },
+    {
+      "epoch": 1.263157894736842,
+      "eval_loss": 0.5720676779747009,
+      "eval_runtime": 27.2065,
+      "eval_samples_per_second": 14.702,
+      "eval_steps_per_second": 14.702,
+      "step": 600
+    },
+    {
+      "epoch": 1.2736842105263158,
+      "grad_norm": 0.8849948644638062,
+      "learning_rate": 6.456181218949096e-05,
+      "loss": 0.5185356140136719,
+      "step": 605
+    },
+    {
+      "epoch": 1.2842105263157895,
+      "grad_norm": 0.8258388638496399,
+      "learning_rate": 6.401721320432604e-05,
+      "loss": 0.45459442138671874,
+      "step": 610
+    },
+    {
+      "epoch": 1.2947368421052632,
+      "grad_norm": 0.7006433606147766,
+      "learning_rate": 6.34708033766909e-05,
+      "loss": 0.42599120140075686,
+      "step": 615
+    },
+    {
+      "epoch": 1.305263157894737,
+      "grad_norm": 0.8149921298027039,
+      "learning_rate": 6.292265329566108e-05,
+      "loss": 0.47135143280029296,
+      "step": 620
+    },
+    {
+      "epoch": 1.3157894736842106,
+      "grad_norm": 0.8304453492164612,
+      "learning_rate": 6.237283377513036e-05,
+      "loss": 0.543507719039917,
+      "step": 625
+    },
+    {
+      "epoch": 1.3263157894736843,
+      "grad_norm": 2.146064281463623,
+      "learning_rate": 6.182141584466247e-05,
+      "loss": 0.525644063949585,
+      "step": 630
+    },
+    {
+      "epoch": 1.3368421052631578,
+      "grad_norm": 0.8405390381813049,
+      "learning_rate": 6.126847074031507e-05,
+      "loss": 0.4994760513305664,
+      "step": 635
+    },
+    {
+      "epoch": 1.3473684210526315,
+      "grad_norm": 0.7558398842811584,
+      "learning_rate": 6.071406989543678e-05,
+      "loss": 0.45131878852844237,
+      "step": 640
+    },
+    {
+      "epoch": 1.3578947368421053,
+      "grad_norm": 1.3861230611801147,
+      "learning_rate": 6.0158284931439177e-05,
+      "loss": 0.45543656349182127,
+      "step": 645
+    },
+    {
+      "epoch": 1.368421052631579,
+      "grad_norm": 0.7160976529121399,
+      "learning_rate": 5.9601187648544056e-05,
+      "loss": 0.4914837837219238,
+      "step": 650
+    },
+    {
+      "epoch": 1.368421052631579,
+      "eval_loss": 0.566149890422821,
+      "eval_runtime": 27.1418,
+      "eval_samples_per_second": 14.737,
+      "eval_steps_per_second": 14.737,
+      "step": 650
+    },
+    {
+      "epoch": 1.3789473684210527,
+      "grad_norm": 0.7861437201499939,
+      "learning_rate": 5.904285001650783e-05,
+      "loss": 0.497259521484375,
+      "step": 655
+    },
+    {
+      "epoch": 1.3894736842105262,
+      "grad_norm": 0.7229910492897034,
+      "learning_rate": 5.8483344165323975e-05,
+      "loss": 0.5121944904327392,
+      "step": 660
+    },
+    {
+      "epoch": 1.4,
+      "grad_norm": 0.8752608299255371,
+      "learning_rate": 5.792274237590471e-05,
+      "loss": 0.5024206161499023,
+      "step": 665
+    },
+    {
+      "epoch": 1.4105263157894736,
+      "grad_norm": 0.8390534520149231,
+      "learning_rate": 5.7361117070743374e-05,
+      "loss": 0.48681044578552246,
+      "step": 670
+    },
+    {
+      "epoch": 1.4210526315789473,
+      "grad_norm": 0.8038208484649658,
+      "learning_rate": 5.679854080455821e-05,
+      "loss": 0.4971346378326416,
+      "step": 675
+    },
+    {
+      "epoch": 1.431578947368421,
+      "grad_norm": 0.7638700008392334,
+      "learning_rate": 5.6235086254919324e-05,
+      "loss": 0.42120847702026365,
+      "step": 680
+    },
+    {
+      "epoch": 1.4421052631578948,
+      "grad_norm": 0.7635045051574707,
+      "learning_rate": 5.567082621285969e-05,
+      "loss": 0.47087574005126953,
+      "step": 685
+    },
+    {
+      "epoch": 1.4526315789473685,
+      "grad_norm": 0.77474045753479,
+      "learning_rate": 5.510583357347149e-05,
+      "loss": 0.48073482513427734,
+      "step": 690
+    },
+    {
+      "epoch": 1.4631578947368422,
+      "grad_norm": 0.8228810429573059,
+      "learning_rate": 5.454018132648897e-05,
+      "loss": 0.46302366256713867,
+      "step": 695
+    },
+    {
+      "epoch": 1.4736842105263157,
+      "grad_norm": 0.8553083539009094,
+      "learning_rate": 5.3973942546859145e-05,
+      "loss": 0.47696647644042967,
+      "step": 700
+    },
+    {
+      "epoch": 1.4736842105263157,
+      "eval_loss": 0.5677127242088318,
+      "eval_runtime": 27.2564,
+      "eval_samples_per_second": 14.675,
+      "eval_steps_per_second": 14.675,
+      "step": 700
+    },
+    {
+      "epoch": 1.4842105263157894,
+      "grad_norm": 0.885428249835968,
+      "learning_rate": 5.3407190385301456e-05,
+      "loss": 0.426132345199585,
+      "step": 705
+    },
+    {
+      "epoch": 1.4947368421052631,
+      "grad_norm": 0.7517284154891968,
+      "learning_rate": 5.283999805885764e-05,
+      "loss": 0.5038762092590332,
+      "step": 710
+    },
+    {
+      "epoch": 1.5052631578947369,
+      "grad_norm": 0.6059720516204834,
+      "learning_rate": 5.227243884143306e-05,
+      "loss": 0.40855984687805175,
+      "step": 715
+    },
+    {
+      "epoch": 1.5157894736842106,
+      "grad_norm": 0.883471667766571,
+      "learning_rate": 5.170458605433059e-05,
+      "loss": 0.48758525848388673,
+      "step": 720
+    },
+    {
+      "epoch": 1.526315789473684,
+      "grad_norm": 0.8542289733886719,
+      "learning_rate": 5.113651305677856e-05,
+      "loss": 0.4801379680633545,
+      "step": 725
+    },
+    {
+      "epoch": 1.5368421052631578,
+      "grad_norm": 0.7095292210578918,
+      "learning_rate": 5.0568293236453614e-05,
+      "loss": 0.4451273441314697,
+      "step": 730
+    },
+    {
+      "epoch": 1.5473684210526315,
+      "grad_norm": 0.7466174960136414,
+      "learning_rate": 5e-05,
+      "loss": 0.4858893871307373,
+      "step": 735
+    },
+    {
+      "epoch": 1.5578947368421052,
+      "grad_norm": 0.8140954971313477,
+      "learning_rate": 4.94317067635464e-05,
+      "loss": 0.5117893695831299,
+      "step": 740
+    },
+    {
+      "epoch": 1.568421052631579,
+      "grad_norm": 0.8628697991371155,
+      "learning_rate": 4.886348694322145e-05,
+      "loss": 0.48410491943359374,
+      "step": 745
+    },
+    {
+      "epoch": 1.5789473684210527,
+      "grad_norm": 0.7672389149665833,
+      "learning_rate": 4.829541394566942e-05,
+      "loss": 0.4889042854309082,
+      "step": 750
+    },
+    {
+      "epoch": 1.5789473684210527,
+      "eval_loss": 0.5602394938468933,
+      "eval_runtime": 27.2938,
+      "eval_samples_per_second": 14.655,
+      "eval_steps_per_second": 14.655,
+      "step": 750
+    },
+    {
+      "epoch": 1.5894736842105264,
+      "grad_norm": 0.8665842413902283,
+      "learning_rate": 4.772756115856695e-05,
+      "loss": 0.4589890480041504,
+      "step": 755
+    },
+    {
+      "epoch": 1.6,
+      "grad_norm": 0.948796808719635,
+      "learning_rate": 4.7160001941142365e-05,
+      "loss": 0.4775981426239014,
+      "step": 760
+    },
+    {
+      "epoch": 1.6105263157894738,
+      "grad_norm": 0.8106200695037842,
+      "learning_rate": 4.6592809614698556e-05,
+      "loss": 0.5602635860443115,
+      "step": 765
+    },
+    {
+      "epoch": 1.6210526315789475,
+      "grad_norm": 1.2499663829803467,
+      "learning_rate": 4.602605745314087e-05,
+      "loss": 0.5022463321685791,
+      "step": 770
+    },
+    {
+      "epoch": 1.631578947368421,
+      "grad_norm": 0.8327662944793701,
+      "learning_rate": 4.545981867351104e-05,
+      "loss": 0.4951312065124512,
+      "step": 775
+    },
+    {
+      "epoch": 1.6421052631578947,
+      "grad_norm": 0.7749590277671814,
+      "learning_rate": 4.4894166426528524e-05,
+      "loss": 0.4875455379486084,
+      "step": 780
+    },
+    {
+      "epoch": 1.6526315789473685,
+      "grad_norm": 0.764395534992218,
+      "learning_rate": 4.432917378714032e-05,
+      "loss": 0.47612872123718264,
+      "step": 785
+    },
+    {
+      "epoch": 1.663157894736842,
+      "grad_norm": 0.8650004863739014,
+      "learning_rate": 4.3764913745080695e-05,
+      "loss": 0.4810147285461426,
+      "step": 790
+    },
+    {
+      "epoch": 1.6736842105263157,
+      "grad_norm": 0.852967381477356,
+      "learning_rate": 4.32014591954418e-05,
+      "loss": 0.46249094009399416,
+      "step": 795
+    },
+    {
+      "epoch": 1.6842105263157894,
+      "grad_norm": 0.752549946308136,
+      "learning_rate": 4.263888292925664e-05,
+      "loss": 0.463092565536499,
+      "step": 800
+    },
+    {
+      "epoch": 1.6842105263157894,
+      "eval_loss": 0.557573139667511,
+      "eval_runtime": 27.1956,
+      "eval_samples_per_second": 14.708,
+      "eval_steps_per_second": 14.708,
+      "step": 800
+    },
+    {
+      "epoch": 1.694736842105263,
+      "grad_norm": 0.7666327953338623,
+      "learning_rate": 4.207725762409529e-05,
+      "loss": 0.549990177154541,
+      "step": 805
+    },
+    {
+      "epoch": 1.7052631578947368,
+      "grad_norm": 1.2041642665863037,
+      "learning_rate": 4.151665583467604e-05,
+      "loss": 0.473134708404541,
+      "step": 810
+    },
+    {
+      "epoch": 1.7157894736842105,
+      "grad_norm": 0.9291930794715881,
+      "learning_rate": 4.095714998349218e-05,
+      "loss": 0.43207721710205077,
+      "step": 815
+    },
+    {
+      "epoch": 1.7263157894736842,
+      "grad_norm": 0.8421230316162109,
+      "learning_rate": 4.0398812351455955e-05,
+      "loss": 0.42284450531005857,
+      "step": 820
+    },
+    {
+      "epoch": 1.736842105263158,
+      "grad_norm": 0.7550533413887024,
+      "learning_rate": 3.9841715068560835e-05,
+      "loss": 0.4533982276916504,
+      "step": 825
+    },
+    {
+      "epoch": 1.7473684210526317,
+      "grad_norm": 0.7380033731460571,
+      "learning_rate": 3.9285930104563234e-05,
+      "loss": 0.47853550910949705,
+      "step": 830
+    },
+    {
+      "epoch": 1.7578947368421054,
+      "grad_norm": 1.0466768741607666,
+      "learning_rate": 3.873152925968495e-05,
+      "loss": 0.5334872245788574,
+      "step": 835
+    },
+    {
+      "epoch": 1.768421052631579,
+      "grad_norm": 0.8049618601799011,
+      "learning_rate": 3.8178584155337525e-05,
+      "loss": 0.4399109363555908,
+      "step": 840
+    },
+    {
+      "epoch": 1.7789473684210526,
+      "grad_norm": 0.7510071992874146,
+      "learning_rate": 3.762716622486965e-05,
+      "loss": 0.521062707901001,
+      "step": 845
+    },
+    {
+      "epoch": 1.7894736842105263,
+      "grad_norm": 1.6552070379257202,
+      "learning_rate": 3.7077346704338935e-05,
+      "loss": 0.45749530792236326,
+      "step": 850
+    },
+    {
+      "epoch": 1.7894736842105263,
+      "eval_loss": 0.552564799785614,
+      "eval_runtime": 27.3375,
+      "eval_samples_per_second": 14.632,
+      "eval_steps_per_second": 14.632,
+      "step": 850
+    },
+    {
+      "epoch": 1.8,
+      "grad_norm": 0.7396476864814758,
+      "learning_rate": 3.6529196623309115e-05,
+      "loss": 0.38660905361175535,
+      "step": 855
+    },
+    {
+      "epoch": 1.8105263157894735,
+      "grad_norm": 0.8277924656867981,
+      "learning_rate": 3.598278679567397e-05,
+      "loss": 0.5064235210418702,
+      "step": 860
+    },
+    {
+      "epoch": 1.8210526315789473,
+      "grad_norm": 0.9554834365844727,
+      "learning_rate": 3.543818781050906e-05,
+      "loss": 0.4140141010284424,
+      "step": 865
+    },
+    {
+      "epoch": 1.831578947368421,
+      "grad_norm": 0.7180286049842834,
+      "learning_rate": 3.4895470022952536e-05,
+      "loss": 0.5401397705078125,
+      "step": 870
+    },
+    {
+      "epoch": 1.8421052631578947,
+      "grad_norm": 0.972764790058136,
+      "learning_rate": 3.4354703545116185e-05,
+      "loss": 0.504509973526001,
+      "step": 875
+    },
+    {
+      "epoch": 1.8526315789473684,
+      "grad_norm": 0.7626535892486572,
+      "learning_rate": 3.381595823702784e-05,
+      "loss": 0.46036224365234374,
+      "step": 880
+    },
+    {
+      "epoch": 1.8631578947368421,
+      "grad_norm": 0.9077037572860718,
+      "learning_rate": 3.3279303697606354e-05,
+      "loss": 0.4127952098846436,
+      "step": 885
+    },
+    {
+      "epoch": 1.8736842105263158,
+      "grad_norm": 0.764255702495575,
+      "learning_rate": 3.274480925567036e-05,
+      "loss": 0.4129067897796631,
+      "step": 890
+    },
+    {
+      "epoch": 1.8842105263157896,
+      "grad_norm": 0.9645035266876221,
+      "learning_rate": 3.2212543960981845e-05,
+      "loss": 0.486614990234375,
+      "step": 895
+    },
+    {
+      "epoch": 1.8947368421052633,
+      "grad_norm": 0.7822158932685852,
+      "learning_rate": 3.168257657532584e-05,
+      "loss": 0.43874049186706543,
+      "step": 900
+    },
+    {
+      "epoch": 1.8947368421052633,
+      "eval_loss": 0.5511002540588379,
+      "eval_runtime": 27.8563,
+      "eval_samples_per_second": 14.359,
+      "eval_steps_per_second": 14.359,
+      "step": 900
+    },
+    {
+      "epoch": 1.905263157894737,
+      "grad_norm": 0.823441207408905,
+      "learning_rate": 3.115497556362727e-05,
+      "loss": 0.4594260692596436,
+      "step": 905
+    },
+    {
+      "epoch": 1.9157894736842105,
+      "grad_norm": 0.821818470954895,
+      "learning_rate": 3.06298090851062e-05,
+      "loss": 0.4467416763305664,
+      "step": 910
+    },
+    {
+      "epoch": 1.9263157894736842,
+      "grad_norm": 0.8790761232376099,
+      "learning_rate": 3.0107144984472502e-05,
+      "loss": 0.4823316097259521,
+      "step": 915
+    },
+    {
+      "epoch": 1.936842105263158,
+      "grad_norm": 0.7914945483207703,
+      "learning_rate": 2.9587050783161252e-05,
+      "loss": 0.4314936637878418,
+      "step": 920
+    },
+    {
+      "epoch": 1.9473684210526314,
+      "grad_norm": 0.805029571056366,
+      "learning_rate": 2.9069593670609775e-05,
+      "loss": 0.4740307331085205,
+      "step": 925
+    },
+    {
+      "epoch": 1.9578947368421051,
+      "grad_norm": 0.7928342223167419,
+      "learning_rate": 2.8554840495577682e-05,
+      "loss": 0.4155329704284668,
+      "step": 930
+    },
+    {
+      "epoch": 1.9684210526315788,
+      "grad_norm": 0.8041934370994568,
+      "learning_rate": 2.8042857757510877e-05,
+      "loss": 0.5049230098724365,
+      "step": 935
+    },
+    {
+      "epoch": 1.9789473684210526,
+      "grad_norm": 0.8169903755187988,
+      "learning_rate": 2.753371159795065e-05,
+      "loss": 0.3966445684432983,
+      "step": 940
+    },
+    {
+      "epoch": 1.9894736842105263,
+      "grad_norm": 0.86372309923172,
+      "learning_rate": 2.702746779198912e-05,
+      "loss": 0.5153264999389648,
+      "step": 945
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.97676020860672,
+      "learning_rate": 2.6524191739771815e-05,
+      "loss": 0.4318437099456787,
+      "step": 950
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.5508978962898254,
+      "eval_runtime": 27.8163,
+      "eval_samples_per_second": 14.38,
+      "eval_steps_per_second": 14.38,
+      "step": 950
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 1425,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 50,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.602735927581737e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-950/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:757b7d78e07bd589f7cd9db1a1a3019f58127f01f91cba75e4a6cbb7ea4b4a6c
+size 5713