Training in progress, step 200, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0409da1086edbc2028c336b653dae9d611a2ab91e30da2994f81efc62cc28b4b
+size 323014168

last-checkpoint/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37b1965037800682a978d82e22194ae3c66d1bd084495b41052dd3cb428f2629
+size 164464564

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b92f2f9d4d6a9a319559b2b2c4ae60261e0254bb401d4132de53c6fdea81e5d4
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5a199a26d86813df5ece274ab1439d2c840332fa531fa36edaa4679b0c6ee56
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
+size 11421896

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,207 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,1458 @@
+{
+  "best_metric": 1.6234830617904663,
+  "best_model_checkpoint": "miner_id_24/checkpoint-200",
+  "epoch": 0.03214788024914607,
+  "eval_steps": 200,
+  "global_step": 200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.00016073940124573035,
+      "grad_norm": 0.6146062612533569,
+      "learning_rate": 6.666666666666667e-06,
+      "loss": 2.4562,
+      "step": 1
+    },
+    {
+      "epoch": 0.00016073940124573035,
+      "eval_loss": 1.8800766468048096,
+      "eval_runtime": 46.1041,
+      "eval_samples_per_second": 5.444,
+      "eval_steps_per_second": 2.733,
+      "step": 1
+    },
+    {
+      "epoch": 0.0003214788024914607,
+      "grad_norm": 0.5687244534492493,
+      "learning_rate": 1.3333333333333333e-05,
+      "loss": 1.9791,
+      "step": 2
+    },
+    {
+      "epoch": 0.0004822182037371911,
+      "grad_norm": 0.5558443665504456,
+      "learning_rate": 2e-05,
+      "loss": 2.0161,
+      "step": 3
+    },
+    {
+      "epoch": 0.0006429576049829214,
+      "grad_norm": 0.5337421298027039,
+      "learning_rate": 2.6666666666666667e-05,
+      "loss": 1.9071,
+      "step": 4
+    },
+    {
+      "epoch": 0.0008036970062286518,
+      "grad_norm": 0.5502483248710632,
+      "learning_rate": 3.3333333333333335e-05,
+      "loss": 2.0349,
+      "step": 5
+    },
+    {
+      "epoch": 0.0009644364074743822,
+      "grad_norm": 0.546974778175354,
+      "learning_rate": 4e-05,
+      "loss": 2.1145,
+      "step": 6
+    },
+    {
+      "epoch": 0.0011251758087201125,
+      "grad_norm": 0.6227491497993469,
+      "learning_rate": 4.666666666666667e-05,
+      "loss": 2.0561,
+      "step": 7
+    },
+    {
+      "epoch": 0.0012859152099658428,
+      "grad_norm": 0.5480761528015137,
+      "learning_rate": 5.333333333333333e-05,
+      "loss": 1.7662,
+      "step": 8
+    },
+    {
+      "epoch": 0.0014466546112115732,
+      "grad_norm": 0.5654782652854919,
+      "learning_rate": 6e-05,
+      "loss": 1.842,
+      "step": 9
+    },
+    {
+      "epoch": 0.0016073940124573037,
+      "grad_norm": 0.5720047950744629,
+      "learning_rate": 6.666666666666667e-05,
+      "loss": 1.7238,
+      "step": 10
+    },
+    {
+      "epoch": 0.001768133413703034,
+      "grad_norm": 0.488892138004303,
+      "learning_rate": 7.333333333333333e-05,
+      "loss": 1.5829,
+      "step": 11
+    },
+    {
+      "epoch": 0.0019288728149487644,
+      "grad_norm": 0.5832931995391846,
+      "learning_rate": 8e-05,
+      "loss": 1.9504,
+      "step": 12
+    },
+    {
+      "epoch": 0.002089612216194495,
+      "grad_norm": 0.5047398805618286,
+      "learning_rate": 8.666666666666667e-05,
+      "loss": 1.6885,
+      "step": 13
+    },
+    {
+      "epoch": 0.002250351617440225,
+      "grad_norm": 0.5305578112602234,
+      "learning_rate": 9.333333333333334e-05,
+      "loss": 1.6642,
+      "step": 14
+    },
+    {
+      "epoch": 0.0024110910186859553,
+      "grad_norm": 0.57685786485672,
+      "learning_rate": 0.0001,
+      "loss": 1.8461,
+      "step": 15
+    },
+    {
+      "epoch": 0.0025718304199316856,
+      "grad_norm": 0.654364287853241,
+      "learning_rate": 0.00010666666666666667,
+      "loss": 1.7632,
+      "step": 16
+    },
+    {
+      "epoch": 0.0027325698211774162,
+      "grad_norm": 0.6196379661560059,
+      "learning_rate": 0.00011333333333333334,
+      "loss": 1.6611,
+      "step": 17
+    },
+    {
+      "epoch": 0.0028933092224231465,
+      "grad_norm": 0.6956469416618347,
+      "learning_rate": 0.00012,
+      "loss": 1.795,
+      "step": 18
+    },
+    {
+      "epoch": 0.0030540486236688767,
+      "grad_norm": 0.8601729273796082,
+      "learning_rate": 0.00012666666666666666,
+      "loss": 2.1705,
+      "step": 19
+    },
+    {
+      "epoch": 0.0032147880249146074,
+      "grad_norm": 0.6868798136711121,
+      "learning_rate": 0.00013333333333333334,
+      "loss": 1.7211,
+      "step": 20
+    },
+    {
+      "epoch": 0.0033755274261603376,
+      "grad_norm": 0.6202433705329895,
+      "learning_rate": 0.00014,
+      "loss": 1.5298,
+      "step": 21
+    },
+    {
+      "epoch": 0.003536266827406068,
+      "grad_norm": 0.6545760035514832,
+      "learning_rate": 0.00014666666666666666,
+      "loss": 1.9106,
+      "step": 22
+    },
+    {
+      "epoch": 0.003697006228651798,
+      "grad_norm": 0.6812022924423218,
+      "learning_rate": 0.00015333333333333334,
+      "loss": 1.7376,
+      "step": 23
+    },
+    {
+      "epoch": 0.0038577456298975288,
+      "grad_norm": 0.6827796697616577,
+      "learning_rate": 0.00016,
+      "loss": 1.6954,
+      "step": 24
+    },
+    {
+      "epoch": 0.004018485031143259,
+      "grad_norm": 0.6145235300064087,
+      "learning_rate": 0.0001666666666666667,
+      "loss": 1.963,
+      "step": 25
+    },
+    {
+      "epoch": 0.00417922443238899,
+      "grad_norm": 0.6488357186317444,
+      "learning_rate": 0.00017333333333333334,
+      "loss": 1.8421,
+      "step": 26
+    },
+    {
+      "epoch": 0.00433996383363472,
+      "grad_norm": 0.6817061901092529,
+      "learning_rate": 0.00018,
+      "loss": 1.7975,
+      "step": 27
+    },
+    {
+      "epoch": 0.00450070323488045,
+      "grad_norm": 0.7456655502319336,
+      "learning_rate": 0.0001866666666666667,
+      "loss": 1.614,
+      "step": 28
+    },
+    {
+      "epoch": 0.00466144263612618,
+      "grad_norm": 0.5470588207244873,
+      "learning_rate": 0.00019333333333333333,
+      "loss": 1.4163,
+      "step": 29
+    },
+    {
+      "epoch": 0.004822182037371911,
+      "grad_norm": 0.7085727453231812,
+      "learning_rate": 0.0002,
+      "loss": 1.6551,
+      "step": 30
+    },
+    {
+      "epoch": 0.004982921438617641,
+      "grad_norm": 0.8203132152557373,
+      "learning_rate": 0.00019999999857863972,
+      "loss": 1.7143,
+      "step": 31
+    },
+    {
+      "epoch": 0.005143660839863371,
+      "grad_norm": 0.6805604696273804,
+      "learning_rate": 0.00019999999431455888,
+      "loss": 1.8898,
+      "step": 32
+    },
+    {
+      "epoch": 0.005304400241109102,
+      "grad_norm": 0.6662057638168335,
+      "learning_rate": 0.00019999998720775754,
+      "loss": 1.5896,
+      "step": 33
+    },
+    {
+      "epoch": 0.0054651396423548325,
+      "grad_norm": 0.7231802344322205,
+      "learning_rate": 0.00019999997725823603,
+      "loss": 1.7113,
+      "step": 34
+    },
+    {
+      "epoch": 0.005625879043600563,
+      "grad_norm": 0.7186694145202637,
+      "learning_rate": 0.00019999996446599453,
+      "loss": 1.7156,
+      "step": 35
+    },
+    {
+      "epoch": 0.005786618444846293,
+      "grad_norm": 0.5861710906028748,
+      "learning_rate": 0.00019999994883103348,
+      "loss": 1.7798,
+      "step": 36
+    },
+    {
+      "epoch": 0.005947357846092023,
+      "grad_norm": 0.622209906578064,
+      "learning_rate": 0.00019999993035335327,
+      "loss": 1.4324,
+      "step": 37
+    },
+    {
+      "epoch": 0.006108097247337753,
+      "grad_norm": 0.6815835237503052,
+      "learning_rate": 0.00019999990903295443,
+      "loss": 1.8903,
+      "step": 38
+    },
+    {
+      "epoch": 0.006268836648583484,
+      "grad_norm": 0.6280589699745178,
+      "learning_rate": 0.00019999988486983758,
+      "loss": 1.7587,
+      "step": 39
+    },
+    {
+      "epoch": 0.006429576049829215,
+      "grad_norm": 0.5866552591323853,
+      "learning_rate": 0.00019999985786400343,
+      "loss": 1.533,
+      "step": 40
+    },
+    {
+      "epoch": 0.006590315451074945,
+      "grad_norm": 0.6842410564422607,
+      "learning_rate": 0.0001999998280154527,
+      "loss": 1.6896,
+      "step": 41
+    },
+    {
+      "epoch": 0.006751054852320675,
+      "grad_norm": 0.6225414276123047,
+      "learning_rate": 0.00019999979532418625,
+      "loss": 1.7056,
+      "step": 42
+    },
+    {
+      "epoch": 0.0069117942535664055,
+      "grad_norm": 0.5919414758682251,
+      "learning_rate": 0.00019999975979020506,
+      "loss": 1.6127,
+      "step": 43
+    },
+    {
+      "epoch": 0.007072533654812136,
+      "grad_norm": 0.7076655030250549,
+      "learning_rate": 0.00019999972141351003,
+      "loss": 1.7316,
+      "step": 44
+    },
+    {
+      "epoch": 0.007233273056057866,
+      "grad_norm": 0.693417489528656,
+      "learning_rate": 0.00019999968019410237,
+      "loss": 1.6352,
+      "step": 45
+    },
+    {
+      "epoch": 0.007394012457303596,
+      "grad_norm": 0.640809178352356,
+      "learning_rate": 0.00019999963613198324,
+      "loss": 2.0136,
+      "step": 46
+    },
+    {
+      "epoch": 0.007554751858549327,
+      "grad_norm": 0.6811522841453552,
+      "learning_rate": 0.00019999958922715377,
+      "loss": 1.9979,
+      "step": 47
+    },
+    {
+      "epoch": 0.0077154912597950575,
+      "grad_norm": 0.6250655055046082,
+      "learning_rate": 0.00019999953947961543,
+      "loss": 1.8884,
+      "step": 48
+    },
+    {
+      "epoch": 0.007876230661040788,
+      "grad_norm": 0.6390895843505859,
+      "learning_rate": 0.00019999948688936958,
+      "loss": 1.8005,
+      "step": 49
+    },
+    {
+      "epoch": 0.008036970062286517,
+      "grad_norm": 0.7204399704933167,
+      "learning_rate": 0.0001999994314564177,
+      "loss": 1.848,
+      "step": 50
+    },
+    {
+      "epoch": 0.008197709463532248,
+      "grad_norm": 0.6431483626365662,
+      "learning_rate": 0.0001999993731807614,
+      "loss": 1.5865,
+      "step": 51
+    },
+    {
+      "epoch": 0.00835844886477798,
+      "grad_norm": 0.6491577625274658,
+      "learning_rate": 0.00019999931206240233,
+      "loss": 1.5799,
+      "step": 52
+    },
+    {
+      "epoch": 0.008519188266023709,
+      "grad_norm": 0.6422392129898071,
+      "learning_rate": 0.00019999924810134217,
+      "loss": 1.8216,
+      "step": 53
+    },
+    {
+      "epoch": 0.00867992766726944,
+      "grad_norm": 0.6124479174613953,
+      "learning_rate": 0.00019999918129758284,
+      "loss": 1.499,
+      "step": 54
+    },
+    {
+      "epoch": 0.00884066706851517,
+      "grad_norm": 0.5777962803840637,
+      "learning_rate": 0.00019999911165112616,
+      "loss": 1.5374,
+      "step": 55
+    },
+    {
+      "epoch": 0.0090014064697609,
+      "grad_norm": 0.5487947463989258,
+      "learning_rate": 0.00019999903916197417,
+      "loss": 1.8562,
+      "step": 56
+    },
+    {
+      "epoch": 0.00916214587100663,
+      "grad_norm": 0.5540621876716614,
+      "learning_rate": 0.00019999896383012883,
+      "loss": 1.4139,
+      "step": 57
+    },
+    {
+      "epoch": 0.00932288527225236,
+      "grad_norm": 0.5495641231536865,
+      "learning_rate": 0.00019999888565559238,
+      "loss": 1.7241,
+      "step": 58
+    },
+    {
+      "epoch": 0.009483624673498092,
+      "grad_norm": 0.5788256525993347,
+      "learning_rate": 0.000199998804638367,
+      "loss": 1.5788,
+      "step": 59
+    },
+    {
+      "epoch": 0.009644364074743821,
+      "grad_norm": 0.5524935126304626,
+      "learning_rate": 0.00019999872077845502,
+      "loss": 1.6883,
+      "step": 60
+    },
+    {
+      "epoch": 0.009805103475989552,
+      "grad_norm": 0.5422489643096924,
+      "learning_rate": 0.0001999986340758588,
+      "loss": 1.5146,
+      "step": 61
+    },
+    {
+      "epoch": 0.009965842877235282,
+      "grad_norm": 0.559472918510437,
+      "learning_rate": 0.00019999854453058081,
+      "loss": 1.6218,
+      "step": 62
+    },
+    {
+      "epoch": 0.010126582278481013,
+      "grad_norm": 0.6507068872451782,
+      "learning_rate": 0.0001999984521426236,
+      "loss": 1.5814,
+      "step": 63
+    },
+    {
+      "epoch": 0.010287321679726742,
+      "grad_norm": 0.5180988311767578,
+      "learning_rate": 0.00019999835691198977,
+      "loss": 1.4473,
+      "step": 64
+    },
+    {
+      "epoch": 0.010448061080972473,
+      "grad_norm": 0.6105542778968811,
+      "learning_rate": 0.00019999825883868207,
+      "loss": 1.3512,
+      "step": 65
+    },
+    {
+      "epoch": 0.010608800482218204,
+      "grad_norm": 0.6106534004211426,
+      "learning_rate": 0.0001999981579227033,
+      "loss": 1.7397,
+      "step": 66
+    },
+    {
+      "epoch": 0.010769539883463934,
+      "grad_norm": 0.5836473107337952,
+      "learning_rate": 0.00019999805416405625,
+      "loss": 1.6683,
+      "step": 67
+    },
+    {
+      "epoch": 0.010930279284709665,
+      "grad_norm": 0.5084603428840637,
+      "learning_rate": 0.00019999794756274394,
+      "loss": 1.391,
+      "step": 68
+    },
+    {
+      "epoch": 0.011091018685955394,
+      "grad_norm": 0.5803837776184082,
+      "learning_rate": 0.00019999783811876936,
+      "loss": 1.5025,
+      "step": 69
+    },
+    {
+      "epoch": 0.011251758087201125,
+      "grad_norm": 0.6616635322570801,
+      "learning_rate": 0.00019999772583213562,
+      "loss": 1.7606,
+      "step": 70
+    },
+    {
+      "epoch": 0.011412497488446855,
+      "grad_norm": 0.5798701643943787,
+      "learning_rate": 0.00019999761070284598,
+      "loss": 1.6723,
+      "step": 71
+    },
+    {
+      "epoch": 0.011573236889692586,
+      "grad_norm": 0.5627177953720093,
+      "learning_rate": 0.00019999749273090362,
+      "loss": 1.599,
+      "step": 72
+    },
+    {
+      "epoch": 0.011733976290938317,
+      "grad_norm": 0.5469239950180054,
+      "learning_rate": 0.00019999737191631197,
+      "loss": 1.4389,
+      "step": 73
+    },
+    {
+      "epoch": 0.011894715692184046,
+      "grad_norm": 0.6494703888893127,
+      "learning_rate": 0.0001999972482590744,
+      "loss": 1.5761,
+      "step": 74
+    },
+    {
+      "epoch": 0.012055455093429777,
+      "grad_norm": 0.6485840678215027,
+      "learning_rate": 0.00019999712175919449,
+      "loss": 1.6965,
+      "step": 75
+    },
+    {
+      "epoch": 0.012216194494675507,
+      "grad_norm": 0.5390245318412781,
+      "learning_rate": 0.00019999699241667575,
+      "loss": 1.4385,
+      "step": 76
+    },
+    {
+      "epoch": 0.012376933895921238,
+      "grad_norm": 0.5867407321929932,
+      "learning_rate": 0.00019999686023152195,
+      "loss": 1.6617,
+      "step": 77
+    },
+    {
+      "epoch": 0.012537673297166967,
+      "grad_norm": 0.5673894882202148,
+      "learning_rate": 0.0001999967252037368,
+      "loss": 1.6431,
+      "step": 78
+    },
+    {
+      "epoch": 0.012698412698412698,
+      "grad_norm": 0.6245514750480652,
+      "learning_rate": 0.00019999658733332415,
+      "loss": 1.7781,
+      "step": 79
+    },
+    {
+      "epoch": 0.01285915209965843,
+      "grad_norm": 0.5269338488578796,
+      "learning_rate": 0.0001999964466202879,
+      "loss": 1.4988,
+      "step": 80
+    },
+    {
+      "epoch": 0.013019891500904159,
+      "grad_norm": 0.64048832654953,
+      "learning_rate": 0.0001999963030646321,
+      "loss": 1.4648,
+      "step": 81
+    },
+    {
+      "epoch": 0.01318063090214989,
+      "grad_norm": 0.6036165356636047,
+      "learning_rate": 0.00019999615666636074,
+      "loss": 1.5798,
+      "step": 82
+    },
+    {
+      "epoch": 0.01334137030339562,
+      "grad_norm": 0.5646976828575134,
+      "learning_rate": 0.00019999600742547807,
+      "loss": 1.796,
+      "step": 83
+    },
+    {
+      "epoch": 0.01350210970464135,
+      "grad_norm": 0.6046854853630066,
+      "learning_rate": 0.00019999585534198828,
+      "loss": 1.6529,
+      "step": 84
+    },
+    {
+      "epoch": 0.01366284910588708,
+      "grad_norm": 0.6673730611801147,
+      "learning_rate": 0.0001999957004158957,
+      "loss": 1.6296,
+      "step": 85
+    },
+    {
+      "epoch": 0.013823588507132811,
+      "grad_norm": 0.5602283477783203,
+      "learning_rate": 0.0001999955426472048,
+      "loss": 1.7198,
+      "step": 86
+    },
+    {
+      "epoch": 0.013984327908378542,
+      "grad_norm": 0.6285644769668579,
+      "learning_rate": 0.00019999538203591997,
+      "loss": 1.5761,
+      "step": 87
+    },
+    {
+      "epoch": 0.014145067309624271,
+      "grad_norm": 0.6506261825561523,
+      "learning_rate": 0.0001999952185820458,
+      "loss": 1.9915,
+      "step": 88
+    },
+    {
+      "epoch": 0.014305806710870003,
+      "grad_norm": 0.6009521484375,
+      "learning_rate": 0.000199995052285587,
+      "loss": 1.5645,
+      "step": 89
+    },
+    {
+      "epoch": 0.014466546112115732,
+      "grad_norm": 0.6257418990135193,
+      "learning_rate": 0.00019999488314654827,
+      "loss": 1.6225,
+      "step": 90
+    },
+    {
+      "epoch": 0.014627285513361463,
+      "grad_norm": 0.5345893502235413,
+      "learning_rate": 0.00019999471116493434,
+      "loss": 1.4062,
+      "step": 91
+    },
+    {
+      "epoch": 0.014788024914607192,
+      "grad_norm": 0.5701767802238464,
+      "learning_rate": 0.00019999453634075018,
+      "loss": 1.6436,
+      "step": 92
+    },
+    {
+      "epoch": 0.014948764315852923,
+      "grad_norm": 0.6432588696479797,
+      "learning_rate": 0.00019999435867400073,
+      "loss": 1.5179,
+      "step": 93
+    },
+    {
+      "epoch": 0.015109503717098655,
+      "grad_norm": 0.5169962644577026,
+      "learning_rate": 0.00019999417816469106,
+      "loss": 1.6126,
+      "step": 94
+    },
+    {
+      "epoch": 0.015270243118344384,
+      "grad_norm": 0.5223938226699829,
+      "learning_rate": 0.0001999939948128263,
+      "loss": 1.5733,
+      "step": 95
+    },
+    {
+      "epoch": 0.015430982519590115,
+      "grad_norm": 0.5852014422416687,
+      "learning_rate": 0.00019999380861841163,
+      "loss": 1.7466,
+      "step": 96
+    },
+    {
+      "epoch": 0.015591721920835844,
+      "grad_norm": 0.5293176174163818,
+      "learning_rate": 0.00019999361958145237,
+      "loss": 1.5117,
+      "step": 97
+    },
+    {
+      "epoch": 0.015752461322081576,
+      "grad_norm": 0.5117928981781006,
+      "learning_rate": 0.0001999934277019539,
+      "loss": 1.429,
+      "step": 98
+    },
+    {
+      "epoch": 0.015913200723327307,
+      "grad_norm": 0.4751776158809662,
+      "learning_rate": 0.00019999323297992169,
+      "loss": 1.3713,
+      "step": 99
+    },
+    {
+      "epoch": 0.016073940124573034,
+      "grad_norm": 0.5721314549446106,
+      "learning_rate": 0.0001999930354153612,
+      "loss": 1.3649,
+      "step": 100
+    },
+    {
+      "epoch": 0.016234679525818765,
+      "grad_norm": 0.5435962677001953,
+      "learning_rate": 0.00019999283500827812,
+      "loss": 1.54,
+      "step": 101
+    },
+    {
+      "epoch": 0.016395418927064496,
+      "grad_norm": 0.4738254249095917,
+      "learning_rate": 0.0001999926317586781,
+      "loss": 1.2795,
+      "step": 102
+    },
+    {
+      "epoch": 0.016556158328310228,
+      "grad_norm": 0.5297900438308716,
+      "learning_rate": 0.00019999242566656698,
+      "loss": 1.309,
+      "step": 103
+    },
+    {
+      "epoch": 0.01671689772955596,
+      "grad_norm": 0.5342819690704346,
+      "learning_rate": 0.00019999221673195052,
+      "loss": 1.7379,
+      "step": 104
+    },
+    {
+      "epoch": 0.016877637130801686,
+      "grad_norm": 0.5869962573051453,
+      "learning_rate": 0.00019999200495483477,
+      "loss": 1.5729,
+      "step": 105
+    },
+    {
+      "epoch": 0.017038376532047417,
+      "grad_norm": 0.5355800986289978,
+      "learning_rate": 0.00019999179033522565,
+      "loss": 1.5899,
+      "step": 106
+    },
+    {
+      "epoch": 0.01719911593329315,
+      "grad_norm": 0.6515146493911743,
+      "learning_rate": 0.00019999157287312936,
+      "loss": 1.7698,
+      "step": 107
+    },
+    {
+      "epoch": 0.01735985533453888,
+      "grad_norm": 0.5469892024993896,
+      "learning_rate": 0.00019999135256855202,
+      "loss": 1.4373,
+      "step": 108
+    },
+    {
+      "epoch": 0.01752059473578461,
+      "grad_norm": 0.572085976600647,
+      "learning_rate": 0.0001999911294214999,
+      "loss": 1.5545,
+      "step": 109
+    },
+    {
+      "epoch": 0.01768133413703034,
+      "grad_norm": 0.6045322418212891,
+      "learning_rate": 0.00019999090343197936,
+      "loss": 1.6601,
+      "step": 110
+    },
+    {
+      "epoch": 0.01784207353827607,
+      "grad_norm": 0.5288821458816528,
+      "learning_rate": 0.00019999067459999676,
+      "loss": 1.4338,
+      "step": 111
+    },
+    {
+      "epoch": 0.0180028129395218,
+      "grad_norm": 0.6269596815109253,
+      "learning_rate": 0.0001999904429255587,
+      "loss": 1.7966,
+      "step": 112
+    },
+    {
+      "epoch": 0.01816355234076753,
+      "grad_norm": 0.6155001521110535,
+      "learning_rate": 0.0001999902084086717,
+      "loss": 1.4822,
+      "step": 113
+    },
+    {
+      "epoch": 0.01832429174201326,
+      "grad_norm": 0.5912888050079346,
+      "learning_rate": 0.00019998997104934247,
+      "loss": 1.5701,
+      "step": 114
+    },
+    {
+      "epoch": 0.01848503114325899,
+      "grad_norm": 0.5434785485267639,
+      "learning_rate": 0.00019998973084757771,
+      "loss": 1.5136,
+      "step": 115
+    },
+    {
+      "epoch": 0.01864577054450472,
+      "grad_norm": 0.6013758182525635,
+      "learning_rate": 0.0001999894878033843,
+      "loss": 1.6653,
+      "step": 116
+    },
+    {
+      "epoch": 0.018806509945750453,
+      "grad_norm": 0.5760058760643005,
+      "learning_rate": 0.00019998924191676914,
+      "loss": 1.5809,
+      "step": 117
+    },
+    {
+      "epoch": 0.018967249346996184,
+      "grad_norm": 0.5517994165420532,
+      "learning_rate": 0.00019998899318773917,
+      "loss": 1.598,
+      "step": 118
+    },
+    {
+      "epoch": 0.01912798874824191,
+      "grad_norm": 0.6126778721809387,
+      "learning_rate": 0.0001999887416163015,
+      "loss": 1.6497,
+      "step": 119
+    },
+    {
+      "epoch": 0.019288728149487643,
+      "grad_norm": 0.5817735195159912,
+      "learning_rate": 0.00019998848720246328,
+      "loss": 1.3496,
+      "step": 120
+    },
+    {
+      "epoch": 0.019449467550733374,
+      "grad_norm": 0.5723729729652405,
+      "learning_rate": 0.00019998822994623174,
+      "loss": 1.3819,
+      "step": 121
+    },
+    {
+      "epoch": 0.019610206951979105,
+      "grad_norm": 0.6477592587471008,
+      "learning_rate": 0.0001999879698476142,
+      "loss": 1.6774,
+      "step": 122
+    },
+    {
+      "epoch": 0.019770946353224836,
+      "grad_norm": 0.6544284820556641,
+      "learning_rate": 0.00019998770690661803,
+      "loss": 1.8183,
+      "step": 123
+    },
+    {
+      "epoch": 0.019931685754470563,
+      "grad_norm": 0.5253116488456726,
+      "learning_rate": 0.00019998744112325074,
+      "loss": 1.3606,
+      "step": 124
+    },
+    {
+      "epoch": 0.020092425155716295,
+      "grad_norm": 0.5207403302192688,
+      "learning_rate": 0.00019998717249751984,
+      "loss": 1.3521,
+      "step": 125
+    },
+    {
+      "epoch": 0.020253164556962026,
+      "grad_norm": 0.6499771475791931,
+      "learning_rate": 0.000199986901029433,
+      "loss": 1.7398,
+      "step": 126
+    },
+    {
+      "epoch": 0.020413903958207757,
+      "grad_norm": 0.5993442535400391,
+      "learning_rate": 0.00019998662671899794,
+      "loss": 1.6894,
+      "step": 127
+    },
+    {
+      "epoch": 0.020574643359453484,
+      "grad_norm": 0.5501679182052612,
+      "learning_rate": 0.0001999863495662224,
+      "loss": 1.658,
+      "step": 128
+    },
+    {
+      "epoch": 0.020735382760699216,
+      "grad_norm": 0.5819227695465088,
+      "learning_rate": 0.0001999860695711144,
+      "loss": 1.7308,
+      "step": 129
+    },
+    {
+      "epoch": 0.020896122161944947,
+      "grad_norm": 0.5369065403938293,
+      "learning_rate": 0.00019998578673368172,
+      "loss": 1.5093,
+      "step": 130
+    },
+    {
+      "epoch": 0.021056861563190678,
+      "grad_norm": 0.5569128394126892,
+      "learning_rate": 0.00019998550105393252,
+      "loss": 1.5935,
+      "step": 131
+    },
+    {
+      "epoch": 0.02121760096443641,
+      "grad_norm": 0.5772825479507446,
+      "learning_rate": 0.00019998521253187487,
+      "loss": 1.6905,
+      "step": 132
+    },
+    {
+      "epoch": 0.021378340365682136,
+      "grad_norm": 0.5244413018226624,
+      "learning_rate": 0.00019998492116751702,
+      "loss": 1.5972,
+      "step": 133
+    },
+    {
+      "epoch": 0.021539079766927868,
+      "grad_norm": 0.5664756298065186,
+      "learning_rate": 0.00019998462696086718,
+      "loss": 1.5403,
+      "step": 134
+    },
+    {
+      "epoch": 0.0216998191681736,
+      "grad_norm": 0.5392235517501831,
+      "learning_rate": 0.00019998432991193377,
+      "loss": 1.6381,
+      "step": 135
+    },
+    {
+      "epoch": 0.02186055856941933,
+      "grad_norm": 0.586540162563324,
+      "learning_rate": 0.00019998403002072522,
+      "loss": 1.5034,
+      "step": 136
+    },
+    {
+      "epoch": 0.02202129797066506,
+      "grad_norm": 0.6401904225349426,
+      "learning_rate": 0.00019998372728725005,
+      "loss": 1.4966,
+      "step": 137
+    },
+    {
+      "epoch": 0.02218203737191079,
+      "grad_norm": 0.8401621580123901,
+      "learning_rate": 0.0001999834217115169,
+      "loss": 1.6566,
+      "step": 138
+    },
+    {
+      "epoch": 0.02234277677315652,
+      "grad_norm": 0.6305237412452698,
+      "learning_rate": 0.00019998311329353437,
+      "loss": 1.5463,
+      "step": 139
+    },
+    {
+      "epoch": 0.02250351617440225,
+      "grad_norm": 0.584318220615387,
+      "learning_rate": 0.00019998280203331132,
+      "loss": 1.9107,
+      "step": 140
+    },
+    {
+      "epoch": 0.022664255575647982,
+      "grad_norm": 0.6372249126434326,
+      "learning_rate": 0.00019998248793085655,
+      "loss": 1.7393,
+      "step": 141
+    },
+    {
+      "epoch": 0.02282499497689371,
+      "grad_norm": 0.6097559332847595,
+      "learning_rate": 0.00019998217098617902,
+      "loss": 1.6399,
+      "step": 142
+    },
+    {
+      "epoch": 0.02298573437813944,
+      "grad_norm": 0.5757220983505249,
+      "learning_rate": 0.00019998185119928769,
+      "loss": 1.5608,
+      "step": 143
+    },
+    {
+      "epoch": 0.02314647377938517,
+      "grad_norm": 0.6350814700126648,
+      "learning_rate": 0.0001999815285701917,
+      "loss": 1.819,
+      "step": 144
+    },
+    {
+      "epoch": 0.023307213180630903,
+      "grad_norm": 0.582371711730957,
+      "learning_rate": 0.0001999812030989002,
+      "loss": 1.6436,
+      "step": 145
+    },
+    {
+      "epoch": 0.023467952581876634,
+      "grad_norm": 0.5847258567810059,
+      "learning_rate": 0.00019998087478542245,
+      "loss": 1.5109,
+      "step": 146
+    },
+    {
+      "epoch": 0.02362869198312236,
+      "grad_norm": 0.5616840720176697,
+      "learning_rate": 0.00019998054362976777,
+      "loss": 1.5682,
+      "step": 147
+    },
+    {
+      "epoch": 0.023789431384368093,
+      "grad_norm": 0.45165371894836426,
+      "learning_rate": 0.00019998020963194558,
+      "loss": 1.306,
+      "step": 148
+    },
+    {
+      "epoch": 0.023950170785613824,
+      "grad_norm": 0.5621107220649719,
+      "learning_rate": 0.00019997987279196536,
+      "loss": 1.6454,
+      "step": 149
+    },
+    {
+      "epoch": 0.024110910186859555,
+      "grad_norm": 0.5720399022102356,
+      "learning_rate": 0.00019997953310983674,
+      "loss": 1.5393,
+      "step": 150
+    },
+    {
+      "epoch": 0.024271649588105286,
+      "grad_norm": 0.5469704866409302,
+      "learning_rate": 0.00019997919058556933,
+      "loss": 1.5437,
+      "step": 151
+    },
+    {
+      "epoch": 0.024432388989351014,
+      "grad_norm": 0.607407808303833,
+      "learning_rate": 0.00019997884521917288,
+      "loss": 1.4615,
+      "step": 152
+    },
+    {
+      "epoch": 0.024593128390596745,
+      "grad_norm": 0.5676476955413818,
+      "learning_rate": 0.00019997849701065717,
+      "loss": 1.5862,
+      "step": 153
+    },
+    {
+      "epoch": 0.024753867791842476,
+      "grad_norm": 0.5178918838500977,
+      "learning_rate": 0.00019997814596003214,
+      "loss": 1.4128,
+      "step": 154
+    },
+    {
+      "epoch": 0.024914607193088207,
+      "grad_norm": 0.555792510509491,
+      "learning_rate": 0.00019997779206730777,
+      "loss": 1.3723,
+      "step": 155
+    },
+    {
+      "epoch": 0.025075346594333935,
+      "grad_norm": 0.5667513608932495,
+      "learning_rate": 0.00019997743533249412,
+      "loss": 1.6411,
+      "step": 156
+    },
+    {
+      "epoch": 0.025236085995579666,
+      "grad_norm": 0.6014764904975891,
+      "learning_rate": 0.00019997707575560128,
+      "loss": 1.7638,
+      "step": 157
+    },
+    {
+      "epoch": 0.025396825396825397,
+      "grad_norm": 0.6081196665763855,
+      "learning_rate": 0.00019997671333663958,
+      "loss": 1.6497,
+      "step": 158
+    },
+    {
+      "epoch": 0.025557564798071128,
+      "grad_norm": 0.6969033479690552,
+      "learning_rate": 0.00019997634807561923,
+      "loss": 1.8579,
+      "step": 159
+    },
+    {
+      "epoch": 0.02571830419931686,
+      "grad_norm": 0.6027001738548279,
+      "learning_rate": 0.0001999759799725506,
+      "loss": 1.4378,
+      "step": 160
+    },
+    {
+      "epoch": 0.025879043600562587,
+      "grad_norm": 0.5889347791671753,
+      "learning_rate": 0.00019997560902744425,
+      "loss": 1.5519,
+      "step": 161
+    },
+    {
+      "epoch": 0.026039783001808318,
+      "grad_norm": 0.551889181137085,
+      "learning_rate": 0.00019997523524031062,
+      "loss": 1.4729,
+      "step": 162
+    },
+    {
+      "epoch": 0.02620052240305405,
+      "grad_norm": 0.6988946795463562,
+      "learning_rate": 0.00019997485861116042,
+      "loss": 1.8647,
+      "step": 163
+    },
+    {
+      "epoch": 0.02636126180429978,
+      "grad_norm": 0.5777518153190613,
+      "learning_rate": 0.00019997447914000432,
+      "loss": 1.5285,
+      "step": 164
+    },
+    {
+      "epoch": 0.02652200120554551,
+      "grad_norm": 0.6907120943069458,
+      "learning_rate": 0.0001999740968268531,
+      "loss": 1.7111,
+      "step": 165
+    },
+    {
+      "epoch": 0.02668274060679124,
+      "grad_norm": 0.6089081168174744,
+      "learning_rate": 0.00019997371167171767,
+      "loss": 1.621,
+      "step": 166
+    },
+    {
+      "epoch": 0.02684348000803697,
+      "grad_norm": 0.6227884292602539,
+      "learning_rate": 0.0001999733236746089,
+      "loss": 1.6838,
+      "step": 167
+    },
+    {
+      "epoch": 0.0270042194092827,
+      "grad_norm": 0.6011916995048523,
+      "learning_rate": 0.00019997293283553788,
+      "loss": 1.661,
+      "step": 168
+    },
+    {
+      "epoch": 0.027164958810528432,
+      "grad_norm": 0.5606417059898376,
+      "learning_rate": 0.00019997253915451569,
+      "loss": 1.4559,
+      "step": 169
+    },
+    {
+      "epoch": 0.02732569821177416,
+      "grad_norm": 0.6192370653152466,
+      "learning_rate": 0.00019997214263155354,
+      "loss": 1.5256,
+      "step": 170
+    },
+    {
+      "epoch": 0.02748643761301989,
+      "grad_norm": 0.6023882627487183,
+      "learning_rate": 0.0001999717432666627,
+      "loss": 1.6291,
+      "step": 171
+    },
+    {
+      "epoch": 0.027647177014265622,
+      "grad_norm": 0.5901930928230286,
+      "learning_rate": 0.00019997134105985453,
+      "loss": 1.5135,
+      "step": 172
+    },
+    {
+      "epoch": 0.027807916415511353,
+      "grad_norm": 0.5880249738693237,
+      "learning_rate": 0.00019997093601114045,
+      "loss": 1.4506,
+      "step": 173
+    },
+    {
+      "epoch": 0.027968655816757084,
+      "grad_norm": 0.5684868097305298,
+      "learning_rate": 0.00019997052812053198,
+      "loss": 1.6294,
+      "step": 174
+    },
+    {
+      "epoch": 0.02812939521800281,
+      "grad_norm": 0.5687061548233032,
+      "learning_rate": 0.00019997011738804072,
+      "loss": 1.5584,
+      "step": 175
+    },
+    {
+      "epoch": 0.028290134619248543,
+      "grad_norm": 0.48914963006973267,
+      "learning_rate": 0.00019996970381367835,
+      "loss": 1.3336,
+      "step": 176
+    },
+    {
+      "epoch": 0.028450874020494274,
+      "grad_norm": 0.6474471092224121,
+      "learning_rate": 0.00019996928739745662,
+      "loss": 1.6332,
+      "step": 177
+    },
+    {
+      "epoch": 0.028611613421740005,
+      "grad_norm": 0.5832691192626953,
+      "learning_rate": 0.00019996886813938733,
+      "loss": 1.4721,
+      "step": 178
+    },
+    {
+      "epoch": 0.028772352822985733,
+      "grad_norm": 0.527232825756073,
+      "learning_rate": 0.00019996844603948247,
+      "loss": 1.4806,
+      "step": 179
+    },
+    {
+      "epoch": 0.028933092224231464,
+      "grad_norm": 0.634027898311615,
+      "learning_rate": 0.00019996802109775396,
+      "loss": 1.7349,
+      "step": 180
+    },
+    {
+      "epoch": 0.029093831625477195,
+      "grad_norm": 0.5957772731781006,
+      "learning_rate": 0.00019996759331421395,
+      "loss": 1.649,
+      "step": 181
+    },
+    {
+      "epoch": 0.029254571026722926,
+      "grad_norm": 0.5642923712730408,
+      "learning_rate": 0.00019996716268887458,
+      "loss": 1.6033,
+      "step": 182
+    },
+    {
+      "epoch": 0.029415310427968657,
+      "grad_norm": 0.6112290024757385,
+      "learning_rate": 0.00019996672922174808,
+      "loss": 1.5424,
+      "step": 183
+    },
+    {
+      "epoch": 0.029576049829214385,
+      "grad_norm": 0.5684464573860168,
+      "learning_rate": 0.0001999662929128468,
+      "loss": 1.5925,
+      "step": 184
+    },
+    {
+      "epoch": 0.029736789230460116,
+      "grad_norm": 0.6065992116928101,
+      "learning_rate": 0.00019996585376218308,
+      "loss": 1.7697,
+      "step": 185
+    },
+    {
+      "epoch": 0.029897528631705847,
+      "grad_norm": 0.5482378005981445,
+      "learning_rate": 0.00019996541176976943,
+      "loss": 1.5841,
+      "step": 186
+    },
+    {
+      "epoch": 0.030058268032951578,
+      "grad_norm": 0.5863565802574158,
+      "learning_rate": 0.00019996496693561845,
+      "loss": 1.5255,
+      "step": 187
+    },
+    {
+      "epoch": 0.03021900743419731,
+      "grad_norm": 0.5618999600410461,
+      "learning_rate": 0.0001999645192597428,
+      "loss": 1.4487,
+      "step": 188
+    },
+    {
+      "epoch": 0.030379746835443037,
+      "grad_norm": 0.6198532581329346,
+      "learning_rate": 0.00019996406874215513,
+      "loss": 1.7207,
+      "step": 189
+    },
+    {
+      "epoch": 0.030540486236688768,
+      "grad_norm": 0.7240882515907288,
+      "learning_rate": 0.0001999636153828683,
+      "loss": 2.0277,
+      "step": 190
+    },
+    {
+      "epoch": 0.0307012256379345,
+      "grad_norm": 0.5573434233665466,
+      "learning_rate": 0.00019996315918189517,
+      "loss": 1.4414,
+      "step": 191
+    },
+    {
+      "epoch": 0.03086196503918023,
+      "grad_norm": 0.5780049562454224,
+      "learning_rate": 0.00019996270013924875,
+      "loss": 1.5145,
+      "step": 192
+    },
+    {
+      "epoch": 0.031022704440425958,
+      "grad_norm": 0.6340463757514954,
+      "learning_rate": 0.00019996223825494201,
+      "loss": 1.5458,
+      "step": 193
+    },
+    {
+      "epoch": 0.03118344384167169,
+      "grad_norm": 0.5492103099822998,
+      "learning_rate": 0.0001999617735289882,
+      "loss": 1.3887,
+      "step": 194
+    },
+    {
+      "epoch": 0.03134418324291742,
+      "grad_norm": 0.6537782549858093,
+      "learning_rate": 0.0001999613059614004,
+      "loss": 1.4066,
+      "step": 195
+    },
+    {
+      "epoch": 0.03150492264416315,
+      "grad_norm": 0.5743723511695862,
+      "learning_rate": 0.00019996083555219203,
+      "loss": 1.7008,
+      "step": 196
+    },
+    {
+      "epoch": 0.03166566204540888,
+      "grad_norm": 0.5864811539649963,
+      "learning_rate": 0.00019996036230137635,
+      "loss": 1.4182,
+      "step": 197
+    },
+    {
+      "epoch": 0.03182640144665461,
+      "grad_norm": 0.5790386199951172,
+      "learning_rate": 0.00019995988620896687,
+      "loss": 1.3932,
+      "step": 198
+    },
+    {
+      "epoch": 0.031987140847900344,
+      "grad_norm": 0.5721076130867004,
+      "learning_rate": 0.00019995940727497713,
+      "loss": 1.6413,
+      "step": 199
+    },
+    {
+      "epoch": 0.03214788024914607,
+      "grad_norm": 0.6769488453865051,
+      "learning_rate": 0.0001999589254994207,
+      "loss": 1.6489,
+      "step": 200
+    },
+    {
+      "epoch": 0.03214788024914607,
+      "eval_loss": 1.6234830617904663,
+      "eval_runtime": 46.2592,
+      "eval_samples_per_second": 5.426,
+      "eval_steps_per_second": 2.724,
+      "step": 200
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 18663,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "EarlyStoppingCallback": {
+      "args": {
+        "early_stopping_patience": 3,
+        "early_stopping_threshold": 0.0
+      },
+      "attributes": {
+        "early_stopping_patience_counter": 0
+      }
+    },
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.493067239194624e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a9c6197306c352b5391f840a153ba2f8800f388ce43ecf7ab06f484098606a5d
+size 6776