Training in progress, step 200, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: unsloth/Qwen2-1.5B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "unsloth/Qwen2-1.5B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "q_proj",
+    "gate_proj",
+    "k_proj",
+    "up_proj",
+    "o_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:639a9dc884067a2de59968620e0fe69deee8f3615abd9758c9bb3b71744d79d4
+size 147770496

last-checkpoint/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6811e77e22966683ef645acda6777e8c9027b3aa1f640bdad54fb25317ac974
+size 75471860

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69fc3d7774646d7ddf16686711b0b2091649fcf18ac5dd847d948683f549a860
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a2350b710e44d7cc917768dc19341444fa0c9c55c94ce3bd1d08bce5b6afca7
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,20 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bcfe42da0a4497e8b2b172c1f9f4ec423a46dc12907f4349c55025f670422ba9
+size 11418266

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,44 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "left",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,1458 @@
+{
+  "best_metric": 0.39890384674072266,
+  "best_model_checkpoint": "miner_id_24/checkpoint-200",
+  "epoch": 0.1128429367374286,
+  "eval_steps": 200,
+  "global_step": 200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.000564214683687143,
+      "grad_norm": 1.1520037651062012,
+      "learning_rate": 6.666666666666667e-06,
+      "loss": 1.4003,
+      "step": 1
+    },
+    {
+      "epoch": 0.000564214683687143,
+      "eval_loss": 1.408501386642456,
+      "eval_runtime": 15.3841,
+      "eval_samples_per_second": 9.295,
+      "eval_steps_per_second": 4.68,
+      "step": 1
+    },
+    {
+      "epoch": 0.001128429367374286,
+      "grad_norm": 1.0998706817626953,
+      "learning_rate": 1.3333333333333333e-05,
+      "loss": 1.4,
+      "step": 2
+    },
+    {
+      "epoch": 0.0016926440510614288,
+      "grad_norm": 1.0686613321304321,
+      "learning_rate": 2e-05,
+      "loss": 1.4384,
+      "step": 3
+    },
+    {
+      "epoch": 0.002256858734748572,
+      "grad_norm": 1.0820649862289429,
+      "learning_rate": 2.6666666666666667e-05,
+      "loss": 1.4037,
+      "step": 4
+    },
+    {
+      "epoch": 0.002821073418435715,
+      "grad_norm": 0.9210293889045715,
+      "learning_rate": 3.3333333333333335e-05,
+      "loss": 1.3524,
+      "step": 5
+    },
+    {
+      "epoch": 0.0033852881021228577,
+      "grad_norm": 0.8373953700065613,
+      "learning_rate": 4e-05,
+      "loss": 1.3229,
+      "step": 6
+    },
+    {
+      "epoch": 0.003949502785810001,
+      "grad_norm": 0.7442015409469604,
+      "learning_rate": 4.666666666666667e-05,
+      "loss": 1.3318,
+      "step": 7
+    },
+    {
+      "epoch": 0.004513717469497144,
+      "grad_norm": 0.7030277252197266,
+      "learning_rate": 5.333333333333333e-05,
+      "loss": 1.2398,
+      "step": 8
+    },
+    {
+      "epoch": 0.005077932153184287,
+      "grad_norm": 0.6552405953407288,
+      "learning_rate": 6e-05,
+      "loss": 1.1855,
+      "step": 9
+    },
+    {
+      "epoch": 0.00564214683687143,
+      "grad_norm": 0.6781073212623596,
+      "learning_rate": 6.666666666666667e-05,
+      "loss": 1.1832,
+      "step": 10
+    },
+    {
+      "epoch": 0.0062063615205585725,
+      "grad_norm": 0.6454867720603943,
+      "learning_rate": 7.333333333333333e-05,
+      "loss": 1.1001,
+      "step": 11
+    },
+    {
+      "epoch": 0.006770576204245715,
+      "grad_norm": 0.7071139812469482,
+      "learning_rate": 8e-05,
+      "loss": 1.0291,
+      "step": 12
+    },
+    {
+      "epoch": 0.007334790887932858,
+      "grad_norm": 0.7745906114578247,
+      "learning_rate": 8.666666666666667e-05,
+      "loss": 1.0417,
+      "step": 13
+    },
+    {
+      "epoch": 0.007899005571620002,
+      "grad_norm": 0.7966415882110596,
+      "learning_rate": 9.333333333333334e-05,
+      "loss": 0.8854,
+      "step": 14
+    },
+    {
+      "epoch": 0.008463220255307144,
+      "grad_norm": 0.7034145593643188,
+      "learning_rate": 0.0001,
+      "loss": 0.8154,
+      "step": 15
+    },
+    {
+      "epoch": 0.009027434938994288,
+      "grad_norm": 0.6648619174957275,
+      "learning_rate": 0.00010666666666666667,
+      "loss": 0.795,
+      "step": 16
+    },
+    {
+      "epoch": 0.00959164962268143,
+      "grad_norm": 0.6274858713150024,
+      "learning_rate": 0.00011333333333333334,
+      "loss": 0.7041,
+      "step": 17
+    },
+    {
+      "epoch": 0.010155864306368573,
+      "grad_norm": 0.6386944055557251,
+      "learning_rate": 0.00012,
+      "loss": 0.7059,
+      "step": 18
+    },
+    {
+      "epoch": 0.010720078990055715,
+      "grad_norm": 0.6311229467391968,
+      "learning_rate": 0.00012666666666666666,
+      "loss": 0.6616,
+      "step": 19
+    },
+    {
+      "epoch": 0.01128429367374286,
+      "grad_norm": 0.5835394859313965,
+      "learning_rate": 0.00013333333333333334,
+      "loss": 0.6301,
+      "step": 20
+    },
+    {
+      "epoch": 0.011848508357430003,
+      "grad_norm": 0.4852144122123718,
+      "learning_rate": 0.00014,
+      "loss": 0.5493,
+      "step": 21
+    },
+    {
+      "epoch": 0.012412723041117145,
+      "grad_norm": 0.5399675965309143,
+      "learning_rate": 0.00014666666666666666,
+      "loss": 0.5637,
+      "step": 22
+    },
+    {
+      "epoch": 0.012976937724804289,
+      "grad_norm": 0.5119890570640564,
+      "learning_rate": 0.00015333333333333334,
+      "loss": 0.5722,
+      "step": 23
+    },
+    {
+      "epoch": 0.01354115240849143,
+      "grad_norm": 0.4267195165157318,
+      "learning_rate": 0.00016,
+      "loss": 0.5791,
+      "step": 24
+    },
+    {
+      "epoch": 0.014105367092178574,
+      "grad_norm": 0.42489948868751526,
+      "learning_rate": 0.0001666666666666667,
+      "loss": 0.4884,
+      "step": 25
+    },
+    {
+      "epoch": 0.014669581775865716,
+      "grad_norm": 0.35310837626457214,
+      "learning_rate": 0.00017333333333333334,
+      "loss": 0.5424,
+      "step": 26
+    },
+    {
+      "epoch": 0.01523379645955286,
+      "grad_norm": 0.3092120289802551,
+      "learning_rate": 0.00018,
+      "loss": 0.5203,
+      "step": 27
+    },
+    {
+      "epoch": 0.015798011143240004,
+      "grad_norm": 0.3228914141654968,
+      "learning_rate": 0.0001866666666666667,
+      "loss": 0.5178,
+      "step": 28
+    },
+    {
+      "epoch": 0.016362225826927146,
+      "grad_norm": 0.3223153054714203,
+      "learning_rate": 0.00019333333333333333,
+      "loss": 0.4951,
+      "step": 29
+    },
+    {
+      "epoch": 0.016926440510614288,
+      "grad_norm": 0.29833337664604187,
+      "learning_rate": 0.0002,
+      "loss": 0.4455,
+      "step": 30
+    },
+    {
+      "epoch": 0.017490655194301433,
+      "grad_norm": 0.2626526653766632,
+      "learning_rate": 0.00019999998233899478,
+      "loss": 0.5323,
+      "step": 31
+    },
+    {
+      "epoch": 0.018054869877988575,
+      "grad_norm": 0.2821914851665497,
+      "learning_rate": 0.00019999992935598528,
+      "loss": 0.5032,
+      "step": 32
+    },
+    {
+      "epoch": 0.018619084561675717,
+      "grad_norm": 0.26937922835350037,
+      "learning_rate": 0.00019999984105099027,
+      "loss": 0.4646,
+      "step": 33
+    },
+    {
+      "epoch": 0.01918329924536286,
+      "grad_norm": 0.23818109929561615,
+      "learning_rate": 0.00019999971742404096,
+      "loss": 0.369,
+      "step": 34
+    },
+    {
+      "epoch": 0.019747513929050005,
+      "grad_norm": 0.2889389991760254,
+      "learning_rate": 0.00019999955847518094,
+      "loss": 0.4943,
+      "step": 35
+    },
+    {
+      "epoch": 0.020311728612737147,
+      "grad_norm": 0.2832432985305786,
+      "learning_rate": 0.00019999936420446644,
+      "loss": 0.492,
+      "step": 36
+    },
+    {
+      "epoch": 0.02087594329642429,
+      "grad_norm": 0.28265583515167236,
+      "learning_rate": 0.00019999913461196602,
+      "loss": 0.4996,
+      "step": 37
+    },
+    {
+      "epoch": 0.02144015798011143,
+      "grad_norm": 0.2630419433116913,
+      "learning_rate": 0.00019999886969776078,
+      "loss": 0.424,
+      "step": 38
+    },
+    {
+      "epoch": 0.022004372663798576,
+      "grad_norm": 0.2762552797794342,
+      "learning_rate": 0.00019999856946194432,
+      "loss": 0.4803,
+      "step": 39
+    },
+    {
+      "epoch": 0.02256858734748572,
+      "grad_norm": 0.2654506266117096,
+      "learning_rate": 0.0001999982339046227,
+      "loss": 0.4646,
+      "step": 40
+    },
+    {
+      "epoch": 0.02313280203117286,
+      "grad_norm": 0.26834630966186523,
+      "learning_rate": 0.0001999978630259144,
+      "loss": 0.4721,
+      "step": 41
+    },
+    {
+      "epoch": 0.023697016714860006,
+      "grad_norm": 0.2593581974506378,
+      "learning_rate": 0.00019999745682595043,
+      "loss": 0.4408,
+      "step": 42
+    },
+    {
+      "epoch": 0.024261231398547148,
+      "grad_norm": 0.26713791489601135,
+      "learning_rate": 0.0001999970153048743,
+      "loss": 0.469,
+      "step": 43
+    },
+    {
+      "epoch": 0.02482544608223429,
+      "grad_norm": 0.2825049161911011,
+      "learning_rate": 0.00019999653846284194,
+      "loss": 0.5197,
+      "step": 44
+    },
+    {
+      "epoch": 0.025389660765921432,
+      "grad_norm": 0.27239227294921875,
+      "learning_rate": 0.0001999960263000218,
+      "loss": 0.4925,
+      "step": 45
+    },
+    {
+      "epoch": 0.025953875449608577,
+      "grad_norm": 0.23411159217357635,
+      "learning_rate": 0.00019999547881659478,
+      "loss": 0.3998,
+      "step": 46
+    },
+    {
+      "epoch": 0.02651809013329572,
+      "grad_norm": 0.2628365457057953,
+      "learning_rate": 0.00019999489601275425,
+      "loss": 0.4285,
+      "step": 47
+    },
+    {
+      "epoch": 0.02708230481698286,
+      "grad_norm": 0.25419920682907104,
+      "learning_rate": 0.00019999427788870605,
+      "loss": 0.4431,
+      "step": 48
+    },
+    {
+      "epoch": 0.027646519500670003,
+      "grad_norm": 0.2781168818473816,
+      "learning_rate": 0.00019999362444466856,
+      "loss": 0.4458,
+      "step": 49
+    },
+    {
+      "epoch": 0.02821073418435715,
+      "grad_norm": 0.2818092107772827,
+      "learning_rate": 0.0001999929356808726,
+      "loss": 0.4627,
+      "step": 50
+    },
+    {
+      "epoch": 0.02877494886804429,
+      "grad_norm": 0.26697826385498047,
+      "learning_rate": 0.00019999221159756135,
+      "loss": 0.4223,
+      "step": 51
+    },
+    {
+      "epoch": 0.029339163551731433,
+      "grad_norm": 0.2988968789577484,
+      "learning_rate": 0.00019999145219499068,
+      "loss": 0.5094,
+      "step": 52
+    },
+    {
+      "epoch": 0.02990337823541858,
+      "grad_norm": 0.2580755054950714,
+      "learning_rate": 0.0001999906574734288,
+      "loss": 0.3957,
+      "step": 53
+    },
+    {
+      "epoch": 0.03046759291910572,
+      "grad_norm": 0.26758626103401184,
+      "learning_rate": 0.00019998982743315641,
+      "loss": 0.4355,
+      "step": 54
+    },
+    {
+      "epoch": 0.031031807602792862,
+      "grad_norm": 0.23946945369243622,
+      "learning_rate": 0.00019998896207446668,
+      "loss": 0.4124,
+      "step": 55
+    },
+    {
+      "epoch": 0.03159602228648001,
+      "grad_norm": 0.2528536915779114,
+      "learning_rate": 0.0001999880613976653,
+      "loss": 0.3928,
+      "step": 56
+    },
+    {
+      "epoch": 0.03216023697016715,
+      "grad_norm": 0.2776258587837219,
+      "learning_rate": 0.0001999871254030704,
+      "loss": 0.3699,
+      "step": 57
+    },
+    {
+      "epoch": 0.03272445165385429,
+      "grad_norm": 0.29236239194869995,
+      "learning_rate": 0.00019998615409101258,
+      "loss": 0.4456,
+      "step": 58
+    },
+    {
+      "epoch": 0.033288666337541434,
+      "grad_norm": 0.27252358198165894,
+      "learning_rate": 0.00019998514746183497,
+      "loss": 0.4349,
+      "step": 59
+    },
+    {
+      "epoch": 0.033852881021228576,
+      "grad_norm": 0.28167688846588135,
+      "learning_rate": 0.00019998410551589306,
+      "loss": 0.3709,
+      "step": 60
+    },
+    {
+      "epoch": 0.03441709570491572,
+      "grad_norm": 0.254842072725296,
+      "learning_rate": 0.00019998302825355492,
+      "loss": 0.4889,
+      "step": 61
+    },
+    {
+      "epoch": 0.03498131038860287,
+      "grad_norm": 0.27143117785453796,
+      "learning_rate": 0.0001999819156752011,
+      "loss": 0.4439,
+      "step": 62
+    },
+    {
+      "epoch": 0.03554552507229001,
+      "grad_norm": 0.298893004655838,
+      "learning_rate": 0.00019998076778122452,
+      "loss": 0.4976,
+      "step": 63
+    },
+    {
+      "epoch": 0.03610973975597715,
+      "grad_norm": 0.2314186990261078,
+      "learning_rate": 0.0001999795845720307,
+      "loss": 0.3786,
+      "step": 64
+    },
+    {
+      "epoch": 0.03667395443966429,
+      "grad_norm": 0.25155794620513916,
+      "learning_rate": 0.0001999783660480375,
+      "loss": 0.4546,
+      "step": 65
+    },
+    {
+      "epoch": 0.037238169123351435,
+      "grad_norm": 0.2556498944759369,
+      "learning_rate": 0.00019997711220967542,
+      "loss": 0.4008,
+      "step": 66
+    },
+    {
+      "epoch": 0.03780238380703858,
+      "grad_norm": 0.2440929114818573,
+      "learning_rate": 0.00019997582305738724,
+      "loss": 0.4534,
+      "step": 67
+    },
+    {
+      "epoch": 0.03836659849072572,
+      "grad_norm": 0.22852113842964172,
+      "learning_rate": 0.0001999744985916284,
+      "loss": 0.4104,
+      "step": 68
+    },
+    {
+      "epoch": 0.03893081317441286,
+      "grad_norm": 0.26607510447502136,
+      "learning_rate": 0.00019997313881286668,
+      "loss": 0.4906,
+      "step": 69
+    },
+    {
+      "epoch": 0.03949502785810001,
+      "grad_norm": 0.25621435046195984,
+      "learning_rate": 0.00019997174372158238,
+      "loss": 0.4276,
+      "step": 70
+    },
+    {
+      "epoch": 0.04005924254178715,
+      "grad_norm": 0.24058878421783447,
+      "learning_rate": 0.0001999703133182683,
+      "loss": 0.4567,
+      "step": 71
+    },
+    {
+      "epoch": 0.040623457225474294,
+      "grad_norm": 0.27006158232688904,
+      "learning_rate": 0.00019996884760342968,
+      "loss": 0.4667,
+      "step": 72
+    },
+    {
+      "epoch": 0.041187671909161436,
+      "grad_norm": 0.24476608633995056,
+      "learning_rate": 0.00019996734657758424,
+      "loss": 0.3976,
+      "step": 73
+    },
+    {
+      "epoch": 0.04175188659284858,
+      "grad_norm": 0.241770938038826,
+      "learning_rate": 0.00019996581024126219,
+      "loss": 0.4358,
+      "step": 74
+    },
+    {
+      "epoch": 0.04231610127653572,
+      "grad_norm": 0.2684559226036072,
+      "learning_rate": 0.00019996423859500612,
+      "loss": 0.4813,
+      "step": 75
+    },
+    {
+      "epoch": 0.04288031596022286,
+      "grad_norm": 0.21936559677124023,
+      "learning_rate": 0.00019996263163937127,
+      "loss": 0.3678,
+      "step": 76
+    },
+    {
+      "epoch": 0.04344453064391001,
+      "grad_norm": 0.2588251829147339,
+      "learning_rate": 0.00019996098937492514,
+      "loss": 0.447,
+      "step": 77
+    },
+    {
+      "epoch": 0.04400874532759715,
+      "grad_norm": 0.23832190036773682,
+      "learning_rate": 0.00019995931180224794,
+      "loss": 0.4577,
+      "step": 78
+    },
+    {
+      "epoch": 0.044572960011284295,
+      "grad_norm": 0.2559312880039215,
+      "learning_rate": 0.0001999575989219321,
+      "loss": 0.4276,
+      "step": 79
+    },
+    {
+      "epoch": 0.04513717469497144,
+      "grad_norm": 0.24614478647708893,
+      "learning_rate": 0.00019995585073458272,
+      "loss": 0.4665,
+      "step": 80
+    },
+    {
+      "epoch": 0.04570138937865858,
+      "grad_norm": 0.23288443684577942,
+      "learning_rate": 0.0001999540672408173,
+      "loss": 0.4616,
+      "step": 81
+    },
+    {
+      "epoch": 0.04626560406234572,
+      "grad_norm": 0.23826882243156433,
+      "learning_rate": 0.00019995224844126573,
+      "loss": 0.4016,
+      "step": 82
+    },
+    {
+      "epoch": 0.04682981874603286,
+      "grad_norm": 0.222177654504776,
+      "learning_rate": 0.0001999503943365705,
+      "loss": 0.3967,
+      "step": 83
+    },
+    {
+      "epoch": 0.04739403342972001,
+      "grad_norm": 0.2404349148273468,
+      "learning_rate": 0.0001999485049273865,
+      "loss": 0.405,
+      "step": 84
+    },
+    {
+      "epoch": 0.047958248113407154,
+      "grad_norm": 0.24770587682724,
+      "learning_rate": 0.00019994658021438115,
+      "loss": 0.3752,
+      "step": 85
+    },
+    {
+      "epoch": 0.048522462797094296,
+      "grad_norm": 0.25180116295814514,
+      "learning_rate": 0.00019994462019823426,
+      "loss": 0.4651,
+      "step": 86
+    },
+    {
+      "epoch": 0.04908667748078144,
+      "grad_norm": 0.21408092975616455,
+      "learning_rate": 0.00019994262487963815,
+      "loss": 0.3532,
+      "step": 87
+    },
+    {
+      "epoch": 0.04965089216446858,
+      "grad_norm": 0.2462555319070816,
+      "learning_rate": 0.0001999405942592976,
+      "loss": 0.4462,
+      "step": 88
+    },
+    {
+      "epoch": 0.05021510684815572,
+      "grad_norm": 0.23737366497516632,
+      "learning_rate": 0.00019993852833792987,
+      "loss": 0.4305,
+      "step": 89
+    },
+    {
+      "epoch": 0.050779321531842864,
+      "grad_norm": 0.2529670298099518,
+      "learning_rate": 0.0001999364271162647,
+      "loss": 0.4584,
+      "step": 90
+    },
+    {
+      "epoch": 0.05134353621553001,
+      "grad_norm": 0.23798632621765137,
+      "learning_rate": 0.00019993429059504428,
+      "loss": 0.4209,
+      "step": 91
+    },
+    {
+      "epoch": 0.051907750899217155,
+      "grad_norm": 0.22876866161823273,
+      "learning_rate": 0.00019993211877502326,
+      "loss": 0.3389,
+      "step": 92
+    },
+    {
+      "epoch": 0.0524719655829043,
+      "grad_norm": 0.2262330800294876,
+      "learning_rate": 0.00019992991165696875,
+      "loss": 0.3991,
+      "step": 93
+    },
+    {
+      "epoch": 0.05303618026659144,
+      "grad_norm": 0.2369421273469925,
+      "learning_rate": 0.00019992766924166043,
+      "loss": 0.416,
+      "step": 94
+    },
+    {
+      "epoch": 0.05360039495027858,
+      "grad_norm": 0.21275782585144043,
+      "learning_rate": 0.00019992539152989024,
+      "loss": 0.3845,
+      "step": 95
+    },
+    {
+      "epoch": 0.05416460963396572,
+      "grad_norm": 0.2287372499704361,
+      "learning_rate": 0.00019992307852246283,
+      "loss": 0.3677,
+      "step": 96
+    },
+    {
+      "epoch": 0.054728824317652865,
+      "grad_norm": 0.2228369414806366,
+      "learning_rate": 0.00019992073022019518,
+      "loss": 0.3967,
+      "step": 97
+    },
+    {
+      "epoch": 0.05529303900134001,
+      "grad_norm": 0.21206596493721008,
+      "learning_rate": 0.00019991834662391668,
+      "loss": 0.4168,
+      "step": 98
+    },
+    {
+      "epoch": 0.055857253685027156,
+      "grad_norm": 0.22784358263015747,
+      "learning_rate": 0.00019991592773446935,
+      "loss": 0.44,
+      "step": 99
+    },
+    {
+      "epoch": 0.0564214683687143,
+      "grad_norm": 0.21377602219581604,
+      "learning_rate": 0.00019991347355270754,
+      "loss": 0.3569,
+      "step": 100
+    },
+    {
+      "epoch": 0.05698568305240144,
+      "grad_norm": 0.2376997470855713,
+      "learning_rate": 0.00019991098407949815,
+      "loss": 0.4444,
+      "step": 101
+    },
+    {
+      "epoch": 0.05754989773608858,
+      "grad_norm": 0.22973109781742096,
+      "learning_rate": 0.0001999084593157205,
+      "loss": 0.449,
+      "step": 102
+    },
+    {
+      "epoch": 0.058114112419775724,
+      "grad_norm": 0.22671952843666077,
+      "learning_rate": 0.00019990589926226635,
+      "loss": 0.3506,
+      "step": 103
+    },
+    {
+      "epoch": 0.058678327103462866,
+      "grad_norm": 0.22656241059303284,
+      "learning_rate": 0.00019990330392004001,
+      "loss": 0.3417,
+      "step": 104
+    },
+    {
+      "epoch": 0.05924254178715001,
+      "grad_norm": 0.23045070469379425,
+      "learning_rate": 0.00019990067328995822,
+      "loss": 0.4611,
+      "step": 105
+    },
+    {
+      "epoch": 0.05980675647083716,
+      "grad_norm": 0.23033630847930908,
+      "learning_rate": 0.0001998980073729501,
+      "loss": 0.4768,
+      "step": 106
+    },
+    {
+      "epoch": 0.0603709711545243,
+      "grad_norm": 0.24220503866672516,
+      "learning_rate": 0.00019989530616995737,
+      "loss": 0.4647,
+      "step": 107
+    },
+    {
+      "epoch": 0.06093518583821144,
+      "grad_norm": 0.22092081606388092,
+      "learning_rate": 0.00019989256968193413,
+      "loss": 0.4052,
+      "step": 108
+    },
+    {
+      "epoch": 0.06149940052189858,
+      "grad_norm": 0.24017955362796783,
+      "learning_rate": 0.00019988979790984697,
+      "loss": 0.4395,
+      "step": 109
+    },
+    {
+      "epoch": 0.062063615205585725,
+      "grad_norm": 0.24283204972743988,
+      "learning_rate": 0.0001998869908546749,
+      "loss": 0.4037,
+      "step": 110
+    },
+    {
+      "epoch": 0.06262782988927287,
+      "grad_norm": 0.2208131104707718,
+      "learning_rate": 0.0001998841485174095,
+      "loss": 0.4412,
+      "step": 111
+    },
+    {
+      "epoch": 0.06319204457296002,
+      "grad_norm": 0.24447640776634216,
+      "learning_rate": 0.00019988127089905466,
+      "loss": 0.4223,
+      "step": 112
+    },
+    {
+      "epoch": 0.06375625925664716,
+      "grad_norm": 0.2286946326494217,
+      "learning_rate": 0.00019987835800062685,
+      "loss": 0.3759,
+      "step": 113
+    },
+    {
+      "epoch": 0.0643204739403343,
+      "grad_norm": 0.2328777313232422,
+      "learning_rate": 0.00019987540982315498,
+      "loss": 0.4488,
+      "step": 114
+    },
+    {
+      "epoch": 0.06488468862402144,
+      "grad_norm": 0.23397162556648254,
+      "learning_rate": 0.00019987242636768037,
+      "loss": 0.38,
+      "step": 115
+    },
+    {
+      "epoch": 0.06544890330770858,
+      "grad_norm": 0.2335141897201538,
+      "learning_rate": 0.00019986940763525688,
+      "loss": 0.4089,
+      "step": 116
+    },
+    {
+      "epoch": 0.06601311799139573,
+      "grad_norm": 0.2624657452106476,
+      "learning_rate": 0.0001998663536269507,
+      "loss": 0.444,
+      "step": 117
+    },
+    {
+      "epoch": 0.06657733267508287,
+      "grad_norm": 0.24512571096420288,
+      "learning_rate": 0.0001998632643438407,
+      "loss": 0.4207,
+      "step": 118
+    },
+    {
+      "epoch": 0.06714154735877001,
+      "grad_norm": 0.23586347699165344,
+      "learning_rate": 0.00019986013978701798,
+      "loss": 0.423,
+      "step": 119
+    },
+    {
+      "epoch": 0.06770576204245715,
+      "grad_norm": 0.22816482186317444,
+      "learning_rate": 0.00019985697995758623,
+      "loss": 0.4182,
+      "step": 120
+    },
+    {
+      "epoch": 0.0682699767261443,
+      "grad_norm": 0.22102321684360504,
+      "learning_rate": 0.00019985378485666154,
+      "loss": 0.3796,
+      "step": 121
+    },
+    {
+      "epoch": 0.06883419140983144,
+      "grad_norm": 0.2527519166469574,
+      "learning_rate": 0.0001998505544853725,
+      "loss": 0.4037,
+      "step": 122
+    },
+    {
+      "epoch": 0.06939840609351858,
+      "grad_norm": 0.22393916547298431,
+      "learning_rate": 0.00019984728884486018,
+      "loss": 0.3694,
+      "step": 123
+    },
+    {
+      "epoch": 0.06996262077720573,
+      "grad_norm": 0.26279133558273315,
+      "learning_rate": 0.000199843987936278,
+      "loss": 0.4212,
+      "step": 124
+    },
+    {
+      "epoch": 0.07052683546089288,
+      "grad_norm": 0.23101545870304108,
+      "learning_rate": 0.00019984065176079194,
+      "loss": 0.4161,
+      "step": 125
+    },
+    {
+      "epoch": 0.07109105014458002,
+      "grad_norm": 0.24067720770835876,
+      "learning_rate": 0.00019983728031958042,
+      "loss": 0.4456,
+      "step": 126
+    },
+    {
+      "epoch": 0.07165526482826716,
+      "grad_norm": 0.23498354852199554,
+      "learning_rate": 0.00019983387361383428,
+      "loss": 0.3891,
+      "step": 127
+    },
+    {
+      "epoch": 0.0722194795119543,
+      "grad_norm": 0.22644099593162537,
+      "learning_rate": 0.00019983043164475682,
+      "loss": 0.4328,
+      "step": 128
+    },
+    {
+      "epoch": 0.07278369419564144,
+      "grad_norm": 0.23138949275016785,
+      "learning_rate": 0.0001998269544135639,
+      "loss": 0.4277,
+      "step": 129
+    },
+    {
+      "epoch": 0.07334790887932859,
+      "grad_norm": 0.23250654339790344,
+      "learning_rate": 0.00019982344192148363,
+      "loss": 0.4202,
+      "step": 130
+    },
+    {
+      "epoch": 0.07391212356301573,
+      "grad_norm": 0.21570515632629395,
+      "learning_rate": 0.00019981989416975674,
+      "loss": 0.3965,
+      "step": 131
+    },
+    {
+      "epoch": 0.07447633824670287,
+      "grad_norm": 0.2185685932636261,
+      "learning_rate": 0.0001998163111596364,
+      "loss": 0.4445,
+      "step": 132
+    },
+    {
+      "epoch": 0.07504055293039001,
+      "grad_norm": 0.24563764035701752,
+      "learning_rate": 0.00019981269289238816,
+      "loss": 0.4604,
+      "step": 133
+    },
+    {
+      "epoch": 0.07560476761407715,
+      "grad_norm": 0.21496614813804626,
+      "learning_rate": 0.00019980903936929007,
+      "loss": 0.3697,
+      "step": 134
+    },
+    {
+      "epoch": 0.0761689822977643,
+      "grad_norm": 0.2316853106021881,
+      "learning_rate": 0.00019980535059163266,
+      "loss": 0.403,
+      "step": 135
+    },
+    {
+      "epoch": 0.07673319698145144,
+      "grad_norm": 0.24481691420078278,
+      "learning_rate": 0.00019980162656071886,
+      "loss": 0.3937,
+      "step": 136
+    },
+    {
+      "epoch": 0.07729741166513858,
+      "grad_norm": 0.21267257630825043,
+      "learning_rate": 0.00019979786727786404,
+      "loss": 0.3288,
+      "step": 137
+    },
+    {
+      "epoch": 0.07786162634882572,
+      "grad_norm": 0.23100951313972473,
+      "learning_rate": 0.0001997940727443961,
+      "loss": 0.4385,
+      "step": 138
+    },
+    {
+      "epoch": 0.07842584103251288,
+      "grad_norm": 0.21640829741954803,
+      "learning_rate": 0.00019979024296165533,
+      "loss": 0.3881,
+      "step": 139
+    },
+    {
+      "epoch": 0.07899005571620002,
+      "grad_norm": 0.23516759276390076,
+      "learning_rate": 0.00019978637793099447,
+      "loss": 0.4579,
+      "step": 140
+    },
+    {
+      "epoch": 0.07955427039988716,
+      "grad_norm": 0.2463797777891159,
+      "learning_rate": 0.00019978247765377877,
+      "loss": 0.4243,
+      "step": 141
+    },
+    {
+      "epoch": 0.0801184850835743,
+      "grad_norm": 0.2400227189064026,
+      "learning_rate": 0.00019977854213138586,
+      "loss": 0.4519,
+      "step": 142
+    },
+    {
+      "epoch": 0.08068269976726145,
+      "grad_norm": 0.22081589698791504,
+      "learning_rate": 0.00019977457136520583,
+      "loss": 0.4142,
+      "step": 143
+    },
+    {
+      "epoch": 0.08124691445094859,
+      "grad_norm": 0.2160605937242508,
+      "learning_rate": 0.00019977056535664124,
+      "loss": 0.3695,
+      "step": 144
+    },
+    {
+      "epoch": 0.08181112913463573,
+      "grad_norm": 0.23759673535823822,
+      "learning_rate": 0.00019976652410710712,
+      "loss": 0.3825,
+      "step": 145
+    },
+    {
+      "epoch": 0.08237534381832287,
+      "grad_norm": 0.24334338307380676,
+      "learning_rate": 0.00019976244761803091,
+      "loss": 0.4083,
+      "step": 146
+    },
+    {
+      "epoch": 0.08293955850201001,
+      "grad_norm": 0.20011739432811737,
+      "learning_rate": 0.00019975833589085248,
+      "loss": 0.3613,
+      "step": 147
+    },
+    {
+      "epoch": 0.08350377318569716,
+      "grad_norm": 0.23955340683460236,
+      "learning_rate": 0.0001997541889270242,
+      "loss": 0.3821,
+      "step": 148
+    },
+    {
+      "epoch": 0.0840679878693843,
+      "grad_norm": 0.25828778743743896,
+      "learning_rate": 0.00019975000672801087,
+      "loss": 0.4336,
+      "step": 149
+    },
+    {
+      "epoch": 0.08463220255307144,
+      "grad_norm": 0.2364976704120636,
+      "learning_rate": 0.0001997457892952897,
+      "loss": 0.4593,
+      "step": 150
+    },
+    {
+      "epoch": 0.08519641723675858,
+      "grad_norm": 0.22080567479133606,
+      "learning_rate": 0.00019974153663035037,
+      "loss": 0.4089,
+      "step": 151
+    },
+    {
+      "epoch": 0.08576063192044572,
+      "grad_norm": 0.21779291331768036,
+      "learning_rate": 0.00019973724873469505,
+      "loss": 0.4328,
+      "step": 152
+    },
+    {
+      "epoch": 0.08632484660413288,
+      "grad_norm": 0.21748045086860657,
+      "learning_rate": 0.00019973292560983825,
+      "loss": 0.3948,
+      "step": 153
+    },
+    {
+      "epoch": 0.08688906128782002,
+      "grad_norm": 0.2201155573129654,
+      "learning_rate": 0.00019972856725730705,
+      "loss": 0.4198,
+      "step": 154
+    },
+    {
+      "epoch": 0.08745327597150716,
+      "grad_norm": 0.24108703434467316,
+      "learning_rate": 0.00019972417367864086,
+      "loss": 0.4381,
+      "step": 155
+    },
+    {
+      "epoch": 0.0880174906551943,
+      "grad_norm": 0.23850011825561523,
+      "learning_rate": 0.00019971974487539157,
+      "loss": 0.4486,
+      "step": 156
+    },
+    {
+      "epoch": 0.08858170533888145,
+      "grad_norm": 0.2331485152244568,
+      "learning_rate": 0.00019971528084912357,
+      "loss": 0.4,
+      "step": 157
+    },
+    {
+      "epoch": 0.08914592002256859,
+      "grad_norm": 0.21300041675567627,
+      "learning_rate": 0.00019971078160141366,
+      "loss": 0.4332,
+      "step": 158
+    },
+    {
+      "epoch": 0.08971013470625573,
+      "grad_norm": 0.21950708329677582,
+      "learning_rate": 0.00019970624713385097,
+      "loss": 0.4032,
+      "step": 159
+    },
+    {
+      "epoch": 0.09027434938994287,
+      "grad_norm": 0.2099374234676361,
+      "learning_rate": 0.00019970167744803725,
+      "loss": 0.4385,
+      "step": 160
+    },
+    {
+      "epoch": 0.09083856407363002,
+      "grad_norm": 0.2099723219871521,
+      "learning_rate": 0.00019969707254558657,
+      "loss": 0.3907,
+      "step": 161
+    },
+    {
+      "epoch": 0.09140277875731716,
+      "grad_norm": 0.23632076382637024,
+      "learning_rate": 0.00019969243242812546,
+      "loss": 0.4239,
+      "step": 162
+    },
+    {
+      "epoch": 0.0919669934410043,
+      "grad_norm": 0.22461844980716705,
+      "learning_rate": 0.00019968775709729294,
+      "loss": 0.4303,
+      "step": 163
+    },
+    {
+      "epoch": 0.09253120812469144,
+      "grad_norm": 0.2540529668331146,
+      "learning_rate": 0.00019968304655474043,
+      "loss": 0.4093,
+      "step": 164
+    },
+    {
+      "epoch": 0.09309542280837858,
+      "grad_norm": 0.24908940494060516,
+      "learning_rate": 0.00019967830080213177,
+      "loss": 0.4695,
+      "step": 165
+    },
+    {
+      "epoch": 0.09365963749206573,
+      "grad_norm": 0.22242751717567444,
+      "learning_rate": 0.00019967351984114323,
+      "loss": 0.3822,
+      "step": 166
+    },
+    {
+      "epoch": 0.09422385217575287,
+      "grad_norm": 0.24038168787956238,
+      "learning_rate": 0.0001996687036734636,
+      "loss": 0.357,
+      "step": 167
+    },
+    {
+      "epoch": 0.09478806685944002,
+      "grad_norm": 0.22208353877067566,
+      "learning_rate": 0.00019966385230079398,
+      "loss": 0.3571,
+      "step": 168
+    },
+    {
+      "epoch": 0.09535228154312717,
+      "grad_norm": 0.21464687585830688,
+      "learning_rate": 0.00019965896572484804,
+      "loss": 0.3443,
+      "step": 169
+    },
+    {
+      "epoch": 0.09591649622681431,
+      "grad_norm": 0.23776409029960632,
+      "learning_rate": 0.00019965404394735176,
+      "loss": 0.4293,
+      "step": 170
+    },
+    {
+      "epoch": 0.09648071091050145,
+      "grad_norm": 0.22006605565547943,
+      "learning_rate": 0.00019964908697004365,
+      "loss": 0.3919,
+      "step": 171
+    },
+    {
+      "epoch": 0.09704492559418859,
+      "grad_norm": 0.22542230784893036,
+      "learning_rate": 0.00019964409479467462,
+      "loss": 0.3788,
+      "step": 172
+    },
+    {
+      "epoch": 0.09760914027787573,
+      "grad_norm": 0.22490368783473969,
+      "learning_rate": 0.00019963906742300795,
+      "loss": 0.36,
+      "step": 173
+    },
+    {
+      "epoch": 0.09817335496156288,
+      "grad_norm": 0.205672949552536,
+      "learning_rate": 0.00019963400485681948,
+      "loss": 0.3854,
+      "step": 174
+    },
+    {
+      "epoch": 0.09873756964525002,
+      "grad_norm": 0.2202606499195099,
+      "learning_rate": 0.00019962890709789735,
+      "loss": 0.3717,
+      "step": 175
+    },
+    {
+      "epoch": 0.09930178432893716,
+      "grad_norm": 0.22787775099277496,
+      "learning_rate": 0.00019962377414804223,
+      "loss": 0.447,
+      "step": 176
+    },
+    {
+      "epoch": 0.0998659990126243,
+      "grad_norm": 0.2405763864517212,
+      "learning_rate": 0.00019961860600906713,
+      "loss": 0.5048,
+      "step": 177
+    },
+    {
+      "epoch": 0.10043021369631144,
+      "grad_norm": 0.24401172995567322,
+      "learning_rate": 0.00019961340268279766,
+      "loss": 0.441,
+      "step": 178
+    },
+    {
+      "epoch": 0.10099442837999859,
+      "grad_norm": 0.20982909202575684,
+      "learning_rate": 0.00019960816417107158,
+      "loss": 0.3983,
+      "step": 179
+    },
+    {
+      "epoch": 0.10155864306368573,
+      "grad_norm": 0.2188471257686615,
+      "learning_rate": 0.00019960289047573935,
+      "loss": 0.4545,
+      "step": 180
+    },
+    {
+      "epoch": 0.10212285774737287,
+      "grad_norm": 0.21007180213928223,
+      "learning_rate": 0.0001995975815986637,
+      "loss": 0.4321,
+      "step": 181
+    },
+    {
+      "epoch": 0.10268707243106003,
+      "grad_norm": 0.23281577229499817,
+      "learning_rate": 0.00019959223754171984,
+      "loss": 0.4981,
+      "step": 182
+    },
+    {
+      "epoch": 0.10325128711474717,
+      "grad_norm": 0.22246459126472473,
+      "learning_rate": 0.00019958685830679543,
+      "loss": 0.3988,
+      "step": 183
+    },
+    {
+      "epoch": 0.10381550179843431,
+      "grad_norm": 0.20650137960910797,
+      "learning_rate": 0.00019958144389579046,
+      "loss": 0.3636,
+      "step": 184
+    },
+    {
+      "epoch": 0.10437971648212145,
+      "grad_norm": 0.21173138916492462,
+      "learning_rate": 0.00019957599431061747,
+      "loss": 0.3361,
+      "step": 185
+    },
+    {
+      "epoch": 0.1049439311658086,
+      "grad_norm": 0.22891078889369965,
+      "learning_rate": 0.00019957050955320133,
+      "loss": 0.4348,
+      "step": 186
+    },
+    {
+      "epoch": 0.10550814584949574,
+      "grad_norm": 0.2189328968524933,
+      "learning_rate": 0.00019956498962547936,
+      "loss": 0.4106,
+      "step": 187
+    },
+    {
+      "epoch": 0.10607236053318288,
+      "grad_norm": 0.22498829662799835,
+      "learning_rate": 0.00019955943452940134,
+      "loss": 0.479,
+      "step": 188
+    },
+    {
+      "epoch": 0.10663657521687002,
+      "grad_norm": 0.2466239482164383,
+      "learning_rate": 0.0001995538442669294,
+      "loss": 0.4616,
+      "step": 189
+    },
+    {
+      "epoch": 0.10720078990055716,
+      "grad_norm": 0.2336626648902893,
+      "learning_rate": 0.00019954821884003818,
+      "loss": 0.4205,
+      "step": 190
+    },
+    {
+      "epoch": 0.1077650045842443,
+      "grad_norm": 0.24595655500888824,
+      "learning_rate": 0.00019954255825071465,
+      "loss": 0.4428,
+      "step": 191
+    },
+    {
+      "epoch": 0.10832921926793145,
+      "grad_norm": 0.23127540946006775,
+      "learning_rate": 0.00019953686250095832,
+      "loss": 0.432,
+      "step": 192
+    },
+    {
+      "epoch": 0.10889343395161859,
+      "grad_norm": 0.23463448882102966,
+      "learning_rate": 0.00019953113159278094,
+      "loss": 0.4252,
+      "step": 193
+    },
+    {
+      "epoch": 0.10945764863530573,
+      "grad_norm": 0.21903161704540253,
+      "learning_rate": 0.00019952536552820684,
+      "loss": 0.403,
+      "step": 194
+    },
+    {
+      "epoch": 0.11002186331899287,
+      "grad_norm": 0.22358113527297974,
+      "learning_rate": 0.0001995195643092727,
+      "loss": 0.3646,
+      "step": 195
+    },
+    {
+      "epoch": 0.11058607800268001,
+      "grad_norm": 0.24384388327598572,
+      "learning_rate": 0.00019951372793802766,
+      "loss": 0.3766,
+      "step": 196
+    },
+    {
+      "epoch": 0.11115029268636717,
+      "grad_norm": 0.23756861686706543,
+      "learning_rate": 0.00019950785641653315,
+      "loss": 0.3691,
+      "step": 197
+    },
+    {
+      "epoch": 0.11171450737005431,
+      "grad_norm": 0.22769290208816528,
+      "learning_rate": 0.00019950194974686323,
+      "loss": 0.369,
+      "step": 198
+    },
+    {
+      "epoch": 0.11227872205374145,
+      "grad_norm": 0.22047221660614014,
+      "learning_rate": 0.00019949600793110417,
+      "loss": 0.3739,
+      "step": 199
+    },
+    {
+      "epoch": 0.1128429367374286,
+      "grad_norm": 0.2341870814561844,
+      "learning_rate": 0.00019949003097135474,
+      "loss": 0.3641,
+      "step": 200
+    },
+    {
+      "epoch": 0.1128429367374286,
+      "eval_loss": 0.39890384674072266,
+      "eval_runtime": 15.3993,
+      "eval_samples_per_second": 9.286,
+      "eval_steps_per_second": 4.676,
+      "step": 200
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 5316,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "EarlyStoppingCallback": {
+      "args": {
+        "early_stopping_patience": 3,
+        "early_stopping_threshold": 0.0
+      },
+      "attributes": {
+        "early_stopping_patience_counter": 0
+      }
+    },
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.64884088471552e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d40aa022b0cba9324db847c078a28e0a2d3222e36a186e8de00ae24ad2a533f0
+size 6776