Training in progress, step 240, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: NousResearch/Nous-Hermes-2-SOLAR-10.7B
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "NousResearch/Nous-Hermes-2-SOLAR-10.7B",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "down_proj",
+    "v_proj",
+    "o_proj",
+    "gate_proj",
+    "q_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1ab16bdd5a0254c0cc8231a4440657492e83beaef1ee5a789415b4f594a60599
+size 125918320

last-checkpoint/added_tokens.json

@@ -0,0 +1,4 @@
+{
+  "<|im_end|>": 32000,
+  "<|im_start|>": 32001
+}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c0d8f296e9762bc02ff5df3984a473f8996db91a2b9658d5a993a2c65ea0ccf6
+size 64683604

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f431ee1b4b36656f1f0e4cfa0a9db856ae2f1272bc810a889b2dd727ac31ff27
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b86692377b6b2dccc36c1bb7def8505d2f39dd62a155b80f10db88dc75e77aa
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,62 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": true,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32000": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32001": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [],
+  "bos_token": "<s>",
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "legacy": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "trust_remote_code": false,
+  "unk_token": "<unk>",
+  "use_default_system_prompt": true,
+  "use_fast": true
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,1713 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.25,
+  "eval_steps": 500,
+  "global_step": 240,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0010416666666666667,
+      "grad_norm": 3.89841890335083,
+      "learning_rate": 2e-05,
+      "loss": 1.3045,
+      "step": 1
+    },
+    {
+      "epoch": 0.0020833333333333333,
+      "grad_norm": 3.155778646469116,
+      "learning_rate": 4e-05,
+      "loss": 1.2908,
+      "step": 2
+    },
+    {
+      "epoch": 0.003125,
+      "grad_norm": 3.7164533138275146,
+      "learning_rate": 6e-05,
+      "loss": 1.3904,
+      "step": 3
+    },
+    {
+      "epoch": 0.004166666666666667,
+      "grad_norm": 3.635389566421509,
+      "learning_rate": 8e-05,
+      "loss": 1.3371,
+      "step": 4
+    },
+    {
+      "epoch": 0.005208333333333333,
+      "grad_norm": 4.847501277923584,
+      "learning_rate": 0.0001,
+      "loss": 1.2251,
+      "step": 5
+    },
+    {
+      "epoch": 0.00625,
+      "grad_norm": 4.314141273498535,
+      "learning_rate": 9.999972945929386e-05,
+      "loss": 1.0889,
+      "step": 6
+    },
+    {
+      "epoch": 0.007291666666666667,
+      "grad_norm": 3.0280661582946777,
+      "learning_rate": 9.999891784010313e-05,
+      "loss": 0.9793,
+      "step": 7
+    },
+    {
+      "epoch": 0.008333333333333333,
+      "grad_norm": 3.5087926387786865,
+      "learning_rate": 9.999756515121088e-05,
+      "loss": 1.0261,
+      "step": 8
+    },
+    {
+      "epoch": 0.009375,
+      "grad_norm": 2.9475789070129395,
+      "learning_rate": 9.999567140725535e-05,
+      "loss": 0.88,
+      "step": 9
+    },
+    {
+      "epoch": 0.010416666666666666,
+      "grad_norm": 2.6721155643463135,
+      "learning_rate": 9.999323662872997e-05,
+      "loss": 0.9092,
+      "step": 10
+    },
+    {
+      "epoch": 0.011458333333333333,
+      "grad_norm": 2.524888038635254,
+      "learning_rate": 9.9990260841983e-05,
+      "loss": 0.8489,
+      "step": 11
+    },
+    {
+      "epoch": 0.0125,
+      "grad_norm": 2.4814159870147705,
+      "learning_rate": 9.998674407921731e-05,
+      "loss": 0.8187,
+      "step": 12
+    },
+    {
+      "epoch": 0.013541666666666667,
+      "grad_norm": 2.0676493644714355,
+      "learning_rate": 9.998268637849e-05,
+      "loss": 0.8502,
+      "step": 13
+    },
+    {
+      "epoch": 0.014583333333333334,
+      "grad_norm": 2.7224721908569336,
+      "learning_rate": 9.997808778371196e-05,
+      "loss": 0.8064,
+      "step": 14
+    },
+    {
+      "epoch": 0.015625,
+      "grad_norm": 1.890153408050537,
+      "learning_rate": 9.99729483446475e-05,
+      "loss": 0.6713,
+      "step": 15
+    },
+    {
+      "epoch": 0.016666666666666666,
+      "grad_norm": 1.9269518852233887,
+      "learning_rate": 9.996726811691375e-05,
+      "loss": 0.7606,
+      "step": 16
+    },
+    {
+      "epoch": 0.017708333333333333,
+      "grad_norm": 2.558152437210083,
+      "learning_rate": 9.996104716197996e-05,
+      "loss": 0.8049,
+      "step": 17
+    },
+    {
+      "epoch": 0.01875,
+      "grad_norm": 2.1918888092041016,
+      "learning_rate": 9.995428554716705e-05,
+      "loss": 0.7751,
+      "step": 18
+    },
+    {
+      "epoch": 0.019791666666666666,
+      "grad_norm": 2.094609498977661,
+      "learning_rate": 9.994698334564666e-05,
+      "loss": 0.8373,
+      "step": 19
+    },
+    {
+      "epoch": 0.020833333333333332,
+      "grad_norm": 2.2064154148101807,
+      "learning_rate": 9.993914063644052e-05,
+      "loss": 0.9063,
+      "step": 20
+    },
+    {
+      "epoch": 0.021875,
+      "grad_norm": 2.315425157546997,
+      "learning_rate": 9.993075750441952e-05,
+      "loss": 0.8035,
+      "step": 21
+    },
+    {
+      "epoch": 0.022916666666666665,
+      "grad_norm": 1.9188307523727417,
+      "learning_rate": 9.992183404030277e-05,
+      "loss": 0.7029,
+      "step": 22
+    },
+    {
+      "epoch": 0.023958333333333335,
+      "grad_norm": 1.783411979675293,
+      "learning_rate": 9.991237034065673e-05,
+      "loss": 0.7527,
+      "step": 23
+    },
+    {
+      "epoch": 0.025,
+      "grad_norm": 2.344759225845337,
+      "learning_rate": 9.990236650789397e-05,
+      "loss": 0.8031,
+      "step": 24
+    },
+    {
+      "epoch": 0.026041666666666668,
+      "grad_norm": 2.0993213653564453,
+      "learning_rate": 9.989182265027232e-05,
+      "loss": 0.8108,
+      "step": 25
+    },
+    {
+      "epoch": 0.027083333333333334,
+      "grad_norm": 2.659205913543701,
+      "learning_rate": 9.988073888189344e-05,
+      "loss": 0.8461,
+      "step": 26
+    },
+    {
+      "epoch": 0.028125,
+      "grad_norm": 1.8281415700912476,
+      "learning_rate": 9.986911532270176e-05,
+      "loss": 0.7059,
+      "step": 27
+    },
+    {
+      "epoch": 0.029166666666666667,
+      "grad_norm": 1.9623245000839233,
+      "learning_rate": 9.985695209848313e-05,
+      "loss": 0.7234,
+      "step": 28
+    },
+    {
+      "epoch": 0.030208333333333334,
+      "grad_norm": 2.053891658782959,
+      "learning_rate": 9.984424934086346e-05,
+      "loss": 0.7011,
+      "step": 29
+    },
+    {
+      "epoch": 0.03125,
+      "grad_norm": 1.9886325597763062,
+      "learning_rate": 9.98310071873072e-05,
+      "loss": 0.7637,
+      "step": 30
+    },
+    {
+      "epoch": 0.03229166666666667,
+      "grad_norm": 10.491949081420898,
+      "learning_rate": 9.981722578111609e-05,
+      "loss": 0.839,
+      "step": 31
+    },
+    {
+      "epoch": 0.03333333333333333,
+      "grad_norm": 7.943881988525391,
+      "learning_rate": 9.980290527142734e-05,
+      "loss": 0.7733,
+      "step": 32
+    },
+    {
+      "epoch": 0.034375,
+      "grad_norm": 2.602108955383301,
+      "learning_rate": 9.978804581321219e-05,
+      "loss": 0.6808,
+      "step": 33
+    },
+    {
+      "epoch": 0.035416666666666666,
+      "grad_norm": 2.1296403408050537,
+      "learning_rate": 9.977264756727418e-05,
+      "loss": 0.5759,
+      "step": 34
+    },
+    {
+      "epoch": 0.036458333333333336,
+      "grad_norm": 2.6959056854248047,
+      "learning_rate": 9.97567107002474e-05,
+      "loss": 0.7861,
+      "step": 35
+    },
+    {
+      "epoch": 0.0375,
+      "grad_norm": 1.9752788543701172,
+      "learning_rate": 9.97402353845947e-05,
+      "loss": 0.8189,
+      "step": 36
+    },
+    {
+      "epoch": 0.03854166666666667,
+      "grad_norm": 2.088995933532715,
+      "learning_rate": 9.972322179860583e-05,
+      "loss": 0.7071,
+      "step": 37
+    },
+    {
+      "epoch": 0.03958333333333333,
+      "grad_norm": 2.0158233642578125,
+      "learning_rate": 9.970567012639546e-05,
+      "loss": 0.5708,
+      "step": 38
+    },
+    {
+      "epoch": 0.040625,
+      "grad_norm": 2.4239232540130615,
+      "learning_rate": 9.968758055790132e-05,
+      "loss": 0.7652,
+      "step": 39
+    },
+    {
+      "epoch": 0.041666666666666664,
+      "grad_norm": 2.455787420272827,
+      "learning_rate": 9.966895328888194e-05,
+      "loss": 0.7408,
+      "step": 40
+    },
+    {
+      "epoch": 0.042708333333333334,
+      "grad_norm": 2.0814132690429688,
+      "learning_rate": 9.964978852091474e-05,
+      "loss": 0.6772,
+      "step": 41
+    },
+    {
+      "epoch": 0.04375,
+      "grad_norm": 2.4373435974121094,
+      "learning_rate": 9.963008646139367e-05,
+      "loss": 0.6243,
+      "step": 42
+    },
+    {
+      "epoch": 0.04479166666666667,
+      "grad_norm": 2.3441109657287598,
+      "learning_rate": 9.960984732352713e-05,
+      "loss": 0.711,
+      "step": 43
+    },
+    {
+      "epoch": 0.04583333333333333,
+      "grad_norm": 2.287811756134033,
+      "learning_rate": 9.958907132633554e-05,
+      "loss": 0.5885,
+      "step": 44
+    },
+    {
+      "epoch": 0.046875,
+      "grad_norm": 2.8655076026916504,
+      "learning_rate": 9.956775869464901e-05,
+      "loss": 0.7893,
+      "step": 45
+    },
+    {
+      "epoch": 0.04791666666666667,
+      "grad_norm": 2.881870746612549,
+      "learning_rate": 9.954590965910493e-05,
+      "loss": 0.7088,
+      "step": 46
+    },
+    {
+      "epoch": 0.04895833333333333,
+      "grad_norm": 2.6930508613586426,
+      "learning_rate": 9.952352445614541e-05,
+      "loss": 0.5712,
+      "step": 47
+    },
+    {
+      "epoch": 0.05,
+      "grad_norm": 2.178975820541382,
+      "learning_rate": 9.950060332801485e-05,
+      "loss": 0.5696,
+      "step": 48
+    },
+    {
+      "epoch": 0.051041666666666666,
+      "grad_norm": 3.1573169231414795,
+      "learning_rate": 9.947714652275711e-05,
+      "loss": 0.705,
+      "step": 49
+    },
+    {
+      "epoch": 0.052083333333333336,
+      "grad_norm": 3.3601553440093994,
+      "learning_rate": 9.945315429421306e-05,
+      "loss": 0.5505,
+      "step": 50
+    },
+    {
+      "epoch": 0.053125,
+      "grad_norm": 4.582420349121094,
+      "learning_rate": 9.942862690201768e-05,
+      "loss": 1.1858,
+      "step": 51
+    },
+    {
+      "epoch": 0.05416666666666667,
+      "grad_norm": 3.9910128116607666,
+      "learning_rate": 9.940356461159727e-05,
+      "loss": 1.0166,
+      "step": 52
+    },
+    {
+      "epoch": 0.05520833333333333,
+      "grad_norm": 2.5004897117614746,
+      "learning_rate": 9.937796769416663e-05,
+      "loss": 0.8907,
+      "step": 53
+    },
+    {
+      "epoch": 0.05625,
+      "grad_norm": 2.0037131309509277,
+      "learning_rate": 9.935183642672608e-05,
+      "loss": 0.8983,
+      "step": 54
+    },
+    {
+      "epoch": 0.057291666666666664,
+      "grad_norm": 1.9318690299987793,
+      "learning_rate": 9.932517109205849e-05,
+      "loss": 0.8029,
+      "step": 55
+    },
+    {
+      "epoch": 0.058333333333333334,
+      "grad_norm": 1.9894757270812988,
+      "learning_rate": 9.929797197872619e-05,
+      "loss": 0.8087,
+      "step": 56
+    },
+    {
+      "epoch": 0.059375,
+      "grad_norm": 1.8991605043411255,
+      "learning_rate": 9.927023938106788e-05,
+      "loss": 0.8239,
+      "step": 57
+    },
+    {
+      "epoch": 0.06041666666666667,
+      "grad_norm": 2.642719268798828,
+      "learning_rate": 9.92419735991954e-05,
+      "loss": 0.8338,
+      "step": 58
+    },
+    {
+      "epoch": 0.06145833333333333,
+      "grad_norm": 2.018235683441162,
+      "learning_rate": 9.921317493899057e-05,
+      "loss": 0.7295,
+      "step": 59
+    },
+    {
+      "epoch": 0.0625,
+      "grad_norm": 1.6592891216278076,
+      "learning_rate": 9.918384371210176e-05,
+      "loss": 0.7484,
+      "step": 60
+    },
+    {
+      "epoch": 0.06354166666666666,
+      "grad_norm": 2.1405680179595947,
+      "learning_rate": 9.915398023594063e-05,
+      "loss": 0.7541,
+      "step": 61
+    },
+    {
+      "epoch": 0.06458333333333334,
+      "grad_norm": 2.276129961013794,
+      "learning_rate": 9.912358483367856e-05,
+      "loss": 0.9185,
+      "step": 62
+    },
+    {
+      "epoch": 0.065625,
+      "grad_norm": 2.695638418197632,
+      "learning_rate": 9.909265783424337e-05,
+      "loss": 0.7996,
+      "step": 63
+    },
+    {
+      "epoch": 0.06666666666666667,
+      "grad_norm": 2.3187263011932373,
+      "learning_rate": 9.906119957231548e-05,
+      "loss": 0.811,
+      "step": 64
+    },
+    {
+      "epoch": 0.06770833333333333,
+      "grad_norm": 2.0456435680389404,
+      "learning_rate": 9.902921038832455e-05,
+      "loss": 0.786,
+      "step": 65
+    },
+    {
+      "epoch": 0.06875,
+      "grad_norm": 2.2249789237976074,
+      "learning_rate": 9.899669062844563e-05,
+      "loss": 0.9081,
+      "step": 66
+    },
+    {
+      "epoch": 0.06979166666666667,
+      "grad_norm": 1.973425269126892,
+      "learning_rate": 9.896364064459547e-05,
+      "loss": 0.6911,
+      "step": 67
+    },
+    {
+      "epoch": 0.07083333333333333,
+      "grad_norm": 1.6641013622283936,
+      "learning_rate": 9.893006079442868e-05,
+      "loss": 0.8201,
+      "step": 68
+    },
+    {
+      "epoch": 0.071875,
+      "grad_norm": 3.695594549179077,
+      "learning_rate": 9.889595144133397e-05,
+      "loss": 0.9104,
+      "step": 69
+    },
+    {
+      "epoch": 0.07291666666666667,
+      "grad_norm": 1.972715139389038,
+      "learning_rate": 9.886131295443003e-05,
+      "loss": 0.7935,
+      "step": 70
+    },
+    {
+      "epoch": 0.07395833333333333,
+      "grad_norm": 1.976627230644226,
+      "learning_rate": 9.882614570856172e-05,
+      "loss": 0.8116,
+      "step": 71
+    },
+    {
+      "epoch": 0.075,
+      "grad_norm": 1.9199652671813965,
+      "learning_rate": 9.879045008429588e-05,
+      "loss": 0.7099,
+      "step": 72
+    },
+    {
+      "epoch": 0.07604166666666666,
+      "grad_norm": 1.676329255104065,
+      "learning_rate": 9.87542264679173e-05,
+      "loss": 0.806,
+      "step": 73
+    },
+    {
+      "epoch": 0.07708333333333334,
+      "grad_norm": 1.8327699899673462,
+      "learning_rate": 9.871747525142447e-05,
+      "loss": 0.6798,
+      "step": 74
+    },
+    {
+      "epoch": 0.078125,
+      "grad_norm": 2.0520970821380615,
+      "learning_rate": 9.868019683252543e-05,
+      "loss": 0.7673,
+      "step": 75
+    },
+    {
+      "epoch": 0.07916666666666666,
+      "grad_norm": 2.136255979537964,
+      "learning_rate": 9.864239161463334e-05,
+      "loss": 0.7315,
+      "step": 76
+    },
+    {
+      "epoch": 0.08020833333333334,
+      "grad_norm": 2.5858349800109863,
+      "learning_rate": 9.860406000686221e-05,
+      "loss": 0.7961,
+      "step": 77
+    },
+    {
+      "epoch": 0.08125,
+      "grad_norm": 1.6929069757461548,
+      "learning_rate": 9.856520242402248e-05,
+      "loss": 0.7337,
+      "step": 78
+    },
+    {
+      "epoch": 0.08229166666666667,
+      "grad_norm": 1.5802732706069946,
+      "learning_rate": 9.852581928661642e-05,
+      "loss": 0.5857,
+      "step": 79
+    },
+    {
+      "epoch": 0.08333333333333333,
+      "grad_norm": 1.929983377456665,
+      "learning_rate": 9.848591102083375e-05,
+      "loss": 0.8097,
+      "step": 80
+    },
+    {
+      "epoch": 0.084375,
+      "grad_norm": 1.926419973373413,
+      "learning_rate": 9.844547805854687e-05,
+      "loss": 0.6829,
+      "step": 81
+    },
+    {
+      "epoch": 0.08541666666666667,
+      "grad_norm": 2.231621026992798,
+      "learning_rate": 9.840452083730624e-05,
+      "loss": 0.6391,
+      "step": 82
+    },
+    {
+      "epoch": 0.08645833333333333,
+      "grad_norm": 2.233232021331787,
+      "learning_rate": 9.836303980033572e-05,
+      "loss": 0.6513,
+      "step": 83
+    },
+    {
+      "epoch": 0.0875,
+      "grad_norm": 1.8242605924606323,
+      "learning_rate": 9.832103539652766e-05,
+      "loss": 0.7875,
+      "step": 84
+    },
+    {
+      "epoch": 0.08854166666666667,
+      "grad_norm": 1.8912043571472168,
+      "learning_rate": 9.82785080804381e-05,
+      "loss": 0.689,
+      "step": 85
+    },
+    {
+      "epoch": 0.08958333333333333,
+      "grad_norm": 1.8962242603302002,
+      "learning_rate": 9.823545831228183e-05,
+      "loss": 0.6863,
+      "step": 86
+    },
+    {
+      "epoch": 0.090625,
+      "grad_norm": 2.398500442504883,
+      "learning_rate": 9.819188655792746e-05,
+      "loss": 0.7515,
+      "step": 87
+    },
+    {
+      "epoch": 0.09166666666666666,
+      "grad_norm": 2.078723192214966,
+      "learning_rate": 9.814779328889229e-05,
+      "loss": 0.843,
+      "step": 88
+    },
+    {
+      "epoch": 0.09270833333333334,
+      "grad_norm": 2.020238161087036,
+      "learning_rate": 9.810317898233731e-05,
+      "loss": 0.8008,
+      "step": 89
+    },
+    {
+      "epoch": 0.09375,
+      "grad_norm": 2.178727626800537,
+      "learning_rate": 9.805804412106198e-05,
+      "loss": 0.8205,
+      "step": 90
+    },
+    {
+      "epoch": 0.09479166666666666,
+      "grad_norm": 1.8760087490081787,
+      "learning_rate": 9.801238919349894e-05,
+      "loss": 0.6174,
+      "step": 91
+    },
+    {
+      "epoch": 0.09583333333333334,
+      "grad_norm": 1.8878923654556274,
+      "learning_rate": 9.796621469370889e-05,
+      "loss": 0.6422,
+      "step": 92
+    },
+    {
+      "epoch": 0.096875,
+      "grad_norm": 2.049712657928467,
+      "learning_rate": 9.791952112137505e-05,
+      "loss": 0.8606,
+      "step": 93
+    },
+    {
+      "epoch": 0.09791666666666667,
+      "grad_norm": 2.280137300491333,
+      "learning_rate": 9.787230898179794e-05,
+      "loss": 0.7064,
+      "step": 94
+    },
+    {
+      "epoch": 0.09895833333333333,
+      "grad_norm": 2.386488199234009,
+      "learning_rate": 9.782457878588977e-05,
+      "loss": 0.7416,
+      "step": 95
+    },
+    {
+      "epoch": 0.1,
+      "grad_norm": 2.596226215362549,
+      "learning_rate": 9.777633105016897e-05,
+      "loss": 0.6648,
+      "step": 96
+    },
+    {
+      "epoch": 0.10104166666666667,
+      "grad_norm": 1.9814091920852661,
+      "learning_rate": 9.772756629675464e-05,
+      "loss": 0.6358,
+      "step": 97
+    },
+    {
+      "epoch": 0.10208333333333333,
+      "grad_norm": 2.1925013065338135,
+      "learning_rate": 9.767828505336075e-05,
+      "loss": 0.6378,
+      "step": 98
+    },
+    {
+      "epoch": 0.103125,
+      "grad_norm": 2.372941017150879,
+      "learning_rate": 9.762848785329064e-05,
+      "loss": 0.4603,
+      "step": 99
+    },
+    {
+      "epoch": 0.10416666666666667,
+      "grad_norm": 2.133225679397583,
+      "learning_rate": 9.757817523543109e-05,
+      "loss": 0.5506,
+      "step": 100
+    },
+    {
+      "epoch": 0.10520833333333333,
+      "grad_norm": 1.969484806060791,
+      "learning_rate": 9.752734774424653e-05,
+      "loss": 0.9866,
+      "step": 101
+    },
+    {
+      "epoch": 0.10625,
+      "grad_norm": 1.7341316938400269,
+      "learning_rate": 9.747600592977318e-05,
+      "loss": 0.8812,
+      "step": 102
+    },
+    {
+      "epoch": 0.10729166666666666,
+      "grad_norm": 1.6266231536865234,
+      "learning_rate": 9.742415034761308e-05,
+      "loss": 0.8481,
+      "step": 103
+    },
+    {
+      "epoch": 0.10833333333333334,
+      "grad_norm": 1.4822884798049927,
+      "learning_rate": 9.737178155892805e-05,
+      "loss": 0.8431,
+      "step": 104
+    },
+    {
+      "epoch": 0.109375,
+      "grad_norm": 1.5820056200027466,
+      "learning_rate": 9.731890013043368e-05,
+      "loss": 0.7855,
+      "step": 105
+    },
+    {
+      "epoch": 0.11041666666666666,
+      "grad_norm": 1.4592959880828857,
+      "learning_rate": 9.72655066343931e-05,
+      "loss": 0.769,
+      "step": 106
+    },
+    {
+      "epoch": 0.11145833333333334,
+      "grad_norm": 1.7101143598556519,
+      "learning_rate": 9.721160164861087e-05,
+      "loss": 0.8562,
+      "step": 107
+    },
+    {
+      "epoch": 0.1125,
+      "grad_norm": 1.946768045425415,
+      "learning_rate": 9.715718575642674e-05,
+      "loss": 0.7703,
+      "step": 108
+    },
+    {
+      "epoch": 0.11354166666666667,
+      "grad_norm": 1.7165383100509644,
+      "learning_rate": 9.710225954670925e-05,
+      "loss": 0.824,
+      "step": 109
+    },
+    {
+      "epoch": 0.11458333333333333,
+      "grad_norm": 1.8618409633636475,
+      "learning_rate": 9.704682361384941e-05,
+      "loss": 0.8197,
+      "step": 110
+    },
+    {
+      "epoch": 0.115625,
+      "grad_norm": 2.930100679397583,
+      "learning_rate": 9.699087855775429e-05,
+      "loss": 0.6797,
+      "step": 111
+    },
+    {
+      "epoch": 0.11666666666666667,
+      "grad_norm": 17.515884399414062,
+      "learning_rate": 9.693442498384048e-05,
+      "loss": 0.7555,
+      "step": 112
+    },
+    {
+      "epoch": 0.11770833333333333,
+      "grad_norm": 1.614569902420044,
+      "learning_rate": 9.687746350302757e-05,
+      "loss": 0.8411,
+      "step": 113
+    },
+    {
+      "epoch": 0.11875,
+      "grad_norm": 1.7483561038970947,
+      "learning_rate": 9.681999473173155e-05,
+      "loss": 0.7561,
+      "step": 114
+    },
+    {
+      "epoch": 0.11979166666666667,
+      "grad_norm": 1.792314887046814,
+      "learning_rate": 9.676201929185809e-05,
+      "loss": 0.6711,
+      "step": 115
+    },
+    {
+      "epoch": 0.12083333333333333,
+      "grad_norm": 1.8255658149719238,
+      "learning_rate": 9.670353781079582e-05,
+      "loss": 0.8284,
+      "step": 116
+    },
+    {
+      "epoch": 0.121875,
+      "grad_norm": 1.903437852859497,
+      "learning_rate": 9.664455092140963e-05,
+      "loss": 0.9174,
+      "step": 117
+    },
+    {
+      "epoch": 0.12291666666666666,
+      "grad_norm": 1.5207189321517944,
+      "learning_rate": 9.658505926203367e-05,
+      "loss": 0.7751,
+      "step": 118
+    },
+    {
+      "epoch": 0.12395833333333334,
+      "grad_norm": 2.0220348834991455,
+      "learning_rate": 9.652506347646459e-05,
+      "loss": 0.9067,
+      "step": 119
+    },
+    {
+      "epoch": 0.125,
+      "grad_norm": 1.8411701917648315,
+      "learning_rate": 9.646456421395446e-05,
+      "loss": 0.7233,
+      "step": 120
+    },
+    {
+      "epoch": 0.12604166666666666,
+      "grad_norm": 1.6238988637924194,
+      "learning_rate": 9.640356212920383e-05,
+      "loss": 0.6868,
+      "step": 121
+    },
+    {
+      "epoch": 0.12708333333333333,
+      "grad_norm": 1.544351577758789,
+      "learning_rate": 9.634205788235456e-05,
+      "loss": 0.7206,
+      "step": 122
+    },
+    {
+      "epoch": 0.128125,
+      "grad_norm": 2.071779727935791,
+      "learning_rate": 9.628005213898274e-05,
+      "loss": 0.6462,
+      "step": 123
+    },
+    {
+      "epoch": 0.12916666666666668,
+      "grad_norm": 1.5759272575378418,
+      "learning_rate": 9.621754557009151e-05,
+      "loss": 0.7485,
+      "step": 124
+    },
+    {
+      "epoch": 0.13020833333333334,
+      "grad_norm": 1.542765736579895,
+      "learning_rate": 9.615453885210369e-05,
+      "loss": 0.7167,
+      "step": 125
+    },
+    {
+      "epoch": 0.13125,
+      "grad_norm": 1.80550217628479,
+      "learning_rate": 9.609103266685456e-05,
+      "loss": 0.7768,
+      "step": 126
+    },
+    {
+      "epoch": 0.13229166666666667,
+      "grad_norm": 1.5399523973464966,
+      "learning_rate": 9.602702770158446e-05,
+      "loss": 0.6797,
+      "step": 127
+    },
+    {
+      "epoch": 0.13333333333333333,
+      "grad_norm": 2.0065791606903076,
+      "learning_rate": 9.596252464893133e-05,
+      "loss": 0.7057,
+      "step": 128
+    },
+    {
+      "epoch": 0.134375,
+      "grad_norm": 2.507671594619751,
+      "learning_rate": 9.589752420692321e-05,
+      "loss": 0.6818,
+      "step": 129
+    },
+    {
+      "epoch": 0.13541666666666666,
+      "grad_norm": 7.731146335601807,
+      "learning_rate": 9.583202707897074e-05,
+      "loss": 0.7506,
+      "step": 130
+    },
+    {
+      "epoch": 0.13645833333333332,
+      "grad_norm": 2.061361789703369,
+      "learning_rate": 9.576603397385948e-05,
+      "loss": 0.7969,
+      "step": 131
+    },
+    {
+      "epoch": 0.1375,
+      "grad_norm": 1.4545551538467407,
+      "learning_rate": 9.569954560574227e-05,
+      "loss": 0.6203,
+      "step": 132
+    },
+    {
+      "epoch": 0.13854166666666667,
+      "grad_norm": 1.5206444263458252,
+      "learning_rate": 9.563256269413155e-05,
+      "loss": 0.698,
+      "step": 133
+    },
+    {
+      "epoch": 0.13958333333333334,
+      "grad_norm": 2.03974986076355,
+      "learning_rate": 9.556508596389144e-05,
+      "loss": 0.6328,
+      "step": 134
+    },
+    {
+      "epoch": 0.140625,
+      "grad_norm": 1.8613979816436768,
+      "learning_rate": 9.549711614523007e-05,
+      "loss": 0.7363,
+      "step": 135
+    },
+    {
+      "epoch": 0.14166666666666666,
+      "grad_norm": 2.0089638233184814,
+      "learning_rate": 9.542865397369152e-05,
+      "loss": 0.6602,
+      "step": 136
+    },
+    {
+      "epoch": 0.14270833333333333,
+      "grad_norm": 1.96244478225708,
+      "learning_rate": 9.535970019014797e-05,
+      "loss": 0.6621,
+      "step": 137
+    },
+    {
+      "epoch": 0.14375,
+      "grad_norm": 1.8381259441375732,
+      "learning_rate": 9.529025554079164e-05,
+      "loss": 0.6835,
+      "step": 138
+    },
+    {
+      "epoch": 0.14479166666666668,
+      "grad_norm": 2.0366594791412354,
+      "learning_rate": 9.522032077712669e-05,
+      "loss": 0.7131,
+      "step": 139
+    },
+    {
+      "epoch": 0.14583333333333334,
+      "grad_norm": 2.140258312225342,
+      "learning_rate": 9.514989665596114e-05,
+      "loss": 0.6861,
+      "step": 140
+    },
+    {
+      "epoch": 0.146875,
+      "grad_norm": 1.8923845291137695,
+      "learning_rate": 9.507898393939868e-05,
+      "loss": 0.6579,
+      "step": 141
+    },
+    {
+      "epoch": 0.14791666666666667,
+      "grad_norm": 1.9693942070007324,
+      "learning_rate": 9.500758339483032e-05,
+      "loss": 0.5021,
+      "step": 142
+    },
+    {
+      "epoch": 0.14895833333333333,
+      "grad_norm": 2.193319797515869,
+      "learning_rate": 9.493569579492624e-05,
+      "loss": 0.7578,
+      "step": 143
+    },
+    {
+      "epoch": 0.15,
+      "grad_norm": 1.9430756568908691,
+      "learning_rate": 9.486332191762731e-05,
+      "loss": 0.6479,
+      "step": 144
+    },
+    {
+      "epoch": 0.15104166666666666,
+      "grad_norm": 2.0702779293060303,
+      "learning_rate": 9.479046254613673e-05,
+      "loss": 0.6854,
+      "step": 145
+    },
+    {
+      "epoch": 0.15208333333333332,
+      "grad_norm": 2.7211689949035645,
+      "learning_rate": 9.471711846891151e-05,
+      "loss": 0.7528,
+      "step": 146
+    },
+    {
+      "epoch": 0.153125,
+      "grad_norm": 3.0110549926757812,
+      "learning_rate": 9.464329047965404e-05,
+      "loss": 0.6203,
+      "step": 147
+    },
+    {
+      "epoch": 0.15416666666666667,
+      "grad_norm": 2.2492358684539795,
+      "learning_rate": 9.456897937730332e-05,
+      "loss": 0.6386,
+      "step": 148
+    },
+    {
+      "epoch": 0.15520833333333334,
+      "grad_norm": 2.6757397651672363,
+      "learning_rate": 9.449418596602648e-05,
+      "loss": 0.6138,
+      "step": 149
+    },
+    {
+      "epoch": 0.15625,
+      "grad_norm": 2.7412190437316895,
+      "learning_rate": 9.441891105521006e-05,
+      "loss": 0.4725,
+      "step": 150
+    },
+    {
+      "epoch": 0.15729166666666666,
+      "grad_norm": 1.6710883378982544,
+      "learning_rate": 9.43431554594511e-05,
+      "loss": 0.9123,
+      "step": 151
+    },
+    {
+      "epoch": 0.15833333333333333,
+      "grad_norm": 1.3830335140228271,
+      "learning_rate": 9.426691999854853e-05,
+      "loss": 0.7745,
+      "step": 152
+    },
+    {
+      "epoch": 0.159375,
+      "grad_norm": 1.5382630825042725,
+      "learning_rate": 9.419020549749417e-05,
+      "loss": 0.7957,
+      "step": 153
+    },
+    {
+      "epoch": 0.16041666666666668,
+      "grad_norm": 1.481225609779358,
+      "learning_rate": 9.411301278646382e-05,
+      "loss": 0.6751,
+      "step": 154
+    },
+    {
+      "epoch": 0.16145833333333334,
+      "grad_norm": 1.372977614402771,
+      "learning_rate": 9.403534270080829e-05,
+      "loss": 0.796,
+      "step": 155
+    },
+    {
+      "epoch": 0.1625,
+      "grad_norm": 1.4860180616378784,
+      "learning_rate": 9.395719608104441e-05,
+      "loss": 0.7453,
+      "step": 156
+    },
+    {
+      "epoch": 0.16354166666666667,
+      "grad_norm": 1.3220758438110352,
+      "learning_rate": 9.387857377284582e-05,
+      "loss": 0.763,
+      "step": 157
+    },
+    {
+      "epoch": 0.16458333333333333,
+      "grad_norm": 1.8010667562484741,
+      "learning_rate": 9.37994766270339e-05,
+      "loss": 0.7671,
+      "step": 158
+    },
+    {
+      "epoch": 0.165625,
+      "grad_norm": 1.506685495376587,
+      "learning_rate": 9.371990549956857e-05,
+      "loss": 0.8321,
+      "step": 159
+    },
+    {
+      "epoch": 0.16666666666666666,
+      "grad_norm": 1.5654361248016357,
+      "learning_rate": 9.3639861251539e-05,
+      "loss": 0.7656,
+      "step": 160
+    },
+    {
+      "epoch": 0.16770833333333332,
+      "grad_norm": 1.5490511655807495,
+      "learning_rate": 9.355934474915428e-05,
+      "loss": 0.682,
+      "step": 161
+    },
+    {
+      "epoch": 0.16875,
+      "grad_norm": 1.359991431236267,
+      "learning_rate": 9.347835686373407e-05,
+      "loss": 0.7185,
+      "step": 162
+    },
+    {
+      "epoch": 0.16979166666666667,
+      "grad_norm": 1.7046531438827515,
+      "learning_rate": 9.339689847169915e-05,
+      "loss": 0.8576,
+      "step": 163
+    },
+    {
+      "epoch": 0.17083333333333334,
+      "grad_norm": 1.369984745979309,
+      "learning_rate": 9.331497045456195e-05,
+      "loss": 0.7514,
+      "step": 164
+    },
+    {
+      "epoch": 0.171875,
+      "grad_norm": 1.5297783613204956,
+      "learning_rate": 9.323257369891703e-05,
+      "loss": 0.7693,
+      "step": 165
+    },
+    {
+      "epoch": 0.17291666666666666,
+      "grad_norm": 2.0187017917633057,
+      "learning_rate": 9.314970909643143e-05,
+      "loss": 0.7888,
+      "step": 166
+    },
+    {
+      "epoch": 0.17395833333333333,
+      "grad_norm": 1.7571396827697754,
+      "learning_rate": 9.306637754383511e-05,
+      "loss": 0.8196,
+      "step": 167
+    },
+    {
+      "epoch": 0.175,
+      "grad_norm": 1.5250581502914429,
+      "learning_rate": 9.29825799429111e-05,
+      "loss": 0.6703,
+      "step": 168
+    },
+    {
+      "epoch": 0.17604166666666668,
+      "grad_norm": 1.5196141004562378,
+      "learning_rate": 9.289831720048593e-05,
+      "loss": 0.6638,
+      "step": 169
+    },
+    {
+      "epoch": 0.17708333333333334,
+      "grad_norm": 1.5836235284805298,
+      "learning_rate": 9.281359022841965e-05,
+      "loss": 0.6461,
+      "step": 170
+    },
+    {
+      "epoch": 0.178125,
+      "grad_norm": 1.3843772411346436,
+      "learning_rate": 9.272839994359606e-05,
+      "loss": 0.7191,
+      "step": 171
+    },
+    {
+      "epoch": 0.17916666666666667,
+      "grad_norm": 1.5738234519958496,
+      "learning_rate": 9.264274726791276e-05,
+      "loss": 0.7675,
+      "step": 172
+    },
+    {
+      "epoch": 0.18020833333333333,
+      "grad_norm": 1.7716032266616821,
+      "learning_rate": 9.255663312827116e-05,
+      "loss": 0.7056,
+      "step": 173
+    },
+    {
+      "epoch": 0.18125,
+      "grad_norm": 1.524893045425415,
+      "learning_rate": 9.247005845656647e-05,
+      "loss": 0.7369,
+      "step": 174
+    },
+    {
+      "epoch": 0.18229166666666666,
+      "grad_norm": 1.6304320096969604,
+      "learning_rate": 9.238302418967756e-05,
+      "loss": 0.6334,
+      "step": 175
+    },
+    {
+      "epoch": 0.18333333333333332,
+      "grad_norm": 1.6756341457366943,
+      "learning_rate": 9.229553126945699e-05,
+      "loss": 0.7805,
+      "step": 176
+    },
+    {
+      "epoch": 0.184375,
+      "grad_norm": 1.8465510606765747,
+      "learning_rate": 9.220758064272054e-05,
+      "loss": 0.7257,
+      "step": 177
+    },
+    {
+      "epoch": 0.18541666666666667,
+      "grad_norm": 1.470868468284607,
+      "learning_rate": 9.211917326123724e-05,
+      "loss": 0.711,
+      "step": 178
+    },
+    {
+      "epoch": 0.18645833333333334,
+      "grad_norm": 1.9779666662216187,
+      "learning_rate": 9.203031008171891e-05,
+      "loss": 0.7962,
+      "step": 179
+    },
+    {
+      "epoch": 0.1875,
+      "grad_norm": 1.5155447721481323,
+      "learning_rate": 9.194099206580982e-05,
+      "loss": 0.7253,
+      "step": 180
+    },
+    {
+      "epoch": 0.18854166666666666,
+      "grad_norm": 1.6814624071121216,
+      "learning_rate": 9.185122018007635e-05,
+      "loss": 0.718,
+      "step": 181
+    },
+    {
+      "epoch": 0.18958333333333333,
+      "grad_norm": 1.8656744956970215,
+      "learning_rate": 9.176099539599647e-05,
+      "loss": 0.7188,
+      "step": 182
+    },
+    {
+      "epoch": 0.190625,
+      "grad_norm": 1.6747164726257324,
+      "learning_rate": 9.167031868994924e-05,
+      "loss": 0.7856,
+      "step": 183
+    },
+    {
+      "epoch": 0.19166666666666668,
+      "grad_norm": 1.5895557403564453,
+      "learning_rate": 9.157919104320427e-05,
+      "loss": 0.7989,
+      "step": 184
+    },
+    {
+      "epoch": 0.19270833333333334,
+      "grad_norm": 1.7184622287750244,
+      "learning_rate": 9.148761344191109e-05,
+      "loss": 0.677,
+      "step": 185
+    },
+    {
+      "epoch": 0.19375,
+      "grad_norm": 1.9188364744186401,
+      "learning_rate": 9.139558687708845e-05,
+      "loss": 0.6746,
+      "step": 186
+    },
+    {
+      "epoch": 0.19479166666666667,
+      "grad_norm": 1.7534639835357666,
+      "learning_rate": 9.130311234461362e-05,
+      "loss": 0.6389,
+      "step": 187
+    },
+    {
+      "epoch": 0.19583333333333333,
+      "grad_norm": 2.1047608852386475,
+      "learning_rate": 9.12101908452116e-05,
+      "loss": 0.6245,
+      "step": 188
+    },
+    {
+      "epoch": 0.196875,
+      "grad_norm": 2.1767189502716064,
+      "learning_rate": 9.111682338444431e-05,
+      "loss": 0.7218,
+      "step": 189
+    },
+    {
+      "epoch": 0.19791666666666666,
+      "grad_norm": 2.293118953704834,
+      "learning_rate": 9.102301097269974e-05,
+      "loss": 0.6994,
+      "step": 190
+    },
+    {
+      "epoch": 0.19895833333333332,
+      "grad_norm": 1.8105974197387695,
+      "learning_rate": 9.092875462518091e-05,
+      "loss": 0.6865,
+      "step": 191
+    },
+    {
+      "epoch": 0.2,
+      "grad_norm": 2.2758266925811768,
+      "learning_rate": 9.083405536189498e-05,
+      "loss": 0.5878,
+      "step": 192
+    },
+    {
+      "epoch": 0.20104166666666667,
+      "grad_norm": 2.7756974697113037,
+      "learning_rate": 9.073891420764215e-05,
+      "loss": 0.7338,
+      "step": 193
+    },
+    {
+      "epoch": 0.20208333333333334,
+      "grad_norm": 1.9865261316299438,
+      "learning_rate": 9.064333219200464e-05,
+      "loss": 0.611,
+      "step": 194
+    },
+    {
+      "epoch": 0.203125,
+      "grad_norm": 1.642616629600525,
+      "learning_rate": 9.054731034933549e-05,
+      "loss": 0.5778,
+      "step": 195
+    },
+    {
+      "epoch": 0.20416666666666666,
+      "grad_norm": 1.953162670135498,
+      "learning_rate": 9.045084971874738e-05,
+      "loss": 0.5745,
+      "step": 196
+    },
+    {
+      "epoch": 0.20520833333333333,
+      "grad_norm": 2.0923852920532227,
+      "learning_rate": 9.035395134410139e-05,
+      "loss": 0.5294,
+      "step": 197
+    },
+    {
+      "epoch": 0.20625,
+      "grad_norm": 1.5819649696350098,
+      "learning_rate": 9.025661627399573e-05,
+      "loss": 0.5277,
+      "step": 198
+    },
+    {
+      "epoch": 0.20729166666666668,
+      "grad_norm": 1.8375093936920166,
+      "learning_rate": 9.015884556175431e-05,
+      "loss": 0.4854,
+      "step": 199
+    },
+    {
+      "epoch": 0.20833333333333334,
+      "grad_norm": 2.7594127655029297,
+      "learning_rate": 9.006064026541548e-05,
+      "loss": 0.6309,
+      "step": 200
+    },
+    {
+      "epoch": 0.209375,
+      "grad_norm": 1.5776251554489136,
+      "learning_rate": 8.99620014477204e-05,
+      "loss": 0.8689,
+      "step": 201
+    },
+    {
+      "epoch": 0.21041666666666667,
+      "grad_norm": 1.5562249422073364,
+      "learning_rate": 8.98629301761017e-05,
+      "loss": 0.8683,
+      "step": 202
+    },
+    {
+      "epoch": 0.21145833333333333,
+      "grad_norm": 1.446951985359192,
+      "learning_rate": 8.976342752267186e-05,
+      "loss": 0.8116,
+      "step": 203
+    },
+    {
+      "epoch": 0.2125,
+      "grad_norm": 1.6022385358810425,
+      "learning_rate": 8.966349456421159e-05,
+      "loss": 0.873,
+      "step": 204
+    },
+    {
+      "epoch": 0.21354166666666666,
+      "grad_norm": 1.3617318868637085,
+      "learning_rate": 8.956313238215824e-05,
+      "loss": 0.8467,
+      "step": 205
+    },
+    {
+      "epoch": 0.21458333333333332,
+      "grad_norm": 1.7591134309768677,
+      "learning_rate": 8.946234206259403e-05,
+      "loss": 0.8555,
+      "step": 206
+    },
+    {
+      "epoch": 0.215625,
+      "grad_norm": 1.722296118736267,
+      "learning_rate": 8.93611246962343e-05,
+      "loss": 0.8652,
+      "step": 207
+    },
+    {
+      "epoch": 0.21666666666666667,
+      "grad_norm": 1.5404603481292725,
+      "learning_rate": 8.925948137841579e-05,
+      "loss": 0.8483,
+      "step": 208
+    },
+    {
+      "epoch": 0.21770833333333334,
+      "grad_norm": 1.703816533088684,
+      "learning_rate": 8.915741320908468e-05,
+      "loss": 0.823,
+      "step": 209
+    },
+    {
+      "epoch": 0.21875,
+      "grad_norm": 1.7601414918899536,
+      "learning_rate": 8.905492129278478e-05,
+      "loss": 0.8513,
+      "step": 210
+    },
+    {
+      "epoch": 0.21979166666666666,
+      "grad_norm": 1.7160789966583252,
+      "learning_rate": 8.895200673864547e-05,
+      "loss": 0.819,
+      "step": 211
+    },
+    {
+      "epoch": 0.22083333333333333,
+      "grad_norm": 1.6923937797546387,
+      "learning_rate": 8.884867066036984e-05,
+      "loss": 0.7305,
+      "step": 212
+    },
+    {
+      "epoch": 0.221875,
+      "grad_norm": 1.6672377586364746,
+      "learning_rate": 8.874491417622248e-05,
+      "loss": 0.7434,
+      "step": 213
+    },
+    {
+      "epoch": 0.22291666666666668,
+      "grad_norm": 1.4038190841674805,
+      "learning_rate": 8.86407384090175e-05,
+      "loss": 0.7008,
+      "step": 214
+    },
+    {
+      "epoch": 0.22395833333333334,
+      "grad_norm": 1.7491308450698853,
+      "learning_rate": 8.853614448610631e-05,
+      "loss": 0.7553,
+      "step": 215
+    },
+    {
+      "epoch": 0.225,
+      "grad_norm": 1.5492684841156006,
+      "learning_rate": 8.843113353936549e-05,
+      "loss": 0.7248,
+      "step": 216
+    },
+    {
+      "epoch": 0.22604166666666667,
+      "grad_norm": 1.304273247718811,
+      "learning_rate": 8.832570670518444e-05,
+      "loss": 0.7486,
+      "step": 217
+    },
+    {
+      "epoch": 0.22708333333333333,
+      "grad_norm": 1.553187370300293,
+      "learning_rate": 8.82198651244532e-05,
+      "loss": 0.7149,
+      "step": 218
+    },
+    {
+      "epoch": 0.228125,
+      "grad_norm": 1.7741554975509644,
+      "learning_rate": 8.811360994254996e-05,
+      "loss": 0.6769,
+      "step": 219
+    },
+    {
+      "epoch": 0.22916666666666666,
+      "grad_norm": 1.582899570465088,
+      "learning_rate": 8.800694230932884e-05,
+      "loss": 0.771,
+      "step": 220
+    },
+    {
+      "epoch": 0.23020833333333332,
+      "grad_norm": 1.4296000003814697,
+      "learning_rate": 8.78998633791073e-05,
+      "loss": 0.6962,
+      "step": 221
+    },
+    {
+      "epoch": 0.23125,
+      "grad_norm": 1.7563673257827759,
+      "learning_rate": 8.77923743106537e-05,
+      "loss": 0.733,
+      "step": 222
+    },
+    {
+      "epoch": 0.23229166666666667,
+      "grad_norm": 1.4017952680587769,
+      "learning_rate": 8.768447626717481e-05,
+      "loss": 0.6637,
+      "step": 223
+    },
+    {
+      "epoch": 0.23333333333333334,
+      "grad_norm": 1.4028128385543823,
+      "learning_rate": 8.757617041630313e-05,
+      "loss": 0.6207,
+      "step": 224
+    },
+    {
+      "epoch": 0.234375,
+      "grad_norm": 1.2909876108169556,
+      "learning_rate": 8.74674579300843e-05,
+      "loss": 0.6572,
+      "step": 225
+    },
+    {
+      "epoch": 0.23541666666666666,
+      "grad_norm": 1.7121574878692627,
+      "learning_rate": 8.735833998496444e-05,
+      "loss": 0.7566,
+      "step": 226
+    },
+    {
+      "epoch": 0.23645833333333333,
+      "grad_norm": 1.7302300930023193,
+      "learning_rate": 8.72488177617774e-05,
+      "loss": 0.8499,
+      "step": 227
+    },
+    {
+      "epoch": 0.2375,
+      "grad_norm": 1.895605206489563,
+      "learning_rate": 8.713889244573197e-05,
+      "loss": 0.6588,
+      "step": 228
+    },
+    {
+      "epoch": 0.23854166666666668,
+      "grad_norm": 1.7216819524765015,
+      "learning_rate": 8.702856522639902e-05,
+      "loss": 0.7614,
+      "step": 229
+    },
+    {
+      "epoch": 0.23958333333333334,
+      "grad_norm": 1.903328537940979,
+      "learning_rate": 8.691783729769874e-05,
+      "loss": 0.7812,
+      "step": 230
+    },
+    {
+      "epoch": 0.240625,
+      "grad_norm": 1.9232820272445679,
+      "learning_rate": 8.680670985788757e-05,
+      "loss": 0.7435,
+      "step": 231
+    },
+    {
+      "epoch": 0.24166666666666667,
+      "grad_norm": 1.4913100004196167,
+      "learning_rate": 8.669518410954541e-05,
+      "loss": 0.6241,
+      "step": 232
+    },
+    {
+      "epoch": 0.24270833333333333,
+      "grad_norm": 1.3420971632003784,
+      "learning_rate": 8.658326125956237e-05,
+      "loss": 0.657,
+      "step": 233
+    },
+    {
+      "epoch": 0.24375,
+      "grad_norm": 1.8332111835479736,
+      "learning_rate": 8.647094251912601e-05,
+      "loss": 0.7692,
+      "step": 234
+    },
+    {
+      "epoch": 0.24479166666666666,
+      "grad_norm": 1.355997085571289,
+      "learning_rate": 8.635822910370792e-05,
+      "loss": 0.583,
+      "step": 235
+    },
+    {
+      "epoch": 0.24583333333333332,
+      "grad_norm": 1.640322208404541,
+      "learning_rate": 8.624512223305081e-05,
+      "loss": 0.6874,
+      "step": 236
+    },
+    {
+      "epoch": 0.246875,
+      "grad_norm": 1.8437845706939697,
+      "learning_rate": 8.613162313115519e-05,
+      "loss": 0.7437,
+      "step": 237
+    },
+    {
+      "epoch": 0.24791666666666667,
+      "grad_norm": 2.5125062465667725,
+      "learning_rate": 8.601773302626612e-05,
+      "loss": 0.676,
+      "step": 238
+    },
+    {
+      "epoch": 0.24895833333333334,
+      "grad_norm": 1.6971981525421143,
+      "learning_rate": 8.590345315086e-05,
+      "loss": 0.6431,
+      "step": 239
+    },
+    {
+      "epoch": 0.25,
+      "grad_norm": 2.3452751636505127,
+      "learning_rate": 8.578878474163115e-05,
+      "loss": 0.7476,
+      "step": 240
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 960,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 240,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.531900159604818e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0bf2b98c902c6e1a7b13795b92f73fed99df6daefaacfc0107a7bb5d655508c6
+size 6776