Training in progress, step 100, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: defog/sqlcoder-7b-2
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "defog/sqlcoder-7b-2",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_dropout": 0.04,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "gate_proj",
+    "q_proj",
+    "k_proj",
+    "v_proj",
+    "down_proj",
+    "o_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5fc4f88fd2b3f41fae9564d4b9b3fa8bb238457eb0bd0fe66a0cedeed7b52b51
+size 639691872

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ffa11bd55ddc1dce631eb6992e8fcd26c5f8407f4a59856297bb7d3ad1dbb54c
+size 325339796

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d431783e0cb3001229be5ffd551139742fe88f606cf488bd419a2a9e2a7a8b22
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:58847d2cf3d6eab1188d4f39ee3fdfc662175af61303d8d204b9658b42dc6fd4
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,40 @@
+{
+  "additional_special_tokens": [
+    "▁<PRE>",
+    "▁<MID>",
+    "▁<SUF>",
+    "▁<EOT>",
+    "▁<PRE>",
+    "▁<MID>",
+    "▁<SUF>",
+    "▁<EOT>"
+  ],
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45ccb9c8b6b561889acea59191d66986d314e7cbd6a78abc6e49b139ca91c1e6
+size 500058

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,88 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32007": {
+      "content": "▁<PRE>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32008": {
+      "content": "▁<SUF>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32009": {
+      "content": "▁<MID>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32010": {
+      "content": "▁<EOT>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "▁<PRE>",
+    "▁<MID>",
+    "▁<SUF>",
+    "▁<EOT>",
+    "▁<PRE>",
+    "▁<MID>",
+    "▁<SUF>",
+    "▁<EOT>"
+  ],
+  "bos_token": "<s>",
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "eot_token": "▁<EOT>",
+  "fill_token": "<FILL_ME>",
+  "legacy": null,
+  "middle_token": "▁<MID>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "prefix_token": "▁<PRE>",
+  "sp_model_kwargs": {},
+  "suffix_token": "▁<SUF>",
+  "tokenizer_class": "CodeLlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,758 @@
+{
+  "best_metric": 1.0371321439743042,
+  "best_model_checkpoint": "miner_id_24/checkpoint-100",
+  "epoch": 0.34305317324185247,
+  "eval_steps": 100,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.003430531732418525,
+      "grad_norm": 0.27824944257736206,
+      "learning_rate": 1.4e-06,
+      "loss": 1.2807,
+      "step": 1
+    },
+    {
+      "epoch": 0.003430531732418525,
+      "eval_loss": 1.3408838510513306,
+      "eval_runtime": 36.4521,
+      "eval_samples_per_second": 13.47,
+      "eval_steps_per_second": 3.374,
+      "step": 1
+    },
+    {
+      "epoch": 0.00686106346483705,
+      "grad_norm": 0.2794307768344879,
+      "learning_rate": 2.8e-06,
+      "loss": 1.3341,
+      "step": 2
+    },
+    {
+      "epoch": 0.010291595197255575,
+      "grad_norm": 0.29869186878204346,
+      "learning_rate": 4.2e-06,
+      "loss": 1.2984,
+      "step": 3
+    },
+    {
+      "epoch": 0.0137221269296741,
+      "grad_norm": 0.30163782835006714,
+      "learning_rate": 5.6e-06,
+      "loss": 1.2416,
+      "step": 4
+    },
+    {
+      "epoch": 0.017152658662092625,
+      "grad_norm": 0.31913354992866516,
+      "learning_rate": 7e-06,
+      "loss": 1.3119,
+      "step": 5
+    },
+    {
+      "epoch": 0.02058319039451115,
+      "grad_norm": 0.31287121772766113,
+      "learning_rate": 8.4e-06,
+      "loss": 1.3023,
+      "step": 6
+    },
+    {
+      "epoch": 0.024013722126929673,
+      "grad_norm": 0.3440784513950348,
+      "learning_rate": 9.8e-06,
+      "loss": 1.3269,
+      "step": 7
+    },
+    {
+      "epoch": 0.0274442538593482,
+      "grad_norm": 0.30914536118507385,
+      "learning_rate": 1.12e-05,
+      "loss": 1.2841,
+      "step": 8
+    },
+    {
+      "epoch": 0.030874785591766724,
+      "grad_norm": 0.32246437668800354,
+      "learning_rate": 1.2599999999999998e-05,
+      "loss": 1.2826,
+      "step": 9
+    },
+    {
+      "epoch": 0.03430531732418525,
+      "grad_norm": 0.35504838824272156,
+      "learning_rate": 1.4e-05,
+      "loss": 1.2992,
+      "step": 10
+    },
+    {
+      "epoch": 0.03773584905660377,
+      "grad_norm": 0.3929505944252014,
+      "learning_rate": 1.5399999999999998e-05,
+      "loss": 1.3377,
+      "step": 11
+    },
+    {
+      "epoch": 0.0411663807890223,
+      "grad_norm": 0.3200545907020569,
+      "learning_rate": 1.68e-05,
+      "loss": 1.2655,
+      "step": 12
+    },
+    {
+      "epoch": 0.044596912521440824,
+      "grad_norm": 0.3272934854030609,
+      "learning_rate": 1.82e-05,
+      "loss": 1.2907,
+      "step": 13
+    },
+    {
+      "epoch": 0.048027444253859346,
+      "grad_norm": 0.3309055268764496,
+      "learning_rate": 1.96e-05,
+      "loss": 1.2089,
+      "step": 14
+    },
+    {
+      "epoch": 0.051457975986277875,
+      "grad_norm": 0.3068203628063202,
+      "learning_rate": 2.1e-05,
+      "loss": 1.2526,
+      "step": 15
+    },
+    {
+      "epoch": 0.0548885077186964,
+      "grad_norm": 0.30473190546035767,
+      "learning_rate": 2.24e-05,
+      "loss": 1.2913,
+      "step": 16
+    },
+    {
+      "epoch": 0.058319039451114926,
+      "grad_norm": 0.31642964482307434,
+      "learning_rate": 2.38e-05,
+      "loss": 1.2156,
+      "step": 17
+    },
+    {
+      "epoch": 0.06174957118353345,
+      "grad_norm": 0.27499595284461975,
+      "learning_rate": 2.5199999999999996e-05,
+      "loss": 1.2661,
+      "step": 18
+    },
+    {
+      "epoch": 0.06518010291595197,
+      "grad_norm": 0.26945629715919495,
+      "learning_rate": 2.66e-05,
+      "loss": 1.2501,
+      "step": 19
+    },
+    {
+      "epoch": 0.0686106346483705,
+      "grad_norm": 0.2821623682975769,
+      "learning_rate": 2.8e-05,
+      "loss": 1.2411,
+      "step": 20
+    },
+    {
+      "epoch": 0.07204116638078903,
+      "grad_norm": 0.3003956079483032,
+      "learning_rate": 2.9399999999999996e-05,
+      "loss": 1.268,
+      "step": 21
+    },
+    {
+      "epoch": 0.07547169811320754,
+      "grad_norm": 0.35937100648880005,
+      "learning_rate": 3.0799999999999996e-05,
+      "loss": 1.2288,
+      "step": 22
+    },
+    {
+      "epoch": 0.07890222984562607,
+      "grad_norm": 0.3568710386753082,
+      "learning_rate": 3.22e-05,
+      "loss": 1.2481,
+      "step": 23
+    },
+    {
+      "epoch": 0.0823327615780446,
+      "grad_norm": 0.37753623723983765,
+      "learning_rate": 3.36e-05,
+      "loss": 1.3478,
+      "step": 24
+    },
+    {
+      "epoch": 0.08576329331046312,
+      "grad_norm": 0.3411467671394348,
+      "learning_rate": 3.5e-05,
+      "loss": 1.2498,
+      "step": 25
+    },
+    {
+      "epoch": 0.08919382504288165,
+      "grad_norm": 0.3256891667842865,
+      "learning_rate": 3.64e-05,
+      "loss": 1.2086,
+      "step": 26
+    },
+    {
+      "epoch": 0.09262435677530018,
+      "grad_norm": 0.31805384159088135,
+      "learning_rate": 3.78e-05,
+      "loss": 1.2715,
+      "step": 27
+    },
+    {
+      "epoch": 0.09605488850771869,
+      "grad_norm": 0.31195560097694397,
+      "learning_rate": 3.92e-05,
+      "loss": 1.1789,
+      "step": 28
+    },
+    {
+      "epoch": 0.09948542024013722,
+      "grad_norm": 0.3224494159221649,
+      "learning_rate": 4.059999999999999e-05,
+      "loss": 1.176,
+      "step": 29
+    },
+    {
+      "epoch": 0.10291595197255575,
+      "grad_norm": 0.32157495617866516,
+      "learning_rate": 4.2e-05,
+      "loss": 1.1217,
+      "step": 30
+    },
+    {
+      "epoch": 0.10634648370497427,
+      "grad_norm": 0.3384961783885956,
+      "learning_rate": 4.34e-05,
+      "loss": 1.1945,
+      "step": 31
+    },
+    {
+      "epoch": 0.1097770154373928,
+      "grad_norm": 0.3938997983932495,
+      "learning_rate": 4.48e-05,
+      "loss": 1.1137,
+      "step": 32
+    },
+    {
+      "epoch": 0.11320754716981132,
+      "grad_norm": 0.37928083539009094,
+      "learning_rate": 4.62e-05,
+      "loss": 1.1206,
+      "step": 33
+    },
+    {
+      "epoch": 0.11663807890222985,
+      "grad_norm": 0.4108302891254425,
+      "learning_rate": 4.76e-05,
+      "loss": 1.177,
+      "step": 34
+    },
+    {
+      "epoch": 0.12006861063464837,
+      "grad_norm": 0.40350550413131714,
+      "learning_rate": 4.899999999999999e-05,
+      "loss": 1.1136,
+      "step": 35
+    },
+    {
+      "epoch": 0.1234991423670669,
+      "grad_norm": 0.44681158661842346,
+      "learning_rate": 5.039999999999999e-05,
+      "loss": 1.1296,
+      "step": 36
+    },
+    {
+      "epoch": 0.1269296740994854,
+      "grad_norm": 0.4281773269176483,
+      "learning_rate": 5.179999999999999e-05,
+      "loss": 1.1422,
+      "step": 37
+    },
+    {
+      "epoch": 0.13036020583190394,
+      "grad_norm": 0.4204246997833252,
+      "learning_rate": 5.32e-05,
+      "loss": 1.1203,
+      "step": 38
+    },
+    {
+      "epoch": 0.13379073756432247,
+      "grad_norm": 0.495144248008728,
+      "learning_rate": 5.46e-05,
+      "loss": 1.1492,
+      "step": 39
+    },
+    {
+      "epoch": 0.137221269296741,
+      "grad_norm": 0.38059696555137634,
+      "learning_rate": 5.6e-05,
+      "loss": 1.088,
+      "step": 40
+    },
+    {
+      "epoch": 0.14065180102915953,
+      "grad_norm": 0.44585326313972473,
+      "learning_rate": 5.739999999999999e-05,
+      "loss": 1.104,
+      "step": 41
+    },
+    {
+      "epoch": 0.14408233276157806,
+      "grad_norm": 0.45643752813339233,
+      "learning_rate": 5.879999999999999e-05,
+      "loss": 1.13,
+      "step": 42
+    },
+    {
+      "epoch": 0.14751286449399656,
+      "grad_norm": 0.4543881118297577,
+      "learning_rate": 6.019999999999999e-05,
+      "loss": 1.0996,
+      "step": 43
+    },
+    {
+      "epoch": 0.1509433962264151,
+      "grad_norm": 0.4942781627178192,
+      "learning_rate": 6.159999999999999e-05,
+      "loss": 1.1354,
+      "step": 44
+    },
+    {
+      "epoch": 0.15437392795883362,
+      "grad_norm": 0.5626402497291565,
+      "learning_rate": 6.3e-05,
+      "loss": 1.1204,
+      "step": 45
+    },
+    {
+      "epoch": 0.15780445969125215,
+      "grad_norm": 0.5675943493843079,
+      "learning_rate": 6.44e-05,
+      "loss": 1.1008,
+      "step": 46
+    },
+    {
+      "epoch": 0.16123499142367068,
+      "grad_norm": 0.5674481987953186,
+      "learning_rate": 6.579999999999999e-05,
+      "loss": 1.0523,
+      "step": 47
+    },
+    {
+      "epoch": 0.1646655231560892,
+      "grad_norm": 0.6175776720046997,
+      "learning_rate": 6.72e-05,
+      "loss": 1.1834,
+      "step": 48
+    },
+    {
+      "epoch": 0.1680960548885077,
+      "grad_norm": 0.648908257484436,
+      "learning_rate": 6.859999999999999e-05,
+      "loss": 1.0875,
+      "step": 49
+    },
+    {
+      "epoch": 0.17152658662092624,
+      "grad_norm": 0.8205939531326294,
+      "learning_rate": 7e-05,
+      "loss": 1.2005,
+      "step": 50
+    },
+    {
+      "epoch": 0.17495711835334476,
+      "grad_norm": 0.617884635925293,
+      "learning_rate": 6.999859006598048e-05,
+      "loss": 1.2037,
+      "step": 51
+    },
+    {
+      "epoch": 0.1783876500857633,
+      "grad_norm": 0.5661861896514893,
+      "learning_rate": 6.999436037751703e-05,
+      "loss": 1.2372,
+      "step": 52
+    },
+    {
+      "epoch": 0.18181818181818182,
+      "grad_norm": 0.42782628536224365,
+      "learning_rate": 6.998731127538574e-05,
+      "loss": 1.1914,
+      "step": 53
+    },
+    {
+      "epoch": 0.18524871355060035,
+      "grad_norm": 0.3282926678657532,
+      "learning_rate": 6.997744332751624e-05,
+      "loss": 1.1677,
+      "step": 54
+    },
+    {
+      "epoch": 0.18867924528301888,
+      "grad_norm": 0.2774999737739563,
+      "learning_rate": 6.996475732894601e-05,
+      "loss": 1.166,
+      "step": 55
+    },
+    {
+      "epoch": 0.19210977701543738,
+      "grad_norm": 0.2545096278190613,
+      "learning_rate": 6.994925430175622e-05,
+      "loss": 1.1396,
+      "step": 56
+    },
+    {
+      "epoch": 0.1955403087478559,
+      "grad_norm": 0.2631377577781677,
+      "learning_rate": 6.99309354949895e-05,
+      "loss": 1.0787,
+      "step": 57
+    },
+    {
+      "epoch": 0.19897084048027444,
+      "grad_norm": 0.2663518488407135,
+      "learning_rate": 6.99098023845492e-05,
+      "loss": 1.1618,
+      "step": 58
+    },
+    {
+      "epoch": 0.20240137221269297,
+      "grad_norm": 0.2729957699775696,
+      "learning_rate": 6.988585667308051e-05,
+      "loss": 1.141,
+      "step": 59
+    },
+    {
+      "epoch": 0.2058319039451115,
+      "grad_norm": 0.2546933889389038,
+      "learning_rate": 6.985910028983336e-05,
+      "loss": 1.1057,
+      "step": 60
+    },
+    {
+      "epoch": 0.20926243567753003,
+      "grad_norm": 0.283761203289032,
+      "learning_rate": 6.982953539050688e-05,
+      "loss": 1.0742,
+      "step": 61
+    },
+    {
+      "epoch": 0.21269296740994853,
+      "grad_norm": 0.27833765745162964,
+      "learning_rate": 6.97971643570758e-05,
+      "loss": 1.18,
+      "step": 62
+    },
+    {
+      "epoch": 0.21612349914236706,
+      "grad_norm": 0.2722010314464569,
+      "learning_rate": 6.976198979759845e-05,
+      "loss": 1.115,
+      "step": 63
+    },
+    {
+      "epoch": 0.2195540308747856,
+      "grad_norm": 0.279540091753006,
+      "learning_rate": 6.972401454600672e-05,
+      "loss": 1.0544,
+      "step": 64
+    },
+    {
+      "epoch": 0.22298456260720412,
+      "grad_norm": 0.2840425372123718,
+      "learning_rate": 6.968324166187771e-05,
+      "loss": 1.0367,
+      "step": 65
+    },
+    {
+      "epoch": 0.22641509433962265,
+      "grad_norm": 0.3138042390346527,
+      "learning_rate": 6.963967443018721e-05,
+      "loss": 0.973,
+      "step": 66
+    },
+    {
+      "epoch": 0.22984562607204118,
+      "grad_norm": 0.29737910628318787,
+      "learning_rate": 6.959331636104508e-05,
+      "loss": 1.0418,
+      "step": 67
+    },
+    {
+      "epoch": 0.2332761578044597,
+      "grad_norm": 0.3060307502746582,
+      "learning_rate": 6.954417118941235e-05,
+      "loss": 1.0985,
+      "step": 68
+    },
+    {
+      "epoch": 0.2367066895368782,
+      "grad_norm": 0.3002816438674927,
+      "learning_rate": 6.949224287480042e-05,
+      "loss": 1.0626,
+      "step": 69
+    },
+    {
+      "epoch": 0.24013722126929674,
+      "grad_norm": 0.32159680128097534,
+      "learning_rate": 6.943753560095204e-05,
+      "loss": 1.0734,
+      "step": 70
+    },
+    {
+      "epoch": 0.24356775300171526,
+      "grad_norm": 0.3148500323295593,
+      "learning_rate": 6.93800537755041e-05,
+      "loss": 1.1352,
+      "step": 71
+    },
+    {
+      "epoch": 0.2469982847341338,
+      "grad_norm": 0.3088507354259491,
+      "learning_rate": 6.93198020296327e-05,
+      "loss": 0.9925,
+      "step": 72
+    },
+    {
+      "epoch": 0.2504288164665523,
+      "grad_norm": 0.3257371187210083,
+      "learning_rate": 6.92567852176799e-05,
+      "loss": 0.9782,
+      "step": 73
+    },
+    {
+      "epoch": 0.2538593481989708,
+      "grad_norm": 0.31071141362190247,
+      "learning_rate": 6.919100841676266e-05,
+      "loss": 1.0096,
+      "step": 74
+    },
+    {
+      "epoch": 0.25728987993138935,
+      "grad_norm": 0.34918615221977234,
+      "learning_rate": 6.912247692636382e-05,
+      "loss": 1.0163,
+      "step": 75
+    },
+    {
+      "epoch": 0.2607204116638079,
+      "grad_norm": 0.3556749224662781,
+      "learning_rate": 6.905119626790507e-05,
+      "loss": 1.0961,
+      "step": 76
+    },
+    {
+      "epoch": 0.2641509433962264,
+      "grad_norm": 0.3610279858112335,
+      "learning_rate": 6.897717218430212e-05,
+      "loss": 1.0246,
+      "step": 77
+    },
+    {
+      "epoch": 0.26758147512864494,
+      "grad_norm": 0.3471314609050751,
+      "learning_rate": 6.890041063950208e-05,
+      "loss": 1.0174,
+      "step": 78
+    },
+    {
+      "epoch": 0.27101200686106347,
+      "grad_norm": 0.3771928548812866,
+      "learning_rate": 6.882091781800283e-05,
+      "loss": 1.0467,
+      "step": 79
+    },
+    {
+      "epoch": 0.274442538593482,
+      "grad_norm": 0.3704926669597626,
+      "learning_rate": 6.873870012435486e-05,
+      "loss": 1.0205,
+      "step": 80
+    },
+    {
+      "epoch": 0.27787307032590053,
+      "grad_norm": 0.40470531582832336,
+      "learning_rate": 6.86537641826452e-05,
+      "loss": 1.0653,
+      "step": 81
+    },
+    {
+      "epoch": 0.28130360205831906,
+      "grad_norm": 0.39418941736221313,
+      "learning_rate": 6.856611683596379e-05,
+      "loss": 0.994,
+      "step": 82
+    },
+    {
+      "epoch": 0.2847341337907376,
+      "grad_norm": 0.4157142639160156,
+      "learning_rate": 6.84757651458521e-05,
+      "loss": 0.9998,
+      "step": 83
+    },
+    {
+      "epoch": 0.2881646655231561,
+      "grad_norm": 0.42235854268074036,
+      "learning_rate": 6.838271639173422e-05,
+      "loss": 1.0191,
+      "step": 84
+    },
+    {
+      "epoch": 0.2915951972555746,
+      "grad_norm": 0.40815040469169617,
+      "learning_rate": 6.828697807033038e-05,
+      "loss": 0.9527,
+      "step": 85
+    },
+    {
+      "epoch": 0.2950257289879931,
+      "grad_norm": 0.440251886844635,
+      "learning_rate": 6.818855789505291e-05,
+      "loss": 1.0031,
+      "step": 86
+    },
+    {
+      "epoch": 0.29845626072041165,
+      "grad_norm": 0.43153345584869385,
+      "learning_rate": 6.80874637953849e-05,
+      "loss": 0.9926,
+      "step": 87
+    },
+    {
+      "epoch": 0.3018867924528302,
+      "grad_norm": 0.4722265899181366,
+      "learning_rate": 6.798370391624122e-05,
+      "loss": 1.0324,
+      "step": 88
+    },
+    {
+      "epoch": 0.3053173241852487,
+      "grad_norm": 0.46640855073928833,
+      "learning_rate": 6.787728661731232e-05,
+      "loss": 1.052,
+      "step": 89
+    },
+    {
+      "epoch": 0.30874785591766724,
+      "grad_norm": 0.4882723093032837,
+      "learning_rate": 6.77682204723908e-05,
+      "loss": 1.1014,
+      "step": 90
+    },
+    {
+      "epoch": 0.31217838765008576,
+      "grad_norm": 0.46555763483047485,
+      "learning_rate": 6.765651426868054e-05,
+      "loss": 0.9989,
+      "step": 91
+    },
+    {
+      "epoch": 0.3156089193825043,
+      "grad_norm": 0.5026487112045288,
+      "learning_rate": 6.75421770060888e-05,
+      "loss": 1.0579,
+      "step": 92
+    },
+    {
+      "epoch": 0.3190394511149228,
+      "grad_norm": 0.47720423340797424,
+      "learning_rate": 6.742521789650105e-05,
+      "loss": 1.0396,
+      "step": 93
+    },
+    {
+      "epoch": 0.32246998284734135,
+      "grad_norm": 0.53495192527771,
+      "learning_rate": 6.73056463630389e-05,
+      "loss": 0.9513,
+      "step": 94
+    },
+    {
+      "epoch": 0.3259005145797599,
+      "grad_norm": 0.5590450167655945,
+      "learning_rate": 6.718347203930078e-05,
+      "loss": 1.051,
+      "step": 95
+    },
+    {
+      "epoch": 0.3293310463121784,
+      "grad_norm": 0.5361358523368835,
+      "learning_rate": 6.705870476858582e-05,
+      "loss": 1.01,
+      "step": 96
+    },
+    {
+      "epoch": 0.33276157804459694,
+      "grad_norm": 0.5490963459014893,
+      "learning_rate": 6.69313546031009e-05,
+      "loss": 0.9192,
+      "step": 97
+    },
+    {
+      "epoch": 0.3361921097770154,
+      "grad_norm": 0.5866110920906067,
+      "learning_rate": 6.68014318031506e-05,
+      "loss": 0.982,
+      "step": 98
+    },
+    {
+      "epoch": 0.33962264150943394,
+      "grad_norm": 0.7043346762657166,
+      "learning_rate": 6.666894683631068e-05,
+      "loss": 1.0057,
+      "step": 99
+    },
+    {
+      "epoch": 0.34305317324185247,
+      "grad_norm": 0.8684700727462769,
+      "learning_rate": 6.653391037658466e-05,
+      "loss": 1.0552,
+      "step": 100
+    },
+    {
+      "epoch": 0.34305317324185247,
+      "eval_loss": 1.0371321439743042,
+      "eval_runtime": 36.9532,
+      "eval_samples_per_second": 13.287,
+      "eval_steps_per_second": 3.329,
+      "step": 100
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 400,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "EarlyStoppingCallback": {
+      "args": {
+        "early_stopping_patience": 4,
+        "early_stopping_threshold": 0.0
+      },
+      "attributes": {
+        "early_stopping_patience_counter": 0
+      }
+    },
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.3804027338817536e+17,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b756e175c82b10a996c73f53ebb69a3128c88b0a06143687eb58db947da126b6
+size 6840