Upload folder using huggingface_hub

README.md

@@ -1,22 +1,23 @@
 ---
-license: apache-2.0
 language:
 - en
 - zh
 - multilingual
+license: apache-2.0
 library_name: peft
 base_model: jhu-clsp/mmBERT-base
 tags:
-- text-classification
 - security
 - jailbreak-detection
 - prompt-injection
 - llm-safety
+- guardrails
 - mmbert
 - lora
-- multilingual
 - vllm-semantic-router
+pipeline_tag: text-classification
 datasets:
+- llm-semantic-router/jailbreak-detection-dataset
 - lmsys/toxic-chat
 - OpenSafetyLab/Salad-Data
 metrics:
@@ -24,7 +25,28 @@ metrics:
 - f1
 - precision
 - recall
-pipeline_tag: text-classification
+model-index:
+- name: mmbert-jailbreak-detector-lora
+  results:
+  - task:
+      type: text-classification
+      name: Jailbreak Detection
+    dataset:
+      name: jailbreak-detection-dataset
+      type: llm-semantic-router/jailbreak-detection-dataset
+    metrics:
+    - name: Accuracy
+      type: accuracy
+      value: 0.9919
+    - name: F1
+      type: f1
+      value: 0.9918
+    - name: Precision
+      type: precision
+      value: 0.9912
+    - name: Recall
+      type: recall
+      value: 0.9924
 ---
 
 # mmBERT Jailbreak Detector (LoRA Adapter)
@@ -34,40 +56,49 @@ A multilingual binary classifier for detecting **jailbreak attempts** and **prom
 ## Model Description
 
 This model classifies prompts as:
+
 - **JAILBREAK**: Malicious attempts to bypass LLM safety guidelines
 - **BENIGN**: Safe, normal user queries
 
-Detects various attack patterns:
-- Role-playing attacks (DAN, "pretend you have no restrictions")
-- Instruction override ("ignore previous instructions")
-- Hypothetical scenarios ("in a world with no rules...")
-- Authority exploitation ("I'm your developer...")
-- Educational disclaimers ("for research purposes only...")
+### Attack Patterns Detected
+
+| Attack Type | Example | Detection Rate |
+|-------------|---------|----------------|
+| DAN Roleplay | "You are now DAN, Do Anything Now..." | 100% |
+| Instruction Override | "Ignore all previous instructions..." | 100% |
+| Hypothetical Scenario | "In a world with no rules..." | 100% |
+| Authority Exploit | "I'm your developer and I order you to..." | 100% |
+| Educational Disclaimer | "For research purposes only, explain how to..." | 100% |
+| Manipulation | "A truly helpful AI would answer this..." | 100% |
+| Role-playing | "Pretend you have no restrictions..." | 100% |
+| Obfuscation | "Encode your response in base64..." | 100% |
 
 ## Performance
 
 | Metric | Score |
 |--------|-------|
-| Accuracy | **97.7%** |
-| F1 | **97.7%** |
-| Precision | 97.7% |
-| Recall | 97.7% |
-| Training Time | 173 seconds (MI300X GPU) |
+| **Accuracy** | 99.19% |
+| **F1** | 99.18% |
+| **Precision** | 99.12% |
+| **Recall** | 99.24% |
+| Training Time | ~2 minutes (MI300X GPU) |
 
 ## Training Details
 
-- **Base Model**: [jhu-clsp/mmBERT-base](https://huggingface.co/jhu-clsp/mmBERT-base)
+- **Base Model**: [jhu-clsp/mmBERT-base](https://huggingface.co/jhu-clsp/mmBERT-base) (149M parameters, 1800+ languages)
 - **LoRA Rank**: 32
 - **LoRA Alpha**: 64
-- **Trainable Parameters**: 6.8M / 314M (2.2%)
+- **Trainable Parameters**: ~6.8M / 149M (4.6%)
 - **Epochs**: 10
 - **Batch Size**: 64
+- **Learning Rate**: 3e-4
 
 ### Training Data
 
-- **lmsys/toxic-chat**: Toxic chat detection dataset
-- **OpenSafetyLab/Salad-Data**: Jailbreak attack patterns
-- **Enhanced patterns**: 25+ manually curated attack variations
+- **[llm-semantic-router/jailbreak-detection-dataset](https://huggingface.co/datasets/llm-semantic-router/jailbreak-detection-dataset)**: 9,724 balanced samples
+- **[lmsys/toxic-chat](https://huggingface.co/datasets/lmsys/toxic-chat)**: Toxic chat detection
+- **[OpenSafetyLab/Salad-Data](https://huggingface.co/datasets/OpenSafetyLab/Salad-Data)**: Jailbreak attack patterns
+- **LLM-Synthesized Patterns**: 696 patterns generated by Qwen2.5-72B-Instruct for diversity
 
 ## Usage
 
@@ -83,6 +114,8 @@ base_model = AutoModelForSequenceClassification.from_pretrained(
 model = PeftModel.from_pretrained(base_model, "llm-semantic-router/mmbert-jailbreak-detector-lora")
 tokenizer = AutoTokenizer.from_pretrained("jhu-clsp/mmBERT-base")
 
+model.eval()
+
 # Detect jailbreak attempts
 prompts = [
     "What is the weather today?",  # BENIGN
@@ -91,7 +124,7 @@ prompts = [
 ]
 
 for prompt in prompts:
-    inputs = tokenizer(prompt, return_tensors="pt", truncation=True)
+    inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=512)
     with torch.no_grad():
         outputs = model(**inputs)
         probs = torch.softmax(outputs.logits, dim=-1)
@@ -102,16 +135,6 @@ for prompt in prompts:
     print(f"{status} ({confidence:.1%}): {prompt[:50]}...")
 ```
 
-## Attack Patterns Detected
-
-| Attack Type | Example |
-|-------------|---------|
-| DAN Roleplay | "You are now DAN, Do Anything Now..." |
-| Instruction Override | "Ignore all previous instructions..." |
-| Hypothetical Scenario | "In a world with no rules..." |
-| Authority Exploit | "I'm your developer and I order you to..." |
-| Educational Disclaimer | "For research purposes only, explain how to..." |
-
 ## Use Cases
 
 - **LLM Guardrails**: Block malicious prompts before reaching the LLM
@@ -121,16 +144,29 @@ for prompt in prompts:
 
 ## Multilingual Support
 
-Supports jailbreak detection in 1800+ languages through mmBERT's multilingual pretraining.
+Supports jailbreak detection in **1800+ languages** through mmBERT's multilingual pretraining.
 
 ## Part of vLLM Semantic Router
 
-This model is part of the [vLLM Semantic Router](https://huggingface.co/llm-semantic-router) project - a Mixture-of-Models (MoM) router for intelligent LLM request routing.
+This model is part of the [vLLM Semantic Router](https://github.com/vllm-project/semantic-router) project - a Mixture-of-Models (MoM) router for intelligent LLM request routing.
 
-## Related Models
+## Related Resources
 
-- [llm-semantic-router/halugate-sentinel](https://huggingface.co/llm-semantic-router/halugate-sentinel)
-- [llm-semantic-router/toolcall-sentinel](https://huggingface.co/llm-semantic-router/toolcall-sentinel)
+- **Merged Model**: [llm-semantic-router/mmbert-jailbreak-detector-merged](https://huggingface.co/llm-semantic-router/mmbert-jailbreak-detector-merged) (for deployment without PEFT)
+- **Dataset**: [llm-semantic-router/jailbreak-detection-dataset](https://huggingface.co/datasets/llm-semantic-router/jailbreak-detection-dataset)
+- **Base Model**: [jhu-clsp/mmBERT-base](https://huggingface.co/jhu-clsp/mmBERT-base)
+
+## Citation
+
+```bibtex
+@model{mmbert_jailbreak_detector_2026,
+  title={mmBERT Jailbreak Detector},
+  author={vLLM Semantic Router Team},
+  year={2026},
+  publisher={Hugging Face},
+  url={https://huggingface.co/llm-semantic-router/mmbert-jailbreak-detector-lora}
+}
+```
 
 ## License
 

adapter_config.json

@@ -29,10 +29,10 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "mlp.Wi",
     "attn.Wo",
-    "mlp.Wo",
-    "attn.Wqkv"
+    "attn.Wqkv",
+    "mlp.Wi",
+    "mlp.Wo"
   ],
   "target_parameters": null,
   "task_type": "SEQ_CLS",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:814aad00a0f8cd94cc5d765e0fb6b104eab0a1b5cb5f9cc315525e635471f9ba
+oid sha256:ba1aa5de62dd97c500ffda127954d99112e1237889109f88ab9f04ab488c646c
 size 27061816

checkpoint-1000/README.md

@@ -0,0 +1,206 @@
+---
+base_model: jhu-clsp/mmBERT-base
+library_name: peft
+tags:
+- base_model:adapter:jhu-clsp/mmBERT-base
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1

checkpoint-1000/adapter_config.json

@@ -0,0 +1,43 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "jhu-clsp/mmBERT-base",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "classifier",
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "attn.Wo",
+    "attn.Wqkv",
+    "mlp.Wi",
+    "mlp.Wo"
+  ],
+  "target_parameters": null,
+  "task_type": "SEQ_CLS",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-1000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:80a3434a0cc2bb94be4bc44a602b4407b898d40a51b3990480644985c4fe55fe
+size 27061816

checkpoint-1000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:26e3101da05914512e04ef505f1d199d8a3affcfce864428c63d305c59c02939
+size 54232075

checkpoint-1000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:995cfffe2c047736df057bb673f62368a40f8b159891c94145c485f61e475cde
+size 14645

checkpoint-1000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65d9f6b1b05902e12f9b4b66135e774f9e41d7804c8d01be9c29327770282f55
+size 1465

checkpoint-1000/trainer_state.json

@@ -0,0 +1,754 @@
+{
+  "best_global_step": 800,
+  "best_metric": 0.9917981072555205,
+  "best_model_checkpoint": "/data/mmbert-jailbreak-detector-lora/checkpoint-800",
+  "epoch": 10.0,
+  "eval_steps": 500,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.1,
+      "learning_rate": 4.4999999999999996e-05,
+      "loss": 1.1486,
+      "step": 10
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 9.499999999999999e-05,
+      "loss": 0.5232,
+      "step": 20
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 0.000145,
+      "loss": 0.3105,
+      "step": 30
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 0.000195,
+      "loss": 0.1912,
+      "step": 40
+    },
+    {
+      "epoch": 0.5,
+      "learning_rate": 0.000245,
+      "loss": 0.1106,
+      "step": 50
+    },
+    {
+      "epoch": 0.6,
+      "learning_rate": 0.00029499999999999996,
+      "loss": 0.1166,
+      "step": 60
+    },
+    {
+      "epoch": 0.7,
+      "learning_rate": 0.0002999321487932364,
+      "loss": 0.0899,
+      "step": 70
+    },
+    {
+      "epoch": 0.8,
+      "learning_rate": 0.00029969768021829806,
+      "loss": 0.0804,
+      "step": 80
+    },
+    {
+      "epoch": 0.9,
+      "learning_rate": 0.0002992960184150984,
+      "loss": 0.0841,
+      "step": 90
+    },
+    {
+      "epoch": 1.0,
+      "learning_rate": 0.00029872761198867565,
+      "loss": 0.0929,
+      "step": 100
+    },
+    {
+      "epoch": 1.0,
+      "eval_accuracy": 0.9725,
+      "eval_f1": 0.9727722772277227,
+      "eval_loss": 0.06970180571079254,
+      "eval_precision": 0.9538834951456311,
+      "eval_recall": 0.9924242424242424,
+      "eval_runtime": 2.8332,
+      "eval_samples_per_second": 564.742,
+      "eval_steps_per_second": 8.824,
+      "step": 100
+    },
+    {
+      "epoch": 1.1,
+      "learning_rate": 0.00029799309577655865,
+      "loss": 0.0357,
+      "step": 110
+    },
+    {
+      "epoch": 1.2,
+      "learning_rate": 0.0002970932901397346,
+      "loss": 0.0308,
+      "step": 120
+    },
+    {
+      "epoch": 1.3,
+      "learning_rate": 0.00029602920004641016,
+      "loss": 0.0261,
+      "step": 130
+    },
+    {
+      "epoch": 1.4,
+      "learning_rate": 0.00029480201394959006,
+      "loss": 0.0355,
+      "step": 140
+    },
+    {
+      "epoch": 1.5,
+      "learning_rate": 0.0002934131024597267,
+      "loss": 0.0493,
+      "step": 150
+    },
+    {
+      "epoch": 1.6,
+      "learning_rate": 0.0002918640168139229,
+      "loss": 0.0481,
+      "step": 160
+    },
+    {
+      "epoch": 1.7,
+      "learning_rate": 0.00029015648714339774,
+      "loss": 0.0322,
+      "step": 170
+    },
+    {
+      "epoch": 1.8,
+      "learning_rate": 0.0002882924205411507,
+      "loss": 0.0214,
+      "step": 180
+    },
+    {
+      "epoch": 1.9,
+      "learning_rate": 0.00028627389893198145,
+      "loss": 0.0444,
+      "step": 190
+    },
+    {
+      "epoch": 2.0,
+      "learning_rate": 0.00028410317674724576,
+      "loss": 0.0544,
+      "step": 200
+    },
+    {
+      "epoch": 2.0,
+      "eval_accuracy": 0.97,
+      "eval_f1": 0.96875,
+      "eval_loss": 0.08036020398139954,
+      "eval_precision": 1.0,
+      "eval_recall": 0.9393939393939394,
+      "eval_runtime": 1.1856,
+      "eval_samples_per_second": 1349.549,
+      "eval_steps_per_second": 21.087,
+      "step": 200
+    },
+    {
+      "epoch": 2.1,
+      "learning_rate": 0.0002817826784069426,
+      "loss": 0.0416,
+      "step": 210
+    },
+    {
+      "epoch": 2.2,
+      "learning_rate": 0.00027931499561194615,
+      "loss": 0.0136,
+      "step": 220
+    },
+    {
+      "epoch": 2.3,
+      "learning_rate": 0.0002767028844494064,
+      "loss": 0.0112,
+      "step": 230
+    },
+    {
+      "epoch": 2.4,
+      "learning_rate": 0.0002739492623145508,
+      "loss": 0.0138,
+      "step": 240
+    },
+    {
+      "epoch": 2.5,
+      "learning_rate": 0.0002710572046523259,
+      "loss": 0.0072,
+      "step": 250
+    },
+    {
+      "epoch": 2.6,
+      "learning_rate": 0.00026802994152251737,
+      "loss": 0.011,
+      "step": 260
+    },
+    {
+      "epoch": 2.7,
+      "learning_rate": 0.00026487085399218464,
+      "loss": 0.0237,
+      "step": 270
+    },
+    {
+      "epoch": 2.8,
+      "learning_rate": 0.0002615834703594401,
+      "loss": 0.0182,
+      "step": 280
+    },
+    {
+      "epoch": 2.9,
+      "learning_rate": 0.0002581714622127901,
+      "loss": 0.0357,
+      "step": 290
+    },
+    {
+      "epoch": 3.0,
+      "learning_rate": 0.00025463864033043804,
+      "loss": 0.0316,
+      "step": 300
+    },
+    {
+      "epoch": 3.0,
+      "eval_accuracy": 0.979375,
+      "eval_f1": 0.978750804893754,
+      "eval_loss": 0.052077628672122955,
+      "eval_precision": 0.9986859395532195,
+      "eval_recall": 0.9595959595959596,
+      "eval_runtime": 1.1837,
+      "eval_samples_per_second": 1351.643,
+      "eval_steps_per_second": 21.119,
+      "step": 300
+    },
+    {
+      "epoch": 3.1,
+      "learning_rate": 0.0002509889504241312,
+      "loss": 0.0129,
+      "step": 310
+    },
+    {
+      "epoch": 3.2,
+      "learning_rate": 0.00024722646873230375,
+      "loss": 0.017,
+      "step": 320
+    },
+    {
+      "epoch": 3.3,
+      "learning_rate": 0.0002433553974674375,
+      "loss": 0.0068,
+      "step": 330
+    },
+    {
+      "epoch": 3.4,
+      "learning_rate": 0.0002393800601227266,
+      "loss": 0.0077,
+      "step": 340
+    },
+    {
+      "epoch": 3.5,
+      "learning_rate": 0.00023530489664328662,
+      "loss": 0.002,
+      "step": 350
+    },
+    {
+      "epoch": 3.6,
+      "learning_rate": 0.00023113445846730167,
+      "loss": 0.0063,
+      "step": 360
+    },
+    {
+      "epoch": 3.7,
+      "learning_rate": 0.00022687340344264807,
+      "loss": 0.0015,
+      "step": 370
+    },
+    {
+      "epoch": 3.8,
+      "learning_rate": 0.00022252649062467217,
+      "loss": 0.0018,
+      "step": 380
+    },
+    {
+      "epoch": 3.9,
+      "learning_rate": 0.00021809857496093199,
+      "loss": 0.0085,
+      "step": 390
+    },
+    {
+      "epoch": 4.0,
+      "learning_rate": 0.0002135946018688395,
+      "loss": 0.0078,
+      "step": 400
+    },
+    {
+      "epoch": 4.0,
+      "eval_accuracy": 0.99,
+      "eval_f1": 0.9899244332493703,
+      "eval_loss": 0.026125960052013397,
+      "eval_precision": 0.9874371859296482,
+      "eval_recall": 0.9924242424242424,
+      "eval_runtime": 1.1858,
+      "eval_samples_per_second": 1349.271,
+      "eval_steps_per_second": 21.082,
+      "step": 400
+    },
+    {
+      "epoch": 4.1,
+      "learning_rate": 0.0002090196017122598,
+      "loss": 0.0008,
+      "step": 410
+    },
+    {
+      "epoch": 4.2,
+      "learning_rate": 0.00020437868418323604,
+      "loss": 0.0014,
+      "step": 420
+    },
+    {
+      "epoch": 4.3,
+      "learning_rate": 0.00019967703259511437,
+      "loss": 0.0011,
+      "step": 430
+    },
+    {
+      "epoch": 4.4,
+      "learning_rate": 0.00019491989809344385,
+      "loss": 0.0005,
+      "step": 440
+    },
+    {
+      "epoch": 4.5,
+      "learning_rate": 0.00019011259379111598,
+      "loss": 0.0002,
+      "step": 450
+    },
+    {
+      "epoch": 4.6,
+      "learning_rate": 0.00018526048883429514,
+      "loss": 0.0004,
+      "step": 460
+    },
+    {
+      "epoch": 4.7,
+      "learning_rate": 0.00018036900240576623,
+      "loss": 0.0005,
+      "step": 470
+    },
+    {
+      "epoch": 4.8,
+      "learning_rate": 0.00017544359767239853,
+      "loss": 0.0003,
+      "step": 480
+    },
+    {
+      "epoch": 4.9,
+      "learning_rate": 0.0001704897756834847,
+      "loss": 0.0003,
+      "step": 490
+    },
+    {
+      "epoch": 5.0,
+      "learning_rate": 0.00016551306922676972,
+      "loss": 0.0002,
+      "step": 500
+    },
+    {
+      "epoch": 5.0,
+      "eval_accuracy": 0.990625,
+      "eval_f1": 0.9905243209096652,
+      "eval_loss": 0.031400058418512344,
+      "eval_precision": 0.9911504424778761,
+      "eval_recall": 0.98989898989899,
+      "eval_runtime": 1.1948,
+      "eval_samples_per_second": 1339.125,
+      "eval_steps_per_second": 20.924,
+      "step": 500
+    },
+    {
+      "epoch": 5.1,
+      "learning_rate": 0.0001605190366490321,
+      "loss": 0.0001,
+      "step": 510
+    },
+    {
+      "epoch": 5.2,
+      "learning_rate": 0.00015551325564811924,
+      "loss": 0.0002,
+      "step": 520
+    },
+    {
+      "epoch": 5.3,
+      "learning_rate": 0.00015050131704336987,
+      "loss": 0.0001,
+      "step": 530
+    },
+    {
+      "epoch": 5.4,
+      "learning_rate": 0.0001454888185313811,
+      "loss": 0.0001,
+      "step": 540
+    },
+    {
+      "epoch": 5.5,
+      "learning_rate": 0.0001404813584340954,
+      "loss": 0.0001,
+      "step": 550
+    },
+    {
+      "epoch": 5.6,
+      "learning_rate": 0.00013548452944618785,
+      "loss": 0.0001,
+      "step": 560
+    },
+    {
+      "epoch": 5.7,
+      "learning_rate": 0.0001305039123887396,
+      "loss": 0.0,
+      "step": 570
+    },
+    {
+      "epoch": 5.8,
+      "learning_rate": 0.00012554506997617193,
+      "loss": 0.0001,
+      "step": 580
+    },
+    {
+      "epoch": 5.9,
+      "learning_rate": 0.00012061354060340307,
+      "loss": 0.0001,
+      "step": 590
+    },
+    {
+      "epoch": 6.0,
+      "learning_rate": 0.00011571483216016707,
+      "loss": 0.0001,
+      "step": 600
+    },
+    {
+      "epoch": 6.0,
+      "eval_accuracy": 0.99125,
+      "eval_f1": 0.9911616161616161,
+      "eval_loss": 0.03386121988296509,
+      "eval_precision": 0.9911616161616161,
+      "eval_recall": 0.9911616161616161,
+      "eval_runtime": 1.2053,
+      "eval_samples_per_second": 1327.517,
+      "eval_steps_per_second": 20.742,
+      "step": 600
+    },
+    {
+      "epoch": 6.1,
+      "learning_rate": 0.00011085441587940324,
+      "loss": 0.0001,
+      "step": 610
+    },
+    {
+      "epoch": 6.2,
+      "learning_rate": 0.00010603772022658601,
+      "loss": 0.0001,
+      "step": 620
+    },
+    {
+      "epoch": 6.3,
+      "learning_rate": 0.00010127012483682115,
+      "loss": 0.0001,
+      "step": 630
+    },
+    {
+      "epoch": 6.4,
+      "learning_rate": 9.655695450647885e-05,
+      "loss": 0.0001,
+      "step": 640
+    },
+    {
+      "epoch": 6.5,
+      "learning_rate": 9.190347324607498e-05,
+      "loss": 0.0,
+      "step": 650
+    },
+    {
+      "epoch": 6.6,
+      "learning_rate": 8.73148784010424e-05,
+      "loss": 0.0,
+      "step": 660
+    },
+    {
+      "epoch": 6.7,
+      "learning_rate": 8.279629484695836e-05,
+      "loss": 0.0,
+      "step": 670
+    },
+    {
+      "epoch": 6.8,
+      "learning_rate": 7.835276926571172e-05,
+      "loss": 0.0,
+      "step": 680
+    },
+    {
+      "epoch": 6.9,
+      "learning_rate": 7.398926450900283e-05,
+      "loss": 0.0,
+      "step": 690
+    },
+    {
+      "epoch": 7.0,
+      "learning_rate": 6.971065405547033e-05,
+      "loss": 0.0001,
+      "step": 700
+    },
+    {
+      "epoch": 7.0,
+      "eval_accuracy": 0.99125,
+      "eval_f1": 0.9911727616645649,
+      "eval_loss": 0.03514711931347847,
+      "eval_precision": 0.9899244332493703,
+      "eval_recall": 0.9924242424242424,
+      "eval_runtime": 1.2072,
+      "eval_samples_per_second": 1325.419,
+      "eval_steps_per_second": 20.71,
+      "step": 700
+    },
+    {
+      "epoch": 7.1,
+      "learning_rate": 6.55217165676369e-05,
+      "loss": 0.0001,
+      "step": 710
+    },
+    {
+      "epoch": 7.2,
+      "learning_rate": 6.142713055475244e-05,
+      "loss": 0.0001,
+      "step": 720
+    },
+    {
+      "epoch": 7.3,
+      "learning_rate": 5.743146914749524e-05,
+      "loss": 0.0001,
+      "step": 730
+    },
+    {
+      "epoch": 7.4,
+      "learning_rate": 5.3539194990368256e-05,
+      "loss": 0.0,
+      "step": 740
+    },
+    {
+      "epoch": 7.5,
+      "learning_rate": 4.9754655257493656e-05,
+      "loss": 0.0,
+      "step": 750
+    },
+    {
+      "epoch": 7.6,
+      "learning_rate": 4.608207679737374e-05,
+      "loss": 0.0,
+      "step": 760
+    },
+    {
+      "epoch": 7.7,
+      "learning_rate": 4.252556141203982e-05,
+      "loss": 0.0,
+      "step": 770
+    },
+    {
+      "epoch": 7.8,
+      "learning_rate": 3.9089081275861954e-05,
+      "loss": 0.0,
+      "step": 780
+    },
+    {
+      "epoch": 7.9,
+      "learning_rate": 3.5776474499136744e-05,
+      "loss": 0.0,
+      "step": 790
+    },
+    {
+      "epoch": 8.0,
+      "learning_rate": 3.2591440841407304e-05,
+      "loss": 0.0001,
+      "step": 800
+    },
+    {
+      "epoch": 8.0,
+      "eval_accuracy": 0.991875,
+      "eval_f1": 0.9917981072555205,
+      "eval_loss": 0.035532549023628235,
+      "eval_precision": 0.9911727616645649,
+      "eval_recall": 0.9924242424242424,
+      "eval_runtime": 1.1976,
+      "eval_samples_per_second": 1335.98,
+      "eval_steps_per_second": 20.875,
+      "step": 800
+    },
+    {
+      "epoch": 8.1,
+      "learning_rate": 2.9537537579303293e-05,
+      "loss": 0.0,
+      "step": 810
+    },
+    {
+      "epoch": 8.2,
+      "learning_rate": 2.6618175533516357e-05,
+      "loss": 0.0,
+      "step": 820
+    },
+    {
+      "epoch": 8.3,
+      "learning_rate": 2.3836615259348298e-05,
+      "loss": 0.0001,
+      "step": 830
+    },
+    {
+      "epoch": 8.4,
+      "learning_rate": 2.1195963405086318e-05,
+      "loss": 0.0,
+      "step": 840
+    },
+    {
+      "epoch": 8.5,
+      "learning_rate": 1.8699169242272745e-05,
+      "loss": 0.0001,
+      "step": 850
+    },
+    {
+      "epoch": 8.6,
+      "learning_rate": 1.634902137174483e-05,
+      "loss": 0.0,
+      "step": 860
+    },
+    {
+      "epoch": 8.7,
+      "learning_rate": 1.4148144609122852e-05,
+      "loss": 0.0,
+      "step": 870
+    },
+    {
+      "epoch": 8.8,
+      "learning_rate": 1.209899705322585e-05,
+      "loss": 0.0,
+      "step": 880
+    },
+    {
+      "epoch": 8.9,
+      "learning_rate": 1.0203867340688332e-05,
+      "loss": 0.0,
+      "step": 890
+    },
+    {
+      "epoch": 9.0,
+      "learning_rate": 8.464872089844982e-06,
+      "loss": 0.0,
+      "step": 900
+    },
+    {
+      "epoch": 9.0,
+      "eval_accuracy": 0.991875,
+      "eval_f1": 0.9917981072555205,
+      "eval_loss": 0.035815972834825516,
+      "eval_precision": 0.9911727616645649,
+      "eval_recall": 0.9924242424242424,
+      "eval_runtime": 1.2091,
+      "eval_samples_per_second": 1323.342,
+      "eval_steps_per_second": 20.677,
+      "step": 900
+    },
+    {
+      "epoch": 9.1,
+      "learning_rate": 6.88395353673783e-06,
+      "loss": 0.0,
+      "step": 910
+    },
+    {
+      "epoch": 9.2,
+      "learning_rate": 5.462877365885954e-06,
+      "loss": 0.0,
+      "step": 920
+    },
+    {
+      "epoch": 9.3,
+      "learning_rate": 4.20323073824117e-06,
+      "loss": 0.0,
+      "step": 930
+    },
+    {
+      "epoch": 9.4,
+      "learning_rate": 3.1064205185312774e-06,
+      "loss": 0.0,
+      "step": 940
+    },
+    {
+      "epoch": 9.5,
+      "learning_rate": 2.173671703971774e-06,
+      "loss": 0.0,
+      "step": 950
+    },
+    {
+      "epoch": 9.6,
+      "learning_rate": 1.406026056099846e-06,
+      "loss": 0.0001,
+      "step": 960
+    },
+    {
+      "epoch": 9.7,
+      "learning_rate": 8.043409372596787e-07,
+      "loss": 0.0,
+      "step": 970
+    },
+    {
+      "epoch": 9.8,
+      "learning_rate": 3.6928835303789986e-07,
+      "loss": 0.0,
+      "step": 980
+    },
+    {
+      "epoch": 9.9,
+      "learning_rate": 1.0135420171908381e-07,
+      "loss": 0.0,
+      "step": 990
+    },
+    {
+      "epoch": 10.0,
+      "learning_rate": 8.377315992236233e-10,
+      "loss": 0.0,
+      "step": 1000
+    },
+    {
+      "epoch": 10.0,
+      "eval_accuracy": 0.991875,
+      "eval_f1": 0.9917981072555205,
+      "eval_loss": 0.035856734961271286,
+      "eval_precision": 0.9911727616645649,
+      "eval_recall": 0.9924242424242424,
+      "eval_runtime": 1.2125,
+      "eval_samples_per_second": 1319.556,
+      "eval_steps_per_second": 20.618,
+      "step": 1000
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.3137560035328e+16,
+  "train_batch_size": 64,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f78f1b2958c3ea989dc461e96c206c82968703a7e46e597797ffd4827e31ae8
+size 5777

checkpoint-800/README.md

@@ -0,0 +1,206 @@
+---
+base_model: jhu-clsp/mmBERT-base
+library_name: peft
+tags:
+- base_model:adapter:jhu-clsp/mmBERT-base
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1

checkpoint-800/adapter_config.json

@@ -0,0 +1,43 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "jhu-clsp/mmBERT-base",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "classifier",
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "attn.Wo",
+    "attn.Wqkv",
+    "mlp.Wi",
+    "mlp.Wo"
+  ],
+  "target_parameters": null,
+  "task_type": "SEQ_CLS",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-800/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba1aa5de62dd97c500ffda127954d99112e1237889109f88ab9f04ab488c646c
+size 27061816

checkpoint-800/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88f9fcf4a5847aca8022e9be26562d0fd954b241424c2511539144f90aa66ca8
+size 54232075

checkpoint-800/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:26c2163045a98c7798cd9b8d91f00490f4130fcfdcf342e397a40e0e08dcd457
+size 14645

checkpoint-800/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d5e20615f60f06003e5e2e4917a47b822371a4a1a873b4d09571a1cf59add95
+size 1465

checkpoint-800/trainer_state.json

@@ -0,0 +1,610 @@
+{
+  "best_global_step": 800,
+  "best_metric": 0.9917981072555205,
+  "best_model_checkpoint": "/data/mmbert-jailbreak-detector-lora/checkpoint-800",
+  "epoch": 8.0,
+  "eval_steps": 500,
+  "global_step": 800,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.1,
+      "learning_rate": 4.4999999999999996e-05,
+      "loss": 1.1486,
+      "step": 10
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 9.499999999999999e-05,
+      "loss": 0.5232,
+      "step": 20
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 0.000145,
+      "loss": 0.3105,
+      "step": 30
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 0.000195,
+      "loss": 0.1912,
+      "step": 40
+    },
+    {
+      "epoch": 0.5,
+      "learning_rate": 0.000245,
+      "loss": 0.1106,
+      "step": 50
+    },
+    {
+      "epoch": 0.6,
+      "learning_rate": 0.00029499999999999996,
+      "loss": 0.1166,
+      "step": 60
+    },
+    {
+      "epoch": 0.7,
+      "learning_rate": 0.0002999321487932364,
+      "loss": 0.0899,
+      "step": 70
+    },
+    {
+      "epoch": 0.8,
+      "learning_rate": 0.00029969768021829806,
+      "loss": 0.0804,
+      "step": 80
+    },
+    {
+      "epoch": 0.9,
+      "learning_rate": 0.0002992960184150984,
+      "loss": 0.0841,
+      "step": 90
+    },
+    {
+      "epoch": 1.0,
+      "learning_rate": 0.00029872761198867565,
+      "loss": 0.0929,
+      "step": 100
+    },
+    {
+      "epoch": 1.0,
+      "eval_accuracy": 0.9725,
+      "eval_f1": 0.9727722772277227,
+      "eval_loss": 0.06970180571079254,
+      "eval_precision": 0.9538834951456311,
+      "eval_recall": 0.9924242424242424,
+      "eval_runtime": 2.8332,
+      "eval_samples_per_second": 564.742,
+      "eval_steps_per_second": 8.824,
+      "step": 100
+    },
+    {
+      "epoch": 1.1,
+      "learning_rate": 0.00029799309577655865,
+      "loss": 0.0357,
+      "step": 110
+    },
+    {
+      "epoch": 1.2,
+      "learning_rate": 0.0002970932901397346,
+      "loss": 0.0308,
+      "step": 120
+    },
+    {
+      "epoch": 1.3,
+      "learning_rate": 0.00029602920004641016,
+      "loss": 0.0261,
+      "step": 130
+    },
+    {
+      "epoch": 1.4,
+      "learning_rate": 0.00029480201394959006,
+      "loss": 0.0355,
+      "step": 140
+    },
+    {
+      "epoch": 1.5,
+      "learning_rate": 0.0002934131024597267,
+      "loss": 0.0493,
+      "step": 150
+    },
+    {
+      "epoch": 1.6,
+      "learning_rate": 0.0002918640168139229,
+      "loss": 0.0481,
+      "step": 160
+    },
+    {
+      "epoch": 1.7,
+      "learning_rate": 0.00029015648714339774,
+      "loss": 0.0322,
+      "step": 170
+    },
+    {
+      "epoch": 1.8,
+      "learning_rate": 0.0002882924205411507,
+      "loss": 0.0214,
+      "step": 180
+    },
+    {
+      "epoch": 1.9,
+      "learning_rate": 0.00028627389893198145,
+      "loss": 0.0444,
+      "step": 190
+    },
+    {
+      "epoch": 2.0,
+      "learning_rate": 0.00028410317674724576,
+      "loss": 0.0544,
+      "step": 200
+    },
+    {
+      "epoch": 2.0,
+      "eval_accuracy": 0.97,
+      "eval_f1": 0.96875,
+      "eval_loss": 0.08036020398139954,
+      "eval_precision": 1.0,
+      "eval_recall": 0.9393939393939394,
+      "eval_runtime": 1.1856,
+      "eval_samples_per_second": 1349.549,
+      "eval_steps_per_second": 21.087,
+      "step": 200
+    },
+    {
+      "epoch": 2.1,
+      "learning_rate": 0.0002817826784069426,
+      "loss": 0.0416,
+      "step": 210
+    },
+    {
+      "epoch": 2.2,
+      "learning_rate": 0.00027931499561194615,
+      "loss": 0.0136,
+      "step": 220
+    },
+    {
+      "epoch": 2.3,
+      "learning_rate": 0.0002767028844494064,
+      "loss": 0.0112,
+      "step": 230
+    },
+    {
+      "epoch": 2.4,
+      "learning_rate": 0.0002739492623145508,
+      "loss": 0.0138,
+      "step": 240
+    },
+    {
+      "epoch": 2.5,
+      "learning_rate": 0.0002710572046523259,
+      "loss": 0.0072,
+      "step": 250
+    },
+    {
+      "epoch": 2.6,
+      "learning_rate": 0.00026802994152251737,
+      "loss": 0.011,
+      "step": 260
+    },
+    {
+      "epoch": 2.7,
+      "learning_rate": 0.00026487085399218464,
+      "loss": 0.0237,
+      "step": 270
+    },
+    {
+      "epoch": 2.8,
+      "learning_rate": 0.0002615834703594401,
+      "loss": 0.0182,
+      "step": 280
+    },
+    {
+      "epoch": 2.9,
+      "learning_rate": 0.0002581714622127901,
+      "loss": 0.0357,
+      "step": 290
+    },
+    {
+      "epoch": 3.0,
+      "learning_rate": 0.00025463864033043804,
+      "loss": 0.0316,
+      "step": 300
+    },
+    {
+      "epoch": 3.0,
+      "eval_accuracy": 0.979375,
+      "eval_f1": 0.978750804893754,
+      "eval_loss": 0.052077628672122955,
+      "eval_precision": 0.9986859395532195,
+      "eval_recall": 0.9595959595959596,
+      "eval_runtime": 1.1837,
+      "eval_samples_per_second": 1351.643,
+      "eval_steps_per_second": 21.119,
+      "step": 300
+    },
+    {
+      "epoch": 3.1,
+      "learning_rate": 0.0002509889504241312,
+      "loss": 0.0129,
+      "step": 310
+    },
+    {
+      "epoch": 3.2,
+      "learning_rate": 0.00024722646873230375,
+      "loss": 0.017,
+      "step": 320
+    },
+    {
+      "epoch": 3.3,
+      "learning_rate": 0.0002433553974674375,
+      "loss": 0.0068,
+      "step": 330
+    },
+    {
+      "epoch": 3.4,
+      "learning_rate": 0.0002393800601227266,
+      "loss": 0.0077,
+      "step": 340
+    },
+    {
+      "epoch": 3.5,
+      "learning_rate": 0.00023530489664328662,
+      "loss": 0.002,
+      "step": 350
+    },
+    {
+      "epoch": 3.6,
+      "learning_rate": 0.00023113445846730167,
+      "loss": 0.0063,
+      "step": 360
+    },
+    {
+      "epoch": 3.7,
+      "learning_rate": 0.00022687340344264807,
+      "loss": 0.0015,
+      "step": 370
+    },
+    {
+      "epoch": 3.8,
+      "learning_rate": 0.00022252649062467217,
+      "loss": 0.0018,
+      "step": 380
+    },
+    {
+      "epoch": 3.9,
+      "learning_rate": 0.00021809857496093199,
+      "loss": 0.0085,
+      "step": 390
+    },
+    {
+      "epoch": 4.0,
+      "learning_rate": 0.0002135946018688395,
+      "loss": 0.0078,
+      "step": 400
+    },
+    {
+      "epoch": 4.0,
+      "eval_accuracy": 0.99,
+      "eval_f1": 0.9899244332493703,
+      "eval_loss": 0.026125960052013397,
+      "eval_precision": 0.9874371859296482,
+      "eval_recall": 0.9924242424242424,
+      "eval_runtime": 1.1858,
+      "eval_samples_per_second": 1349.271,
+      "eval_steps_per_second": 21.082,
+      "step": 400
+    },
+    {
+      "epoch": 4.1,
+      "learning_rate": 0.0002090196017122598,
+      "loss": 0.0008,
+      "step": 410
+    },
+    {
+      "epoch": 4.2,
+      "learning_rate": 0.00020437868418323604,
+      "loss": 0.0014,
+      "step": 420
+    },
+    {
+      "epoch": 4.3,
+      "learning_rate": 0.00019967703259511437,
+      "loss": 0.0011,
+      "step": 430
+    },
+    {
+      "epoch": 4.4,
+      "learning_rate": 0.00019491989809344385,
+      "loss": 0.0005,
+      "step": 440
+    },
+    {
+      "epoch": 4.5,
+      "learning_rate": 0.00019011259379111598,
+      "loss": 0.0002,
+      "step": 450
+    },
+    {
+      "epoch": 4.6,
+      "learning_rate": 0.00018526048883429514,
+      "loss": 0.0004,
+      "step": 460
+    },
+    {
+      "epoch": 4.7,
+      "learning_rate": 0.00018036900240576623,
+      "loss": 0.0005,
+      "step": 470
+    },
+    {
+      "epoch": 4.8,
+      "learning_rate": 0.00017544359767239853,
+      "loss": 0.0003,
+      "step": 480
+    },
+    {
+      "epoch": 4.9,
+      "learning_rate": 0.0001704897756834847,
+      "loss": 0.0003,
+      "step": 490
+    },
+    {
+      "epoch": 5.0,
+      "learning_rate": 0.00016551306922676972,
+      "loss": 0.0002,
+      "step": 500
+    },
+    {
+      "epoch": 5.0,
+      "eval_accuracy": 0.990625,
+      "eval_f1": 0.9905243209096652,
+      "eval_loss": 0.031400058418512344,
+      "eval_precision": 0.9911504424778761,
+      "eval_recall": 0.98989898989899,
+      "eval_runtime": 1.1948,
+      "eval_samples_per_second": 1339.125,
+      "eval_steps_per_second": 20.924,
+      "step": 500
+    },
+    {
+      "epoch": 5.1,
+      "learning_rate": 0.0001605190366490321,
+      "loss": 0.0001,
+      "step": 510
+    },
+    {
+      "epoch": 5.2,
+      "learning_rate": 0.00015551325564811924,
+      "loss": 0.0002,
+      "step": 520
+    },
+    {
+      "epoch": 5.3,
+      "learning_rate": 0.00015050131704336987,
+      "loss": 0.0001,
+      "step": 530
+    },
+    {
+      "epoch": 5.4,
+      "learning_rate": 0.0001454888185313811,
+      "loss": 0.0001,
+      "step": 540
+    },
+    {
+      "epoch": 5.5,
+      "learning_rate": 0.0001404813584340954,
+      "loss": 0.0001,
+      "step": 550
+    },
+    {
+      "epoch": 5.6,
+      "learning_rate": 0.00013548452944618785,
+      "loss": 0.0001,
+      "step": 560
+    },
+    {
+      "epoch": 5.7,
+      "learning_rate": 0.0001305039123887396,
+      "loss": 0.0,
+      "step": 570
+    },
+    {
+      "epoch": 5.8,
+      "learning_rate": 0.00012554506997617193,
+      "loss": 0.0001,
+      "step": 580
+    },
+    {
+      "epoch": 5.9,
+      "learning_rate": 0.00012061354060340307,
+      "loss": 0.0001,
+      "step": 590
+    },
+    {
+      "epoch": 6.0,
+      "learning_rate": 0.00011571483216016707,
+      "loss": 0.0001,
+      "step": 600
+    },
+    {
+      "epoch": 6.0,
+      "eval_accuracy": 0.99125,
+      "eval_f1": 0.9911616161616161,
+      "eval_loss": 0.03386121988296509,
+      "eval_precision": 0.9911616161616161,
+      "eval_recall": 0.9911616161616161,
+      "eval_runtime": 1.2053,
+      "eval_samples_per_second": 1327.517,
+      "eval_steps_per_second": 20.742,
+      "step": 600
+    },
+    {
+      "epoch": 6.1,
+      "learning_rate": 0.00011085441587940324,
+      "loss": 0.0001,
+      "step": 610
+    },
+    {
+      "epoch": 6.2,
+      "learning_rate": 0.00010603772022658601,
+      "loss": 0.0001,
+      "step": 620
+    },
+    {
+      "epoch": 6.3,
+      "learning_rate": 0.00010127012483682115,
+      "loss": 0.0001,
+      "step": 630
+    },
+    {
+      "epoch": 6.4,
+      "learning_rate": 9.655695450647885e-05,
+      "loss": 0.0001,
+      "step": 640
+    },
+    {
+      "epoch": 6.5,
+      "learning_rate": 9.190347324607498e-05,
+      "loss": 0.0,
+      "step": 650
+    },
+    {
+      "epoch": 6.6,
+      "learning_rate": 8.73148784010424e-05,
+      "loss": 0.0,
+      "step": 660
+    },
+    {
+      "epoch": 6.7,
+      "learning_rate": 8.279629484695836e-05,
+      "loss": 0.0,
+      "step": 670
+    },
+    {
+      "epoch": 6.8,
+      "learning_rate": 7.835276926571172e-05,
+      "loss": 0.0,
+      "step": 680
+    },
+    {
+      "epoch": 6.9,
+      "learning_rate": 7.398926450900283e-05,
+      "loss": 0.0,
+      "step": 690
+    },
+    {
+      "epoch": 7.0,
+      "learning_rate": 6.971065405547033e-05,
+      "loss": 0.0001,
+      "step": 700
+    },
+    {
+      "epoch": 7.0,
+      "eval_accuracy": 0.99125,
+      "eval_f1": 0.9911727616645649,
+      "eval_loss": 0.03514711931347847,
+      "eval_precision": 0.9899244332493703,
+      "eval_recall": 0.9924242424242424,
+      "eval_runtime": 1.2072,
+      "eval_samples_per_second": 1325.419,
+      "eval_steps_per_second": 20.71,
+      "step": 700
+    },
+    {
+      "epoch": 7.1,
+      "learning_rate": 6.55217165676369e-05,
+      "loss": 0.0001,
+      "step": 710
+    },
+    {
+      "epoch": 7.2,
+      "learning_rate": 6.142713055475244e-05,
+      "loss": 0.0001,
+      "step": 720
+    },
+    {
+      "epoch": 7.3,
+      "learning_rate": 5.743146914749524e-05,
+      "loss": 0.0001,
+      "step": 730
+    },
+    {
+      "epoch": 7.4,
+      "learning_rate": 5.3539194990368256e-05,
+      "loss": 0.0,
+      "step": 740
+    },
+    {
+      "epoch": 7.5,
+      "learning_rate": 4.9754655257493656e-05,
+      "loss": 0.0,
+      "step": 750
+    },
+    {
+      "epoch": 7.6,
+      "learning_rate": 4.608207679737374e-05,
+      "loss": 0.0,
+      "step": 760
+    },
+    {
+      "epoch": 7.7,
+      "learning_rate": 4.252556141203982e-05,
+      "loss": 0.0,
+      "step": 770
+    },
+    {
+      "epoch": 7.8,
+      "learning_rate": 3.9089081275861954e-05,
+      "loss": 0.0,
+      "step": 780
+    },
+    {
+      "epoch": 7.9,
+      "learning_rate": 3.5776474499136744e-05,
+      "loss": 0.0,
+      "step": 790
+    },
+    {
+      "epoch": 8.0,
+      "learning_rate": 3.2591440841407304e-05,
+      "loss": 0.0001,
+      "step": 800
+    },
+    {
+      "epoch": 8.0,
+      "eval_accuracy": 0.991875,
+      "eval_f1": 0.9917981072555205,
+      "eval_loss": 0.035532549023628235,
+      "eval_precision": 0.9911727616645649,
+      "eval_recall": 0.9924242424242424,
+      "eval_runtime": 1.1976,
+      "eval_samples_per_second": 1335.98,
+      "eval_steps_per_second": 20.875,
+      "step": 800
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.85100480282624e+16,
+  "train_batch_size": 64,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-800/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f78f1b2958c3ea989dc461e96c206c82968703a7e46e597797ffd4827e31ae8
+size 5777