| language: en | |
| library_name: transformers | |
| base_model: roberta-base | |
| tags: | |
| - text-classification | |
| - rte | |
| - parameter-efficient-fine-tuning | |
| # RoBERTa-base fine-tuned on GLUE RTE (lora) | |
| This checkpoint is one learning experiment comparing Full Fine-Tuning, | |
| BitFit, adapters, and LoRA on GLUE Recognizing Textual Entailment (RTE). | |
| It uses seed 42 and selects the highest validation epoch. | |
| | Metric | Value | | |
| | --- | ---: | | |
| | Best validation accuracy | 0.7148 | | |
| | Best epoch | 6 | | |
| | Trainable parameters | 887,042 | | |
| | Total parameters | 125,534,212 | | |
| ## Method | |
| `lora`. See the project README for the exact shared training configuration. | |
| This is an educational experiment, not a benchmark-level performance claim. | |
| ## Load | |
| ```python | |
| from peft import PeftModel | |
| from transformers import AutoModelForSequenceClassification | |
| base = AutoModelForSequenceClassification.from_pretrained("roberta-base", num_labels=2) | |
| model = PeftModel.from_pretrained(base, "Anirudh7003/roberta-base-rte-lora") | |
| ``` | |