metadata
library_name: transformers
license: mit
base_model: FacebookAI/xlm-roberta-large
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: panoncology-RE-sp
results: []
panoncology-RE-sp
This model is a fine-tuned version of FacebookAI/xlm-roberta-large. It achieves the following results on the evaluation set:
- Loss: 0.1095
- F1 Macro: 0.9791
- F1 Weighted: 0.9878
- Precision Macro: 0.9834
- Recall Macro: 0.9749
- Accuracy: 0.9878
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 64
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 0.1
- num_epochs: 15
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | F1 Macro | F1 Weighted | Precision Macro | Recall Macro | Accuracy |
|---|---|---|---|---|---|---|---|---|
| 6.4623 | 1.0 | 65 | 1.5947 | 0.1778 | 0.1949 | 0.1840 | 0.2552 | 0.3520 |
| 5.4292 | 2.0 | 130 | 1.0449 | 0.6247 | 0.7446 | 0.6485 | 0.7039 | 0.7408 |
| 1.8966 | 3.0 | 195 | 0.3466 | 0.8689 | 0.9233 | 0.8803 | 0.8765 | 0.9304 |
| 0.9579 | 4.0 | 260 | 0.2411 | 0.9352 | 0.9573 | 0.9319 | 0.9387 | 0.9574 |
| 0.5551 | 5.0 | 325 | 0.1844 | 0.9308 | 0.9528 | 0.9279 | 0.9346 | 0.9536 |
| 0.5579 | 6.0 | 390 | 0.2231 | 0.9218 | 0.9431 | 0.9051 | 0.9512 | 0.9381 |
| 0.2832 | 7.0 | 455 | 0.1940 | 0.9505 | 0.9666 | 0.9511 | 0.9504 | 0.9671 |
| 0.4714 | 8.0 | 520 | 0.1696 | 0.9655 | 0.9750 | 0.9608 | 0.9705 | 0.9749 |
| 0.1448 | 9.0 | 585 | 0.2455 | 0.9573 | 0.9705 | 0.9602 | 0.9553 | 0.9710 |
| 0.1025 | 10.0 | 650 | 0.2173 | 0.9637 | 0.9749 | 0.9605 | 0.9672 | 0.9749 |
| 0.1946 | 11.0 | 715 | 0.1978 | 0.9653 | 0.9770 | 0.9593 | 0.9717 | 0.9768 |
Framework versions
- Transformers 5.9.0
- Pytorch 2.11.0+cu128
- Datasets 4.8.5
- Tokenizers 0.22.2