metadata
library_name: transformers
base_model: khairi/life2lang-base-pt
tags:
- generated_from_trainer
model-index:
- name: life2lang-base-pt-go-ft
results: []
life2lang-base-pt-go-ft
This model is a fine-tuned version of khairi/life2lang-base-pt on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.4810
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 64
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 500
- num_epochs: 3.0
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.8673 | 0.0617 | 1000 | 1.4130 |
| 1.4816 | 0.1234 | 2000 | 1.0700 |
| 1.3071 | 0.1851 | 3000 | 0.9358 |
| 1.2306 | 0.2468 | 4000 | 0.8501 |
| 1.1215 | 0.3084 | 5000 | 0.7879 |
| 1.0657 | 0.3701 | 6000 | 0.7463 |
| 1.0089 | 0.4318 | 7000 | 0.7196 |
| 0.9697 | 0.4935 | 8000 | 0.6891 |
| 0.9490 | 0.5552 | 9000 | 0.6623 |
| 0.9127 | 0.6169 | 10000 | 0.6550 |
| 0.8766 | 0.6786 | 11000 | 0.6356 |
| 0.8518 | 0.7403 | 12000 | 0.6125 |
| 0.8372 | 0.8020 | 13000 | 0.6012 |
| 0.8266 | 0.8637 | 14000 | 0.5974 |
| 0.8069 | 0.9253 | 15000 | 0.5724 |
| 0.8043 | 0.9870 | 16000 | 0.5724 |
| 0.7688 | 1.0487 | 17000 | 0.5634 |
| 0.7590 | 1.1104 | 18000 | 0.5518 |
| 0.7441 | 1.1721 | 19000 | 0.5421 |
| 0.7556 | 1.2337 | 20000 | 0.5423 |
| 0.7281 | 1.2954 | 21000 | 0.5385 |
| 0.7259 | 1.3571 | 22000 | 0.5214 |
| 0.7311 | 1.4188 | 23000 | 0.5160 |
| 0.6925 | 1.4805 | 24000 | 0.5235 |
| 0.6947 | 1.5422 | 25000 | 0.5170 |
| 0.6953 | 1.6039 | 26000 | 0.5016 |
| 0.6842 | 1.6656 | 27000 | 0.5043 |
| 0.6778 | 1.7273 | 28000 | 0.5027 |
| 0.6860 | 1.7889 | 29000 | 0.5034 |
| 0.6725 | 1.8506 | 30000 | 0.5014 |
| 0.6638 | 1.9123 | 31000 | 0.4949 |
| 0.6742 | 1.9740 | 32000 | 0.4877 |
| 0.6828 | 2.0357 | 33000 | 0.4910 |
| 0.6776 | 2.0973 | 34000 | 0.4883 |
| 0.6662 | 2.1590 | 35000 | 0.4907 |
| 0.6611 | 2.2207 | 36000 | 0.4838 |
| 0.6419 | 2.2824 | 37000 | 0.4840 |
| 0.6525 | 2.3441 | 38000 | 0.4829 |
| 0.6622 | 2.4058 | 39000 | 0.4845 |
| 0.6746 | 2.4675 | 40000 | 0.4805 |
| 0.6524 | 2.5292 | 41000 | 0.4810 |
| 0.6329 | 2.5909 | 42000 | 0.4798 |
| 0.6578 | 2.6526 | 43000 | 0.4788 |
| 0.6513 | 2.7142 | 44000 | 0.4802 |
| 0.6596 | 2.7759 | 45000 | 0.4820 |
| 0.6544 | 2.8376 | 46000 | 0.4804 |
| 0.6418 | 2.8993 | 47000 | 0.4812 |
| 0.6396 | 2.9610 | 48000 | 0.4810 |
| 0.6532 | 3.0 | 48633 | 0.4810 |
Framework versions
- Transformers 5.8.1
- Pytorch 2.11.0+cu128
- Datasets 3.1.0
- Tokenizers 0.22.2