life2lang-base-pt-go-ft

This model is a fine-tuned version of khairi/life2lang-base-pt on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4810

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 500
• num_epochs: 3.0

Training results

Training Loss	Epoch	Step	Validation Loss
1.8673	0.0617	1000	1.4130
1.4816	0.1234	2000	1.0700
1.3071	0.1851	3000	0.9358
1.2306	0.2468	4000	0.8501
1.1215	0.3084	5000	0.7879
1.0657	0.3701	6000	0.7463
1.0089	0.4318	7000	0.7196
0.9697	0.4935	8000	0.6891
0.9490	0.5552	9000	0.6623
0.9127	0.6169	10000	0.6550
0.8766	0.6786	11000	0.6356
0.8518	0.7403	12000	0.6125
0.8372	0.8020	13000	0.6012
0.8266	0.8637	14000	0.5974
0.8069	0.9253	15000	0.5724
0.8043	0.9870	16000	0.5724
0.7688	1.0487	17000	0.5634
0.7590	1.1104	18000	0.5518
0.7441	1.1721	19000	0.5421
0.7556	1.2337	20000	0.5423
0.7281	1.2954	21000	0.5385
0.7259	1.3571	22000	0.5214
0.7311	1.4188	23000	0.5160
0.6925	1.4805	24000	0.5235
0.6947	1.5422	25000	0.5170
0.6953	1.6039	26000	0.5016
0.6842	1.6656	27000	0.5043
0.6778	1.7273	28000	0.5027
0.6860	1.7889	29000	0.5034
0.6725	1.8506	30000	0.5014
0.6638	1.9123	31000	0.4949
0.6742	1.9740	32000	0.4877
0.6828	2.0357	33000	0.4910
0.6776	2.0973	34000	0.4883
0.6662	2.1590	35000	0.4907
0.6611	2.2207	36000	0.4838
0.6419	2.2824	37000	0.4840
0.6525	2.3441	38000	0.4829
0.6622	2.4058	39000	0.4845
0.6746	2.4675	40000	0.4805
0.6524	2.5292	41000	0.4810
0.6329	2.5909	42000	0.4798
0.6578	2.6526	43000	0.4788
0.6513	2.7142	44000	0.4802
0.6596	2.7759	45000	0.4820
0.6544	2.8376	46000	0.4804
0.6418	2.8993	47000	0.4812
0.6396	2.9610	48000	0.4810
0.6532	3.0	48633	0.4810

Framework versions

• Transformers 5.8.1
• Pytorch 2.11.0+cu128
• Datasets 3.1.0
• Tokenizers 0.22.2