arabic_model

This model is a fine-tuned version of UBC-NLP/AraT5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Gleu: 0.4037
• Loss: 0.2622

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 1
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 8
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Gleu	Validation Loss
19.1173	0.0837	1000	0.2732	1.3794
7.7961	0.1673	2000	0.3625	0.7437
5.7143	0.2510	3000	0.3744	0.5760
4.9653	0.3347	4000	0.3777	0.5058
5.0289	0.4184	5000	0.3823	0.4535
4.1525	0.5020	6000	0.3844	0.4582
4.0185	0.5857	7000	0.3864	0.4210
3.8394	0.6694	8000	0.3888	0.4059
3.8096	0.7530	9000	0.3883	0.3957
3.3862	0.8367	10000	0.3917	0.3658
3.1032	0.9204	11000	0.3708
2.7585	1.0040	12000	0.3583
2.5355	1.0877	13000	0.3544
3.0999	1.1714	14000	0.3324
2.6779	1.2550	15000	0.3954	0.3324
2.2995	1.3387	16000	0.3963	0.3189
2.4669	1.4224	17000	0.3980	0.3114
2.4679	1.5060	18000	0.3981	0.3076
2.3149	1.5897	19000	0.3991	0.2955
2.5368	1.6734	20000	0.3989	0.2899
2.5924	1.7571	21000	0.3991	0.2882
2.4474	1.8407	22000	0.4004	0.2902
2.1439	1.9244	23000	0.4000	0.2912
1.9272	2.0080	24000	0.4013	0.2832
1.8141	2.0917	25000	0.4018	0.2764
1.6279	2.1754	26000	0.4021	0.2811
1.8335	2.2590	27000	0.4021	0.2780
1.9612	2.3427	28000	0.4029	0.2749
1.8228	2.4264	29000	0.4027	0.2739
1.7412	2.5101	30000	0.4031	0.2702
1.6284	2.5937	31000	0.4032	0.2709
1.6856	2.6774	32000	0.4034	0.2667
1.7036	2.7611	33000	0.4037	0.2629
1.6959	2.8447	34000	0.4037	0.2604
1.5777	2.9284	35000	0.4037	0.2622

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2