mbart-large-50-stage1-summary-only

This model is a fine-tuned version of facebook/mbart-large-50 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.6078

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss
3.8522	1.0	127	1.3287
2.3178	2.0	254	1.0195
1.6341	3.0	381	0.8336
1.3558	4.0	508	0.8433
1.1516	5.0	635	0.8453
1.1106	6.0	762	1.0699
0.7112	7.0	889	0.9277
0.5613	8.0	1016	1.0087
0.4208	9.0	1143	1.0854
0.3320	10.0	1270	1.1444
0.2509	11.0	1397	1.1901
0.2093	12.0	1524	1.2487
0.1730	13.0	1651	1.2882
0.1405	14.0	1778	1.3194
0.1198	15.0	1905	1.3390
0.0999	16.0	2032	1.3764
0.0878	17.0	2159	1.3972
0.0753	18.0	2286	1.4139
0.0671	19.0	2413	1.4172
0.0574	20.0	2540	1.4491
0.0492	21.0	2667	1.4690
0.0418	22.0	2794	1.4903
0.0376	23.0	2921	1.5011
0.0316	24.0	3048	1.5058
0.0301	25.0	3175	1.5156
0.0250	26.0	3302	1.5335
0.0217	27.0	3429	1.5414
0.0195	28.0	3556	1.5563
0.0187	29.0	3683	1.5464
0.0164	30.0	3810	1.5691
0.0149	31.0	3937	1.5858
0.0129	32.0	4064	1.5768
0.0117	33.0	4191	1.5857
0.0115	34.0	4318	1.5894
0.0099	35.0	4445	1.6025
0.0093	36.0	4572	1.5977
0.0088	37.0	4699	1.6049
0.0084	38.0	4826	1.6112
0.0075	39.0	4953	1.6085
0.0077	40.0	5080	1.6078

Framework versions

• Transformers 5.8.0
• Pytorch 2.11.0+cu130
• Datasets 4.8.5
• Tokenizers 0.22.2