legal_mt5_large_finetuned_e12

This model is a fine-tuned version of google/mt5-large on the None dataset. It achieves the following results on the evaluation set:

• Loss: 26.9787

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 7

Training results

Training Loss	Epoch	Step	Validation Loss
31.9236	0.7641	50	27.4731
31.8339	1.5196	100	27.4236
31.6682	2.2751	150	27.4717
31.7347	3.0306	200	27.4181
32.0255	3.7947	250	27.4156
31.2847	4.5501	300	27.2741
31.4576	5.3056	350	27.1971
31.3904	6.0611	400	27.0051
31.3494	6.8252	450	26.9787

Framework versions

• Transformers 4.49.0
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0