finetuned_byT5large_ddo

This model is a fine-tuned version of google/byt5-large on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0727
• Chrf++: 86.7438
• Gen Len: 132.782

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Chrf++	Gen Len
7.0165	1.9938	444	0.4881	34.1949	150.8764
1.6653	3.9848	888	0.1125	78.7733	134.2022
5.4223	5.9758	1332	0.0687	85.6668	133.9663
0.3970	7.9668	1776	0.0638	86.5696	132.1618
0.3085	9.9578	2220	0.0587	87.5918	132.8899
0.2588	11.9488	2664	0.0609	87.9516	131.4629
0.2339	13.9399	3108	0.0719	85.8205	134.0539
0.2169	15.9309	3552	0.0679	87.2247	131.6112
0.1951	17.9219	3996	0.0727	86.7438	132.782

Framework versions

• Transformers 5.12.1
• Pytorch 2.6.0+cu124
• Datasets 5.0.0
• Tokenizers 0.22.2