finetuned_byT5large_ddo_joint

This model is a fine-tuned version of google/byt5-large on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0738
• Chrf++: 87.4594
• Gen Len: 134.7708

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Chrf++	Gen Len
6.8153	1.9938	444	0.4818	33.9646	122.1124
5.2444	3.9848	888	0.1417	74.6167	126.582
0.6217	5.9758	1332	0.0677	85.3335	132.6539
0.4057	7.9668	1776	0.0613	86.9431	134.4382
0.2987	9.9578	2220	0.0625	88.0924	132.1663
0.2541	11.9488	2664	0.0666	86.4618	133.3775
0.2310	13.9399	3108	0.0727	85.7648	139.1393
0.2057	15.9309	3552	0.0738	87.4594	134.7708

Framework versions

• Transformers 5.12.1
• Pytorch 2.6.0+cu124
• Datasets 5.0.0
• Tokenizers 0.22.2