mms-khotbah-lora-final

This model is a fine-tuned version of facebook/mms-1b-all on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.0736
• Wer: 0.4429

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 128
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
19.3181	0.1580	50	2.1154	0.6563
16.0984	0.3160	100	1.7825	0.6077
15.0136	0.4739	150	1.4714	0.5597
12.6032	0.6319	200	1.3219	0.5053
12.0595	0.7899	250	1.2435	0.4842
12.1743	0.9479	300	1.2041	0.4737
11.6856	1.1043	350	1.1855	0.4672
11.5580	1.2622	400	1.1728	0.4643
11.5327	1.4202	450	1.1705	0.4610
11.9941	1.5782	500	1.1495	0.4581
11.2616	1.7362	550	1.1428	0.4588
11.1071	1.8942	600	1.1425	0.4558
11.4070	2.0506	650	1.1365	0.4550
10.7593	2.2085	700	1.1325	0.4543
11.2756	2.3665	750	1.1304	0.4529
11.0277	2.5245	800	1.1315	0.4522
11.4287	2.6825	850	1.1267	0.4516
11.3707	2.8404	900	1.1256	0.4520
10.8637	2.9984	950	1.0736	0.4429

Framework versions

• PEFT 0.18.1
• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.8.3
• Tokenizers 0.22.2