File size: 5,203 Bytes
acdff73 20b3703 acdff73 20b3703 acdff73 20b3703 acdff73 1c1b7f6 706762d acdff73 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 | ---
language:
- ny
tags:
- automatic-speech-recognition
- whisper
- chichewa
- nyanja
- fleurs
license: apache-2.0
pipeline_tag: automatic-speech-recognition
base_model: openai/whisper-medium
---
# FLEURS-Trigram Whisper Medium No-Language LoRA Adapter
## Summary
This repository contains a Whisper checkpoint for Chichewa/Nyanja
automatic speech recognition, fine-tuned from `openai/whisper-medium`.
- Experiment type: `fleurs-trigram`
- Base model: `openai/whisper-medium`
- Training condition: `no_language`
- Release artifact: LoRA adapter checkpoint selected from the best training checkpoint
## Intended use
This adapter is intended for research and evaluation on Chichewa/Nyanja ASR. It must be used together with the base Whisper model.
It is not a production-ready speech system and should be validated carefully
before downstream use.
## How to use
This repository contains an adapter, not a fully merged standalone model. Load
the base model first, then attach the adapter.
```python
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
from peft import PeftModel
base_model_id = "openai/whisper-medium"
adapter_repo_id = "ai4good-labyrinth/fleurs-trigram-hours0p25-whisper-medium-no-language-lora-adapter"
processor = AutoProcessor.from_pretrained(base_model_id)
base_model = AutoModelForSpeechSeq2Seq.from_pretrained(base_model_id)
model = PeftModel.from_pretrained(base_model, adapter_repo_id)
```
The local evaluation script in this repository can also load the adapter
directly because it reads `adapter_config.json` and automatically fetches the
base model.
## Training data
- Training source: `FLEURS train + Chichewa Trigrams train`
- Evaluation source during training: `FLEURS dev`
- Train examples before duration filtering: `9139`
- Train examples after duration filtering: `9098`
- Dev examples before duration filtering: `311`
- Dev examples after duration filtering: `305`
- Duration filter used during training: `min_duration_seconds=0.0`, `max_duration_seconds=30.0`
## Training procedure
- Fine-tuning script: `experiments/whisper_finetune/finetune_whisper.py`
- Base model: `openai/whisper-medium`
- Task: `transcribe`
- Language hint during training/evaluation: none, corresponding to `--language auto` in standalone evaluation
- LoRA: `yes`
- LoRA rank: `32`
- LoRA alpha: `64`
- LoRA dropout: `0.05`
- LoRA target modules: `fc1,out_proj,k_proj,v_proj,fc2,q_proj`
- Extra trainable modules: `embed_tokens,proj_out`
- Mixed precision: `fp16`
- Gradient checkpointing: `True`
- Selected checkpoint step: `4050`
- Selected checkpoint epoch: `3.56`
## Training-time dev selection
The best checkpoint was selected using trainer-side dev evaluation on the
duration-filtered FLEURS dev split.
- Dev WER: `0.4697`
- Dev CER: `0.1132`
- Dev loss: `0.6200`
These values come from the training pipeline and may differ slightly from
standalone post-hoc evaluation because the decoding path is not perfectly
identical.
## Evaluation protocol
Standalone evaluation is recommended for the final release. Filtered and
unfiltered results should be reported separately.
- Filtered evaluation: `min_duration_seconds=0`, `max_duration_seconds=30`
- Unfiltered evaluation: no duration constraint
- Decoding task: `transcribe`
- Language hint: `auto`
## Evaluation summary
| Dataset | Split | Setting | Num examples | WER | CER | Notes |
|---|---|---|---:|---:|---:|---|
| FLEURS | dev | filtered | 305 | 0.4522 | 0.1040 | Filtered to 30 seconds |
| FLEURS | dev | unfiltered | TBD | TBD | TBD | Standalone eval pending |
| FLEURS | test | filtered | 745 | 0.4609 | 0.1137 | Filtered to 30 seconds |
| FLEURS | test | unfiltered | TBD | TBD | TBD | Standalone eval pending |
| Zambezi | dev | filtered | 613 | 0.6597 | 0.1871 | Filtered to 30 seconds |
| Zambezi | dev | unfiltered | TBD | TBD | TBD | Standalone eval pending |
| Zambezi | test | filtered | 427 | 0.7294 | 0.2059 | Filtered to 30 seconds |
| Zambezi | test | unfiltered | 428 | 0.7262 | 0.2047 | No duration filter |
## Files in this repository
- Adapter weights and config: repository root
- Processor/tokenizer files: repository root
- Evaluation JSON files: `eval/...`
## Known limitations
- Whisper does not provide an official Nyanja/Chichewa language token.
- This repository contains an adapter only, so users must also comply with the upstream base model license and dataset licenses.
- Standalone evaluation and trainer-side evaluation can differ slightly even on the same split and duration filter.
- Cross-dataset results should be interpreted carefully because transcription conventions may differ across corpora.
## Citation
If you use this checkpoint, please cite:
- the Whisper paper
- the FLEURS dataset
- this repository
```bibtex
@misc{fleurs_trigram_hours0p25_whisper_medium_no_language_lora_adapter_2026,
title = {FLEURS-Trigram Whisper Medium No-Language LoRA Adapter},
author = {AI4Good Labyrinth Team},
year = {2026},
howpublished = {\url{https://huggingface.co/ai4good-labyrinth/fleurs-trigram-hours0p25-whisper-medium-no-language-lora-adapter}},
note = {Whisper LoRA adapter fine-tuning for Chichewa/Nyanja ASR}
}
```
|