Automatic Speech Recognition
Transformers
ONNX
Safetensors
Igbo
whisper
igbo
asr
on-device
olu-igbo
Eval Results (legacy)
theelvace commited on
Commit
b924e40
·
verified ·
1 Parent(s): f58c12c

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +87 -0
README.md ADDED
@@ -0,0 +1,87 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - ig
4
+ license: cc-by-nc-sa-4.0
5
+ library_name: transformers
6
+ pipeline_tag: automatic-speech-recognition
7
+ tags:
8
+ - automatic-speech-recognition
9
+ - whisper
10
+ - igbo
11
+ - asr
12
+ - on-device
13
+ - olu-igbo
14
+ datasets:
15
+ - google/fleurs
16
+ - naijavoices/naijavoices-dataset
17
+ - benjaminogbonna/nigerian_common_voice_dataset
18
+ metrics:
19
+ - wer
20
+ base_model: openai/whisper-small
21
+ model-index:
22
+ - name: whisper-small-igbo-25k
23
+ results:
24
+ - task:
25
+ type: automatic-speech-recognition
26
+ name: Automatic Speech Recognition
27
+ dataset:
28
+ name: FLEURS (Igbo, ig_ng)
29
+ type: google/fleurs
30
+ config: ig_ng
31
+ split: test
32
+ metrics:
33
+ - type: wer
34
+ value: 41.95
35
+ name: Test WER
36
+ ---
37
+
38
+ # Whisper Small — Igbo (Olu Igbo)
39
+
40
+ A full fine-tune of [openai/whisper-small](https://huggingface.co/openai/whisper-small) for **Igbo** automatic speech recognition, built to run **fully on-device** on mid-range Android phones (developed and tested on a Redmi Note 10 / Snapdragon 678, CPU-only).
41
+
42
+ - **WER: 41.95%** on the FLEURS Igbo test set (969 samples, normalized), down from a 68.95% zero-shot baseline.
43
+ - Part of the **Olu Igbo** project: <https://github.com/Elvaceishim/olu-igbo>
44
+ - INT8 ONNX exports of this model power the on-device Android app in that repo.
45
+
46
+ ## Language token
47
+
48
+ Igbo is not one of Whisper's 99 languages, so this model uses the **Yoruba token `<|yo|>` (id 50325)** as a language-token proxy, at both training and inference. Supply the forced decoder prefix shown below.
49
+
50
+ ## Usage
51
+
52
+ ```python
53
+ import numpy as np, torch
54
+ from transformers import WhisperProcessor, WhisperForConditionalGeneration
55
+
56
+ processor = WhisperProcessor.from_pretrained("openai/whisper-small")
57
+ model = WhisperForConditionalGeneration.from_pretrained("theelvace/whisper-small-igbo-25k").eval()
58
+ model.config.forced_decoder_ids = None
59
+
60
+ # audio: 16 kHz mono float32 numpy array
61
+ feats = processor.feature_extractor(audio, sampling_rate=16000, return_tensors="pt").input_features
62
+ ids = model.generate(
63
+ feats,
64
+ forced_decoder_ids=[[1, 50325], [2, 50359], [3, 50363]], # <|yo|>, transcribe, no-timestamps
65
+ num_beams=5,
66
+ max_new_tokens=100,
67
+ )
68
+ print(processor.tokenizer.decode(ids[0], skip_special_tokens=True))
69
+ ```
70
+
71
+ ## Training
72
+
73
+ - **Base:** `openai/whisper-small`, warm-started from a LoRA adapter (q/v, r=32) that reached 62.45% WER, then **full fine-tuned** (all parameters — same on-device size as the base).
74
+ - **Data:** FLEURS Igbo + Nigerian Common Voice (Igbo) + ~25k streamed utterances of [NaijaVoices](https://huggingface.co/datasets/naijavoices/naijavoices-dataset) (a ~600-hour Igbo read-speech corpus).
75
+ - **SpecAugment** enabled; checkpoints selected on the FLEURS **validation** split, WER reported on the held-out **test** split.
76
+ - **WER journey:** 68.95% (zero-shot) → 62.45% (LoRA) → 48.58% (full fine-tune) → **41.95%** (+ NaijaVoices).
77
+ - Full training, export, and on-device inference code: <https://github.com/Elvaceishim/olu-igbo>
78
+
79
+ ## Limitations
80
+
81
+ - Short, clear utterances transcribe well; long or complex sentences show the model's real error rate.
82
+ - **Live-mic audio is harder than clean test audio** — on long/complex live speech the model may fail to emit an end-of-sentence token and ramble.
83
+ - Borrowed words, proper nouns, and numbers are the weakest spots.
84
+
85
+ ## License
86
+
87
+ **CC-BY-NC-SA-4.0** (non-commercial, share-alike) — inherited from the NaijaVoices training data. (Base Whisper is MIT; FLEURS is CC-BY-4.0; Common Voice is CC0.) If you need a permissively-licensed variant, the FLEURS + Common Voice–only model [`theelvace/whisper-small-igbo`](https://huggingface.co/theelvace/whisper-small-igbo) (62.45% WER) is not subject to the non-commercial restriction.