Professor commited on
Commit
d1add0f
·
verified ·
1 Parent(s): 3042d25

Add comprehensive model card (WAXALNet benchmark)

Browse files
Files changed (1) hide show
  1. README.md +82 -18
README.md CHANGED
@@ -2,37 +2,66 @@
2
  library_name: transformers
3
  license: apache-2.0
4
  base_model: openai/whisper-small
 
 
5
  tags:
6
- - generated_from_trainer
 
 
 
 
 
 
7
  metrics:
8
  - wer
9
- model-index:
10
- - name: whisper-small-kpo-gbotemi
11
- results: []
12
  ---
 
13
 
14
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
- should probably proofread and complete it, then remove this comment. -->
16
 
17
- # whisper-small-kpo-gbotemi
18
 
19
- This model is a fine-tuned version of [openai/whisper-small](https://huggingface.co/openai/whisper-small) on an unknown dataset.
20
- It achieves the following results on the evaluation set:
21
- - Loss: 0.5751
22
- - Wer: 0.7239
23
- - Cer: 0.2642
 
 
 
 
 
24
 
25
- ## Model description
26
 
27
- More information needed
28
 
29
- ## Intended uses & limitations
30
 
31
- More information needed
32
 
33
- ## Training and evaluation data
34
 
35
- More information needed
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36
 
37
  ## Training procedure
38
 
@@ -68,3 +97,38 @@ The following hyperparameters were used during training:
68
  - Pytorch 2.10.0+cu128
69
  - Datasets 4.0.0
70
  - Tokenizers 0.22.2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  library_name: transformers
3
  license: apache-2.0
4
  base_model: openai/whisper-small
5
+ language:
6
+ - kpo
7
  tags:
8
+ - automatic-speech-recognition
9
+ - african-languages
10
+ - waxal
11
+ - waxalnet
12
+ - kpo
13
+ datasets:
14
+ - waxal-benchmarking/waxal
15
  metrics:
16
  - wer
17
+ - cer
 
 
18
  ---
19
+ # Whisper Small fine-tuned on WAXAL — Ikposo
20
 
21
+ This model is part of **[WAXALNet](https://huggingface.co/waxal-benchmarking)**, a suite of ASR models fine-tuned on the [WAXAL corpus](https://huggingface.co/waxal-benchmarking) across 19 African languages, developed as part of the WAXAL ASR Benchmark study.
 
22
 
23
+ ## Model Details
24
 
25
+ | | |
26
+ |---|---|
27
+ | **Language** | Ikposo (`kpo`) |
28
+ | **Language Family** | Niger-Congo (Kwa) |
29
+ | **Architecture** | Whisper Small (244M parameters) |
30
+ | **Base Model** | [openai/whisper-small](https://huggingface.co/openai/whisper-small) |
31
+ | **Training Data** | WAXAL corpus (conversational spontaneous speech) |
32
+ | **Test WER** | 77.5% |
33
+ | **Test CER** | 26.5% |
34
+ | **License** | apache-2.0 |
35
 
36
+ ## Intended Use
37
 
38
+ This model is intended for automatic speech recognition of **Ikposo** conversational speech. It was evaluated on the WAXAL test set (spontaneous, image-prompted speech) and partially on FLEURS (read speech). It is suitable for research and low-resource ASR applications. It is not recommended for high-stakes production use without further validation.
39
 
40
+ ## Training Data
41
 
42
+ Fine-tuned on the [WAXAL corpus](https://huggingface.co/waxal-benchmarking), a large-scale dataset of transcribed, image-prompted spontaneous speech across 19 African languages recorded in participants' natural environments. The Ikposo training split contains conversational speech across diverse speakers. Data is released under CC-BY 4.0.
43
 
44
+ ## Usage
45
 
46
+ ```python
47
+ from transformers import pipeline
48
+
49
+ asr = pipeline("automatic-speech-recognition",
50
+ model="waxal-benchmarking/whisper-small-waxal-kpo")
51
+ result = asr("audio.wav")
52
+ print(result["text"])
53
+ ```
54
+
55
+ ## Test Set Performance (WAXAL Benchmark)
56
+
57
+ Evaluated on the filtered WAXAL test set (duration >= 1.5s, speech rate >= 4 WPS).
58
+
59
+ | Metric | Score |
60
+ |---|---|
61
+ | **WER** | 77.5% |
62
+ | **CER** | 26.5% |
63
+
64
+ Full benchmark results across all 19 languages and 6 models are reported in the WAXAL ASR Benchmark paper (citation below).
65
 
66
  ## Training procedure
67
 
 
97
  - Pytorch 2.10.0+cu128
98
  - Datasets 4.0.0
99
  - Tokenizers 0.22.2
100
+
101
+
102
+ ## Citation
103
+
104
+ ```bibtex
105
+ @article{waxalnet2026,
106
+ title = {The WAXAL ASR Benchmark: Fine-Tuned Edge Models Across 19 African Languages},
107
+ author = {Olufemi, Victor Tolulope and Babatunde, Oreoluwa and Njema, Ramsey and
108
+ Gbotemi, Bolarinwa and Yen, Wanchi Lucia and Uzodinma, John and
109
+ Ajayi, Sunday and Williams, Oluwademilade and Moshood, Kausar and
110
+ Anyaele, Innocent Elendu and Arefaine, Akebert Tesfahunegn and
111
+ Hunzwi, Candace and Daniel, Wongel Dawit and Namuganga, Emmilly Immaculate and
112
+ Kadima, Cleophas and Bahizire, Athanase Biluge and Ranaivoson, Onitsiky and
113
+ Aaron, Emmanuel and Ladislaus, Nicholaus Dismas and Muhammed, Idris and
114
+ Simenya, Jonathan Enoch and Koome, Martin and Endaylalu, Matewos Tegete and
115
+ Adeyemo, Peter Ifeoluwa and Birindwa, Hondi Prisca and Eze-Mbey, Ukachi Agnes and
116
+ Oduro-Yeboah, Yacoba and Aremu, Toluwani and Adjovi, Pericles and
117
+ Ngueajio, Mikel K and Mitra, Prasenjit},
118
+ year = {2026},
119
+ note = {Preprint coming soon}
120
+ }
121
+ ```
122
+
123
+ ## Authors
124
+
125
+ Victor Tolulope Olufemi · Oreoluwa Babatunde · Ramsey Njema · Bolarinwa Gbotemi · Wanchi Lucia Yen · John Uzodinma · Sunday Ajayi · Oluwademilade Williams · Kausar Moshood · Innocent Elendu Anyaele · Akebert Tesfahunegn Arefaine · Candace Hunzwi · Wongel Dawit Daniel · Emmilly Immaculate Namuganga · Cleophas Kadima · Athanase Biluge Bahizire · Onitsiky Ranaivoson · Emmanuel Aaron · Nicholaus Dismas Ladislaus · Idris Muhammed · Jonathan Enoch Simenya · Martin Koome · Matewos Tegete Endaylalu · Peter Ifeoluwa Adeyemo · Hondi Prisca Birindwa · Ukachi Agnes Eze-Mbey · Yacoba Oduro-Yeboah · Toluwani Aremu · Pericles Adjovi · Mikel K Ngueajio · Prasenjit Mitra
126
+
127
+ ## Acknowledgements
128
+
129
+ We thank the following contributors for their language expertise and native-speaker evaluation support:
130
+ Ajara Oyinloye, Abubakari Sadic Mohammed, Hafiz Adjei, Aliga Norah Lele, Marie-Louise B. Ndamuso, and Odong Diana.
131
+
132
+ This work was supported by **[Lynguallabs](https://lynguallabs.org/)** (compute, researchers & storage),
133
+ **[Open Token](https://opentoken.global/)** (compute resources), and
134
+ **[CMU Africa](https://www.africa.engineering.cmu.edu/)** (researchers & native speakers).