Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,98 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-nc-4.0
|
| 3 |
+
language:
|
| 4 |
+
- sna
|
| 5 |
+
base_model: facebook/omniASR-LLM-300M
|
| 6 |
+
tags:
|
| 7 |
+
- automatic-speech-recognition
|
| 8 |
+
- african-languages
|
| 9 |
+
- waxal
|
| 10 |
+
- omnilingual-asr
|
| 11 |
+
- fairseq2
|
| 12 |
+
datasets:
|
| 13 |
+
- google/WaxalNLP
|
| 14 |
+
metrics:
|
| 15 |
+
- wer
|
| 16 |
+
- cer
|
| 17 |
+
---
|
| 18 |
+
|
| 19 |
+
# omniasr-llm-300m-waxal-sna
|
| 20 |
+
|
| 21 |
+
Fine-tune of Meta's **Omnilingual ASR** (wav2vec2_llama (autoregressive LLM decoder)) on **Shona** conversational
|
| 22 |
+
speech from the [WAXAL corpus](https://huggingface.co/datasets/google/WaxalNLP),
|
| 23 |
+
part of a 19-language WAXAL×Omnilingual ASR benchmark.
|
| 24 |
+
|
| 25 |
+
## Results (held-out WAXAL test set)
|
| 26 |
+
|
| 27 |
+
| Metric | Score |
|
| 28 |
+
|---|---|
|
| 29 |
+
| WER | 21.6% |
|
| 30 |
+
| UER (unit/char error rate) | 3.7% |
|
| 31 |
+
|
| 32 |
+
Fine-tuned for 5,000 steps from [`facebook/omniASR-LLM-300M`](https://huggingface.co/facebook/omniASR-LLM-300M) on 2×H200.
|
| 33 |
+
|
| 34 |
+
## Usage
|
| 35 |
+
|
| 36 |
+
```python
|
| 37 |
+
# pip install git+https://github.com/facebookresearch/omnilingual-asr.git
|
| 38 |
+
from pathlib import Path
|
| 39 |
+
import torch
|
| 40 |
+
from huggingface_hub import snapshot_download
|
| 41 |
+
from fairseq2.data.tokenizers.hub import load_tokenizer
|
| 42 |
+
from omnilingual_asr.models.inference.pipeline import ASRInferencePipeline
|
| 43 |
+
from omnilingual_asr.models.wav2vec2_llama.hub import get_wav2vec2_llama_model_hub as get_hub
|
| 44 |
+
|
| 45 |
+
ckpt = snapshot_download("waxal-benchmarking/omniasr-llm-300m-waxal-sna")
|
| 46 |
+
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
| 47 |
+
dtype = torch.bfloat16 if device.type == "cuda" else torch.float32
|
| 48 |
+
|
| 49 |
+
hub = get_hub()
|
| 50 |
+
config = hub.get_arch_config("300m")
|
| 51 |
+
model = hub.load_custom_model(Path(ckpt) / "model", config, device=device, dtype=dtype)
|
| 52 |
+
tokenizer = load_tokenizer("omniASR_tokenizer_v1")
|
| 53 |
+
|
| 54 |
+
pipe = ASRInferencePipeline(model_card=None, model=model, tokenizer=tokenizer, device=device, dtype=dtype)
|
| 55 |
+
texts = pipe.transcribe(["your_audio.flac"], lang=["sna_Latn"]) # LLM: condition on the Shona token
|
| 56 |
+
print(texts)
|
| 57 |
+
```
|
| 58 |
+
|
| 59 |
+
Audio should be mono 16 kHz (the pipeline resamples if needed); `.transcribe()` also accepts a
|
| 60 |
+
list of raw bytes or `np.int8` arrays. Keep clips under 40 s.
|
| 61 |
+
|
| 62 |
+
## Checkpoint format
|
| 63 |
+
|
| 64 |
+
Native **fairseq2** sharded checkpoint (`model/pp_00/tp_00/sdp_00.pt` + `model.yaml`) —
|
| 65 |
+
**not** a `transformers` model, so `AutoModel` will not load it. Load with the
|
| 66 |
+
[`omnilingual_asr`](https://github.com/facebookresearch/omnilingual-asr) / `fairseq2`
|
| 67 |
+
libraries, pointing the ASR recipe's `model.path` at the downloaded `model/` directory
|
| 68 |
+
with `model.family=wav2vec2_llama`, `model.arch=300m`, `tokenizer=omniASR_tokenizer_v1`.
|
| 69 |
+
|
| 70 |
+
## Citation
|
| 71 |
+
|
| 72 |
+
This model accompanies the **WAXAL ASR Benchmark** ([arXiv:2606.02375](https://arxiv.org/abs/2606.02375)).
|
| 73 |
+
|
| 74 |
+
```bibtex
|
| 75 |
+
@article{waxalnet2026,
|
| 76 |
+
title = {The WAXAL ASR Benchmark: Fine-Tuned Edge Models Across 19 African Languages},
|
| 77 |
+
author = {Olufemi, Victor Tolulope and Babatunde, Oreoluwa and Njema, Ramsey and
|
| 78 |
+
Gbotemi, Bolarinwa and Yen, Wanchi Lucia and Uzodinma, John and
|
| 79 |
+
Ajayi, Sunday and Williams, Oluwademilade and Moshood, Kausar and
|
| 80 |
+
Anyaele, Innocent Elendu and Arefaine, Akebert Tesfahunegn and
|
| 81 |
+
Hunzwi, Candace and Daniel, Wongel Dawit and Namuganga, Emmilly Immaculate and
|
| 82 |
+
Kadima, Cleophas and Bahizire, Athanase Biluge and Ranaivoson, Onitsiky and
|
| 83 |
+
Aaron, Emmanuel and Ladislaus, Nicholaus Dismas and Muhammed, Idris and
|
| 84 |
+
Simenya, Jonathan Enoch and Koome, Martin and Endaylalu, Matewos Tegete and
|
| 85 |
+
Adeyemo, Peter Ifeoluwa and Birindwa, Hondi Prisca and Eze-Mbey, Ukachi Agnes and
|
| 86 |
+
Oduro-Yeboah, Yacoba and Aremu, Toluwani and Adjovi, Pericles and
|
| 87 |
+
Ngueajio, Mikel K and Mitra, Prasenjit},
|
| 88 |
+
year = {2026},
|
| 89 |
+
note = {arXiv preprint arXiv:2606.02375}
|
| 90 |
+
}
|
| 91 |
+
```
|
| 92 |
+
|
| 93 |
+
## Acknowledgements
|
| 94 |
+
|
| 95 |
+
We thank the native-speaker contributors for their language expertise and evaluation support.
|
| 96 |
+
This work was supported by **[Lynguallabs](https://lynguallabs.org/)** (compute, researchers & storage),
|
| 97 |
+
**[Open Token](https://opentoken.global/)** (compute resources), and
|
| 98 |
+
**[CMU Africa](https://www.africa.engineering.cmu.edu/)** (researchers & native speakers).
|