Professor commited on
Commit
7f0bd42
·
verified ·
1 Parent(s): 6807d3b

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +98 -0
README.md ADDED
@@ -0,0 +1,98 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-4.0
3
+ language:
4
+ - sna
5
+ base_model: facebook/omniASR-LLM-300M
6
+ tags:
7
+ - automatic-speech-recognition
8
+ - african-languages
9
+ - waxal
10
+ - omnilingual-asr
11
+ - fairseq2
12
+ datasets:
13
+ - google/WaxalNLP
14
+ metrics:
15
+ - wer
16
+ - cer
17
+ ---
18
+
19
+ # omniasr-llm-300m-waxal-sna
20
+
21
+ Fine-tune of Meta's **Omnilingual ASR** (wav2vec2_llama (autoregressive LLM decoder)) on **Shona** conversational
22
+ speech from the [WAXAL corpus](https://huggingface.co/datasets/google/WaxalNLP),
23
+ part of a 19-language WAXAL×Omnilingual ASR benchmark.
24
+
25
+ ## Results (held-out WAXAL test set)
26
+
27
+ | Metric | Score |
28
+ |---|---|
29
+ | WER | 21.6% |
30
+ | UER (unit/char error rate) | 3.7% |
31
+
32
+ Fine-tuned for 5,000 steps from [`facebook/omniASR-LLM-300M`](https://huggingface.co/facebook/omniASR-LLM-300M) on 2×H200.
33
+
34
+ ## Usage
35
+
36
+ ```python
37
+ # pip install git+https://github.com/facebookresearch/omnilingual-asr.git
38
+ from pathlib import Path
39
+ import torch
40
+ from huggingface_hub import snapshot_download
41
+ from fairseq2.data.tokenizers.hub import load_tokenizer
42
+ from omnilingual_asr.models.inference.pipeline import ASRInferencePipeline
43
+ from omnilingual_asr.models.wav2vec2_llama.hub import get_wav2vec2_llama_model_hub as get_hub
44
+
45
+ ckpt = snapshot_download("waxal-benchmarking/omniasr-llm-300m-waxal-sna")
46
+ device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
47
+ dtype = torch.bfloat16 if device.type == "cuda" else torch.float32
48
+
49
+ hub = get_hub()
50
+ config = hub.get_arch_config("300m")
51
+ model = hub.load_custom_model(Path(ckpt) / "model", config, device=device, dtype=dtype)
52
+ tokenizer = load_tokenizer("omniASR_tokenizer_v1")
53
+
54
+ pipe = ASRInferencePipeline(model_card=None, model=model, tokenizer=tokenizer, device=device, dtype=dtype)
55
+ texts = pipe.transcribe(["your_audio.flac"], lang=["sna_Latn"]) # LLM: condition on the Shona token
56
+ print(texts)
57
+ ```
58
+
59
+ Audio should be mono 16 kHz (the pipeline resamples if needed); `.transcribe()` also accepts a
60
+ list of raw bytes or `np.int8` arrays. Keep clips under 40 s.
61
+
62
+ ## Checkpoint format
63
+
64
+ Native **fairseq2** sharded checkpoint (`model/pp_00/tp_00/sdp_00.pt` + `model.yaml`) —
65
+ **not** a `transformers` model, so `AutoModel` will not load it. Load with the
66
+ [`omnilingual_asr`](https://github.com/facebookresearch/omnilingual-asr) / `fairseq2`
67
+ libraries, pointing the ASR recipe's `model.path` at the downloaded `model/` directory
68
+ with `model.family=wav2vec2_llama`, `model.arch=300m`, `tokenizer=omniASR_tokenizer_v1`.
69
+
70
+ ## Citation
71
+
72
+ This model accompanies the **WAXAL ASR Benchmark** ([arXiv:2606.02375](https://arxiv.org/abs/2606.02375)).
73
+
74
+ ```bibtex
75
+ @article{waxalnet2026,
76
+ title = {The WAXAL ASR Benchmark: Fine-Tuned Edge Models Across 19 African Languages},
77
+ author = {Olufemi, Victor Tolulope and Babatunde, Oreoluwa and Njema, Ramsey and
78
+ Gbotemi, Bolarinwa and Yen, Wanchi Lucia and Uzodinma, John and
79
+ Ajayi, Sunday and Williams, Oluwademilade and Moshood, Kausar and
80
+ Anyaele, Innocent Elendu and Arefaine, Akebert Tesfahunegn and
81
+ Hunzwi, Candace and Daniel, Wongel Dawit and Namuganga, Emmilly Immaculate and
82
+ Kadima, Cleophas and Bahizire, Athanase Biluge and Ranaivoson, Onitsiky and
83
+ Aaron, Emmanuel and Ladislaus, Nicholaus Dismas and Muhammed, Idris and
84
+ Simenya, Jonathan Enoch and Koome, Martin and Endaylalu, Matewos Tegete and
85
+ Adeyemo, Peter Ifeoluwa and Birindwa, Hondi Prisca and Eze-Mbey, Ukachi Agnes and
86
+ Oduro-Yeboah, Yacoba and Aremu, Toluwani and Adjovi, Pericles and
87
+ Ngueajio, Mikel K and Mitra, Prasenjit},
88
+ year = {2026},
89
+ note = {arXiv preprint arXiv:2606.02375}
90
+ }
91
+ ```
92
+
93
+ ## Acknowledgements
94
+
95
+ We thank the native-speaker contributors for their language expertise and evaluation support.
96
+ This work was supported by **[Lynguallabs](https://lynguallabs.org/)** (compute, researchers & storage),
97
+ **[Open Token](https://opentoken.global/)** (compute resources), and
98
+ **[CMU Africa](https://www.africa.engineering.cmu.edu/)** (researchers & native speakers).