omniasr-llm-300m-waxal-bantu

A Bantu (Niger-Congo) multilingual fine-tune of Meta's Omnilingual ASR (wav2vec2_llama, 300M-parameter shared wav2vec2 encoder + autoregressive Llama-style decoder), trained jointly on 6 WAXAL languages. Part of the WAXAL ASR benchmark's training-granularity study (monolingual vs. language-family vs. all-19 pooling).

• Base model: facebook/omniASR-LLM-300M
• Languages (6): Luganda, Nyankole, Shona, Lingala, Masaaba, Soga
• Macro-averaged WER (this model): 31.6%  |  monolingual baselines: 30.6%

Training data

Fine-tuned on the pooled train splits of the WAXAL corpus for these languages (16 kHz mono; transcripts NFC-normalized and lower-cased, punctuation removed, phonemic diacritics/tone marks preserved). Total: 53,830 clips / 308.9 hours.

Language	Train clips	Train hours
Luganda	5,455	37.3
Nyankole	6,783	40.8
Shona	14,109	79.7
Lingala	14,399	71.9
Masaaba	6,865	39.3
Soga	6,219	40.0

Training procedure

Fine-tuned with the Omnilingual-ASR wav2vec2_llama recipe (fairseq2) on 2× NVIDIA H200 (DistributedDataParallel). All granularity conditions use an identical budget so the only variable is the language mixture.

Hyperparameter	Value
Base checkpoint	facebook/omniASR-LLM-300M
Tokenizer	omniASR_tokenizer_v1 (SentencePiece, 9,812 units)
Training steps	5,000
Optimizer	AdamW
Learning rate	5e-5
Weight decay	0.01
Batch size	3,000,000 audio tokens/batch (dynamic)
Gradient accumulation	4
Precision	bfloat16 mixed
Audio length filter	0.5 s – 30 s
Hardware	2× H200 (DDP)

Evaluation

Scored on each language's held-out test split (utterances ≥ 1.5 s, matching the benchmark's filtered-test protocol). WER and CER computed with jiwer on NFC-normalized, lower-cased text (diacritics preserved). The Monolingual WER column is the corresponding per-language model (omniasr-llm-300m-waxal-<iso>) evaluated identically, for a same-protocol comparison.

Language	Code	WER	CER	Monolingual WER	Δ vs mono
Luganda	lug_Latn	11.6	2.8	10.9	+0.6
Nyankole	nyn_Latn	33.6	8.7	32.3	+1.3
Shona	sna_Latn	22.2	3.9	21.6	+0.6
Lingala	lin_Latn	31.5	13.2	30.4	+1.1
Masaaba	myx_Latn	47.0	10.4	45.5	+1.4
Soga	xog_Latn	44.0	8.7	42.8	+1.2

Usage

# pip install git+https://github.com/facebookresearch/omnilingual-asr.git
from pathlib import Path
import torch
from huggingface_hub import snapshot_download
from fairseq2.data.tokenizers.hub import load_tokenizer
from omnilingual_asr.models.inference.pipeline import ASRInferencePipeline
from omnilingual_asr.models.wav2vec2_llama.hub import get_wav2vec2_llama_model_hub

ckpt = snapshot_download("waxal-benchmarking/omniasr-llm-300m-waxal-bantu")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
dtype = torch.bfloat16 if device.type == "cuda" else torch.float32

hub = get_wav2vec2_llama_model_hub()
model = hub.load_custom_model(Path(ckpt) / "model", hub.get_arch_config("300m"), device=device, dtype=dtype)
tokenizer = load_tokenizer("omniASR_tokenizer_v1")

pipe = ASRInferencePipeline(model_card=None, model=model, tokenizer=tokenizer, device=device, dtype=dtype)
# pass the target language's Omnilingual token, e.g. Luganda -> "lug_Latn"
texts = pipe.transcribe(["your_audio.flac"], lang=["lug_Latn"])
print(texts)

Language tokens for this model: Luganda lug_Latn, Nyankole nyn_Latn, Shona sna_Latn, Lingala lin_Latn, Masaaba myx_Latn, Soga xog_Latn. Audio should be mono 16 kHz (the pipeline resamples if needed); keep clips under 40 s.

Checkpoint format

Native fairseq2 sharded checkpoint (model/pp_00/tp_00/sdp_00.pt + model.yaml) — not a transformers model, so AutoModel will not load it. Load with omnilingual_asr / fairseq2 as shown above.

Citation

Part of the WAXAL ASR Benchmark (arXiv:2606.02375).

@article{waxalnet2026,
  title  = {The WAXAL ASR Benchmark: Fine-Tuned Edge Models Across 19 African Languages},
  author = {Olufemi, Victor Tolulope and Babatunde, Oreoluwa and Njema, Ramsey and others},
  year   = {2026},
  note   = {arXiv preprint arXiv:2606.02375}
}

Acknowledgements

Supported by Lynguallabs (compute, researchers & storage), Open Token (compute), and CMU Africa (researchers & native speakers).