WAXAL LLM-ASR 1B fine-tune (Lingala, Shona, Luganda)
Fine-tune of Meta's omniASR_LLM_1B (Omnilingual ASR) on Lingala, Shona,
and Luganda for the WAXAL Hackathon (Zindi).
Full project writeup, training pipeline, and inference script: https://github.com/osinkolu/waxal-hackathon
Checkpoints in this repo
Each step_N/ directory is a full fairseq2 training checkpoint (model +
optimizer + trainer + data-reader state) -- resumable for continued
training, not just inference. For inference, only step_N/model/pp_00/tp_00/sdp_00.pt
(a plain PyTorch state_dict) is needed.
Best: step_124000 (WER ~7.6% on a small in-loop validation sample;
step_122500 scored marginally better at the time but wasn't checkpointed).
Important: output format
This model outputs lowercase, unpunctuated "spoken form" text --
omniASR_tokenizer_v1's vocabulary has no entries for uppercase letters or
for most punctuation (. , ! ? ; : etc. all map to <unk>; confirmed via
direct tokenizer round-trip tests). This isn't a training shortfall, it's
what the tokenizer can represent. If you need written-form output, run the
transcription through a separate punctuation-restoration tool.
Usage
See inference/inference.py in the GitHub repo linked above -- it
downloads a checkpoint from this repo and transcribes audio standalone
(Colab-friendly, no training-pod setup needed). Pass nbest=5, compression_window=100, compression_threshold=4.0 for beam search (the
inference pipeline's own default of nbest=1 gives noticeably worse
results -- less room to recover from an uncertain first token).
Training summary
- Base:
omniASR_LLM_1B(full pretrained checkpoint, not a fresh decoder) - Language conditioning:
lang_embeddings_p=0.001(near-zero, per the paper's Table 22 ablation, matching WAXAL Phase 2's zero-metadata inference constraint) - 128,000 total training steps across several continuations, driven by real measured throughput and validation trend rather than a pre-committed schedule
- Two tokenizer-vocabulary bugs found and fixed mid-training (capitalization, then punctuation -- see the GitHub writeup for the full story and the exact before/after WER impact of each)