PrinceAlhassanNasamu/tekyerema-audio-native-4k

Tɛkyerɛma-1 audio-native, 4,000 clips (arm 3 v2)

Author: Prince Nasamu Alhassan

Overview

Qwen2-Audio-7B-Instruct + LoRA, trained on 4,000 synthesised clips instead of the 1,200 used by tekyerema-audio-native.

Measured on utterances it has genuinely never seen — 60 rows whose text is absent from the training slice, after skipping 139 candidates that appeared there verbatim:

path	correct
audio → ASR → tekyerema-1-native-tool	5/52 · 9.6%
audio → tekyerema-audio-native (1,200 clips)	23/52 · 44.2%
audio → this model (4,000 clips)	45/52 · 86.5%

An earlier version of this card said 92.3%, and that was memorisation. The corpus has 8,488 rows and 2,943 distinct utterances — 65% duplicates, single sentences repeated up to 25 times — so holding out by ROW INDEX left 40 of 60 test rows appearing word for word in training, and the bigger model looked better mainly because it had seen more of the test. AGENT/eval_audio_native.py now splits by utterance and prints what it skipped.

The conclusion survived the correction and got cleaner: more data genuinely helps this architecture (44.2 → 86.5), and it genuinely beats the cascade — by more than the leaky measurement showed, because removing the leaked rows cost the cascade more than it cost this model.

Training loss moved 4.649 to 4.424 between the two runs, which is worth noting on its own: loss on a generative model averages over every token and tracked neither the inflated score nor the real one.

Use it

import torch, soundfile as sf
from peft import PeftModel
from transformers import AutoProcessor, Qwen2AudioForConditionalGeneration

BASE = "Qwen/Qwen2-Audio-7B-Instruct"
proc  = AutoProcessor.from_pretrained("PrinceAlhassanNasamu/tekyerema-audio-native-4k")
model = PeftModel.from_pretrained(
    Qwen2AudioForConditionalGeneration.from_pretrained(
        BASE, dtype=torch.bfloat16, device_map="auto"),
    "PrinceAlhassanNasamu/tekyerema-audio-native-4k").eval()

convo = [{"role": "system",
          "content": "Convert the spoken command into one tool call as JSON."},
         {"role": "user",
          "content": [{"type": "audio", "audio_url": "x.wav"}]}]
prompt = proc.apply_chat_template(convo, add_generation_prompt=True,
                                  tokenize=False)

wav, sr = sf.read("command.wav", dtype="float32")
inp = proc(text=prompt, audio=[wav], sampling_rate=sr,
           return_tensors="pt").to(model.device)
with torch.no_grad():
    out = model.generate(**inp, max_new_tokens=48, do_sample=False)
print(proc.batch_decode(out[:, inp["input_ids"].shape[1]:],
                        skip_special_tokens=True)[0])

Training data

Trained on the Ghana Speech dataset and related Ghanaian corpora, licensed CC BY-NC 4.0.

Intended use & license

Non-commercial use only (CC BY-NC 4.0). This is inherited from the training data and required by the terms under which the compute was granted: models trained in that window are non-commercial by condition of access, not by inference.

Limitations, stated plainly

• Dagbani had no recogniser of its own for this whole project, and the reason given for that was wrong. Every card here said "one fine-tuning session on 74 validation rows would not change that". Those 74 rows are the eng-dag machine-translation validation split. The Dagbani speech data in this same account is waxal_dag: 13,228 training rows, 1,750 validation rows, ~71 hours, 1,041 speakers with the largest at 1% — more data and better speaker diversity than Ewe, which produced a working 42.19 WER recogniser. A number was carried across from a translation table into a speech claim, and then repeated on every model card on the account. It is training now, on 2026-08-31. Until it is scored, the honest statement is that Dagbani's best available recogniser scores 86.6 WER and nobody had tried fine-tuning on the data already in hand.
• Evaluation is on read and machine-translated text. No recordings of people speaking agent commands in these languages exist. Numbers measured this way are optimistic about phrasing and pessimistic about code-switching, and should not be read as field performance.
• Research work from a hackathon entry, not a supported product.

The rest of the family

Recognisers

• whisper-large-v3-turbo-tekyerema-eng-foundation — Ghanaian English ASR — course 1 (foundation)
• kusaal-whisper-small-lora — Kusaal ASR (Whisper-small LoRA, superseded)
• kasa42-asr — KASA-42 (Kusaal, third-party export)
• tekyerema-asr-ctc — Twi ASR (w2v-BERT CTC)
• tekyerema-asr-mms-ewe — Ewe ASR (MMS adapter)
• tekyerema-asr-mms-dag — Dagbani ASR (MMS adapter)
• tekyerema-asr-mms-hau — Hausa ASR (MMS adapter)
• tekyerema-asr-mms-kus — Kusaal ASR (MMS adapter)
• whisper-large-v3-turbo-tekyerema-eng — Ghanaian English ASR (Whisper large-v3-turbo)

Voices

• tekyerema-tts-twi — Twi TTS (VITS)
• tekyerema-tts-kus — Kusaal TTS (VITS)
• tekyerema-tts-ewe — Ewe TTS (VITS)
• tekyerema-tts-hau — Hausa TTS (VITS)
• tekyerema-tts-eng — Ghanaian English TTS (VITS)

Agent models

• tekyerema-1-reply — Tɛkyerɛma-1 reply adapter (arm ①)
• tekyerema-1-native-reply — Tɛkyerɛma-1 reply adapter (arm ②)
• tekyerema-1-tool — Tɛkyerɛma-1 tool adapter (arm 1)
• tekyerema-audio-native — Tɛkyerɛma-1 audio-native (arm 3)
• tekyerema-audio-native-4k — Tɛkyerɛma-1 audio-native, 4,000 clips (arm 3 v2)
• tekyerema-1-native-tool — Tɛkyerɛma-1 tool adapter (arm 2)

Translation

• tekyerema-nllb600m-v1 — Tɛkyerɛma MT v1 (NLLB-600M)
• kusaal-nllb-600M — Kusaal MT specialist (NLLB-600M)

Routing

• tekyerema-intent-afroxlmr — Intent classifier (AfroXLMR)

Acknowledgements

Compute resources provided by AI Skills and Compute Africa (AISCA). Trained on the Ghana NLP H200 GPU. Please keep derivatives non-commercial and share improvements back with the Ghana NLP community (ghananlpcommunity).