---
language:
- kln
license: cc-by-4.0
library_name: transformers
pipeline_tag: automatic-speech-recognition
base_model: nvidia/parakeet-tdt-0.6b-v3
tags:
- automatic-speech-recognition
- kalenjin
- kipsigis
- nandi
- low-resource
- african-languages
- parakeet
- fastconformer
- tdt
- fine-tune
metrics:
- cer
- wer
model-index:
- name: parakeet-tdt-0.6b-kalenjin
  results:
  - task:
      type: automatic-speech-recognition
      name: Automatic Speech Recognition
    dataset:
      type: Anv-ke/Kalenjin
      name: KaleBench-ASR (held-out unscripted, n=198)
    metrics:
    - type: cer
      value: 0.2604
      name: CER (greedy)
    - type: wer
      value: 0.7563
      name: WER (greedy)
---

# parakeet-tdt-0.6b-kalenjin

> ⚠️ **Experimental — not recommended for use.** This Parakeet fine-tune was a research experiment.
> On real-world Kalenjin audio it is less reliable than the recommended model,
> [`Tonykip/whisper-kalenjin-v3-turbo`](https://huggingface.co/Tonykip/whisper-kalenjin-v3-turbo)
> — **use that instead.** This repo is kept for reproducibility and research transparency.

**An open automatic speech recognition (ASR) model for Kalenjin** — a Highland Nilotic
language of Kenya's Rift Valley (~2M speakers). This is an **honest fine-tune** of NVIDIA's
[`parakeet-tdt-0.6b-v3`](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) (a 0.6B-parameter
FastConformer encoder with a Token-and-Duration Transducer decoder). **It is not trained from
scratch**, and it does **not** beat the strongest closed baseline overall — see *Results* for the
honest comparison.

Adaptation rebuilds the SentencePiece tokenizer to Kalenjin orthography (vocab 2048, BPE,
**preserving the velar-nasal apostrophe in `ng'`** — a real Kalenjin letter, not punctuation),
reinitializes the decoder embedding + joint head to the new vocabulary, and keeps the acoustic
encoder warm. Covered dialects: **Kipsigis** and **Nandi**.

## Results (KaleBench-ASR, held-out unscripted, n=198)

Character Error Rate (CER) is the primary metric — Kalenjin has no single agreed orthography,
which inflates WER. 95% confidence intervals are paired bootstrap (B=2000, seed=1234).

| System | CER | 95% CI | WER | Kipsigis / Nandi CER |
|---|---|---|---|---|
| `parakeet-tdt-0.6b-v3` (base, 0 Kalenjin) | 0.534 | [0.514, 0.558] | 1.00 | 0.524 / 0.661 |
| Whisper-large-v3-turbo+LoRA (closed incumbent, **beam-5**) | **0.211** | [0.199, 0.224] | 0.656 | 0.212 / 0.196 |
| **this model — v3 (full-corpus, greedy)** | **0.260** | [0.251, 0.270] | 0.756 | 0.260 / 0.267 |

**Honest reading:**
- This is the **best _open_ Kalenjin ASR model**, and it is **dialect-balanced** (Kipsigis 0.260 ≈ Nandi 0.267), unlike the incumbent's Nandi tilt.
- Under **matched greedy decoding** it leads the closed incumbent decisively (the incumbent's *greedy* CER is 0.483). The incumbent's headline 0.211 uses **beam-5** search; Parakeet-TDT in 🤗 Transformers currently supports **greedy decoding only**, so the comparison is asymmetric in decoding budget.
- The reported scores are **comparable but not yet a formally citable KaleBench score**: all 198 clips currently carry `needs_native_validation=true`. Numbers are reproducible to the digit and use the same gold + normalizer for every system; native by-ear validation (in progress) is what makes them citable.

## Usage

Parakeet-TDT's joint head emits token logits **and** duration logits; during greedy decoding you
must mask the duration columns so the argmax can never select a duration slot as a token.

```python
import torch, soundfile as sf
from transformers import (ParakeetForTDT, ParakeetProcessor, ParakeetTokenizer,
                          LogitsProcessorList)

repo = "Tonykip/parakeet-tdt-0.6b-kalenjin"
model = ParakeetForTDT.from_pretrained(repo, dtype=torch.bfloat16).to("cuda").eval()
processor = ParakeetProcessor.from_pretrained(repo)
processor.tokenizer = ParakeetTokenizer.from_pretrained(repo)   # the rebuilt Kalenjin tokenizer

V = model.config.vocab_size
class MaskDurations:                      # keep argmax inside the real vocab
    def __call__(self, input_ids, scores):
        if scores.shape[-1] > V:
            scores[..., V:] = float("-inf")
        return scores

wav, sr = sf.read("clip.wav")             # mono; resample to 16 kHz if needed
feats = processor.feature_extractor(wav, sampling_rate=16000, return_tensors="pt")
feats = {k: (v.to("cuda", dtype=model.dtype) if torch.is_floating_point(v) else v.to("cuda"))
         for k, v in feats.items()}
with torch.no_grad():
    out = model.generate(**feats, logits_processor=LogitsProcessorList([MaskDurations()]))
seq = out.sequences if hasattr(out, "sequences") else out
print(processor.tokenizer.decode(seq[0], skip_special_tokens=True))
```

## Training

- **Base:** `nvidia/parakeet-tdt-0.6b-v3` (FastConformer + TDT, 0.6B, CC-BY-4.0).
- **Data:** [`Anv-ke/Kalenjin`](https://huggingface.co/datasets/Anv-ke/Kalenjin) (AfriVoices-KE; Wanzare et al., arXiv:2604.08448) — full training corpus, 82,378 clips, streamed (~3.4 passes over clips ≤32s).
- **Tokenizer:** rebuilt Kalenjin SentencePiece (BPE, vocab 2048; `ng'` preserved as a piece); decoder embedding + joint head reinitialized to the new vocabulary; encoder kept warm.
- **Recipe:** L40S (Modal), bf16, gradient checkpointing, per-device batch 8 × grad-accum 4, lr 1e-4, `max_steps=7723`. A TDT-loss correction (`sigma=0.05`; the 🤗 default `0.0` collapses training to empty output) and int64 gather indices were applied.

## Intended use & limitations

- **Intended:** transcription of spoken Kalenjin (Kipsigis / Nandi), research on low-resource African ASR, and as a baseline for the KaleBench-ASR benchmark.
- **Limitations:** only Kipsigis + Nandi are covered (no Marakwet/Sabaot/Tugen/Pokot evaluation); greedy decoding only (no beam in Transformers for TDT); code-switched English/Swahili spans are transcribed phonetically rather than preserved; training transcripts are themselves unvalidated, which likely caps accuracy. Not intended for surveillance, profiling, or any non-consensual use.

## License & attribution

This fine-tune and rebuilt tokenizer are released under **CC-BY-4.0**, inheriting the base model's
license. The training audio comes from `Anv-ke/Kalenjin` (CC-BY-4.0, gated) — please cite
AfriVoices-KE (arXiv:2604.08448) in any derivative work. Built by **Tony Kipkemboi**, a native
Kalenjin speaker.

## Citation

```bibtex
@misc{kipkemboi2026parakeetkalenjin,
  title  = {Open Kalenjin Automatic Speech Recognition: Fine-tuned Parakeet Models and the KaleBench-ASR Benchmark},
  author = {Kipkemboi, Tony},
  year   = {2026},
  note   = {Preprint}
}
```