Yemeni ASR

whisper-small-yemeni is a fine-tuned version of openai/whisper-small for automatic speech recognition of Yemeni Arabic speech. The model was fine-tuned on the Yemen subset of UBC-NLP/Casablanca.

Model Details

Model type: Whisper encoder-decoder Transformer
Base model: openai/whisper-small
Task: Automatic Speech Recognition
Language: Arabic
Target speech variety: Yemeni Arabic
Framework: Hugging Face Transformers
Model author: tahaalselwii

Results

The pretrained base model and the fine-tuned model were evaluated on the same held-out custom test split.

Model	Test loss	WER (%)
`openai/whisper-small`	3.6458	84.3240
`tahaalselwii/whisper-small-yemeni`	0.9923	63.4929

The fine-tuned model achieved a 24.70% relative improvement, with WER decreasing from 84.32% to 63.49%, a reduction of 20.83 percentage points.

Usage

pip install -U torch transformers accelerate "datasets[audio]"

import torch
from datasets import Audio, load_dataset
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor

model_id = "tahaalselwii/whisper-small-yemeni"
device = "cuda" if torch.cuda.is_available() else "cpu"

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForSpeechSeq2Seq.from_pretrained(model_id).to(device)
model.eval()

# Load the Yemeni test split
dataset = load_dataset(
    "UBC-NLP/Casablanca",
    "Yemen",
    split="test",
)

# Resample the audio to 16 kHz as required by Whisper
dataset = dataset.cast_column(
    "audio",
    Audio(sampling_rate=16_000),
)

# Select one test sample
example = dataset[0]
audio = example["audio"]

inputs = processor(
    audio["array"],
    sampling_rate=audio["sampling_rate"],
    return_tensors="pt",
    return_attention_mask=True,
)

with torch.no_grad():
    predicted_ids = model.generate(
        input_features=inputs.input_features.to(device),
        attention_mask=inputs.attention_mask.to(device),
        language="ar",
        task="transcribe",
    )

prediction = processor.batch_decode(
    predicted_ids,
    skip_special_tokens=True,
)[0]

print("Reference:", example["transcription"])
print("Prediction:", prediction)

Training and Evaluation Data

The model was trained using the Yemen configuration of UBC-NLP/Casablanca. The dataset snapshot used by the training initially contained 1,606 examples across its available source splits. All available source splits were concatenated, shuffled with seed 42, and divided into a new custom split:

Custom split	Percentage	Examples used
Train	80%	1,284
Validation	10%	161
Test	10%	160

Limitations

Yemeni Arabic speech data is scarce, and no sufficiently large, suitable dataset was available for this work. As a result, the model was trained and evaluated on a limited dataset, including only 160 test examples. The reported results should therefore be considered preliminary, and performance may vary with new speakers and recording conditions.

Citation

If you use this model, please cite the model repository, the Casablanca dataset paper, and the Whisper paper.

Model

@misc{tahaalselwii2026whisper_yemeni,
  author       = {Taha Alselwi},
  title        = {Whisper Small Yemeni},
  year         = {2026}
}

Training Dataset

@article{talafha2024casablanca,
  title   = {Casablanca: Data and Models for Multidialectal Arabic Speech Recognition},
  author  = {Talafha, Bashar and others},
  journal = {arXiv preprint arXiv:2410.04527},
  year    = {2024}
}

Base Model

@article{radford2022robust,
  title   = {Robust Speech Recognition via Large-Scale Weak Supervision},
  author  = {Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
  journal = {arXiv preprint arXiv:2212.04356},
  year    = {2022}
}