---
language:
- ar
license: apache-2.0
base_model: facebook/wav2vec2-large-xlsr-53
datasets:
- tunis-ai/arabic_speech_corpus
tags:
- automatic-speech-recognition
- arabic
- phoneme
- ctc
- wav2vec2
metrics:
- wer
model-index:
- name: wav2vec2-large-xlsr-53-arabic-phoneme
  results:
  - task:
      type: automatic-speech-recognition
      name: Automatic Speech Recognition
    dataset:
      name: tunis-ai/arabic_speech_corpus
      type: tunis-ai/arabic_speech_corpus
      split: test
    metrics:
    - type: wer
      value: 0.0395
      name: PER (Phoneme Error Rate)
    - type: cer
      value: 0.0293
      name: CER
---

# 🎙️ Arabic Phoneme ASR — wav2vec2-large-xlsr-53

Fine-tuned [`facebook/wav2vec2-large-xlsr-53`](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) for **phoneme-level Arabic speech recognition** using CTC, trained on the [`tunis-ai/arabic_speech_corpus`](https://huggingface.co/datasets/tunis-ai/arabic_speech_corpus) dataset.

---

## Model Description

This model transcribes Arabic speech directly into a sequence of phoneme tokens rather than graphemes or words. It is intended for linguistic analysis, pronunciation research, and downstream tasks that benefit from sub-word acoustic representations.

The base model's convolutional feature encoder is frozen; only the transformer layers and a freshly initialized CTC head (61 phoneme tokens) are fine-tuned.

---

## Training Details

| Parameter | Value |
|---|---|
| Base model | `facebook/wav2vec2-large-xlsr-53` |
| Dataset | `tunis-ai/arabic_speech_corpus` |
| Target label | `phonetic` column (phoneme sequences) |
| Sample rate | 16 kHz |
| Epochs | 40 (early stopping, patience = 10) |
| Effective batch size | 16 (2 × 8 gradient accumulation) |
| Learning rate | 1e-4 |
| Warmup steps | 500 |
| Precision | fp16 |

### Data Splits

The original train/test splits were pooled and re-split deterministically (seed 42):

| Split | Samples | % |
|---|---|---|
| Train | 1,530 | 80% |
| Validation | 191 | 10% |
| Test | 192 | 10% |

### Preprocessing

- Audio resampled to 16 kHz
- Phoneme sequences cleaned: punctuation removed, lowercased, whitespace normalised
- Gemination (doubled tokens like `bb`, `dd`) mapped to their single form to reduce vocabulary size
- Noise/distortion marker `dist` mapped to `sil`
- Phonemes joined with `|` (word delimiter token) before tokenisation to prevent character-level splitting

---

## Vocabulary

The model uses a 59-phoneme vocabulary plus three special tokens (`|`, `[UNK]`, `[PAD]`), for a total of **61 tokens**.

Phoneme inventory after normalisation:

```
$ $$ * ** - ^ ^^ a a' aa aa' ah b d e ee f g h i i0 i0' i1 i1'
ii0 ii0' ii1 ii1' j k l m n p pp q r s sh sil t th u u0 u0' u1
u1' uu0 uu0' uu1 uu1' v w x y z
```

---

## Evaluation Results

Evaluated on the held-out test set (192 samples):

| Metric | Score |
|---|---|
| **PER** (Phoneme Error Rate) | **3.95%** |
| **CER** (Character Error Rate) | **2.93%** |
| Test Loss | 0.0505 |

### Training Curve Summary

The model converged rapidly:
- Epochs 1–4: high loss, near-random output
- Epoch 5: PER drops from ~95% to ~38%
- Epoch 6: PER drops to 7.5%
- Epochs 7–40: steady improvement to final PER of 3.95%

---

## Error Analysis

The most common error types (on 100 test samples) are **vowel length and stress confusions** — the model occasionally substitutes short vowels for long ones and vice versa. No consonant confusions appear in the top-15 substitutions.

**Top substitutions (reference → prediction):**

| Reference | Predicted | Count |
|---|---|---|
| `a` | `aa` | 9× |
| `a` | `a'` | 7× |
| `a'` | `a` | 5× |
| `aa'` | `aa` | 5× |
| `i0` | `ii0` | 5× |

**Most common deletion:** `sil` boundary tokens (96× in 100 samples). These are silence/pause markers at utterance boundaries; their deletion has minimal impact on phonemic content.

---

## Usage

```python
from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
import torch

model_path = "path/to/wav2vec2-arabic-phoneme-final"

processor = Wav2Vec2Processor.from_pretrained(model_path)
model = Wav2Vec2ForCTC.from_pretrained(model_path)
model.eval()

# audio_array: numpy array, 16 kHz
inputs = processor(audio_array, sampling_rate=16000, return_tensors="pt", padding=True)

with torch.no_grad():
    logits = model(**inputs).logits

predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.decode(predicted_ids[0])

# Convert delimiter back to spaces
phonemes = transcription.replace("|", " ").strip()
print(phonemes)
# e.g. → "sil w a r a' jj a h a tt a q r ii0' r u0 ..."
```

---

## Limitations

- Trained on a single corpus (1,913 utterances); performance on out-of-domain speakers or dialects may degrade.
- The model over-generates `sil` and `$` tokens at utterance boundaries; strip trailing occurrences in post-processing if needed.
- Vowel length and stress diacritics (`'`, `0`, `1`) represent the most frequent error class — tasks that do not distinguish length/stress can benefit from collapsing these distinctions in post-processing.
- The dataset's `text` column contains Buckwalter transliteration, not Arabic Unicode; this model is trained solely on the `phonetic` column and produces phoneme sequences, not orthographic Arabic script.

---

## Citation

If you use this model, please cite the base model and dataset:

```bibtex
@misc{conneau2020unsupervised,
  title={Unsupervised Cross-lingual Representation Learning for Speech Recognition},
  author={Conneau, Alexis and others},
  year={2020},
  eprint={2006.13979},
  archivePrefix={arXiv}
}
```

---

## License

Inherits the license of `facebook/wav2vec2-large-xlsr-53`. Please review the [original model card](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) before use in commercial applications.