---
language:
- en
base_model: openai/whisper-tiny
library_name: pytorch
tags:
- audio
- speech
- pronunciation-assessment
- phoneme-level
- ordinal-regression
---

# Phone-Level Accentedness Scorer — E16

This repository contains the exact promoted E16 checkpoint for the
[Phone-Level Accentedness Scoring challenge](https://github.com/aviadarn/Accentedness-Scoring-Challenge).
Given a speech recording and its expected phoneme sequence, the model returns
one continuous `0`–`100` American-English accentedness score per phoneme.

## Try it

- [Open the live Gradio demo](https://aviadarn--phone-accentedness-scorer-web.modal.run)
- [Open the verified Colab notebook](https://colab.research.google.com/github/aviadarn/Accentedness-Scoring-Challenge/blob/main/notebooks/phone_accentedness_colab.ipynb)
- [View the complete source, experiments, and write-up](https://github.com/aviadarn/Accentedness-Scoring-Challenge)
- [Open the project Space](https://huggingface.co/spaces/Aviadara/phone-accentedness-scorer)

The checkpoint uses a frozen Whisper-tiny encoder, constrained CTC phone
alignment, acoustic and alignment features, a bidirectional GRU, and an ordinal
prediction head with per-phone class weights proportional to `n_c^-0.54`.

## Validation results

| Metric | E16 result |
|---|---:|
| Balanced MAE | 21.8496 |
| MAE | 18.0081 |
| Quadratic-weighted kappa | 0.5786 |
| Macro-F1 | 0.5682 |
| Balanced accuracy | 0.6642 |
| Spearman correlation | 0.5583 |

Balanced MAE improved by `0.7248` over the E01 incumbent, with paired
utterance-bootstrap 95% CI `[-1.3549, -0.0812]`. The supplied validation split
has substantial inferred-speaker and prompt overlap with training, so these
numbers must not be interpreted as new-speaker performance.

## Artifact integrity

The model file SHA-256 is:

```text
ead3144c82ab87ad9d6406511c6348a99c944a9f8ac1097756a6a61d78e80338
```

`deployment_manifest.json` binds the promoted checkpoint to its accepted
confirmation evidence and records hashes for every deployed file. The nine
artifact files in this repository are unchanged from `submission/model/` at
Git commit `e05549fc0b8e89fef05d6f111a623b3f397240e1`.

## Usage

This is a custom architecture and is **not** compatible with
`transformers.AutoModel`. The simplest supported path is the linked Colab
notebook. For local use, clone the source repository, install the environment
from `submission/`, download this model snapshot, and point inference at it:

```bash
git clone https://github.com/aviadarn/Accentedness-Scoring-Challenge.git
cd Accentedness-Scoring-Challenge/submission
uv sync --python 3.11
uv run python -c 'from huggingface_hub import snapshot_download; print(snapshot_download("Aviadara/phone-accentedness-scorer"))'
export ACCENT_MODEL_DIR=/path/printed/by/the/previous/command
uv run python -c 'from inference import score_phonemes; print(score_phonemes("sample.wav", ["n", "oʊ", "s", "ɝ"]))'
```

The public API is `score_phonemes(audio_path, phonemes) -> list[float]`. Input
phones must use the model vocabulary recorded in `accent_model_config.json`.

## Intended use and limitations

This artifact is for research, challenge evaluation, and formative pronunciation
feedback. “Native-like” is a subjective annotation target, not a measure of
intelligence, identity, employability, nationality, or general English
proficiency. The challenge data lacks documented speaker IDs, rater agreement,
consent, and authoritative continuous calibration. Do not use the model for
high-stakes decisions. The private challenge audio and labels are not included
in this repository.