---
license: apache-2.0
pipeline_tag: automatic-speech-recognition
library_name: executorch
tags:
  - arm
  - arm-optimized
  - premium-smartphone
  - executorch
base_model: openai/whisper-small
base_model_relation: quantized
---

# Whisper Small optimized for Arm-based Premium Smartphone

Whisper Small, an encoder-decoder Transformer for automatic speech recognition, quantized to INT8 and exported to ExecuTorch for on-device inference on Arm-based Premium Smartphone devices.

## Summary

This repository contains an Arm-optimized version of openai/whisper-small for automatic speech recognition. The model is provided in ExecuTorch (`.pte`) format, targeting Premium Smartphone systems.

This version is intended to demonstrate efficient inference on Arm-based platforms while preserving the original model's intended behavior. Arm has evaluated this model on LibriSpeech ASR and measured performance on Vivo X300.

**Key results**

| Area | Result |
|---|---|
| Model format | ExecuTorch (`.pte`) |
| Target device class | Premium Smartphone |
| Reference device | Vivo X300 (C1-Ultra, C1-Premium, C1-Pro; Android 16 / OriginOS 6) |
| Primary performance result | 7802.5 ms p50 latency, RTFx 0.90 |
| Accuracy result | Normalised WER 3.41%, CER 1.29% |
| Size / memory result | 395.05 MB, 2.72 x smaller than the baseline (1074.76 MB) |

## Original model

| Field | Value |
|---|---|
| Original model | openai/whisper-small |
| Original source | Hugging Face |
| Original developer | OpenAI |
| Original model card | [openai/whisper-small](https://huggingface.co/openai/whisper-small) |
| Original license | [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) |

## Model files

| File | Description |
|---|---|
| `whisper_small_vivo_executorch_optimized.pte` | Arm-optimized model for deployment |
| `whisper_preprocessor.pte` | ExecuTorch module that computes the log-mel spectrogram from raw audio; loaded by `example.py` when present, with a Python-side fallback otherwise |
| `tokenizer.json` | Fast Whisper tokenizer vocabulary and tokenisation rules |
| `tokenizer_config.json` | Whisper tokenizer configuration |
| `special_tokens_map.json` | Whisper special-token definitions |
| `pte_original/` | Original baseline model and tokenizer artefacts; not used by the optimised example |
| `example.py` | Minimal inference example |
| `pyproject.toml` | Pinned runtime dependencies for `example.py`, resolved with uv |
| `uv.lock` | Locked dependency resolution for `pyproject.toml` |
| `config.yaml` | Model I/O contract used by the example |
| `benchmarks/` | FP32 baseline and Arm-optimized benchmark records |

## Performance

Performance was measured on the reference configuration below. Results are intended to make the optimization reproducible but do not guarantee identical performance on every Arm-based system.

**Reference configuration**

| Field | Value |
|---|---|
| Device / platform | Vivo X300 |
| CPU / accelerator | C1-Ultra, C1-Premium, C1-Pro (aarch64, 8 cores), CPU execution backend |
| OS | Android 16 / OriginOS 6 |
| Runtime | ExecuTorch 1.1.0 |
| Backend / delegate | XNNPACK, KleidiAI |
| Batch size | 1 |
| Precision | INT8 weights (per-channel symmetric) and INT8 dynamic activations (PTQ-dynamic, no calibration required) |
| Runs | 10 warmup, 50 measured |

**Performance results**

| Metric | Original / baseline | Arm-optimized | Improvement |
|---|---:|---:|---:|
| p50 latency | 10927.0 ms | 7802.5 ms | 1.40 x faster |
| p90 latency | 11742.0 ms | 8027.0 ms | 1.46 x faster |
| p99 latency | 11742.0 ms | 8027.0 ms | 1.46 x faster |
| Model size | 1074.76 MB | 395.05 MB | 2.72 x smaller |
| Peak memory | 5226.97 MB | 4662.29 MB | 1.12 x less |
| RTFx | 0.64 | 0.90 | 1.40 x higher |
| Model load time | 1327.0 ms | 1015.0 ms | 1.31 x faster |

## Accuracy

Accuracy was evaluated using the same preprocessing, input resolution, and evaluation protocol described below. Where possible, the optimized model is compared against the original model under the same evaluation conditions.

**Evaluation setup**

| Field | Value |
|---|---|
| Dataset | LibriSpeech ASR (librispeech_asr) |
| Split | test-clean |
| Number of samples | 2620 |
| Metric(s) | Normalised WER (Whisper-style), CER |
| Evaluation runtime | ExecuTorch |

**Accuracy results**

| Metric | Original / baseline | Arm-optimized | Change |
|---|---:|---:|---:|
| Normalised WER | 3.45% | 3.41% | -0.04 pp |
| CER | 1.33% | 1.29% | -0.04 pp |

Accuracy was measured using the evaluation setup described above. Users should re-evaluate the model on their own data before production use.

## Arm optimization approach

Arm optimized this model for efficient inference on Arm-based platforms using a hardware-aware conversion and validation flow.

For this release, Arm used:

| Optimization area | Applied? | Notes |
|---|---|---|
| Model conversion | Yes | Converted to ExecuTorch `.pte` via an Optimum ExecuTorch seq2seq export |
| Quantization | Yes | PTQ-dynamic INT8: 8-bit weights (per-channel symmetric), 8-bit dynamic activations; no calibration required |
| Runtime/backend selection | Yes | XNNPACK and KleidiAI optimisations |
| Graph/runtime compatibility updates | Yes | Performed as part of the ExecuTorch export pipeline |
| Accuracy validation | Yes | Compared against the original model or published baseline |
| Performance validation | Yes | Measured on the reference Arm platform |

The goal of this process is to improve deployment characteristics such as latency, memory use, model size, and runtime compatibility while preserving the model's intended behavior. Detailed conversion scripts, calibration configuration, or backend-specific implementation details may be provided separately where appropriate.

## Using this model

### Install dependencies

Dependencies are declared in `pyproject.toml`, which ships with this repository. Resolve and install them into a local virtual environment with uv:

```bash
uv python install
uv sync --frozen
```

### Run the example

```bash
uv run example.py
```

Note: The Python/uv example runs on AWS Graviton (Ubuntu arm64) to confirm runtime compatibility only, and is intended as a guideline for building an equivalent run script on smartphone devices.

### Expected input

| Property | Value |
|---|---|
| Input shape | [1, 80, 3000] |
| Input type | float32 |
| Input range | N/A (log-mel spectrogram magnitude, not a fixed bounded range) |
| Preprocessing | Load audio as a 16 kHz mono waveform; compute a log-mel spectrogram (mel bins 80, hop length 160, n_fft 400, sample rate 16000, duration 30 seconds); pad or trim to 3000 time frames |

### Expected output

| Property | Value |
|---|---|
| Output shape | N/A (variable-length token ID sequence, autoregressive generation) |
| Output type | Token ID sequence |
| Postprocessing | Greedy decoding with a fixed token-suppression list and a repetition-guard heuristic that trims repeated trailing token patterns; decode with the Whisper tokenizer, skipping special tokens |

## Intended use

This model is intended for developers evaluating automatic speech recognition workloads on Arm-based platforms. It is suitable as a reference implementation for benchmarking, prototyping, and integration exploration.

## Limitations

- Performance depends on the target device, runtime version, backend/delegate support, memory configuration, and system load.
- Accuracy was evaluated on LibriSpeech ASR and may not generalize to all domains.
- This release preserves the original model's intended task and behavior, but users should validate it for their own application, data, and deployment environment.
- This repository is not a replacement for the original model documentation.

## Additional notes

Quantization keeps a small set of layers in FP32 to preserve accuracy: proj_out/lm_head, the encoder and decoder positional embeddings, and encoder.conv1/encoder.conv2. Audio inputs are limited to 30 seconds (3000 mel time frames) per chunk; longer audio must be chunked externally before inference.

`example.py` forces English transcription by hardcoding the decoder prefix to `<|en|>, <|transcribe|>, <|notimestamps|>`. This is an example-level default, not a model restriction: the bundled tokenizer and decoder support the full multilingual Whisper vocabulary (98 language tokens) and the `<|translate|>` task, so other languages or the translate task can be enabled by changing the forced-prefix token IDs in `example.py`, with no re-export required.
- Sample input: `sample_input.flac` is utterance `5338-24615-0014` of the [LibriSpeech ASR corpus](https://www.openslr.org/12) (dev-clean split) by Panayotov et al., via OpenSLR ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)).

## About this version

This repository contains a converted version of the openai/whisper-small model, originally developed by OpenAI.
Arm has converted the model to enable efficient execution on Arm-based platforms. No changes have been made to the model's intended behavior.

## Original model and documentation

For full details of the original model, please refer to the original [model card](https://huggingface.co/openai/whisper-small).

## Purpose of this release

This version is provided by Arm as a reference implementation to demonstrate performance on Arm-based systems. It is not a production-ready or supported solution. Users should evaluate the model independently for their use-case. Arm provides no warranties or ongoing support for this version.