---
license: cc-by-nc-4.0
language:
- da
library_name: transformers
pipeline_tag: automatic-speech-recognition
tags:
- asr
- speech-recognition
- danish
- distillation
base_model: syvai/hviske-v5.3
---
# hviske-v5-tiny
**hviske-v5-tiny** is a 263M-parameter Danish ASR model distilled from the
syv-transcribe ensemble (~2.1B). About **8× smaller** than its teachers while staying
competitive with them, and roughly **2× faster** than any 2B model on the
[Danish ASR leaderboard](https://github.com/Rye-A1/danish-asr-leaderboard).
Audio is expected at 16 kHz mono, clips up to 35 s. Danish only, offline transcription
(no timestamps, diarization, or streaming).
**[Try it live in your browser](https://huggingface.co/spaces/syvai/hviske-v5-tiny-demo)** —
live dictation with VAD-based streaming, running entirely client-side on WebGPU
(onnxruntime-web + a transformers.js tokenizer). Nothing is uploaded; the model is
downloaded once and cached.
Prefer not to run it yourself? The model is also available as a hosted API at
**[platform.syv.ai](https://platform.syv.ai)**.
## Accuracy
Measured with the leaderboard harness, unmodified (`--backend cohere-asr`), on a single
RTX 3090.
| Dataset | WER | CER |
|---|---|---|
| CoRal conversation | 26.07 | 15.97 |
| CoRal read-aloud | 14.76 | 6.00 |
| Common Voice 17 (da) | 9.89 | 3.57 |
| FLEURS (da) | 11.32 | 4.49 |
| FTSpeech | **7.15** | 3.84 |
| **Mean** | **13.84** | **6.77** |
FTSpeech 7.15 is the best score on the leaderboard at the time of submission, ahead of
the 2.1B syv-transcribe ensemble (8.02).
## Speed
RTFx = audio seconds transcribed per wall-clock second. "Single" is one clip at a
time (the interactive case); "batched" is the best measured batched/concurrent
configuration for that runtime. Where clip length matters the batched cell shows
both: longer clips amortise per-request cost, so 30 s audio yields a higher RTFx
than the 11 s FLEURS average.
| Hardware | Runtime | Single | Batched | Batch config | WER |
|---|---|---|---|---|---|
| RTX 3090 | **custom CUDA kernels ([`cuda/`](cuda/))** | **320×** | **3833×** (11 s) · **4341×** (30 s) | batch 64 | 11.47 |
| RTX 3090 | vLLM 0.19.0 + CUDA kernels | 182× | 2646× (11 s) · 3587× (30 s) | 512 concurrent | 11.21 |
| RTX 3090 | vLLM 0.19.0 stock | 176× | 1783× (11 s) · 2936× (30 s) | 256–384 concurrent | 11.27 |
| RTX 3090 | PyTorch bf16 (leaderboard harness) | 158× | 323× | batch 8 | 11.32 |
| Apple M4 (base) | MLX int4 | 87× | 93× | batch 8 | 10.60† |
| Apple M4 (base) | MLX int8 | 63× | 110× | batch 8 | 10.60† |
| Apple M4 (base) | MLX fp16 | 39× | 78× | batch 8 | 10.46† |
| Apple M4 (base) | GGUF q4_k via CrispASR (Metal) | 56× | — | sequential CLI | 10.51† |
| Apple M4 (base) | GGUF q8_0 via CrispASR (Metal) | 45× | — | sequential CLI | 10.44† |
| Apple M4 (base) | ONNX Runtime CPU, 2 threads | 31× | — | batch axis fixed at 1 | 10.55† |
| Apple M4 (base) | GGUF q4_k via CrispASR (CPU, 8 threads) | 24× | — | sequential CLI | 10.88† |
| Apple M4 (base) | PyTorch CPU fp32 | 11× | — | | 11.03† |
| x86 CPU, 4 threads | PyTorch fp32 | 3.2× | — | | — |
WER is FLEURS-da (lowercase, punctuation-stripped, same normaliser as the
leaderboard). RTX rows are the **full 930-clip test set**; † rows are the
**200-clip subset** used in the per-build sections below, which skews ~0.8
lower because its clips are shorter — compare within a group, not across.
The spread inside each group (±0.1–0.2) is quantisation/bf16 noise, not a
real accuracy difference between runtimes.
Notes on the dashes: the CrispASR CLI processes files sequentially (passing many
files in one invocation only amortises model load), and the published ONNX graphs
were exported with the batch axis fixed at 1, so neither supports true batching
today. Both are fixable — the ONNX one just needs a re-export with a dynamic
batch axis.
Batching pays off very differently per backend: **12× on the 3090 with the custom
kernels, 9–15× under vLLM**, but only **1.1–2× on a base M4**, where a single
stream already keeps the GPU busy.
The Apple rows were measured back-to-back within each build, but the M4 had other
work running and its throughput moves by roughly ±25% with machine load (the MLX
int4 single figure measured as high as 139× on an idle machine, 73–87× loaded).
Treat all Apple numbers as a band, not a point; per-build details are in the MLX,
ONNX and GGUF sections below.
## Usage
All paths below are self-hosted. For a managed option, the model is served through the
hosted API at [platform.syv.ai](https://platform.syv.ai).
### PyTorch
```python
import soundfile as sf
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
model = AutoModelForSpeechSeq2Seq.from_pretrained(
"syvai/hviske-v5-tiny", trust_remote_code=True).eval()
processor = AutoProcessor.from_pretrained("syvai/hviske-v5-tiny", trust_remote_code=True)
audio, sr = sf.read("clip.wav", dtype="float32")
text = model.transcribe(processor=processor, language="da",
audio_arrays=[audio], sample_rates=[sr])[0]
print(text)
```
- Requires `trust_remote_code=True` and `transformers` 4.57.x — 5.x regressed
remote-model loading for this architecture.
- Use **bf16** on GPU, not fp16: the architecture's masking values overflow in fp16.
### CUDA kernels (NVIDIA GPUs)
The fastest GPU path is the custom Triton kernel stack in [`cuda/`](cuda/): flash-style
relative-position attention (no `rel_shift` copies or score-tensor materialization),
single-query decode attention, fused depthwise-conv+BatchNorm+SiLU, channels-last
subsampling convs, a GPU log-mel frontend, and a CUDA-graphed greedy decode loop.
Same weights, same math: WER on the full FLEURS-da test set is 11.47 vs 11.45 through
the reference path on identical audio. Speed is in the table above — the batched
figure corresponds to about 48% MFU on the 3090, so the remaining headroom on this
card is small.
`cuda/hviske_enc_kernels.py` also patches the encoder of vLLM 0.19.0 in place
(instructions in [`cuda/README.md`](cuda/README.md)), lifting batched serving by
25–35% at unchanged WER (the two vLLM rows above). After the patch, vLLM is
CPU-frontend-bound: use concurrency ≥512 to saturate the GPU.
### Server (vLLM)
A bf16 checkpoint with the serving fixes already applied lives in [`vllm/`](./vllm).
Tested with vLLM 0.19.0.
```bash
pip install vllm==0.19.0
huggingface-cli download syvai/hviske-v5-tiny --include 'vllm/*' --local-dir ./hviske
vllm serve ./hviske/vllm --served-model-name hviske-v5-tiny \
--trust-remote-code --dtype bfloat16 --gpu-memory-utilization 0.90 \
--max-model-len 1024 --max-num-seqs 512 --api-server-count 4 --port 18010
```
```bash
curl http://127.0.0.1:18010/v1/audio/transcriptions \
-F file=@clip.wav -F model=hviske-v5-tiny -F language=da
```
Single-request p50 is 62 ms on an RTX 3090; throughput and WER are in the Speed table
above. See [`vllm/README.md`](./vllm/README.md) for tuning notes and the reasons that
directory differs from the root checkpoint.
### Apple silicon (MLX)
Native [MLX](https://github.com/ml-explore/mlx) builds live in [`mlx/`](./mlx) — fp16,
int8 and int4 weights plus a small pure-MLX runtime, so the model runs on a Mac with no
PyTorch installed.
```bash
pip install mlx numpy sentencepiece soundfile huggingface_hub
```
```python
import sys
import mlx.core as mx
import soundfile as sf
from huggingface_hub import snapshot_download
path = snapshot_download("syvai/hviske-v5-tiny", allow_patterns=["mlx/int4/*", "mlx/hviske_mlx/*"])
sys.path.insert(0, f"{path}/mlx")
from hviske_mlx.transcribe import Hviske
model = Hviske(f"{path}/mlx/int4")
audio, sr = sf.read("clip.wav", dtype="float32") # 16 kHz mono
print(model.generate(mx.array(audio))["text"])
```
Batched greedy decoding is available via `model.generate_batch([a1, a2, ...])`.
Build sizes: `fp16` 526 MB, `int8` 300 MB, `int4` 179 MB; speed and WER are in the
Speed table above. **int4 is the recommended default** — it matches int8 on accuracy
while being 41% smaller and faster; `fp16` tracks the PyTorch model most closely.
The port is verified against PyTorch stage by stage (encoder output agrees to 1.5e-06;
38 of 40 clips decode byte-identically) — see [`mlx/README.md`](./mlx/README.md) for
details, limitations, and the alternative
[`mlx-speech`](https://pypi.org/project/mlx-speech/) runtime.
### CPU (ONNX Runtime)
ONNX graphs live in [`onnx/`](./onnx) — portable CPU inference with no PyTorch, on
macOS, Linux or Windows.
```bash
pip install onnxruntime numpy sentencepiece soundfile huggingface_hub
```
```python
import sys
import soundfile as sf
from huggingface_hub import snapshot_download
path = snapshot_download("syvai/hviske-v5-tiny", allow_patterns=["onnx/*"])
sys.path.insert(0, f"{path}/onnx")
from hviske_onnx.runtime import HviskeOnnx
model = HviskeOnnx(f"{path}/onnx", encoder_int8=False, decoder_int8=True, threads=2)
audio, sr = sf.read("clip.wav", dtype="float32") # 16 kHz mono
print(model.generate(audio)["text"])
```
Speed and WER are in the Speed table above (2.8× faster than the PyTorch CPU path at
equal accuracy). The encoder is fp32 and the decoder int8: dynamic int8 helps the
decoder (3× faster, no WER cost) but *hurts* the conv-heavy encoder on ARM (2× slower,
+1.9 WER). Use 2 threads — the graphs are too small to profit from more. See
[`onnx/README.md`](./onnx/README.md).
### GGUF (CrispASR)
GGUF builds for the [CrispASR](https://github.com/CrispStrobe/CrispASR) C++/ggml runtime
live in [`gguf/`](./gguf) — no Python at inference, with Metal/CUDA/Vulkan/CPU backends.
```bash
./build/bin/crispasr --backend cohere -m hviske-v5-tiny-q4_k.gguf -f clip.wav -l da -t 4
```
| Build | Size | RTFx (Metal) | WER |
|---|---|---|---|
| `q4_k` | 160 MB | **56.4×** | 10.51 |
| `q5_0` | 190 MB | 52.5× | 10.57 |
| `q6_k` | 243 MB | 50.5× | 10.53 |
| `q8_0` | 281 MB | 45.0× | 10.44 |
| `f16` | 527 MB | 39.1× | 10.51 |
q4_k is the recommended build — smallest, fastest, and no measurable accuracy cost
against f16 (ggml's per-block k-quants cost nothing here, unlike ORT's dynamic int8).
The CPU-only figure is in the Speed table above. See
[`gguf/README.md`](./gguf/README.md).
## Limitations
- Danish only, despite the multilingual tokenizer inherited from the teacher.
- Conversational/spontaneous speech is the weakest domain (CoRal conversation 26.07
WER), consistent with the teacher family.
- Distilled from teacher pseudo-labels, so it inherits the teacher's biases and cannot
exceed it on material where the teacher is wrong.
- No timestamps, diarization, or streaming.
## License
CC BY-NC 4.0, inherited from the teacher [syvai/hviske-v5.3](https://huggingface.co/syvai/hviske-v5.3).
