parakeet-tdt-0.6b-v3-UltiMed-onnx
Want faster improvements? This project is entirely self-funded on my minimum-wage salary, and every training run competes for a single consumer GPU. If you or your organisation can donate an RTX 5090, or the money to buy one second-hand, it would directly speed up the next versions of the dataset and the fine-tuned models. Reach out via olicorne.org or open a discussion on this page.
More improvements are planned for October 2026, stay tuned.
The UltiMed effort has three parts:
- the dataset: Olicorne/UltiMed-ASR-FR-v1: 3,000+ hours of French medical dictation, to train or evaluate ASR models;
- the finetuned model (this repository): Olicorne/parakeet-tdt-0.6b-v3-UltiMed-onnx: parakeet-ultra finetuned on that dataset;
- the regex fixes: regex-fixes/ in the finetune's repository: drug-name and medical-term fix rules that should catch many of the remaining errors at almost no cost (text substitution after decoding), and also help models that were not finetuned.
French-medical fine-tune of moondream/parakeet-ultra, moondream's full-precision post-train of nvidia/parakeet-tdt-0.6b-v3. Training started from the NeMo rebuild of those weights published in Olicorne/parakeet-tdt-0.6b-v3-ultra-onnx, and the result is exported to ONNX for browser and CPU inference, with several encoder precisions in one repository.
See it on real speech: the real-voice stress test shows the author's own phone recordings of 10 dense medical dictations (15 recordings, 5 texts read twice), transcribed by the base model and this finetune in every setup, with the typical transcription mistakes the UltiMed finetune corrects (normalised CER, beam 5: 4.9 % for the base model, 2.3 % for this finetune, 1.9 % with phrase boosting and the fix rules).
Try it without installing anything: pick this model in parakeetweb.olicorne.org, my browser-based ASR app (source). It downloads the ONNX files below and transcribes entirely client-side, so nothing you dictate leaves your machine.
The export targets onnx-asr and the Parakeet Web browser app. Each precision lives in its own folder (see Layout); the basenames are the canonical onnx-asr ones, and the fp32 encoder is split across encoder-model.onnx.data.000 and .data.001, which ONNX resolves relative to the graph file, so the shards simply sit beside it in fp32/.
Contents
Benchmarks
For transcripts rather than numbers, the real-voice stress test shows the typical mistakes the finetune corrects on real phone recordings of the author reading 10 medical texts (unseen in training and by the fix rules, no cherry picking).
Word and character error rate on French-medical eval sets that the model never trained on:
| set | clips | what it is |
|---|---|---|
| dictionary | 5,285 | test split of UltiMed-ASR-FR-v1, medical dictionary terms in dictation sentences |
| parhaf | 2,498 | test split, PARHAF hospital report paragraphs |
| drugs | 2,059 | test split, drug names and dosages (complete) |
| acronyms | 151 | test split, medical acronyms (complete) |
| drug_sentence | 205 | separate drug and dosage sentence set, validation split: never trained on, but watched during training, so slightly optimistic |
| parrot | 1,549 | the full French PARROT subset, radiology reports, eval-only (never in any training manifest) |
That is 11,747 clips. The four UltiMed test splits are the dataset's own test split (dictionary and parhaf uniformly downsampled, drugs and acronyms complete), scored against the relabelled v1.3 text. Text is normalised before scoring (case and punctuation folded, accents kept). Lower is better. All columns share one CUDA backend and the int8 decoder; base is parakeet-tdt-0.6b-v3-ultra-onnx, the ONNX export of moondream/parakeet-ultra that the fine-tune started from, scored on exactly the same clips, and it is the column that says whether the fine-tune was worth doing. Part of the base model's error is formatting rather than recognition: it writes mg and deux where the labels say milligrammes and 2, which the normalisation does not fold. Folding unit words to symbols and spelled numbers to digits on both sides takes base from 8.93 % to 6.17 % macro WER (drugs 10.79 % to 7.12 %) and the fine-tune from 3.40 % to 3.22 %, so the fine-tune still roughly halves the error (fp32, greedy, 2026-10-04).
These numbers are not comparable with the ones the previous version of this repository reported, which used different evaluation sets and older labels.
Greedy decoding
Each cell is WER % / CER %, lower is better.
The fp16 column was re-measured on the full test split (11,747 clips, 2026-10-05): it matches fp32 to 0.1 point on every set (macro 3.4 % / 0.8 % for both). An earlier 150-clip run had shown fp16 lower on drugs (2.6 % against 3.9 %), but that came from the 150-clip slice being easier, not from a better encoder.
| set | fp32 | fp16 | int8 | w4a8 | base fp32 | base int8 |
|---|---|---|---|---|---|---|
| dictionary | 2.9 / 0.6 | 2.9 / 0.6 | 2.9 / 0.6 | 3.0 / 0.6 | 7.5 / 2.7 | 7.5 / 2.7 |
| parhaf | 3.1 / 0.8 | 3.1 / 0.8 | 3.1 / 0.8 | 3.1 / 0.8 | 7.5 / 3.7 | 7.5 / 3.7 |
| drugs | 3.9 / 1.0 | 3.8 / 0.9 | 3.9 / 1.0 | 4.0 / 1.0 | 11.0 / 6.4 | 11.0 / 6.4 |
| acronyms | 2.5 / 0.6 | 2.5 / 0.5 | 2.5 / 0.6 | 3.0 / 0.8 | 5.3 / 2.2 | 5.3 / 2.2 |
| drug_sentence | 4.0 / 1.0 | 4.0 / 1.0 | 4.0 / 1.0 | 4.0 / 1.0 | 14.4 / 11.0 | 14.4 / 11.0 |
| parrot (held out) | 4.2 / 0.8 | 4.2 / 0.8 | 4.2 / 0.8 | 4.4 / 0.9 | 8.5 / 3.1 | 8.5 / 3.1 |
| macro average | 3.4 / 0.8 | 3.4 / 0.8 | 3.4 / 0.8 | 3.6 / 0.9 | 9.0 / 4.8 | 9.0 / 4.8 |
| micro average | 3.3 / 0.7 | 3.3 / 0.7 | 3.3 / 0.7 | 3.4 / 0.8 | 8.1 / 3.5 | 8.1 / 3.5 |
Beam 5 (MAES)
Beam 5 is scored for fp32 and int8 only. Cells are WER % / CER %. Every column covers the full test split (11,747 clips, 2026-10-05 to 08). base fp32 uses paired early stopping against base int8 on the same clips (each set stops once the 95 % confidence interval of the WER difference is within ±0.05 point, 150 to 1,350 clips per set), and reports base int8 plus that difference.
| set | fp32 | int8 | base fp32 | base int8 |
|---|---|---|---|---|
| dictionary | 2.8 / 0.5 | 2.8 / 0.5 | 7.3 / 2.6 | 7.4 / 2.7 |
| parhaf | 3.0 / 0.8 | 3.0 / 0.8 | 7.3 / 3.6 | 7.4 / 3.7 |
| drugs | 3.8 / 0.9 | 3.8 / 0.9 | 10.5 / 6.3 | 10.6 / 6.3 |
| acronyms | 2.5 / 0.5 | 2.5 / 0.5 | 5.0 / 2.0 | 5.2 / 2.1 |
| drug_sentence | 3.9 / 1.0 | 3.9 / 1.0 | 14.3 / 11.0 | 14.4 / 11.0 |
| parrot (held out) | 4.1 / 0.8 | 4.1 / 0.8 | 8.2 / 3.0 | 8.2 / 3.0 |
| macro average | 3.3 / 0.8 | 3.4 / 0.8 | 8.8 / 4.7 | 8.9 / 4.8 |
| micro average | 3.2 / 0.7 | 3.2 / 0.7 | 7.8 / 3.4 | 7.9 / 3.5 |
The fp16 encoder runs about 12x slower than int8 on the CUDA execution provider here, which is why it gets no beam 5 column.
Multilingual retention (FLEURS)
The fine-tune was trained on French medical speech. The question this table answers is what that cost on ordinary multilingual speech, which is the risk an earlier training run actually hit: unprotected Swedish and Finnish drifted by 2.9 and 4.0 WER points before Common Voice rehearsal was added.
Scored on the FLEURS validation split of the 11 languages this model is realistically used near,
3,773 clips, normalised, greedy. base is the non-fine-tuned
parakeet-tdt-0.6b-v3-ultra-onnx
export on exactly the same clips. Cells are WER % / CER %, and the delta column is the int8 difference in percentage points for each (a negative delta means the fine-tune is better).
The base columns were re-measured on the same CUDA harness as the UltiMed columns, so every delta compares the fine-tune with the exact weights it started from. Both tables (greedy and beam 5) were rerun on 2026-10-07 with the <unk> ban in every decoder (see above); against the earlier unbanned run the greedy int8 delta went from +1.3 to +1.0 macro WER, the largest moves being Russian (+2.4 to +1.2) and Ukrainian (+3.1 to +1.6), where the fine-tune's own cells dropped while the base cells barely moved.
One methodological note worth stating, because getting it wrong is easy: this harness pairs an
int8 decoder with every encoder quant, fp32 included. A base fp32 figure taken from a sweep
that also ran an fp32 decoder is therefore not the same measurement and does not belong in this
column, even on identical clips. Measured both ways for base fp32 over the same 3,773 clips (2026-10-04), the decoder alone
moves a language by up to 0.27 pp (uk 6.05 % against 5.78 %, de 4.80 % against 4.60 %) and the
macro by 0.14 pp (5.24 % against 5.10 %), always in favour of the fp32 decoder.
| language | base fp32 | base int8 | UltiMed fp32 | UltiMed int8 | UltiMed w4a8 | delta (int8) |
|---|---|---|---|---|---|---|
| French | 4.7 / 1.9 | 4.7 / 1.9 | 5.2 / 2.2 | 5.2 / 2.2 | 5.7 / 2.4 | +0.5 / +0.3 |
| English | 5.5 / 2.5 | 5.5 / 2.5 | 6.3 / 3.1 | 6.3 / 3.1 | 6.5 / 3.3 | +0.8 / +0.7 |
| German | 4.8 / 1.4 | 4.8 / 1.4 | 5.1 / 1.7 | 5.1 / 1.7 | 5.3 / 1.7 | +0.3 / +0.2 |
| Italian | 2.2 / 0.7 | 2.2 / 0.7 | 3.0 / 1.3 | 3.0 / 1.3 | 3.4 / 1.5 | +0.8 / +0.6 |
| Spanish | 3.2 / 1.2 | 3.2 / 1.2 | 3.7 / 1.5 | 3.7 / 1.5 | 3.8 / 1.6 | +0.5 / +0.3 |
| Portuguese | 4.0 / 1.6 | 4.0 / 1.6 | 4.7 / 2.3 | 4.6 / 2.2 | 4.7 / 2.2 | +0.6 / +0.6 |
| Dutch | 7.0 / 2.3 | 7.1 / 2.3 | 7.5 / 2.7 | 7.5 / 2.8 | 7.9 / 2.8 | +0.4 / +0.5 |
| Polish | 5.8 / 1.9 | 5.8 / 1.8 | 7.0 / 2.4 | 6.9 / 2.4 | 7.5 / 2.6 | +1.1 / +0.6 |
| Slovak | 7.4 / 2.5 | 7.4 / 2.5 | 10.5 / 3.5 | 10.5 / 3.5 | 12.6 / 4.0 | +3.1 / +1.0 |
| Russian | 5.8 / 1.8 | 5.8 / 1.8 | 7.0 / 2.2 | 7.0 / 2.3 | 7.5 / 2.3 | +1.2 / +0.5 |
| Ukrainian | 5.8 / 1.6 | 5.8 / 1.6 | 7.4 / 2.4 | 7.4 / 2.4 | 8.0 / 2.6 | +1.6 / +0.8 |
| macro average | 5.1 / 1.8 | 5.1 / 1.8 | 6.1 / 2.3 | 6.1 / 2.3 | 6.6 / 2.5 | +1.0 / +0.5 |
Beam 5 (MAES), same 3,773 clips:
| language | base fp32 | base int8 | UltiMed fp32 | UltiMed int8 | delta (int8) |
|---|---|---|---|---|---|
| French | 4.5 / 1.9 | 4.5 / 1.8 | 5.0 / 2.1 | 4.9 / 2.1 | +0.5 / +0.3 |
| English | 5.4 / 2.4 | 5.5 / 2.4 | 6.1 / 3.0 | 6.1 / 3.0 | +0.6 / +0.6 |
| German | 4.7 / 1.4 | 4.6 / 1.4 | 4.9 / 1.6 | 4.9 / 1.6 | +0.2 / +0.2 |
| Italian | 2.1 / 0.6 | 2.1 / 0.6 | 2.8 / 1.2 | 2.8 / 1.2 | +0.7 / +0.6 |
| Spanish | 3.2 / 1.2 | 3.1 / 1.2 | 3.5 / 1.4 | 3.5 / 1.4 | +0.4 / +0.2 |
| Portuguese | 3.9 / 1.6 | 4.0 / 1.6 | 4.4 / 2.1 | 4.5 / 2.1 | +0.5 / +0.5 |
| Dutch | 7.0 / 2.3 | 6.9 / 2.2 | 7.2 / 2.7 | 7.1 / 2.7 | +0.2 / +0.5 |
| Polish | 5.6 / 1.8 | 5.7 / 1.8 | 6.7 / 2.4 | 6.7 / 2.4 | +1.0 / +0.6 |
| Slovak | 7.3 / 2.5 | 7.4 / 2.5 | 10.5 / 3.5 | 10.4 / 3.5 | +2.9 / +1.0 |
| Russian | 5.6 / 1.7 | 5.7 / 1.7 | 6.6 / 2.1 | 6.7 / 2.2 | +1.0 / +0.4 |
| Ukrainian | 5.7 / 1.6 | 5.7 / 1.6 | 7.1 / 2.4 | 7.2 / 2.4 | +1.4 / +0.8 |
| macro average | 5.0 / 1.7 | 5.0 / 1.7 | 5.9 / 2.2 | 5.9 / 2.2 | +0.9 / +0.5 |
French and English are broken out because they are the two that matter most here: French is the fine-tuning language, so it is where a gain is plausible, and English is the largest slice of the base model's pretraining, so it is where forgetting would show first.
Phrase boosting and drug-name rules
Two extra configurations, both scored with the JavaScript greedy decoder that parakeet_web runs in the browser (so these numbers match what a user of that app gets):
- boost: the decoder is biased toward the terms of the
french_medical.pwcphrase list shipped with parakeet_web (strength 1). - boost + regex: the same boosted transcripts, with the two fix-rule files of
regex-fixes/applied on top, drug rules (drug_fix_rules.jsonl, 3,428 rules) then medical-term rules (term_fix_rules.jsonl, 25,045 rules), the 2026-10-05 build (the 2026-10-07 rebuild now in the folder, 3,963 + 35,996 rules, also learned from parakeet-redux, gives this model the same WER to 0.01 point). They are learned from the errors of this model and of parakeet-ultra on ALL splits of UltiMed, test included (a list of fixes for every mistake seen on the dataset, not a model), so on the medical test sets this column shows how many seen mistakes they fix, not how they generalize. French only; regex-fixes/README.md explains how they are built (with thevenlafaxineexample), their overcorrection rate and how to apply them.
Cells are WER % / CER %. fp16 decoding is slow, so both fp16 columns use paired early stopping (2026-10-08): fp16 runs on the same clips as the full-split fp32 run, in a random order, and each set or language stops once the 95 % confidence interval of its WER difference to fp32 is within ±0.05 point (100 to 200 clips per medical set, 175 to all of the clips per FLEURS language). The cell is the fp32 full-split value plus that difference, so it compares with the other columns. On every set fp16 lands within 0.04 point of fp32. The earlier 150-clip fp16 cells read lower (drugs 2.3 %) only because the first 150 clips were easier. On FLEURS fp16 reads higher than fp32 on Russian (+0.34), Ukrainian (+0.29) and English (+0.14), but that is the <unk> ban, not fp16: the fp32 reference predates the ban in the shipped decoder (2026-10-06) while the fp16 run uses the banned one, and a paired fp32 rerun with the banned decoder (2026-10-08) moves by the same amount on its own (Russian +0.30, Ukrainian +0.25, English +0.09, every other language within 0.07). Net of the ban, fp16 is within 0.05 point of fp32 on 10 of the 11 languages; the exception is Slovak, where fp16 reads 0.13 point lower (a small difference in fp16's favour).
Medical sets, boost:
| set | fp32 | fp16 | int8 | w4a8 | base fp32 | base int8 |
|---|---|---|---|---|---|---|
| dictionary | 2.7 / 0.5 | 2.7 / 0.5 | 2.7 / 0.5 | 2.8 / 0.5 | 7.1 / 2.6 | 7.1 / 2.6 |
| parhaf | 3.1 / 0.8 | 3.0 / 0.8 | 3.0 / 0.8 | 3.1 / 0.8 | 7.2 / 3.5 | 7.2 / 3.5 |
| drugs | 3.6 / 0.9 | 3.6 / 0.9 | 3.6 / 0.9 | 3.7 / 0.9 | 10.6 / 6.3 | 10.6 / 6.3 |
| acronyms | 2.6 / 0.5 | 2.6 / 0.5 | 2.6 / 0.5 | 2.8 / 0.6 | 5.0 / 2.0 | 4.9 / 2.0 |
| drug_sentence | 3.6 / 1.0 | 3.7 / 1.0 | 3.6 / 0.9 | 3.7 / 1.0 | 13.7 / 10.9 | 13.7 / 10.9 |
| parrot (held out) | 4.3 / 0.8 | 4.3 / 0.8 | 4.3 / 0.8 | 4.5 / 0.9 | 8.0 / 3.0 | 8.1 / 3.0 |
| macro average | 3.3 / 0.8 | 3.3 / 0.8 | 3.3 / 0.7 | 3.4 / 0.8 | 8.6 / 4.7 | 8.6 / 4.7 |
| micro average | 3.2 / 0.7 | 3.1 / 0.7 | 3.1 / 0.7 | 3.3 / 0.7 | 7.7 / 3.4 | 7.7 / 3.4 |
Medical sets, boost + regex:
| set | fp32 | fp16 | int8 | w4a8 | base fp32 | base int8 |
|---|---|---|---|---|---|---|
| dictionary | 2.2 / 0.4 | 2.5 / 0.5 | 2.2 / 0.4 | 2.3 / 0.4 | 5.8 / 2.3 | 5.8 / 2.3 |
| parhaf | 2.8 / 0.7 | 2.6 / 0.7 | 2.8 / 0.7 | 2.9 / 0.7 | 6.3 / 3.3 | 6.3 / 3.3 |
| drugs | 2.2 / 0.5 | 1.5 / 0.3 | 2.2 / 0.5 | 2.3 / 0.6 | 7.9 / 5.6 | 7.8 / 5.5 |
| acronyms | 2.6 / 0.5 | 2.6 / 0.5 | 2.6 / 0.5 | 2.8 / 0.6 | 4.6 / 1.9 | 4.6 / 1.9 |
| drug_sentence | 2.7 / 0.8 | 2.7 / 0.9 | 2.7 / 0.8 | 2.8 / 0.8 | 11.6 / 10.3 | 11.5 / 10.3 |
| parrot (held out) | 4.1 / 0.8 | 3.8 / 0.6 | 4.1 / 0.8 | 4.2 / 0.8 | 7.4 / 2.8 | 7.4 / 2.8 |
| macro average | 2.8 / 0.6 | 2.6 / 0.6 | 2.8 / 0.6 | 2.9 / 0.7 | 7.3 / 4.4 | 7.2 / 4.4 |
| micro average | 2.7 / 0.6 | 2.7 / 0.6 | 2.7 / 0.6 | 2.8 / 0.6 | 6.5 / 3.1 | 6.5 / 3.1 |
FLEURS, boost (checks the medical phrase list does not hurt general speech):
| language | base fp32 | base int8 | UltiMed fp32 | UltiMed int8 | UltiMed w4a8 | UltiMed fp16 | delta (int8) |
|---|---|---|---|---|---|---|---|
| French | 4.6 / 1.9 | 4.6 / 1.9 | 5.4 / 2.2 | 5.4 / 2.2 | 5.9 / 2.5 | 5.4 / 2.3 | +0.8 / +0.3 |
| English | 5.5 / 2.4 | 5.5 / 2.4 | 6.2 / 3.0 | 6.2 / 3.0 | 6.2 / 3.1 | 6.4 / 3.2 | +0.7 / +0.6 |
| German | 4.7 / 1.3 | 4.7 / 1.4 | 5.1 / 1.6 | 5.1 / 1.6 | 5.3 / 1.7 | 5.0 / 1.6 | +0.4 / +0.3 |
| Italian | 2.2 / 0.7 | 2.2 / 0.7 | 3.0 / 1.3 | 3.0 / 1.3 | 3.3 / 1.5 | 3.0 / 1.3 | +0.8 / +0.7 |
| Spanish | 3.2 / 1.2 | 3.2 / 1.2 | 3.7 / 1.5 | 3.7 / 1.5 | 4.0 / 1.7 | 3.8 / 1.5 | +0.5 / +0.3 |
| Portuguese | 4.0 / 1.7 | 4.0 / 1.7 | 4.6 / 2.2 | 4.6 / 2.2 | 4.8 / 2.3 | 4.7 / 2.2 | +0.6 / +0.5 |
| Dutch | 7.1 / 2.4 | 7.1 / 2.3 | 7.7 / 2.8 | 7.7 / 2.8 | 7.8 / 2.8 | 7.7 / 2.8 | +0.6 / +0.5 |
| Polish | 5.9 / 1.9 | 5.9 / 1.8 | 7.0 / 2.4 | 7.0 / 2.4 | 7.7 / 2.7 | 7.0 / 2.3 | +1.0 / +0.5 |
| Slovak | 7.6 / 2.5 | 7.6 / 2.5 | 10.8 / 3.6 | 10.7 / 3.6 | 12.9 / 4.1 | 10.7 / 3.6 | +3.1 / +1.1 |
| Russian | 5.9 / 1.8 | 5.8 / 1.8 | 6.9 / 2.3 | 7.0 / 2.3 | 7.2 / 2.3 | 7.3 / 2.4 | +1.2 / +0.5 |
| Ukrainian | 5.8 / 1.6 | 5.9 / 1.6 | 7.2 / 2.3 | 7.3 / 2.4 | 7.8 / 2.6 | 7.5 / 2.5 | +1.4 / +0.7 |
| macro average | 5.1 / 1.8 | 5.1 / 1.8 | 6.2 / 2.3 | 6.2 / 2.3 | 6.6 / 2.5 | 6.2 / 2.3 | +1.0 / +0.5 |
FLEURS, boost + regex:
| language | base fp32 | base int8 | UltiMed fp32 | UltiMed int8 | UltiMed w4a8 | UltiMed fp16 | delta (int8) |
|---|---|---|---|---|---|---|---|
| French | 4.6 / 1.9 | 4.6 / 1.9 | 5.4 / 2.2 | 5.4 / 2.2 | 5.9 / 2.5 | 5.3 / 2.0 | +0.8 / +0.3 |
| English | 5.5 / 2.4 | 5.5 / 2.4 | 6.2 / 3.0 | 6.2 / 3.0 | 6.2 / 3.1 | 5.8 / 2.8 | +0.7 / +0.6 |
| German | 4.7 / 1.3 | 4.7 / 1.4 | 5.1 / 1.6 | 5.1 / 1.6 | 5.3 / 1.7 | 4.4 / 1.4 | +0.4 / +0.3 |
| Italian | 2.3 / 0.7 | 2.3 / 0.7 | 3.1 / 1.3 | 3.1 / 1.3 | 3.3 / 1.5 | 3.3 / 1.5 | +0.8 / +0.7 |
| Spanish | 3.2 / 1.2 | 3.2 / 1.2 | 3.7 / 1.5 | 3.7 / 1.5 | 4.0 / 1.7 | 3.0 / 1.2 | +0.5 / +0.3 |
| Portuguese | 4.1 / 1.7 | 4.1 / 1.7 | 4.7 / 2.3 | 4.7 / 2.3 | 4.9 / 2.3 | 5.3 / 2.6 | +0.6 / +0.5 |
| Dutch | 7.1 / 2.4 | 7.1 / 2.4 | 7.7 / 2.9 | 7.7 / 2.9 | 7.9 / 2.9 | 8.0 / 3.0 | +0.6 / +0.5 |
| Polish | 5.9 / 1.9 | 5.9 / 1.8 | 7.0 / 2.4 | 7.0 / 2.4 | 7.7 / 2.7 | 6.6 / 2.4 | +1.0 / +0.5 |
| Slovak | 7.6 / 2.5 | 7.6 / 2.5 | 10.8 / 3.6 | 10.7 / 3.6 | 12.9 / 4.1 | 10.4 / 3.5 | +3.1 / +1.1 |
| Russian | 5.9 / 1.8 | 5.8 / 1.8 | 6.9 / 2.3 | 7.0 / 2.3 | 7.2 / 2.3 | 6.7 / 2.1 | +1.2 / +0.5 |
| Ukrainian | 5.8 / 1.6 | 5.9 / 1.6 | 7.2 / 2.3 | 7.3 / 2.4 | 7.8 / 2.6 | 7.7 / 2.5 | +1.4 / +0.7 |
| macro average | 5.2 / 1.8 | 5.2 / 1.8 | 6.2 / 2.3 | 6.2 / 2.3 | 6.6 / 2.5 | 6.0 / 2.3 | +1.0 / +0.5 |
Long audio: a 10 minute single pass (VoxPopuli, French)
A naive int8 cast is known to degrade as a single pass runs on, so long audio is checked separately. The clip is 607.2 s of French European Parliament speech from the VoxPopuli validation split (CC0), 16 kHz mono, built from consecutive segments with the first 54 s dropped so an introduction or applause cannot sit at the start. It starts and ends on segment boundaries, so no utterance is cut mid-word at either end. The reference is fp32 decoding the same file in independent sections, so these are long-pass drift figures, not absolute accuracy. Both the sectioned oracle and the single pass read a byte-identical file, so the segment joins cannot bias the comparison. This is parliamentary speech, nothing like the medical audio this model was fine-tuned on, so it also doubles as a check that the fine-tune still holds together on ordinary French over a long pass.
The pass runs to completion in one go, which is inherited from the optimized export recipe: its encoder builds positional encoding at run time, so there is no ~400 s / 5000 frame wall. Attention memory is still quadratic in the window, so VRAM, not the graph, is what bounds a single pass.
| encoder | WER | CER | dur/proc | peak host RAM | peak VRAM |
|---|---|---|---|---|---|
| int8 | 7.5 % | 2.5 % | 62x | 2,680 MB | 16,810 MiB |
| w4a8 | 8.4 % | 2.6 % | 56x | 2,319 MB | 16,298 MiB |
| fp32 | 7.7 % | 2.5 % | 51x | 2,226 MB | 18,862 MiB |
The dur/proc column comes from a single pass and varies from run to run: three repeated passes per encoder on an idle GPU (2026-10-04) gave int8 43x, fp32 46x and w4a8 46x, so on CUDA all three run at roughly 45 to 60x real time and speed is not a reason to pick one.
VRAM is the peak of the pass's own process, sampled once a second and attributed by session id, so other work on the same card is excluded. Each pass runs alone, and the fp32 oracle over the same sections peaks at only 3,876 MiB by comparison.
Per-section WER across the pass, one row per minute:
| window | int8 | w4a8 | fp32 |
|---|---|---|---|
| 0-60 s | 2.7 % | 2.7 % | 2.7 % |
| 60-120 s | 7.8 % | 8.7 % | 7.8 % |
| 120-180 s | 7.9 % | 10.9 % | 6.9 % |
| 180-240 s | 7.5 % | 7.5 % | 7.5 % |
| 240-300 s | 9.0 % | 9.9 % | 9.9 % |
| 300-360 s | 1.6 % | 3.2 % | 3.2 % |
| 360-420 s | 4.9 % | 4.9 % | 4.9 % |
| 420-480 s | 9.0 % | 12.0 % | 9.0 % |
| 480-540 s | 12.4 % | 14.1 % | 12.4 % |
| 540-600 s | 8.6 % | 8.1 % | 8.6 % |
For comparison, the base ultra int8 export reaches 11.5 % and 10.5 % WER on the last two windows (480-540 s and 540-600 s) of the same file.
CER is reported for the pass as a whole only; the per-section path in wer-quants.py computes WER
only. Beam 5 is not available for this test: the long-audio harness is greedy-only.
Files
| File | Size | What it is |
|---|---|---|
fp32/encoder-model.onnx + .data.000 + .data.001 |
2,435 MB | fp32 encoder, sharded external data |
fp16/encoder-model.fp16.onnx |
1,162 MB | fp16 encoder, single file, for GPU back ends (see Benchmarks) |
int8/encoder-model.int8.onnx |
650 MB | 8-bit weights, int8 activations at run time |
w4a8/encoder-model.w4a8.onnx |
384 MB | 4-bit weights, int8 activations at run time |
fp32/decoder_joint-model.onnx + .data |
69 MB | fp32 decoder and joint network |
int8/decoder_joint-model.int8.onnx |
17 MB | int8 decoder and joint network |
nemo128.onnx |
137 KB | 128-bin mel preprocessor |
vocab.txt, config.json |
92 KB | tokenizer vocabulary and onnx-asr config |
parakeet-tdt-0.6b-v3-ultimed.nemo |
2,393 MB | the NeMo training checkpoint, produced by UltiMed-ASR-FR-v1-NeMo_training_scripts |
The decoder graphs carry in-graph top-K outputs (topk_logits, topk_ids, duration_logits, lse_token, lse_duration) alongside the full outputs tensor, so a greedy decoder can fetch only the reduced set per step instead of reading back the whole logits row.
Layout
A file's directory is a pure function of its basename, and the basenames never change:
| basename | directory |
|---|---|
encoder-model.onnx, encoder-model.onnx.data.NNN, decoder_joint-model.onnx, decoder_joint-model.onnx.data |
fp32/ |
*.int8.onnx |
int8/ |
*.w4a8.onnx |
w4a8/ |
*.fp16.onnx |
fp16/ |
vocab.txt, config.json, nemo128.onnx, README.md, *.nemo |
repo root |
This is the same rule, folder for folder, as the sibling Olicorne/parakeet-tdt-0.6b-v3-optimized-onnx repository, whose README carries the full version of this table together with the legacy-layout notes and the loader details. A consumer written for one repository works on the other.
One consequence worth stating: downloading a single precision is a one-liner, which matters when the full repository is several GB.
from huggingface_hub import snapshot_download
snapshot_download("Olicorne/parakeet-tdt-0.6b-v3-UltiMed-onnx",
allow_patterns=["int8/*", "vocab.txt", "nemo128.onnx", "config.json"])
Swap "int8/*" for "fp32/*", "w4a8/*" or "fp16/*". The 4-bit and fp16 encoders ship no decoder of their own, so pair them with "int8/decoder_joint-model.int8.onnx" or "fp32/decoder_joint-model.onnx*".
Not shipped: a local/ folder is used locally as a scratch area for stale builds and working files. It is excluded from every upload and is not part of this repository.
Related repositories
This model is one piece of a French medical ASR stack; every other piece is public.
| Repository | What it is |
|---|---|
| Olicorne/UltiMed-ASR-FR-v1 | The dataset the fine-tune was trained on: 601,338 clips / 3,105 h of synthesized French medical speech, plus an eval-only PARROT subset. |
| This repository | The French medical fine-tune documented above, exported to ONNX. |
| Olicorne/parakeet-tdt-0.6b-v3-ultra-onnx | The base the fine-tune starts from: moondream/parakeet-ultra rebuilt as a .nemo and exported to ONNX, and the base column of every benchmark here. |
| Olicorne/parakeet-tdt-0.6b-v3-redux-UltiMed-onnx | Experimental: this fine-tune's decoder change (UltiMed minus ultra, times 1.5) added to the decoder of moondream's 2-bit parakeet-redux. A 159 MB download for CPU / WASM, less accurate than this repository. |
| Olicorne/parakeet-tdt-0.6b-v3-optimized-onnx | The stock NVIDIA export whose re-quantization and graph-optimization recipe (int8 accuracy on long audio, browser speed) every export here reuses. |
| UltiMed-ASR-FR-v1-scripts | The open recipe that built that dataset end to end: text sources, spoken-form normalization, the batch synthesis client, and the transcribe-and-rescore QC pass. |
| UltiMed-ASR-FR-v1-NeMo_training_scripts | The NeMo fork and training configs used to run the fine-tune itself. |
| UltiMed-ASR-FR-v1-Voxtral | The Voxtral TTS container that spoke every clip of the dataset: Dockerfile, tuning, and the vllm-omni patches. |
| Parakeet Web | The in-browser ASR app that loads either ONNX model, live at parakeetweb.olicorne.org. Everything runs client-side. |
No <unk> output (since 2026-10-06). Both decoders have the joint's output bias for <unk> (id 0) set to -1e4, so greedy and beam search never emit it and take the next best token, or blank, instead. Parakeet emits <unk> for characters missing from its vocab, mostly ° ("37,8 degrés" came out as "37,8C"), + and some apostrophes; onnx-asr printed it raw and parakeet_web silently deleted it. The patch changes that one float and nothing else (ban-unk.py). On a paired check (this model, int8 encoder and decoder, CUDA greedy, 2026-10-06) it lowered FLEURS fr WER from 4.98 % to 4.79 % and moved the four UltiMed medical test slices (PARROT, drugs, PARHAF, dictionary) by at most 0.01 point, so the medical tables, measured before the ban, still hold. The other benchmarks dated before 2026-10-06 ran without it. Done with Claude Code.
Usage
Intended use
Built for French medical dictation and transcription assistance: clinical notes, drug and dosage mentions, anatomical and technical vocabulary, in a workflow where a human reads and corrects the output before it counts.
It is a drop-in replacement for the base Parakeet model when the audio is French and medical. Outside that domain it is not better than the base model, and on non-French audio it is at best equal to it.
Out of scope. Do not use it as an autonomous system, for clinical decision support, for triage or diagnosis, for any decision affecting a patient without a human in the loop, or for identifying or profiling speakers.
Which encoder to use
Both quantised encoders store narrow weights and let the kernel quantise activations to int8 at run time, so neither needs calibration data. Note that this makes the int8 file here a different recipe from the static QDQ int8 that ships with the upstream optimised repository: that one freezes activation ranges from a calibration campaign, this one recomputes them per run.
They trade size for speed rather than buying both. Measured in-browser on the CPU path over a 90 second profile, fp32 runs at about 2.8x real time, int8 at about 2.5x, and w4a8 at about 2.4x. Read the three as a download-size ladder with flat accuracy, not a speed ladder. Pick fp32 when bandwidth is free, w4a8 when it is not, and int8 in between. The fp16 encoder is the GPU option: half the fp32 download at the same accuracy (checked on matching clips, see the note under the first table), but it needs a back end with fp16 kernels (WebGPU in the browser, CUDA with onnx-asr). On the CPU execution provider it ran about eight times slower than real time in a smoke test, so it is not a candidate for the WASM path.
Loading it with onnx-asr
onnx-asr's stock loader globs the repository root only, so onnx_asr.load_model(..., path=<repo>) does not find these files on its own: they are one folder deeper. Point each entry at its layout folder first, which takes about ten lines. The optimized repository's README carries the full, commented snippet, reproduced here in short:
from pathlib import Path
import onnx_asr
from onnx_asr.loader import create_asr_resolver
MODEL_DIR = "path/to/this/repo"
QUANT = "int8" # None for fp32, or "int8" / "w4a8" / "fp16"
def relocate(pattern):
# onnx-asr resolves each entry with path.glob(pattern) and hands out
# basename-shaped patterns ('encoder-model?int8.onnx'), so prepending the
# directory is enough; the pattern stays the last segment.
basename = pattern.replace("?", ".")
for folder in ("int8/", "w4a8/", "fp16/", "fp32/", ""):
if (Path(MODEL_DIR) / folder / basename).is_file():
return folder + pattern
return pattern
model_type = create_asr_resolver("nemo-parakeet-tdt-0.6b-v3", MODEL_DIR).model_type
original = model_type._get_model_files
model_type._get_model_files = staticmethod(
lambda quantization=None: {k: relocate(v) for k, v in original(quantization).items()})
try:
model = onnx_asr.load_model("nemo-parakeet-tdt-0.6b-v3", path=MODEL_DIR, quantization=QUANT)
finally:
model_type._get_model_files = staticmethod(original)
print(model.recognize("audio.wav"))
Pass quantization=None for fp32. Note that onnx-asr applies one suffix to both encoder and decoder, so quantization="int8" selects int8/encoder-model.int8.onnx together with int8/decoder_joint-model.int8.onnx; the w4a8 and fp16 encoders have no decoder at their own width, so loading one that way needs a mixed-precision override (see scripts/wer-quants.py in the Parakeet Web repository for a worked example).
Known caveats and limitations
Not a medical device. This is a speech recognition model, not a medical device, and it must not be used to make clinical decisions on its own. It makes errors, and it makes more of them on continuous drug-name speech than anywhere else (see the drug_sentence row of the tables above). Any transcript intended for a patient record needs human review.
The shipped checkpoint and the ONNX. parakeet-tdt-0.6b-v3-ultimed.nemo is the exact model the ONNX files were exported from, and the one the benchmarks score: the ONNX graphs were produced from this very file by the NeMo fork's perso/export_onnx.py. Use it as the starting point for further fine-tuning in the same NeMo fork.
Batching. The encoder masks padded frames with a key-only attention bias, so onnx-asr's model.recognize([a, b, c]) works on clips of different lengths with no manual padding, and batch 1 is bit-identical to the previous unmasked export (the published numbers stay valid). On 1,000 UltiMed dictionary clips (UltiMed w4a8 encoder, fp32 decoder, CUDA, onnx-asr with a lockstep batched greedy TDT decode as in the ultra repo's scripts/drug-rules/transcribe_onnx.py), batches of 8, 16 and 32 took 0.06, 0.05 and 0.04 s per clip against 0.16 s one clip at a time, changed 3 to 4 transcripts out of 1,000, and kept WER at 3.26 % (3.26 % unbatched). Use the fp32 decoder when batching: the int8 decoder quantizes activations with one scale per batch, so a clip's transcript can depend on its batch neighbours. Stock onnx-asr decodes each clip on its own, so it gets the encoder speedup but not the decoder one.
Bias and coverage
The most important thing to know about this model is how narrow its medical training audio is.
One synthetic voice. Every hour of medical speech it learned from was generated by a single TTS voice, fr_female, from written text. There is no real clinician or patient speech in the medical portion at all. The model has therefore seen essentially none of the variation that real dictation contains: accents, age, pitch range, speech rate, disfluency, hesitation, correction mid-sentence, or any pathological or accented speech. White noise, gain and room impulse response augmentation plus SpecAugment were the only defences against overfitting to that one voice's acoustics, and they are a partial defence, not a substitute for varied speakers.
Expect measured accuracy here to be optimistic against real recordings, and validate on your own audio before trusting the numbers above.
Read speech, not spontaneous speech. Both the synthetic medical audio and the Common Voice rehearsal are read aloud from text. Spontaneous dictation, with its restarts and filler, is out of distribution.
French from France. The drug and terminology sources are French national open data, so coverage of Belgian, Swiss, Quebec and African French medical usage and brand names is weaker.
Inherited skew. The Common Voice portion carries Common Voice's own well-documented volunteer-contributor imbalances in gender, age and accent, and those are the only human voices in the mix.
Training
Data
| Corpus | In training | Licence |
|---|---|---|
| UltiMed-ASR-FR-v1: dictionary, PARHAF, drugs, acronyms | 2,398 h after a 38 s clip cap, val and test rows excluded | CC BY 4.0 |
| Drug and dosage sentences read by TTS | 4,100 clips | CC BY 4.0, names from French open data |
| In-house medical set | small, hand-built | CC BY 4.0 |
| Common Voice 22.0 rehearsal: fr 200 h, en 100 h, de 20 h, it 20 h, sv-SE 9.1 h, fi 2.6 h | 352 h, roughly 11 % of audio-seconds | CC0-1.0 |
The medical audio is synthetic, generated from text with Voxtral TTS at 24 kHz and resampled to 16 kHz during training. By hours it is 85.6 % dictionary terms, 11.8 % PARHAF hospital reports, 2.5 % drug names and 0.1 % acronyms.
The Common Voice portion is replay rehearsal rather than new capability. Common Voice is part of the base model's own pretraining mix, so replaying it holds back the multilingual forgetting that earlier runs showed once the medical data dominated. Unprotected Swedish and Finnish had drifted by 2.9 and 4.0 WER points against baseline before this was added.
Deliberately excluded
Three corpora were kept out of training on purpose, which is what makes the CC BY 4.0 release below clean:
- PARROT (radiology reports, CC BY-NC-SA 4.0). Evaluation only, never in a training manifest. Its NonCommercial and ShareAlike terms would otherwise have propagated to these weights.
- United Synthetic Medical (CC BY-SA 4.0). Removed from training for the ShareAlike clause alone. Still scored in validation, since evaluating is not redistributing.
- FLEURS. Held out so its validation splits stay genuinely out-of-corpus probes for every monitored language.
Procedure
Fine-tuned from parakeet-tdt-0.6b-v3-ultra.nemo, the NeMo rebuild of moondream/parakeet-ultra shipped in Olicorne/parakeet-tdt-0.6b-v3-ultra-onnx (same architecture and tokenizer as nvidia/parakeet-tdt-0.6b-v3, only the learned weights differ). The tokenizer was kept as-is, so the vocabulary is byte-identical to the base model's.
The run is reproducible: the NeMo fork, the patches it needed and the launcher are published at UltiMed-ASR-FR-v1-NeMo_training_scripts. The table below is a summary of perso/training_config.yaml, which is annotated at length with the VRAM cost model behind each value, the ten measured OOM points it was fitted on, and what to change first when a setting has to move.
| Setting | Value |
|---|---|
| Trainable parameters | Last 4 encoder layers, plus the full decoder and joint network. Lower encoder layers frozen to preserve multilinguality. |
| Optimiser | AdamSPD (selective projection decay), betas 0.9 / 0.999, weight decay 0.01. Decays toward the pretrained weights rather than toward zero, to limit multilingual forgetting |
| Learning rate | 2e-5, cosine annealing to 2e-6, 800 warmup steps |
| Schedule | 20,000-step cosine schedule, stopped by hand at step 14,255 (about 1.3 epochs) once validation had plateaued |
| Final weights | Uniform average of the step 13,255 and 14,255 checkpoints. Picked out of four checkpoint averages and the single checkpoints, all re-scored offline on the validation sets (normalised macro WER 5.08 % against 5.12 % for the best single checkpoint, and lower French FLEURS WER); every candidate was within about 0.1 pp, so averaging bought robustness more than accuracy |
| Labels | UltiMed-ASR-FR-v1 text as relabelled in v1.3 (consistent spellings for œ, aiguë, suture gauges, digits after staging words and the like) |
| Batching | DurationCostBatchSampler: sizes each batch to a VRAM budget rather than a fixed clip count, 38 s clip cap. See patch 1 |
| SpecAugment | 2 frequency masks (width 25), 5 time masks (width 0.05) |
| Audio augmentation | White noise, gain, room impulse response convolution |
| Hardware | One 24 GB GPU |
| Seed | 42424242 |
Validation during training tracked downsampled slices of the dataset's validation split (never the test split scored above) alongside FLEURS validation splits for six European languages, so multilingual behaviour was monitored rather than assumed. What each patch does walks through every change the fork makes to upstream NeMo, including the normalised, per-source macro-averaged WER metric those validations were scored with.
Licence and attribution
Released under CC BY 4.0, matching the base model and the training corpora. You may use these weights commercially and build on them, including under different terms, provided you keep the attributions below and note that the material was modified.
- Base model: moondream/parakeet-ultra by moondream, CC BY 4.0, itself a post-train of
nvidia/parakeet-tdt-0.6b-v3by NVIDIA, CC BY 4.0. Converted to NeMo in Olicorne/parakeet-tdt-0.6b-v3-ultra-onnx. These weights are a modified version: fine-tuned on French medical speech and exported to ONNX. - PARHAF: HealthDataHub / Plateforme des Donnees de Sante, CC BY 4.0 and Etalab Open Licence 2.0. Dataset. Training split only; the test split is embargoed and was not used or redistributed. Data was modified: text was synthesized to audio. No endorsement by the authors is implied.
- Drug names: French open data under the Licence Ouverte / Etalab 2.0, from OPEN_MEDIC and RETROCEDAM (CNAM / Assurance Maladie) and the BDPM public medicines database. Names and public dosages only.
- Medical terms and acronyms: individual terms drawn from public French sources; no definitions or article text were redistributed.
- Common Voice 22.0: Mozilla Foundation, CC0-1.0. No obligation, credited anyway.
Please cite PARHAF if you build on this model:
@misc{tannier2026parhaf,
title = {PARHAF, a human-authored corpus of clinical reports for fictitious patients in French},
author = {Tannier, Xavier and Abbara, Salam and Flicoteaux, R{\'e}mi and Khalil, Youness and N{\'e}v{\'e}ol, Aur{\'e}lie and Zweigenbaum, Pierre and Bacry, Emmanuel},
year = {2026},
howpublished = {arXiv preprint; see the PARHAF dataset page},
url = {https://huggingface.co/datasets/HealthDataHub/PARHAF}
}
Contact
Open a discussion on this repository. Issues in the browser app that loads these files belong in the Parakeet Web repository instead.
Exported, quantised and measured with Claude Code.
- Downloads last month
- 286