FredrikKarlssonSpeech/whisper-large-v2-finnish-onnx

whisper-large-v2-finnish (ONNX)

ONNX conversion of Finnish-NLP/whisper-large-v2-finnish with multiple precision variants.

Available Precisions

Variant	Description	Files
fp32	Full precision (float32)	encoder_model.onnx, decoder_model.onnx, decoder_with_past_model.onnx
fp16	Half precision (float16)	*_fp16.onnx
int8	8-bit integer (signed)	*_int8.onnx
uint8	8-bit integer (unsigned)	*_uint8.onnx
quantized	Dynamic quantization (avx512_vnni)	*_quantized.onnx
q4	4-bit weight quantization	*_q4.onnx
q4f16	4-bit weights + fp16 scales	*_q4f16.onnx
bnb4	BitsAndBytes NF4 quantization	*_bnb4.onnx

Usage with Optimum

from optimum.onnxruntime import ORTModelForSpeechSeq2Seq
from transformers import AutoProcessor
import librosa

model_id = "path/to/whisper-large-v2-finnish"
processor = AutoProcessor.from_pretrained(model_id)
model = ORTModelForSpeechSeq2Seq.from_pretrained(model_id, subfolder="onnx")

audio, sr = librosa.load("audio.wav", sr=16000, mono=True)

inputs = processor.feature_extractor(audio, sampling_rate=16000, return_tensors="pt")
gen_tokens = model.generate(**inputs, max_length=300)
print(processor.decode(gen_tokens[0], skip_special_tokens=True))

Usage with transformers.js

In a browser environment with AudioContext available:

import { pipeline } from "@huggingface/transformers";

const transcriber = await pipeline(
  "automatic-speech-recognition",
  "path/to/whisper-large-v2-finnish",
);
const result = await transcriber("audio.wav");
console.log(result.text);

In Node.js (AudioContext is not available, so audio must be loaded manually):

import { pipeline } from "@huggingface/transformers";
import fs from "fs";
import wavefile from "wavefile";

// Load and resample audio to 16kHz mono
const buffer = fs.readFileSync("audio.wav");
const wav = new wavefile.WaveFile(buffer);
wav.toBitDepth("32f");
wav.toSampleRate(16000);
const audioData = wav.getSamples(false, Float32Array);
// Mix to mono if stereo
const audio = Array.isArray(audioData)
  ? audioData[0].map((v, i) => (v + audioData[1][i]) / 2)
  : audioData;

const transcriber = await pipeline(
  "automatic-speech-recognition",
  "path/to/whisper-large-v2-finnish",
);
const result = await transcriber(audio);
console.log(result.text);

Original Model

This is an ONNX conversion of Finnish-NLP/whisper-large-v2-finnish. See the original model card for training details, performance metrics, and dataset information.

• Language: Finnish
• Task: Automatic Speech Recognition
• License: Apache 2.0
• Original authors: Finnish-NLP

Known Limitations

• No merged decoder: decoder_model_merged.onnx is not available for this model. The ONNX post-processing step that merges the decoder fails for large models due to the protobuf 2GB size limit. The separate decoder_model.onnx and decoder_with_past_model.onnx files are used instead — this is handled automatically by Optimum and does not affect functionality or accuracy. Quantized variants (*_merged_*) are also unavailable.

Conversion Details

• Exported with Hugging Face Optimum
• Quantized with ONNX Runtime quantization tools
• fp16 conversion via onnxconverter-common
• 4-bit quantization via MatMulNBitsQuantizer (q4) and MatMulBnb4Quantizer (bnb4)