File size: 4,843 Bytes
23ce103 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 | ---
license: other
license_name: nvidia-open-model-license
license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
language:
- en
library_name: asrjs/speech-recognition
pipeline_tag: automatic-speech-recognition
tags:
- automatic-speech-recognition
- speech
- audio
- onnx
- onnxruntime
- asrjs
- asrjs/speech-recognition
- nemo
- NeMo
- FastConformer
- RNNT
- streaming-asr
- eou
- end-of-utterance
- parakeet
- voice-agent
base_model: nvidia/parakeet_realtime_eou_120m-v1
models:
- nvidia/parakeet_realtime_eou_120m-v1
widget:
- example_title: Librispeech sample 1
src: https://cdn-media.huggingface.co/speech_samples/sample1.flac
- example_title: Librispeech sample 2
src: https://cdn-media.huggingface.co/speech_samples/sample2.flac
---
# Parakeet Realtime EOU 120M v1 ONNX
Converted ONNX package of [`nvidia/parakeet_realtime_eou_120m-v1`](https://huggingface.co/nvidia/parakeet_realtime_eou_120m-v1) for use with [`@asrjs/speech-recognition`](https://github.com/asrjs/speech-recognition).
This repository is not the original NVIDIA training checkpoint repo. It contains exported runtime artifacts for browser and Node.js inference.
## Included Artifacts
- `encoder-model.onnx`
- `decoder_joint-model.onnx`
- `encoder-model.fp16.onnx`
- `decoder_joint-model.fp16.onnx`
- `encoder-model.int8.onnx`
- `decoder_joint-model.int8.onnx`
- `vocab.txt`
- `config.json`
## Model Summary
Parakeet Realtime EOU 120M v1 is a streaming English ASR model with:
- cache-aware FastConformer encoder
- RNNT decoder
- explicit `<EOU>` token emission for end-of-utterance detection
- low-latency voice-agent-oriented output
The model:
- supports English only
- does not emit punctuation or capitalization
- may emit empty visible text for non-speech audio
- keeps `<EOU>` in raw/native output while user-visible text should strip it
Architecture details:
- FastConformer-RNNT
- 17 encoder layers
- about 120M parameters
## Frontend / Preprocessing
The upstream model expects raw 16 kHz mono audio and uses a NeMo mel frontend internally.
For `@asrjs/speech-recognition`, this ONNX package is intended to run with the shared in-repo JavaScript NeMo frontend. A dedicated `nemo80.onnx` or `nemo128.onnx` preprocessor is intentionally not required.
Frontend assumptions:
- sample rate: `16000`
- mono audio
- mel bins: `128`
- valid length mode: `centered`
- frontend output: raw log-mel features
This matters: this model does **not** use the normalized `nemo128` frontend contract reused by some other NeMo exports.
## Quantization Notes
Included variants:
- FP32
- FP16
- INT8 encoder
- INT8 decoder
Port validation summary on the smoke fixture:
- FP32: exact token/text/raw-text parity
- FP16: exact token/text/raw-text parity
- decoder-only INT8: exact token/text/raw-text parity
- encoder-only INT8: not exact
- full `int8/int8`: not exact
Recommended default pairings:
- `fp32/fp32`
- `fp16/fp16`
- `fp32/int8` if you specifically want decoder-only INT8
## Usage with `@asrjs/speech-recognition`
### Preset usage
```ts
import { createSpeechPipeline, PcmAudioBuffer } from '@asrjs/speech-recognition';
const pipeline = createSpeechPipeline({ cacheModels: true });
const loaded = await pipeline.loadModel({
preset: 'parakeet',
modelId: 'nvidia/parakeet_realtime_eou_120m-v1',
backend: 'wasm',
});
const audio = PcmAudioBuffer.fromMono(pcmFloat32, 16000);
const result = await loaded.transcribe(audio, {
detail: 'detailed',
responseFlavor: 'canonical+native',
});
console.log(result.canonical.text);
console.log(result.native.rawUtteranceText);
```
### Direct source usage
```ts
const loaded = await pipeline.loadModel({
family: 'nemo-rnnt',
modelId: 'nvidia/parakeet_realtime_eou_120m-v1',
backend: 'wasm',
options: {
source: {
kind: 'huggingface',
repoId: 'ysdede/parakeet-realtime-eou-120m-v1-onnx',
preprocessorBackend: 'js',
encoderQuant: 'fp32',
decoderQuant: 'fp32',
},
},
});
```
## Voice-Agent Context
The original model card highlights voice-agent usage, especially streaming end-of-utterance detection.
<div align="center">
<img src="./figure-streaming.png" width="450" />
</div>
<div align="center">
<img src="./voice-agent.png" width="750" />
</div>
## Upstream Model and License
Original model:
- [`nvidia/parakeet_realtime_eou_120m-v1`](https://huggingface.co/nvidia/parakeet_realtime_eou_120m-v1)
This converted package follows the upstream **NVIDIA Open Model License** terms.
## References
- [Fast Conformer With Linearly Scalable Attention For Efficient Speech Recognition](https://arxiv.org/abs/2305.05084)
- [Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognition](https://arxiv.org/abs/2312.17279)
- [NVIDIA NeMo](https://github.com/NVIDIA/NeMo)
|