README.md

---
license: other
license_name: nvidia-open-model-license
license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
language:
- en
library_name: asrjs/speech-recognition
pipeline_tag: automatic-speech-recognition
tags:
- automatic-speech-recognition
- speech
- audio
- onnx
- onnxruntime
- asrjs
- asrjs/speech-recognition
- nemo
- NeMo
- FastConformer
- RNNT
- streaming-asr
- eou
- end-of-utterance
- parakeet
- voice-agent
base_model: nvidia/parakeet_realtime_eou_120m-v1
models:
- nvidia/parakeet_realtime_eou_120m-v1
widget:
- example_title: Librispeech sample 1
  src: https://cdn-media.huggingface.co/speech_samples/sample1.flac
- example_title: Librispeech sample 2
  src: https://cdn-media.huggingface.co/speech_samples/sample2.flac
---

# Parakeet Realtime EOU 120M v1 ONNX

Converted ONNX package of [`nvidia/parakeet_realtime_eou_120m-v1`](https://huggingface.co/nvidia/parakeet_realtime_eou_120m-v1) for use with [`@asrjs/speech-recognition`](https://github.com/asrjs/speech-recognition).

This repository is not the original NVIDIA training checkpoint repo. It contains exported runtime artifacts for browser and Node.js inference.

## Included Artifacts

- `encoder-model.onnx`
- `decoder_joint-model.onnx`
- `encoder-model.fp16.onnx`
- `decoder_joint-model.fp16.onnx`
- `encoder-model.int8.onnx`
- `decoder_joint-model.int8.onnx`
- `vocab.txt`
- `config.json`

## Model Summary

Parakeet Realtime EOU 120M v1 is a streaming English ASR model with:

- cache-aware FastConformer encoder
- RNNT decoder
- explicit `<EOU>` token emission for end-of-utterance detection
- low-latency voice-agent-oriented output

The model:

- supports English only
- does not emit punctuation or capitalization
- may emit empty visible text for non-speech audio
- keeps `<EOU>` in raw/native output while user-visible text should strip it

Architecture details:

- FastConformer-RNNT
- 17 encoder layers
- about 120M parameters

## Frontend / Preprocessing

The upstream model expects raw 16 kHz mono audio and uses a NeMo mel frontend internally.

For `@asrjs/speech-recognition`, this ONNX package is intended to run with the shared in-repo JavaScript NeMo frontend. A dedicated `nemo80.onnx` or `nemo128.onnx` preprocessor is intentionally not required.

Frontend assumptions:

- sample rate: `16000`
- mono audio
- mel bins: `128`
- valid length mode: `centered`
- frontend output: raw log-mel features

This matters: this model does **not** use the normalized `nemo128` frontend contract reused by some other NeMo exports.

## Quantization Notes

Included variants:

- FP32
- FP16
- INT8 encoder
- INT8 decoder

Port validation summary on the smoke fixture:

- FP32: exact token/text/raw-text parity
- FP16: exact token/text/raw-text parity
- decoder-only INT8: exact token/text/raw-text parity
- encoder-only INT8: not exact
- full `int8/int8`: not exact

Recommended default pairings:

- `fp32/fp32`
- `fp16/fp16`
- `fp32/int8` if you specifically want decoder-only INT8

## Usage with `@asrjs/speech-recognition`

### Preset usage

```ts
import { createSpeechPipeline, PcmAudioBuffer } from '@asrjs/speech-recognition';

const pipeline = createSpeechPipeline({ cacheModels: true });

const loaded = await pipeline.loadModel({
  preset: 'parakeet',
  modelId: 'nvidia/parakeet_realtime_eou_120m-v1',
  backend: 'wasm',
});

const audio = PcmAudioBuffer.fromMono(pcmFloat32, 16000);
const result = await loaded.transcribe(audio, {
  detail: 'detailed',
  responseFlavor: 'canonical+native',
});

console.log(result.canonical.text);
console.log(result.native.rawUtteranceText);
```

### Direct source usage

```ts
const loaded = await pipeline.loadModel({
  family: 'nemo-rnnt',
  modelId: 'nvidia/parakeet_realtime_eou_120m-v1',
  backend: 'wasm',
  options: {
    source: {
      kind: 'huggingface',
      repoId: 'ysdede/parakeet-realtime-eou-120m-v1-onnx',
      preprocessorBackend: 'js',
      encoderQuant: 'fp32',
      decoderQuant: 'fp32',
    },
  },
});
```

## Voice-Agent Context

The original model card highlights voice-agent usage, especially streaming end-of-utterance detection.

<div align="center">
  <img src="./figure-streaming.png" width="450" />
</div>

<div align="center">
  <img src="./voice-agent.png" width="750" />
</div>

## Upstream Model and License

Original model:

- [`nvidia/parakeet_realtime_eou_120m-v1`](https://huggingface.co/nvidia/parakeet_realtime_eou_120m-v1)

This converted package follows the upstream **NVIDIA Open Model License** terms.

## References

- [Fast Conformer With Linearly Scalable Attention For Efficient Speech Recognition](https://arxiv.org/abs/2305.05084)
- [Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognition](https://arxiv.org/abs/2312.17279)
- [NVIDIA NeMo](https://github.com/NVIDIA/NeMo)