---
language:
- de
license: mit
library_name: onnxruntime-genai
pipeline_tag: automatic-speech-recognition
tags:
- asr
- speech-recognition
- streaming
- conformer
- rnn-t
- transducer
- onnx
- onnxruntime
- onnxruntime-genai
- int8
- quantization
- german
- feather
- microsoft
base_model: nvidia/nemotron-speech-streaming-en-0.6b
model-index:
- name: feather-0.2b-onnx-int8
  results:
  - task:
      type: automatic-speech-recognition
      name: Automatic Speech Recognition
    dataset:
      type: google/fleurs
      name: FLEURS (de_de)
      config: de_de
      split: test
    metrics:
    - type: wer
      value: 10.22
      name: WER
  - task:
      type: automatic-speech-recognition
      name: Automatic Speech Recognition
    dataset:
      type: mozilla-foundation/common_voice_17_0
      name: Common Voice 17 (de)
      config: de
      split: test
    metrics:
    - type: wer
      value: 10.28
      name: WER
  - task:
      type: automatic-speech-recognition
      name: Automatic Speech Recognition
    dataset:
      type: facebook/multilingual_librispeech
      name: Multilingual LibriSpeech (German)
      config: german
      split: test
    metrics:
    - type: wer
      value: 9.28
      name: WER
  - task:
      type: automatic-speech-recognition
      name: Automatic Speech Recognition
    dataset:
      type: facebook/voxpopuli
      name: VoxPopuli (de)
      config: de
      split: test
    metrics:
    - type: wer
      value: 13.75
      name: WER
---

# Feather 0.2B — German ONNX (INT8)

> [!Note]
> **Feather** is Microsoft's newest family of mini speech-recognition models, designed
> for fast on-device transcription with a tiny CPU footprint. This checkpoint —
> `feather-0.2b-onnx-int8` — is the German-specialised member of the family: a
> streaming Conformer-Transducer (RNN-T) of about **200 M parameters**, quantised to
> **INT8 weight-only** and packaged for [`onnxruntime-genai`](https://github.com/microsoft/onnxruntime-genai).

The encoder is a "**LiteConformer**" — a slimmed-down Conformer variant that
diverges from the standard NVIDIA Nemotron Conformer in three ways, specifically
chosen to reduce the per-layer op count on CPU:

| Block | Standard Nemotron Conformer        | Feather LiteConformer encoder        |
|-------|------------------------------------|--------------------------------------|
| FFN   | **Macaron**: two FFN blocks per layer (`feed_forward1` + `feed_forward2`, each with `fc_factor = 0.5`) | **Single FFN** per layer (`feed_forward2` only, `fc_factor = 1.0`) — drops one `LayerNorm` + two `Linear` ops per layer |
| Norm  | `nn.LayerNorm` (weight + bias, mean-subtract + variance-normalise) | `nn.RMSNorm` (weight-only, no bias, no mean subtraction) — fewer ops + no centring pass |
| Activation | `Swish` / `SiLU` (`x * sigmoid(x)`) in FFN and convolution | `GELU` (matches faster ORT fused `BiasGelu` kernel) in FFN and convolution |

Across the 20-layer encoder this removes roughly **40 LayerNorm sub-ops** (centring
+ bias add) and **40 Linear projections** worth of FFN compute compared to the
Macaron baseline, while RMSNorm and GELU map directly to fused
`SkipSimplifiedLayerNormalization` / `BiasGelu` kernels in ONNX Runtime CPU EP.

After export, the encoder is graph-fused with `OrtTransformersOptimization`
(`model_type=conformer`, MHA fusion disabled for accuracy) and the encoder weights
are quantised with `MatMulNBitsQuantizer` (`bits=8`, `block_size=32`, symmetric,
k-quant). The decoder and joint networks are kept FP32 — they are tiny and
benefit little from quantisation. A bundled Silero VAD is included for
optional voice-activity-driven streaming.

## Model details

| Property                | Value                                        |
|-------------------------|----------------------------------------------|
| Architecture            | Conformer-Transducer (RNN-T), LiteConformer encoder + RNN-T decoder + joint |
| Parameters              | ~200 M                                       |
| Language                | German (`de`)                                |
| Sample rate             | 16 kHz                                       |
| Mel features            | 128                                          |
| Encoder layers / dim / heads | 20 / 768 / 12                           |
| Predictor (decoder)     | 2-layer LSTM, hidden 640                     |
| Tokenizer               | SentencePiece BPE, vocab 8192 (+1 blank)     |
| Streaming chunk         | 560 ms (`chunk_samples=8960` @ 16 kHz)       |
| Left context            | 70 frames                                    |
| Subsampling             | depthwise-striding, factor 8                 |
| Encoder quantisation    | INT8 weight-only (`MatMulNBits`, block 32, symmetric) |
| Decoder / joint dtype   | FP32                                         |
| Inference runtime       | `onnxruntime-genai` (Microsoft)              |
| File size (encoder)     | ~358 MB (graph + external data)              |

## Evaluation — German test sets, full splits

All numbers are streaming, greedy-RNNT, **CPU**, no VAD. Reference normaliser is
`BasicMultilingualTextNormalizer` from
[`huggingface/open_asr_leaderboard`](https://github.com/huggingface/open_asr_leaderboard).

| Dataset            | Split | Utts  | Hours | **WER** | RTF (CPU) |
|--------------------|-------|------:|------:|--------:|----------:|
| FLEURS `de_de`     | test  |   862 |  3.15 | **10.22 %** | 14.9× |
| Common Voice 17 `de`| test | 16,206| 27.96 | **10.28 %** | 12.5× |
| MLS `german`       | test  | 3,394 | 14.29 |  **9.28 %** | 14.1× |
| VoxPopuli `de`     | test  | 1,968 |  4.92 | **13.75 %** | 13.1× |

RTF measured single-threaded on a Linux x86-64 CPU host with `onnxruntime-genai`'s
streaming `Generator`. No batching.

### Comparison vs NVIDIA Nemotron Multilingual (German)

For reference, the official
[`nvidia/nemotron-asr-streaming-multilingual-0.6b`](https://huggingface.co/nvidia/nemotron-asr-streaming-multilingual-0.6b)
model — same architecture family, ~3× the parameters, covers 40+ languages —
evaluated on the same German test splits scores:

| Dataset            | Nemotron Multilingual (0.6B) | **Feather 0.2B (this model)** | Δ (Feather − Nemotron) |
|--------------------|-----------------------------:|------------------------------:|-----------------------:|
| FLEURS `de_de`     | 12.25 % | **10.22 %** | **−2.03** |
| Common Voice 17 `de`| 12.80 % | **10.28 %** | **−2.52** |
| MLS `german`       |  9.91 % |  **9.28 %** | **−0.63** |
| VoxPopuli `de`     | 16.90 % | **13.75 %** | **−3.15** |

Feather is German-specialised and a third of the size in terms of number of parameters, smaller than int4 quantised version of multilingual Nemotron, and it both transcribes
German more accurately and runs noticeably faster on CPU, in our streaming
benchmarks Feather achieves **~2× the RTF of Nemotron Multilingual 0.6B ONNX INT4** on the
same CPU under identical decoding settings.

## Usage (`onnxruntime-genai`)

```bash
pip install onnxruntime-genai soundfile huggingface_hub
```

```python
from huggingface_hub import snapshot_download
import onnxruntime_genai as og
import soundfile as sf
import numpy as np

model_dir = snapshot_download("nenad1002/feather-0.2b-onnx-int8")

# Load audio (16 kHz mono float32)
audio, sr = sf.read("your_audio.wav", dtype="float32")
if audio.ndim > 1:
    audio = audio.mean(axis=1)
assert sr == 16000

config = og.Config(model_dir)
model = og.Model(config)
tokenizer = og.Tokenizer(model)
params = og.GeneratorParams(model)

processor = og.StreamingProcessor(model)
processor.set_option("use_vad", "false")
generator = og.Generator(model, params)
tok_stream = tokenizer.create_stream()

CHUNK = 8960  # 560 ms @ 16 kHz, matches genai_config.json
text = ""
def drain():
    global text
    while not generator.is_done():
        generator.generate_next_token()
        toks = generator.get_next_tokens()
        if len(toks) > 0:
            t = tok_stream.decode(toks[0])
            if t: text += t

for s in range(0, len(audio), CHUNK):
    inputs = processor.process(audio[s:s+CHUNK].astype(np.float32))
    if inputs is not None:
        generator.set_inputs(inputs); drain()
inputs = processor.flush()
if inputs is not None:
    generator.set_inputs(inputs); drain()

print(text)
```

Or use the reference script
[`nemotron_speech.py`](https://github.com/microsoft/onnxruntime-genai/blob/main/examples/python/nemotron_speech.py)
from the `onnxruntime-genai` repository:

```bash
python nemotron_speech.py \
    --model_path /path/to/feather-0.2b-onnx-int8 \
    --audio_file your_audio.wav \
    --use_vad false -e cpu
```

## Files

| File                          | Purpose                                   |
|-------------------------------|-------------------------------------------|
| `encoder.onnx` (+ `.data`)    | INT8-quantised LiteConformer encoder      |
| `decoder.onnx` (+ `.data`)    | FP32 RNN-T predictor                      |
| `joint.onnx` (+ `.data`)      | FP32 RNN-T joiner                         |
| `silero_vad.onnx`             | Optional VAD (Silero, ONNX)               |
| `genai_config.json`           | `onnxruntime-genai` model config          |
| `audio_processor_config.json` | Audio front-end / mel config              |
| `tokenizer.json`, `tokenizer_config.json`, `vocab.txt` | SentencePiece BPE tokenizer files |

## Limitations

- **German only.** Inference on other languages is unsupported and will produce
  garbled output.
- **Streaming chunk = 560 ms.** The encoder caches are sized for this latency
  setting and cannot be changed at inference time.
- **CPU-optimised.** The encoder is INT8 weight-only quantised and graph-fused
  for ONNX Runtime CPU EP; GPU EP will run but will not benefit from the
  optimisations.
- **Reference text normalisation.** WER numbers above use the
  open-asr-leaderboard `BasicMultilingualTextNormalizer`; other normalisers
  (e.g. Whisper-style) will give different absolute numbers.

## License

MIT.

## Acknowledgements

- Base architecture: NVIDIA Nemotron Conformer-Transducer family.
- Optimisation stack: Microsoft Olive + ONNX Runtime + `onnxruntime-genai`.
- Voice activity: [Silero VAD](https://github.com/snakers4/silero-vad).

## Author

- **Nenad Banfic** — CoreAI, Microsoft