Model card

.gitattributes

@@ -38,3 +38,4 @@ lm-4gram-ro.arpa filter=lfs diff=lfs merge=lfs -text
 lm-4gram-ro.nemo filter=lfs diff=lfs merge=lfs -text
 gguf/jackrabbit-110m-ro.f16.gguf filter=lfs diff=lfs merge=lfs -text
 gguf/jackrabbit-110m-ro.q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+jackrabbit-hf-card.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,159 @@
+---
+license: cc-by-nc-4.0
+language:
+  - ro
+library_name: nemo
+pipeline_tag: automatic-speech-recognition
+base_model: nvidia/parakeet-tdt_ctc-110m
+tags:
+  - surogate
+  - surogate-speech
+  - jackrabbit
+  - fastconformer
+  - romanian
+model-index:
+  - name: jackrabbit-110m-ro
+    results:
+      - task: {type: automatic-speech-recognition}
+        dataset: {name: FLEURS ro_ro test (Open ASR Leaderboard protocol), type: google/fleurs, config: ro_ro, split: test}
+        metrics:
+          - {type: wer, value: 5.69, name: WER (CTC + 4-gram)}
+          - {type: wer, value: 7.56, name: WER (TDT greedy)}
+      - task: {type: automatic-speech-recognition}
+        dataset: {name: Common Voice 21 ro test (SpeD clip list), type: common_voice, config: ro, split: test}
+        metrics:
+          - {type: wer, value: 2.07, name: WER (CTC + 4-gram)}
+          - {type: wer, value: 2.19, name: WER (TDT greedy)}
+---
+
+<p align="center">
+  <img src="./jackrabbit-hf-card.png" width="800px"/>
+</p>
+
+<p align="center">
+    <a href="https://huggingface.co/collections/surogate/surogate-speech" target="_blank">Surogate Speech</a> |
+    <a href="https://github.com/invergent-ai/surogate" target="_blank">Training and Serving Engine</a> |
+    <a href="https://github.com/invergent-ai/surogate-speech" target="_blank">Toolkit and Evals</a> |
+    <br>
+    <b>License</b>: CC-BY-NC-4.0 | <b>Authors</b>: <a href="https://invergent.ai" target="_blank">Invergent</a>
+</p>
+
+# Jackrabbit 110M (Romanian)
+
+**Jackrabbit is Surogate's speech recognition family. This is the offline Romanian model:** a 116M-parameter
+FastConformer with two decoders on one encoder, TDT and CTC, that writes cased, punctuated Romanian and
+runs comfortably on a CPU. For live audio use its sibling,
+[Jackrabbit 110M Streaming](https://huggingface.co/surogate/jackrabbit-110m-ro-streaming).
+
+It is served fastest by the [**surogate** engine](https://github.com/invergent-ai/surogate), which runs the
+encoder and both decoders natively, with no Python in the serving path. Every number on this page was
+measured, and the harnesses are named, so you can reproduce them.
+
+## Accuracy
+
+Word error rate in %, lower is better. Every model ran through the same code on one RTX 5090 (bf16);
+other rows are our own runs of the public checkpoints, not their authors' figures.
+
+**FLEURS Romanian test** (883 clips), with the Open ASR Leaderboard's own runner
+(`nemo_asr/run_eval_ml.py`, commit `b2e6f04`) and its multilingual normalizer:
+
+| Model | Params | WER | Speed |
+|---|---:|---:|---:|
+| **Surogate Jackrabbit 110M**, CTC + 4-gram | 116M | **5.69** | 2,531× real time |
+| NVIDIA Canary 1B v2 | 1B | 5.95 | 853× |
+| **Surogate Jackrabbit 110M**, TDT greedy | 116M | 7.56 | 2,774× |
+| OpenAI Whisper large-v3 | 1.55B | 8.42 | 102× |
+| SpeD 110M | 110M | 8.60 | 2,810× |
+| NVIDIA Parakeet TDT 0.6B v3 | 0.6B | 11.58 | 2,246× |
+
+The CTC + 4-gram row switches the decoder with a seven-line change to the runner, published with the results.
+
+**Common Voice 21 Romanian test** (3,929 clips, the clip list from the
+[SpeD paper](https://arxiv.org/abs/2511.03361)), with the leaderboard normalizer and with SpeD's:
+
+| Model | WER, leaderboard norm | WER, SpeD norm |
+|---|---:|---:|
+| **Surogate Jackrabbit 110M**, CTC + 4-gram | **2.07** | 2.19 |
+| **Surogate Jackrabbit 110M**, TDT greedy | 2.19 | 2.30 |
+| SpeD 110M | 3.47 | 3.58 |
+| NVIDIA Canary 1B v2 | 8.65 | 8.87 |
+| OpenAI Whisper large-v3 | 8.94 | 9.16 |
+| NVIDIA Parakeet TDT 0.6B v3 | 10.06 | 10.19 |
+
+Our run of SpeD reproduces its published 3.57% on this set. Transcripts for every clip and model are in
+[surogate-speech-evals](https://huggingface.co/datasets/surogate/surogate-speech-evals).
+
+## Runs on
+
+| Runtime | Where | Decoding |
+|---|---|---|
+| [surogate engine](https://github.com/invergent-ai/surogate) | Linux, CPU or NVIDIA GPU | TDT, CTC, CTC + 4-gram |
+| NeMo (`jackrabbit-110m-ro.nemo`) | Linux or macOS; CPU or GPU | TDT, CTC, CTC + 4-gram |
+| NeMo-Speech.cpp (`gguf/`) | Linux CPU, no PyTorch | TDT (7.99% on FLEURS under the same scoring as the NeMo file's 8.15%) |
+
+
+## Running it
+
+### With surogate (recommended)
+
+```bash
+docker pull ghcr.io/invergent-ai/surogate:<ENGINE_VERSION>
+docker run --gpus all -p 8000:8000 ghcr.io/invergent-ai/surogate:<ENGINE_VERSION> \
+  serve --stt surogate/jackrabbit-110m-ro --host 0.0.0.0 --port 8000
+
+curl http://localhost:8000/v1/audio/transcriptions -F file=@interviu.wav -F model=surogate/jackrabbit-110m-ro
+```
+
+The API is OpenAI-compatible. Drop `--gpus all` to run on CPU. Server options are in the
+[engine's speech docs](https://github.com/invergent-ai/surogate/blob/main/docs/inference/speech.md).
+
+### With surogate-speech (local, CPU is fine)
+
+```bash
+pip install "surogate-speech[asr] @ git+https://github.com/invergent-ai/surogate-speech"
+surogate-speech transcribe interviu.wav
+```
+
+### With NeMo
+
+```python
+from huggingface_hub import hf_hub_download
+from nemo.collections.asr.models import ASRModel
+from omegaconf import OmegaConf
+
+model = ASRModel.restore_from(hf_hub_download("surogate/jackrabbit-110m-ro", "jackrabbit-110m-ro.nemo"))
+model.change_decoding_strategy(decoder_type="rnnt")      # TDT greedy
+print(model.transcribe(["audio.wav"])[0].text)
+
+lm = hf_hub_download("surogate/jackrabbit-110m-ro", "lm-4gram-ro.nemo")   # loads in seconds
+model.change_decoding_strategy(OmegaConf.create({         # CTC + 4-gram, the best accuracy
+    "strategy": "beam_batch",
+    "beam": {"beam_size": 32, "ngram_lm_model": lm, "ngram_lm_alpha": 0.5, "beam_beta": 2.0,
+             "return_best_hypothesis": True, "allow_cuda_graphs": False},
+}), decoder_type="ctc")
+print(model.transcribe(["audio.wav"])[0].text)
+```
+
+## Files
+
+| file | size | what it is |
+|---|---|---|
+| `jackrabbit-110m-ro.nemo` | 466 MB | the model: FastConformer encoder (17 layers, width 512, 80 ms frames), TDT and CTC decoders, 2,048-piece SentencePiece vocabulary with case and punctuation |
+| `lm-4gram-ro.nemo` | 2.07 GB | the 4-gram language model for CTC beam search, in NeMo's fast-loading format |
+| `lm-4gram-ro.arpa` | 2.28 GB | the same language model as standard ARPA, for other decoders |
+| `gguf/` | 257 MB (F16), 150 MB (Q8_0) | the model for [NeMo-Speech.cpp](https://github.com/NVIDIA/NeMo-Speech.cpp), TDT decoder |
+
+The language model is built over the model's own 2,048 SentencePiece pieces, so it cannot be paired with
+another tokenizer. Input is 16 kHz mono; the engine and surogate-speech resample other formats.
+
+## Limitations
+
+Romanian only. Accuracy was measured on read and parliamentary speech; telephone audio, heavy noise,
+overlapping speakers and strong regional accents are not measured here. Numbers, acronyms and rare proper
+names cause a large share of the remaining errors. The model transcribes; it does not identify speakers.
+
+## License
+
+CC-BY-NC-4.0: free for research and other non-commercial use; for commercial use, contact
+[Invergent](https://invergent.ai). Fine-tuned from
+[nvidia/parakeet-tdt_ctc-110m](https://huggingface.co/nvidia/parakeet-tdt_ctc-110m) by NVIDIA (CC-BY-4.0).