trevest/parakeet-ultra-GGUF

Parakeet Ultra GGUF

GGUF conversion of moondream/parakeet-ultra for mudler/parakeet.cpp (MIT), a ggml ASR engine with an OpenAI-compatible POST /v1/audio/transcriptions HTTP server.

parakeet-ultra is a post-train of nvidia/parakeet-tdt-0.6b-v3 by Moondream -- same architecture, same tokenizer, same 0.6B parameter count, better WER than v3 on every benchmark on its model card. License is CC-BY-4.0, inherited unchanged from the original.

Files

File	Notes
parakeet-ultra.f16.gguf	1.34 GiB. All weights F32 except the allowlisted ggml_mul_mat linear layers, which are F16.
parakeet-ultra.q8_0.gguf	897 MiB. Same allowlist quantized to Q8_0 instead.

Both were produced by the exact metadata/tensor-writing logic of parakeet.cpp's own scripts/convert_parakeet_to_gguf.py (same should_quantize allowlist, same KV schema parakeet.* that the current C++ loader (src/model_loader.cpp) reads) at parakeet.cpp commit e75de9b (tag v0.5.0, 2026-08-18).

Why not the stock converter

parakeet.cpp's own converter requires nemo_toolkit[asr] and loads the checkpoint through nemo.collections.asr.models.ASRModel.from_pretrained/.restore_from, i.e. it expects a NeMo .nemo archive. moondream/parakeet-ultra ships HF-transformers format only (model.safetensors + config.json, architecture ParakeetForTDT) -- there is no .nemo for it, so the stock converter cannot load it directly.

Conversion method

Rather than guess a tensor-name mapping, this used the exact inverse of the mapping HF's own transformers library uses to go the other way: NEMO_TO_HF_WEIGHT_MAPPING + NEMO_TDT_WEIGHT_MAPPING in transformers/models/parakeet/convert_nemo_to_hf.py.

1. Metadata + featurizer buffers (encoder/decoder/joint hyperparameters, the mel filterbank and window buffers, the 8192-piece vocabulary) were sourced from nvidia/parakeet-tdt-0.6b-v3's own .nemo archive, since every hyperparameter in ultra's HF config.json matches v3's NeMo config exactly (confirmed field by field) -- ultra is architecturally identical to v3, only the weights differ.
2. Every weight tensor came from ultra's own model.safetensors, renamed from HF-Parakeet naming back to NeMo naming via the inverse mapping above.
3. Validation, since v3 uniquely ships both formats on its own HF repo: applying the same inverse map to v3's own model.safetensors reproduces v3's real .nemo tensor names and shapes with 0 missing and 0 mismatches across 699 non-scalar tensors (the only extras are num_batches_tracked scalars, which the converter already discards). The resulting GGUF's KV keys and tensor names were also diffed directly against parakeet.cpp's C++ loader source (kv_u32(gguf_, "parakeet.encoder.d_model")-style reads in model_loader.cpp, and the preprocessor.featurizer.* / decoder.prediction.dec_rnn.lstm.weight_ih_l0-style tensor reads in mel.cpp / prediction.cpp) to confirm every emitted name is exactly what the engine expects.
4. Transcription check: parakeet-cli transcribe (built locally, CUDA, sm_120) against tests/fixtures/speech.wav (7.435 s LibriSpeech-style clip) produced a byte-identical transcript from this ultra GGUF and from a v3 GGUF built the same way: "Well, I don't wish to see it any more, observed Phoebe, turning away her eyes. It is certainly very like the old portrait."
5. Server check: parakeet-server (built locally from the same commit, CUDA, sm_120) served POST /v1/audio/transcriptions with response_format=verbose_json and timestamp_granularities[]=word correctly (per-word start/end/confidence). Measured on an RTX 5090 Laptop GPU: RTF ~39x realtime (7.435 s audio in 0.189 s wall), 1.75 GiB VRAM resident.

Deliberately excluded: vad_head.*

moondream/parakeet-ultra's checkpoint carries six extra tensors (vad_head.proj/ctx/out, ~213K params) that do not exist in v3 and are not part of the base NeMo Parakeet-TDT architecture. Per the model's own card, these back a small voice-activity head Moondream's own Photon runtime uses to segment long audio at pauses. parakeet.cpp's engine has no long-audio segmentation pipeline that would read them, so they are excluded from this GGUF rather than shipped unused.

Credits

• moondream/parakeet-ultra by Moondream -- the weights converted here. CC-BY-4.0.
• nvidia/parakeet-tdt-0.6b-v3 by NVIDIA -- the base architecture ultra was post-trained from, and the source of the metadata/featurizer buffers used in this conversion (both are the same math given the identical preprocessor config). CC-BY-4.0.
• mudler/parakeet.cpp by the LocalAI team -- the ggml engine and OpenAI-compatible server this GGUF targets. MIT.
• huggingface/transformers -- convert_nemo_to_hf.py supplied the NeMo<->HF tensor-name mapping this conversion inverted. Apache-2.0.