smdesai/canary-1b-v2-coreml

Canary 1B v2 — CoreML (FP16, KV-cache)

On-device CoreML port of NVIDIA canary-1b-v2 for Apple silicon (iOS / macOS): FP32 preprocessor, FP16 Apple Neural Engine-resident FastConformer encoder, and a stateful KV-cache Transformer decoder driven by a host-side greedy loop. This is the reference (uncompressed) build; the memory-optimized variants below are derived from it and match its accuracy.

Base model: nvidia/canary-1b-v2 (NVIDIA NeMo EncDecMultiTaskModel, 1B parameters, 25 European languages, ASR + speech translation). License: the base model is released under CC-BY-4.0; this conversion carries the same license. Please attribute NVIDIA for the model.

This build

Encoder	FP16 weights, Neural Engine-resident, 1.58 GB
Decoder / cross-KV	FP16, 271 MB + 34 MB
Download	1.89 GB
iOS RAM while transcribing	1.9 GB (measured, 6-minute file)

Build family

All 1B v2 builds share the same preprocessor, tokenizer, package layout, and decode contract; only the weight format of the encoder and/or decoder differs. Download sizes are as hosted on the Hub; iOS RAM was measured in an app while transcribing a 6-minute file.

Repo	Build	Encoder weights	Decoder weights	Download	iOS RAM
canary-1b-v2-coreml (this repo)	FP16 (reference)	FP16	FP16	1.89 GB	1.9 GB (measured)
canary-1b-v2-int8-coreml	INT8 encoder	INT8 per-channel	FP16	1.10 GB	1.2 GB (measured)
canary-1b-v2-pal6-coreml	6-bit encoder	6-bit palette, g=16	FP16	0.92 GB	1.0 GB (measured)
canary-1b-v2-int8full-coreml	INT8 full	INT8 per-channel	INT8 per-channel	0.95 GB	1.0 GB (measured)
canary-1b-v2-pal6-int8-coreml	6-bit + INT8	6-bit palette, g=16	INT8 per-channel	0.77 GB	910 MB (measured)
canary-180m-flash-coreml	180M Flash	FP16 (17L, d=512)	FP16 (4L)	0.37 GB	470 MB (measured)

Files

File	Contents
canary_preprocessor.mlmodelc	FP32 mel front end (128 mel bins, 16 kHz mono, 15 s window)
canary_encoder.mlmodelc	FastConformer encoder, 32 layers, d=1024, subsampling 8
canary_cross_kv.mlmodelc	Cross-attention K/V projection, 8 layers (run once per window)
canary_decoder_kv.mlmodelc	Stateful single-step Transformer decoder, 8 layers, 16384-way LM head
canary_spe.model	SentencePiece tokenizer (16384 pieces)
metadata.json	Shapes, decode geometry, seed tokens, and the weight recipe of this build

Credits

Model: NVIDIA NeMo team, canary-1b-v2, CC-BY-4.0.