push folder mlx

README.md

@@ -0,0 +1,25 @@
+# canary-1b-v2 — MLX bf16
+
+MLX-native conversion of [nvidia/canary-1b-v2](https://huggingface.co/nvidia/canary-1b-v2)
+(FastConformer AED ASR, 25 European languages incl. French) for Apple Silicon, via
+[mlx-audio](https://github.com/Blaizzy/mlx-audio).
+
+- **Precision**: bf16 (~1.8 GB)
+- **Architecture**: unchanged from canary-1b-v2 (32-layer FastConformer encoder + 8-layer
+  Transformer decoder, 128-mel, `dw_striding` ×8, `rel_pos`).
+- **Files**: `model.safetensors`, `config.json` (mlx-audio canary schema), `tokenizer.model` (SPE).
+
+## How it was made
+Converted from the official `.nemo` via a raw NeMo state_dict dump (drop `preprocessor.*` and
+`num_batches_tracked`, cast bf16). mlx-audio `Canary.sanitize()` handles the NeMo→MLX key rename
+and the single conv transpose — so conv weights are kept in **raw PyTorch layout**. Pre-transposing
+them is the bug in earlier community checkpoints (`weight (256,3,1,3)` at load).
+
+## Usage notes (mlx-audio 0.4.3)
+- Decoder masks are float32 vs bf16 qkv → run the decoder in float32 (cast at load) **or** patch the
+  two `scaled_dot_product_attention` call sites to cast the mask to `q.dtype`.
+- Pass `source_lang`/`target_lang` to `model.generate` (the `language=` kwarg is silently filtered).
+- Greedy decoding loops on hard far-field audio → use `no_repeat_ngram_size=3`.
+
+## License
+**CC-BY-4.0**, derived from **nvidia/canary-1b-v2** (© NVIDIA). Attribution to NVIDIA required.

config.json

@@ -0,0 +1 @@
+{"preprocessor": {"sample_rate": 16000, "normalize": "per_feature", "window_size": 0.025, "window_stride": 0.01, "window": "hann", "features": 128, "n_fft": 512, "dither": 1e-05, "pad_to": 0, "pad_value": 0.0, "preemph": 0.97, "mag_power": 2.0}, "encoder": {"feat_in": 128, "n_layers": 32, "d_model": 1024, "n_heads": 8, "ff_expansion_factor": 4, "subsampling_factor": 8, "self_attention_model": "rel_pos", "subsampling": "dw_striding", "conv_kernel_size": 9, "subsampling_conv_channels": 256, "pos_emb_max_len": 5000, "causal_downsampling": false, "use_bias": true, "xscaling": false, "subsampling_conv_chunking_factor": 1, "att_context_size": [-1, -1]}, "transf_decoder": {"vocab_size": 16384, "hidden_size": 1024, "inner_size": 4096, "num_layers": 8, "num_attention_heads": 8, "pre_ln": true, "hidden_act": "relu", "pre_ln_final_layer_norm": true, "learn_positional_encodings": false, "max_sequence_length": 1024}, "head": {"num_layers": 1, "hidden_size": 1024, "num_classes": 16384}, "prompt_format": "canary2", "tokenizer": {"type": "sentencepiece", "model_path": "tokenizer.model"}, "model_type": "canary"}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8cf72fc07289b1c9e10af7c8a153a96bd7f91bc5a7726c6f351ef3dc93733457
+size 1960268929

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c36395c4fc6074512648baa557586c535f92b9d9682f66bf967bf4cc3ab749b8
+size 503803