iky1e/granite-speech-5.0-470m-turboctc-mlx-q6

Granite Speech 5.0 470M TurboCTC MLX — Q6

This repository contains 6-bit affine weight quantization, group size 64 for native Apple-silicon inference with the Granite-MLX Swift runtime.

It is converted from IBM's ibm-granite/granite-speech-5.0-470m-turboctc at revision 7e74c6438b7cfb5090cb6a131538f5e8515a7de3. The source model.safetensors SHA-256 is 8b98a8c34fd5fcb081caef719638eded31bb6d197d62053eefc5c1703aaf1ad4.

The conversion transposes PyTorch depthwise Conv1d kernels into MLX layout, removes training-only batch counters, converts retained floating-point tensors to FP16, and applies weight-only affine quantization where applicable. Activations remain floating point at runtime.

Family comparison

Variant	Repository	Weight file	Size vs source	Agreement with source
IBM source	ibm-granite/granite-speech-5.0-470m-turboctc	902.35 MiB	100.00%	100.0000%
FP16	iky1e/granite-speech-5.0-470m-turboctc-mlx-fp16	902.22 MiB	99.99%	99.9706%
Q8	iky1e/granite-speech-5.0-470m-turboctc-mlx-q8	466.03 MiB	51.65%	99.8825%
Q6 (this repository)	iky1e/granite-speech-5.0-470m-turboctc-mlx-q6	367.51 MiB	40.73%	99.6989%
Q5	iky1e/granite-speech-5.0-470m-turboctc-mlx-q5	311.22 MiB	34.49%	99.4712%
Q4	iky1e/granite-speech-5.0-470m-turboctc-mlx-q4	254.93 MiB	28.25%	98.4135%

“Agreement with source” is word-level transcript agreement, calculated as 100 − Levenshtein word edits / source words. It is not WER and does not measure correctness against a human transcript. The reference is the matching original IBM checkpoint loaded by the same native Swift runtime.

The test recording is a 6,118.72-second (101m58.72s) single-speaker Stanford CME295 lecture. All checkpoints used Granite-MLX's bounded-memory defaults: 122.88-second chunks, 20.48-second context, FP16 activations, greedy CTC decoding, and a 64 MiB MLX cache. The source transcript contained 13,615 words. Raw benchmark JSON and transcripts are preserved with the Granite-MLX project and will be published with its source repository.

Usage

granite-mlx /path/to/audio-or-video --model iky1e/granite-speech-5.0-470m-turboctc-mlx-q6

Granite-MLX accepts common audio and video files, downloading this repository automatically on first use. It can export TXT, SRT, WebVTT, JSON, or all formats.

Reproducing the conversion

uv run python Scripts/convert_granite.py \
  /path/to/source-checkpoint \
  /path/to/output-q6 \
  --precision fp16 \
  --quantization-bits 6 \
  --group-size 64 \
  --source-model-id ibm-granite/granite-speech-5.0-470m-turboctc \
  --source-revision 7e74c6438b7cfb5090cb6a131538f5e8515a7de3

The publication converter (Scripts/convert_granite.py) has SHA-256 83141299b6e680fbdc020aed6674683b5cac03685dec839a30767f55de0942fb. Validation used Granite-MLX's native Swift release build with Apple Swift 6.2.3, MLX Swift 0.31.4 (dc43e62d7055353c7f99fa071a4e71d29dfddc44), swift-transformers 1.3.3 (2fa33e1f5e7131a7fc64c28e6d161dcec0d24820), macOS 26.5.2, and Xcode 26.2.

License

These converted model weights retain the source model's Apache 2.0 terms. The Granite-MLX Swift software is separate from the weights; this model repository does not set the software's license.