aufklarer/CosyVoice3-0.5B-MLX-8bit-full

CosyVoice3 0.5B — MLX 8-bit (LLM + Flow)

MLX port of FunAudioLLM/Fun-CosyVoice3-0.5B-2512 for Apple Silicon, with 8-bit quantization on both the LLM and the flow-matching DiT (the "full 8-bit" bundle). HiFi-GAN stays float32 — vocoder quality degrades noticeably under quantization.

Sibling bundles in the aufklarer/ org:

• CosyVoice3-0.5B-MLX-4bit — smallest (~600 MB), 4-bit everywhere
• CosyVoice3-0.5B-MLX-8bit — LLM 8-bit, DiT 4-bit (~900 MB)
• CosyVoice3-0.5B-MLX-8bit-full (this repo) — LLM 8-bit, DiT 8-bit (~1.1 GB)

Bundle contents

file	size	purpose
llm.safetensors	640 MB	Qwen2-derived text→speech-token LLM
flow.safetensors	342 MB	DiT flow-matching speech-token→mel
hifigan.safetensors	79 MB	mel→24 kHz waveform vocoder
speech_tokenizer.safetensors	484 MB	s3tokenizer-v3 (reference audio → FSQ codes; required for zero-shot voice cloning)
config.json	—	model + quantization config

Total: ~1.5 GB.

Usage with speech-swift

import CosyVoiceTTS

let model = try await CosyVoiceTTSModel.fromPretrained(
    modelId: "aufklarer/CosyVoice3-0.5B-MLX-8bit-full"
)
let tokenizer = try SpeechTokenizerModel.fromSafetensors(
    at: cacheDir.appendingPathComponent("speech_tokenizer.safetensors"))

// Zero-shot voice cloning: prompt_token + prompt_feat carry the voice.
let refSamples = try AudioFileLoader.load(url: refURL, targetSampleRate: 16000)
let profile = try model.extractVoiceProfile(
    audio: refSamples, sampleRate: 16000,
    speechTokenizer: tokenizer,
    referenceTranscript: "Exact transcript of the reference clip.")

let audio = model.synthesize(
    text: "Hello world.",
    promptToken: profile.promptToken,
    promptFeat: profile.promptFeat,
    promptText: profile.promptText)

Conversion

Converted from the PyTorch original via speech-models' models/cosyvoice-tts/export/convert.py:

python convert.py --llm-bits 8 --quantize-dit --flow-bits 8 \
    --output-dir cosyvoice3-mlx-8bit-full

Group size 64 for both LLM and Flow quantization. HiFi-GAN kept at float32. See the upstream README for the underlying model architecture and license.

License

Apache 2.0, inherited from the upstream FunAudioLLM/Fun-CosyVoice3-0.5B-2512.