mlx-community/fish-audio-drama-3-3bit

Drama 3 MLX 3-bit

Built with Fish Audio.

Fish Audio Drama 3 converted for mlx-audio on Apple Silicon. This is an unofficial community conversion; Fish Audio did not release or endorse it.

• routed experts: 3-bit, dequantized from the block-FP8 release
• dense projections and audio tower: 8-bit
• 18.5 GB on disk, about 22.5–24 GB peak memory while generating, so it fits a 32 GB Mac

In our tests it kept the quality of 8-bit experts. It is the smallest setting that did.

Converted and measured by Navneet Sharma (@Nav8ha).

Quality, measured

20 voice-cloned English clips (10 sentences × 2 seeds), M5 Max. Whisper large-v3-turbo word error rate, ECAPA speaker similarity to a held-out clip of the reference speaker, UTMOS. "Failed" = WER above 20 % or no end of speech.

Experts / dense	Backbone	WER	Failed	Speaker sim	UTMOS
8 / 8	34.3 GB	1.6 %	0/20	0.625	3.92
4 / 8	19.3 GB	2.5 %	0/20	0.619	3.92
3 / 8 (this model)	16.9 GB	1.4 %	0/20	0.610	4.04
2 / 8	11.8 GB	0.9 %	0/20	0.624	3.84
3 / 3	14.1 GB	26.7 %	9/20	0.561	3.13

• The row for this model was measured on this exact checkpoint through mlx-audio. Other rows come from the same sweep in a standalone MLX port.
• In a blind listening test (6 lines × 3 versions), the listener did not prefer 8-bit over this model.
• 2-bit experts dropped the last two lines of a multi-speaker dialogue, so they are not offered.
• Also checked on this setting: two-speaker dialogue, [sighs] / [laughs] cues, Hinglish, Hindi and a 43 s long-form script.

Usage

Drama 3 support is merged into mlx-audio (#1018). Until it is in a release, install from main:

pip install "mlx-audio[tts] @ git+https://github.com/Blaizzy/mlx-audio"

from mlx_audio.tts.utils import load_model

model = load_model("mlx-community/fish-audio-drama-3-3bit")

for result in model.generate(
    text="<|speaker:0|> So what did you ship this week? <|speaker:1|> [laughs] A spreadsheet.",
    ref_audio="reference.wav",
    ref_text="Transcript of the reference clip.",
):
    audio = result.audio  # 44.1 kHz

python -m mlx_audio.tts.generate --model mlx-community/fish-audio-drama-3-3bit \
    --text "[sighs] Fine. If we miss it, it's on you." \
    --ref_audio reference.wav --ref_text "Transcript of the reference clip." \
    --temperature 0.7 --top_p 0.7 --top_k 30

Use a 44.1 kHz reference recording if you can: Drama 3 copies the bandwidth of the reference. Speaker tags (<|speaker:0|> to <|speaker:15|>), multiple references and streaming are described in the model docs.

Speed: about 36 frames/s on an M5 Max (1.7× real time).

Conversion

Made with mlx-audio's converter:

python -m mlx_audio.tts.models.fish_drama3.convert --src drama-3 --dst drama-3-mlx-3bit \
    --expert-bits 3 --dense-bits 8 --tower-bits 8

Other versions: 4-bit (22 GB) and bf16 (67 GB).

NOTICE lists every change to the original weights.

License

The Fish Audio Community License applies, including its use restrictions and its terms for commercial use and hosted services. Do not clone a person's voice without their consent.