---
license: mit
language:
  - en
tags:
  - text-to-speech
  - tts
  - voice-cloning
  - coreml
  - apple-silicon
  - chatterbox
library_name: coreml
base_model: ResembleAI/chatterbox
inference: false
---

# Chatterbox TTS — CoreML

CoreML port of [Chatterbox TTS](https://github.com/resemble-ai/chatterbox) by
Resemble AI. The full pipeline (T3 transformer → s3 speech tokenizer → s3gen
flow + flow estimator → mel-to-wav vocoder, plus the voice encoder and
CAMPPlus speaker encoder used for one-shot voice cloning) compiled to
`.mlpackage` for on-device inference on Apple Silicon Macs.

## Intended use

Drop-in for Apple platform apps that want offline, on-device TTS with optional
voice cloning. Built specifically to back the narration feature in
[Screen Cut Pro](https://example.invalid) — but the files are the model only;
nothing here is app-specific.

## Files

| File                                | Role                                                       |
| ----------------------------------- | ---------------------------------------------------------- |
| `t3_text_emb.mlpackage`             | T3 text token embedding                                    |
| `t3_text_pos_emb.mlpackage`         | T3 text positional embedding                               |
| `t3_speech_emb.mlpackage`           | T3 speech-token embedding                                  |
| `t3_speech_pos_emb.mlpackage`       | T3 speech-token positional embedding                       |
| `t3_cond_enc.mlpackage`             | T3 conditioning encoder (speaker + emotion + prompt)       |
| `t3_tfmr.mlpackage`                 | T3 Llama-style transformer (autoregressive backbone)       |
| `t3_speech_head.mlpackage`          | T3 speech-token output projection                          |
| `s3_tokenizer.mlpackage`            | s3 speech tokenizer (mel → speech tokens)                  |
| `voice_encoder.mlpackage`           | T3 speaker encoder (256-d embedding)                       |
| `campplus.mlpackage`                | CAMPPlus speaker encoder for s3gen (192-d embedding)       |
| `flow_encoder.mlpackage`            | s3gen flow encoder                                         |
| `flow_estimator.mlpackage`          | s3gen flow estimator (diffusion)                           |
| `mel2wav.mlpackage`                 | Vocoder (mel → 24 kHz waveform)                            |
| `default_t3_speaker_emb.bin`        | Bundled default voice — T3 speaker embedding               |
| `default_t3_cond_prompt_tokens.bin` | Bundled default voice — T3 conditioning prompt tokens      |
| `default_flow_prompt_token.bin`     | Bundled default voice — s3gen prompt tokens                |
| `default_flow_prompt_feat.bin`      | Bundled default voice — s3gen prompt features              |
| `default_flow_speaker_embedding.bin`| Bundled default voice — s3gen speaker embedding            |

## Conversion notes

Converted from the upstream PyTorch checkpoints with `coremltools`. A few
non-obvious patches were required to make the ONNX→CoreML path work:

- **s3 tokenizer RoPE** — the upstream uses `complex64` rotary embeddings,
  which CoreML does not support. Replaced with split real `(cos, sin)`
  tensors at trace time.
- **Fixed-length traces** — CoreML traces shapes statically; the flow encoder
  is traced at 400 prompt tokens / 1024 mel frames. Inputs longer than that
  must be chunked by the host application.
- **Mel features** — the upstream uses `librosa.filters.mel(slaney)` for
  voice encoding and `kaldi.fbank(htk)` for CAMPPlus. The conversion script
  bakes those windowing assumptions into the model where possible; the host
  app reproduces them where not. See `Tools/compare_mels.py` in the source
  app for verification.

The conversion script is in the source app's `Tools/` directory:
[`convert_chatterbox_to_coreml.py`](https://example.invalid).

## License

MIT, inherited from upstream Chatterbox. See [`LICENSE`](./LICENSE) for the
full text and [`NOTICE`](./NOTICE) for attribution and a summary of
modifications.

## Citation

If you use this in research, cite the upstream model:

```bibtex
@misc{chatterbox2024,
  title  = {Chatterbox},
  author = {Resemble AI},
  year   = {2024},
  url    = {https://github.com/resemble-ai/chatterbox}
}
```