README.md

---
license: apache-2.0
language:
  - yue
  - zh
  - nl
  - en
  - fr
  - de
  - it
  - ja
  - ko
  - pl
  - es
library_name: onnxruntime
pipeline_tag: text-to-speech
base_model: Audio8/Audio8-TTS-Preview-0.6b
tags:
  - onnx
  - int4
  - audio
  - text-to-speech
  - tts
  - voice-cloning
  - zero-shot
  - multilingual
---

<div align="center">

<img src="./20260729-124515.jpeg" alt="Audio8" width="760">

<h1>&nbsp;&nbsp;Audio8 TTS Preview 0.6B ONNX INT4</h1>

**SOTA-class multilingual TTS at compact scale, packaged for low-resource CPU inference.**

[![GitHub](https://img.shields.io/badge/GitHub-Audio8__TTS-black?style=for-the-badge&logo=github)](https://github.com/Audio8-AI/Audio8_TTS)
[![Base Model](https://img.shields.io/badge/%F0%9F%A4%97%20Base%20Model-0.6B-FFD21E?style=for-the-badge)](https://huggingface.co/Audio8/Audio8-TTS-Preview-0.6b)
[![Demo](https://img.shields.io/badge/Live-Demo-brightgreen?style=for-the-badge&logo=githubpages)](https://audio8-ai.github.io/Audio8_TTS/)
[![ONNX Runtime](https://img.shields.io/badge/ONNX-Runtime-005CED?style=for-the-badge&logo=onnx)](https://onnxruntime.ai/)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue?style=for-the-badge)](https://github.com/Audio8-AI/Audio8_TTS/blob/master/LICENSE)

</div>

Audio8 TTS Preview is a 0.6B-parameter multilingual text-to-speech model with
zero-shot voice cloning. This repository provides its CPU-oriented ONNX
deployment: weight-only INT4 DualAR models, an FP16 neural audio codec, the
tokenizer, and the optional FP16 encoder used to register reference voices.

> **Model files only.** Inference, streaming service, and voice-registration
> code live in the
> [Audio8 TTS repository](https://github.com/Audio8-AI/Audio8_TTS/tree/master/onnx_runtime).

## Why this ONNX release

| | Deployment characteristic |
|---|---|
| **CPU native** | ONNX Runtime `CPUExecutionProvider`; no CUDA requirement |
| **Small runtime** | No PyTorch, Transformers, or Hugging Face Hub dependency after download |
| **Low memory** | About 1 GiB after loading in the tested Apple M2 configuration |
| **Voice cloning** | Bundled FP16 codec encoder for reusable local voice profiles |
| **Local service** | CLI, web UI, HTTP API, streaming PCM, and OpenAI-compatible endpoint |

### Precision and footprint

| Component | Precision |
|---|---|
| Slow/Fast AR weights | Weight-only INT4 |
| Activations, hidden states, and KV cache | FP16 |
| Codec encoder and decoder | FP16 |
| Waveform output | FP32, 44.1 kHz mono |

Normal synthesis loads only the Slow AR, Fast AR, and codec decoder sessions.
On a 16 GB Apple M2 MacBook Air with five ONNX Runtime threads, the service
used about **1004 MiB after loading** and approximately **1.1-1.2 GiB at
synthesis peak**. Voice registration releases the online sessions before
loading the codec encoder; the measured registration peak was approximately
**1.55 GiB**. Actual memory use varies by platform and allocator behavior.

The online model files occupy about **572 MiB**. The complete repository,
including the optional voice-registration encoder, is about **968 MiB**.

## Supported Languages

<p align="center">
  <strong>Cantonese</strong> &nbsp;·&nbsp;
  <strong>Chinese</strong> &nbsp;·&nbsp;
  <strong>Dutch</strong> &nbsp;·&nbsp;
  <strong>English</strong><br>
  <strong>French</strong> &nbsp;·&nbsp;
  <strong>German</strong> &nbsp;·&nbsp;
  <strong>Italian</strong> &nbsp;·&nbsp;
  <strong>Japanese</strong><br>
  <strong>Korean</strong> &nbsp;·&nbsp;
  <strong>Polish</strong> &nbsp;·&nbsp;
  <strong>Spanish</strong>
</p>

> **Preview status:** Language coverage is intentionally limited in this
> release. For the best results, use one of the 11 recommended languages
> above. Broader multilingual coverage and Chinese dialect support are
> planned for future releases.

## Model Details

Audio8 TTS uses a DualAR architecture inspired by
[Fish Audio S2 Pro](https://github.com/fishaudio/fish-speech). The slow AR
transformer predicts one semantic token for each audio frame. The fast AR
transformer predicts the frame's codec codebooks, conditioned on the slow
hidden state and preceding codebooks.

| Component | Configuration |
|---|---|
| Main model | 601,159,424 parameters, excluding the codec |
| Slow AR | 24 layers, width 896, 14 attention heads, 2 KV heads |
| Fast AR | 4 layers, width 896, 14 attention heads, 2 KV heads |
| Acoustic tokens | 10 codebooks, 4,096 entries per codebook |
| Codec | 44.1 kHz, 2,048 samples per model frame (~21.5 frames/s) |
| Context | Up to 2,048 packed text/audio positions |
| Execution provider | ONNX Runtime CPU |

## Quick Start

Python 3.11 or newer is required. The current release is tested on macOS
arm64.

### 1. Download the code and model

```bash
git clone https://github.com/Audio8-AI/Audio8_TTS.git
cd Audio8_TTS/onnx_runtime

python3 -m pip install -U "huggingface_hub[cli]"
hf download Audio8/Audio8-TTS-Preview-0.6B-ONNX-INT4 --local-dir model
bash setup.sh
```

The model files are stored at this Hugging Face repository's root. Downloading
with `--local-dir model` creates the exact layout expected by the runtime:

```text
model/
├── slow_ar_int4.onnx(.data)
├── fast_ar_int4.onnx(.data)
├── codec_decoder_fp16.onnx(.data)
├── runtime_manifest.json
├── tokenizer/tokenizer.json
└── registration/
    ├── codec_encoder_fp16.onnx(.data)
    └── registration_manifest.json
```

### 2. Register a reference voice

Start the local service and open <http://127.0.0.1:8024>. Upload a 0.5-30
second reference recording, its exact transcript, and a voice name.

```bash
bash start_server.sh
```

The same operation is available through HTTP:

```bash
curl http://127.0.0.1:8024/api/voices/register \
  -F 'audio=@/absolute/path/reference.wav' \
  -F 'text=The exact transcript of the reference recording.' \
  -F 'name=speaker_a' \
  -F 'overwrite=false'
```

The encoder in `registration/` is loaded only while registering a voice. The
generated profile is stored locally and can be reused across requests.

### 3. Generate speech

```bash
bash run_infer.sh \
  --text "Welcome to Audio8 TTS ONNX Runtime." \
  --voice speaker_a \
  --max-new-tokens 256 \
  --output outputs/example.wav
```

The command writes `outputs/example.wav` and `[10, T]` codec codes to
`outputs/example.npy`.

### HTTP API

```bash
curl http://127.0.0.1:8024/api/tts \
  -H 'Content-Type: application/json' \
  -d '{"text":"Welcome to Audio8 TTS.","voice_name":"speaker_a","max_new_tokens":256}' \
  -o outputs/api.wav
```

### OpenAI-compatible API

```bash
curl http://127.0.0.1:8024/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"model":"arktts","input":"Welcome to Audio8 TTS.","voice":"speaker_a","response_format":"wav"}' \
  -o outputs/openai.wav
```

See the complete
[ONNX Runtime guide](https://github.com/Audio8-AI/Audio8_TTS/tree/master/onnx_runtime)
for streaming output, configuration, memory management, and service controls.

## Evaluation

The source Audio8 TTS Preview checkpoint is a compact first-tier model on
Seed-TTS and CV3 multilingual evaluation. See the
[base model card](https://huggingface.co/Audio8/Audio8-TTS-Preview-0.6b#evaluation)
for benchmark tables, methodology, and comparison notes.

INT4 quantization can change sampled token sequences, so quality should be
evaluated for each target language, voice, and deployment setting rather than
assuming bit-for-bit equivalence with the source checkpoint.

## Limitations and Responsible Use

- This is a Preview checkpoint with limited multilingual and dialect coverage.
- Very long, noisy, or incorrectly transcribed references can reduce stability
  and speaker similarity.
- Generated speech can be misused for impersonation or misinformation. Obtain
  consent before cloning a voice and clearly disclose synthetic audio where
  appropriate.
- Evaluate the model for accuracy, safety, and legal compliance before
  deployment.

## License and Acknowledgements

The code and model weights are released under the
[Apache License 2.0](https://github.com/Audio8-AI/Audio8_TTS/blob/master/LICENSE).
See the upstream
[NOTICE](https://github.com/Audio8-AI/Audio8_TTS/blob/master/NOTICE) for
attribution details.

We thank the Fish Audio team for publishing the DualAR architecture used in
Fish Audio S2 Pro.