---
license: apache-2.0
library_name: ggml
pipeline_tag: text-to-speech
language:
  - en
base_model: owensong/Inflect-Nano-v1
base_model_relation: quantized
tags:
  - text-to-speech
  - gguf
  - quantized
  - inflect-nano
  - english
---

<p align="center">
  <img src="https://huggingface.co/owensong/Inflect-Nano-v1/resolve/main/assets/inflect-nano-banner.png" alt="Inflect-Nano banner" width="100%">
</p>

# Inflect-Nano GGUF Quantized Weights

This repository contains quantized GGUF weights of [Inflect-Nano-v1 model](https://huggingface.co/owensong/Inflect-Nano-v1) for the [InflectNanoTTS C++/GGML runtime](https://github.com/remixer-dec/InflectNanoTTS.cpp).
The original model was published by its author, [owensong](https://huggingface.co/owensong) under Apache-2.0 license. Inflect-Nano-v1 aims to deliver ultra-lightweight english text to speech to edge devices.
  
The goal of this port was to run TTS on an ESP32 device and it has been [achieved](https://github.com/remixer-dec/PocketInkOS/tree/master/src/tts/). From a cold start to a spoken single word, the full neural vocoder takes around 26 seconds. In Griffin-Lim mode it takes around 6 seconds and is good enough for short phrases.

[![Static Badge](https://img.shields.io/badge/Fresh%20model%20releases%20%26%20news-Join%20the%20community-blue?style=flat&link=https%3A%2F%2Ft.me%2Fgenaimon)](https://t.me/genaimon)

## Repository Contents

```text
acoustic/
  inflect_acoustic_f16.gguf
  inflect_acoustic_q2_k.gguf
  inflect_acoustic_q3_k.gguf
  inflect_acoustic_q3_k_dynamic.gguf
  inflect_acoustic_q4_0.gguf
  inflect_acoustic_q4_0_E.gguf
  inflect_acoustic_q4_k.gguf
  inflect_acoustic_q5_0.gguf
  inflect_acoustic_q5_k.gguf
  inflect_acoustic_q6_k.gguf
  inflect_acoustic_q8_0.gguf
vocoder/
  inflect_vocoder_f16.gguf
  inflect_vocoder_q4_0.gguf
  inflect_vocoder_q4_0_E.gguf
  inflect_vocoder_q5_k.gguf
  inflect_vocoder_q8_0.gguf
cmudict.bin
cmudict.idx
```

Quality drop becomes significant lower than q3_k, others work relatively well.

The `inflect_acoustic_q4_0_E.gguf` and `inflect_vocoder_q4_0_E.gguf` files are experimental low-memory quants. They keep biases in fp16 and the last layer is also quantized to q4_0, making the pair small enough to fit in an ESP32's 8MB PSRAM.

## Usage

Build the runtime from the [InflectNanoTTS.cpp](https://github.com/remixer-dec/InflectNanoTTS.cpp) project:

```bash
./tools/build.sh
```

Run inference:

```bash
build/linux-arm64/inflect-nano \
  -a acoustic/inflect_acoustic_q4_0.gguf \
  -v vocoder/inflect_vocoder_q4_0.gguf \
  -d cmudict.bin \
  -t "Hello, this is a test." \
  -o output.wav
```

Adjust the binary path for your platform, for example `build/macos-arm64/inflect-nano` on Apple Silicon.

**cmudict.bin** is a compact binary pronunciation dictionary used by the text frontend.

**cmudict.idx** is an index file for `cmudict.bin` that makes dictionary initialization faster.

It is derived from CMU Pronouncing Dictionary: English words mapped to ARPAbet phoneme sequences. The project converts that text dictionary into a faster binary lookup format so the runtime can avoid parsing the full text dictionary at startup.

Its purpose is to turn English words into phoneme IDs before acoustic inference. In practice: input text -> normalize/tokenize -> dictionary lookup for pronunciations -> phoneme/tone/lang IDs -> acoustic model -> mel -> vocoder audio.

cmudict.bin was obtained by converting [cmudict.rep from the original repo](https://huggingface.co/owensong/Inflect-Nano-v1/blob/main/third_party/tiny_tts_frontend/tiny_tts/text/cmudict.rep) with [this script](https://github.com/remixer-dec/InflectNanoTTS.cpp/blob/master/tools/compile_cmudict.py):

```python
python tools/compile_cmudict.py \
    third_party/tiny_tts_frontend/tiny_tts/text/cmudict.rep \
    cmudict.bin
```