tidelganesh/svara-tts-v1-bf16

bf16 re-upload of kenpath/svara-tts-v1. Weights cast from the original fp32/fp16 checkpoint to bfloat16; no quantization-aware training, no calibration. Quality is lossless relative to the upstream model.

Why bf16

	fp32 (upstream)	bf16 (this repo)
VRAM	~32 GB	~16 GB
Throughput vs fp32	1.0×	~1.8–2.2× on Ampere/Ada/Hopper
Quality	baseline	indistinguishable
Bitsandbytes / calibration	n/a	not required

bf16 is the sweet spot for serving TTS on a single 24 GB+ GPU. If you need to push VRAM further, see the int8 sister repo.

Hardware

GPU	bf16 fit	Notes
A100 40/80 GB	✅ comfortable	best throughput
A40 48 GB	✅ very comfortable	plenty of headroom for concurrency
L4 24 GB	✅ tight	use attn_implementation="flash_attention_2"
RTX 4090 24 GB	✅ tight	same
T4 16 GB	❌	use int8 instead
RTX 3090 24 GB	⚠️	may OOM on long contexts; use int8

Software: CUDA ≥ 11.8, PyTorch ≥ 2.3, transformers >= 4.45, accelerate, flash-attn (strongly recommended).

Usage

import torch
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained(
    "tidelganesh/svara-tts-v1-bf16",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="cuda",
    attn_implementation="flash_attention_2",
)
tokenizer = AutoTokenizer.from_pretrained(
    "tidelganesh/svara-tts-v1-bf16",
    trust_remote_code=True,
)

trust_remote_code=True is required because the upstream model ships custom modeling code in modeling_svara.py.

Serving

This repo is intended to back a self-hosted Svara-TTS FastAPI server. Point the server's model loader at this repo instead of the upstream one:

- model = AutoModel.from_pretrained("kenpath/svara-tts-v1", trust_remote_code=True)
+ model = AutoModel.from_pretrained(
+     "tidelganesh/svara-tts-v1-bf16",
+     trust_remote_code=True,
+     torch_dtype=torch.bfloat16,
+     device_map="cuda",
+     attn_implementation="flash_attention_2",
+ )

Server contract (POST /v1/text-to-speech, 24 kHz PCM16 mono) is unchanged.

How it was made

pip install -U "transformers>=4.45" accelerate "huggingface_hub[cli]" hf_transfer
pip install -U flash-attn --no-build-isolation
export HF_HUB_ENABLE_HF_TRANSFER=1
huggingface-cli login
export HF_TOKEN=hf_xxx...

python backend/quantize_and_push_svara.py   # ~10–20 min on A40
python backend/verify_svara_bf16.py         # round-trip check

The script:

1. Loads kenpath/svara-tts-v1 in bf16 with Flash-Attention 2.
2. Casts any stray fp32 parameters (custom code occasionally re-introduces them) back to bf16.
3. Asserts every parameter is bfloat16.
4. Saves with safe_serialization=True (safetensors) sharded at 2 GB.
5. Copies the upstream tokenizer / preprocessor / *.py files so trust_remote_code still works.
6. Pushes everything via model.push_to_hub + tokenizer.push_to_hub to this repo.

Limitations

• bf16 is not a quantization (int8/int4); it's a precision downcast with the same numeric range as fp32. There is no measurable quality loss.
• Custom modeling code is re-shipped from the upstream repo verbatim. If kenpath/svara-tts-v1 updates its modeling_svara.py, this repo must be re-pushed.
• Not a fine-tune. The model is identical to upstream except for dtype.

Provenance

• Base model: kenpath/svara-tts-v1
• Conversion: bf16 cast + safetensors re-serialization
• Tooling: transformers, accelerate, bitsandbytes (not used for this repo), huggingface_hub, flash-attn

Related

• tidelganesh/svara-tts-v1-bnb-8bit — bitsandbytes int8 weights (~50% of bf16 VRAM, minimal quality loss on TTS, listen-test before production).

License

Inherits the upstream license. Please refer to kenpath/svara-tts-v1 for terms.