speech-enhancement / README.md
jj-wool's picture
Update speech-enhancement model card
2289e83 verified
|
Raw History Blame Contribute Delete
9.95 kB
---
license: other
license_name: aic-sdk-binary-license
license_link: https://huggingface.co/ai-coustics/speech-enhancement/blob/main/LICENSE
pipeline_tag: audio-to-audio
library_name: aic-sdk
tags:
- audio
- speech-enhancement
- speech-to-speech
- noise-reduction
- denoising
- dereverberation
- voice-isolation
- asr
- stt
- voice-ai
- voice-agents
- real-time
- ai-coustics
---
# Speech Enhancement β€” ai-coustics
**Real-time, ASR-optimized speech enhancement models for machines.**
Quail and Rook are production speech-to-speech models by [ai-coustics](https://ai-coustics.com/) that remove noise, reverb, and competing speakers from microphone audio in real time β€” **30 ms latency, no GPU needed, no ONNX dependency**. They run on the contact side of Voice AI stacks to make downstream STT, VAD, turn-taking, and speech-to-speech models reliable: *real-time audio intelligence that makes Voice AI work in production. Not just in the lab.* Inference runs through the [`aic-sdk`](https://pypi.org/project/aic-sdk/) Python package (also available for Rust, C/C++, Node.js, and WebAssembly).
## Setup
1. Install the SDK:
```bash
pip install aic-sdk
```
2. [Create a free SDK key](https://developers.ai-coustics.com) on the ai-coustics playground β€” 30-day trial included, no credit card required.
## Usage
```python
# initialize the speech enhancement processor
import aic_sdk as aic
model = aic.Model.from_file(aic.Model.download("quail-vf-2.2-l-16khz", "./models"))
config = aic.ProcessorConfig.optimal(model)
processor = aic.Processor(model, "{aic-sdk-key}", config) # get a free SDK key: https://developers.ai-coustics.com/login
# enhance mono float32 audio, one block at a time β€” real-time safe
enhanced_block = processor.process(audio_block) # shape: (config.block_size,)
```
### Complete example: enhance a WAV file
```python
import wave
import numpy as np
import aic_sdk as aic
license_key = "YOUR_SDK_KEY" # get a free SDK key: https://developers.ai-coustics.com/login
model = aic.Model.from_file(aic.Model.download("quail-vf-2.2-l-16khz", "./models"))
with wave.open("input.wav", "rb") as wf:
assert wf.getnchannels() == 1, "mono input required"
sample_rate = wf.getframerate()
pcm = np.frombuffer(wf.readframes(wf.getnframes()), dtype=np.int16).astype(np.float32) / 32768.0
config = aic.ProcessorConfig.optimal(model, sample_rate=sample_rate)
processor = aic.Processor(model, license_key, config)
enhanced = np.concatenate([
processor.process(pcm[i : i + config.block_size])
for i in range(0, len(pcm) - config.block_size + 1, config.block_size)
])
with wave.open("enhanced.wav", "wb") as wf:
wf.setnchannels(1)
wf.setsampwidth(2)
wf.setframerate(sample_rate)
wf.writeframes((enhanced * 32767).astype(np.int16).tobytes())
```
Tune the enhancement strength at runtime (useful to optimize for a specific STT engine):
```python
ctx = processor.get_context()
ctx.set_parameter(aic.ProcessorParameter.EnhancementLevel, 0.8) # 0.0–1.0
```
## Which model should I use?
| Family | Use it for | Notes |
|---|---|---|
| **Quail Voice Focus** | Voice agents with one primary speaker | Voice isolation: suppresses competing voices and background speech, isolates the foreground speaker. Up to 43% fewer word errors across major STT providers. 16 kHz (usable in 8 kHz pipelines). v2.2 adds far-field single-speaker support. Not suited for multi-speaker scenarios. |
| **Quail Multi Speaker** | Speech-to-text primer in noisy, far-field, multi-speaker conditions | Denoiser optimized for STT accuracy; up to 30% word-error-rate reduction. 8 and 16 kHz. |
| **Rook** | Human-to-human listening: conferencing, telephony | Perceptual enhancement of noise, reverb, and distortion at up to 48 kHz. Not recommended for Voice AI use cases β€” use Quail there. |
## Model variants
Weights are hosted on the ai-coustics artifact CDN. `aic.Model.download()` fetches the latest compatible artifact and verifies its checksum automatically; the direct links below are for manual download. **L** = best quality, **S** = smaller/faster.
### Quail Voice Focus (16 kHz)
| SDK model ID | Artifact | SHA-256 (prefix) |
|---|---|---|
| **`quail-vf-2.2-l-16khz`** β€” latest, recommended | [v7](https://artifacts.ai-coustics.io/models/quail-vf-2-2-l-16khz/v7/quail_vf_2_2_l_16khz_horgwub0_v14.aicmodel) | `9645543da790` |
| `quail-vf-2.2-s-16khz` | [v7](https://artifacts.ai-coustics.io/models/quail-vf-2-2-s-16khz/v7/quail_vf_2_2_s_16khz_gf70x7zf_v14.aicmodel) | `59a57bae4bcf` |
| `quail-vf-2.1-l-16khz` | [v5](https://artifacts.ai-coustics.io/models/quail-vf-2-1-l-16khz/v5/quail_vf_2_1_l_16khz_8xope536_v11.aicmodel) | `e3f6cd3fda62` |
| `quail-vf-2.1-s-16khz` | [v5](https://artifacts.ai-coustics.io/models/quail-vf-2-1-s-16khz/v5/quail_vf_2_1_s_16khz_5i8jb8of_v12.aicmodel) | `7d0cc51114b7` |
| `quail-vf-2.0-l-16khz` | [v2](https://artifacts.ai-coustics.io/models/quail-vf-2-0-l-16khz/v2/quail_vf_2_0_l_16khz_d42jls1e_v18.aicmodel) | `c33a73442e25` |
| `quail-vf-1.1-l-16khz` | [v1](https://artifacts.ai-coustics.io/models/quail-vf-1-1-l-16khz/v1/quail_vf_1_1_l_16khz_d00ghjzn_v15.aicmodel) | `e0337ec3388f` |
| `quail-vf-l-16khz` | [v1](https://artifacts.ai-coustics.io/models/quail-vf-l-16khz/v1/quail_vf_l_16khz_jc5pk1aa_v17.aicmodel) | `1509e36bd30c` |
### Quail Multi Speaker (speech-to-text primer)
| SDK model ID | Sample rate | Artifact | SHA-256 (prefix) |
|---|---|---|---|
| **`quail-ms-l-16khz`** β€” recommended | 16 kHz | [v7](https://artifacts.ai-coustics.io/models/quail-ms-l-16khz/v7/quail_ms_l_16khz_dtv5nvgu_v20.aicmodel) | `6c55eab06add` |
| `quail-ms-s-16khz` | 16 kHz | [v7](https://artifacts.ai-coustics.io/models/quail-ms-s-16khz/v7/quail_ms_s_16khz_ilmexgyt_v8.aicmodel) | `c468df3f0687` |
| `quail-ms-l-8khz` | 8 kHz | [v7](https://artifacts.ai-coustics.io/models/quail-ms-l-8khz/v7/quail_ms_l_8khz_mp51agn0_v13.aicmodel) | `9943a0e57123` |
| `quail-ms-s-8khz` | 8 kHz | [v7](https://artifacts.ai-coustics.io/models/quail-ms-s-8khz/v7/quail_ms_s_8khz_9q57ojki_v20.aicmodel) | `2f0c21c45972` |
| `quail-l-16khz` (legacy) | 16 kHz | [v6](https://artifacts.ai-coustics.io/models/quail-l-16khz/v6/quail_l_16khz_dtv5nvgu_v20.aicmodel) | `80d212880568` |
| `quail-s-16khz` (legacy) | 16 kHz | [v6](https://artifacts.ai-coustics.io/models/quail-s-16khz/v6/quail_s_16khz_ilmexgyt_v8.aicmodel) | `c41e369e7a33` |
| `quail-l-8khz` (legacy) | 8 kHz | [v6](https://artifacts.ai-coustics.io/models/quail-l-8khz/v6/quail_l_8khz_mp51agn0_v13.aicmodel) | `e9d0768cc276` |
| `quail-s-8khz` (legacy) | 8 kHz | [v6](https://artifacts.ai-coustics.io/models/quail-s-8khz/v6/quail_s_8khz_9q57ojki_v20.aicmodel) | `e666888f67c0` |
### Rook Multi Speaker (human listening)
| SDK model ID | Sample rate | Artifact | SHA-256 (prefix) |
|---|---|---|---|
| **`rook-ms-l-48khz`** β€” recommended | 48 kHz | [v7](https://artifacts.ai-coustics.io/models/rook-ms-l-48khz/v7/rook_ms_l_48khz_7n1r44ri_v23.aicmodel) | `e53fb6151ac5` |
| `rook-ms-s-48khz` | 48 kHz | [v7](https://artifacts.ai-coustics.io/models/rook-ms-s-48khz/v7/rook_ms_s_48khz_kg8kp9qm_v52.aicmodel) | `d08b1b773722` |
| `rook-l-48khz` | 48 kHz | [v6](https://artifacts.ai-coustics.io/models/rook-l-48khz/v6/rook_l_48khz_7n1r44ri_v23.aicmodel) | `be355b90be0e` |
| `rook-s-48khz` | 48 kHz | [v6](https://artifacts.ai-coustics.io/models/rook-s-48khz/v6/rook_s_48khz_kg8kp9qm_v52.aicmodel) | `8e2ac9750b10` |
| `rook-xs-48khz` | 48 kHz | [v1](https://artifacts.ai-coustics.io/models/rook-xs-48khz/v1/rook_xs_48khz_g4qwc6lv_v21.aicmodel) | `63bffca72730` |
| `rook-xxs-48khz` | 48 kHz | [v1](https://artifacts.ai-coustics.io/models/rook-xxs-48khz/v1/rook_xxs_48khz_wsur2zkw_v7.aicmodel) | `914b8deb45a4` |
| `rook-ms-l-16khz` | 16 kHz | [v7](https://artifacts.ai-coustics.io/models/rook-ms-l-16khz/v7/rook_ms_l_16khz_dtv5nvgu_v20.aicmodel) | `c85d729aca04` |
| `rook-ms-s-16khz` | 16 kHz | [v7](https://artifacts.ai-coustics.io/models/rook-ms-s-16khz/v7/rook_ms_s_16khz_ilmexgyt_v8.aicmodel) | `b3d4c49b6184` |
| `rook-l-16khz` | 16 kHz | [v6](https://artifacts.ai-coustics.io/models/rook-l-16khz/v6/rook_l_16khz_dtv5nvgu_v20.aicmodel) | `85bd01c975e9` |
| `rook-s-16khz` | 16 kHz | [v6](https://artifacts.ai-coustics.io/models/rook-s-16khz/v6/rook_s_16khz_ilmexgyt_v8.aicmodel) | `b6d129e32f79` |
| `rook-ms-l-8khz` | 8 kHz | [v7](https://artifacts.ai-coustics.io/models/rook-ms-l-8khz/v7/rook_ms_l_8khz_mp51agn0_v13.aicmodel) | `3b617e2d122e` |
| `rook-ms-s-8khz` | 8 kHz | [v7](https://artifacts.ai-coustics.io/models/rook-ms-s-8khz/v7/rook_ms_s_8khz_9q57ojki_v20.aicmodel) | `ca0734051114` |
| `rook-l-8khz` | 8 kHz | [v6](https://artifacts.ai-coustics.io/models/rook-l-8khz/v6/rook_l_8khz_mp51agn0_v13.aicmodel) | `07eca5332b03` |
| `rook-s-8khz` | 8 kHz | [v6](https://artifacts.ai-coustics.io/models/rook-s-8khz/v6/rook_s_8khz_9q57ojki_v20.aicmodel) | `0a1e0ff071ea` |
Full SHA-256 checksums for every artifact are listed next to each download at [artifacts.ai-coustics.io](https://artifacts.ai-coustics.io).
## Related ai-coustics models
- πŸŽ™οΈ **[Voice Activity Detection](https://huggingface.co/ai-coustics/voice-activity-detection)** β€” real-time, noise-robust turn-taking for voice agents
- πŸ” **[Audio Insight](https://huggingface.co/ai-coustics/audio-insight)** β€” predict and diagnose downstream failures in Voice AI stacks
## Resources
- πŸ“– Documentation & model guide: [docs.ai-coustics.com](https://docs.ai-coustics.com)
- 🧩 Framework quickstarts: [LiveKit](https://docs.ai-coustics.com/models/get-started/livekit-quickstart), [Pipecat](https://github.com/pipecat-ai/pipecat/blob/main/examples/voice/voice-aicoustics.py)
- πŸ“¦ Python SDK: [pypi.org/project/aic-sdk](https://pypi.org/project/aic-sdk/)
- πŸ”‘ Free SDK key: [developers.ai-coustics.com](https://developers.ai-coustics.com)
- 🏒 Enterprise licensing: [ai-coustics.com](https://ai-coustics.com/)