README.md

---
license: cc-by-sa-4.0
base_model: FunAudioLLM/CosyVoice2-0.5B
tags:
  - nanites
  - tts
  - text-to-speech
  - cosyvoice2
  - anonymized-voice
  - british-isles
  - scottish
  - irish
  - welsh
  - dialect
language:
  - en
pipeline_tag: text-to-speech
---

# nanites-isles-speech-{hamish,maisie,seamus,tegan,rhys}

Anonymized, dialect+gender-pooled British Isles English text-to-speech
voices — Scottish, Irish, and Welsh English as crowdsourced in the
**OpenSLR 83** dataset (Google, 2018–19). Built on
[CosyVoice2-0.5B](https://huggingface.co/FunAudioLLM/CosyVoice2-0.5B)
(Apache-2.0, from FunAudioLLM), fine-tuned separately per pool.

## Personas

Pooled voices ship under dialect-appropriate codenames so each identity
is a *persona*, never a single OpenSLR speaker:

| Persona | Dialect/gender | Pool | WER (medium.en) | Anon max cos |
|---|---|---|---|---|
| **Maisie** | Scottish female | 6 speakers / 54.4 min | 26%* | 0.911 |
| **Hamish** | Scottish male | 6 speakers / 53.4 min | 22%* | 0.782 |
| **Seamus** | Irish male | 3 speakers / 22.7 min (thin) | 4.2% | 0.895 |
| **Tegan** | Welsh female | 6 speakers / 60.1 min | 8.9% | 0.871 |
| **Rhys** | Welsh male | 6 speakers / 59.8 min | 0.0% | 0.883 |

\* Scots lines are deliberately written in Scots orthography (e.g.
"dinna fash", "ged your tea"); whisper normalizes them to standard
spelling, so WER is inflated by orthography, not speech errors. The
Irish/Welsh lines use standard English spelling and their WERs are
reflective.

In code/config the voices keep the functional names
`isles-speech-scottish-female` etc. (engine + dialect + gender are
metadata); the codenames are the display/persona names.

**Hugging Face repos are persona-named** (a persona gets its own repo;
a future voice gets a new codename, not a gender suffix):

- `Nanite-Labs/nanites-isles-speech-hamish` (Scottish male)
- `Nanite-Labs/nanites-isles-speech-maisie` (Scottish female)
- `Nanite-Labs/nanites-isles-speech-seamus` (Irish male)
- `Nanite-Labs/nanites-isles-speech-tegan` (Welsh female)
- `Nanite-Labs/nanites-isles-speech-rhys` (Welsh male)

## Why this approach

- **Engine.** Same rationale as the sister `smoky` suite: modern line is
  **CosyVoice 2.0** (FunAudioLLM), Apache-2.0. **XTTS is retired** in
  this project (dormant, CPML license, poor clustering/cloning control).
- **Identity.** A *regional anon voice* must not replicate any single
  OpenSLR speaker. CosyVoice2 SFT collapses each dialect+gender pool
  into **one pooled speaker ID** (`<|{pool}_pool|>`), so no single
  speaker is recoverable at synthesis time.
- **Anonymization.** Generation feeds the pooled campplus embedding
  (average of the pool's members) — no reference audio from any
  individual. Max cosine of the pooled embedding to the closest member
  is reported per voice above (0.78–0.91; smoky reference ~0.89–0.91).
  **Seamus's pool is thin** (3 speakers — the smallest OpenSLR set; the
  Irish female pool does not exist in OpenSLR 83, so Irish female is
  not offered): its anonymization margin and training-data size are
  inherently smaller than the 6-speaker pools.

## Stable persona (zero-shot anchoring)

SFT mode conditions identity only through the pooled embedding, and the
base speech flow can leak timbre from the token sequence — some
short/locked-phrase sentences **deterministically** land in a different
timbre region (diagnosed on Hamish: one sentence regen 3× identical but
cosine 0.58 from his anchor). Fix, adopted for all five voices:
**zero-shot anchoring.** Each voice dir carries `anchor.wav` +
`anchor.txt` (a pooled-blend clip chosen by the listener, so the anchor
is itself anonymized); `cosyvoice_infer` clones that clip's timbre for
every new sentence (cosine 0.83–0.94 vs the anchor across novel lines).
Inference is therefore persona-consistent per codename.

## Data & license

- **OpenSLR 83** — "Crowdsourced high-quality UK and Ireland English
  Dialect speech data set" (Google, 2018–19), **CC BY-SA 4.0**.
  Citation: Demirsahin et al., "Open-source Multi-speaker Corpora of
  the English Accents in the British Isles", LREC 2020.
- **CC BY-SA carries over to these voices**: derivatives (incl.
  synthesized redistribution) inherit share-alike. This differs from
  smoky (public-domain Hall corpus).
- OpenSLR 83 offers 6 dialects × male/female; this release covers
  Scottish (f+m), Irish (male), Welsh (f+m). The Irish female set is
  not offered because OpenSLR 83 only ships `irish_english_male.zip`.
- Audio: 48 kHz mono studio recordings → pooled 24 kHz clean sets with
  light spectral-gate denoise, energy-VAD trim, SNR ≥ 8 dB filter,
  per-pool caps (6 speakers / ~60 min / 10 min per speaker).

## Training

- **Base:** `FunAudioLLM/CosyVoice2-0.5B` (llm + flow + hift).
- **Data:** pooled chunk sets → kaldi-style dirs (10% cv) → campplus
  embeddings + speech tokens → parquet →
  `cosyvoice/bin/train.py` (`--model llm`, 1×GPU, bf16 AMP,
  constant-lr 1e-5, 60 epochs). One speaker ID per pool.
- **Guarding (8 GB GPU):** bitsandbytes AdamW8bit; non-LLM params
  frozen; bf16; gradient checkpointing; `KEEP_CKPT=12` in-run pruning
  of checkpoints (multi-speaker SFT writes ~1 GB/epoch → ~60 GB if
  unpruned).
- **Outputs:** `llm_avg.pt` per voice (**best-8 by val loss** via
  `average_model.py --val_best`); final voice dirs assembled with base
  flow/hift + baked `spk2info.pt` + anchor + samples.

## Evaluation

- WER: whisper `medium.en` on novel pool dialect lines (see table;
  target <12%; Scots lines orthography-inflated).
  Tools: `experts/isles/eval_wer.py`, `eval_texts.py`.
- Anonymization: `check_anon.py` — pooled embedding max cosine to a
  member, per voice (see table).
- Persona consistency: anchor-mode synthesis lands at cosine 0.83–0.94
  vs the anchor across novel lines (vs 0.58 drift pre-fix).

## Report to User

- [x] Whisper WER < 12% on held-out lines (per voice; see table)
- [x] Anonymization check (max cos 0.78–0.91)
- [x] Persona-consistent anchored synthesis (0.83–0.94 vs anchor)
- [x] All five voices signed off by listening; training data pruned
- [ ] Publish zips via `publish_cosyvoice.py --voice ...` (or all)