Upload README.md

README.md

@@ -1,21 +1,256 @@
 ---
-base_model: hypaai/Hypa-Orpheus-3b-TTS-VC
-tags:
-- text-generation-inference
-- transformers
-- unsloth
-- llama
+library_name: transformers
+pipeline_tag: text-generation
 license: apache-2.0
+base_model:
+  - hypaai/Hypa-Orpheus-3b-TTS-VC
+datasets:
+  - hypaai/Hypa-Voices-snac
 language:
-- en
+  - en
+  - fr
+  - es
+  - pt
+  - ar
+  - sw
+  - pcm
+  - yo
+  - ha
+  - ig
+  - ibb
+  - tiv
+  - efi
+  - anw
+  - idc
+  - igl
+  - nup
+  - ego
+  - fon
+  - urh
+  - tw
+  - ibr
+tags:
+  - orpheus
+  - llama
+  - text-to-speech
+  - voice-cloning
+  - speech-synthesis
+  - snac
+  - multilingual
+  - low-resource
+  - underrepresented-languages
+  - unsloth
+  - lora
+  - qlora
+  - continued-finetuning
+  - hypa-ai
+---
+
+<div align="center">
+
+![Hypa-Orpheus 3B](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC/resolve/main/Hypa_Orpheus.jpg)
+
+**Extended continued fine-tune of Hypa-Orpheus 3B with a higher-capacity LoRA adapter (r=2048), merged to 16-bit bf16 for inference.**
+
+[![License: Apache 2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0)
+[![Base: Hypa-Orpheus 3B](https://img.shields.io/badge/Base-Hypa--Orpheus%203B-blue)](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC)
+[![Collection: Hypa-Orpheus](https://img.shields.io/badge/%F0%9F%A4%97%20Collection-Hypa--Orpheus-yellow)](https://huggingface.co/collections/hypaai/hypa-orpheus)
+[![Primary release](https://img.shields.io/badge/Primary-Hypa--Orpheus%203B-green)](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC)
+[![Trained with Unsloth](https://img.shields.io/badge/Trained%20with-Unsloth-orange)](https://github.com/unslothai/unsloth)
+
+</div>
+
+## Model Description
+
+**Hypa-Orpheus 3B Extended** (`hypaai/Hypa-Orpheus-3b-TTS-VC-ext`) is a **continued fine-tune** of the production [Hypa-Orpheus 3B](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC) release. It uses the same Hypa-Voices corpus, prompt formats, and SNAC training pipeline as the primary model, but attaches a **much higher-rank LoRA adapter** (`r=2048`, `α=2048`) on top of the merged Step **3a** checkpoint and trains further on the same precomputed integer SNAC shards.
+
+This is an **experimental capacity extension**, not a replacement for the primary release. It exists for researchers and deployers who want the strongest held-out loss we observed after continuing training with a larger adapter.
+
+**Collection:** [hypaai/hypa-orpheus](https://huggingface.co/collections/hypaai/hypa-orpheus)
+
+| Repository | Role | Best for |
+| --- | --- | --- |
+| [`hypaai/Hypa-Orpheus-3b-TTS-VC`](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC) | Primary merged release (r=512, step 44,334) | Default production deployment |
+| **`hypaai/Hypa-Orpheus-3b-TTS-VC-ext` (this repo)** | Extended merged release (r=2048, step 4,992) | Higher-capacity continuation checkpoint |
+| [`hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs`](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs) | Primary LoRA + TensorBoard | Primary adapter inspection and retraining |
+| [`hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs-ext`](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs-ext) | Extended LoRA + TensorBoard | Extended adapter inspection and retraining |
+
+| Property | Primary release | **Extended (this repo)** |
+| --- | --- | --- |
+| **Starting checkpoint** | `unsloth/orpheus-3b-0.1-ft` | `hypaai/Hypa-Orpheus-3b-TTS-VC` |
+| **LoRA rank / alpha** | r=512, α=512 | **r=2048, α=2048** |
+| **Trainable parameters** | 778M (19.07%) | **~3.11B (~76%)** |
+| **Adapter size** | ~3.1 GB | **~12.4 GB** |
+| **Effective batch size** | 96 | **256** |
+| **Optimizer steps (shipped)** | 44,334 | **4,992** |
+| **Best validation loss** | 3.513 | **3.438** |
+| **Approx. validation PPL** | 33.5 | **31.1** |
+
+Capabilities, languages, prompt structure, SNAC decode path, and intended use cases are identical to the primary model. See the main model card for the full description:
+
+**[hypaai/Hypa-Orpheus-3b-TTS-VC README](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC)**
+
+---
+
+## What Changed in This Run
+
+The extended run kept the same data mixture and response-only SNAC supervision, but increased adapter capacity and batch size while continuing from the already merged production weights:
+
+| Hyperparameter | Primary release | Extended run |
+| --- | --- | --- |
+| Base weights | 4-bit QLoRA on Orpheus FT | **Merged bf16 Hypa-Orpheus 3B** |
+| LoRA rank (r) | 512 | **2048** |
+| LoRA alpha (α) | 512 | **2048** |
+| Per-device batch size | 16 | 16 |
+| Gradient accumulation | 6 | **16** |
+| **Effective batch size** | **96** | **256** |
+| Learning rate | 5e-5 | 5e-5 |
+| LR schedule | cosine, 3% warmup | cosine, **498 warmup steps** |
+| Optimizer | AdamW 8-bit | AdamW 8-bit |
+| Weight decay | 0.01 | 0.01 |
+| Max grad norm | 1.0 | 1.0 |
+| Eval / save interval | 2,217 steps | **832 steps** |
+| Planned step budget | 44,334 | **16,626** |
+| **Shipped checkpoint** | step 44,334 | **step 4,992** |
+| Precision | bfloat16 | bfloat16 |
+| Random seed | 3407 | 3407 |
+
+The published extended checkpoint reflects **4,992 optimizer steps**, about **30% of one effective epoch** on the same interleaved TTS/VC mixture. Training was stopped and released at the validation-best step observed in that continuation segment.
+
+---
+
+## Evaluation and Checkpoint Selection
+
+TensorBoard logs for the extended adapter:
+
+**[Hypa-Orpheus 3B Extended LoRA training logs](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs-ext/tensorboard)**
+
+Headline metrics from the shipped continuation:
+
+| Metric | Value | Step |
+| --- | --- | --- |
+| Initial training loss | **3.582** | 1 |
+| Final training loss | **3.433** | 4,992 |
+| Initial validation loss | **3.496** | 832 |
+| **Best validation loss (shipped)** | **3.438** | **4,992** |
+
+Compared with the primary release validation loss of **3.513** at step **44,334**, the extended checkpoint improves held-out loss by **0.075** (approximate validation perplexity **33.5 → 31.1**). Validation loss improved at every logged evaluation interval in this continuation.
+
+| Step | Training Loss | Validation Loss | Val PPL | Gen. Gap |
+| --- | --- | --- | --- | --- |
+| 832 | 3.531 | 3.496 | 33.0 | −0.035 |
+| 1,664 | 3.491 | 3.478 | 32.4 | −0.013 |
+| 2,496 | 3.483 | 3.464 | 31.9 | −0.019 |
+| 3,328 | 3.469 | 3.456 | 31.7 | −0.012 |
+| 4,160 | 3.463 | 3.451 | 31.5 | −0.012 |
+| **4,992** | **3.433** | **3.438** | **31.1** | **+0.006** |
+
+As with the primary release, these scalars measure uncertainty over the supervised target SNAC stream. They do not, by themselves, prove better perceptual quality. The extended checkpoint should be treated as a **loss-improved continuation** pending formal listening and task benchmarks.
+
+Gradient norms remained stable throughout the run (roughly **0.18 → 0.11**).
+
 ---
 
-# Uploaded finetuned  model
+## Which Checkpoint Should I Use?
 
-- **Developed by:** hypaai
-- **License:** apache-2.0
-- **Finetuned from model :** hypaai/Hypa-Orpheus-3b-TTS-VC
+| Use case | Recommendation |
+| --- | --- |
+| **Default production** | [`hypaai/Hypa-Orpheus-3b-TTS-VC`](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC) |
+| **Smaller adapter / full primary training budget** | Primary LoRA repo |
+| **Lowest published held-out loss after continuation** | **This repo (`-ext`)** |
+| **Further high-rank experimentation** | Extended LoRA repo |
 
-This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
+Choose the primary release when you want the fully budgeted Step **3a** training run and the smaller r=512 adapter footprint. Choose this extended release when you want the best validation loss we published after continuing from that checkpoint with a r=2048 adapter, and can accept the larger merged weights and shorter continuation schedule.
+
+---
+
+## Quick Start
+
+### Load the extended merged model
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
+
+model_id = "hypaai/Hypa-Orpheus-3b-TTS-VC-ext"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+)
+```
+
+Prompt assembly, SNAC encode/decode, and vLLM serving follow the same conventions as the primary release. This repository includes the same `handler.py` inference entrypoint pattern as [`hypaai/Hypa-Orpheus-3b-TTS-VC`](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC).
+
+```bash
+vllm serve hypaai/Hypa-Orpheus-3b-TTS-VC-ext --dtype bfloat16 --max-model-len 4096
+```
+
+### Load the extended LoRA adapter
+
+See [`hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs-ext`](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs-ext) for adapter loading, merging, and TensorBoard links.
+
+---
+
+## Limitations (Extended Release)
+
+- **Shorter continuation budget.** The shipped extended checkpoint covers **4,992** steps, not the full planned **16,626**-step schedule.
+- **Much larger adapter.** r=2048 increases adapter storage and merge size roughly **4×** versus the primary release.
+- **Experimental status.** The primary release remains our default production artifact; this continuation is published for transparency and research comparison.
+- **Same perceptual eval gap.** Improved held-out loss does not yet come with published WER/CER, speaker-similarity, or MOS results.
+
+For shared limitations on languages, prompt sensitivity, and context length, see the [primary model card](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC).
+
+---
+
+## Released Artifacts
+
+- **Extended merged 16-bit model (this repo):** [`hypaai/Hypa-Orpheus-3b-TTS-VC-ext`](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC-ext)
+- **Extended LoRA + TensorBoard:** [`hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs-ext`](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs-ext)
+- **Primary merged release:** [`hypaai/Hypa-Orpheus-3b-TTS-VC`](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC)
+- **Primary LoRA + TensorBoard:** [`hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs`](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs)
+- **Training data (public subset):** [Hypa-Voices collection](https://huggingface.co/collections/hypaai/hypa-voices)
+
+For questions or contributions, contact **chris@hypaintelligence.com** or open an issue on Hugging Face.
+
+---
+
+## Citation
+
+If you use the extended Hypa-Orpheus checkpoint, please cite both the primary release and this continuation:
+
+```bibtex
+@misc{hypaorpheus2026,
+  title        = {Hypa-Orpheus 3B: Multilingual Text-to-Speech and Voice Cloning for Underrepresented Languages},
+  author       = {{Hypa Intelligence and AfroVoices}},
+  year         = {2026},
+  publisher    = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC}}
+}
+
+@misc{hypaorpheus2026ext,
+  title        = {Hypa-Orpheus 3B Extended: Continued Fine-Tuning with Higher-Capacity LoRA},
+  author       = {{Hypa Intelligence and AfroVoices}},
+  year         = {2026},
+  publisher    = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC-ext}}
+}
+```
+
+---
+
+## License
+
+Released under the **Apache License 2.0**.
+
+---
+
+## Acknowledgments
+
+Same upstream acknowledgments as the [primary Hypa-Orpheus release](https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC#acknowledgments): Canopy Labs (Orpheus), Meta AI (Llama), Hubert Siuzdak (SNAC), Unsloth, RunPod, Mozilla Common Voice, AfroVoices, and Hypa Intelligence Research (HaIR).
+
+---
 
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
+**Hypa Intelligence** • [Website](https://hypaintelligence.com) • [Hugging Face](https://huggingface.co/hypaai) • [Updates](https://hypaintelligence.com/updates)