| title: VocabFusion Pythia160m Fused | |
| emoji: 🧬 | |
| colorFrom: purple | |
| colorTo: blue | |
| sdk: docker | |
| app_port: 7860 | |
| pinned: false | |
| license: apache-2.0 | |
| tags: | |
| - symbiogenesis | |
| - vocab-fusion | |
| - pythia | |
| - pca-fusion | |
| - openai-compatible | |
| # VocabFusion-Pythia160m-Fused | |
| VocabFusion-SymbioGPT architecture with knowledge from Pythia-160M (12 layers, d=768) projected via PCA decomposition and blended at alpha=0.3 (92% variance retained). 17.5M params. | |
| The donor embedding (GPTNeoX, ~50k tokens, 128-dim) bridges to the 320-dim SymbioGPT body (4-organelle decoder) via a junction layer (Linear + LayerNorm). Output is weight-tied to the donor embedding. | |
| ## API | |
| OpenAI-compatible endpoints: | |
| - `GET /` — model info | |
| - `GET /v1/models` — list models | |
| - `POST /v1/chat/completions` — generate (supports `stream: true`) | |
| ## Links | |
| - **Fused model**: [LisaMegaWatts/VocabFusion-Pythia160m-Fused](https://huggingface.co/LisaMegaWatts/VocabFusion-Pythia160m-Fused) | |
| - **Base VocabFusion**: [LisaMegaWatts/VocabFusion-SymbioGPT-10M](https://huggingface.co/LisaMegaWatts/VocabFusion-SymbioGPT-10M) | |
| - **Source model**: [EleutherAI/pythia-160m](https://huggingface.co/EleutherAI/pythia-160m) | |
| - **Base architecture**: [LisaMegaWatts/SymbioGPT-10M](https://huggingface.co/LisaMegaWatts/SymbioGPT-10M) | |
| - **GitHub**: [DavinciDreams/SymbioGPT](https://github.com/DavinciDreams/SymbioGPT) | |