---
title: VocabFusion Pythia160m Fused
emoji: 🧬
colorFrom: purple
colorTo: blue
sdk: docker
app_port: 7860
pinned: false
license: apache-2.0
tags:
  - symbiogenesis
  - vocab-fusion
  - pythia
  - pca-fusion
  - openai-compatible
---

# VocabFusion-Pythia160m-Fused

VocabFusion-SymbioGPT architecture with knowledge from Pythia-160M (12 layers, d=768) projected via PCA decomposition and blended at alpha=0.3 (92% variance retained). 17.5M params.

The donor embedding (GPTNeoX, ~50k tokens, 128-dim) bridges to the 320-dim SymbioGPT body (4-organelle decoder) via a junction layer (Linear + LayerNorm). Output is weight-tied to the donor embedding.

## API

OpenAI-compatible endpoints:

- `GET /` — model info
- `GET /v1/models` — list models
- `POST /v1/chat/completions` — generate (supports `stream: true`)

## Links

- **Fused model**: [LisaMegaWatts/VocabFusion-Pythia160m-Fused](https://huggingface.co/LisaMegaWatts/VocabFusion-Pythia160m-Fused)
- **Base VocabFusion**: [LisaMegaWatts/VocabFusion-SymbioGPT-10M](https://huggingface.co/LisaMegaWatts/VocabFusion-SymbioGPT-10M)
- **Source model**: [EleutherAI/pythia-160m](https://huggingface.co/EleutherAI/pythia-160m)
- **Base architecture**: [LisaMegaWatts/SymbioGPT-10M](https://huggingface.co/LisaMegaWatts/SymbioGPT-10M)
- **GitHub**: [DavinciDreams/SymbioGPT](https://github.com/DavinciDreams/SymbioGPT)