Hugging Face's logo

Spaces:
LisaMegaWatts
/
VocabFusion-Pythia160m-Fused
Sleeping

App Files Community
VocabFusion-Pythia160m-Fused / README.md
LisaMegaWatts's picture
LisaMegaWatts
Upload folder using huggingface_hub
331c4cc verified
preview code
|
Raw History Blame Contribute Delete
1.38 kB
metadata 
title: VocabFusion Pythia160m Fused
emoji: 🧬
colorFrom: purple
colorTo: blue
sdk: docker
app_port: 7860
pinned: false
license: apache-2.0
tags:
  - symbiogenesis
  - vocab-fusion
  - pythia
  - pca-fusion
  - openai-compatible

VocabFusion-Pythia160m-Fused

VocabFusion-SymbioGPT architecture with knowledge from Pythia-160M (12 layers, d=768) projected via PCA decomposition and blended at alpha=0.3 (92% variance retained). 17.5M params.

The donor embedding (GPTNeoX, ~50k tokens, 128-dim) bridges to the 320-dim SymbioGPT body (4-organelle decoder) via a junction layer (Linear + LayerNorm). Output is weight-tied to the donor embedding.

API

OpenAI-compatible endpoints:

  • GET / — model info
  • GET /v1/models — list models
  • POST /v1/chat/completions — generate (supports stream: true)

Links