Fractus CTE
A living AI that thinks continuously, remembers forever, and grows on its own.
What is Fractus?
Fractus is not a chatbot. It's not GPT. It's not a transformer.
Fractus is a Continuous Cognitive Agent — an AI that works like a brain, not a calculator. Instead of processing input → output in one pass, Fractus ticks like a biological system: it maintains a persistent thought state, advances it through multiple blocks of processing, remembers everything across sessions, and can grow new capacity by itself.
What makes it different from GPT/Claude?
| GPT-4 / Claude | Fractus | |
|---|---|---|
| Thinking | One pass, done | Continuous ticks (like a heartbeat) |
| Memory | Forgets when context window fills | Remembers forever (survives restarts) |
| Learning | Retrain from scratch ($$$) | Learns from every interaction |
| Growth | Fixed size forever | Grows new experts at runtime |
| Mental states | One mode always | Shifts between cognitive modes |
| Where it runs | Corporate cloud | Your machine |
The 12 Building Blocks
| Block | What it does |
|---|---|
| Continuous Thought Engine | The brain — thinks tick by tick through 16 blocks |
| Persistent Memory | Remembers you across sessions, never forgets |
| Cognitive Modes | Shifts mental states (focused, creative, exploratory...) |
| RAG Knowledge Base | Learns facts instantly — no retraining needed |
| Cognitive Plugins | Hot-swappable modes: analyst, coder, creative, teacher |
| MetaCognition | Decides its own actions: retrieve, learn, generate |
| Progressive Growth | Grows from 6M to 1B+ params, palier by palier |
| Self-Modification | Adds new experts at runtime when it needs them |
| PhaseRoutedMoE | Sparse experts routed by oscillator phases |
| Kuramoto Clock | A dynamical system that drives routing decisions |
| Online Trainer | Learns continuously, one chunk at a time |
| HF Space | Live chat demo with shared memory |
How to Use
Install
git clone https://github.com/AFKmoney/fractus-cte.git
cd fractus-cte
pip install torch numpy tokenizers matplotlib fastapi uvicorn pydantic
Run tests
pytest tests/ -q
# → 28 passed (CTE + memory + MoE + multi-block + continuous thought)
Build a corpus
python scripts/build_quality_corpus.py
Train on CPU (progressive growth)
# Paliers 0-3: grows from 6M to 350M params
python scripts/train_progressive.py --paliers 0,1,2,3 --accumulation-steps 8
Train on GPU (1B scale)
# Grows from palier 3 checkpoint to 1B, then trains
python scripts/train_1b_gpu.py \
--checkpoint checkpoints/fractus_palier3.pt \
--tokens 500000000 \
--batch-size 8 \
--bf16 \
--accumulation-steps 4
Use the agent
from fractus.continuous_engine import ContinuousThoughtEngine
from fractus.memory import PersistentMemory
from fractus.tokenizer import FractusTokenizer
# Build the brain
engine = ContinuousThoughtEngine(
vocab_size=50257, d_model=128, n_heads=2, d_head=64,
n_layers=2, n_levels=2, n_oscillators=8, coupling_rank=4,
n_experts=4, top_k=2, expert_d_ff=128, siren_rank=32)
# Give it memory
memory = PersistentMemory(d_model=128, path="~/.fractus/memory.pt")
engine.attach_memory(memory)
# Think
engine.reset_thought(batch_size=1)
logits, confidence = engine.tick(torch.tensor([42]))
print(f"Confidence: {confidence.item():.2f}")
The Growth Path
Fractus grows like a brain — small at first, bigger over time:
| Stage | Size | Blocks | Experts | What it can do |
|---|---|---|---|---|
| Palier 0 | 6.6M | 1 | 4 | Learn basic patterns |
| Palier 1 | 25M | 2 | 8 | Simple text generation |
| Palier 2 | 120M | 4 | 16 | Coherent fragments |
| Palier 3 | 350M | 8 | 32 | Decent text quality |
| Palier 4 | 1B | 16 | 128 | Full language model |
Each stage inherits the previous one's knowledge. The model never starts from zero.
Architecture (for developers)
fractus-cte/
├── fractus/
│ ├── continuous_engine.py ← The brain (CTE + CTEBlock)
│ │ ├── CTEBlock One block: attention + Kuramoto + MoE
│ │ └── ContinuousThoughtEngine Stacks N blocks, carries thought state
│ ├── memory.py ← Cross-session persistent memory
│ ├── cognitive_modes.py ← Unsupervised mental state detection
│ ├── grow.py ← Progressive growth operator
│ ├── rag.py ← Knowledge base + plugins + metacognition
│ ├── tokenizer.py ← GPT-2 BPE tokenizer
│ ├── nn/
│ │ ├── moe.py ← PhaseRoutedMoE (sparse, low-rank, differentiable)
│ │ ├── attention.py ← Multi-level causal linear attention
│ │ ├── phase_ode.py ← Kuramoto RK4 oscillators
│ │ └── lazy_siren.py ← Low-rank weight storage
│ └── train/
│ └── online.py ← Online trainer (SGD/AdamW, accumulation)
├── tests/ 28 tests
├── scripts/ Training + corpus + GPU scripts
├── space/ HF Space demo
├── docs/ Optimization analysis
├── Fractus_White_Paper.pdf Technical white paper v2.0
└── arxiv/ LaTeX source for arXiv submission
Key concepts
Tick: one step of thinking. The engine processes an observation, updates its thought state through all blocks, and optionally emits output.
Thought state: a vector h ∈ R^d_model that persists across ticks. It's the engine's "consciousness" — it carries context forward.
Chunk: 32 tokens processed in one forward pass (for speed). The thought state and per-block attention state carry between chunks.
Expert: a small neural network (low-rank W = scale·U@V^T) that specializes in certain types of thoughts. Only 2 out of 128 are active per token (sparse routing).
Kuramoto: coupled oscillators that produce phase vectors. These phases route tokens to the right experts. Think of it as the engine's "internal clock" — different phase patterns = different cognitive modes.
Research Results (Honest)
We tested alternative training methods. Both failed:
- Expert Decoupled Training (EDT): claimed 189x speedup. Reality: 19% worse than standard training. The pre-training objective doesn't align with the final task.
- Forward-Forward (Hinton 2022): local goodness signal. Reality: the model got worse. Local learning can't replace global backpropagation.
What works: standard gradient descent + our architectural optimizations = 1345 tokens/second on CPU (was 4 tok/s before).
License
MIT. Fractus belongs to you, not to a corporation.
Author
Philippe-Antoine Robert — 2026 — rpa.tu@proton.me
Links
- GitHub: github.com/AFKmoney/fractus-cte
- HuggingFace: huggingface.co/thefinalboss/fractus-cte
- White Paper: Fractus_White_Paper.pdf
- arXiv source: arxiv/main.tex