# fractus-opt

Training-optimization workshop for
[fractus-cte](https://huggingface.co/thefinalboss/fractus-cte)
(Continuous Thought Engine — a dynamical system, not a transformer).

**Mission (2026-08-22): make phase-2 training (8×RTX 5090) faster and more
efficient — without ever breaking a living checkpoint.**

## The open-heart rule

Fractus is operable "open-heart": organs of the code are replaced while the
patient (the live training) keeps going. The rule is strict:

1. **Every change to the production path must be PROVEN mathematically
   equivalent to the reference** — forward, gradients, carried states
   (`tests/test_attention_equivalence.py`, 14 proofs).
2. The old implementation stays in the code as an executable reference
   (`_linear_attention_causal_einsum`).
3. The `.pt` checkpoints never change format; resumption happens at exact
   token offsets (`RESUME_MANIFEST`), zero progress thrown away.
4. Honest documented caveat: topk over von Mises gates is discrete — fp32
   rounding near a gate tie can flip the expert choice for isolated tokens
   (zero net effect). The engine tests bound this behavior instead of
   denying it.

## Contents

| Path | Role |
|---|---|
| `fractus/nn/attention.py` | Causal linear-attention kernel: einsum (reference O(C²·dH²)), **cumsum** (O(C·dH²)), **chunked** (flat memory, block²+dH²). Dispatcher `_linear_attention_causal_vectorized` + env `FRACTUS_ATTN_IMPL`. |
| `fractus/nn/ce.py` | `chunked_cross_entropy` (per-chunk checkpointing → logits never held for backward) + `sample_tokens_chunked` (memory-flat sampling for scheduled sampling). |
| `fractus/continuous_engine.py` | + method `tick_chunk_train_ce(obs, targets, ce_chunk, return_hidden)` — same forward, CE without dense logits; + pure-core block checkpointing (`_tick_chunk_core_pure`, `BLOCK_CKPT`). Nothing removed. |
| `scripts/fast4gpu_boost_v3.py` | Trainer v3 = consolidation of everything since v2: exact v1 semantics at ACCUM=1, zero-copy int32 memmap fetch (~27 GB RAM saved/pod), `CE_CHUNK`, `ACCUM`, `COMPILE`, `BLOCK_CKPT`, atomic checkpoint writes (tmp + rename), periodic resume manifest. |
| `scripts/fast4gpu_boost_v2.py` | Trainer v2 (kept for provenance; v3 supersedes it). |
| `scripts/pod_deploy_x8.sh` | One-shot detached pod deployer: env → assets from HF → smoke → launch → verify; stage markers make re-runs cheap. |
| `tests/` | 46 tests: repo suite (28) + equivalences (14) + v2 loop smoke (2) + block-ckpt (2). **46/46 passing** (2026-08-24). |
| `benchmarks/` | Attention micro-bench at real 1B shapes (subprocess-isolated cells) + end-to-end tok/s bench. |
| `docs/OPTIMIZATION_2026-08-22.md` | **THE document**: bottleneck diagnosis, mathematical proofs, step-by-step pod deployment guide, measured results (CPU / 6×5060 Ti / production 8×5090). |
| `docs/…` | Copies of the project documents (DISCOVERY_LOG, TRUSTED_LOSS, HOW_FRACTUS_IS_TRAINED, …). |

## Key results

CPU local (torch 2.9.1+cpu) — attention micro-bench, real 1B shapes
(G=B·40, C=128, dH=64), fwd+bwd:

| Impl | B=2 | B=4 | B=8 |
|---|---|---|---|
| einsum (ref) | 572 ms | 1127 ms | crash (memory) |
| cumsum | 799 ms | crash (memory) | crash |
| **chunked** | **32 ms** | **72 ms** | **150 ms** |

- chunked = **×15–25 vs reference**, flat memory ((G, block²+dH²)) where
  einsum/cumsum materialize (G, C, dH, dH) and explode.
- End-to-end CPU: 703 → **764 tok/s** (+8.7 %).
- Absolute CPU timings do not transfer to CUDA; relative ordering and memory
  profile do.

GPU validation (6× RTX 5060 Ti 16 GB, torch 2.11+cu128):

- chunked beats cumsum ×1.27 on GPU; full trainer sustained ~365 tok/s/GPU
  at B=3; without `BLOCK_CKPT` the 1B config does not fit in 16 GB.

Production run (8× RTX 5090 32 GB, 2026-08-26, live):

- Smoke B=8: **1,527 tok/s/GPU** @ 19.85 GB; sustained all-8:
  **~1,554–1,604 tok/s/GPU ≈ 12,600 aggregate**, lb = 14.028 stable.
- Phase-2 finish projected in **≈ 3.1 days** (~3,420M tokens remaining).

## Usage

```bash
# tests (no GPU required)
py -m pytest tests/ -q                      # 46 passed

# benchmarks
py benchmarks/bench_attention.py --iters 8  # per-kernel table
py benchmarks/bench_engine.py               # end-to-end tok/s

# trainer v3, one process per GPU (details: docs/OPTIMIZATION_2026-08-22.md §3)
CUDA_VISIBLE_DEVICES=$i GPU_ID=$i START_TOKEN=<manifest offset> \
  BATCH=8 CE_CHUNK=2048 FRACTUS_ATTN_IMPL=chunked BLOCK_CKPT=1 \
  python -u scripts/fast4gpu_boost_v3.py
```

Recommended escalation on a pod (validate ema_tf at every step):
code swap with same settings → `CE_CHUNK=2048` → `FRACTUS_ATTN_IMPL=chunked`
→ `BLOCK_CKPT=1` → `COMPILE=1` + higher `BATCH`.

## Local Windows environment

Python via the `py` launcher (torch 2.9.1+cpu); bare `python` = msys64
without torch. cp1252 console: no non-ASCII characters in prints.

---
*Work done 2026-08-22/23 with Claude Code; GPU validation 2026-08-24;
production x8 launch 2026-08-26. Model and corpus:
HF [thefinalboss](https://huggingface.co/thefinalboss).*