docs: QuickPod 8x5090 exact-token resume 2026-08-28

README.md

@@ -1,127 +1,471 @@
 ---
 license: mit
 language:
-- en
-- fr
+  - en
+  - fr
 tags:
-- continuous-thought
-- linear-attention
-- kuramoto
-- moe
-- rnn
+  - continuous-thought-engine
+  - cognitive-agent
+  - mixture-of-experts
+  - kuramoto
+  - self-modifying
+  - progressive-growth
+  - decentralized-ai
+  - personal-ai
+  - neuroscience
 library_name: pytorch
+pipeline_tag: text-generation
+models:
+  - thefinalboss/fractus-cte
+datasets:
+  - thefinalboss/fractus-datasets
 ---
 
-# Fractus-CTE
+# Fractus CTE
 
-**Continuous Thought Engine.** A dynamical system with a fixed-size recurrent state, not a transformer.
+**A living AI that thinks continuously, remembers forever, and grows on its own.**
 
-Fractus belongs to the linear-attention RNN family (Katharopoulos 2020, RetNet, RWKV, GLA, DeltaNet, Mamba/S6) **plus** phase-routed MoE and a persistent thought state across chunks. Depth is time. The checkpoint is a living state (weights + phases + attention memory), not a frozen function.
+**Fractus is NOT a transformer.** It's a Continuous Cognitive Agent — a dynamical system that maintains a persistent thought state, advances it tick by tick through 16 blocks, and routes via Kuramoto oscillator phases. The checkpoint is a living seed: it never freezes, grows at runtime, and trains forever.
 
-| | Transformer | Fractus |
-|---|---|---|
-| Computation | one forward per prompt | continuous ticks / chunks |
-| Memory | KV cache grows with length | fixed-size \(S, z\), thought state |
-| Routing | (optional) learned softmax MoE | Kuramoto phases on a circle |
-| Knowledge after train | fine-tune / RAG | **Vorax** organs, append-only `.kn` |
-| Operability | replace the blob | open-heart: code changes, `.pt` shapes stay |
+> **Status (2026-08-28):** Training live again on **QuickPod 8× RTX 5090** after a RunPod death. Exact-token resume from `checkpoints/x8run` (GPU0 at 79.9M / 430M of this Phase-2 pass). Trainer: `scripts/fast4gpu_boost_v4.py`, SS_RATE=1.0, BATCH=4. Speech gate is **unique@40 greedy PREFIX** — last offline probe was NO-GO (3/12/1/6). Not a finished assistant. See [Live Training](#live-training-status) and [`docs/2026-08-28-QUICKPOD-RESUME.md`](docs/2026-08-28-QUICKPOD-RESUME.md).
+
+## Quick Start
+
+```bash
+git clone https://github.com/AFKmoney/fractus-cte.git
+cd fractus-cte && pip install torch numpy tokenizers
+
+# Load the trained 1B and generate
+python -c "
+from fractus.continuous_engine import ContinuousThoughtEngine
+engine = ContinuousThoughtEngine.from_pretrained('checkpoints/fractus_1b_gpu3.pt')
+import torch
+logits, confidence = engine.tick(torch.tensor([42]))
+print(f'Fractus is thinking. Confidence: {confidence.item():.2f}')
+"
+```
+
+The `.pt` checkpoint contains the full model (weights + dynamic state). You need this repo's code to run it — Fractus is a custom architecture, not a transformer. Checkpoints are on [HF](https://huggingface.co/thefinalboss/fractus-cte).
 
-**Author:** Philippe-Antoine Robert ([thefinalboss](https://huggingface.co/thefinalboss))  
-**License:** MIT  
-**Sibling repos:** [fractus-p0](https://huggingface.co/thefinalboss/fractus-p0) (routing + v4 + DiffusionBlocks body) · [fractus-vorax](https://huggingface.co/thefinalboss/fractus-vorax) (sealed brain + ingest) · [fractus-datasets](https://huggingface.co/datasets/thefinalboss/fractus-datasets)
+![License](https://img.shields.io/badge/license-MIT-blue)
+![Python](https://img.shields.io/badge/python-3.13-green)
+![PyTorch](https://img.shields.io/badge/PyTorch-2.11-orange)
+![Params](https://img.shields.io/badge/params-1.05B-red)
+![Active](https://img.shields.io/badge/active%20params-119M-yellow)
+![Status](https://img.shields.io/badge/status-training%20live-brightgreen)
+![Datasets](https://img.shields.io/badge/datasets-4.2B%20tokens-purple)
 
-Full chronology: [`docs/EVOLUTION.md`](docs/EVOLUTION.md) · français [`docs/EVOLUTION.fr.md`](docs/EVOLUTION.fr.md)
+---
+
+## What is Fractus?
+
+Fractus is not a chatbot. It's not GPT. It's not a transformer.
+
+Fractus is a **Continuous Cognitive Agent** — an AI that works like a brain, not a calculator. Instead of processing input → output in one pass, Fractus **ticks** like a biological system: it maintains a persistent thought state, advances it through multiple blocks of processing, remembers everything across sessions, and can grow new capacity by itself.
+
+### What makes it different from GPT/Claude?
+
+| | GPT-4 / Claude | Fractus |
+|---|---|---|
+| **Thinking** | One pass, done | Continuous ticks (like a heartbeat) |
+| **Memory** | Forgets when context window fills | Remembers forever (survives restarts) |
+| **Learning** | Retrain from scratch ($$$) | Learns from every interaction |
+| **Growth** | Fixed size forever | Grows new experts at runtime |
+| **Mental states** | One mode always | Shifts between cognitive modes |
+| **Where it runs** | Corporate cloud | Your machine |
+| **Training** | Fixed, done once | Perpetual, never stops |
 
 ---
 
-## Architecture (1B production config)
+## Architecture (1.05B total / ~119M active per token)
 
-| | |
+```
+ContinuousThoughtEngine
+├── d_model=1280, 16 layers, 16 heads
+├── FractalLinearAttention   multi-level causal linear attention (O(L)),
+│                            carry state (S, z) persists across chunks
+├── Kuramoto phase clock     RK4-integrated oscillators → phase vectors
+│                            (learned end-to-end; feeds routing)
+├── PhaseRoutedMoE           128 experts, top-2 active per token,
+│                            von Mises gate over phases, Farey-sequence
+│                            expert phases, load-balance loss in objective
+├── Low-rank experts         W = scale · U@Vᵀ (rank 64) → 64× less compute
+├── Tied embedding/head      GPT-2 BPE vocab (50257)
+└── Persistent thought state residual stream carried across ticks
+```
+
+**Parameter accounting:** ~1.05B total, but only ~118.8M are active per token (dense attention + top-2 of 128 sparse experts) — ~183.1M including the tied embedding. An 11.3% sparsity ratio. This is the basis of the adapted scaling target below.
+
+### The 12 Building Blocks
+
+| Block | What it does |
 |---|---|
-| Params | ~1.165B |
-| `d_model` | 1280 |
-| `n_layers` | 16 CTEBlocks |
-| Attention | causal **linear** (cumsum / chunked). Env `FRACTUS_ATTN_IMPL` |
-| Oscillators / block | 16, coupling rank 8 |
-| Experts / block | 128, top-2, phase-gated |
-| Vocab | 50257 GPT-2 BPE |
-| Train objective | dense next-token CE on the chunk + Switch load-balance (+ optional SS / anti-repeat in v4) |
+| **Continuous Thought Engine** | The brain — thinks tick by tick through 16 blocks |
+| **Persistent Memory** | Vector bank surviving sessions, cosine recall, 5% blend injection, salience head gates impact |
+| **Cognitive Modes** | Mental states discovered unsupervised (k-means on Kuramoto phase features): focused, creative, exploratory |
+| **RAG Knowledge Base** | Learns facts instantly — no retraining needed |
+| **Cognitive Plugins** | Hot-swappable modes: analyst, coder, creative, teacher |
+| **MetaCognition** | Decides its own actions: retrieve, learn, generate |
+| **Progressive Growth** | Grows from 6M to 1B+ params, palier by palier (`maybe_grow`: width + depth + experts) |
+| **Self-Modification** | Adds new experts at runtime when routing is imbalanced (zero-init, placed near the dominant expert) |
+| **PhaseRoutedMoE** | Sparse experts routed by oscillator phases |
+| **Kuramoto Clock** | A dynamical system that drives routing decisions |
+| **Online Trainer** | Learns continuously, one chunk at a time |
+| **Vision (`tick_vec`)** | Multimodal input path — image patches drive the engine directly, bypassing the token embedding (CIFAR "eyes" prototype trained while the text run continued) |
+
+---
+
+## Live Training Status (28 August 2026)
+
+**Hardware:** QuickPod 8× RTX 5090. One independent Python process per GPU. No DDP / no gradient sync. Consolidation = mean-merge of the 8 `.pt` when we need a single brain.
 
-Each block: attention → Kuramoto → PhaseRoutedMoE. \(S,z\) and phases are **per-block** and carried across chunk boundaries.
+**This pass (Phase 2 shards):**
+- 8 GPT-2 BPE int32 memmaps, `~429,896,462` tokens/GPU — dataset: [thefinalboss/fractus-datasets](https://huggingface.co/datasets/thefinalboss/fractus-datasets) `tokenized/phase2/shard_phase2_gpu{i}.npy`
+- Resume point **2026-08-28 14:40 UTC** after RunPod SSH death: GPU0 **79,922,176** tokens (~18.6% of this pass). Other GPUs 76.2M–78.4M. Exact JSON: `checkpoints/x8run/RESUME_gpu{i}.json`
+- Trainer: `scripts/fast4gpu_boost_v4.py`
+- Config: **BATCH=4**, SEQ=128, LR 7e-4, SGD+momentum 0.9, bf16, TF32, compile off, `FRACTUS_ATTN_IMPL=chunked`, `BLOCK_CKPT=1`, **SS_RATE=1.0**, SS_PROB 0.2→0.5 over 50M tokens, REPEAT_COEF=0.1, P0 on, **PROBE_EVERY=0** (live unique@40 probe crashed the first QuickPod launch — probes are offline)
+- Signals: `tf`/`ema_tf`, `ss`/`ema_ss`, `rep`/`ema_rep`, `lb` (~14)
+- Throughput after resume: ~840–940 tok/s/GPU. Remainder of this pass ≈ **4.1–4.5 days** if 8 GPUs stay up
+- **Torch on RTX 5090:** image default `2.2.1+cu121` has **no Blackwell kernels**. Required: **torch ≥ 2.11 + cu128**
+
+**Crash recovery:** hourly upload of `checkpoints/x8run/fractus_1b_gpu{i}.pt` + `RESUME_gpu{i}.json`. That folder is the source of truth. See [`docs/2026-08-28-QUICKPOD-RESUME.md`](docs/2026-08-28-QUICKPOD-RESUME.md).
+
+**Speech gate (not the loss):** `unique@40` greedy PREFIX — 40 generated tokens, count distinct ids, no ban, no temperature. Last offline numbers: Hello=3, Once upon a time=12, meaning of life=1, Fractus is=6. **NO-GO.** Teacher-force CE going down does **not** mean Fractus speaks. Do not cite banned-decode diversity as speech.
+
+## Vision — the CIFAR "eyes" prototype
+
+Fractus is not text-only. The engine exposes **`tick_vec(obs_vec)`**: a multimodal entry point that accepts a precomputed `(B, d_model)` vector and injects it directly into the residual thought state — **bypassing the token embedding entirely**. Any modality that can be embedded into `d_model` dimensions can drive continuous thought: images, audio, sensor streams.
+
+```
+image → PatchEmbed → (B, d_model) patch vectors
+                            │
+                            ▼
+              engine.tick_vec(patch)     ← no tokenizer involved
+                            │
+              thought state advances through all 16 blocks
+              (same Kuramoto routing + MoE as text)
+```
+
+**Proof of concept — CIFAR-10 eyes:** a small CTE + PatchEmbed stack (`fractus/nn/vision.py`) was trained on CPU on real CIFAR-10 images (`fractus_eyes_cifar_final.pt`), demonstrating that image patches can drive the continuous-thought loop. Crucially, this ran as a **parallel track**: the 8-GPU text run was never interrupted for eyes work — GPU text digestion and CPU vision learning proceed simultaneously on the same living system.
+
+This mirrors the biological premise: eyes evolved as a peripheral sense feeding a central dynamical brain, not as a core feature of it. The text 1B is the brain; vision is a front-end that plugs into `tick_vec`.
+
+---
+
+### Adapted Chinchilla target
+
+Standard Chinchilla (20 tokens/param) assumes dense transformers where all parameters train on every token. Fractus violates that: only ~119M of 1.05B params are active per token, and the Kuramoto clock trains end-to-end now. The adapted target is **20 × active params ≈ 2.4–3.7B tokens**; with a warm-started checkpoint, the practical target is **~2.5–3B tokens** — which is why the phase-2 corpus is 3.44B. Since Fractus grows continuously (`maybe_grow`), this is an instantaneous cumulative-exposure requirement that grows with the model, not a final-stop condition. Details: [`docs/2026-08-12-fractus-chinchilla.md`](docs/2026-08-12-fractus-chinchilla.md).
 
 ---
 
-## What is true right now (2026-08-27)
+## The Surgeries (mid-training interventions, no weight wipes)
 
-**Done**
-- Multi-GPU data-parallel shards, exact `START_TOKEN` resume, hourly mean-merge of same-shape `.pt`
-- Open-heart speed work (22–26 Aug): chunked linear attention, `chunked_cross_entropy`, `BLOCK_CKPT`, atomic ckpts — **shapes unchanged**
-- Phase-2 corpus on HF; freeze then x8 resume (~35M tokens/GPU on ~430M-token shards at last pushed manifest)
-- P0 routing body (atan2 encode, phase carry, per-token MoE, Switch LB) in [fractus-p0](https://huggingface.co/thefinalboss/fractus-p0)
-- PREFIX vs CARRY decode hole **named and measured** on a CPU mini
-- Fractus-native DiffusionBlocks prototype (one CTEBlock / step, 0 new params) — experiment, not default
+A defining discovery of this run: **the brain (.pt) and the code are separable.** Bottlenecks were fixed by live surgery — save the checkpoint, patch the code, reload weights, resume at the exact recorded token offset. Multi-day digestion is never thrown away.
+
+| Phase | Intervention | Result |
+|---|---|---|
+| **A — Initial** | Last-position-only CE, 4 independent GPUs | Loss fell; generation collapsed into single-token loops |
+| **B — Routing surgery** | Kuramoto was frozen under `no_grad` (order parameter r ≈ 0.01–0.03); LB loss was detached → ~70% of experts dead. Fixed: gradients enabled (state kept detached for carry), CE + 0.02·lb, gate temp 1.0 → 2.5, omega scale ×4 | lb ≈ 14 live on all GPUs; experts alive |
+| **C — Dense CE** | Replaced last-position CE (1 target / 128 tokens) with CE over all positions | Sharp loss drop; token-to-token chaining enforced |
+| **D — Decode surgery** | Phase/thought noise, frequency penalties, cycle bans, forced escape tokens | Loop lock broken; still no coherent English |
+| **E — Train/gen mismatch** | Training used causal attention + RK4 Kuramoto; generation used a simpler single-tick Euler path. Aligned decode via `generate_aligned.py` | Decode now matches the training path |
+| **F — Loss recalibration** | Cumulative-average CE was misleading near 2.0 → batch CE + EMA; LR 1e-3 → 5e-4 | Trustworthy metrics |
+| **G — Scheduled sampling** | Two-pass training: TF CE + LB, plus SS steps mixing model samples into inputs | Live; now SS_RATE=1.0 on boost_v4 |
+| **H — Pod death / exact resume** | RunPod SSH died 2026-08-28. Reloaded 8× `.pt` + manifests on a new QuickPod 8×5090 at the recorded `start_token_next`. Torch upgraded 2.2→2.11+cu128 for Blackwell | Weights kept. Pass continues from ~80M/430M |
 
-**Open (do not skip)**
-- Attention state \(S\) still looks like \(S_t = S_{t-1} + k\otimes v\) — **no learned decay** yet (RWKV/RetNet/Mamba all added one)
-- Mean-merge of independent MoE shards ≠ DDP; expert #k is not aligned across GPUs
-- Kuramoto decision is still a **circle** (need \(\mathrm{MI}(\bar\theta, \text{token})\) vs tick)
-- No published **matched-compute PPL** vs a vanilla transformer on a clean held-out
-- CARRY length-1 generation is not the trained graph (PREFIX is the language gate)
-- Eval must be split by **source**, not by line (identity text in the corpus)
+Full logs: [`docs/MASTER_RUN_LOG.md`](docs/MASTER_RUN_LOG.md), [`docs/DISCOVERY_LOG.md`](docs/DISCOVERY_LOG.md), [`docs/OPERABILITY_MIDTRAIN.md`](docs/OPERABILITY_MIDTRAIN.md).
 
-Default next run: **v4 + P0 body + resume offsets + 24GB+ identical GPUs (x4 is enough)**.  
-DiffusionBlocks and DDP are options. Finishing the pass without a baseline PPL is not a paper.
+### Composable checkpoints
+
+Because shapes stay compatible and manifests record token offsets, the following operations are proven on this run:
+- **Parallel independent training** — N GPUs on separate shards
+- **Mean-merge** — per-GPU checkpoints averaged into one unified model that still generates (424/440 tensors on the 4-GPU merge; stateful buffers reset)
+- **Iterative fusion** — train → merge → train cycles; the model absorbs compatible checkpoints and keeps going
+- **Exact-token resume** — across pod reboots and driver crashes
+- **Compositional growth ≠ structural growth** — weight averaging vs `maybe_grow` paliers
+
+Details: [`docs/COMPOSABILITY_AND_SURGERY.md`](docs/COMPOSABILITY_AND_SURGERY.md).
 
 ---
 
-## Load
+## Trusted Loss (reading the numbers)
+
+Three metrics, three different questions:
+
+| Metric | What it measures | Where |
+|---|---|---|
+| `ema_tf` | CE with ground-truth history + continuous internal state — is the model digesting data? | live |
+| `ema_ss` | CE after scheduled sampling mixes model-generated tokens in — partial free-run robustness | live |
+| **AR** | Warm on 32 true tokens, greedily free-run 32 steps, CE vs truth — actual generation quality | offline |
+
+**A single CE number cannot represent both teacher-forced learning and free-run generation.** Live ema_tf after the QuickPod resume is mid-teens on the lead GPUs (first steps inflate EMA — wait). AR stays the offline generation metric and last unique@40 was NO-GO. Random-vocab CE ≈ 10.82; AR in the thousands (random guessing over the GPT-2 vocab is ~10.82): free-running compounds every error, and teacher forcing always supplies the correct past. Trust `ema_tf`/`ema_ss` for learning progress, AR + text probes for generation progress. The convergence signal is AR falling toward the ss/tf order of magnitude, plus readable output. Details: [`docs/TRUSTED_LOSS.md`](docs/TRUSTED_LOSS.md).
+
+---
+
+## Research Results (Honest)
+
+**Refuted:**
+- **EDT** (Expert Decoupled Training) — all 5 variants ~19–20% worse than from-scratch; MSE objective misaligned with CE, router ignores half the experts.
+- **Forward-Forward** (Hinton 2022) — NLL rose from 124 to 221; local objectives can't replace global backprop here.
+
+**Validated:**
+- **Progressive growth** — warm start converges faster; trained through palier 3 (350M, loss 23.0, ~5 days CPU).
+- **Sparse low-rank MoE** — 2/128 experts = 64× less compute.
+- **Open-heart operability** — live surgery on a training model works (see above).
+- **Routing pathology as a first-class debug target** — expert-hit histograms and the Kuramoto order parameter catch failures that loss curves hide (they caught the frozen clock and the dead experts).
+
+**Training optimizations (measured):** tied head (~1.1×), head-partial training (~2×), sparse gathered low-rank MoE (8× at 16 experts, 64× at 128), detached-state Kuramoto, gradient accumulation (~1.4×), SGD+momentum over AdamW (~1.37×), batching (335 → 1345 tok/s at B=8), bf16 (~2×). Combined CPU: ~336× over naive. Measured on CPU: 707 tok/s single-stream, **1345 tok/s batched**.
+
+---
+
+## Datasets (4.2 Billion Tokens)
+
+Source of truth: [huggingface.co/datasets/thefinalboss/fractus-datasets](https://huggingface.co/datasets/thefinalboss/fractus-datasets).
+
+| Dataset | Tokens | Content |
+|---|---|---|
+| **neuro-paradigms-1b** | **~1B** | 100 neuroscience → software architecture paradigms (300 chunked files) |
+| **neuro-code-math** | **~900M** | Neuro-inspired coding, mathematics, algorithms (incl. 40 applied-neuroscience topics) |
+| **cognitive-skills** | **~780M** | Coding, reasoning, speaking, thinking, understanding |
+| **fractus-generated-corpus** | **340M** | Bilingual FR/EN generated by Fractus ontology engine |
+| **paradigms-full** | **191M** | 140 paradigms (neuroscience, CS, architecture) |
+| **gutenberg-esoteric** | **~58M** | 487 public-domain esoteric / masonic / hermetic books |
+| **neuro-arch-full** | **86M** | 60 neuroscience paradigms (neuro-software-architecture) |
+| **all-github-repos** | **54M+** | 80+ repos (public + private, secret-filtered) |
+| **mega-corpus-v3** | **20M** | Literature, philosophy, occult, masonry, science, medicine |
+| **wordnet** | **3M** | 117K dictionary synset entries |
+| **Total** | **~4.2B** | |
+
+Tokenized streams: Phase 1 = ~1.52B tokens (`.pt` files); Phase 2 = ~3.44B tokens (8 GPT-2 BPE int32 memmap shards). Phases are kept separate to avoid re-ingesting the same ordered stream.
+
+---
+
+## Applied Neuroscience — the theoretical core
+
+Fractus is a neuroscience-grounded architecture: real brain mechanisms are mapped to software/AI patterns, and that mapping is itself training data. Every entry below is present in the dataset — verified by file listing, not just claimed.
+
+### 100 neuroscience → software-architecture paradigms (`neuro_paradigms_1b`, 300 chunked files)
+
+Each paradigm maps a biological mechanism to an engineering pattern (e.g. *adenosine sleep pressure* → cache-stampede recovery; *myelin sheath* → caching; *hippocampal replay* → trajectory consolidation).
+
+<details><summary><b>show all 100 paradigms</b></summary>
 
-```python
-from fractus.continuous_engine import ContinuousThoughtEngine
-eng = ContinuousThoughtEngine.from_pretrained("checkpoints/FRACTUS_1B_PHASE2_FROZEN_MERGED.pt")
-# .pt = weights + live state. fractus/ = the body that ticks.
 ```
+adenosine_sleep_pressure            amygdala_prefrontal_topdown       anterior_cingulate_conflict_monitor
+apoptosis_self_destructing_service  arc_gene_plasticity_marker        astrocyte_tripartite_synapse
+axon_initial_segment_trigger        basal_ganglia_loop_arbitration    bdnf_growth_factor_scaling
+bergmann_glia_purkinje              binaural_cross_correlation_localization   brainstem_vital_functions
+broca_area_api_generator            calcium_transmitter_coupling      camp_second_messenger_amplifier
+cerebellar_forward_model            cholinergic_attentional_filter    circadian_gene_expression
+climbing_fiber_error_broadcast      cochlear_compressive_nonlinearity cortical_area_specialization
+cortical_minicolumn_pipeline        cortico_cortical_pathways         corticotropin_releasing_hormone
+cortisol_slow_stress_recovery       critical_period_learning_rate     dendritic_compartmentalization
+endocannabinoid_retrograde          enteric_glia_gut_brain            ependymal_cell_barrier
+fusiform_face_service_registry      gaba_inhibitory_bus               gap_junction_electrical_sync
+ghrelin_hunger_signal               glomerular_convergence_gateway    glutamate_excitatory_bus
+glycine_coagonist_modulator         granule_cell_inhibitory_relay     hair_cell_banks_event_clusters
+hippocampal_4ec_loop_replay         histamine_wakefulness_keeper      hox_gene_service_specialization
+hypercolumn_module_federation       hypercolumn_sharding              hypothalamus_homeostasis
+insula_interoception_monitor        ip3_inositol_cascade              k_complex_event_trigger
+kcc2_chloride_shift_inhibitor       leptin_satiety_signal             locus_coeruleus_ne_global_signal
+melatonin_circadian_scheduler       microglia_active_surveillance     mitral_tufted_cell_dual_path
+morphogen_gradient_config           muller_glia_retina_repair         myelin_sheath_caching
+neural_crest_migration_deploy       neuropeptide_y_stress_buffer      ng2_glia_pool_renewal
+nitric_oxide_gas_signal             node_of_ranvier_bypass            nrem_slow_wave_cleanup
+nucleus_accumbens_reward_routing    oligodendrocyte_myelination_dynamic      orexin_stability_keeper
+orientation_column_indexing         oscillatory_phase_locking_io      oxytocin_trust_protocol
+parahippocampal_place_topology      parallel_fiber_fanout_aggregation pineal_circadian_release
+pinwheel_central_layout             pituitary_master_gland            posterior_parietal_integration
+prolactin_parental_care             quantal_release_batching          radial_glia_neural_stem
+radial_glial_scaffold               raphe_serotonin_rate_limit        rem_paradoxical_processing
+replay_consolidation_trajectory     reticular_activating_system       retinotopic_data_layout
+satellite_glial_ganglion            schwann_cell_peripheral_repair    sleep_pressure_forced_maintenance
+sleep_spindle_memory_transfer       slow_oscillation_sync             subplate_wait_state
+suprachiasmatic_clock               synaptic_vesicle_pool            synaptogenesis_service_wiring
+tanycyte_metabolic_sensor           temporal_pole_semantic_cache      thalamocortical_loop_api
+tonotopic_stream_partitioning       vasopressin_loyalty_aware_routing vta_dopamine_rpe_scheduler
+wernicke_area_api_parser
+```
+</details>
+
+### 40 applied-neuroscience topics (`neuro_code_math/applied_neuroscience/`)
 
-Same-shape checkpoints can be mean-merged. Different `d_model` / layer / expert counts **cannot**. See `docs/CPU_MINI_MERGE_AND_DIMENSIONS.md`.
+Deep dives on computational neuroscience theories — the science Fractus's design draws from.
+
+<details><summary><b>show all 40 topics</b></summary>
+
+```
+active_inference          axonal_computation         basal_ganglia_circuits     bayesian_brain
+cerebellar_computation    consolidation              cortical_minicolumns       cross_frequency_coupling
+dendritic_computation     dopamine_reward            entorhinal_grid_cells      free_energy_principle
+gamma_oscillations        global_workspace_theory    head_direction_cells       hierarchical_processing
+higher_order_theories     hippocampal_formation      homeostatic_plasticity     integrated_information_theory
+long_term_depression      long_term_potentiation    metaplasticity             neural_coding
+neural_decoding           neural_manifolds          neuromodulation            place_cells
+population_coding         predictive_coding          predictive_processing      rate_coding
+serotonin_modulation      sharp_wave_ripples         sleep_replay               sparse_coding
+spike_timing_dependent_plasticity   temporal_coding  thalamic_reticular_nucleus theta_oscillations
+```
+</details>
+
+### Foundational researchers & concepts honored in the corpus
+
+**Hebb** (Hebbian learning), **Bi & Poo** (STDP timing curves), **Friston** (free energy / active inference), **Buzsáki** (hippocampal sharp-wave ripples, replay), **Moser & Moser** (grid cells), **Hodgkin & Huxley** (axon dynamics), **Izhikevich** (spike models), **Tononi** (integrated information), **Baars/Dehaene** (global workspace), **O'Keefe** (place cells), **Kandel** (memory consolidation), plus neuromodulators (dopamine RPE, serotonin, oxytocin, vasopressin) and glial biology (astrocytes, microglia, oligodendrocytes, Schwann cells).
 
 ---
 
-## Train (resume, never from token 0 unless you mean it)
+## How to Use
+
+### Install
 
 ```bash
-# offsets: checkpoints/X8_MANIFEST.json or FROZEN_RESUME_MANIFEST.json
-CUDA_VISIBLE_DEVICES=0 GPU_ID=0 START_TOKEN=<manifest> \
-  BATCH=2 CE_CHUNK=2048 FRACTUS_ATTN_IMPL=chunked BLOCK_CKPT=1 \
-  python -u scripts/fast4gpu_boost_v3.py
+git clone https://github.com/AFKmoney/fractus-cte.git
+cd fractus-cte
+pip install torch numpy tokenizers matplotlib fastapi uvicorn pydantic
 ```
 
-P0 + v4 launcher lives in [fractus-p0](https://huggingface.co/thefinalboss/fractus-p0) (`scripts/fast4gpu_boost_v4.py`).  
-PREFIX `unique@40` is the language gate. Do not raise `REPEAT_COEF` above 0.1.
+### Run tests
+
+```bash
+pytest tests/ -q
+```
+
+### Train on CPU (progressive growth)
+
+```bash
+python scripts/train_progressive.py --paliers 0,1,2,3 --accumulation-steps 8
+```
+
+### Train the 1B (GPU, sharded)
+
+```bash
+# Phase-2 1B on 8 GPUs — resume from HF x8run (do not restart at token 0)
+# 1) torch >= 2.11+cu128 on RTX 5090
+# 2) download checkpoints/x8run/*.pt + RESUME_gpu*.json
+# 3) download tokenized/phase2/shard_phase2_gpu{i}.npy, symlink to data/shard_gpu{i}.npy
+CUDA_VISIBLE_DEVICES=$i GPU_ID=$i START_TOKEN=<from RESUME_gpu$i.json> \
+  BATCH=4 SEQ=128 CE_CHUNK=2048 FRACTUS_ATTN_IMPL=chunked BLOCK_CKPT=1 \
+  SS_RATE=1.0 P0=1 COMPILE=0 PROBE_EVERY=0 \
+  CKPT_IN=checkpoints/x8run/fractus_1b_gpu$i.pt \
+  CKPT_OUT=checkpoints/x8run/fractus_1b_gpu$i.pt \
+  SHARD=data/shard_gpu$i.npy \
+  python -u scripts/fast4gpu_boost_v4.py
+```
+Speech metric (offline, not in the trainer loop): unique@40 greedy PREFIX. Gate ≈ mean unique ≥ 20 and not a single-token lock.
+
+### Use the agent
+
+```python
+from fractus.continuous_engine import ContinuousThoughtEngine
+from fractus.memory import PersistentMemory
+from fractus.tokenizer import FractusTokenizer
+
+# Build the brain
+engine = ContinuousThoughtEngine(
+    vocab_size=50257, d_model=128, n_heads=2, d_head=64,
+    n_layers=2, n_levels=2, n_oscillators=8, coupling_rank=4,
+    n_experts=4, top_k=2, expert_d_ff=128, siren_rank=32)
+
+# Give it memory
+memory = PersistentMemory(d_model=128, path="~/.fractus/memory.pt")
+engine.attach_memory(memory)
+
+# Think
+engine.reset_thought(batch_size=1)
+logits, confidence = engine.tick(torch.tensor([42]))
+print(f"Confidence: {confidence.item():.2f}")
+
+# Vision: drive thought with an image patch vector (no tokenizer)
+patch_vec = torch.randn(1, 128)          # any (B, d_model) embedding
+logits, confidence = engine.tick_vec(patch_vec)
+```
 
 ---
 
-## Docs worth reading first
+## The Growth Path
 
-| Doc | Why |
-|---|---|
-| `docs/EVOLUTION.md` | timestamped history |
-| `docs/HOW_FRACTUS_IS_TRAINED.md` | phase-2 recipe |
-| `docs/KURAMOTO_BOTTLENECK_AND_FIX.md` | dead experts / 25° arc |
-| `docs/OPTIMIZATION_2026-08-22.md` | open-heart kernels |
-| `docs/TRAIN_GEN_MISMATCH.md` / p0 `V4_AR.md` | PREFIX vs CARRY |
-| `docs/TRUSTED_LOSS.md` | loss ≠ generation |
-| `Fractus_White_Paper_v2.md` | architecture write-up |
+| Stage | Size | Blocks | Experts | What it can do |
+|---|---|---|---|---|
+| Palier 0 | 6.6M | 1 | 4 | Learn basic patterns |
+| Palier 1 | 25M | 2 | 8 | Simple text generation |
+| Palier 2 | 120M | 4 | 16 | Coherent fragments |
+| Palier 3 | 350M | 8 | 32 | Decent text quality |
+| **Palier 4** | **1B** | **16** | **128** | **Full language model (training now)** |
+
+Each stage inherits the previous one's knowledge via zero-padded warm starts. The model never starts from zero.
 
 ---
 
-## Citation
+## Repository Layout
 
 ```
-@misc{robert2026fractus,
-  title={Fractus: a Continuous Thought Engine},
-  author={Robert, Philippe-Antoine},
-  year={2026},
-  howpublished={https://huggingface.co/thefinalboss/fractus-cte},
-  note={MIT. Dynamical system; linear-attention RNN lineage + phase-routed MoE.}
-}
+fractus-cte/
+├── fractus/
+│   ├── continuous_engine.py      ← The brain (CTE + CTEBlock)
+│   ├── memory.py                 ← Cross-session persistent memory
+│   ├── cognitive_modes.py        ← Unsupervised mental state detection
+│   ├── grow.py                   ← Progressive growth (width + depth + experts)
+│   ├── rag.py                    ← Knowledge base + plugins + metacognition
+│   ├── tokenizer.py              ← GPT-2 BPE tokenizer
+│   ├── nn/
+│   │   ├── moe.py                ← PhaseRoutedMoE (sparse, low-rank)
+│   │   ├── attention.py          ← Multi-level causal linear attention + carry
+│   │   ├── phase_ode.py          ← Kuramoto RK4 oscillators
+│   │   └── lazy_siren.py         ← Low-rank weight storage
+│   └── train/online.py           ← Online trainer
+├── tests/
+├── scripts/                      fast_tokenize, shard_corpus, launch_4gpu,
+│                                 fast4gpu_boost_v4, generate_aligned, hourly HF x8run sync
+├── checkpoints/                  per-GPU + merged + recovery alias
+├── docs/                         run logs, surgeries, trusted loss, scaling
+├── space/                        HF Space demo
+├── Fractus_White_Paper_v2.md     White paper v2.0
+└── arxiv/                        LaTeX source for arXiv submission
 ```
+
+---
+
+## Key Concepts
+
+**Tick**: one step of thinking. The engine processes an observation, updates its thought state through all blocks, and optionally emits output.
+
+**Thought state**: a vector that persists across ticks — the engine's "consciousness."
+
+**Chunk**: 32 tokens processed in one forward pass. The thought state and per-block attention state carry between chunks.
+
+**Expert**: a small low-rank network (`W = scale·U@Vᵀ`) that specializes in certain thoughts. Only 2 of 128 active per token.
+
+**Kuramoto clock**: coupled oscillators producing phase vectors that route tokens to experts — learned end-to-end since the routing surgery.
+
+**`tick_vec`**: the multimodal tick — feed any precomputed `(B, d_model)` vector (image patches, embeddings from any encoder) straight into the thought state without tokens. This is how vision plugs in.
+
+**Surgery**: patching code around a preserved checkpoint mid-training, resuming at the exact token offset. Weights are never wiped for a routing, objective, or decode bug.
+
+---
+
+## Limitations (stated plainly)
+
+- Generation is not yet coherent English — word-level repetition loops / lexical noise. Exposure bias is being addressed by scheduled sampling; AR is the metric to watch.
+- GPT-2 vocab dominates parameters (81% at d=768) — vocab reduction is a known lever.
+- "Remembers forever" and "grows on its own" describe the architecture's design; no independent benchmarks are provided.
+- This is a research artifact and a live training run, not a production assistant.
+
+---
+
+## License
+
+MIT. Fractus belongs to you, not to a corporation.
+
+## Author
+
+**Philippe-Antoine Robert** — 2026 — rpa.tu@proton.me
+
+## Links
+
+- **GitHub:** [github.com/AFKmoney/fractus-cte](https://github.com/AFKmoney/fractus-cte)
+- **HuggingFace Model:** [huggingface.co/thefinalboss/fractus-cte](https://huggingface.co/thefinalboss/fractus-cte)
+- **HuggingFace Datasets:** [huggingface.co/datasets/thefinalboss/fractus-datasets](https://huggingface.co/datasets/thefinalboss/fractus-datasets)
+- **White Paper:** [Fractus_White_Paper_v2.md](Fractus_White_Paper_v2.md) / [PDF](Fractus_White_Paper.pdf)
+- **Run logs:** [MASTER_RUN_LOG](docs/MASTER_RUN_LOG.md) · [TRAINING_LOG_1B](docs/TRAINING_LOG_1B.md) · [DISCOVERY_LOG](docs/DISCOVERY_LOG.md)
+- **arXiv source:** [arxiv/main.tex](arxiv/main.tex)
+