Upload docs/COMPOSABILITY_AND_SURGERY.md with huggingface_hub

docs/COMPOSABILITY_AND_SURGERY.md

@@ -0,0 +1,123 @@
+# Fractus Emergent Properties — In-Training Surgery and Checkpoint Composition
+
+**Updated:** 2026-08-17 00:26 UTC
+
+These properties were **discovered in production** while correcting Fractus mid-run.
+They are not marketing claims; they were forced by real bugs, resumes, merges, and continued training.
+
+---
+
+## 1. In-training surgery (open-heart)
+
+**Claim:** Fractus can be modified **during training** without throwing away learned weights.
+
+### What we did live
+- Unfroze Kuramoto (removed no_grad) while runs continued from the same token offsets
+- Wired load-balance into the loss mid-run
+- Raised MoE gate temperature
+- Switched from last-token CE to **dense full-chunk CE** (stage2)
+- Added scheduled sampling
+- Recalibrated LR and logging
+- Fixed train/gen path mismatch (RK4 alignment, tick_chunk decode)
+
+### What was preserved
+- All previously digested tokens (exact resume manifests)
+- Model state_dict tensors (same shapes)
+- Multi-day progress across pod restarts
+
+### Operational rule
+1. Save / keep current .pt
+2. Patch body (code / objective / routing)
+3. Reload weights
+4. Resume at recorded token offset
+5. Document the surgery
+
+**This is mid-training operability**, not a full retrain.
+
+See also: docs/OPERABILITY_MIDTRAIN.md
+
+---
+
+## 2. Parallel brains then merge
+
+**Claim:** You can train **many different .pt files** (shards / GPUs / runs) and **merge** them into one Fractus.
+
+### Proven path (this run)
+- 4 independent GPU runs on 4 data shards
+- Each writes checkpoints/fractus_1b_gpu0.pt ... gpu3.pt
+- **Mean-merge** of floating tensors -> unified checkpoint
+  - Example artifact: checkpoints/FRACTUS_1B_STAGE2_MERGED.pt
+- Unified .pt still generates (different attractors than a single shard)
+
+### Workflow
+
+
+Constraint: **same architecture shapes** (d_model, layers, experts, etc.).
+
+---
+
+## 3. Compose an already-trained Fractus with more checkpoints
+
+**Claim:** Take a Fractus already trained, merge it with other compatible .pt files, then keep training — the system **accumulates** rather than restarting from zero.
+
+### Composition loop
+
+
+### What grow means here
+Two related notions:
+
+1. **Compositional growth (proven in this run)**
+   - Knowledge / weights from multiple runs can be averaged into one brain
+   - Then training continues on the merged brain
+   - Digestion is not discarded
+
+2. **Architectural growth (design of Fractus / paliers)**
+   - Progressive growth: width, depth, experts can increase across paliers
+   - maybe_grow adds capacity when routing demands it
+   - Documented in the Chinchilla/growth notes
+
+Merging many same-shape .pt files is **composition**.
+Growing parameter count is **structural growth**.
+Both are part of the Fractus operating model; this production run proved composition + mid-train surgery under fire.
+
+---
+
+## 4. Why this matters
+
+Standard large training runs treat the job as fragile:
+- change the objective -> often restart
+- multi-GPU shard brains -> hard to recombine
+- decode bugs -> blamed on needs more train only
+
+Fractus production forced a different picture:
+
+| Property | Observation |
+|----------|-------------|
+| Operable mid-train | Yes — weights kept, body patched |
+| Multi-pt parallel train | Yes — 4 shards |
+| Merge to one Fractus | Yes — mean-merge works |
+| Resume after surgery | Yes — exact token offsets |
+| Train then merge then train | Yes — intended workflow |
+
+---
+
+## 5. Limits (honest)
+
+- Merge requires **matching tensor shapes**
+- Naive mean-merge of very divergent specialists can **soup** skills
+- Composition is not magic AGI; it is a **workflow property** of separable brain (.pt) + body (code)
+- Generation coherence still lags teacher-forced loss (see docs/LOSS_VS_GEN.md)
+
+---
+
+## 6. Related documents
+
+- docs/OPERABILITY_MIDTRAIN.md — open-heart principle and stage2 surgery
+- docs/DISCOVERY_LOG.md — bugs and emergent features list
+- docs/MASTER_RUN_LOG.md — full chronology
+- docs/2026-08-12-fractus-chinchilla.md — progressive growth / paliers
+- docs/LOSS_VS_GEN.md — why TF loss is not gen quality
+
+---
+
+*Discovered by operating Fractus under real training pressure, not by design slide.*