# Fractus Emergent Properties — In-Training Surgery and Checkpoint Composition

**Updated:** 2026-08-17 00:26 UTC

These properties were **discovered in production** while correcting Fractus mid-run.
They are not marketing claims; they were forced by real bugs, resumes, merges, and continued training.

---

## 1. In-training surgery (open-heart)

**Claim:** Fractus can be modified **during training** without throwing away learned weights.

### What we did live
- Unfroze Kuramoto (removed no_grad) while runs continued from the same token offsets
- Wired load-balance into the loss mid-run
- Raised MoE gate temperature
- Switched from last-token CE to **dense full-chunk CE** (stage2)
- Added scheduled sampling
- Recalibrated LR and logging
- Fixed train/gen path mismatch (RK4 alignment, tick_chunk decode)

### What was preserved
- All previously digested tokens (exact resume manifests)
- Model state_dict tensors (same shapes)
- Multi-day progress across pod restarts

### Operational rule
1. Save / keep current .pt
2. Patch body (code / objective / routing)
3. Reload weights
4. Resume at recorded token offset
5. Document the surgery

**This is mid-training operability**, not a full retrain.

See also: docs/OPERABILITY_MIDTRAIN.md

---

## 2. Parallel brains then merge

**Claim:** You can train **many different .pt files** (shards / GPUs / runs) and **merge** them into one Fractus.

### Proven path (this run)
- 4 independent GPU runs on 4 data shards
- Each writes checkpoints/fractus_1b_gpu0.pt ... gpu3.pt
- **Mean-merge** of floating tensors -> unified checkpoint
  - Example artifact: checkpoints/FRACTUS_1B_STAGE2_MERGED.pt
- Unified .pt still generates (different attractors than a single shard)

### Workflow


Constraint: **same architecture shapes** (d_model, layers, experts, etc.).

---

## 3. Compose an already-trained Fractus with more checkpoints

**Claim:** Take a Fractus already trained, merge it with other compatible .pt files, then keep training — the system **accumulates** rather than restarting from zero.

### Composition loop


### What grow means here
Two related notions:

1. **Compositional growth (proven in this run)**
   - Knowledge / weights from multiple runs can be averaged into one brain
   - Then training continues on the merged brain
   - Digestion is not discarded

2. **Architectural growth (design of Fractus / paliers)**
   - Progressive growth: width, depth, experts can increase across paliers
   - maybe_grow adds capacity when routing demands it
   - Documented in the Chinchilla/growth notes

Merging many same-shape .pt files is **composition**.
Growing parameter count is **structural growth**.
Both are part of the Fractus operating model; this production run proved composition + mid-train surgery under fire.

---

## 4. Why this matters

Standard large training runs treat the job as fragile:
- change the objective -> often restart
- multi-GPU shard brains -> hard to recombine
- decode bugs -> blamed on needs more train only

Fractus production forced a different picture:

| Property | Observation |
|----------|-------------|
| Operable mid-train | Yes — weights kept, body patched |
| Multi-pt parallel train | Yes — 4 shards |
| Merge to one Fractus | Yes — mean-merge works |
| Resume after surgery | Yes — exact token offsets |
| Train then merge then train | Yes — intended workflow |

---

## 5. Limits (honest)

- Merge requires **matching tensor shapes**
- Naive mean-merge of very divergent specialists can **soup** skills
- Composition is not magic AGI; it is a **workflow property** of separable brain (.pt) + body (code)
- Generation coherence still lags teacher-forced loss (see docs/LOSS_VS_GEN.md)

---

## 6. Related documents

- docs/OPERABILITY_MIDTRAIN.md — open-heart principle and stage2 surgery
- docs/DISCOVERY_LOG.md — bugs and emergent features list
- docs/MASTER_RUN_LOG.md — full chronology
- docs/2026-08-12-fractus-chinchilla.md — progressive growth / paliers
- docs/LOSS_VS_GEN.md — why TF loss is not gen quality

---

*Discovered by operating Fractus under real training pressure, not by design slide.*