Legacy Brain Crystallization

Training the law before the noise

Modern pretraining is built on a brute assumption: if you replay enough text, the model will eventually figure out the underlying structure. For general language, with trillions of tokens of essentially free supply, that assumption is fine. It is also enormously wasteful, and the waste is invisible until you try to train inside a real budget.

This is the second devlog in our series on building a frontier decision system on consumer hardware. The discovery here is uncomfortable for anyone who has spent the last three years scaling token counts. We will state it plainly:

Most of the tokens in a domain-specific training corpus are not teaching the model anything new.

They are restating, in slightly different surface forms, the same handful of underlying laws. The model has to slog through every restatement to extract the law once. We started asking what would happen if we extracted the law first and trained on that.

From replay to crystallization

A traditional finance-data training row is a long piece of structured text describing a market state, an action, and an outcome. Two rows describing structurally identical situations -- same regime, same setup, same outcome -- produce nearly identical gradient signal. Multiply that by an entire corpus and a large fraction of the wall-clock is being spent re-learning lessons the model already knows.

We started building what we call state-action memory atoms: compact units that distill repeated experience into a single weighted row. Each atom carries a structural fingerprint of a class of market states, a directional label, an outcome, and a bounded weight that lets one row contribute more gradient than one row's worth of tokens -- without duplicating any text.

The early measurement was striking. Our first compressed shard collapsed the equivalent of a much larger replay corpus into 4,602 rows representing roughly 700K source tokens but carrying over 1M weighted token mass -- a 24.3x row-compression factor we have since reproduced across follow-on shards.

We are deliberately not describing how these atoms are generated. The selector, the fingerprint, the weighting bounds, and the action-balance gating are the parts of the recipe we are keeping in-house for now. What is publishable is the effect.

Weighted learning density

The right mental model is not "fewer rows." It is higher learning density per row.

When the trainer sees a compressed row, it is not seeing a single example. It is seeing a law backed by many observations, with a weight that flows through the data pipeline end to end. The gradient on that row reflects the evidence behind it, not the surface area of one piece of text.

This composes cleanly with two other ideas we will publish more about in later entries:

• An outcome-weighted loss that scales gradient by the realized quality of the decision the row represents.
• A curriculum prior that determines which phase of training each row is meaningful in.

The result is a training process where the information content per step is dramatically higher than in conventional replay. The model is not learning faster because the hardware is faster. It is learning faster because each step is doing more work.

Training the law before the noise

One of the deeper consequences took us by surprise. When the early stages of training are dominated by crystallized law, and broader replay is held back for later phases, the model develops a different kind of competence.

It does not memorize the surface form of trades. It internalizes a prior over structural setups before it is ever asked to handle messy real rows. By the time noisy long-form data arrives in later phases, the model has a frame to fit it into rather than a vacuum to drown in.

We have started calling this training the law before the noise, and we now treat it as a first-class curriculum principle. It is not a clever ablation; it is a default phase ordering. The crystals come first. The replay anneals in afterward.

What it does not do

Crystallization is not a free lunch and we are careful not to claim it is one. Specifically:

• It does not eliminate the need for raw replay. Some skills -- pixel-grounded chart competence, broad language fluency, filing-style comprehension -- require the actual rows. The compressed memory makes the early stages cheaper, not the whole pipeline trivial.
• It does not guarantee generalization. An atom can encode a real law or a stale one. Stale atoms are an active failure mode we have to detect and retire -- a discipline we now treat as its own component, including iterative corrector passes that re-mine failure cases back into the next compression round.
• It does not produce results without honest evaluation gates (G1-G6 in our internal taxonomy). A model that scores well on the compressed corpus and then fails on fresh held-out data has learned the index, not the territory. We are very strict about which evals count.

We are publishing the architectural idea, not a finished verdict. There are at least a dozen ablations still queued.

Why this matters

Two reasons.

Economic. A 24x row-compression effect changes what is possible to train inside a consumer budget. It is the difference between a recipe that demands a rented cluster and a recipe that runs locally on a single GPU over a long weekend. Combined with the throughput envelope we describe in the first entry -- a sustained ~13.6K tokens/sec on a single RTX 5070 Ti -- this puts a serious training run inside a single day.

Epistemic. Crystallization forces you to make explicit what your corpus is actually teaching. You cannot compress a lesson you cannot name. The exercise of building the atom set surfaces structural assumptions that were previously hidden inside billions of redundant tokens. That is valuable on its own, regardless of the compute saving.

What comes next

The crystallization layer is the foundation. The next phase, which we will write about in a later entry, is the EVO20 Training Genome: a 20-step curriculum that uses the crystals as its first bootcamp, then layers a language spine, a finance spine, chart anchors, and a verifier-first replay anneal on top of them.

We will also be writing about how crystallization interacts with the execution reality layer -- the part of the system that ensures the model is being scored under realistic friction, not just paper alpha. A model trained on crystallized law still has to survive contact with 10bps of cost. That confrontation is where the discipline lives.

Until then: the rule is simple. Compress the law. Train it first. Make every row count.

Implementation details and certain training procedures are intentionally withheld.

Part 2 of an ongoing devlog series. The work is in progress and the claims are provisional. Implementation details are intentionally withheld.

If this work is useful to you and you want to support it: ko-fi.com/tjarvis91