Anytime-unrolled EqLM (121M): parity ratio 0.991 vs param-matched explicit baseline

README.md

@@ -1,37 +1,62 @@
 ---
-license: mit
+license: apache-2.0
+language: [en]
+tags: [deep-equilibrium, weight-tying, babylm, equilibrium-model, kinetic-ai]
+datasets: [BabyLM-community/BabyLM-2026-Strict]
 ---
 
-# EqLM — kinetic-eqlm-anytime-121m-babylm
+# kinetic-eqlm-anytime-121m-babylm
 
-This is an equilibrium language model from the **Kinetic AI** research project.
+A weight-tied transformer block solved to a fixed point, trained with anytime
+unrolled supervision. Its depth is a stopping criterion rather than an
+architecture: the same 120.7M parameters are applied repeatedly until the
+representation stops moving, instead of being spread across distinct layers.
 
-## Architecture
+## What it is for
 
-Config: {'vocab_size': 50257, 'd_model': 1704, 'n_heads': 12, 'd_ff': 6807, 'max_seq_len': 128, 'deq_max_iter': 12, 'deq_tol': 0.001, 'solver': 'anderson', 'jfb': False, 'dropout': 0.1, 'spectral_norm': True, 'residual_damping': 0.2, 'map_form': 'postln', 'aux_residual': False, 'lambda_aux': 0.1}
+This checkpoint exists to answer one question at a like-for-like setting: can a
+model whose depth is an equilibrium computation match a conventional stack of
+the same size trained on the same data for the same number of steps?
 
-## Provenance
+| | this model | explicit baseline |
+|---|---|---|
+| parameters | 120.7M | 123.8M |
+| depth | one tied block, iterated | 12 distinct layers |
+| BLiMP ratio vs baseline | **0.991** (3 seeds: 0.971 / 1.033 / 0.970) | 1.000 |
 
-- **Config Hash:** `33b106ea36f6f2db58eb1748a6169f269559b5d008ea87822816d214e0689b3d`
-- **Git Commit:** `unknown`
-- **Run Directory:** `/home/sharaths/projects/game-llm/results/exp13_seed43`
+The mean ratio of 0.991 is statistical parity, and one seed exceeds its baseline.
+Parity is the honest description; this model is not better than a conventional
+transformer of the same size, it is equal to one while storing its depth as
+iteration rather than as parameters.
 
-All numbers in this model card trace to documented runs; see [research/memory/findings.md](https://github.com/SharathSPhD/game-llm/blob/main/research/memory/findings.md) for validation details.
+## Training
 
-## Metrics
+Anytime-unrolled supervision: cross-entropy is applied to intermediate iterates
+at depths 4, 8 and 12 rather than only at the final one, which makes truncated
+solves usable and closes the width gap that earlier fixed-point training left
+open. BabyLM strict-small, matched data, steps and batch against the baseline.
 
-| Metric | Value |
-|--------|-------|
-| final_loss | 2.8680 |
+## Honest limitations
 
+Training supervises plain unrolled iterates while evaluation budgets cap an
+Anderson-accelerated solver, so the two use different algorithms at the same
+nominal depth; the parity figure is a claim about the end-to-end configuration
+rather than about the solver in isolation. Converting a *pretrained* large model
+to this topology is a different matter and does not work at budgets we could
+afford: at 1.7B, conversion retained 0.354 of above-chance benchmark capability
+and grade-school arithmetic collapsed from 0.457 to 0.017 while perplexity moved
+only to 1.19x baseline. Train in this topology rather than converting into it.
 
-## Disclaimer
+## Provenance
 
-This is research-stage code. Numbers reported here are from `results.json` in the named run directory and trace to `research/memory/findings.md`. For baseline comparisons and metric definitions, consult the paper at https://github.com/SharathSPhD/game-llm/tree/main/paper.
+Every number traces to a results file with a resolved-config hash and commit in
+https://github.com/SharathSPhD/game-llm — findings F16 through F24 for this
+model, F25/F26/F35 for the conversion results.
 
-## Links
+## Loading
 
-- [GitHub Repository](https://github.com/SharathSPhD/game-llm)
-- [Paper (arXiv/Site)](https://github.com/SharathSPhD/game-llm/tree/main/paper)
-- [Validated Findings](https://github.com/SharathSPhD/game-llm/blob/main/research/memory/findings.md)
-- [Kinetic AI Home](https://kinetic.kinetic-ai.workers.dev)
+```python
+import torch
+blob = torch.load("pytorch_model.pt", map_location="cpu", weights_only=True)
+# blob["config_dict"] -> EqLMConfig fields; blob["model_class"] == "EqLM"
+```

pytorch_model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24704232f3263a10494df06b70370b6f7ee87e5a2e2e25c58fb567c6dbec3010
+size 482926309