Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
---
|
| 4 |
+
|
| 5 |
+
# EqLM — kinetic-eqlm-anytime-121m-babylm
|
| 6 |
+
|
| 7 |
+
This is an equilibrium language model from the **Kinetic AI** research project.
|
| 8 |
+
|
| 9 |
+
## Architecture
|
| 10 |
+
|
| 11 |
+
Config: {'vocab_size': 50257, 'd_model': 1704, 'n_heads': 12, 'd_ff': 6807, 'max_seq_len': 128, 'deq_max_iter': 12, 'deq_tol': 0.001, 'solver': 'anderson', 'jfb': False, 'dropout': 0.1, 'spectral_norm': True, 'residual_damping': 0.2, 'map_form': 'postln', 'aux_residual': False, 'lambda_aux': 0.1}
|
| 12 |
+
|
| 13 |
+
## Provenance
|
| 14 |
+
|
| 15 |
+
- **Config Hash:** `33b106ea36f6f2db58eb1748a6169f269559b5d008ea87822816d214e0689b3d`
|
| 16 |
+
- **Git Commit:** `unknown`
|
| 17 |
+
- **Run Directory:** `/home/sharaths/projects/game-llm/results/exp13_seed43`
|
| 18 |
+
|
| 19 |
+
All numbers in this model card trace to documented runs; see [research/memory/findings.md](https://github.com/SharathSPhD/game-llm/blob/main/research/memory/findings.md) for validation details.
|
| 20 |
+
|
| 21 |
+
## Metrics
|
| 22 |
+
|
| 23 |
+
| Metric | Value |
|
| 24 |
+
|--------|-------|
|
| 25 |
+
| final_loss | 2.8680 |
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
## Disclaimer
|
| 29 |
+
|
| 30 |
+
This is research-stage code. Numbers reported here are from `results.json` in the named run directory and trace to `research/memory/findings.md`. For baseline comparisons and metric definitions, consult the paper at https://github.com/SharathSPhD/game-llm/tree/main/paper.
|
| 31 |
+
|
| 32 |
+
## Links
|
| 33 |
+
|
| 34 |
+
- [GitHub Repository](https://github.com/SharathSPhD/game-llm)
|
| 35 |
+
- [Paper (arXiv/Site)](https://github.com/SharathSPhD/game-llm/tree/main/paper)
|
| 36 |
+
- [Validated Findings](https://github.com/SharathSPhD/game-llm/blob/main/research/memory/findings.md)
|
| 37 |
+
- [Kinetic AI Home](https://kinetic.kinetic-ai.workers.dev)
|