etwk commited on
Commit ·
35ad1a4
1
Parent(s): f704813
Link GitHub research repository
Browse files
README.md
CHANGED
|
@@ -23,6 +23,14 @@ the same **carry-aware TCN** (~10.7M params total across two shared weight files
|
|
| 23 |
than memorising finite multiplication tables, and it verifiably generalises to primes never seen
|
| 24 |
in training.
|
| 25 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
## The idea
|
| 27 |
|
| 28 |
Direct classification of the bilinear map `(a, b) -> a*b mod p` does not generalise across
|
|
@@ -137,11 +145,14 @@ position. Combined with gradient accumulation (effective batch ~26k) and the wor
|
|
| 137 |
loss, this took tier 9 from **0.73 -> 0.99**, even across prime widths (held-out value-uniform
|
| 138 |
validation 0.99; per-width 1015-1024 all ~0.99).
|
| 139 |
|
| 140 |
-
The training scripts
|
| 141 |
-
research
|
| 142 |
-
|
| 143 |
-
|
| 144 |
-
|
|
|
|
|
|
|
|
|
|
| 145 |
|
| 146 |
```bash
|
| 147 |
# Historical small-prime cells were first trained width-matched, then absorbed into the shared cell.
|
|
|
|
| 23 |
than memorising finite multiplication tables, and it verifiably generalises to primes never seen
|
| 24 |
in training.
|
| 25 |
|
| 26 |
+
## Source and reproducibility
|
| 27 |
+
|
| 28 |
+
The complete training code, evaluation utilities, experiment notes, and reproducibility guide are
|
| 29 |
+
available in the public GitHub repository
|
| 30 |
+
[`Xllent-AI/modular_arithmetic`](https://github.com/Xllent-AI/modular_arithmetic). This Hugging Face
|
| 31 |
+
repository remains the standalone submission artifact containing the model code, manifest, and
|
| 32 |
+
trained weights.
|
| 33 |
+
|
| 34 |
## The idea
|
| 35 |
|
| 36 |
Direct classification of the bilinear map `(a, b) -> a*b mod p` does not generalise across
|
|
|
|
| 145 |
loss, this took tier 9 from **0.73 -> 0.99**, even across prime widths (held-out value-uniform
|
| 146 |
validation 0.99; per-width 1015-1024 all ~0.99).
|
| 147 |
|
| 148 |
+
The training scripts live under `exploration/` in the
|
| 149 |
+
[GitHub research repository](https://github.com/Xllent-AI/modular_arithmetic); they are not shipped
|
| 150 |
+
in this standalone model repository. The commands below document *how the weights were obtained*
|
| 151 |
+
(the provenance the rules ask for) and are not runnable as-is from this repository alone.
|
| 152 |
+
Intermediate warm-start files are absent from the current revision but remain available in the
|
| 153 |
+
repository history; for example,
|
| 154 |
+
[`weights_shared_64_512.pt`](https://huggingface.co/XllentAI/modular_arithmetic/blob/fff63d15412d615df043af59c90ff28f372173e8/weights_shared_64_512.pt)
|
| 155 |
+
is the prior shared 64–512 cell.
|
| 156 |
|
| 157 |
```bash
|
| 158 |
# Historical small-prime cells were first trained width-matched, then absorbed into the shared cell.
|