Docstring precision: pinning cost = intervention artifact; LayerNorm = candidate mechanism (not isolated)
Browse files- model_v2.py +3 -2
model_v2.py
CHANGED
|
@@ -18,8 +18,9 @@ Everything is operator-derived → operator-only bottleneck holds. STALE-PREDICT
|
|
| 18 |
NOTE (2026-09-10, kept for the record): "the R_state→identity bypass collapses to
|
| 19 |
unigram" was written for the PRE-TOWER readout and was true of it; with the grade
|
| 20 |
tower (shipped config) the readout has a direct route to B_t..B_{t-2} and identity-
|
| 21 |
-
scan costs only ~1.2x (
|
| 22 |
-
~1.0x;
|
|
|
|
| 23 |
At LM inference the transported state is causally decorative; in sole-channel/tape
|
| 24 |
configs it is load-bearing. See paper §3.2.
|
| 25 |
|
|
|
|
| 18 |
NOTE (2026-09-10, kept for the record): "the R_state→identity bypass collapses to
|
| 19 |
unigram" was written for the PRE-TOWER readout and was true of it; with the grade
|
| 20 |
tower (shipped config) the readout has a direct route to B_t..B_{t-2} and identity-
|
| 21 |
+
scan costs only ~1.2x (an intervention artifact, not lost history — time-shuffling S
|
| 22 |
+
costs ~1.0x; shared-LayerNorm shift is the candidate mechanism, not yet isolated;
|
| 23 |
+
measured internally and independently replicated by N. Watson 2026-09-10).
|
| 24 |
At LM inference the transported state is causally decorative; in sole-channel/tape
|
| 25 |
configs it is load-bearing. See paper §3.2.
|
| 26 |
|