Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,59 @@
+---
+license: mit
+base_model: syvb/nanonla-qwen3-8b-L24-av
+tags:
+- nla
+- natural-language-autoencoder
+- qwen3
+---
+
+# nanonla-qwen3-8b-L24-av-ctrl-singletag-bs64
+
+**1-slot control arm** (status-quo single injection, matched compute).
+
+Continued AV-SFT warm-started from [`syvb/nanonla-qwen3-8b-L24-av`](https://huggingface.co/syvb/nanonla-qwen3-8b-L24-av)
+(Qwen3-8B, injection layer 24, d_model 4096).
+
+## Multi-input affine experiment
+
+Hypothesis: repeating the injection marker **N times**, each slot getting its own
+learned affine `A_i·v_norm + b_i` over the (normalized) injected activation, gives
+the backbone N independently-readable "views" of one activation and improves
+verbalization vs. the status-quo single injection.
+
+- **Experiment arm:** N=16 markers, 16 full d×d+bias affines, identity-init,
+  full continued AV-SFT (affines **and** backbone trained).
+- **Control arm:** N=1, no affine — same warm-start, same data, same step budget
+  (matched compute).
+
+Both arms: 1000 steps, eff. batch 64, 1 epoch (64k rows), lr 2.5e-5, single-tag, `injection_scale = sqrt(d_model)`. Trained with the
+standalone (miles-free) trainer `tools/train_av_standalone.py`.
+
+## Held-out val NLL (lower is better)
+
+Evaluated on **4001 doc-disjoint** held-out rows (val rows whose `doc_id`
+does **not** appear in the training split — a true held-out set, not just a row
+split), gold activation injected. Both arms scored on the **same** rows (paired).
+
+| arm | val NLL/token | perplexity |
+|---|---|---|
+| **16-slot affine (experiment)** | 1.4580 | 4.298 |
+| **1-slot (control)** | 1.4753 | 4.372 |
+| **Δ (control − experiment)** | **+0.0172** | — |
+
+Paired row-level bootstrap (10k resamples): Δ = **+0.0172** nats/token,
+95% CI **[+0.0167, +0.0178]**.
+
+✅ **The 16-slot affine significantly improves val NLL** (Δ=+0.0172, 95% CI [+0.0167, +0.0178] excludes 0).
+
+### Caveats
+- This is a **system-level** comparison (16 markers + per-slot affine vs. 1 marker).
+  It does **not** isolate the affine from the effect of simply repeating the
+  injection 16× — a "16 markers, no affine" arm would be needed for that.
+- Continued warm-start, not a converged run (1000 steps, eff. batch 64, 1 epoch (64k rows), lr 2.5e-5, single-tag).
+
+For the 16-slot model the per-slot affines are in `nla_affine.safetensors`
+(`weight [16,4096,4096]`, `bias [16,4096]`); apply them at injection time — see
+`launch/eval_av_val_loss.py --multi-input-slots 16 --affine-path ...`.
+
+wandb: https://wandb.ai/octahedral-systems/nla-multi-affine-experiment