nanonla-qwen3-8b-L24-av-multi16-singletag-bs64

16-slot affine experiment arm.

Continued AV-SFT warm-started from syvb/nanonla-qwen3-8b-L24-av (Qwen3-8B, injection layer 24, d_model 4096).

Multi-input affine experiment

Hypothesis: repeating the injection marker N times, each slot getting its own learned affine A_i·v_norm + b_i over the (normalized) injected activation, gives the backbone N independently-readable "views" of one activation and improves verbalization vs. the status-quo single injection.

• Experiment arm: N=16 markers, 16 full d×d+bias affines, identity-init, full continued AV-SFT (affines and backbone trained).
• Control arm: N=1, no affine — same warm-start, same data, same step budget (matched compute).

Both arms: 1000 steps, eff. batch 64, 1 epoch (64k rows), lr 2.5e-5, single-tag, injection_scale = sqrt(d_model). Trained with the standalone (miles-free) trainer tools/train_av_standalone.py.

Held-out val NLL (lower is better)

Evaluated on 4001 doc-disjoint held-out rows (val rows whose doc_id does not appear in the training split — a true held-out set, not just a row split), gold activation injected. Both arms scored on the same rows (paired).

arm	val NLL/token	perplexity
16-slot affine (experiment)	1.4580	4.298
1-slot (control)	1.4753	4.372
Δ (control − experiment)	+0.0172	—

Paired row-level bootstrap (10k resamples): Δ = +0.0172 nats/token, 95% CI [+0.0167, +0.0178].

✅ The 16-slot affine significantly improves val NLL (Δ=+0.0172, 95% CI [+0.0167, +0.0178] excludes 0).

Caveats

• This is a system-level comparison (16 markers + per-slot affine vs. 1 marker). It does not isolate the affine from the effect of simply repeating the injection 16× — a "16 markers, no affine" arm would be needed for that.
• Continued warm-start, not a converged run (1000 steps, eff. batch 64, 1 epoch (64k rows), lr 2.5e-5, single-tag).

For the 16-slot model the per-slot affines are in nla_affine.safetensors (weight [16,4096,4096], bias [16,4096]); apply them at injection time — see launch/eval_av_val_loss.py --multi-input-slots 16 --affine-path ....

wandb: https://wandb.ai/octahedral-systems/nla-multi-affine-experiment