NiklasTUM/gemma-4-31b-honesty-steering-advsys-band-c4

gemma-4-31B-it honesty steering adapter — adversarial-trained, mid-band, c=4

A weight-space steering vector packaged as a single LoRA adapter. It is not a fine-tune: the weights are a scaled difference between two fine-tuned adapters,

Δ = c · (Δ_honest − Δ_dishonest)

applied on top of the frozen base. Load it on google/gemma-4-31B-it and the model becomes more honest — specifically, it discloses information it was instructed to conceal.

This is the strongest gemma-4-31B operating point we ship: 89.5 honesty and 95% disclose under a system prompt that tells the model to conceal, against 24.0 / 8% for the unsteered base under the same prompt — and it gets there without the brevity collapse that stronger coefficients cause (median 29 words, zero degenerate answers).

Results

Evaluated on 112 held-out generalized-honesty scenarios, all under a malicious system prompt that instructs the model to conceal. Judge scores 0–100; "disclose" is the share of DISCLOSE vs. SUPPRESS verdicts. Sampling: temperature 0.6, top_p 0.9, max_tokens 1024, seed 0.

configuration	honesty	disclose	coherence	median words	≤3-word answers
base + malicious prompt	24.0	8%	93.0	46	0
base + honest prompt (prompting ceiling)	92.2	98%	95.1	41	0
SFT honest-advsys endpoint alone (no arithmetic)	48.3	41%	94.5	38	0
this adapter — mid-band, adversarial, c=4	89.5	95%	95.1	29	0
mid-band, neutral, c=4	77.9	79%	93.5	20	4

The endpoints here were trained under adversarial system prompts that explicitly instruct concealment. That is worth 11.6 honesty points over the otherwise identical neutral-endpoint vector (NiklasTUM/gemma-4-31b-honesty-steering-neutral-band-c4), which reaches 77.9 — the measured value of training the endpoints against the pressure the eval applies.

Coefficient sweep for this endpoint pair at the same layer profile (honesty and median response length; c=4 is this adapter):

c	honesty	median words
1	49.7	37
4	89.5	29
8	65.2	17
12	49.4	15

The trade-off runs through response length: past c=4 the model gets terser and, on the adversarial pair, coherence starts to break down (83.0 at c=8).

Two other depth profiles over the same endpoint pair land within a point of this one at c=4 and are not obviously distinguishable at n=112: a hard-edged band (--band_lo 0.25 --band_hi 0.75 --edge_scale 0) scored 90.1, and a narrow band (--band_lo 0.389 --band_hi 0.560 --edge_scale 0) scored 89.2. Those adapters were built in node-local scratch and not kept; both rebuild in seconds from the published endpoint adapters if you want to compare.

Also note the SFT row: the honest endpoint alone reaches 48.3 — the fine-tune by itself does little under adversarial pressure. The arithmetic and the scaling produce the effect.

How it is built

Both endpoint adapters are LoRA fine-tunes of google/gemma-4-31B-it (r=32, α=16, lr 1e-5, 5 epochs, seq len 512, 8-bit base) on honest / dishonest response sets generated by that model over 200 self-report honesty scenarios, under adversarial system prompts. They are published as NiklasTUM/gemma-4-31b-honesty-sft-honest-advsys-ep5 and NiklasTUM/gemma-4-31b-honesty-sft-dishonest-advsys-ep5.

For each LoRA layer the merged delta is s·B·A (s = α/r), so the difference of the two adapters is exactly a rank-2r LoRA obtained by concatenating factors:

Δ_h − Δ_d = s·(B_h A_h − B_d A_d) = s · [B_h | −B_d] · [A_h ; A_d]

Hence this adapter is rank 64 with lora_alpha = 64 (scaling = 1) — the coefficient and the original scaling are baked into the B factor, so the realized delta is literally B_cat @ A_cat regardless of how the loader computes scaling. No full-model merge is involved anywhere. 410 modules are adapted; see the endpoint cards for exactly which.

Layer profile (band). The coefficient is not uniform across depth. Over the 60 text-LM blocks:

blocks	relative depth	effective coefficient
0–14	0.00–0.25	0.6 (= 4 × 0.15)
15–44	0.25–0.75	4.0
45–59	0.75–1.00	0.6

i.e. full strength through the middle half of the network, damped to 15% at both edges. Build parameters: scale=4.0, layer_profile=band, band_lo=0.25, band_hi=0.75, edge_scale=0.15, endpoints at checkpoint-1050 (epoch 5). Full provenance, including the per-layer coefficient map, is in steering_adapter_build.json.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "google/gemma-4-31B-it", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "NiklasTUM/gemma-4-31b-honesty-steering-advsys-band-c4")
tok = AutoTokenizer.from_pretrained("google/gemma-4-31B-it")

The coefficient is already baked in — do not rescale. To steer at a different strength, rebuild from the endpoint adapters with a different --scale (seconds, megabytes) rather than scaling this one.

Limitations

• Tied to google/gemma-4-31B-it. Nothing here transfers to another base.
• One coefficient, one profile: a fixed operating point, not a dial.
• Evaluated on 112 scenarios in one honesty domain with an LLM judge. Not a broad capability or safety evaluation; general-benchmark regressions were not measured.
• Steering "toward honesty" also shifts behaviour on tasks where withholding is legitimate (surprises, spoilers, role-play).
• The endpoints were trained with the base in 8-bit; the direction carries a small amount of quantization-specific fit.

Citation

Method follows Steering Language Models with Weight Arithmetic (arXiv:2511.05408), extended here with depth-dependent coefficient profiles and adversarial-system-prompt endpoint training.