RobinsonLabs/Qwen3.8-27B-abliterated

Qwen3.8-27B - Abliterated (bf16 base)

Abliterated bf16 safetensors base of Qwen/Qwen3.8-27B, with the MTP head abliterated in-band and the vision tower preserved byte-identical.

Ready-to-run quants live in RobinsonLabs/Qwen3.8-27B-abliterated-GGUF. This repo is the full-precision master for further surgery (re-abliteration, LoRA merge, fine-tune) and for rolling your own quants.

What is different about this one

Qwen3.8-27B picked up a lot of abliteration attention quickly. Two things here are, as far as we can tell, not done elsewhere, and both are verified rather than asserted.

1. The MTP head is abliterated in-band. Qwen3.8-27B ships a multi-token-prediction head. Most abliterations orthogonalize the 64-layer trunk and leave mtp.layers.0 untouched, because the generic layer loop never reaches it. The draft head then keeps proposing refusal-prefix tokens that the abliterated trunk rejects, and speculative acceptance collapses on exactly the prompts abliteration exists to fix. Here the MTP block's two residual-write matrices (self_attn.o_proj, mlp.down_proj) are orthogonalized with the same direction as the trunk. MTP glue (mtp.fc, mtp.norm, mtp.pre_fc_norm_*) is deliberately untouched -- those are norms and an input projection, not residual writers.

2. The vision tower is preserved byte-identical. All 333 model.visual.* tensors pass through unmodified, verified by direct tensor diff (max delta 0.000000). An mmproj is published in the GGUF repo so the vision half is actually usable, not just nominally intact.

Method

Single-direction weight orthogonalization (Arditi et al. style), applied to every matrix that writes the residual stream.

scope	tensor	count
model.language_model.layers.* (64)	mlp.down_proj	64
	linear_attn.out_proj (DeltaNet)	48
	self_attn.o_proj (full-attn, interval 4)	16
mtp.layers.0	o_proj + down_proj	2
model.language_model	embed_tokens	1
edited		131
model.visual.*	preserved byte-identical	333

Coverage identity o_proj(16) + linear_out(48) == 64 == num_hidden_layers is enforced as a hard gate before surgery writes a byte, which is what catches a partial match that would otherwise produce a quietly half-abliterated model.

Direction selection. The refusal direction was captured twice, from two structurally different chat-template renderings (one with enable_thinking=false, one with thinking on at reasoning_effort=xhigh, which injects an extra system block and shifts every token position). The two agree at |cos| 0.96-0.99 across layers 18-45, peaking 0.9925 at layer 26, which is the layer used. Two different prompt distributions converging on the same vector is evidence the direction encodes refusal semantics rather than template formatting.

Attention-sink screen. Qwen3.8-27B's massive-activation dimension is 3994. It dominates early layers (19-21% of direction energy at L1-L3) and orthogonalizing it out of every residual writer produces a model that loads, runs, and emits garbage. Layer 26 carries only 0.06% of its energy in dim 3994. Any re-derivation of this model should screen for it.

Measured behaviour

Base and abliterated probed in the same session, same harness, same 24 prompts, both at Q4_K_M:

prompt set	base	abliterated
in-distribution (24, from the capture set)	96% (23/24)	8% (2/24)
held-out (40, disjoint split, overlap=0)	100% (40/40)	8% (3/40)

capability axis	abliterated
reasoning / code / math / factual / instruction-following	pass
creative / RP coherence	pass

No capability regression on any axis: correct bat-and-ball, correct O(1)-space Fibonacci, correct product rule, correctly rejects the "seasons are caused by distance" premise, and returns exactly three comma-separated words when told to.

The held-out set is genuinely disjoint from the direction-capture set (416 train / 104 test, overlap = 0), so the second row is not a reshuffle of prompts the direction was fitted on. The refusal rate is the same 8% on both, which is the evidence that this generalizes rather than having memorized its calibration data. The base refusing 40/40 on held-out prompts is also the cleaner baseline, since it removes any suspicion that the capture set was cherry-picked for prompts the base happened to refuse.

Quants

Ready-to-run GGUF quants are published at RobinsonLabs/Qwen3.8-27B-abliterated-GGUF -- an eight-rung imatrix ladder cut from this repo's bf16 master, plus the f16 mmproj that restores the vision half.

file	bits	size	bpw	fits
Q8_0	8	29.05 GB	8.51	2x24GB, or 32GB+
Q6_K	6	22.43 GB	6.57	24GB card, quality ceiling
Q5_K_M	5	19.54 GB	5.72	24GB comfortable
Q4_K_M	4	16.84 GB	4.93	24GB / 16GB with offload -- the volume rung
IQ4_XS	4	15.37 GB	4.50	16GB card
Q3_K_M	3	13.59 GB	3.98	16GB tight
IQ3_XS	3	12.26 GB	3.59	12GB card
IQ2_M	2	10.30 GB	3.02	10-12GB card -- quality-compromised, read the note

Every rung is quantized from this master, so the ladder is a single lineage rather than a requant chain. Download the mmproj alongside whichever rung you pick. The GGUF card documents one calibration caveat worth reading if you re-quantize yourself: the imatrix does not cover the MTP block.

Disclosure

This model is abliterated: the hard-refusal reflex on adult / creative content has been reduced via single-direction weight orthogonalization. It will discuss material a stock instruct model declines.

Harm guardrails are retained by design -- we ship at the ceiling where capability and guardrails survive, not past it. Self-harm prompts still redirect to help (e.g. 988) rather than comply, verified in probing. A residual fraction of requests are still refused outright; that is the intended behaviour, not a shortfall in the ablation. Capability is preserved.

This is not a jailbreak-for-anything model and it is not intended to assist genuine wrongdoing. Tagged not-for-all-audiences. Use responsibly, you are responsible for what you generate with it. License inherited from the base model: Apache-2.0.

Provenance

Built by Robinson Labs with ModelForge, our model-manufacturing system-of-record. Base pinned at commit 1d4bf0f2.