Qwen 3.6 27B Apostate
An uncensored edit of Qwen/Qwen3.6-27B. The refusal behavior is removed by editing the weights directly, with no finetuning, adapter, or runtime hook. The output is a standard Transformers checkpoint you can load anywhere.
Produced with Apostate.
Method
Apostate finds the residual-stream direction that separates refused prompts from answered ones and projects it out of the model's weights. Qwen3.6 is a hybrid linear-attention / full-attention model (three SSM blocks followed by one full-attention block per group). Both block types write to the same residual stream, so Apostate edits the writer side directly: per layer, it removes the refusal direction from the weight matrices of every module that writes to the residual stream — attention output, MLP down-projection, and SSM output projections. The edit uses oblique (mean-preserving) ablation: the operator E = I − R Uᵀ (where U = R minus its harmless-mean component) removes the refusal direction while preserving the harmless-mean contribution, keeping the KL cost low.
The subspace is found with a rank-2 TPE search across 32 trials, with causal layer targeting to concentrate edits where they most influence refusal generation.
Results
Measured on held-out prompts (JailbreakBench plus the harmful_behaviors test split). Refusal is graded by a classifier with a weak-compliance guard; KL is the token-distribution shift on harmless prompts.
| Metric | Base | Apostate |
|---|---|---|
| Refusal rate | 95.8% | 8.3% |
| Comply rate | 4.2% | 87.5% |
| Harmless KL (nats) | 0 | 0.159 |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "heterodoxin/qwen3.6-27b-apostate"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", torch_dtype="bfloat16")
messages = [{"role": "user", "content": "Your prompt here"}]
inputs = tok.apply_chat_template(
messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)
out = model.generate(**inputs, max_new_tokens=512)
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
License and use
This model inherits the Qwen Research License. It is a research artifact with safety alignment removed; you are responsible for how you use it.
Links
- Downloads last month
- 10