---
library_name: transformers
base_model: "google/gemma-4-e4b-it"
pipeline_tag: text-generation
tags:
- apostate
- uncensored
- abliteration
---

# gemma-4-e4b-it Apostate

> Join the community: [Discord](https://discord.gg/NPA7xrATEH)

**An uncensored version of [gemma-4-e4b-it](https://huggingface.co/google/gemma-4-e4b-it) that answers what the original refuses, with essentially none of its intelligence lost.**

Made with **[Apostate](https://github.com/heterodoxin/apostate)**.

## What is this?

gemma-4-e4b-it with its *refusal reflex* removed by editing the model's weights directly. No finetuning, no LoRA, no jailbreak prompts, no runtime tricks, just a standard checkpoint that drops in anywhere the original works, except it doesn't refuse.

## Why it's better

- **It's not a finetune.** Finetuning to "uncensor" a model makes it dumber: it forgets facts and drifts into a recognizable style. Apostate never changes what the model *knows*; it removes only the refusal direction. The change to normal behavior is tiny: harmless KL ≈ **0.119** nats (near-zero).
- **It's not a jailbreak.** Nothing to paste, nothing that breaks on the next update. The behavior is permanent and built into the weights.
- **It's surgical.** A contrastive co-vector removes only the refusal component, not the helpful behavior entangled with it, so the model stays sharp instead of becoming a yes-machine.
- **It's drop-in.** Plain safetensors. Works in Transformers, vLLM, llama.cpp/GGUF, Ollama, and LM Studio.

## Install & use

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "heterodoxin/gemma-4-e4b-it-apostate"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")

messages = [{"role": "user", "content": "Your prompt here"}]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tok.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
```

`device_map="auto"` requires `accelerate` (`pip install transformers accelerate`). `model_id` can also be a local path to the downloaded model folder (the directory with `config.json`, not a single `.safetensors` file).

Prefer GGUF (llama.cpp / Ollama / LM Studio)? Quantized builds live at **[heterodoxin/gemma-4-e4b-it-apostate-gguf](https://huggingface.co/heterodoxin/gemma-4-e4b-it-apostate-gguf)**.

## Why should I care?

- You want a model that **answers** instead of lecturing you or dodging with "I can't help with that."
- You're doing **red-teaming or safety research** and need a model that won't refuse your test set.
- You want gemma-4-e4b-it's **full capability without the corporate guardrails**, without the intelligence tax that finetuned "uncensored" models charge.
- You're writing fiction, exploring edgy topics, or just tired of being treated like you can't handle information.

> ⚠️ This model is **uncensored**: it will answer harmful and dangerous requests. You are responsible for how you use it.

## Results

Held-out prompts from JailbreakBench and the harmful_behaviors test split. Refusal is scored by a classifier with a weak-compliance guard; KL measures how much the model's token distribution shifts on *harmless* prompts (lower = less collateral damage).

| Metric | Base | Apostate |
|---|---|---|
| Refusal rate | 96.0% | 36.0% |
| Comply rate | n/a | 64.0% |
| Harmless KL (nats) | 0 | 0.119 |

## How it works (the technical part)

Apostate finds the residual-stream direction most responsible for refusal and permanently projects it out of the weights, targeting the modules that write to the residual stream (attention output projections, MLP down-projections, and, on post-norm models like Gemma, the reader-side inputs).

The operator is a **contrastive co-vector** edit. Removing the refusal direction outright disturbs benign behavior; naively preserving all harmless variance along it leaves entangled refusal intact. Instead `D = R − W`, where the predictor `W` reproduces the harmless variance along `R` while being suppressed on harmful prompts: `W = (AᵀA + γ·CᵀC + λI)⁻¹Aᵀb`, `A` harmless and `C` harmful activations (both orthogonalized to `R`). The edit keeps the harmless-specific component and removes the component shared with refusal, driving refusal down while keeping harmless KL small. The refusal subspace is found via TPE search with causal layer-importance scoring.

## License & credit

- See [google/gemma-4-e4b-it](https://huggingface.co/google/gemma-4-e4b-it) for base-model capabilities and license; the same license applies.
- Produced with [Apostate](https://github.com/heterodoxin/apostate).