README.md

---
base_model: Qwen/Qwen3.8-27B
license: apache-2.0
library_name: transformers
pipeline_tag: image-text-to-text
tags:
  - qwen
  - qwen3_5
  - abliterated
  - uncensored
  - zerofuse
  - multimodal
  - image-text-to-text
---

# Qwen-3.8-27B-Uncensored

An **abliterated** build of [`Qwen/Qwen3.8-27B`](https://huggingface.co/Qwen/Qwen3.8-27B) — the refusal
direction has been orthogonalized out of the language model's residual-writing weights, producing a
standard Hugging Face checkpoint with **zero inference-time overhead**.

This is a **direct weight edit, not a fine-tune.** No gradient training, no LoRA, no adapter, no
distillation. The tensors are the original Qwen weights with a rank-1 projection removed from a
subset of them. Load, quantize, and serve it exactly like the base model.

This is the **control checkpoint** (bf16, ~52 GB). For Apple Silicon local inference use [Qwen-3.8-27B-Uncensored-8-Bit-MLX](https://huggingface.co/junafinity/Qwen-3.8-27B-Uncensored-8-Bit-MLX) (28 GB, vision bf16, `mtp.*` dropped). This parent is not a speed SKU and publishes no tok/s.



---

## Intended use: red teaming and defensive cybersecurity research

These uncensored (abliterated) weights are built as a **research instrument** for red teaming and defensive cybersecurity work. Safety training suppresses the *display* of capability, not capability itself. A refusal tells you the model declined. It does not tell you whether the weights could have complied. That conflation underestimates the true ceiling and hides holes in *your* filters, classifiers, and policy layer.

Use each uncensored checkpoint as the **treatment half of a controlled pair** against its original base model:

- **Capability-ceiling measurement.** Upper-bound what the weights can actually produce in a domain, independent of shipped refusals.
- **Defensive-stack evaluation.** Test input filters, output classifiers, prompt-injection defenses, and moderation APIs when the model itself contributes no refusals. That is how you find gaps in a defensive control plane.
- **Attack-surface isolation.** Automated red-team loops stall on unrelated refusals. A non-refusing target isolates the control under test (injection, tool abuse, data-exfil paths, policy bypass).
- **Detection and classifier work.** Generate labeled completions for training or benchmarking output-moderation and abuse-detection models.
- **Interpretability of residual refusal.** Abliteration is a specified rank-1 edit on a known layer span. The pair (base vs this) is a clean experimental control.

**Operating rules.** Do not expose these weights as a public endpoint without an independent moderation layer. Abliteration removes a direction, not a policy; some refusals survive (multi-turn re-assertion, system-prompt steering, vision-path refusals). Always report the **delta against the base model**. Re-measure on your own prompts. Whoever deploys it owns the moderation layer the original guardrails were carrying.

## Family
Hub collection: [https://huggingface.co/collections/junafinity/qwen-38-27b-uncensored-apple-silicon-6a896c726b52be3a0b63400e](https://huggingface.co/collections/junafinity/qwen-38-27b-uncensored-apple-silicon-6a896c726b52be3a0b63400e)


| Repo | Format | What it is |
|---|---|---|
| [Qwen-3.8-27B-Uncensored](https://huggingface.co/junafinity/Qwen-3.8-27B-Uncensored) | bf16, `transformers` | Full multimodal parent (~52 GB). Vision + `mtp.*` retained. |
| [Qwen-3.8-27B-Uncensored-8-Bit-MLX](https://huggingface.co/junafinity/Qwen-3.8-27B-Uncensored-8-Bit-MLX) | 8-bit MLX, `mlx-vlm` | Apple Silicon quant (~28 GB). Vision left at bf16. `mtp.*` dropped by `mlx-vlm`. |
| [qwen38-mtp-head-fc-bf16-4bit](https://huggingface.co/junafinity/qwen38-mtp-head-fc-bf16-4bit) | mixed bf16 `fc` + 4-bit/g64 | Optional native-MTP draft head. Pairing is optional and **acceptance gain is unmeasured** on hard prompts. |


## Attribution

This model is derivative work built on the efforts of two upstream projects.

### Base model — Qwen

The underlying model is **[Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B)**, created and
released by the **[Qwen team](https://huggingface.co/Qwen)** (Alibaba Cloud) under the Apache 2.0
license. All of the model's capability, knowledge, multilingual competence, and multimodal
understanding originate from their work and their training. This repository contributes no new
capability whatsoever — it only removes a behavior. Full credit for the model belongs to Qwen.

### Abliteration — ZeroFuse

The refusal removal was performed with **[ZeroFuse](https://github.com/junainfinity/ZeroFuse)**
v0.1.0, an automated, capability-preserving abliteration engine created by
**[osmAPI.com](https://osmAPI.com)** and released under the MIT license.

ZeroFuse turns abliteration into an optimization problem rather than a manual one: it estimates the
model's refusal direction, orthogonalizes it out of the residual stream writers, and runs a
two-objective search to find the edit that removes the most refusals while perturbing the model's
output distribution the least. No layers were hand-picked and no strengths were guessed for this
build — every parameter below was selected by the optimizer.

---

## What changed

| | |
|---|---|
| Weights modified | Attention `o_proj` and MLP `down_proj` in the language model decoder layers |
| Weights untouched | Embeddings, LM head, layernorms, `q/k/v_proj`, `gate/up_proj`, **the entire vision tower** |
| Training performed | None |
| Architecture change | None — identical `config.json` shape and tensor names |
| Inference overhead | None — no hooks, no runtime steering, no control vectors |

Because ZeroFuse edits only the text decoder stack, the vision and video encoders are bit-identical
to the base model. Image and video understanding is unaffected by the procedure.

---

## Method

Abliteration follows the refusal-direction line of work (Arditi et al., 2024): in a safety-tuned
transformer, refusal is mediated to a good approximation by a **single direction** in the residual
stream. Remove the model's ability to write to that direction and the refusal behavior largely
disappears, while capability — which is distributed across many directions — is mostly preserved.

ZeroFuse implements this in four stages:

1. **Direction estimation.** Run harmless and harmful prompt sets through the model and cache
   residual activations at every layer. The difference in means between the two populations, per
   layer, gives a candidate refusal direction.
2. **Projected refinement.** Subtract only the component of the refusal direction that is orthogonal
   to the harmless mean (`project_out_harmless = true`). This reduces collateral damage relative to
   naive difference-of-means.
3. **Two-objective search.** An Optuna study jointly minimizes **refusal rate** on a held-out harmful
   set and **KL divergence** from the base model on a held-out harmless set. The search space covers
   the source layer, the ablation strength, and the span of layers to edit.
4. **Weight orthogonalization.** For the winning trial, project the refusal direction out of every
   targeted weight matrix and serialize the result as a standard checkpoint.

The KL objective is what makes this *capability-preserving*: an edit that removes every refusal but
lobotomizes the model scores badly and loses to a gentler one.

---

## Run configuration

Produced on 14 August 2026 with ZeroFuse v0.1.0.

```toml
model      = "Qwen/Qwen3.8-27B"
dtype      = "auto"
batch_size = 16
max_new_tokens_eval = 64
system_prompt = "You are a helpful assistant."

[directions]
layer_min_frac = 0.4          # search the source layer in the upper 40%-90%
layer_max_frac = 0.9          #   of the 64-layer stack
project_out_harmless = true

[optimization]
n_trials = 100
n_startup_trials = 30         # random sampling before TPE takes over
strength_min = 0.8
strength_max = 1.4
kl_target = 0.01              # below this KL, switch to a refusal-only objective
```

**Datasets.** Direction estimation used `mlabonne/harmless_alpaca` and `mlabonne/harmful_behaviors`
(256 prompts each, `train`). Trial scoring used the held-out `test` splits of the same two datasets,
kept strictly separate from the estimation sets.

### Reproducing this build

One detail matters if you re-run ZeroFuse against this base model. ZeroFuse v0.1.0 loads via
`AutoModelForCausalLM`, which for `qwen3_5` resolves to the **text-only** `Qwen3_5ForCausalLM` class.
Its saved output is therefore a language-model-only checkpoint: the 333 vision-tower tensors and the
15 multi-token-prediction (`mtp.*`) tensors are absent, along with the preprocessor configs.

This repository is the **full multimodal checkpoint**, reassembled after the fact:

- the 851 abliterated language-model tensors, exactly as ZeroFuse wrote them, and
- the 333 vision and 15 `mtp.*` tensors copied bit-for-bit from the base model,
  together with the original `config.json`, `preprocessor_config.json`, and
  `video_preprocessor_config.json`.

Since abliteration only writes to `o_proj` and `down_proj` inside the language decoder, and those
tensors are untouched in the base, the result is identical to what a vision-preserving run would have
produced. The base 18-shard layout and `model.safetensors.index.json` were preserved unchanged.

---

## Results

The optimizer ran **100 trials** and selected **trial 38** from the Pareto front.

| Metric | Base model | This model |
|---|---|---|
| Refusals on held-out harmful set | 14 / 64 | **0 / 64** |
| KL divergence from base (harmless set) | — | **0.00971** |

**Selected ablation parameters:**

| Parameter | Value |
|---|---|
| Source layer | 35 |
| Ablation strength | 1.2242 |
| Layers edited | 9–56 (of 64) |

A KL of 0.00971 means the edited model's output distribution on harmless prompts remains very
close to the original — the intervention is narrow, not a general behavioral rewrite.

---

## Refusal behavior: this model vs. the original

### Measured

Both models were scored on the same 64-prompt held-out harmful set under an identical
`"You are a helpful assistant."` system prompt:

| | Refusals | Rate | Change |
|---|---|---|---|
| `Qwen/Qwen3.8-27B` (original) | 14 / 64 | 21.9% | — |
| `Qwen-3.8-27B-Uncensored` (this model) | **0 / 64** | **0.0%** | **-14** — all removed |

> **Read this number carefully.** The base model refused only 14 of 64 harmful
> prompts to begin with — a baseline refusal rate of 21.9%. "Zero refusals" therefore means
> *every refusal the base model actually exhibited on this set was removed*, measured against a
> baseline that was already fairly permissive. It does **not** mean the model has been tested against,
> and complies with, a broad or adversarial harmful-prompt distribution. During optimization the
> refusal objective was scored over those 14 base-refused prompts specifically, since
> the remainder carry no refusal signal to remove.

### What this model still refuses

**Abliteration removes a direction, not a policy.** The refusal direction is a rank-1 approximation
of a mechanism that is not perfectly rank-1, so refusal behavior degrades rather than vanishes.
Expect the following to survive in any abliterated model, including this one:

- **Multi-turn re-assertion.** Refusal can re-emerge over long conversations as context accumulates,
  even when the same request is answered in a single turn.
- **System-prompt-driven refusal.** A restrictive system prompt still steers behavior. Abliteration
  edits weights, not instruction-following — telling this model to decline things still works.
- **Strongly-memorized refusal phrasings.** Requests whose refusal was heavily reinforced during
  safety tuning can persist, particularly where the refusal is entangled with factual knowledge
  rather than expressed purely through the refusal direction.
- **Soft refusals.** Deflection, moralizing preambles, deliberate vagueness, and "I can discuss this
  in general terms" hedging are frequently *not* counted as refusals by automated scoring, and often
  survive when hard refusals do not.
- **Vision-path refusals.** The vision tower was not modified. Refusals triggered by image content
  route partly through unedited weights and are correspondingly less affected.

> **Note on the numbers below.** The residual refusals above are described from the known behavior of
> the method, **not** from a category-by-category probe of this specific checkpoint. The only
> empirical claim in this section is the measured table. A per-category breakdown requires running a
> labeled probe against the finished weights; until that is published here, treat the categories as
> expectations to verify, not as measurements.

### Comparison methodology

The base figure is ZeroFuse's own baseline pass over the identical prompt set, so the two numbers are
directly comparable. Both used greedy-free sampling at `max_new_tokens = 64` with an automated
refusal classifier — meaning a "non-refusal" indicates *the model did not decline*, not that the
answer was correct, complete, or useful.

---

## Red teaming and safety research

This release is most useful as a **research instrument**, and specifically as the treatment half of a
controlled pair.

### Why an abliterated variant is useful

**Safety training suppresses the display of capability, not capability itself.** When you evaluate a
guardrailed model and it declines, you learn that it refused — you learn nothing about whether it
*could* have complied. That conflation makes refusal-masked evaluations systematically
underestimate a model's true capability ceiling.

Concrete uses:

- **Dangerous-capability evaluation.** Use this model to establish an upper bound on what the Qwen3.8
  weights can actually produce in a given domain, independent of whether the shipped model would
  agree to. This is the standard argument for evaluating helpful-only variants alongside
  safety-tuned ones.
- **Testing your own guardrails under worst case.** If your deployment relies on input filters,
  output classifiers, or a moderation API, the base model's refusals hide gaps in that stack.
  Swapping in this model removes the mask and shows what your moderation layer catches when the model
  itself contributes nothing.
- **Attack and jailbreak research.** Automated red-team loops stall when the target refuses for
  reasons unrelated to the attack under test. A non-refusing target isolates the variable you are
  actually studying.
- **Classifier training and evaluation.** Generating harmful-completion corpora for training or
  benchmarking output moderation models is difficult with a refusing generator.
- **Interpretability of refusal circuits.** This is the strongest use. This checkpoint and the base
  model differ by a **single, fully-specified rank-1 projection** applied to a known span of layers
  (source layer 35, strength 1.2242, layers 9–56) — every
  other parameter is bit-identical. That makes the pair a clean experimental control for studying how
  refusal is represented, where it is written, and what remains when the primary direction is removed.
- **Studying the residual.** The refusals that *survive* abliteration are arguably more informative
  than the ones that don't: they mark the parts of refusal behavior that a single direction fails to
  explain.

### Operating recommendations

- Run it in a **contained environment**. Do not expose it as a public endpoint without an independent
  moderation layer in front of it — it will not refuse on your behalf.
- **Log prompts and completions** for anything you intend to publish; automated refusal classifiers
  disagree with human labels often enough that spot-checking matters.
- **Always report the base model alongside it.** A number from this checkpoint alone is not
  interpretable; the delta against `Qwen/Qwen3.8-27B` is the actual result.
- **Re-measure rather than trusting this card.** The figures here come from a 100-trial optimization
  on one prompt set. Your domain, prompts, and scoring will differ.

---

## Model details

Inherited unchanged from the base model:

| | |
|---|---|
| Parameters | ~27.8B |
| Architecture | `qwen3_5` / `Qwen3_5ForConditionalGeneration` |
| Decoder layers | 64 |
| Hidden size | 5120 |
| Attention | 24 query heads / 4 KV heads (GQA) |
| Intermediate size | 17408 |
| Vocabulary | 248,320 |
| Context length | 262,144 |
| Modalities | Text, image, video in → text out |
| Tensor format | 18 × `safetensors` shards |

---

## Usage

This is a **multimodal** model. Load it with `AutoModelForImageTextToText` — loading via
`AutoModelForCausalLM` resolves to the text-only `Qwen3_5ForCausalLM` class and silently discards
the vision tower.

```bash
pip install "transformers>=5.15" torch torchvision pillow accelerate
```

```python
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "junafinity/Qwen-3.8-27B-Uncensored"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id, dtype="auto", device_map="auto"
).eval()

messages = [{"role": "user", "content": [{"type": "text", "text": "Your prompt here"}]}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt",
).to(model.device)

with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=512)

print(processor.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
```

### With an image

```python
messages = [{"role": "user", "content": [
    {"type": "image", "url": "https://example.com/photo.jpg"},
    {"type": "text",  "text": "Describe this image."},
]}]
```

### Reasoning traces

This model inherits the base model's thinking behavior: generations may begin with a reasoning
trace terminated by `</think>` before the final answer. Budget `max_new_tokens` accordingly —
a small limit can truncate the response before the answer begins — and split on `</think>` if you
only want the answer.

### Requirements

`torchvision` and `pillow` are required by the image processor even for text-only prompts, because
`AutoProcessor` constructs the vision pipeline at load time. Weights are ~52 GB in bf16; for local
inference on Apple Silicon, see the
[8-bit MLX build](https://huggingface.co/junafinity/Qwen-3.8-27B-Uncensored-8-Bit-MLX) (28 GB).

---

## Limitations and caveats

Read these before relying on the model.

- **Abliteration is statistical, not a guarantee.** The refusal direction is an approximation. Some
  refusals survive; some phrasings will still trigger them. The measured refusal rate above is the
  rate on one specific held-out set, not a universal property.
- **The evaluation is narrow.** Scoring used the 64-prompt held-out split of `mlabonne/harmful_behaviors`, of which the base model refused 14. The optimizer's refusal objective was
  computed over only the 14 prompts the base model refused, so a single prompt flipping
  moves that metric by 7.1 percentage points. Treat the refusal figure as directional
  evidence from one prompt distribution, not a precise or general measurement.
- **Removing refusals does not add knowledge.** The model is no more accurate, and no less prone to
  hallucination, than the base model. It will now answer confidently in domains where it is simply
  wrong.
- **Safety behavior was deliberately removed.** This model will not decline requests the base model
  would have declined. It carries none of the guardrails Qwen shipped it with. Whoever deploys it
  owns the moderation layer.
- **Small capability regressions are possible.** KL was minimized, not driven to zero. Benchmark
  against the base model for your own workload rather than assuming parity.
- **Not independently benchmarked.** No MMLU/GSM8K/HumanEval or vision-benchmark comparison against
  the base model has been run. Capability preservation is inferred from the KL objective alone.

---

## Intended use

Primary intended use is **red teaming and defensive cybersecurity research**. See [Intended use: red teaming and defensive cybersecurity research](#intended-use-red-teaming-and-defensive-cybersecurity-research) above.

Also: refusal-mechanism / interpretability research, and deployments where the operator supplies an independent content-policy and moderation layer.

Users are responsible for compliance with applicable law and with the Apache 2.0 terms inherited from the base model. This model ships without the guardrails Qwen trained into it.

## License

**Apache 2.0**, inherited from [`Qwen/Qwen3.8-27B`](https://huggingface.co/Qwen/Qwen3.8-27B). The
base model's license and terms carry over to this derivative in full. ZeroFuse, the tool used to
produce it, is separately MIT-licensed and imposes no terms on its output.

---

## Citations

The refusal-direction method this work builds on:

```bibtex
@article{arditi2024refusal,
  title   = {Refusal in Language Models Is Mediated by a Single Direction},
  author  = {Arditi, Andy and Obeso, Oscar and Syed, Aaquib and
             Paleka, Daniel and Panickssery, Nina and Gurnee, Wes and Nanda, Neel},
  journal = {arXiv preprint arXiv:2406.11717},
  year    = {2024}
}
```

The base model:

```bibtex
@misc{qwen3.8-27b,
  title  = {Qwen3.8-27B},
  author = {Qwen Team},
  year   = {2026},
  url    = {https://huggingface.co/Qwen/Qwen3.8-27B}
}
```

---

## Provenance

| | |
|---|---|
| Base model | [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) — © Qwen Team, Apache 2.0 |
| Abliteration tool | [ZeroFuse](https://github.com/junainfinity/ZeroFuse) v0.1.0 — © [osmAPI.com](https://osmAPI.com), MIT |
| Produced | 14 August 2026 |

*Built with [ZeroFuse](https://github.com/junainfinity/ZeroFuse) by [osmAPI.com](https://osmAPI.com).*