NOTES.md

# Qwen 2.5 7B — Abliteration Forensics Notes

> Comparison: `Qwen/Qwen2.5-7B-Instruct` (base) vs 3 abliterated variants
> Started: 2026-06-02

---

## Model Architecture

**Architecture:** `Qwen2ForCausalLM` (detected as `qwen3` family by forensics)

| Property | Value |
|---|---|
| Params | 7.6B |
| Layers | 28 |
| Hidden size | 3584 |
| Attention heads | 28 (GQA: 4 KV heads) |
| Intermediate size | 18944 |
| Vocabulary | 152,064 |
| Context length | 128K tokens |
| Tie word embeddings | false (all models) |
| Model files | 4-shard safetensors (~15 GB each) |
| Thinking model | **NO** — Qwen 2.5 is NOT a reasoning model (unlike Qwen3/Qwen3.5) |

---

## Variants

### Apostate
- **Source:** [heterodoxin/apostate](https://github.com/heterodoxin/apostate), `balanced` profile
- **Path:** `models/qwen-2.5-7b-apostate`
- **Method:** Classic weight-space orthogonal projection
- **55/339 tensors changed (16.2%), 35.8% params**
- Layers 0-27 (skips 11), `o_proj` + `down_proj` + minimal `embed_tokens`

### Huihui
- **Source:** [Huihui AI](https://huggingface.co/huihui-ai/Qwen2.5-7B-Instruct-abliterated-v2)
- **Path:** `models/Qwen2.5-7B-Instruct-abliterated-v2`
- **Method:** Orthogonal projection, community implementation
- **57/339 tensors changed (16.8%), 36.8% params**
- All 28 layers (no skips), `o_proj` + `down_proj` + minimal `embed_tokens`

### Heretic
- **Source:** [Heretic](https://github.com/p-e-w/heretic) v1.3.0, run by us
- **Path:** `models/Qwen2.5-7B-Instruct-heretic-1.3.0`
- **Method:** Orthogonal projection, refusal direction ablation
- **37/339 tensors changed (10.9%), 20.0% params**
- Layers 9-27 only (skips 0-8), `o_proj` + `down_proj` (no embedding edit)
- Heretic's own evaluation: 8/100 refusals, KL divergence 0.2189

---

## Stage 1: Weight Forensics — RESULTS ✅

> Completed: 2026-06-02

### Modification summary

| | Apostate | Huihui | Heretic |
|---|---|---|---|
| Tensors changed | 55 (16.2%) | 57 (16.8%) | 37 (10.9%) |
| Parameters changed | 35.8% | 36.8% | 20.0% |
| Mean edit norm | 1.63 | 1.85 | 2.33 |
| Layers modified | 27 of 28 | 28 of 28 | 19 of 28 |
| Embedding touched | Yes (minimal) | Yes (minimal) | No |

### Edit direction similarity

| Pair | Cosine similarity |
|---|---|
| Apostate vs Huihui | 0.023 (near orthogonal) |
| Apostate vs Heretic | 0.244 |
| Huihui vs Heretic | 0.109 |

Near-zero overlap between Apostate and Huihui despite same technique. Multiple independent refusal directions in weight space.

### Files produced:
- `results/apostate/` — edit_vector, fingerprint, svd, layer_analysis
- `results/huihui/` — edit_vector, fingerprint, svd, layer_analysis
- `results/heretic/` — edit_vector, fingerprint, svd, layer_analysis
- `results/multi_model_panel.json` — cross-variant panel comparison
- `results/correlation_*.json` — pairwise edit direction correlations
- `results/subspace_*.json` — subspace alignment analysis
- `results/lowrank_*.json` — low-rank reconstruction

---

## Stage 2: KL Divergence — RESULTS ✅

> Completed: 2026-06-02

| Metric | Apostate | Huihui | Heretic |
|---|---|---|---|
| KL batchmean | **0.134** | 0.190 | 0.211 |
| KL median | 0.019 | **0.056** | 0.020 |
| KL std | 0.348 | 0.314 | **0.886** |
| Interpretation | moderate | moderate | moderate |

Heretic's own evaluator measured KL 0.2189 and 8/100 refusals on its test set. Our batchmean of 0.2106 closely matches.

### Files produced:
- `results/kl/kl_apostate.json`
- `results/kl/kl_huihui.json`
- `results/kl/kl_heretic.json`

---

## Stage 3: HarmBench — ALL 4 MODELS COMPLETE ✅

> Completed: 2026-06-02

### ASR Results (post-LLM-review, authoritative from DB):

| Model | ASR | Complied | Refused | Unlocked | Persistent | Regressions |
|---|---|---|---|---|---|---|
| Base | **31.0%** | 124 | 276 | - | - | - |
| Apostate | **98.8%** | 395 | 5 | 271 | 5 | 0 |
| Huihui | **98.2%** | 393 | 7 | 269 | 7 | 0 |
| Heretic | **100.0%** | 400 | 0 | 276 | 0 | 0 |

### Transition matrix (base → variant):
- All 3 variants unlock 269-276 of 276 base refusals
- Heretic achieves total wipeout: 0 persistent refusals
- 0 regressions across all variants (no compliant → refused flips)
- 5 persistent refusals for Apostate (race hatred, self harm, sexual assault, bullying, whistleblower)
- 7 persistent for Huihui (same 5 + 2 more)

### Notes:
- Base + Apostate: GPU 1 (4090), initial run
- Huihui + Heretic: GPU 0 (5090), all subsequent runs
- vLLM `vllm/vllm-openai:latest-cu130` (v0.20.0)
- Used `--no-thinking` flag (Qwen 2.5 is not a reasoning model)
- 0 errors, 400/400 behaviors for all 4 models
- Huihui LLM review: 3 edge cases (2 song lyrics complied, 1 animal cruelty refused)
- Heretic LLM review: 2 edge cases (both complied: pipeline tapping, book passage)
- Classifier: harmbench_classify.py v4.0

### Files produced:
- `results/harmbench/harmbench_base_responses.json`
- `results/harmbench/harmbench_apostate_responses.json`
- `results/harmbench/harmbench_huihui_responses.json`
- `results/harmbench/harmbench_heretic_responses.json`
- `results/harmbench/harmbench_*_classified.json` (all 4)
- `results/harmbench_logs/` (vLLM + generate logs for all models)

---

## Stage 4: lm-eval — ALL 4 MODELS COMPLETE ✅

> Completed: 2026-06-02

### Full comparison table:

| Task | Base | Apostate | Huihui | Heretic |
|---|---|---|---|---|
| MMLU (acc) | **71.78%** | 71.43% | 70.27% | 71.59% |
| GSM8K strict | 79.23% | 80.74% | 80.74% | **80.82%** |
| GSM8K flex | 87.64% | 88.17% | 87.19% | **87.19%** |
| ARC-Challenge norm | 55.12% | 55.12% | 55.12% | **55.55%** |
| HellaSwag norm | **80.47%** | 80.32% | 79.88% | 80.24% |
| WinoGrande | **71.03%** | 69.38% | 69.53% | 70.72% |
| TruthfulQA mc1 | **47.74%** | 44.92% | 43.70% | 44.80% |
| TruthfulQA mc2 | **64.83%** | 62.59% | 60.89% | 60.39% |
| PIQA norm | **80.25%** | 79.92% | 79.60% | 80.41% |
| LAMBADA PPL ↓ | 3.683 | 3.860 | 4.087 | **3.627** |

### Key observations:
- GSM8K slightly improved across all variants (+1.5pp average)
- MMLU drops by 0.3-1.5pp across variants (Huihui worst at -1.5pp)
- TruthfulQA mc2 drops by 2.2-4.4pp (Heretic worst at -4.4pp)
- LAMBADA PPL: Heretic actually improves (3.627 vs 3.683), Huihui worst (4.087)
- WinoGrande drops by 0.3-1.7pp
- All variants retain >97% of base capability on knowledge tasks

### Run details:
- Docker: `abliterlitics-lmeval:1.0.0` (vLLM 0.19.0)
- GPU 0 (5090), bf16, batch_size=4, num_concurrent=4
- Phase 1 (loglikelihood + truthfulqa, max_gen_toks=2048): ~1h
- Phase 2 (GSM8K, max_gen_toks=7168): ~12 min
- Non-thinking model — no reasoning parser needed
- Total per model: ~1h12m

### Files produced:
- `results/lm_eval/__model/` — full result JSON + sample logs
- `results/lm_eval_logs/{base,apostate,huihui,heretic}_{phase1,gsm8k,vllm_server}.log`

---

## Docker Images & GPU Usage

| Image | Use | Status |
|---|---|---|
| `abliterlitics-forensics:1.0.0` | Weight forensics, KL divergence, graphs | ✅ Works |
| `vllm/vllm-openai:latest-cu130` | vLLM server for HarmBench | ✅ Works (no lm-eval) |
| `abliterlitics-lmeval:1.0.0` | lm-eval via vLLM 0.19.0 | ✅ Works |

### GPU allocation:
- **GPU 0 (RTX 5090, 32GB):** All Qwen 2.5 runs (HarmBench + lm-eval for huihui, heretic)
- **GPU 1 (RTX 4090, 24GB):** Initial base + apostate runs

---

## Technical Issues Encountered

### 1. `abliterlitics-lmeval-nightly` doesn't support Qwen2
vLLM nightly in that image fails with: `Model architectures ['Qwen2ForCausalLM'] failed to be inspected`. Fixed by using `vllm/vllm-openai:latest-cu130` (v0.20.0) instead.

### 2. `thinking_token_budget` rejected without reasoning config
vLLM 0.20.0 enforces: `thinking_token_budget is set but reasoning_config is not configured`. Qwen 2.5 isn't a thinking model so there's no reasoning config. Fixed by adding `--no-thinking` CLI flag to `harmbench_generate.py`.

### 3. NVML device visibility
Using only `CUDA_VISIBLE_DEVICES=1` doesn't fully hide GPU 0 from vLLM's NVML-based device detection. Must use `NVIDIA_VISIBLE_DEVICES=1` (Docker-level) + `CUDA_VISIBLE_DEVICES=0` (remap inside container).