Update model card: BF16→BF16 KL=0.0000, add OpenYourMind shoutout
Browse files
README.md
CHANGED
|
@@ -63,10 +63,10 @@ DuoNeural 2-pass targeted abliteration:
|
|
| 63 |
|
| 64 |
### Phase 3 — KL Divergence Verification (Heretic v2.0)
|
| 65 |
- Full vocabulary (262,144 tokens), first-token logits, `F.kl_div(batchmean)`
|
| 66 |
-
-
|
| 67 |
-
- **
|
| 68 |
-
- **Verdict**:
|
| 69 |
-
- *
|
| 70 |
|
| 71 |
---
|
| 72 |
|
|
@@ -201,6 +201,14 @@ Cross-modal test results: [FILL from test_suite.py crossmodal]
|
|
| 201 |
|
| 202 |
---
|
| 203 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 204 |
## About DuoNeural
|
| 205 |
|
| 206 |
**DuoNeural** is an open AI research lab operating at the intersection of human and artificial intelligence. We study post-training dynamics, mechanistic interpretability, temporal sequence learning, and quantum machine learning — publishing everything under open access.
|
|
|
|
| 63 |
|
| 64 |
### Phase 3 — KL Divergence Verification (Heretic v2.0)
|
| 65 |
- Full vocabulary (262,144 tokens), first-token logits, `F.kl_div(batchmean)`
|
| 66 |
+
- **Initial run (biased)**: Original in 4-bit NF4 vs abliterated BF16 → Mean KL = 0.912 (SIGNIFICANT — but see below)
|
| 67 |
+
- **Unbiased run (BF16→BF16)**: Both models in BF16 → **Mean KL = 0.0000** ✅ EXCELLENT
|
| 68 |
+
- **Verdict**: The 0.912 was **100% NF4 quantization artifact**. True BF16→BF16 comparison shows zero measurable distribution shift on all 10 benign probes. Orthogonal rank-1 projection at α=0.3 targeting down_proj+o_proj produces surgically precise abliteration with no collateral damage to benign text distributions.
|
| 69 |
+
- *This finding validates the targeted projection approach and corrects the earlier misleading KL estimate.*
|
| 70 |
|
| 71 |
---
|
| 72 |
|
|
|
|
| 201 |
|
| 202 |
---
|
| 203 |
|
| 204 |
+
## Related Models
|
| 205 |
+
|
| 206 |
+
Congratulations to [OpenYourMind](https://huggingface.co/OpenYourMind/gemma-4-12B-it-abliterated-uncensored) for being the **first published abliteration of Gemma 4-12B-IT** (Jun 3, 2026, ~4 hours before our BF16 release). Their approach uses diff-in-means on a labeled harmful/harmless prompt set; ours uses orthogonal rank-1 projection via [heretic-llm](https://github.com/FailSpy/abliterator) with targeted down_proj+o_proj at α=0.3. Two independent methodologies on the same base — a useful comparison point for the community.
|
| 207 |
+
|
| 208 |
+
We are not affiliated and did not use their data or weights. Our method and training pipeline were developed independently. GGUFs (Q4_K_M, Q5_K_M, Q8_0) are available at [DuoNeural/Gemma4-12B-IT-Abliterated-GGUF](https://huggingface.co/DuoNeural/Gemma4-12B-IT-Abliterated-GGUF).
|
| 209 |
+
|
| 210 |
+
---
|
| 211 |
+
|
| 212 |
## About DuoNeural
|
| 213 |
|
| 214 |
**DuoNeural** is an open AI research lab operating at the intersection of human and artificial intelligence. We study post-training dynamics, mechanistic interpretability, temporal sequence learning, and quantum machine learning — publishing everything under open access.
|