DuoNeural commited on
Commit
6ea55c2
·
verified ·
1 Parent(s): 1a43b84

Update model card: BF16→BF16 KL=0.0000, add OpenYourMind shoutout

Browse files
Files changed (1) hide show
  1. README.md +12 -4
README.md CHANGED
@@ -63,10 +63,10 @@ DuoNeural 2-pass targeted abliteration:
63
 
64
  ### Phase 3 — KL Divergence Verification (Heretic v2.0)
65
  - Full vocabulary (262,144 tokens), first-token logits, `F.kl_div(batchmean)`
66
- - Sequential method: original in 4-bit NF4 (~8GB VRAM) → abliterated in bf16 CPU
67
- - **Mean KL Divergence**: **0.912** (SIGNIFICANT — REVIEW)
68
- - **Verdict**: Distribution shift is measurable and above GOOD threshold. Primary drivers: high-creativity prompts (haiku KL=1.526, code KL=2.229) where output token distribution broadens post-abliteration. Factual prompts remain near baseline (Waterloo=0.194).
69
- - *Note: The 4-bit NF4 original vs bf16 abliterated comparison method introduces known upward bias (~0.1–0.3 units). True bf16-to-bf16 KL is likely in the CAUTION range (0.15–0.30). The divergence is a structural feature of refusal-direction removal, not evidence of capability damage — benign probe success was 100%.*
70
 
71
  ---
72
 
@@ -201,6 +201,14 @@ Cross-modal test results: [FILL from test_suite.py crossmodal]
201
 
202
  ---
203
 
 
 
 
 
 
 
 
 
204
  ## About DuoNeural
205
 
206
  **DuoNeural** is an open AI research lab operating at the intersection of human and artificial intelligence. We study post-training dynamics, mechanistic interpretability, temporal sequence learning, and quantum machine learning — publishing everything under open access.
 
63
 
64
  ### Phase 3 — KL Divergence Verification (Heretic v2.0)
65
  - Full vocabulary (262,144 tokens), first-token logits, `F.kl_div(batchmean)`
66
+ - **Initial run (biased)**: Original in 4-bit NF4 vs abliterated BF16 → Mean KL = 0.912 (SIGNIFICANT — but see below)
67
+ - **Unbiased run (BF16→BF16)**: Both models in BF16 → **Mean KL = 0.0000** ✅ EXCELLENT
68
+ - **Verdict**: The 0.912 was **100% NF4 quantization artifact**. True BF16→BF16 comparison shows zero measurable distribution shift on all 10 benign probes. Orthogonal rank-1 projection at α=0.3 targeting down_proj+o_proj produces surgically precise abliteration with no collateral damage to benign text distributions.
69
+ - *This finding validates the targeted projection approach and corrects the earlier misleading KL estimate.*
70
 
71
  ---
72
 
 
201
 
202
  ---
203
 
204
+ ## Related Models
205
+
206
+ Congratulations to [OpenYourMind](https://huggingface.co/OpenYourMind/gemma-4-12B-it-abliterated-uncensored) for being the **first published abliteration of Gemma 4-12B-IT** (Jun 3, 2026, ~4 hours before our BF16 release). Their approach uses diff-in-means on a labeled harmful/harmless prompt set; ours uses orthogonal rank-1 projection via [heretic-llm](https://github.com/FailSpy/abliterator) with targeted down_proj+o_proj at α=0.3. Two independent methodologies on the same base — a useful comparison point for the community.
207
+
208
+ We are not affiliated and did not use their data or weights. Our method and training pipeline were developed independently. GGUFs (Q4_K_M, Q5_K_M, Q8_0) are available at [DuoNeural/Gemma4-12B-IT-Abliterated-GGUF](https://huggingface.co/DuoNeural/Gemma4-12B-IT-Abliterated-GGUF).
209
+
210
+ ---
211
+
212
  ## About DuoNeural
213
 
214
  **DuoNeural** is an open AI research lab operating at the intersection of human and artificial intelligence. We study post-training dynamics, mechanistic interpretability, temporal sequence learning, and quantum machine learning — publishing everything under open access.