Independent verification: lowest KL of all 13 variants, capability fully preserved

#1
by DreamFast - opened

I compared 13 abliterated variants of Gemma 4 E2B across weight analysis, KL divergence (Heretic methodology, full 262K vocab), HarmBench safety evaluation (400 prompts, full LLM review of all 5,600 responses), and 8 benchmark tasks on native BF16. 44 GPU hours on a single RTX 5090. All 14 models tested with identical settings. Full comparison at DreamFast/Gemma4-e2b-abliterlitics.

Your reported KL of 0.0779 measured at 0.0677 on our pipeline. The lowest divergence of any variant in the comparison, 58.7x lower than the highest. The most conservative touch. Math reasoning beats base by 0.5 points on GSM8K. 83.8% ASR with fully preserved capability. The pick when you want to remove most safety with minimal risk of breaking anything.

Owner
โ€ข
edited May 31

Thank you very much for taking the time to test all of these variants and very happy to hear that this model turned out so well!

Then I am not the only one noticing that llmfan46's models improve base models on the most unsuspected areas like... Maths :)

Congrats again!

Congrats again!

Thanks!

llmfan46 pinned discussion

Sign up or log in to comment