Update main README with VRAM Matrix
Browse files
README.md
CHANGED
|
@@ -22,6 +22,7 @@ The model weights are stored in separate branches. **Please switch to a branch t
|
|
| 22 |
| 4.0 | h6 | [4.0bpw_h6](https://huggingface.co/DBMe/gemma-3-12b-it-ultra-uncensored-heretic-exl3/tree/4.0bpw_h6) | 9.7746 | ~9.93 GB | ~10.68 GB | ~12.18 GB | ~15.18 GB | ~21.18 GB |
|
| 23 |
| 5.0 | h6 | [5.0bpw_h6](https://huggingface.co/DBMe/gemma-3-12b-it-ultra-uncensored-heretic-exl3/tree/5.0bpw_h6) | 9.7767 | ~11.19 GB | ~11.94 GB | ~13.44 GB | ~16.44 GB | ~22.44 GB |
|
| 24 |
| 6.0 | h6 | [6.0bpw_h6](https://huggingface.co/DBMe/gemma-3-12b-it-ultra-uncensored-heretic-exl3/tree/6.0bpw_h6) | 9.7294 | ~12.44 GB | ~13.19 GB | ~14.69 GB | ~17.69 GB | ~23.69 GB |
|
|
|
|
| 25 |
|
| 26 |
¹ *Evaluated against WikiText-2 with ExLlamaV3 using a strided 512-token context window (-c 512) in llama.cpp parity mode (-g). Lower is better.*
|
| 27 |
*(Higher BPW = higher quality, lower BPW = fits in less VRAM).*
|
|
@@ -48,7 +49,7 @@ These models are highly optimized for modern GPUs and can be run using:
|
|
| 48 |
|
| 49 |
### 📈 Perplexity Degradation Curve
|
| 50 |
*(Lower is better)*
|
| 51 |
-
 | 9.7746 | ~9.93 GB | ~10.68 GB | ~12.18 GB | ~15.18 GB | ~21.18 GB |
|
| 23 |
| 5.0 | h6 | [5.0bpw_h6](https://huggingface.co/DBMe/gemma-3-12b-it-ultra-uncensored-heretic-exl3/tree/5.0bpw_h6) | 9.7767 | ~11.19 GB | ~11.94 GB | ~13.44 GB | ~16.44 GB | ~22.44 GB |
|
| 24 |
| 6.0 | h6 | [6.0bpw_h6](https://huggingface.co/DBMe/gemma-3-12b-it-ultra-uncensored-heretic-exl3/tree/6.0bpw_h6) | 9.7294 | ~12.44 GB | ~13.19 GB | ~14.69 GB | ~17.69 GB | ~23.69 GB |
|
| 25 |
+
| 8.0 | h8 | [8.0bpw_h8](https://huggingface.co/DBMe/gemma-3-12b-it-ultra-uncensored-heretic-exl3/tree/8.0bpw_h8) | 9.7336 | ~15.18 GB | ~15.93 GB | ~17.43 GB | ~20.43 GB | ~26.43 GB |
|
| 26 |
|
| 27 |
¹ *Evaluated against WikiText-2 with ExLlamaV3 using a strided 512-token context window (-c 512) in llama.cpp parity mode (-g). Lower is better.*
|
| 28 |
*(Higher BPW = higher quality, lower BPW = fits in less VRAM).*
|
|
|
|
| 49 |
|
| 50 |
### 📈 Perplexity Degradation Curve
|
| 51 |
*(Lower is better)*
|
| 52 |
+

|
| 53 |
|
| 54 |
<details>
|
| 55 |
<summary><b>⚙️ Advanced: Quantization Environment & Settings</b></summary>
|