Update main README with VRAM Matrix
Browse files
README.md
CHANGED
|
@@ -27,6 +27,7 @@ The model weights are stored in separate branches. **Please switch to a branch t
|
|
| 27 |
| 4.0 | h6 | [4.0bpw_h6](https://huggingface.co/DBMe/gemma-4-31B-it-qat-q4_0-unquantized-uncensored-heretic-exl3-mul1/tree/4.0bpw_h6) | 2097.6864 | N/A | N/A | N/A | N/A | N/A |
|
| 28 |
| 5.0 | h6 | [5.0bpw_h6](https://huggingface.co/DBMe/gemma-4-31B-it-qat-q4_0-unquantized-uncensored-heretic-exl3-mul1/tree/5.0bpw_h6) | 1995.7812 | N/A | N/A | N/A | N/A | N/A |
|
| 29 |
| 6.0 | h6 | [6.0bpw_h6](https://huggingface.co/DBMe/gemma-4-31B-it-qat-q4_0-unquantized-uncensored-heretic-exl3-mul1/tree/6.0bpw_h6) | 1995.3135 | N/A | N/A | N/A | N/A | N/A |
|
|
|
|
| 30 |
|
| 31 |
¹ *Evaluated against WikiText-2 with ExLlamaV3 using a strided 512-token context window (-c 512) in llama.cpp parity mode (-g). Lower is better.*
|
| 32 |
|
|
@@ -40,10 +41,10 @@ Ensure you have the CLI installed:
|
|
| 40 |
pip install -U "huggingface_hub[cli]"
|
| 41 |
```
|
| 42 |
|
| 43 |
-
Download a specific branch (e.g., `
|
| 44 |
```bash
|
| 45 |
-
# Example: Downloading the
|
| 46 |
-
huggingface-cli download DBMe/gemma-4-31B-it-qat-q4_0-unquantized-uncensored-heretic-exl3-mul1 --revision
|
| 47 |
```
|
| 48 |
|
| 49 |
## 💻 Supported Engines
|
|
@@ -53,7 +54,7 @@ These models are highly optimized for modern GPUs and can be run using:
|
|
| 53 |
* **[ExLlamaV3 (Native)](https://github.com/turboderp-org/exllamav3):** Python library for custom integration.
|
| 54 |
### 📈 Perplexity Degradation Curve
|
| 55 |
*(Lower is better)*
|
| 56 |
-
 | 2097.6864 | N/A | N/A | N/A | N/A | N/A |
|
| 28 |
| 5.0 | h6 | [5.0bpw_h6](https://huggingface.co/DBMe/gemma-4-31B-it-qat-q4_0-unquantized-uncensored-heretic-exl3-mul1/tree/5.0bpw_h6) | 1995.7812 | N/A | N/A | N/A | N/A | N/A |
|
| 29 |
| 6.0 | h6 | [6.0bpw_h6](https://huggingface.co/DBMe/gemma-4-31B-it-qat-q4_0-unquantized-uncensored-heretic-exl3-mul1/tree/6.0bpw_h6) | 1995.3135 | N/A | N/A | N/A | N/A | N/A |
|
| 30 |
+
| 8.0 | h8 | [8.0bpw_h8](https://huggingface.co/DBMe/gemma-4-31B-it-qat-q4_0-unquantized-uncensored-heretic-exl3-mul1/tree/8.0bpw_h8) | 1985.1562 | N/A | N/A | N/A | N/A | N/A |
|
| 31 |
|
| 32 |
¹ *Evaluated against WikiText-2 with ExLlamaV3 using a strided 512-token context window (-c 512) in llama.cpp parity mode (-g). Lower is better.*
|
| 33 |
|
|
|
|
| 41 |
pip install -U "huggingface_hub[cli]"
|
| 42 |
```
|
| 43 |
|
| 44 |
+
Download a specific branch (e.g., `5.0bpw_h6`):
|
| 45 |
```bash
|
| 46 |
+
# Example: Downloading the 5.0bpw_h6 branch
|
| 47 |
+
huggingface-cli download DBMe/gemma-4-31B-it-qat-q4_0-unquantized-uncensored-heretic-exl3-mul1 --revision 5.0bpw_h6 --local-dir gemma-4-31B-it-qat-q4_0-unquantized-uncensored-heretic-exl3-mul1-5.0bpw_h6
|
| 48 |
```
|
| 49 |
|
| 50 |
## 💻 Supported Engines
|
|
|
|
| 54 |
* **[ExLlamaV3 (Native)](https://github.com/turboderp-org/exllamav3):** Python library for custom integration.
|
| 55 |
### 📈 Perplexity Degradation Curve
|
| 56 |
*(Lower is better)*
|
| 57 |
+

|
| 58 |
|
| 59 |
|
| 60 |
<details>
|