Upload README.md with huggingface_hub

README.md

@@ -1,381 +1,42 @@
 ---
-license: apache-2.0
-base_model:
-- google/gemma-4-31B-it
-- zerofata/G4-MeroMero-31B
-- Jackrong/Gemopus-4-31B-it
-- ConicCat/Gemma4-GarnetV2-31B
-- AuriAetherwiing/G4-31B-Musica-v1
-- Blazed-Forge/Gemma-4-Gemsicle-31B
-- trohrbaugh/gemma-4-31b-it-heretic-ara
+base_model: Nimbz/Gemma-4-Gembrain-31B
+library_name: transformers
 tags:
-- merge
-- mergekit
-- reasoning
-- non-reasoning
-- creative writing
-- roleplay
-- uncensored
-- 31B
-- gemma-4
-new_version: Nimbz/Gemma-4-Gembrain-X-31B
+- quantized
+- w8a16
+- fp8
 ---
 
-<style>
-.gembrain-wrapper {
-background-color: #050505;
-color: #e2e8f0;
-font-family: 'Segoe UI', Roboto, Helvetica, Arial, sans-serif;
-padding: 30px;
-border-radius: 20px;
-border: 1px solid #1a1a1a;
-line-height: 1.6;
-}
+# Gemma-4-Gembrain-31B-W8A16-FP8
 
-.neon-header { text-align: center; padding-bottom: 20px; }
+Quantized version of [Nimbz/Gemma-4-Gembrain-31B](https://huggingface.co/Nimbz/Gemma-4-Gembrain-31B).
 
-.neon-title {
-font-size: 3.5rem;
-font-weight: 800;
-margin: 0;
-background: linear-gradient(90deg, #00f2ff, #ff00ff);
--webkit-background-clip: text;
--webkit-text-fill-color: transparent;
-filter: drop-shadow(0 0 10px rgba(0, 242, 255, 0.4));
-letter-spacing: -2px;
-}
+## Format
 
-.crystal-card {
-background: rgba(255, 255, 255, 0.03);
-border-left: 4px solid #00f2ff;
-padding: 20px;
-margin: 20px 0;
-border-radius: 0 12px 12px 0;
-box-shadow: 10px 0 30px rgba(0, 242, 255, 0.05);
-}
+Offline-quantized **W8A16 FP8** in the
+[compressed-tensors](https://github.com/neuralmagic/compressed-tensors)
+`float-quantized` format: weights in `float8_e4m3fn` with per-output-channel
+symmetric scales, activations kept in bf16/fp16.
 
-.magenta-card {
-border-left-color: #ff00ff;
-box-shadow: 10px 0 30px rgba(255, 0, 255, 0.05);
-}
+## How it was quantized
 
-.gold-card {
-border-left-color: #fbbf24;
-box-shadow: 10px 0 30px rgba(251, 191, 36, 0.05);
-}
+For each linear layer, every output row gets its own scale starting from
+`amax / 448`, refined by an MSE clip search over ~9 clip fractions
+(0.8–1.0× amax) picking the lowest-error scale per row. Weights are then
+quantized `q = e4m3(w / scale)` with round-to-nearest and saturation. This
+per-channel + clipping scheme gives better SNR than vLLM's online per-tensor
+`--quantization fp8` path.
 
-.stat-grid {
-display: grid;
-grid-template-columns: repeat(auto-fit, minmax(150px, 1fr));
-gap: 15px;
-margin: 20px 0;
-}
+Only 2D linear projection weights are quantized (attention q/k/v/o, MLP
+gate/up/down). Embeddings, norms, lm_head, routers/experts, and the vision
+tower stay in bf16 and are listed in the checkpoint's `ignore` list, so vLLM
+leaves them untouched.
 
-.stat-item {
-text-align: center;
-padding: 15px;
-background: #0f172a;
-border: 1px solid #1e293b;
-border-radius: 15px;
-}
+## Supported vLLM platforms
 
-.stat-value {
-display: block;
-font-size: 1.2rem;
-font-weight: bold;
-color: #00f2ff;
-}
-
-.thought-block {
-background: #000;
-border: 1px dashed #334155;
-padding: 15px;
-font-family: 'Courier New', Courier, monospace;
-color: #22c55e;
-margin-top: 10px;
-}
-
-/* 5-Phase Layout */
-.phase-grid {
-display: flex;
-flex-wrap: wrap;
-gap: 15px;
-justify-content: center;
-margin: 20px 0;
-}
-
-.phase-box {
-flex: 1 1 calc(33% - 15px);
-min-width: 180px;
-background: #111;
-padding: 15px;
-border-radius: 10px;
-border: 1px solid #334155;
-}
-
-.tag-pill {
-display: inline-block;
-padding: 4px 12px;
-border-radius: 50px;
-font-size: 12px;
-background: rgba(0, 242, 255, 0.1);
-color: #00f2ff;
-border: 1px solid rgba(0, 242, 255, 0.3);
-margin: 5px;
-}
-</style>
-
-<div class="gembrain-wrapper">
-
-<div class="neon-header">
-<img src="insane-in-the-gembrain-31b-title.png" width="100%" style="border-radius: 12px; border: 1px solid #333; margin-bottom: 20px;">
-<h1 class="neon-title"> 💎 GEMBRAIN-31B 🧠 </h1>
-<div style="color: #94a3b8; text-transform: uppercase; letter-spacing: 5px; font-size: 0.85rem; margin-top: 10px;">
-INSANE IN THE GEMBRAIN
-</div>
-</div>
-
-<div class="stat-grid">
-<div class="stat-item"><span class="stat-value">ADHERENCE</span><span style="font-size: 0.75rem; color: #64748b;">IMPROVED</span></div>
-<div class="stat-item"><span class="stat-value" style="color: #ff00ff;">SWIPE VARIETY</span><span style="font-size: 0.75rem; color: #64748b;">INCREASED</span></div>
-<div class="stat-item"><span class="stat-value">CREATIVE PROSE</span><span style="font-size: 0.75rem; color: #64748b;">PRESERVED</span></div>
-</div>
-
-<div class="crystal-card">
-<h3 style="color: #00f2ff; margin-top: 0;">🧠 About The Model</h3>
-<p style="margin-bottom: 0;">Gembrain-31B is a synthesis of several models, including Gemsicle-31B as important ingredient. The goal of this release was to stabilize and improve the initial Gemsicle-31B, but also to enhance its logical and lateral thinking, both with and without reasoning.</p>
-
-***
-
-<p style="margin-bottom: 0;">It's build to create the most unhinged narratives and construct image prompts about anything accordingly to a given structure with high precision.</p>
-
-***
-
-<p style="margin-bottom: 0;">Expect creative swipe variance, unique and non-robotic prose, and sharper instruction adherence.</p>
-</div>
-
-<div style="display: flex; gap: 20px; flex-wrap: wrap; margin-top: 30px;">
-<div style="flex: 1; min-width: 250px;">
-<h2 style="color: #00f2ff; border-bottom: 1px solid #334155; padding-bottom: 5px;">🎚️ Samplers</h2>
-<table style="width: 100%; border-collapse: collapse; font-size: 0.9rem;">
-
-<tr style="border-bottom: 1px solid #1e293b;">
-<td style="padding: 8px 5; color: #94a3b8;">Temperature</td>
-<td style="padding: 8px 5; text-align: right; color: #00f2ff;">1.0</td>
-</tr>
-
-<tr style="border-bottom: 1px solid #1e293b;">
-<td style="padding: 8px 5; color: #94a3b8;">Top-K</td>
-<td style="padding: 8px 5; text-align: right; color: #00f2ff;">0</td>
-</tr>
-
-<tr style="border-bottom: 1px solid #1e293b;">
-<td style="padding: 8px 5; color: #94a3b8;">Top-P</td>
-<td style="padding: 8px 5; text-align: right; color: #00f2ff;">0.95</td>
-</tr>
-
-<tr style="border-bottom: 1px solid #1e293b;">
-<td style="padding: 8px 5; color: #94a3b8;">Min-P</td>
-<td style="padding: 8px 5; text-align: right; color: #00f2ff;">0.03</td>
-</tr>
-
-<tr style="border-bottom: 1px solid #1e293b;">
-<td style="padding: 8px 5; color: #94a3b8;">DRY Multiplier</td>
-<td style="padding: 8px 5; text-align: right; color: #00f2ff;">0.8</td>
-</tr>
-<tr style="border-bottom: 1px solid #1e293b;">
-<td style="padding: 8px 5; color: #94a3b8;">DRY Base</td>
-<td style="padding: 8px 5; text-align: right; color: #00f2ff;">1.75</td>
-</tr>
-<tr style="border-bottom: 1px solid #1e293b;">
-<td style="padding: 8px 5; color: #94a3b8;">DRY Allowed Length</td>
-<td style="padding: 8px 5; text-align: right; color: #00f2ff;">10</td>
-</tr>
-
-<tr>
-<td style="padding: 8px 5; color: #94a3b8;">Optional: Adaptive-P Target</td>
-<td style="padding: 8px 5; text-align: right; color: #00f2ff;">0.6</td>
-</tr>
-<tr>
-<td style="padding: 8px 5; color: #94a3b8;">Optional: Adaptive-P Decay</td>
-<td style="padding: 8px 5; text-align: right; color: #00f2ff;">0.5</td>
-</tr>
-
-</table>
-</div>
-
-<div class="crystal-card magenta-card" style="flex: 1; min-width: 250px; margin: 0; border-radius: 12px;">
-<h3 style="color: #ff00ff; margin-top: 0;">🫟 GGUF Quants</h3>
-
-| Quant | Size | Download Link |
-| :--- | :--- | :--- | 
-| **Q4_K_S** | `17.8 GB` | [Click](https://huggingface.co/Nimbz/Gemma-4-Gembrain-31B-GGUF/blob/main/Gembrain-31B-Q4_K_S.gguf) | 
-| **Q4_K_M** | `18.7 GB` | [Click](https://huggingface.co/Nimbz/Gemma-4-Gembrain-31B-GGUF/blob/main/Gembrain-31B-Q4_K_M.gguf) | 
-| **Q5_K_S** | `21.3 GB` | [Click](https://huggingface.co/Nimbz/Gemma-4-Gembrain-31B-GGUF/blob/main/Gembrain-31B-Q5_K_S.gguf) | 
-| **Q5_K_M** | `21.8 GB` | [Click](https://huggingface.co/Nimbz/Gemma-4-Gembrain-31B-GGUF/blob/main/Gembrain-31B-Q5_K_M.gguf) | 
-| **Q6_K** | `25.2 GB` | [Click](https://huggingface.co/Nimbz/Gemma-4-Gembrain-31B-GGUF/blob/main/Gembrain-31B-Q6_K.gguf) | 
-| **Q8_0** | `32.6 GB` | [Click](https://huggingface.co/Nimbz/Gemma-4-Gembrain-31B-GGUF/blob/main/Gemsicle-31B-Q8_0.gguf) | 
-
-</ul>
-</div>
-</div>
-
-<div class="crystal-card magenta-card">
-<h3 style="color: #ff00ff; margin-top: 0;">🔮 Prompt Format</h3>
-
-[**Please refer to the original google/gemma-4-31b-it for the correct chat template.**](https://huggingface.co/google/gemma-4-31B-it#2-thinking-mode-configuration)
-
-Let your frontend handle the chat template if possible (e.g., Chat Completion in SillyTavern).
-
-**For Reasoning:** Add `<|think|>` at the very beginning of the system prompt. Thinking happens between `<|channel>thought\n` and
-`<channel|>` tags.
-
-```text
-<|turn>system
-<|think|>
-You are a helpful assistant<turn|>
-<|turn>user
-Hello<turn|>
-<|turn>model
-Hi there<turn|>
-<|turn>user
-How are you?<turn|>
-<|turn>model
-```
-</div>
-
-<h2 style="color: #e2e8f0; border-bottom: 1px solid #334155; padding-bottom: 5px; margin-top: 30px;">🧪 Merge Details</h2>
-<p style="font-size: 0.9rem; color: #94a3b8;">This model was systematically created through a five-stage process of priming models for their given purpose and merging the results:</p>
-
-<div class="phase-grid">
-<div class="phase-box" style="border-color: #00f2ff;">
-<h4 style="color: #00f2ff; margin: 0 0 5px 0;">Phase 01: breadcrumbs_ties</h4>
-<p style="font-size: 0.8rem; margin: 0; color: #cbd5e1;">Gemopus X MeroMero
-<details>
-<summary></summary>
-
-```
-models:
-  - model: ./G4-MeroMero-31B
-  - model: ./G4-Gemopus-4-31B-it
-merge_method: breadcrumbs_ties
-base_model: ./G4-31B-it
-parameters:
-  density: 0.85
-  weight: 0.5
-  int8_mask: true
-dtype: bfloat16
-```
-
-</details></p>
-</div>
-<div class="phase-box" style="border-color: #00f2ff;">
-<h4 style="color: #00f2ff; margin: 0 0 5px 0;">Phase 02: slerp</h4>
-<p style="font-size: 0.8rem; margin: 0; color: #cbd5e1;">GarnetV2 X Musica-v1
-<details>
-<summary></summary>
-
-```
-models:
-  - model: ./G4-Gemma4-GarnetV2-31B
-  - model: ./G4-31B-Musica-v1
-merge_method: slerp
-base_model: ./G4-Gemma4-GarnetV2-31B
-parameters:
-  t:
-    - value: 0.6
-dtype: bfloat16
-```
-
-</details></p>
-</div>
-</div>
-
-<div class="phase-grid">
-<div class="phase-box" style="border-color: #ff00ff;">
-<h4 style="color: #ff00ff; margin: 0 0 5px 0;">Phase 03: della_linear</h4>
-<p style="font-size: 0.8rem; margin: 0; color: #cbd5e1;">Gemsicle X Gemma-4-31B-it-heretic-ara
-<details>
-<summary></summary>
-
-```
-models:
-  - model: ./Gemsicle-31B
-    parameters:
-    weight: 1.0
-  - model: ./G4-gemma-4-31b-it-heretic-ara
-    parameters:
-      weight: 0.75
-      density: 0.65
-merge_method: della_linear
-base_model: ./G4-31B-it
-parameters:
-  weight: 1.0
-  normalize: false
-  epsilon: 0.05
-  lambda: 1.0
-dtype: bfloat16
-```
-
-</details></p>
-</div>
-<div class="phase-box" style="border-color: #ff00ff; flex-grow: 1;">
-<h4 style="color: #ff00ff; margin: 0 0 5px 0;">Phase 04: model_stock</h4>
-<p style="font-size: 0.8rem; margin: 0; color: #cbd5e1;">Phase 01 X Phase 02 X Phase 03
-<details>
-<summary></summary>
-
-```
-models:
-  - model: ./phase01_breadcrumbs_ties
-  - model: ./phase02_slerp
-merge_method: model_stock
-base_model: ./phase03_della_linear
-dtype: bfloat16
-tokenizer_source: "base"
-```
-
-</details></p>
-</div>
-
-</div>
-
-<div class="phase-grid">
-<div class="phase-box" style="border-image: linear-gradient(90deg, #00f2ff, #ff00ff) 1; flex-grow: 1;">
-<h4 style="background: linear-gradient(90deg, #00f2ff, #ff00ff); -webkit-background-clip: text; -webkit-text-fill-color: transparent; margin: 0 0 5px 0;">Phase 05: arcee_fusion</h4>
-<p style="font-size: 0.8rem; margin: 0; color: #cbd5e1;">Gemsicle X Phase 04
-<details>
-<summary></summary>
-
-```
-models:
-  - model: ./Gemsicle-31B
-  - model: ./phase04_model_stock
-merge_method: arcee_fusion
-base_model: ./Gemsicle-31B
-dtype: bfloat16
-tokenizer_source: "base"
-```
-
-</details></p>
-</div>
-</div>
-
-<div style="display: flex; gap: 20px; flex-wrap: wrap; margin-top: 30px;">
-
-<div class="crystal-card gold-card" style="flex: 1; min-width: 250px; margin: 0; border-radius: 12px;">
-
-<h3 style="color: #fbbf24; margin-top: 0;">🏆 Credits & Honors</h3>
-
-<li style="margin-bottom: 8px;"><b>The Open-Source Community:</b> For providing the brilliant base models and fine-tunes that made this synthesis possible.</li>
-
-<li style="margin-bottom: 8px;"><b><a href="https://www.reddit.com/r/BeaverAI/" style="color: #fbbf24; text-decoration: none;">The BeaverAI Community</a>:</b> The people on the BeaverAI Discord - Without your help I wouldn't do all that.</li>
-
-<li style="margin-bottom: 8px;"><b>Mergekit:</b> Once again thank you Arcee AI for the great and easy to use mergekit! And thanks to zerofota and their fork for Gemma 4 support.</li>
-
-<li style="margin-bottom: 8px;"><b><a href="https://huggingface.co/Ateron" style="color: #fbbf24; text-decoration: none;">Ateron</a>:</b> Big kudos for providing me with the first steps for merging models and your relentless testing and support.</li>
-
-<li><b>Google Gemini:</b> For once again helping me to craft this specific model card.</li>
-</div>
-
-</div>
+- **Intel XPU** — `XPUW8A16FP8LinearKernel` (the intended target).
+- **NVIDIA CUDA** (SM75+, i.e. Turing and newer) —
+  `HummingFP8ScaledMMLinearKernel` when the `humming` package is installed,
+  otherwise `MarlinFP8ScaledMMLinearKernel`.
+- **Not supported**: ROCm, CPU, TPU — vLLM has no W8A16-FP8 kernel for these
+  backends yet, so loading will fail with a "no kernel" error.