Update README.md

README.md

@@ -1,3 +1,76 @@
 ---
 license: apache-2.0
 ---
+
+# Qwen3.8-27B GGUF Quantizations (IQ2_E_S & IQ3_E_S)
+
+This repository provides experimental, low-bitrate GGUF quantizations for **Qwen3.8-27B**, focused on extreme VRAM efficiency without catastrophic quality degradation.
+
+## Motivation & Overview
+
+The primary goal of this project was to explore whether Qwen3.8-27B could be loaded and executed on consumer GPUs with strictly **8 GB of VRAM**. 
+
+Quantizing a 27B model down to this range yields two distinct results:
+1. **IQ2_E_S (2.40 BPW, ~7.51 GiB):** Fits tightly into an 8 GB VRAM budget. While low-bit quantization introduces inevitable degradation, perplexity remains bounded, making the model practically usable for lighter or structured generation tasks.
+2. **IQ3_E_S (3.00 BPW, ~9.41 GiB):** A balanced quantization targeting 10–12 GB GPUs, showing theoretical perplexity retention close to the unquantized baseline.
+
+---
+
+## Perplexity Evaluation
+
+Perplexity was evaluated using standard Wiki-style test sets and compared against the FP16/BF16 reference as well as external community baselines.
+
+| Candidate | Size | BPW | Wiki-style PPL | Notes |
+| :--- | :---: | :---: | :---: | :--- |
+| **BF16 GGUF** | 50.89 GiB | 16.00 | 7.15 ± 0.12 | Unquantized Baseline |
+| **IQ3_E_S** *(Ours)* | 9.41 GiB | 3.00 | **7.4453 ± 0.1287** | Best balance (10–12 GB VRAM) |
+| `empero-ai/Qwen3.8-27B-Ridge-GGUF` | 11.73 GiB | 3.69 | 7.82 ± 0.14 | External Reference |
+| **IQ2_E_S** *(Ours)* | 7.51 GiB | 2.40 | **8.8587 ± 0.1576** | Sub-8GB Footprint |
+
+*Note: The IQ3_E_S quantization shows lower perplexity than larger existing baselines at a significantly reduced footprint.*
+
+---
+
+## Usage & Tested Hardware Setup
+
+The **IQ2_E_S** variant was tested on a dedicated secondary GPU with **8 GB of available VRAM** using `llama.cpp` (`llama-server`). Under the following launch parameters, total VRAM consumption stabilized at approximately **7.6 GB**:
+
+```cmd
+llama-server.exe ^
+  -m Qwen3.8-27B-IQ2_E_S.gguf ^
+  --gpu-layers all ^
+  --flash-attn on ^
+  --ctx-size 16384 ^
+  --cache-type-k q8_0 ^
+  --cache-type-v q8_0 ^
+  --batch-size 1024 ^
+  --ubatch-size 512
+  ```
+  
+  ## Backend & Hardware Compatibility Notes (Intel Arc / oneAPI)
+
+> **Important Warning for Intel Arc Users:**  
+> The **IQ2_E_S** build is known to **collapse/fail under the SYCL backend** (producing gibberish, NaNs, or crashes during execution).  
+> If you are running on Intel Arc hardware, **you must use the Vulkan backend instead of SYCL**:
+> * Ensure your `llama.cpp` binary is compiled with `-DGGML_VULKAN=ON`.
+> * Run via Vulkan to avoid kernel execution bugs present in current SYCL implementations for low-bit IQ quantizations.
+  
+# Limitations & Validation Notes
+
+   - **IQ2_E_S**: Subjectively evaluated under the configuration above. Coherence remains acceptable for conversational and summary flows despite the low bitrate, though reasoning and technical accuracy will degrade compared to higher quantizations.
+
+   - **IQ3_E_S**: Due to physical VRAM constraints (8 GB test system) and limited testing time, IQ3_E_S has not been empirically verified in inference, only evaluated for perplexity. Users with 12 GB or 16 GB setups are encouraged to test it.
+
+# Community Feedback
+
+If you test either the `IQ2_E_S` or `IQ3_E_S` builds, please leave feedback in the Discussions tab:
+
+   - Real-world coherence vs. baseline expectations.
+
+   - Inference speeds (tokens/sec) across different backends (CUDA, Vulkan, SYCL, Metal).
+
+   - Context scaling behavior beyond 8k tokens.
+
+# Support
+
+If these quantizations are useful for your local workflows or research, consider supporting further compute and quantization tests: