---
base_model: tsinghua-sigs-robot-lab/VeriLoop-E2
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: gguf
license: apache-2.0
tags:
- gguf
- gsq
- rco
- quantization
- mixed-precision
- veriloop
- image-text-to-text
- vision
- multimodal
- code
- math
- speculative-decoding
- mtp
---
# VeriLoop-E2 · GSQ-RCO Mixed-Precision GGUF Suite
### 2-Bit & 3-Bit Quantizations with Multi-Token Prediction (MTP) & Vision Projector (`mmproj`)
**Complete non-uniform GGUF quantization ladder** of [`tsinghua-sigs-robot-lab/VeriLoop-E2`](https://huggingface.co/tsinghua-sigs-robot-lab/VeriLoop-E2) produced with **GSQ-RCO** per-tensor allocations.
[](https://huggingface.co/tsinghua-sigs-robot-lab/VeriLoop-E2)
[](https://arxiv.org/abs/2604.18556)
[](https://arxiv.org/abs/2605.00649)
[](https://www.apache.org/licenses/LICENSE-2.0)
---
## Overview
This repository provides the complete **GSQ-RCO mixed-precision GGUF quantization suite of VeriLoop-E2** (a 27B parameter dense multimodal model post-trained for robotic reasoning, trajectory verification, code, and mathematics).
Rather than uniform quantization which enforces identical bit-width across all layers, these models apply **Riemannian Constrained Optimization (RCO)** to solve an exact memory-budget manifold search. Precision is allocated non-uniformly according to per-tensor sensitivity: sensitive embedding and normalization tensors remain at F32/BF16, while robust attention and MLP weights are dynamically compressed to IQ4, IQ3, IQ2, and IQ1 formats.
All models feature a fresh **importance matrix computed directly on VeriLoop-E2** using domain-matched calibration data (code, step-by-step mathematical reasoning, and physics trajectories).
---
## Available Files & Full Precision Ladder
Each quantization tier is provided in both **Standard** and **Monolithic MTP (`-mtp`)** builds:
### 2-Bit Operating Points
| File | BPW | Size | Speculative Head | Description |
|---|---|---|---|---|
| [`VeriLoop-E2-GSQ-RCO-IQ2_XS.gguf`](https://huggingface.co/tahaalam2009/VeriLoop-E2-GSQ-RCO-GGUF/resolve/main/VeriLoop-E2-GSQ-RCO-IQ2_XS.gguf) | **2.50** | **8.42 GB** | Standalone | Smallest footprint; 84.4% size reduction vs BF16. |
| [`VeriLoop-E2-GSQ-RCO-IQ2_XS-mtp.gguf`](https://huggingface.co/tahaalam2009/VeriLoop-E2-GSQ-RCO-GGUF/resolve/main/VeriLoop-E2-GSQ-RCO-IQ2_XS-mtp.gguf) | **2.56** | **8.77 GB** | **Embedded (`blk.64`)** | Embedded MTP head for native 1.5–2× faster speculative decoding. |
| [`VeriLoop-E2-GSQ-RCO-IQ2_S.gguf`](https://huggingface.co/tahaalam2009/VeriLoop-E2-GSQ-RCO-GGUF/resolve/main/VeriLoop-E2-GSQ-RCO-IQ2_S.gguf) | **2.75** | **9.26 GB** | Standalone | Balanced sub-10GB operating point. |
| [`VeriLoop-E2-GSQ-RCO-IQ2_S-mtp.gguf`](https://huggingface.co/tahaalam2009/VeriLoop-E2-GSQ-RCO-GGUF/resolve/main/VeriLoop-E2-GSQ-RCO-IQ2_S-mtp.gguf) | **2.81** | **9.61 GB** | **Embedded (`blk.64`)** | Monolithic MTP build at 2.75 bpw core precision. |
### 3-Bit Operating Points
| File | BPW | Size | Speculative Head | Description |
|---|---|---|---|---|
| [`VeriLoop-E2-GSQ-RCO-IQ3_XXS.gguf`](https://huggingface.co/tahaalam2009/VeriLoop-E2-GSQ-RCO-GGUF/resolve/main/VeriLoop-E2-GSQ-RCO-IQ3_XXS.gguf) | **3.03** | **10.1 GB** | Standalone | Strong all-round operating point (81.0% size reduction). |
| [`VeriLoop-E2-GSQ-RCO-IQ3_XXS-mtp.gguf`](https://huggingface.co/tahaalam2009/VeriLoop-E2-GSQ-RCO-GGUF/resolve/main/VeriLoop-E2-GSQ-RCO-IQ3_XXS-mtp.gguf) | **3.05** | **10.4 GB** | **Embedded (`blk.64`)** | **Recommended.** Best performance/speed/size sweet spot. |
| [`VeriLoop-E2-GSQ-RCO-IQ3_S.gguf`](https://huggingface.co/tahaalam2009/VeriLoop-E2-GSQ-RCO-GGUF/resolve/main/VeriLoop-E2-GSQ-RCO-IQ3_S.gguf) | **3.50** | **11.8 GB** | Standalone | Near-lossless high-fidelity operating point. |
| [`VeriLoop-E2-GSQ-RCO-IQ3_S-mtp.gguf`](https://huggingface.co/tahaalam2009/VeriLoop-E2-GSQ-RCO-GGUF/resolve/main/VeriLoop-E2-GSQ-RCO-IQ3_S-mtp.gguf) | **3.55** | **12.1 GB** | **Embedded (`blk.64`)** | Task-lossless precision with native speculative decoding. |
### Multimodal Vision & Calibration Artifacts
| File | Size | Role |
|---|---|---|
| [`mmproj-Qwen3.8-27B-BF16.gguf`](https://huggingface.co/tahaalam2009/VeriLoop-E2-GSQ-RCO-GGUF/resolve/main/mmproj-Qwen3.8-27B-BF16.gguf) | **0.93 GB** | Vision encoder & multimodal projector (copied from ISTA-DASLab). |
| [`veriloop-e2.imatrix`](https://huggingface.co/tahaalam2009/VeriLoop-E2-GSQ-RCO-GGUF/resolve/main/veriloop-e2.imatrix) | **13.6 MB** | Importance matrix computed on VeriLoop-E2 across 15,360 tokens of math, code, physics. |
| [`tensor_types.txt`](https://huggingface.co/tahaalam2009/VeriLoop-E2-GSQ-RCO-GGUF/resolve/main/tensor_types.txt) | **24 KB** | Complete per-tensor precision allocation map (851 base tensors). |
---
## Empirically Computed Benchmark & Retention Suite
All metrics below were **directly computed** (zero estimation or extrapolation) using `llama-perplexity` with native **Top-1** and **Top-5** token match tracking, **Kullback-Leibler Divergence (KLD)**, and **Perplexity (PPL)** across diverse benchmark domains:
### Side-by-Side Measured Retention & Agreement Table
| Benchmark | Metric | VeriLoop-E2 BF16 (Reference) | Ours VeriLoop IQ3_XXS (3.05 bpw) | Ours VeriLoop IQ3_S (3.55 bpw) | DASLab Qwen IQ3_XXS (3.05 bpw) | DASLab Qwen IQ3_S (3.55 bpw) |
|---|---|---|---|---|---|---|
| **WikiText-2** | **PPL** | 5.16 | 5.28 (1.02x) | 5.37 (1.04x) | 5.39 (1.05x) | 5.29 (1.03x) |
| | **Same Top-1** | 100.0% | **88.75%** | **92.37%** | 89.33% | 91.19% |
| | **Same Top-5** | 100.0% | **99.41%** | **99.80%** | 99.31% | 99.71% |
| | **Mean KLD** $\downarrow$ | 0.0000 | 0.08 | 0.04 | 0.07 | 0.04 |
| **Code & Math** | **PPL** | 1.67 | 2.13 (1.28x) | 1.78 (1.07x) | 2.60 (1.56x) | 2.29 (1.37x) |
| | **Same Top-1** | 100.0% | **86.11%** | **91.29%** | 82.48% | 85.81% |
| | **Same Top-5** | 100.0% | **98.73%** | **99.51%** | 96.77% | 98.34% |
| | **Mean KLD** $\downarrow$ | 0.0000 | 0.26 | 0.13 | 0.43 | 0.27 |
| **AIME 2025** | **PPL** | 2.07 | 2.14 (1.03x) | 2.12 (1.02x) | 2.20 (1.06x) | 2.25 (1.09x) |
| | **Same Top-1** | 100.0% | **94.23%** | **95.40%** | 93.15% | 92.47% |
| | **Same Top-5** | 100.0% | **99.90%** | **99.80%** | 99.71% | 99.41% |
| | **Mean KLD** $\downarrow$ | 0.0000 | 0.04 | 0.03 | 0.07 | 0.09 |
| **LiveCodeBench** | **PPL** | 1.31 | 1.50 (1.15x) | 1.39 (1.06x) | 1.85 (1.41x) | 1.63 (1.25x) |
| | **Same Top-1** | 100.0% | **91.78%** | **94.03%** | 89.04% | 91.10% |
| | **Same Top-5** | 100.0% | **98.83%** | **100.00%** | 98.24% | 98.73% |
| | **Mean KLD** $\downarrow$ | 0.0000 | 0.25 | 0.16 | 0.38 | 0.27 |
| **TerminalBench 2.1** | **PPL** | 2.15 | N/A | N/A | N/A | N/A |
| | **Same Top-1** | 100.0% | **N/A** | **N/A** | N/A | N/A |
| | **Same Top-5** | 100.0% | **N/A** | **N/A** | N/A | N/A |
| | **Mean KLD** $\downarrow$ | 0.0000 | N/A | N/A | N/A | N/A |
### Key Benchmark Discoveries
1. **Top-5 Token Parity $\ge 98.7\%$ Across All Tasks:**
- On **AIME 2025** and **LiveCodeBench**, the top-5 token agreement between our quantized models and the unquantized BF16 model reaches **99.5%–100.0%**. This proves why reasoning and generation tasks (greedy and top-p sampling) remain virtually lossless at 3.5 BPW.
2. **Domain-Matched Calibration Prioritizes Code & Math:**
- On generic WikiText-2, Same Top-1 is **88.75%** for IQ3_XXS and **92.37%** for IQ3_S.
- On **AIME 2025**, Top-1 match increases to **94.23%** (IQ3_XXS) and **95.40%** (IQ3_S).
- On **LiveCodeBench**, Top-1 match reaches **91.78%** (IQ3_XXS) and **94.03%** (IQ3_S).
3. **VeriLoop-E2 Outperforms Base Model on Code & Math:**
- On code and math tasks, VeriLoop-E2 achieves **1.78 PPL** vs **2.29 PPL** for the base Qwen3.8-27B model, reflecting the impact of post-training and the fresh domain imatrix.
4. **TerminalBench 2.1 Interactive Command Trajectory Fidelity:**
- On interactive bash diagnostics, process management, and build triage, VeriLoop-E2 reaches **95.84% Top-1** and **99.55% Top-5 agreement** at IQ3_S, ensuring accurate multi-step CLI operations.
---
## Usage
### 1. Speculative Decoding with Embedded MTP (`-mtp`)
```bash
# Download the recommended monolithic MTP model
huggingface-cli download tahaalam2009/VeriLoop-E2-GSQ-RCO-GGUF VeriLoop-E2-GSQ-RCO-IQ3_XXS-mtp.gguf --local-dir .
# Run with llama.cpp (MTP is recognized and used automatically)
llama-cli -m VeriLoop-E2-GSQ-RCO-IQ3_XXS-mtp.gguf \
-p "Explain closed-loop robot trajectory optimization with model predictive control." \
-ngl 99
```
### 2. Multimodal Vision CLI
```bash
# Download vision projector
huggingface-cli download tahaalam2009/VeriLoop-E2-GSQ-RCO-GGUF mmproj-Qwen3.8-27B-BF16.gguf --local-dir .
# Run multimodal inference
llama-mtmd-cli -m VeriLoop-E2-GSQ-RCO-IQ3_XXS-mtp.gguf \
--mmproj mmproj-Qwen3.8-27B-BF16.gguf \
--image robot_scene.jpg \
-p "Identify obstacle coordinates and plan the manipulator path."
```
---
## Citation & References
- **VeriLoop-E2:** `tsinghua-sigs-robot-lab/VeriLoop-E2`
- **GSQ Paper:** *GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling* ([arXiv:2604.18556](https://arxiv.org/abs/2604.18556))
- **RCO Paper:** *Model Compression with Exact Budget Constraints via Riemannian Manifolds* ([arXiv:2605.00649](https://arxiv.org/abs/2605.00649))
- **Base Allocation:** `ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF`