Instructions to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Use Docker

docker model run hf.co/tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Ollama
How to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with Ollama:
```
ollama run hf.co/tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M
```
Unsloth Desktop

How to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with Docker Model Runner:
```
docker model run hf.co/tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M
```

Lemonade

How to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.VeriLoop-E2-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

VeriLoop logo

VeriLoop E2 — GGUF

Official llama.cpp distribution of VeriLoop E2
BF16 reference · Q8_0 high fidelity · Q6_K overall sweet spot · Q5_K_M memory-quality sweet spot · IQ1_M minimum-footprint sweet spot

27B post-trained model for code, mathematics, and physics · 262K native context · Apache License 2.0
Developed by Tsinghua SIGS Robot Lab · Libo Wang

GGUF · Hugging Face · GGUF · ModelScope Global · GGUF · ModelScope China

Parent Model · Technical Report · Evaluation Evidence · Riemann ζ Artifact

Model variants

Use case	File	Main size	BF16-paired retention
Default / overall balance	`VeriLoop-E2-Q6_K.gguf`	20.566 GiB	PPL parity within uncertainty; KLD 0.004409; Same top-p 98.204%
Memory-quality balance	`VeriLoop-E2-Q5_K_M.gguf`	18.965 GiB	PPL +0.4450%; KLD 0.006919; Same top-p 97.251%
Minimum footprint	`VeriLoop-E2-IQ1_M.gguf`	16.790 GiB	PPL +0.3191%; KLD 0.014357; Same top-p 95.870%
Lower-KLD low-footprint alternative	`VeriLoop-E2-IQ2_S.gguf`	16.799 GiB	PPL +0.3457%; KLD 0.014023; Same top-p 95.516%
Low-footprint alternative	`VeriLoop-E2-Q3_K_M.gguf`	16.826 GiB	PPL +0.4090%; KLD 0.014349; Same top-p 95.919%
Balanced compact	`VeriLoop-E2-Q4_K_M.gguf`	18.301 GiB	PPL +0.4821%; KLD 0.009700; Same top-p 96.786%
High fidelity	`VeriLoop-E2-Q8_0.gguf`	26.632 GiB	PPL +0.0643%; KLD 0.002176; Same top-p 98.815%
Reference	`VeriLoop-E2-BF16.gguf`	50.113 GiB	Canonical BF16 reference

Recommended deployment points

Q6_K — overall quality / efficiency sweet spot. It is 58.96% smaller than BF16 while remaining statistically consistent with BF16 PPL parity under the frozen paired protocol.

Q5_K_M — memory-quality sweet spot. It reduces the main-file footprint to 18.965 GiB while preserving wider KLD and Same-top margins than the sub-17 GiB variants.

IQ1_M — minimum-footprint sweet spot. It is 16.790078 GiB, 66.4955% smaller than BF16, and passes the frozen hard gate, engineering-reserve gate, stock llama.cpp runtime validation, and real MTP engagement validation. IQ2_S remains the lower-KLD low-footprint alternative.

Naming note: VeriLoop-E2-IQ1_M.gguf is a mixed-precision artifact, not a uniform 1-bit model. Its measured tensor policy is 353 F32 + 1 IQ1_M + 2 IQ2_S + 64 Q4_K + 429 Q5_K + 2 Q6_K = 851 tensors, with 5.36 effective BPW. VeriLoop-E2-IQ2_S.gguf is likewise mixed precision rather than uniform 2-bit quantization.

Precision ladder

Tier	Role	Main size	Reduction vs BF16	Effective density
BF16	Canonical reference	53.808 GB / 50.113 GiB	—	16-bit-class
Q8_0	High fidelity	28.596 GB / 26.632 GiB	46.86%	8.50 BPW
Q6_K	Overall sweet spot	22.083 GB / 20.566 GiB	58.96%	6.57 BPW
Q5_K_M	Memory-quality sweet spot	20.364 GB / 18.965 GiB	62.16%	6.05 BPW
Q4_K_M	Balanced compact	19.651 GB / 18.301 GiB	63.48%	5.84 BPW
Q3_K_M	Low-footprint alternative	18.067 GB / 16.826 GiB	66.42%	5.37 BPW
IQ2_S	Lower-KLD low-footprint alternative	18.037 GB / 16.799 GiB	66.48%	5.36 BPW
IQ1_M	Minimum-footprint sweet spot	18.028 GB / 16.790 GiB	66.50%	5.36 BPW

Quantization-retention benchmark

All measured tiers use the same frozen BF16 logits and the same paired protocol.

Tier	Mean PPL	PPL ratio vs BF16	Relative PPL change	Mean KLD	Same top-p	log-PPL correlation
BF16	4.840423 ± 0.119931	1.000000	—	0 reference	100% reference	100% reference
Q8_0	4.843536 ± 0.120062	1.000643 ± 0.000754	+0.0643%	0.002176 ± 0.000668	98.815 ± 0.120%	99.95%
Q6_K	4.838514 ± 0.119694	0.999605 ± 0.001222	−0.0395% observed; parity within uncertainty	0.004409 ± 0.000953	98.204 ± 0.147%	99.88%
Q5_K_M	4.861965 ± 0.120630	1.004450 ± 0.001361	+0.4450%	0.006919 ± 0.000945	97.251 ± 0.181%	99.85%
Q4_K_M	4.863760 ± 0.120720	1.004821 ± 0.001722	+0.4821%	0.009700 ± 0.001110	96.786 ± 0.195%	99.76%
Q3_K_M	4.860222 ± 0.120606	1.004090 ± 0.002181	+0.4090%	0.014349 ± 0.001985	95.919 ± 0.219%	99.62%
IQ2_S	4.857159 ± 0.120482	1.003457 ± 0.002100	+0.3457%	0.014023 ± 0.001408	95.516 ± 0.229%	99.64%
IQ1_M	4.855870 ± 0.120480	1.003191 ± 0.002175	+0.3191%	0.014357 ± 0.001317	95.870 ± 0.220%	99.62%

These figures measure quantization retention against the BF16 reference. They are not downstream benchmark-score loss percentages. The nine parent-model benchmarks were not independently rerun for every quantization tier.

IQ1_M validation record

IQ1_M changes exactly one tensor relative to the IQ2_S precision policy: blk.1.ffn_down.weight moves from IQ2_S to IQ1_M. All other tensor assignments remain unchanged.

Precision	Tensor assignment	Count
F32	Non-quantized tensors retained by GGUF conversion	353
Q6_K	`output.weight`, `token_embd.weight`	2
Q5_K	Remaining quantized internal tensors	429
Q4_K	all `*.ffn_up.weight`	64
IQ2_S	`blk.0.ffn_down.weight`, `blk.3.ffn_down.weight`	2
IQ1_M	`blk.1.ffn_down.weight`	1
Total		851

Fidelity and runtime validation

Validation	Requirement	Measured	Result
PPL ratio hard / reserve	≤ 1.0150 / ≤ 1.0135	1.003191	PASS / PASS
Mean KLD hard / reserve	≤ 0.0150 / ≤ 0.0145	0.014357	PASS / PASS
Same top-p hard / reserve	≥ 95.0% / ≥ 95.5%	95.870%	PASS / PASS
Exact tensor structure	851 tensors; declared policy	851 tensors; 0 mismatch	PASS
Stock llama.cpp main runtime	Successful real generation	HTTP 200, non-empty generation	PASS
MTP runtime	Successful real generation	HTTP 200, non-empty generation	PASS
Real MTP engagement	generated > 0; accepted > 0	76 accepted / 104 generated (73.0769%)	PASS
Main vs MTP deterministic audit	Advisory	Identical output SHA256	IDENTICAL

IQ1_M artifact identity

Property	Value
Filename	`VeriLoop-E2-IQ1_M.gguf`
Exact bytes	18,028,208,896
Binary size	16.790078 GiB
SHA256	`e4d395806994cdbcfecf7711e22456af7a663b3ea37c89ef15d5bab4571cf68b`
Effective density	5.36 BPW
Tensor count	851
Quantizer-reported size	17,182.55 MiB
Quantizer time	167.77645 s
llama.cpp validation revision	`42916d83f4a225e56709f873aa8050ac11f5b6a4`

Low-footprint comparison

Metric	Q3_K_M	IQ2_S	IQ1_M
Main size	16.825745 GiB	16.798508 GiB	16.790078 GiB
PPL ratio	1.004090	1.003457	1.003191
Mean KLD	0.014349	0.014023	0.014357
Same top-p	95.919%	95.516%	95.870%
RMS Δp	3.515%	3.679%	3.691%

IQ1_M is 8.633 MiB smaller than IQ2_S and 36.523 MiB smaller than Q3_K_M. IQ2_S retains the lowest Mean KLD of the three; IQ1_M has the smallest footprint, the lowest PPL ratio, and higher Same top-p than IQ2_S. The point-estimate differences remain small relative to the reported uncertainty scale.

Frozen BF16-paired protocol

Item	Value
Reference model	Canonical VeriLoop E2 BF16 GGUF
Corpus	WikiText-2 raw test
Corpus SHA256	`173c87a53759e0201f33e0ccf978e510c2042d7f2cb78229d9a50d79b9e7dd08`
Context	2,048
Chunks	8
Seed	42
GPU layers	40
Flash Attention	Off
KV cache	F16/F16
Batch / micro-batch	512 / 512
Evaluator	`llama-perplexity`
Reference logits	`--kl-divergence-base`
Quantized comparison	`--kl-divergence`
BF16 logits reused across tiers	Yes
llama.cpp revision	`42916d83f4a225e56709f873aa8050ac11f5b6a4`

Parent-model benchmark record

The scores below describe the parent VeriLoop E2 release and are not relabeled as quantization-specific reruns.

Benchmark	VeriLoop E2 parent score	Public evidence
SWE-bench Pro	76.2	Evidence
Terminal-Bench 2.1	88.8	Evidence
DeepSWE v1.1	64.6	Evidence
Terminal-Bench 3.0	29.7	Evidence
Terminal-Bench 4.0	37.9	Evidence
SWE-Marathon v1.1	45.0	Parent release
AIME 2026	98.3	Evidence
GPQA Diamond	93.9	Evidence
Apex 2025	89.6	Evidence

Quick start with llama.cpp

# Overall sweet spot
hf download tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF VeriLoop-E2-Q6_K.gguf --local-dir .

# Minimum-footprint sweet spot
hf download tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF VeriLoop-E2-IQ1_M.gguf --local-dir .

# Run
llama-server \
  -m ./VeriLoop-E2-IQ1_M.gguf \
  -ngl 99 \
  -c 32768 \
  --host 127.0.0.1 \
  --port 8080

Memory guidance

Variant	Main file	Positioning
BF16	50.113 GiB	Reference
Q8_0	26.632 GiB	High fidelity
Q6_K	20.566 GiB	Overall sweet spot
Q5_K_M	18.965 GiB	Memory-quality sweet spot
Q4_K_M	18.301 GiB	Balanced compact
Q3_K_M	16.826 GiB	Low-footprint alternative
IQ2_S	16.799 GiB	Lower-KLD low-footprint alternative
IQ1_M	16.790 GiB	Minimum-footprint sweet spot

A 16.79 GiB model file does not imply full offload on a 16 GiB GPU. Runtime memory also includes KV cache, compute buffers, allocator overhead, and optional speculative-decoding weights.

A standalone Q8_0 llama-bench record exists (pp512 3102.722776 tok/s; tg128 41.999677 tok/s), but there is no frozen paired BF16 throughput campaign. No universal speedup percentage is claimed for the quantization ladder.

Measurement boundaries

PPL, KLD, Same top-p, and token-probability statistics are quantization-retention measurements, not universal downstream capability-loss percentages.
IQ1_M and IQ2_S are mixed-precision artifacts; the tier name does not equal the model-wide effective bit width.
Native 262K context comes from the parent configuration; practical context depends on runtime memory.
MTP acceptance is prompt- and workload-dependent.
The model can still produce incorrect code, mathematics, scientific reasoning, or commands.

Source model and evidence

Resource	Link
Parent model	VeriLoop E2
GGUF · Hugging Face	VeriLoop E2 GGUF
GGUF · ModelScope Global	VeriLoop E2 GGUF
GGUF · ModelScope China	VeriLoop E2 GGUF
Quantization quality record	Quantization Quality
Artifact manifest	Release Manifest
Technical report	OpenReview
Evaluation evidence	VeriLoop E2 Evaluation Evidence
Riemann ζ artifact	Public artifact
llama.cpp	ggml-org/llama.cpp

License

The VeriLoop E2 model weights and this GGUF distribution are released under the Apache License 2.0, consistent with the parent model release.

Citation

@misc{wang2026veriloope2,
  title        = {VeriLoop E2: A 27B Post-Trained Model for Code, Mathematics, and Scientific Reasoning},
  author       = {Wang, Libo},
  year         = {2026},
  note         = {Tsinghua Shenzhen International Graduate School (SIGS)},
  howpublished = {Open model release},
  url          = {https://huggingface.co/tsinghua-sigs-robot-lab/VeriLoop-E2}
}

For quantization-specific comparisons, identify the exact GGUF filename and corresponding manifest identity.

Downloads last month: 2,709

GGUF

Model size

27B params

Architecture

qwen35

Hardware compatibility

1-bit

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

Model tree for tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF

Base model

Qwen/Qwen3.8-27B

Finetuned

tsinghua-sigs-robot-lab/VeriLoop-E2

Quantized

(11)

this model

tsinghua-sigs-robot-lab
/

VeriLoop-E2-GGUF

VeriLoop E2 — GGUF

Model variants

Recommended deployment points

Precision ladder

Quantization-retention benchmark

IQ1_M validation record

Fidelity and runtime validation

Low-footprint comparison

Frozen BF16-paired protocol

Parent-model benchmark record

Quick start with llama.cpp

Memory guidance

Measurement boundaries

Source model and evidence

License

Citation

Model tree for tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF

Space using tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF 1