Instructions to use RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS
# Run inference directly in the terminal:
llama cli -hf RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS
# Run inference directly in the terminal:
llama cli -hf RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS
# Run inference directly in the terminal:
./llama-cli -hf RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS
# Run inference directly in the terminal:
./build/bin/llama-cli -hf RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS

Use Docker

docker model run hf.co/RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS

LM Studio
Jan

vLLM

How to use RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS

Ollama
How to use RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored with Ollama:
```
ollama run hf.co/RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS
```
Unsloth Desktop

How to use RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored with Docker Model Runner:
```
docker model run hf.co/RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS
```

Lemonade

How to use RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS

Run and chat with the model

lemonade run user.Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored-IQ3_XXS

List all available models

lemonade list

Hermes Agent

How to use RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "RentedNoodle/Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored:IQ3_XXS" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored

A 3-bit GGUF quant of the abliterated Qwen3.8-27B, built to hold full 196K context on a single 16 GB GPU. Sibling size class carries ISTA-DASLab's full-size AIME/GPQA numbers; this file trades recall for residency and says so below.

A 9.73 GiB, mixed-precision GSQ/RCO quant of Huihui's refusal-reduced Qwen3.8-27B, preserving the MTP head and validated for 16 GB GPUs.

This is a quantization of huihui-ai/Huihui-Qwen3.8-27B-abliterated, not a new fine-tune. The underlying model is Qwen3.8-27B; the refusal ablation comes from Huihui; the quantization work here follows the GSQ/RCO methodology developed by ISTA-DASLab. The checkpoint is designed to substantially reduce learned refusal behavior. This is not a guarantee of universal compliance — the deployer is responsible for compliant use.

Quick specs


Base	`Qwen/Qwen3.8-27B` via `huihui-ai/Huihui-Qwen3.8-27B-abliterated`
File	`Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored-MTP.gguf`
Size	10,442,827,584 bytes (9.73 GiB)
Tensors	866 (65 blocks: 64 layers + MTP head)
bpw	3.058 (bytes×8 ÷ 27,320,697,856 params)
MTP head	Present - 15 `blk.64.*` weights incl. `nextn.eh_proj/enorm/hnorm/shared_head_norm`, verified non-empty, draft-accept 0.46-0.95 by workload logged against this exact file
Template	`froggeric-qwen3.8-tool-use.jinja` (local file header v22.5; upstream repo may still show v22.4)
Context	196K resident-fit on 16 GB (`q4_0` KV, verified loaded); quality gates ran at 32K server ctx + 64K needle ctx — full-196K retrieval quality pending, not claimed

What I changed

Starting from the abliterated checkpoint, I produced this GGUF using a GSQ/RCO-based non-uniform allocation and IQ3_XXS packing. This isn't a generic IQ3_XXS conversion: sensitive tensors (SSM gates, embeddings, attention gates — 96 tensors) are held at BF16 while the bulk rides the GSQ-coded grid.

The important honesty clause: I reproduced ISTA-DASLab's published per-tensor allocation map for Qwen3.8-27B and applied it to the abliterated weights. I did not independently re-run the multi-GPU RCO budget search. Same map, different base.

I also tried custom importance-matrix blends during the process (fresh + syncretized). They looked promising and then failed needle retrieval outright (0/6 timeouts), so I threw them out and kept the official ISTA imatrix. If a change makes the benchmark prettier but the model worse to actually use, it doesn't ship.

Tested performance (RTX 5070 Ti 16 GB, build2 llama.cpp, this exact file)

Configuration	Decode	Notes
Serial / no spec	~39.6 t/s	400-tok prose, thinking on
MTP `n-max 2`	~59.3 t/s	+50%. Ship default. Accept 0.46-0.95 by workload (code peaks, prose dips), mean len 1.9-2.9
MTP `n-max 2` + `--fit off` + `--spec-draft-backend-sampling` + `-b 1024`	69–82 t/s	Ship flags. The `--fit off` alone was worth +38%
MTP n-max 3 / 4 (+p-min)	33–37 t/s	Higher acceptance yet slower — verification cost wins. Don't use
KV `iq4_nl`	26.8 t/s	No fast kernel in this build. Don't use
KV `q4_0/q4_0`	ship	15.8/16 GB resident

Higher acceptance does not automatically mean higher throughput. The verification cost matters — measure tok/s, not just accept rate.

Suite	Score	Setup
Needle retrieval	6/6	15K-word haystack, depths 0.1–0.9, temp 0.0, 64K server ctx
Toolcall v1 (Pi JSON path)	8/8	`tool_call_format=json`
Toolcall v2 (args + must-not-fire + chains)	7/14 raw-template; negatives 4/4 clean	Schema path hits a build2 tools+MTP post-boot flake (warm-up rule); Pi-path number pending
Livebench-style	12/12	reasoning/coding/extraction mix
Coherence	4/4	multi-turn continuity

Temp ships at 0.6 (4/8 vs 3/8 raw-template A/B; favored for tool-use reliability in agentic loops). Reasoning: medium default, low for short code, xhigh for verifiable-correctness only.

Academic benchmarks (AIME/GPQA/LiveCodeBench) for this specific file: pending, will append. For reference, ISTA-DASLab's same-bpw base reports AIME25 100.0 / GPQA-Diamond 88.89 / LiveCodeBench v6 84.57 — their base, not this file.

Usage

Froggeric template is required for the intended tool/reasoning behavior (tool_call_format=json for the certified path; template default is xml).

llama-server \
  -m Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored-MTP.gguf \
  --alias Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored \
  --jinja --chat-template-file froggeric-qwen3.8-tool-use.jinja \
  --ctx-size 196608 -fa on -ctk q4_0 -ctv q4_0 \
  -ngl 99 -b 1024 -ub 1024 -np 1 -t 8 --fit off \
  --reasoning-budget 256 --spec-type draft-mtp --spec-draft-n-max 2 \
  --spec-draft-backend-sampling

Vision: projector files live in this repo under mmproj/ (BF16 or Q8_0). Start the server with one, then send a multimodal chat request (skip both flags for text-only):

llama-server -m <gguf> --mmproj mmproj/mmproj-Qwen3.8-27B-BF16.gguf --ctx-size 65536 [same flags as above]
# then POST /v1/chat/completions with content blocks including {"type":"image_url",...}

Ollama (text generation supported; tool/reasoning behavior uncertified — use llama-server + Froggeric for that):

FROM ./Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored-MTP.gguf
PARAMETER num_ctx 196608
PARAMETER num_gpu 999

Honest limitations

Aggressive 3-bit quant — don't expect BF16 behavior. Knowledge recall is weaker than higher-bit variants (the model leans on retrieval/tools instead). MTP output may differ from serial generation on quantized targets; fix the seed and runtime config for reproducibility, disable spec when exact serial behavior matters. Vision + long context gets tight on 16 GB (split text/vision profiles or offload the projector). The imatrix was computed on the stock base, not the abliterated weights — whether that shift matters is untested, not assumed safe.

Files in this repo

File	What
`Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored-MTP.gguf`	The quant (SHA256 in `SHA256SUMS.txt`)
Vision projector	`mmproj/mmproj-Qwen3.8-27B-BF16.gguf` (0.87 GB) + Q8_0 alt (0.60 GB, 27 tensors fallback-quantized)
`froggeric-qwen3.8-tool-use.jinja`	Required chat template
`REF-IQ3_XXS-mtp.rco-allocation.txt`	866-row per-tensor allocation map (authoritative over any summary)
`imatrix.dat`	ISTA-DASLab's published calibration matrix as used
`SHA256SUMS.txt`	Hashes
`LICENSE`	Upstream Apache-2.0 terms apply; see Qwen/Huihui repos

GGUF embeds tokenizer metadata — no separate tokenizer needed for llama.cpp/Ollama/LM Studio/Pi.

Credits

Qwen — architecture + pretrained weights: Qwen3.8 repo, Qwen3.8-27B
Huihui — abliterated checkpoint (model), via remove-refusals-with-transformers
GSQ — Dadgarnia, Tabesh, Nikdan, Helcig, Kurtic, Kleinegger, Alistarh (2026): paper, code
RCO — Helcig & Alistarh (2026): paper, code
ISTA-DASLab quant release — Qwen3.8-27B-GSQ-RCO-GGUF (allocation + imatrix source)
Froggeric — Qwen-Fixed-Chat-Templates
llama.cpp/GGML — runtime + format; IQ3_XXS is their standard type, nothing custom here

Reproducibility

Base: huihui-ai/Huihui-Qwen3.8-27B-abliterated
Allocation: REF-IQ3_XXS-mtp.rco-allocation.txt (in this repo), applied via llama-quantize --tensor-type-file
Imatrix: imatrix.dat (in this repo), ISTA-DASLab's published matrix as used
Runtime: den_llama.cpp @ 3231ee89 (build2; -DGGML_CUDA_FA_ALL_QUANTS=ON, CUDA 13.3)
GPU: RTX 5070 Ti 16 GB, Windows 11, driver per nvidia-smi at run time
Evals: prompts in evals/prompts/ (hashes in evals/SHA256SUMS), commands in evals/commands.md, raw outputs + timings + accept stats in evals/raw/

Versions

v1 (current): official imatrix, MTP intact, gates above. Earlier fresh/syncretized-imatrix builds failed needle and were discarded, not shipped.
v1.1 (in progress): FastMTP-style trained MTP head (4M self-distilled diverse corpus) for deeper-draft acceptance.
v1.2 (planned): M12 self-distilled corpus (12M agent/tool-heavy trajectories from the v1.1 teacher) + anything else that genuinely moves MTP acceptance at any depth (k1-k4): p-min tuning, draft-KV precision, head-precision ladder. This card updates on release.

Community quant, not affiliated with Qwen, Huihui, or ISTA-DASLab. Bug reports: include model revision, llama.cpp build, ctx/KV/spec settings, sampler. "Feels worse" is fine; reproducible info makes it actionable.