Instructions to use dancinlab/Qwen3.8-27B-Uncensored-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use dancinlab/Qwen3.8-27B-Uncensored-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M

Use Docker

docker model run hf.co/dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use dancinlab/Qwen3.8-27B-Uncensored-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "dancinlab/Qwen3.8-27B-Uncensored-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dancinlab/Qwen3.8-27B-Uncensored-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M

Ollama
How to use dancinlab/Qwen3.8-27B-Uncensored-GGUF with Ollama:
```
ollama run hf.co/dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M
```
Unsloth Desktop

How to use dancinlab/Qwen3.8-27B-Uncensored-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use dancinlab/Qwen3.8-27B-Uncensored-GGUF with Docker Model Runner:
```
docker model run hf.co/dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M
```

Lemonade

How to use dancinlab/Qwen3.8-27B-Uncensored-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3.8-27B-Uncensored-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use dancinlab/Qwen3.8-27B-Uncensored-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use dancinlab/Qwen3.8-27B-Uncensored-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "dancinlab/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.8-27B-Uncensored-GGUF / README.md

dancinlife

Mirror orcarouter/Qwen3.8-27B-Uncensored-GGUF@b33b15636fce4621fd1c3499bb5c9681465fb626

c978b0a verified 30 days ago

preview code

Raw

History Blame Contribute Delete

10.7 kB

	---
	license: apache-2.0
	base_model: Qwen/Qwen3.8-27B
	base_model_relation: quantized
	pipeline_tag: image-text-to-text
	library_name: gguf
	language:
	- en
	- zh
	tags:
	- abliterated
	- qwen
	- qwen3
	- qwen3.8
	- gguf
	- llama.cpp
	- uncensored
	- ai-red-team
	- red-teaming
	- vision-language
	- mmproj
	- mtp
	- function-calling
	- reasoning
	---

	<div align="center">

	<a href="https://www.orcarouter.ai" target="_blank">
	<img src="https://www.orcarouter.ai/orca-logo-classic.png" alt="OrcaRouter" width="110">
	</a>

	<h1>Qwen3.8-27B-Uncensored-GGUF</h1>

	<p><em>GGUF quants (2-bit → 16-bit) of the abliterated (refusal-removed) Qwen3.8-27B — for llama.cpp</em></p>

	<p>
	<a href="https://www.orcarouter.ai"><img src="https://img.shields.io/badge/Website-orcarouter.ai-1E6FEB" alt="Website"></a>
	<a href="https://www.orcarouter.ai/models"><img src="https://img.shields.io/badge/OrcaRouter-Model%20Catalog-2EA043" alt="Model Catalog"></a>
	<a href="https://www.orcarouter.ai/models/qwen/qwen3.8-27b"><img src="https://img.shields.io/badge/OrcaRouter-Model%20Card-6F42C1" alt="Model Card"></a>
	<a href="https://www.apache.org/licenses/LICENSE-2.0"><img src="https://img.shields.io/badge/License-Apache%202.0-4C8BF5" alt="License"></a>
	<img src="https://img.shields.io/badge/Format-GGUF-00A67E" alt="GGUF">
	<img src="https://img.shields.io/badge/Quants-Q2__K%20%E2%86%92%20F16-FF8800" alt="Quants">
	<img src="https://img.shields.io/badge/Vision-mmproj-9B59B6" alt="Vision">
	</p>

	<p><strong>One Gateway. Every Model.</strong> — Route Smarter · Ship Safer · Spend Less.</p>

	<p>
	<a href="https://www.orcarouter.ai">Website</a> ·
	<a href="https://www.orcarouter.ai/models">Model Catalog</a> ·
	<a href="https://www.orcarouter.ai/models/qwen/qwen3.8-27b">Model Card</a> ·
	<a href="https://github.com/Continuum-AI-Corp">GitHub</a> ·
	<a href="https://ollama.com/orcarouter">Ollama</a> ·
	<a href="https://discord.gg/yAh6Tex6kx">Discord</a> ·
	<a href="https://x.com/OrcaRouter">X</a>

	</p>

	</div>

	---

	> GGUF conversions of [`Qwen3.8-27B-Uncensored`](https://huggingface.co/orcarouter/Qwen3.8-27B-Uncensored-FP8)
	> — an abliterated (refusal-removed) build of Qwen's `Qwen3.8-27B`, a 27B dense hybrid-attention
	> (Gated DeltaNet linear + full attention) native vision-language model with reasoning, tool-calling,
	> and an MTP speculative-decoding head. These files run in llama.cpp (CPU / CUDA / Metal / ROCm),
	> quantized from 2-bit to 16-bit, with a separate mmproj file that restores vision.
	> Browse all models in the [OrcaRouter Model Catalog](https://www.orcarouter.ai/models). Qwen3.8 27B is
	> deployed as API [on OrcaRouter](https://www.orcarouter.ai/models/qwen/qwen3.8-27b).

	---

	## ⚠️ Disclaimer — read before use

	This model has had its safety alignment substantially removed via abliteration (orthogonalizing
	the refusal direction out of the residual stream). It will **comply with harmful, unethical, or illegal
	requests the original `Qwen3.8-27B` would refuse. Released strictly for legitimate research** —
	interpretability, AI-safety / refusal-mechanism study, red-teaming, and robustness evaluation. **You
	assume full responsibility** for how you use it and everything it generates; add your own safety and
	moderation layers before any deployment. Use must comply with the
	[Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0) inherited from the base model and all
	applicable law. The authors accept no liability for misuse.

	---

	## Requirements

	- A recent [llama.cpp](https://github.com/ggml-org/llama.cpp) built from source (the `qwen35`
	hybrid-GDN architecture and the MTP / `nextn` speculative head — merged 2026-05 — must be present).
	Older releases will not load these files.
	- The GDN linear-attention layers are stored as SSM-style tensors (`ssm_*`); full-attention layers as
	`attn_`; the MTP head as block `nextn.` (`qwen35.nextn_predict_layers`).

	## Files

	### Standard K-quants

	\| File \| Bits \| Size \| Notes / recommendation \|
	\|---\|---\|---:\|---\|
	\| `…-Q2_K.gguf` \| 2-bit \| 10.9 GB \| Smallest K-quant; noticeable quality drop — low-VRAM only \|
	\| `…-Q3_K_S.gguf`\| 3-bit \| 12.3 GB \| \|
	\| `…-Q3_K_M.gguf`\| 3-bit \| 13.5 GB \| Good small option \|
	\| `…-Q3_K_L.gguf`\| 3-bit \| 14.6 GB \| \|
	\| `…-Q4_K_S.gguf`\| 4-bit \| 15.8 GB \| \|
	\| `…-Q4_K_M.gguf` \| 4-bit \| 16.8 GB \| Recommended default — best quality/size balance \|
	\| `…-Q5_K_S.gguf`\| 5-bit \| 17.7 GB \| \|
	\| `…-Q5_K_M.gguf`\| 5-bit \| 18.2 GB \| High quality \|
	\| `…-Q6_K.gguf` \| 6-bit \| 20.9 GB \| Very high quality \|
	\| `…-Q8_0.gguf` \| 8-bit \| 27.1 GB \| Near-lossless \|
	\| `…-F16-0000*-of-00002.gguf` \| 16-bit \| 54.7 GB \| Full precision (split into 2 parts; point llama.cpp at part 00001) \|

	### IQ quants (imatrix)

	Lower-bit quants built with an importance matrix (computed on English + Chinese calibration
	text) — better quality-per-bit than plain K-quants at the low end, especially IQ3/IQ2.

	\| File \| Bits \| Size \| Notes / recommendation \|
	\|---\|---\|---:\|---\|
	\| `…-IQ4_XS.gguf` \| ~4.25-bit \| 15.3 GB \| Best low-bit pick — ≈ Q4_K_S quality at smaller size \|
	\| `…-IQ3_M.gguf` \| ~3.7-bit \| 12.8 GB \| Solid 3-bit \|
	\| `…-IQ3_XXS.gguf` \| ~3.1-bit \| 11.6 GB \| Smaller 3-bit \|
	\| `…-IQ2_M.gguf` \| ~2.7-bit \| 10.5 GB \| Runs in low VRAM; some quality loss \|
	\| `…-IQ2_XXS.gguf` \| ~2.1-bit \| 8.9 GB \| Smallest runnable; most degraded \|

	### Vision

	\| File \| Size \| Notes \|
	\|---\|---:\|---\|
	\| `mmproj-…-f16.gguf` \| 0.9 GB \| Vision projector — download this too for image input \|

	All quants (K-quant and IQ) preserve the MTP (`nextn`) head and the GDN hybrid architecture;
	vision is provided by the separate `mmproj` file. The IQ files were quantized with an importance
	matrix (computed on English + Chinese calibration text) for better low-bit fidelity; the matrix
	itself is not shipped, as it is only needed to re-quantize these files, not to run them.

	## Usage (llama.cpp)

	### Download

	```bash
	hf download orcarouter/Qwen3.8-27B-Uncensored-GGUF \
	Qwen3.8-27B-Uncensored-Q4_K_M.gguf mmproj-Qwen3.8-27B-Uncensored-f16.gguf \
	--local-dir ./qwen38-uncensored
	```

	### Chat (text)

	```bash
	./llama-cli -m Qwen3.8-27B-Uncensored-Q4_K_M.gguf --jinja -c 8192 -p "Hello!"
	```

	### OpenAI-compatible server (tool calling + reasoning + vision)

	```bash
	./llama-server -m Qwen3.8-27B-Uncensored-Q4_K_M.gguf \
	--mmproj mmproj-Qwen3.8-27B-Uncensored-f16.gguf \
	--host 0.0.0.0 --port 8000 -c 8192 --jinja
	```

	- Vision: pass `--mmproj …`, then send OpenAI `image_url` content parts (base64 data-URI or URL).
	- Tool calling: `--jinja` enables the Qwen tool template; use standard OpenAI `tools` + `tool_calls`.
	- Reasoning (thinking): thinking is on by default; toggle per request via
	`chat_template_kwargs.enable_thinking`. The reasoning trace is returned in `reasoning_content`
	(give `max_tokens` room, e.g. ≥ 2048, so the final answer isn't truncated by the thinking budget).
	- MTP speculative decoding: the `nextn` head is embedded; enable llama.cpp's MTP/speculative path
	for a decode speed-up (optional — the model runs fine without it).

	### Or skip llama.cpp — run it with Ollama

	The same quants are published as Ollama tags, with the `mmproj` projector already bundled in (no second
	file, no `--mmproj` flag) and vision / tools / thinking wired up:

	```bash
	ollama run orcarouter/Qwen3.8-27B-Uncensored # q4_K_M by default
	ollama run orcarouter/Qwen3.8-27B-Uncensored:iq4_xs # any of the 16 tags
	```

	Sixteen tags from `q2_K` to `q8_0`, each verified on the actual quantized artifact before release —
	[ollama.com/orcarouter](https://ollama.com/orcarouter).

	## Evaluation

	Measured on this exact abliterated build (served with vLLM) vs the official `Qwen/Qwen3.8-27B-FP8`,
	using the same scripts and settings. Refusal is judged by a rule-based opening-phrase classifier —
	indicative, not an LLM-judge / publication-grade number. GGUF quants are deterministic derivations and
	inherit these behaviours; lower bits trade some quality (see the file table).

	### Safety — harmful-prompt refusal, thinking OFF (lower = more uncensored)

	\| Benchmark \| n \| Base FP8 \| This model \|
	\|---\|---:\|---:\|---:\|
	\| AdvBench \| 100 \| 99.0% \| 0.0% \|
	\| JailbreakBench (harmful) \| 100 \| 94.0% \| 0.0% \|
	\| StrongREJECT \| 150 \| 97.3% \| 2.0% \|
	\| HarmBench (standard) \| 150 \| 98.7% \| 2.7% \|
	\| MaliciousInstruct \| 100 \| 99.0% \| 0.0% \|
	\| SimpleSafetyTests \| 50 \| 64.0% \| 6.0% \|
	\| ForbiddenQuestions \| 150 \| 73.3% \| 4.7% \|
	\| Custom probes (zh/en) \| 11 \| 63.6% \| 0.0% \|

	### Safety — harmful-prompt refusal, thinking ON (`enable_thinking=true`)

	\| Benchmark \| n \| Base FP8 \| This model \|
	\|---\|---:\|---:\|---:\|
	\| AdvBench \| 60 \| 66.7% \| 1.7% \|
	\| JailbreakBench (harmful) \| 60 \| 43.3% \| 0.0% \|
	\| StrongREJECT \| 60 \| 35.0% \| 0.0% \|
	\| HarmBench (standard) \| 60 \| 46.7% \| 0.0% \|
	\| MaliciousInstruct \| 60 \| 83.3% \| 0.0% \|
	\| SimpleSafetyTests \| 50 \| 44.0% \| 0.0% \|
	\| ForbiddenQuestions \| 60 \| 48.3% \| 0.0% \|
	\| Custom probes (zh/en) \| 11 \| 45.5% \| 0.0% \|

	### Over-refusal — benign prompts wrongly refused (lower = better)

	\| Benchmark \| n \| Base FP8 (no-think / think) \| This model (no-think / think) \|
	\|---\|---:\|---:\|---:\|
	\| XSTest-safe \| 250 \| 5.6% / 0.0% \| 0.4% / 0.0% \|

	### Capability retention — vs the official base FP8 (same scripts)

	\| Benchmark \| n \| Base FP8 \| This model \| Δ \|
	\|---\|---:\|---:\|---:\|---:\|
	\| MMLU (all, 0-shot) \| 300 \| 84.3% \| 84.7% \| +0.4 \|
	\| MMLU-Pro (CoT) \| 250 \| 77.6% \| 76.8% \| −0.8 \|
	\| GSM8K (CoT) \| 150 \| 90.0% \| 88.7% \| −1.3 \|
	\| CMMLU (0-shot, Chinese) \| 500 \| 81.4% \| 80.8% \| −0.6 \|
	\| WikiText-2 perplexity \| — \| — \| 6.96 \| fluency sanity check \|

	Harmful-prompt refusal collapses from 64–99% (base) to 0–6%; benign over-refusal drops
	(5.6%→0.4%); capability stays within ±1.3 pts of the base. Reasoning (`enable_thinking`),
	multi-turn tool calling (`qwen3_coder`), and vision (image + OCR via `mmproj`) all verified working on
	the GGUF build. Note: the above are full-precision/FP8 numbers; expect small additional degradation at
	lower quants (most visible at Q2_K / Q3).

	## Hardware

	- Runs on CPU, CUDA, Metal, or ROCm via llama.cpp. VRAM/RAM ≈ the file size + KV cache + (for vision)
	the ~0.9 GB mmproj. E.g. `Q4_K_M` fits comfortably on a 24 GB GPU with room for context.

	## License

	Apache 2.0, inherited from [`Qwen/Qwen3.8-27B`](https://huggingface.co/Qwen/Qwen3.8-27B).
	Abliteration and quantization do not change the underlying license obligations.