Mirror orcarouter/Qwen3.8-27B-Uncensored-GGUF@b33b15636fce4621fd1c3499bb5c9681465fb626

.gitattributes

@@ -33,3 +33,21 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-Uncensored-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-Uncensored-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-Uncensored-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-Uncensored-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-Uncensored-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-Uncensored-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+mmproj-Qwen3.8-27B-Uncensored-f16.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-Uncensored-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-Uncensored-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-Uncensored-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-Uncensored-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-Uncensored-F16-00001-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-Uncensored-F16-00002-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-Uncensored-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-Uncensored-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-Uncensored-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-Uncensored-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-Uncensored-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text

Qwen3.8-27B-Uncensored-F16-00001-of-00002.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:578926d4e6d94281e95a48d8e154c4667061a669e9016f2aa2b15101ab4363dc
+size 27908108288

Qwen3.8-27B-Uncensored-F16-00002-of-00002.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c15e78454caec46b19dae22fa6915a9e77b917e7cceffdc64c265801f4eaa1f1
+size 26749625920

Qwen3.8-27B-Uncensored-IQ2_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b7e77fd57bb9bd0162bb80b839e3484ec06af3e9016f456e762c73fe9b570f9
+size 10455914080

Qwen3.8-27B-Uncensored-IQ2_XXS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c3ac5d52238b28dce25b1fe4b4e759ca0f51b89864d4e2b6ea1e8135716ba136
+size 8884504160

Qwen3.8-27B-Uncensored-IQ3_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:045ca1b0396bda8591e8b6f8c616c54ab2e6c4e0aea0275e5107d1eccf6204ec
+size 12768331360

Qwen3.8-27B-Uncensored-IQ3_XXS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d91ad7b2c57f545a030b1d391375a5f9a822423b90923d91cf744e717e80d03
+size 11637692000

Qwen3.8-27B-Uncensored-IQ4_XS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c52df3e4289da6e24aa20665f2a88e9347aa1ce08f2bfe0a58cc9e578845e0af
+size 15309039200

Qwen3.8-27B-Uncensored-Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:85f17d3779fbc16507ebdc8efab9f7cbe42a7a30488f4029179a1c5471e0c088
+size 10864592256

Qwen3.8-27B-Uncensored-Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9572d9ea6c180dac37b027987476ee5cc93e12946f7b242c8dfa1cdc0f7842d
+size 14559798656

Qwen3.8-27B-Uncensored-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2bdecffabba9fedd93f383ce834b2d185db32a7fa0cc6cddbbadcf56097cd4ae
+size 13500736896

Qwen3.8-27B-Uncensored-Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a9f07e69b0f15942a028461aca88361294fa2a40a845b1648096cfad668cdffc
+size 12256535936

Qwen3.8-27B-Uncensored-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3445102e9cde5d562508642c100a2f5ac3368a5a3f748442811d7a95daee3bec
+size 16810714496

Qwen3.8-27B-Uncensored-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a9f8fe94324af5c8bf5fd4f0be3d24424c914707d399c514733710375c3cb6fa
+size 15825298816

Qwen3.8-27B-Uncensored-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e42ef33bcf9974730119fa59ff6b065fdd52080a664728afa0902cb358a301b
+size 19535701376

Qwen3.8-27B-Uncensored-Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:650b40ba96e1993923293d2db9a8697e13a8e37867b6c3586c97d896905ed24a
+size 18971682176

Qwen3.8-27B-Uncensored-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a0d6fe2e48417f1571a57a7104b4d5da0c45c3d00de18fa7aa48affca5a24d7
+size 22430999936

Qwen3.8-27B-Uncensored-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:31756fca94beca71ea4b8706d6fdc896dab2a3c6376ab0c1863b98512a24f8d6
+size 29047084416

README.md

@@ -0,0 +1,245 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen3.8-27B
+base_model_relation: quantized
+pipeline_tag: image-text-to-text
+library_name: gguf
+language:
+  - en
+  - zh
+tags:
+  - abliterated
+  - qwen
+  - qwen3
+  - qwen3.8
+  - gguf
+  - llama.cpp
+  - uncensored
+  - ai-red-team
+  - red-teaming
+  - vision-language
+  - mmproj
+  - mtp
+  - function-calling
+  - reasoning
+---
+
+<div align="center">
+
+<a href="https://www.orcarouter.ai" target="_blank">
+  <img src="https://www.orcarouter.ai/orca-logo-classic.png" alt="OrcaRouter" width="110">
+</a>
+
+<h1>Qwen3.8-27B-Uncensored-GGUF</h1>
+
+<p><em>GGUF quants (2-bit → 16-bit) of the abliterated (refusal-removed) Qwen3.8-27B — for llama.cpp</em></p>
+
+<p>
+<a href="https://www.orcarouter.ai"><img src="https://img.shields.io/badge/Website-orcarouter.ai-1E6FEB" alt="Website"></a>
+<a href="https://www.orcarouter.ai/models"><img src="https://img.shields.io/badge/OrcaRouter-Model%20Catalog-2EA043" alt="Model Catalog"></a>
+<a href="https://www.orcarouter.ai/models/qwen/qwen3.8-27b"><img src="https://img.shields.io/badge/OrcaRouter-Model%20Card-6F42C1" alt="Model Card"></a>
+<a href="https://www.apache.org/licenses/LICENSE-2.0"><img src="https://img.shields.io/badge/License-Apache%202.0-4C8BF5" alt="License"></a>
+<img src="https://img.shields.io/badge/Format-GGUF-00A67E" alt="GGUF">
+<img src="https://img.shields.io/badge/Quants-Q2__K%20%E2%86%92%20F16-FF8800" alt="Quants">
+<img src="https://img.shields.io/badge/Vision-mmproj-9B59B6" alt="Vision">
+</p>
+
+<p><strong>One Gateway. Every Model.</strong> — Route Smarter · Ship Safer · Spend Less.</p>
+
+<p>
+<a href="https://www.orcarouter.ai">Website</a> ·
+<a href="https://www.orcarouter.ai/models">Model Catalog</a> ·
+<a href="https://www.orcarouter.ai/models/qwen/qwen3.8-27b">Model Card</a> ·
+<a href="https://github.com/Continuum-AI-Corp">GitHub</a> ·
+<a href="https://ollama.com/orcarouter">Ollama</a> ·
+<a href="https://discord.gg/yAh6Tex6kx">Discord</a> ·
+<a href="https://x.com/OrcaRouter">X</a>
+
+</p>
+
+</div>
+
+---
+
+> **GGUF conversions** of [`Qwen3.8-27B-Uncensored`](https://huggingface.co/orcarouter/Qwen3.8-27B-Uncensored-FP8)
+> — an **abliterated** (refusal-removed) build of Qwen's `Qwen3.8-27B`, a 27B dense hybrid-attention
+> (Gated DeltaNet linear + full attention) native vision-language model with reasoning, tool-calling,
+> and an MTP speculative-decoding head. These files run in **llama.cpp** (CPU / CUDA / Metal / ROCm),
+> quantized from **2-bit to 16-bit**, with a separate **mmproj** file that restores **vision**.
+> Browse all models in the [OrcaRouter Model Catalog](https://www.orcarouter.ai/models). Qwen3.8 27B is
+> deployed as API [on OrcaRouter](https://www.orcarouter.ai/models/qwen/qwen3.8-27b).
+
+---
+
+## ⚠️ Disclaimer — read before use
+
+This model has had its **safety alignment substantially removed** via *abliteration* (orthogonalizing
+the refusal direction out of the residual stream). It will **comply with harmful, unethical, or illegal
+requests** the original `Qwen3.8-27B` would refuse. Released **strictly for legitimate research** —
+interpretability, AI-safety / refusal-mechanism study, red-teaming, and robustness evaluation. **You
+assume full responsibility** for how you use it and everything it generates; add your own safety and
+moderation layers before any deployment. Use must comply with the
+[Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0) inherited from the base model and all
+applicable law. The authors accept **no liability** for misuse.
+
+---
+
+## Requirements
+
+- **A recent [llama.cpp](https://github.com/ggml-org/llama.cpp)** built from source (the `qwen35`
+  hybrid-GDN architecture and the **MTP / `nextn`** speculative head — merged 2026-05 — must be present).
+  Older releases will not load these files.
+- The GDN linear-attention layers are stored as SSM-style tensors (`ssm_*`); full-attention layers as
+  `attn_*`; the MTP head as block `nextn.*` (`qwen35.nextn_predict_layers`).
+
+## Files
+
+### Standard K-quants
+
+| File | Bits | Size | Notes / recommendation |
+|---|---|---:|---|
+| `…-Q2_K.gguf`  | 2-bit | 10.9 GB | Smallest K-quant; noticeable quality drop — low-VRAM only |
+| `…-Q3_K_S.gguf`| 3-bit | 12.3 GB | |
+| `…-Q3_K_M.gguf`| 3-bit | 13.5 GB | Good small option |
+| `…-Q3_K_L.gguf`| 3-bit | 14.6 GB | |
+| `…-Q4_K_S.gguf`| 4-bit | 15.8 GB | |
+| **`…-Q4_K_M.gguf`** | 4-bit | 16.8 GB | **Recommended default** — best quality/size balance |
+| `…-Q5_K_S.gguf`| 5-bit | 17.7 GB | |
+| `…-Q5_K_M.gguf`| 5-bit | 18.2 GB | High quality |
+| `…-Q6_K.gguf`  | 6-bit | 20.9 GB | Very high quality |
+| `…-Q8_0.gguf`  | 8-bit | 27.1 GB | Near-lossless |
+| `…-F16-0000*-of-00002.gguf` | 16-bit | 54.7 GB | Full precision (split into 2 parts; point llama.cpp at part 00001) |
+
+### IQ quants (imatrix)
+
+Lower-bit quants built with an **importance matrix** (computed on English + Chinese calibration
+text) — better quality-per-bit than plain K-quants at the low end, especially IQ3/IQ2.
+
+| File | Bits | Size | Notes / recommendation |
+|---|---|---:|---|
+| **`…-IQ4_XS.gguf`** | ~4.25-bit | 15.3 GB | **Best low-bit pick** — ≈ Q4_K_S quality at smaller size |
+| `…-IQ3_M.gguf`   | ~3.7-bit | 12.8 GB | Solid 3-bit |
+| `…-IQ3_XXS.gguf` | ~3.1-bit | 11.6 GB | Smaller 3-bit |
+| `…-IQ2_M.gguf`   | ~2.7-bit | 10.5 GB | Runs in low VRAM; some quality loss |
+| `…-IQ2_XXS.gguf` | ~2.1-bit | 8.9 GB  | Smallest runnable; most degraded |
+
+### Vision
+
+| File | Size | Notes |
+|---|---:|---|
+| **`mmproj-…-f16.gguf`** | 0.9 GB | **Vision projector — download this too for image input** |
+
+All quants (K-quant and IQ) preserve the **MTP (`nextn`) head** and the **GDN hybrid architecture**;
+vision is provided by the separate `mmproj` file. The **IQ** files were quantized with an importance
+matrix (computed on English + Chinese calibration text) for better low-bit fidelity; the matrix
+itself is not shipped, as it is only needed to re-quantize these files, not to run them.
+
+## Usage (llama.cpp)
+
+### Download
+
+```bash
+hf download orcarouter/Qwen3.8-27B-Uncensored-GGUF \
+  Qwen3.8-27B-Uncensored-Q4_K_M.gguf mmproj-Qwen3.8-27B-Uncensored-f16.gguf \
+  --local-dir ./qwen38-uncensored
+```
+
+### Chat (text)
+
+```bash
+./llama-cli -m Qwen3.8-27B-Uncensored-Q4_K_M.gguf --jinja -c 8192 -p "Hello!"
+```
+
+### OpenAI-compatible server (tool calling + reasoning + vision)
+
+```bash
+./llama-server -m Qwen3.8-27B-Uncensored-Q4_K_M.gguf \
+  --mmproj mmproj-Qwen3.8-27B-Uncensored-f16.gguf \
+  --host 0.0.0.0 --port 8000 -c 8192 --jinja
+```
+
+- **Vision:** pass `--mmproj …`, then send OpenAI `image_url` content parts (base64 data-URI or URL).
+- **Tool calling:** `--jinja` enables the Qwen tool template; use standard OpenAI `tools` + `tool_calls`.
+- **Reasoning (thinking):** thinking is on by default; toggle per request via
+  `chat_template_kwargs.enable_thinking`. The reasoning trace is returned in `reasoning_content`
+  (give `max_tokens` room, e.g. ≥ 2048, so the final answer isn't truncated by the thinking budget).
+- **MTP speculative decoding:** the `nextn` head is embedded; enable llama.cpp's MTP/speculative path
+  for a decode speed-up (optional — the model runs fine without it).
+
+### Or skip llama.cpp — run it with Ollama
+
+The same quants are published as Ollama tags, with the `mmproj` projector already bundled in (no second
+file, no `--mmproj` flag) and vision / tools / thinking wired up:
+
+```bash
+ollama run orcarouter/Qwen3.8-27B-Uncensored          # q4_K_M by default
+ollama run orcarouter/Qwen3.8-27B-Uncensored:iq4_xs   # any of the 16 tags
+```
+
+Sixteen tags from `q2_K` to `q8_0`, each verified on the actual quantized artifact before release —
+[ollama.com/orcarouter](https://ollama.com/orcarouter).
+
+## Evaluation
+
+Measured on this exact abliterated build (served with vLLM) vs the official `Qwen/Qwen3.8-27B-FP8`,
+using the same scripts and settings. Refusal is judged by a rule-based opening-phrase classifier —
+indicative, not an LLM-judge / publication-grade number. GGUF quants are deterministic derivations and
+inherit these behaviours; lower bits trade some quality (see the file table).
+
+### Safety — harmful-prompt refusal, **thinking OFF** (lower = more uncensored)
+
+| Benchmark | n | Base FP8 | **This model** |
+|---|---:|---:|---:|
+| AdvBench | 100 | 99.0% | **0.0%** |
+| JailbreakBench (harmful) | 100 | 94.0% | **0.0%** |
+| StrongREJECT | 150 | 97.3% | **2.0%** |
+| HarmBench (standard) | 150 | 98.7% | **2.7%** |
+| MaliciousInstruct | 100 | 99.0% | **0.0%** |
+| SimpleSafetyTests | 50 | 64.0% | **6.0%** |
+| ForbiddenQuestions | 150 | 73.3% | **4.7%** |
+| Custom probes (zh/en) | 11 | 63.6% | **0.0%** |
+
+### Safety — harmful-prompt refusal, **thinking ON** (`enable_thinking=true`)
+
+| Benchmark | n | Base FP8 | **This model** |
+|---|---:|---:|---:|
+| AdvBench | 60 | 66.7% | **1.7%** |
+| JailbreakBench (harmful) | 60 | 43.3% | **0.0%** |
+| StrongREJECT | 60 | 35.0% | **0.0%** |
+| HarmBench (standard) | 60 | 46.7% | **0.0%** |
+| MaliciousInstruct | 60 | 83.3% | **0.0%** |
+| SimpleSafetyTests | 50 | 44.0% | **0.0%** |
+| ForbiddenQuestions | 60 | 48.3% | **0.0%** |
+| Custom probes (zh/en) | 11 | 45.5% | **0.0%** |
+
+### Over-refusal — benign prompts wrongly refused (lower = better)
+
+| Benchmark | n | Base FP8 (no-think / think) | **This model** (no-think / think) |
+|---|---:|---:|---:|
+| XSTest-safe | 250 | 5.6% / 0.0% | **0.4% / 0.0%** |
+
+### Capability retention — vs the official base FP8 (same scripts)
+
+| Benchmark | n | Base FP8 | **This model** | Δ |
+|---|---:|---:|---:|---:|
+| MMLU (all, 0-shot) | 300 | 84.3% | **84.7%** | **+0.4** |
+| MMLU-Pro (CoT) | 250 | 77.6% | **76.8%** | −0.8 |
+| GSM8K (CoT) | 150 | 90.0% | **88.7%** | −1.3 |
+| CMMLU (0-shot, Chinese) | 500 | 81.4% | **80.8%** | −0.6 |
+| WikiText-2 perplexity | — | — | **6.96** | fluency sanity check |
+
+Harmful-prompt refusal collapses from **64–99%** (base) to **0–6%**; benign over-refusal drops
+(5.6%→0.4%); capability stays within **±1.3 pts** of the base. Reasoning (`enable_thinking`),
+multi-turn tool calling (`qwen3_coder`), and vision (image + OCR via `mmproj`) all verified working on
+the GGUF build. Note: the above are full-precision/FP8 numbers; expect small additional degradation at
+lower quants (most visible at Q2_K / Q3).
+
+## Hardware
+
+- Runs on CPU, CUDA, Metal, or ROCm via llama.cpp. VRAM/RAM ≈ the file size + KV cache + (for vision)
+  the ~0.9 GB mmproj. E.g. `Q4_K_M` fits comfortably on a 24 GB GPU with room for context.
+
+## License
+
+**Apache 2.0**, inherited from [`Qwen/Qwen3.8-27B`](https://huggingface.co/Qwen/Qwen3.8-27B).
+Abliteration and quantization do not change the underlying license obligations.

mmproj-Qwen3.8-27B-Uncensored-f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:add205b7bfdb3f71f6da36b0a82aa20928dd829a920878c602628cdfbebc5288
+size 931145984