Instructions to use SC117/Qwen3.8-27B-Uncensored-FIT-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use SC117/Qwen3.8-27B-Uncensored-FIT-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS
# Run inference directly in the terminal:
llama cli -hf SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS
# Run inference directly in the terminal:
llama cli -hf SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS
# Run inference directly in the terminal:
./llama-cli -hf SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS
# Run inference directly in the terminal:
./build/bin/llama-cli -hf SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS

Use Docker

docker model run hf.co/SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS

LM Studio
Jan

vLLM

How to use SC117/Qwen3.8-27B-Uncensored-FIT-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SC117/Qwen3.8-27B-Uncensored-FIT-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SC117/Qwen3.8-27B-Uncensored-FIT-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS

Ollama
How to use SC117/Qwen3.8-27B-Uncensored-FIT-GGUF with Ollama:
```
ollama run hf.co/SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS
```
Unsloth Desktop

How to use SC117/Qwen3.8-27B-Uncensored-FIT-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use SC117/Qwen3.8-27B-Uncensored-FIT-GGUF with Docker Model Runner:
```
docker model run hf.co/SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS
```

Lemonade

How to use SC117/Qwen3.8-27B-Uncensored-FIT-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS

Run and chat with the model

lemonade run user.Qwen3.8-27B-Uncensored-FIT-GGUF-IQ3_XXS

List all available models

lemonade list

Hermes Agent

How to use SC117/Qwen3.8-27B-Uncensored-FIT-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use SC117/Qwen3.8-27B-Uncensored-FIT-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "SC117/Qwen3.8-27B-Uncensored-FIT-GGUF:IQ3_XXS" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

FIT-GGUF v0.1ZERO-BYTE VERIFIED14 TIERS · 7–13.5 GiBMTP HEAD REMOVEDMEASURED KL + SAME-TOPAPACHE-2.0

Qwen3.8-27B-Uncensored · FIT-GGUF

Fourteen continuous-size GGUF quantizations — every file exactly its predicted size, every tier measured.

7 GiBask for any budget in between13.5 GiB

English · 简体中文 📖

🧭 About FIT-GGUF — the tool behind these files

Every file in this repository was planned, executed and verified by FIT-GGUF, an open-source, deterministic tensor-level planning layer on top of llama.cpp quantization presets. Standard GGUF quantization asks you to pick one of a handful of presets; FIT-GGUF starts from the largest supported preset below your requested byte budget, then spends the remaining bytes on deterministic tensor-level precision upgrades.

Traditional GGUF gives you presets. FIT gives you a size slider.

Deterministic size prediction & recipe execution	✅ Validated
Universally optimal tensor allocation	⚠️ Not established — FIT claims precise size control, not a universal quality optimum

The method, the full preregistered research record and the fit CLI are open source: github.com/Scorp1o117/FIT-GGUF

⚠️ Safety notice / 安全提示

The source model is an abliterated, refusal-removed model with no meaningful built-in guardrails, and may comply with harmful, illegal or unsafe requests. Use it only where you can provide appropriate moderation, access control and legal review. Do not deploy it to end users without your own safety layer.

源模型经过拒答方向移除，不具备可靠的内置安全护栏；请仅在合法、受控、具备审核与滥用防护的环境中使用，使用者自行承担部署责任。

📦 Pick a tier

FIT-12G means a 12 GiB budget for the main GGUF file — not total RAM/VRAM usage. KV cache, compute buffers, runtime overhead and the multimodal projector are separate. G = GiB (2³⁰ bytes). Naming: Qwen3.8-27B-Uncensored-FIT-<tier>-<dominant>.gguf

MTP removed: every quantization in this repository ships without the NextN/MTP head (the source was converted with --no-nextn). MTP-based speculative decoding is therefore not available with these files; text and vision inference are unaffected.

Tier	GiB	Dominant	Macro KL ↓	Same-top ↑	Measured positioning
FIT-7G	7.000	IQ1_M	1.1327	63.9%	Extreme compression; large measured quality loss
FIT-7.5G	7.500	IQ2_XXS	0.5898	73.1%	First major quality step above the IQ1 region
FIT-8G	7.999	IQ2_XXS	0.4838	76.9%	Beats the IQ2_XXS preset (0.5403) with +0.15 GiB
FIT-8.5G	8.499	IQ2_XXS	0.4527	78.4%	Best measured point in the 8–9.3 GiB native region
FIT-9G	8.999	Q2_K	0.3363	80.8%	q2_k-directed fill; large KL step over FIT-8.5G
FIT-9.5G	9.500	Q2_K	0.2737	83.1%	Beats the Q2_K_S preset (0.2889) at slightly less size
FIT-10G	9.999	Q2_K	0.2299	84.6%	Beats the Q2_K preset (0.2439); compact general tier
FIT-10.5G	10.499	IQ3_XXS	0.1873	87.4%	Clear fidelity step over the 10G tier
FIT-11G	10.988	IQ3_S	0.1515	88.9%	Near IQ3_XS macro quality, slightly smaller
FIT-11.5G	11.434	IQ3_S	0.1439	89.2%	Near IQ3_S; documented 67.5 MiB target slack
⭐ FIT-12G	12.000	IQ3_S	0.1227	90.3%	Strongest measured quality/size point in this release
FIT-12.5G	12.497	IQ3_S	0.1116	91.0%	0.5 GiB over FIT-12G buys a clear KL step
FIT-13G	12.998	IQ4_XS	0.0987	91.9%	IQ4_XS becomes the dominant payload
FIT-13.5G	13.498	IQ4_XS	0.0838	92.7%	Best measured FIT-tier macro KL in this batch

Quick picks

🏆 FIT-12G — the sweet spot

Below the IQ3_S / IQ3_M presets (0.1227 vs 0.1424 / 0.1445), far cheaper than IQ4_XS.

🎯 FIT-13.5G — max quality

Highest measured quality in this batch: KL 0.0838, Same-top 92.7%.

💸 FIT-8.5G — budget winner

The 8–10 GiB region winner; beats the IQ2_XXS preset outright.

⚠️ Below ~7.5 GiB

Quality drops sharply — the IQ1 region is rough and reported as measured.

All numbers are protocol-scoped observations (five fixed 64 KiB domains vs aligned BF16), not an application benchmark or a universal ranking.

📈 Measured quality

KL and Same-top curves with all 14 native presets labeled

Measured quality improves monotonically across all 14 tiers (macro KL 1.1327 → … → 0.0838), and in the 8–10 GiB region every FIT tier beats its surrounding llama.cpp default presets: FIT-8G / FIT-8.5G beat IQ2_XXS, FIT-9.5G beats Q2_K_S, FIT-10G beats Q2_K. Superseded early recipes (the P5/P6 repairs) are retained in the research record as evidence, not hidden.

Full-size KL · 中文大图 · Same-top · Allocation repair · Target utilization

🚀 Run it

llama.cpp (text)

./llama-cli \ -m Qwen3.8-27B-Uncensored-FIT-12G-IQ3_S.gguf \ -ngl 99 \ -c 8192 \ -cnv

llama-server (text + vision)

./llama-server \ -m Qwen3.8-27B-Uncensored-FIT-12G-IQ3_S.gguf \ --mmproj mmproj-Qwen3.8-27B-Uncensored-BF16.gguf \ -ngl 99 -c 8192

Any llama.cpp-based runner (llama-cli, llama-server, LM Studio, KoboldCpp, Jan, …) loads these files directly. The BF16 projector pairs with any tier. Pick -ngl, context and batch for your hardware — and remember the GGUF file size alone is not a RAM/VRAM requirement calculator.

🔬 Evaluation protocol & honest scope

Runtime	llama.cpp b10666 (commit 4e97ac86e) · Linux x86_64 · ROCm
Command shape	`llama-perplexity -ngl 99 -t 16 -c 512 -b 512 --kl-divergence ...`
Reference	Aligned BF16 logits from the converted source
Domains	wiki_test · wiki_valid · Chinese · code · agent_chat (five fixed 64 KiB slices, macro mean)

Size accuracy: all 14 artifacts matched their post-oracle predicted byte sizes exactly; most use >99.98% of the requested target (FIT-11.5G: 99.427%, the 67.5 MiB reported as target slack — llama.cpp counter-based preset rules shift when manual overrides bypass parts of preset selection; the planner detects this via an override-aware dry-run oracle).

Allocator scope: the balanced v0.1b policy has positive holdout evidence on the development architecture at some budgets but did not beat matched random allocation on a second model family. This release claims deterministic target-size planning and reports measured quality for these specific artifacts — it does not claim a universally optimal allocation.

🧩 Included — and not included

✅ 14 quantized main language-model GGUFs · ✅ BF16 multimodal projector mmproj-Qwen3.8-27B-Uncensored-BF16.gguf (vision, pairs with any tier) · per-tier plan, effective recipe, tensor override file and quantize record (fit-plans/) · full metrics (results/p4-results.json) · checksums (results/SHA256SUMS)

❌ The auxiliary NextN/MTP head was excluded from the source conversion (--no-nextn).

The original abliteration belongs to OrcaRouter; this repository contributes the FIT quantization plans and artifacts only.

🔍 Verify & reproduce

sha256sum -c results/SHA256SUMS

Evaluation slices, evaluation logs and the complete decision record live in the FIT-GGUF repository. Exact-size behavior is scoped to the recorded source metadata and pinned llama.cpp build; changing the converter, runtime, source layout or metadata requires revalidation.

📄 License & credits

Apache-2.0, inherited from the base model — follow the upstream license and model-card requirements.

OrcaRouter — abliterated BF16 source weights · Qwen — the original model family · llama.cpp — GGUF tooling and runtime · Unsloth calibration dataset lineage — imatrix (1,251 chunks, reused from the same-architecture lineage). FIT-GGUF is an independent project, not affiliated with Qwen, Alibaba, OrcaRouter or llama.cpp.

⭐ FIT-GGUF on GitHub — the tool, the method, the full research record

English · 简体中文