Instructions to use jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS
# Run inference directly in the terminal:
llama cli -hf jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS
# Run inference directly in the terminal:
llama cli -hf jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS
# Run inference directly in the terminal:
./llama-cli -hf jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS
# Run inference directly in the terminal:
./build/bin/llama-cli -hf jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS

Use Docker

docker model run hf.co/jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS

LM Studio
Jan

vLLM

How to use jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS

Ollama
How to use jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF with Ollama:
```
ollama run hf.co/jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS
```
Unsloth Desktop

How to use jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF with Docker Model Runner:
```
docker model run hf.co/jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS
```

Lemonade

How to use jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS

Run and chat with the model

lemonade run user.Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF-IQ4_XS

List all available models

lemonade list

Hermes Agent

How to use jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF:IQ4_XS" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwopus3.8-27B-Flash ATX-IQ4_XS-M (GGUF)

The ATX-IQ4_XS-M recipe applied to Jackrong/Qwopus3.8-27B-Flash, a fine-tune of Qwen3.8-27B tuned for shorter reasoning and higher MTP acceptance in agent loops. Same architecture, same 27.32B parameters, same 866 tensors as the base model, so the recipe carries over unchanged; what is new here is a model-specific importance matrix.

The recipe was designed for a single RTX 3090 / 3090 Ti (24 GB): fit a populated 200K-token context with an 8-bit key cache and the model's own MTP speculative head, and decode faster per speculative round than the stock Q4 mixes on that card. The original build and its measurements are at jakeatx/Qwen3.8-27B-ATX-IQ4_XS-M-GGUF.

file	size	notes
`Qwopus3.8-27B-Flash-ATX-IQ4_XS-M.gguf`	14.6 GiB (15,724,533,952 bytes)	4.60 bits per weight, MTP layer included
`imatrix_qwopus_flash.gguf`	13.6 MB	importance matrix computed from this model, see below
`tensor_types_ATX-IQ4_XS-M.txt`		the per-tensor type map used by `llama-quantize`

On the name. ATX-IQ4_XS-M reads as: base format IQ4_XS on the bulk tensors, upgrade pattern M. In llama.cpp's vocabulary the XS in IQ4_XS is part of the format's name (the 4.25 bits-per-weight super-block layout of the IQ4 codebook), not a mix size, while the S/M/L suffix on the K-quants describes how many tensors are lifted above the base format. This file lifts the same tensors Q4_K_M does, plus the early-layer tensors Jackrong's Q4_K_S lifts, so it is an M-pattern mix on an IQ4_XS base: 4.60 bits per weight, between Q4_K_S (4.63 for this model) and Q4_K_M (4.92) in size.

Recipe

Built from the Qwopus safetensors converted to a BF16 GGUF with llama.cpp's converter (MTP layer kept), quantized with the per-tensor type map:

tensors	format	share of weight bytes
everything not listed below	IQ4_XS	~68%
attn_output, ssm_out, ffn_down in the layers Unsloth's tier ladder upgrades first	Q5_0	~17%
attention K/V projections, output head	Q6_K	~7%
token embedding (host side)	Q4_K	~5%
the eight attention K/V tensors Q4_K_M keeps at Q8_0 (V in layers 11, 27, 31, 51, 55, 59, 63; K in 31)	Q8_0
FFN down in layers 0-7 and GDN qkv in layers 0-2 (the tensors Jackrong's Q4_K_S lifts to Q5_K)	Q5_0	~3%
MTP draft layer (blk.64)	Q5_0
GDN alpha / beta vectors (96 tiny)	Q8_0

llama-quantize --imatrix imatrix_qwopus_flash.gguf --tensor-type-file tensor_types_ATX-IQ4_XS-M.txt \
               --token-embedding-type q4_K Qwopus3.8-27B-Flash-BF16.gguf Qwopus3.8-27B-Flash-ATX-IQ4_XS-M.gguf iq4_xs

Every tensor's type was checked against the base ATX-IQ4_XS-M file after quantization: identical except the eleven early-layer tensors above, which this build carries at Q5_0 so that every tensor Jackrong's own mix protects stays protected here.

Importance matrix. Rather than reuse the Unsloth base-model imatrix, this build computes its own from Qwopus at Q8_0 precision over about 226K tokens: Bartowski's calibration_datav3 (mixed prose, code, multilingual, chat) plus 64K-token agentic and coding prompts and a 32K-token retrieval-analysis prompt from the production prompt set the recipe was tuned on. A fine-tune shifts activation statistics; measuring them on the model being quantized is the more faithful translation of the recipe.

Why this mix: on SM86 the fastest weight format per tensor at speculative verification widths 1-5 is IQ4_XS, not the 2-3 bit codebook types (IQ3_S, IQ3_XXS, IQ2_S are instruction-bound and slower despite fewer bytes); Q5_0 is about 16% cheaper than Q5_K; Q8_0 is the only format near the memory roof. Extra bits go where Unsloth's tier ladder puts them (attention V/K, attention output, GDN output, FFN down).

Measured (RTX 3090 Ti, TurboQuant+ fork with SM86 kernel work, MTP-3, q8_0 K / turbo3 V)

Same prompts for both files, sampled decode at temperature 1, top-k 20, top-p 0.95, 256-token generations, two runs per cell (second run with the prompt cached). Compared against Jackrong's Q4_K_S, the like-for-like size class (14.7 GiB, 4.63 bpw).

context	workload	ATX-IQ4_XS-M decode tok/s	Q4_K_S decode tok/s	ratio	prefill tok/s (ATX / Q4_K_S)
16K	coding	71.9	59.7	1.20	1378 / 1298
16K	agentic	78.8	70.3	1.12	1393 / 1388
16K	RAG analysis	83.8	67.4	1.24	1385 / 1377
32K	coding	72.7	64.8	1.12	1290 / 1288
32K	agentic	77.1	65.4	1.18	1286 / 1283
32K	RAG analysis	76.7	60.2	1.27	1285 / 1281
64K	coding	66.2	55.1	1.20	1099 / 1099
64K	agentic	68.8	53.0	1.30	1095 / 1103
64K	RAG analysis	67.1	57.9	1.16	1095 / 1104
100K	coding	56.2	57.1	0.98	934 / 940
100K	agentic	57.9	55.7	1.04	933 / 940
100K	RAG analysis	52.6	48.3	1.09	931 / 938
16K mean	all three	78.1	65.8	1.19
32K mean	all three	75.5	63.5	1.19
64K mean	all three	67.4	55.3	1.22
100K mean	all three	55.6	53.7	1.04

Decode is the mean of the two runs per cell; the ratio is ATX-IQ4_XS-M over Q4_K_S. Ready VRAM at a 110,592-token window: about 19,910 MiB (ATX-IQ4_XS-M) against 19,879 MiB (Q4_K_S). The two files prefill at the same rate. Sampled decode swings with draft acceptance (65-80% per cell here), and a few generations stopped early on their own (the model finished its answer before 256 tokens), which makes the 100K cells the noisiest: read the picture as roughly +20% decode from 16K to 64K, narrowing to a few percent at 100K, where attention over the long cache takes a growing share of each round and the weight format matters less. Same runtime, same prompts, same session for both files; Jackrong's Q4_K_S was measured as downloaded.

sha256 of the GGUF: 733198976d76de46326f9d685533827551d14cdef46f03c2a4d6255f42df76a2.

Run it

Runtime with the kernel and memory work: https://github.com/JakeATX/llama-cpp-qwen-ampere, branch main. Build with -DGGML_CUDA=ON -DGGML_CUDA_FA=ON -DCMAKE_CUDA_ARCHITECTURES=86. The file also loads on stock TurboQuant+ and on mainline llama.cpp (without the turbo3 cache there), with less context headroom.

GGML_Q8_TURBO3_MMA_FUSED=1 llama-server -m Qwopus3.8-27B-Flash-ATX-IQ4_XS-M.gguf \
  -c 245760 -b 4096 -ub 1024 -t 8 -tb 8 -ngl 99 -fa on -ctk q8_0 -ctv turbo3 \
  --parallel 1 --jinja --fit off \
  --cache-prompt --cache-ram 8192 --ctx-checkpoints 24 --checkpoint-min-step 10240 \
  --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.45 \
  --spec-draft-type-k q8_0 --spec-draft-type-v turbo3

Single-user configuration (one request at a time). The cache flags are explained on the base model's card; they keep long conversations from re-prefilling and let you edit or regenerate a turn on this partly recurrent architecture. Qwopus ships its own chat template and sampling defaults (temperature 1, top-p 0.95, top-k 20), which the GGUF carries.

Quality was not re-measured for this build; the recipe's quality evidence is on the base model's card and in its paired Terminal-Bench / SciCode comparison against Q4_K_M. Qwopus's own capability trade-off (shorter reasoning, lower MMLU-Pro than the base) is described on Jackrong's card.

Credits

Qwen team for Qwen3.8; Jackrong for Qwopus3.8-27B-Flash; Unsloth for the dynamic-quant tier ladder this recipe follows; Bartowski for the calibration set; TheTom for TurboQuant+.

Qwen3.8 Perian project lineage

This repository is retained in the Qwen3.8 Perian checkpoints collection. Its exact position in the lineage is: Baseline/comparator quant retained with the project checkpoints. It is not a Perian-pruned or Perian-QLoRA artifact.

The final Qwen3.8 Perian GGUF release combines three reductions and one post-training stage:

depth: 48 to 32 transformer layers;
routed-expert width: 384 to 288 experts per layer;
PLE n-gram capacity: 320,001,446 to 160,000,768 rows (50%, about 25.60B parameters removed), using activation-aware bigram and frequency-ranked trigram selections validated on a document-disjoint 5M-token holdout;
rank-32 QLoRA on 12,558 normalized traces spanning math/STEM reasoning, coding/debugging, agentic tool use, retrieval, and general multi-step reasoning. The trace mixture draws from several frontier-model families, including Fable 5, GLM 5.2, Kimi K3, Claude Opus 4.7, Qwen3.8-Max, and GPT-5.6-Sol. The final merged milestone was trained through 9,336,692 supervised assistant tokens.

Earlier checkpoints in this collection do not inherit later stages merely by being listed beside them; the stage statement above is authoritative for this artifact.