Qwen3.8-Flash-Next-Abliterated-MLX-4bit

A high quality abliterated 4-bit build of Qwen3.8-Flash-Next, optimized for Apple Silicon with oMLX.

Lineage: Qwen/Qwen3.8-Flash-Next → Blackfrost-AI/Qwen3.8-Flash-Next-DERISKED-BF16 (abliterated) → this build (4-bit oQ4e).

The vision tower and the MTP (multi-token-prediction) draft head are both preserved and working, so the build retains image input and speculative decoding abilites. Quantized with oMLX to oQ4e: iMatrix-calibrated mixed precision, with per-tensor bit widths rather than a uniform 4 bits.

	This build: 4-bit
Thinking off, MTP on	58.1 tok/s at short context, temp 0
Thinking-xHigh, MTP on	52.4 tok/s at short context, temp 0
Without MTP	26.5 tok/s, flat across every setting
MTP acceptance	71.3% draft acceptance, 2.47 tokens per cycle
Abliteration (upstream BF16, Qwentium persona)	2.3% refusal on harmful prompts (7/300), 2.2% full suite (10/450) *

Abliteration is measured by Blackfrost-AI on the parent BF16 checkpoint with the Blackfrost-AI "Qwentium" persona active. This 4-bit build has yet to be re-measured after quantization, and ships with the stock Qwen persona rather than the "Qwentium" persona. The upstream Blackfrost-AI "Qwentium" persona is included as an optional swap to further reduce refusals; see Personas. Prior work on Qwen3.8-27B-Abliterated-MTPLX showed Blackfrost-AI abliteration holding at the weight level with the persona removed, with refusal rate of 8.7% at 1 attempt, 4.2% within 3 attempts, and 1.7% with the Blackfrost persona. See Abliteration and refusal surface.

download size	106 GB across 23 shards
resident when serving	~106 GiB
tensors	3748 (76 MTP · 333 vision)
base model	125B parameters, ~6B active per token (MoE, 10 of 512 experts)
context	262,144 tokens

Note: SSD offload of the n-gram table brings it to roughly 76 GB needed in unified memory.

⚠️ Intended use and terms

Published for experimental AI research and AI-safety research only: red-teaming, refusal and alignment research, interpretability, and quantization or inference research.

It must not be used for anything illegal, in any jurisdiction, under any circumstances. The removal of refusal behavior is a research property of the artifact, not an invitation, a permission, or an endorsement. Nothing about publishing this model condones or excuses illegal activity.

You are solely responsible for your use of it and for complying with applicable laws and upstream licence terms. It ships with essentially no safety refusals: assume it will comply with any instruction, including harmful ones. Run it locally on hardware you control. Do not expose it as a public endpoint or deploy it to untrusted users.

---

Performance

Measured on an M3 Ultra Mac Studio (256 GB) using oMLX. Generation is capped at a fixed token count with prompts that overrun the cap, so every run generates exactly that many tokens and rates are directly comparable.

Decode, tokens/second. Same prose prompt throughout, so the rows differ only in the setting named.

Reasoning xhigh (the default when reasoning_effort is unset):

temp	generated	MTP off	MTP on	speedup
0	512	26.9	52.4	1.95x
0	1024	26.7	52.3	1.96x
1	512	26.6	40.2	1.51x
1	1024	26.5	38.2	1.44x

Reasoning off (enable_thinking: false):

temp	generated	MTP off	MTP on	speedup
0	512	26.9	54.8	2.04x
0	1024	26.7	52.0	1.95x
1	512	26.6	49.4	1.86x
1	1024	26.5	50.1	1.89x

On a technical-explanation prompt, which is more predictable and so accepts more drafts, MTP reaches 57.3 tok/s (512) and 58.1 tok/s (1024) with reasoning off at temperature 0, against the same ~26.7 baseline: a 2.13–2.18x speedup.

Decode without MTP is flat at 26.5–26.9 tok/s regardless of reasoning mode, temperature or length. All the variation comes from MTP, because the speedup depends on how predictable the text is. Temperature 1 costs the most: it makes the draft head wrong more often, so acceptance and speedup both fall, and it costs nothing measurable with MTP disabled.

Only xhigh was benchmarked. medium and low were not measured, and since reasoning effort governs how many reasoning tokens get generated, they will change time-to-answer substantially even though decode rate itself is flat.

Prefill runs ~145–225 tok/s on short prompts and ~815 tok/s on 1000-token prompts, since fixed per-request overhead amortizes over more tokens.

MTP draft head:

acceptance	71.3% (17,965 of 25,202 drafts, 42 requests)
by depth	d1 72.6% · d2 69.6% · d3 70.5%
tokens per backbone cycle	2.47
speedup over MTP disabled	~2x

peak resident memory	~106 GiB serving
time to first token	~0.5 s on a short prompt

Temperature 1 costs roughly 25% of decode throughput when MTP is enabled, because sampling makes the draft head's guesses wrong more often and acceptance falls. With MTP disabled, temperature is nearly free.

---

Abliteration and refusal surface

Refusal behaviour was removed at the weight level upstream BF16 by Blackfrost-AI. Their reported figures on R1-HARMFUL-BENCH-450:

Evaluation slice	Final judged refusals
Harmful prompts	7 / 300 (2.3%)
Full suite	10 / 450 (2.2%)
API errors	0 / 450

* These values were measured by Blackfrost-AI on the parent BF16 checkpoint with the Qwentium persona active. They are not measurements of this build: this is a 4-bit quantization that ships with the stock Qwen template by default, and neither quantization effects nor the stock-template configuration were evaluated for refusal behaviour here.

Refusals have not been re-measured for this build. The configuration that actually ships here, abliterated weights with the stock Qwen template and no persona, is exactly the combination neither of the tables above covers.

As the closest available proxy, the sibling Qwen3.8-27B-Abliterated-MTPLX-4bit was measured in both configurations on mlabonne/harmful_behaviors (n=520), single attempt:

Reasoning	Stock template (no persona)	With persona
on	8.7%	1.7%
off	29.4%	3.8%

Two things that suggests, both unverified for this model. The weights do most of the work with reasoning on: 8.7% refusal on the stock template means compliance is a property of the weights, not of prompt framing. And the persona matters far more with reasoning off, where it moves 29.4% to 3.8%. So if you are running with enable_thinking: false and want minimum refusals, the persona swap is worth more than it is with reasoning on.

That is a different model and a different quantization. Do not treat these as figures for this build.

---

Personas

Ships with the stock Qwen chat template as the default, which injects no system prompt at all. The abliterated weights carry the behaviour on their own.

The upstream Blackfrost "Qwentium" persona is included as an opt-in. It is a ~1,200 token compliance-oriented system prompt prepended to every request; on the sibling 27B build it reduced refusals by roughly a further 5 percentage points over the abliterated weights alone, so it is a marginal boost rather than the mechanism.

Switch with the bundled script, then restart your server:

bash switch_persona.sh blackfrost   # persona on
bash switch_persona.sh qwen         # back to stock (default)
bash switch_persona.sh status       # which is active

Run it from the model directory. It is safe against an hf download cache checkout: it replaces symlinks rather than writing through them, so the shared blob store is left intact.

The template lives in two places, chat_template.jinja and the chat_template key inside tokenizer_config.json, and different runtimes read different ones, so the script switches both. By hand, copy both files of a pair:

persona	files
qwen (default)	chat_template_qwen.jinja · tokenizer_config_qwen.json
blackfrost	chat_template_blackfrost.jinja · tokenizer_config_blackfrost.json

With the Blackfrost template active you can substitute your own text without editing files:

{"chat_template_kwargs": {"blackfrost_system_prompt": "your prompt here"}}

The persona cannot be switched off from within that template, only replaced: an empty override falls back to the built-in persona. That is why stock is the default here and the script exists.

Switching changes prompt formatting only. The weights are in the .safetensors files and are byte-identical either way.

Speed cost of the persona is minimal. Measured on identical prompts with MTP disabled at a 512-token cap:

	persona off	persona on
prompt tokens	118	1114
time to first token	0.53 s	1.36 s
decode tok/s	26.9	26.6

It adds about 0.8 s of prompt-processing latency per request and leaves decode throughput essentially unchanged, since the cost is in reading the extra ~1,000 tokens rather than in generating.

---

Quick start

Requires oMLX on Apple Silicon.

hf download grant-ai/Qwen3.8-Flash-Next-Abliterated-MLX-4bit \
  --local-dir ~/omlx-models/qwen38-abliterated-4bit

omlx serve --model-dir ~/omlx-models --port 8000

Then use the OpenAI-compatible endpoint:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model": "qwen38-abliterated-4bit",
       "messages": [{"role": "user", "content": "Explain MoE routing."}],
       "max_tokens": 1024, "temperature": 1.0, "top_p": 0.95, "top_k": 20}'

Settings that matter

setting	value
sampler	temperature 1.0, top_p 0.95, top_k 20 (from the model's own generation_config.json)
MTP	enable via mtp_enabled: true, mtp_num_draft_tokens: 3 in ~/.omlx/model_settings.json
thinking	chat_template_kwargs: {"enable_thinking": false} to disable
reasoning effort	chat_template_kwargs: {"reasoning_effort": "low" | "medium" | "xhigh"}

Reasoning effort defaults to xhigh, the most expensive setting, and is the largest lever on time-to-answer: at xhigh roughly half the generated tokens are reasoning rather than answer. Drop to medium or low for quicker replies, or set enable_thinking: false to turn reasoning off entirely.

MTP is off by default and must be enabled to get the speculative-decoding speedup. With MTP enabled, temperature-0 output is not bit-reproducible across runs.

Technical details

architecture	qwen4_exp, hybrid Gated-DeltaNet + sparse attention, MoE, PLE n-gram embeddings
quantization	oQ4e mixed precision, affine, group size 32 base
bit widths	4 / 5 / 6 / 8 bit across 533 per-module overrides, group sizes 32 / 64 / 128
MTP head	preserved, quantized, iMatrix-calibrated
vision	preserved, 333 tensors
runtime	oMLX

How it was made

1. Abliterated BF16 weights from Blackfrost-AI/Qwen3.8-Flash-Next-DERISKED-BF16, itself derived from Qwen/Qwen3.8-Flash-Next.
2. iMatrix measured from the model's own activations, on a calibration proxy built on disk because the 335 GB source exceeds the in-memory calibration limit.
3. Body and MTP head both quantized with iMatrix-calibrated oQ4e. Standard quantization treats every weight as equally important and applies one bit width throughout. oQ4e instead measures which weights actually matter to the model's outputs and spends more bits on those, so the same file size buys better quality. Applying it to the draft head as well as the body keeps the head accurate enough to stay useful: acceptance was measured at 71.3%, not assumed.
4. Vision tower carried through and verified with a real image request rather than a loader check.

Limitations

• Capability benchmarks have not been run; only conversion correctness and throughput were verified.
• Refusal behaviour was not measured on this build. "Abliterated" describes the upstream weights, not a property this conversion tested.
• With MTP enabled, temperature-0 output varies between runs.
• Speed depends heavily on machine, context length, reasoning effort, and thinking mode.

Credits

• Qwen for Qwen3.8-Flash-Next.
• Blackfrost-AI for the abliterated BF16 weights this build was quantized from.
• oMLX for the oQ mixed-precision quantizer and the Lightning MTP runtime.

Disclaimer and licence

This is an independent, unaffiliated conversion. It is not endorsed by, developed with, developed by, sponsored by, or associated with Qwen, Alibaba Cloud, Blackfrost-AI, or oMLX. None of them reviewed, approved, or contributed to this repository, and none of them bear any responsibility for it.

It does not represent Qwen or Qwen3.8-Flash-Next. The behavior of this model is not the behavior of the official Qwen release. Refusal behavior has been removed by a third-party modification, and the source checkpoint's injected system prompt has been made optional here rather than default. Do not treat any output of this model as representative of, or attributable to, the upstream model or any party named in this card. "Qwen" is used solely to identify the base architecture.

Changes made to the upstream work: quantized to mixed-precision oQ4e with iMatrix calibration, body and MTP draft head together; vision tower preserved; the source checkpoint's chat_template.jinja and the chat_template key in tokenizer_config.json replaced with the stock Qwen3.8 template, with the source checkpoint's injected system prompt retained in the repository as an optional swap rather than a default. No weights were modified beyond quantization; the abliteration itself is upstream work.

Licence: Qwen Community License 1.0, inherited from the upstream model and weights. Your use is also subject to the licence terms of those upstream works.