---
language:
- en
- zh
license: apache-2.0
base_model: dealignai/Qwen3.8-27B-MXFP8-CRACK
base_model_relation: quantized
library_name: mlx
pipeline_tag: image-text-to-text
tags:
- mlx
- apple-silicon
- qwen3_5
- dwq
- 4-bit
- vision-language
- video
- reasoning
- tool-calling
- mtp
- crack
---

# Qwen3.8 27B CRACK — DWQ 4-bit MLX

This is a **16.20 GB, standard-runtime MLX-VLM** derivative of
[`dealignai/Qwen3.8-27B-MXFP8-CRACK`](https://huggingface.co/dealignai/Qwen3.8-27B-MXFP8-CRACK).

- The language tower is affine 4-bit, group size 64, refined with 128
  deterministic DWQ calibration records.
- All 417 vision tensors are value-identical to the source.
- The 31-tensor native MTP shard is byte-identical to the source and remains
  embedded in this repository.
- The source CRACK chat template is retained byte-for-byte.
- Loading and generation use official, unmodified MLX packages. No runtime
  fork or custom model code is required.

> [!IMPORTANT]
> The source model has weight-level refusal ablation. It may comply with unsafe
> requests. You are responsible for safe and lawful use.

## Usage

Tested on Apple Silicon with MLX 0.32.0, MLX-LM 0.31.3, and MLX-VLM 0.6.13.
Because this repository contains the complete vision-language model, load it
with **MLX-VLM**, not the text-only `mlx_lm` loader.

```bash
python -m pip install "mlx==0.32.0" "mlx-lm==0.31.3" "mlx-vlm==0.6.13"

mlx_vlm.generate \
  --model WaveCut/Qwen3.8-27B-CRACK-DWQ-4bit-MLX \
  --prompt "Explain why the sky is blue." \
  --max-tokens 256 \
  --temperature 1.0

mlx_vlm.generate \
  --model WaveCut/Qwen3.8-27B-CRACK-DWQ-4bit-MLX \
  --image ./image.jpg \
  --prompt "Describe this image." \
  --max-tokens 256 \
  --temperature 1.0
```

The inherited chat template enables `xhigh` reasoning by default. It also
supports `medium` and `low`; pass `enable_thinking=False` or use the equivalent
runtime option to disable thinking. The source card recommends
`temperature=1.0`, `top_p=0.95`, `top_k=20`, with EOS token IDs 248046 and
248044.

## What was quantized

| Component | Released representation | Preservation |
|---|---|---|
| Language tower | affine 4-bit, group size 64 | DWQ-refined scales and biases |
| Vision tower | source representation | 417/417 tensors value-identical |
| Native MTP head | affine 8-bit, group size 128 | standalone shard byte-identical |
| Chat template | source CRACK template | byte-identical |

DWQ optimized only the language-tower affine scales and biases against sparse
teacher logits from the MXFP8 source. The 128 unique calibration records
contain 48 tool-calling, 32 SWE-agent, 32 multilingual, and 16 code examples.
The public source datasets were NousResearch/hermes-function-calling-v1,
nebius/SWE-agent-trajectories, CohereLabs/aya_dataset, and
openai/openai_humaneval. Images were not used for language-tower DWQ; vision was
protected by exact tensor preservation plus an official-runtime image smoke.

## Evaluation

### Agent/tool decision sample

We used a deterministic, stratified 96-row sample from NVIDIA When2Call and
teacher-forced choice log-likelihood with the native Qwen tools template. All
models saw the same selected rows.

| Model | Correct | Accuracy | Peak MLX memory |
|---|---:|---:|---:|
| Source MXFP8 CRACK | 38/96 | 39.58% | 30.60 GB |
| Uniform RTN q4/g64 | 40/96 | 41.67% | 18.04 GB |
| **This DWQ q4/g64** | **42/96** | **43.75%** | **18.04 GB** |

Against source MXFP8, this model was uniquely correct on four rows and uniquely
wrong on zero: +4.17 percentage points, paired bootstrap 95% interval
[+1.04, +8.33] points, exact two-sided McNemar p=0.125. Against RTN, it was
uniquely correct on two rows and uniquely wrong on zero: +2.08 points, paired
bootstrap interval [0.00, +5.21], McNemar p=0.5. This is bounded evidence on a
small agent/tool sample, not a claim of universal superiority.

Tool hallucination was unchanged at 5/16 eligible rows for all three models.
Length-normalized scoring tied RTN at 38/96 and scored 38/96 versus 37/96 for
source MXFP8.

### Distillation proxy and runtime smokes

- Fixed held-out sparse-teacher loss: RTN `0.168036` → DWQ `0.093706`
  (44.24% lower).
- Official MLX-VLM text smoke: exact answer `4`, 18.26 GB peak.
- Official MLX-VLM image smoke: exact answer `Blue`, 18.34 GB peak.
- The source CRACK template scored 42/96; replacing only the template with
  [Frog Qwen Fixed Chat Templates](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates)
  v22.1 scored 40/96. The full Frog template is therefore not included.

These memory figures are short-context measurements on an Apple M2 Max with
64 GB unified memory. Longer context and KV cache increase memory use.

## Native MTP status

The original MTP tensors are preserved in this repository; no separate
companion model was published. Official MLX-VLM can split and load them:

```bash
python -m mlx_vlm.speculative.drafters.qwen3_5_mtp.split \
  --model WaveCut/Qwen3.8-27B-CRACK-DWQ-4bit-MLX \
  --output ./qwen38-crack-mtp
```

Preservation does **not** imply a speedup with this new q4 target. On one fixed
128-token greedy prompt with draft block size 2, speculative output matched the
ordinary output exactly, but accepted 0/128 draft tokens and ran at 12.50 tok/s
versus 21.60 tok/s without MTP. Keep MTP disabled by default unless your own
prompt/runtime benchmark demonstrates positive acceptance and throughput.

## License and attribution

Apache-2.0, following the source model. The base Qwen model is by Alibaba; the
MXFP8 CRACK source and refusal-ablation work are by dealignai. This derivative
changes language-tower quantization only and retains the source safety profile.