---
license: apache-2.0
language:
- en
- zh
pipeline_tag: text-generation
tags:
- apple-silicon
- metal
- splash
- mtp
- speculative-decoding
- 8bit
---

# Qwen3.8-27B-Splash-HQ (Native 8-bit)

> [!IMPORTANT]
> ### ⚠️ Runtime Package Compatibility Notice
> This model is packed in the **`splash-packed-q8` (schema_version 5)** format developed for high-precision Apple Silicon evaluation. 
> * Official upstream Splash 1.0 (`incoai/splash`) only recognizes 4-bit formats (`splash-packed-q4`, schema 3/4) and will reject Q8 packages with `repository is not a supported Splash runtime package`.
> * To run this model, use the Q8-enabled Metal runtime fork available at:
>   👉 **[https://github.com/npanj/splash/tree/q8](https://github.com/npanj/splash/tree/q8)**
>
> All 81 model files (target layers, embedding, head, draft model, vision model, tokenizer, and manifest) are 100% complete and verified.

---

## Overview

This repository contains the uncompressed native 8-bit weights of **Qwen3.8-27B** with built-in Multi-Token Prediction (MTP) heads, optimized for high-throughput speculative decoding on Apple Silicon.

All 64 layers are packed with true uncompressed 8-bit tiled weights (`MDFL0008`), preserving exact associative reasoning while running speculative verification via Splash's compiled Metal kernel pipeline.

## Performance Benchmark vs Other Engines (Apple M-Series, 64 GiB)

| Model / Engine | Format | Speculative Scheme | Decode Speed (tok/s) | Speedup vs Stock AR |
| :--- | :--- | :--- | :---: | :---: |
| **Qwen3.8-27B-Splash-HQ** | **Native 8-bit** | **Splash Metal MTP** | **36.9 t/s** (peak 54.8) | **3.73x** |
| Splash-Q8 | Transcoded 8-bit | Splash Metal MTP | 36.5 t/s (peak 52.7) | 3.69x |
| Splash-Q4 | 4-bit | Splash Metal MTP | 60.7 t/s (peak 83.3) | 6.13x |
| MTPLX-Optimized-Quality | 8-bit | MTPLX DraftCore D3 | 26.5 t/s (peak 28.8) | 2.68x |
| MLX-Community 8bit | 8-bit | None (Autoregressive) | 9.9 t/s | 1.00x |
| llama.cpp Q8_0 GGUF | Q8_0 | None (Autoregressive) | 9.9 t/s | 1.00x |

---

## Step-by-Step Setup Recipe

### 1. Clone the Q8 Splash Engine Fork
```bash
git clone https://github.com/npanj/splash.git -b q8
cd splash
```

### 2. Build the Metal Kernels
```bash
make -j4
```

### 3. Serve the Model
```bash
./splash serve --model nitinpanj/Qwen3.8-27B-Splash-HQ --port 8000
```
*(On first launch, this automatically verifies and downloads the model into `install/models/nitinpanj/Qwen3.8-27B-Splash-HQ` and starts the OpenAI-compatible API on `http://127.0.0.1:8000`).*

### 4. Connect Coding Agents
```bash
# Connect with Oh My Pi (OMP)
omp --model splash/nitinpanj/Qwen3.8-27B-Splash-HQ

# Or connect with any OpenAI-compatible client at http://127.0.0.1:8000/v1
```

For the complete benchmark report, evaluation scripts, and raw JSON logs:
👉 **[https://github.com/npanj/splash-plus](https://github.com/npanj/splash-plus)**