Instructions to use Noctalin/Qwen3.6-27B-oQ8-fp16-mtp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Noctalin/Qwen3.6-27B-oQ8-fp16-mtp with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("Noctalin/Qwen3.6-27B-oQ8-fp16-mtp")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use Noctalin/Qwen3.6-27B-oQ8-fp16-mtp with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Noctalin/Qwen3.6-27B-oQ8-fp16-mtp"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Noctalin/Qwen3.6-27B-oQ8-fp16-mtp"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use Noctalin/Qwen3.6-27B-oQ8-fp16-mtp with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "Noctalin/Qwen3.6-27B-oQ8-fp16-mtp"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "Noctalin/Qwen3.6-27B-oQ8-fp16-mtp"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "Noctalin/Qwen3.6-27B-oQ8-fp16-mtp",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use Noctalin/Qwen3.6-27B-oQ8-fp16-mtp with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Noctalin/Qwen3.6-27B-oQ8-fp16-mtp"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Noctalin/Qwen3.6-27B-oQ8-fp16-mtp

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Noctalin/Qwen3.6-27B-oQ8-fp16-mtp with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Noctalin/Qwen3.6-27B-oQ8-fp16-mtp"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Noctalin/Qwen3.6-27B-oQ8-fp16-mtp" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.6 oQ8 Text-Only with Native MTP (Optimized for Apple Silicon)

This repository contains custom-quantized, text-only configurations of the Qwen3.6 model family, optimized explicitly for local repository-level agentic coding on Apple Silicon using the oMLX inference engine.

🎯 Why This Was Created

Standard open-weight releases and community pre-converted MLX files (like standard Unsloth) routinely strip out Multi-Token Prediction (mtp.*) tensors during conversion. While this saves minimal disk space, it completely disables the hardware-level Native MTP decoding pipeline on Apple Silicon.

This project was built to solve specific long-context constraints encountered on a MacBook Pro M2 Max (96GB Unified Memory, 400 GB/s memory bandwidth):

The Dense Speed Bottleneck: Dense architectures like the Qwen3.6-27B provide significantly higher coding accuracy, sharper reasoning, and far fewer repetitive loops than sparse MoE setups. However, they naturally run much slower. Preserving the original MTP heads unlocks a ~1.5x generation speedup to make dense coding completely viable locally.
Overcoming the 128k Context Wall: Traditional backends like LM Studio frequently experience severe latency degradation or choke entirely when context windows approach 128k tokens due to unoptimized KV cache regeneration. Moving to oMLX's specialized two-tier caching eliminates this overhead.
Maximized Precision: While 4-bit and 5-bit quants are popular for saving memory, a 96GB unified memory architecture provides plenty of overhead. These builds use oQ8 (8-bit) precision to guarantee lossless adherence to complex syntax, indentation, and nested tool calls.

🚀 Key Differences

Feature / Attribute	Standard Qwen3.6 / Unsloth	This Custom Build (`oQ8-fp16-mtp`)
Native MTP Heads	Stripped out during conversion	Fully Preserved (Enables Native MTP toggle)
Vision Model (VLM)	Included by default (wastes memory space)	Stripped (Text/Coding only; ~2-3% smaller)
Quantization Method	Standard Uniform / GGUF Dynamic	oQ8 (Dynamic mixed-precision calibration)
Non-Quant Weight DType	`bfloat16`	`float16` (~20% faster prefill on M1/M2 Silicon)

💻 Hardware & RAM Recommendations

Mac Hardware Configuration	RAM Recommendation	Status / Performance Expectation
M1 Max / M2 Max / M3 Max (Base 32GB/36GB/48GB)	48GB Unified Memory	Supported (Minimal) — Good for standard queries, but context window will be limited.
M1 / M2 / M3 / M4 Max / Ultra	64GB Unified Memory	Recommended — Great performance, handles deep context windows (~128k) comfortably.
M2 Max / M3 Max (96GB / 128GB)	96GB / 128GB Unified Memory	Optimal / Best Experience — The user's M2 Max 96GB is ideal. Allows running maximum context (262k) with zero slowdowns and plenty of system overhead.

🛠️ Quantization Settings

This model was quantized using oMLX v0.4.3 with the following settings:

Source Model: Qwen/Qwen3.6-27B
Sensitivity Model: None (use source model)
oQ Level: oQ8
Text Only: Enabled (Excludes vision encoder weights, saving ~2-3% memory footprint)
Preserve MTP weights: Enabled (Preserves mtp.* tensors for Native MTP support)
Non-quant weight dtype: float16 (Provides ~20% faster prefill on M1/M2 Apple Silicon)

⚙️ Optimized oMLX Settings (v0.4.3)

To replicate the ultra-long context development workspace used with OpenCode 1.16.2, apply the following server specifications in your oMLX dashboard:

Model Basic Settings

Reasoning Parser: qwen_3_coder (Isolates CoT thoughts to protect IDE syntax)
CTX Window: 262,144
Max Tokens: 32,768
Temperature: 0.6
Top P / Top K: 0.95 / 20
Min P: 0
Repetition / Presence Penalty: 1 / 0

Model Advanced Settings

Enabled Thinking: Checked (True)
Chat Template Kwargs: enable_thinking: true, preserve_thinking: true
Native MTP: Checked (True)

Resource Management & Cache

Memory Guard: Aggressive (Enforces strict macOS memory/swap cleanup)
Hot Cache Limit (RAM): 40GB (Allocated for high-speed Unified Memory history)
Cold Cache Limit (SSD): 371GB (Serialized safetensors storage for context overflow)
Max Concurrent Requests: 2 (Prevents splitting the 400 GB/s bandwidth bus)
Embedding Batch Size: 32
Chunked Prefill: Enabled (Prevents instantaneous out-of-memory crashes on large files)
Burst Decode: Aggressive (Coalesces tokens for maximum raw typing speed)
Initial Cache Blocks: 256
SSE Keepalive Mode: Chunk

🌡️ Thermal Optimization Notice

Sustained local execution across deep context windows will heavily tax the Apple Silicon GPU/CPU, leading to internal heat buildup. Because Apple's default fan profiles prioritize silence over longevity, they delay high-RPM cooling until thermal throttling is already occurring.

To manage this proactively during extended coding agent sessions, use smc-command—an adapted CLI fan utility tailored to trigger more aggressive fan curves based on exact thermal sensors.

Source Repository: smc-command on Codeberg

oMLX Benchmark - Performance

oMLX - LLM inference, optimized for your Mac
https://github.com/jundot/omlx
Benchmark Model: Qwen3.6-27B-oQ8-fp16-mtp
================================================================================

Single Request Results
--------------------------------------------------------------------------------
Test                TTFT(ms)    TPOT(ms)        pp TPS        tg TPS      E2E(s)    Throughput    Peak Mem
pp1024/tg128          5468.7       70.23   187.2 tok/s    14.4 tok/s      14.388    80.1 tok/s    28.81 GB
pp4096/tg128         20370.5       61.99   201.1 tok/s    16.3 tok/s      28.243   149.6 tok/s    30.26 GB
pp8192/tg128         41398.6       54.51   197.9 tok/s    18.5 tok/s      48.321   172.2 tok/s    30.89 GB
pp16384/tg128        85002.6       56.29   192.7 tok/s    17.9 tok/s      92.152   179.2 tok/s    32.14 GB
pp32768/tg128       180387.9       57.55   181.7 tok/s    17.5 tok/s     187.697   175.3 tok/s    34.64 GB
pp65536/tg128       409535.7       62.85   160.0 tok/s    16.0 tok/s     417.518   157.3 tok/s    39.66 GB
pp200000/tg128     1900213.2       89.74   105.3 tok/s    11.2 tok/s    1911.611   104.7 tok/s    56.53 GB

Continuous Batching
pp1024 / tg128
--------------------------------------------------------------------------------
Batch           tg TPS   Speedup        pp TPS    pp TPS/req    TTFT(ms)      E2E(s)
1x          14.4 tok/s     1.00x   187.2 tok/s   187.2 tok/s      5468.7      14.388
2x          23.2 tok/s     1.61x    50.4 tok/s    25.2 tok/s     34893.7      51.639
4x          46.3 tok/s     3.22x   127.0 tok/s    31.8 tok/s     21303.2      43.311
8x          92.5 tok/s     6.42x   108.4 tok/s    13.6 tok/s     42968.2      86.633

Downloads last month: 75

Safetensors

Model size

27B params

Tensor type

U32

F16

MLX

Hardware compatibility

8-bit

Model tree for Noctalin/Qwen3.6-27B-oQ8-fp16-mtp

Base model

Qwen/Qwen3.6-27B

Quantized

(715)

this model

Collection including Noctalin/Qwen3.6-27B-oQ8-fp16-mtp

Qwen3.6 oQ MTP (M1/M2 Apple Silicon)

Collection

Text-only Qwen3.6 oQ + Native MTP models optimized for M1/M2 Apple Silicon. Built for local agentic coding via oMLX and OpenCode. • 6 items • Updated Jun 17