Instructions to use Noctalin/Qwen3.6-27B-oQ5-fp16-mtp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Noctalin/Qwen3.6-27B-oQ5-fp16-mtp with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("Noctalin/Qwen3.6-27B-oQ5-fp16-mtp")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use Noctalin/Qwen3.6-27B-oQ5-fp16-mtp with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Noctalin/Qwen3.6-27B-oQ5-fp16-mtp"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Noctalin/Qwen3.6-27B-oQ5-fp16-mtp"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use Noctalin/Qwen3.6-27B-oQ5-fp16-mtp with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "Noctalin/Qwen3.6-27B-oQ5-fp16-mtp"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "Noctalin/Qwen3.6-27B-oQ5-fp16-mtp"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "Noctalin/Qwen3.6-27B-oQ5-fp16-mtp",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use Noctalin/Qwen3.6-27B-oQ5-fp16-mtp with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Noctalin/Qwen3.6-27B-oQ5-fp16-mtp"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Noctalin/Qwen3.6-27B-oQ5-fp16-mtp

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Noctalin/Qwen3.6-27B-oQ5-fp16-mtp with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Noctalin/Qwen3.6-27B-oQ5-fp16-mtp"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Noctalin/Qwen3.6-27B-oQ5-fp16-mtp" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.6 oQ5 Text-Only with Native MTP (Optimized for Apple Silicon)

This repository contains custom-quantized, text-only configurations of the Qwen3.6 model family, optimized explicitly for local repository-level agentic coding on Apple Silicon using the oMLX inference engine.

🎯 Why This Was Created

Standard open-weight releases and community pre-converted MLX files (like standard Unsloth) routinely strip out Multi-Token Prediction (mtp.*) tensors during conversion. While this saves minimal disk space, it completely disables the hardware-level Native MTP decoding pipeline on Apple Silicon.

This project was built to solve specific long-context constraints encountered on Apple Silicon:

The Dense Speed Bottleneck: Dense architectures like the Qwen3.6-27B provide significantly higher coding accuracy, sharper reasoning, and far fewer repetitive loops than sparse MoE setups. However, they naturally run much slower. Preserving the original MTP heads unlocks a ~1.5x generation speedup to make dense coding completely viable locally.
Overcoming the 128k Context Wall: Traditional backends like LM Studio frequently experience severe latency degradation or choke entirely when context windows approach 128k tokens due to unoptimized KV cache regeneration. Moving to oMLX's specialized two-tier caching eliminates this overhead.
Balanced Performance & Precision: While oQ8 requires a high-end machine (like 64GB/96GB RAM), oQ5 (5-bit) offers a sweet spot of improved perplexity and adherence to complex code instructions over 4-bit quants, running smoothly on Macs with 32GB or 36GB Unified Memory.

🚀 Key Differences

Feature / Attribute	Standard Qwen3.6 / Unsloth	This Custom Build (`oQ5-fp16-mtp`)
Native MTP Heads	Stripped out during conversion	Fully Preserved (Enables Native MTP toggle)
Vision Model (VLM)	Included by default (wastes memory space)	Stripped (Text/Coding only; ~2-3% smaller)
Quantization Method	Standard Uniform / GGUF Dynamic	oQ5 (Dynamic mixed-precision calibration)
Non-Quant Weight DType	`bfloat16`	`float16` (~20% faster prefill on M1/M2 Silicon)

💻 Hardware & RAM Recommendations

Mac Hardware Configuration	RAM Recommendation	Status / Performance Expectation
M1 / M2 / M3 Pro	32GB / 36GB Unified Memory	Recommended (Optimal) — Runs smoothly with ample overhead for deep context windows.
M1 / M2 / M3 / M4 Max	32GB / 64GB / 96GB	Fully Supported — High-speed bandwidth enables peak tokens/sec. (M2 Max 96GB will run this effortlessly).

🛠️ Quantization Settings

This model was quantized using oMLX v0.4.4 with the following settings:

Source Model: Qwen/Qwen3.6-27B
Sensitivity Model: None (use source model)
oQ Level: oQ5
Text Only: Enabled (Excludes vision encoder weights, saving ~2-3% memory footprint)
Preserve MTP weights: Enabled (Preserves mtp.* tensors for Native MTP support)
Non-quant weight dtype: float16 (Provides ~20% faster prefill on M1/M2 Apple Silicon)

⚙️ Optimized oMLX Settings (v0.4.4)

To replicate the ultra-long context development workspace used with OpenCode 1.16.2, apply the following server specifications in your oMLX dashboard:

Model Basic Settings

Reasoning Parser: qwen_3_coder (Isolates CoT thoughts to protect IDE syntax)
CTX Window: 262,144
Max Tokens: 32,768
Temperature: 0.6
Top P / Top K: 0.95 / 20
Min P: 0
Repetition / Presence Penalty: 1 / 0

Model Advanced Settings

Enabled Thinking: Checked (True)
Chat Template Kwargs: enable_thinking: true, preserve_thinking: true
Native MTP: Checked (True)

Resource Management & Cache

Memory Guard: Aggressive (Enforces strict macOS memory/swap cleanup)
Hot Cache Limit (RAM): 25GB (Allocated for high-speed Unified Memory history)
Cold Cache Limit (SSD): 371GB (Serialized safetensors storage for context overflow)
Max Concurrent Requests: 2 (Prevents splitting the 400 GB/s bandwidth bus)
Embedding Batch Size: 32
Chunked Prefill: Enabled (Prevents instantaneous out-of-memory crashes on large files)
Burst Decode: Aggressive (Coalesces tokens for maximum raw typing speed)
Initial Cache Blocks: 256
SSE Keepalive Mode: Chunk

🌡️ Thermal Optimization Notice

Sustained local execution across deep context windows will heavily tax the Apple Silicon GPU/CPU, leading to internal heat buildup. Because Apple's default fan profiles prioritize silence over longevity, they delay high-RPM cooling until thermal throttling is already occurring.

To manage this proactively during extended coding agent sessions, use smc-command—an adapted CLI fan utility tailored to trigger more aggressive fan curves based on exact thermal sensors.