Hugging Face's logo

incoai
/
Qwen3.6-35B-A3B-Splash

Text Generation
splash
apple-silicon
metal
local-inference
dflash2
speculative-decoding
qwen3.6
Mixture of Experts
4-bit precision
Model card Files
xet
 Community
3
Qwen3.6-35B-A3B-Splash / README.md
zhijianliu's picture
zhijianliu
Qwen3.6-35B-A3B packed for Splash: package, manifest, and model card
a93f2a0 verified
preview code
|
Raw History Blame
11.5 kB
metadata 
license: apache-2.0
library_name: splash
pipeline_tag: text-generation
inference: false
base_model:
  - mlx-community/Qwen3.6-35B-A3B-4bit
  - incoai/Qwen3.6-35B-A3B-DFlash2
base_model_relation: quantized
tags:
  - splash
  - apple-silicon
  - metal
  - local-inference
  - dflash2
  - speculative-decoding
  - qwen3.6
  - moe
  - 4-bit

Qwen3.6-35B-A3B-Splash

Qwen3.6-35B-A3B, packed for Splash on Apple silicon.

Splash is Inco AI's open-source inference engine for Apple silicon, built around the model. Each model on its roster gets its own kernels, its own DFlash 2 draft, and a memory plan computed for the machine it runs on.

This package is what Splash maps into memory when you serve Qwen3.6-35B-A3B: the 4-bit target, its DFlash 2 draft, the vision encoder, and the tokenizer, laid out as fixed runtime artifacts. It is not a Transformers or MLX checkpoint and does not load in those frameworks.

Qwen3.6-35B-A3B is the faster of the two launch models in Splash's benchmarks, a mixture-of-experts model with 35B total parameters and about 3B active per token. The other, Qwen3.8-27B-Splash, is a dense 27B model.

Engine 路 Launch post and benchmarks 路 DFlash 2

Model details

Description

  • Developed by: Inco AI. Qwen3.6-35B-A3B is by Alibaba's Qwen team.
  • Model type: a Splash package: the 4-bit Qwen3.6-35B-A3B target, its DFlash 2 draft model, the vision encoder, and the tokenizer, as fixed-layout runtime artifacts.
  • Languages: as Qwen3.6-35B-A3B.
  • License: Apache-2.0. Every upstream component is Apache-2.0 as well: Qwen3.6-35B-A3B (Alibaba), its 4-bit conversion (mlx-community), and the DFlash 2 draft (Inco AI).
  • Packaged from: mlx-community/Qwen3.6-35B-A3B-4bit for the target, tokenizer, and vision encoder, and incoai/Qwen3.6-35B-A3B-DFlash2 for the draft, at the revisions listed under Technical specifications.

Sources

Uses

Direct use

Serve Qwen3.6-35B-A3B locally with Splash: to coding agents (OpenCode, Claude Code, Codex CLI, Hermes Agent) and to any client of the OpenAI Chat Completions, Responses, or Anthropic Messages APIs. Text, images, and inline PDFs in. Streaming, tool calls, and JSON Schema output out.

Out-of-scope use

  • Loading it anywhere but Splash. It is not a Transformers, MLX, or GGUF checkpoint. For other engines, use the upstream checkpoints.
  • Using the draft on its own. It is not a standalone language model.
  • Anything Qwen3.6-35B-A3B's own license and usage policy exclude.

Bias, risks, and limitations

  • The package inherits Qwen3.6-35B-A3B's biases, knowledge cutoff, and failure modes. See the upstream model card.
  • The target is the upstream 4-bit conversion. We have not published a quality comparison against the bf16 checkpoint.
  • The draft proposes tokens and the target verifies every one, so speculative decoding changes the speed and not the output distribution.
  • Reasoning is on by default. Reasoning tokens count toward the output limit and the time to the final answer.
  • Usable context is bounded by the memory left after the weights, so a 36 GB Mac holds less context and fewer concurrent requests than a 48 GB one.
  • The server binds 127.0.0.1 and has no authentication. Put a proxy in front of it before exposing it to a network.

How to get started

Requirements

Apple M3 or newer, macOS 26.4 or later, Homebrew, and 36 GB of unified memory, 48 GB or more recommended. Splash checks available memory at startup and stops with a budget breakdown if the model does not fit. More memory means more context and more concurrent requests.

Quick start 

brew install incoai/tap/splash
splash serve --model incoai/Qwen3.6-35B-A3B-Splash

That is the complete setup. The first run downloads this package (20.9 GB), verifies it, computes a memory plan for this machine, and starts the server. Interrupted downloads resume. Later runs need no network.

When it prints its Ready line, the server is up. Open:

To point a coding agent at it, run one of these in another terminal:

splash opencode    # or: splash claude / splash codex / splash hermes

This launches the agent you already have installed and points it at the serving model. Keep the server terminal open. Ctrl+C stops the server.

Use the API

Splash serves OpenAI Chat Completions and Responses, and Anthropic Messages, all streaming, with tool calls, JSON Schema output, images, and inline PDFs (up to 20 pages and 10 MiB each).

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "incoai/Qwen3.6-35B-A3B-Splash",
    "messages": [{"role": "user", "content": "Explain speculative decoding in one sentence."}]
  }'

model is the package ID you served, exactly as passed to --model. Leave it out to address whatever is serving. GET /v1/models lists it.

Reasoning

Qwen3.6-35B-A3B reasons before it answers by default. Its reasoning is a switch, not a dial: on Chat Completions, "reasoning_effort": "none" turns it off, the other accepted levels (minimal through max) all render the same prompt, and anything else is rejected with a 400. Responses takes the same values in reasoning.effort. Anthropic Messages follows the thinking field and is off unless you send {"type": "enabled"} or {"type": "adaptive"}. Reasoning comes back as reasoning_content and is counted in completion_tokens_details.reasoning_tokens.

The benchmarks below were run with reasoning on.

Context and memory

The model's native context window is 256K tokens. Splash grows its 8-bit KV cache as requests need it, within the memory left after the weights and the requests in flight, so the usable window depends on the machine and the workload. The settings are ceilings that leave room for other work on the same Mac:

splash serve --model incoai/Qwen3.6-35B-A3B-Splash --max-memory 28G    # cap Metal allocations
splash serve --model incoai/Qwen3.6-35B-A3B-Splash --max-context 100K  # cap one request

--max-memory is a ceiling, not a reservation. The README lists every flag with its default.

Training details

This package adds no training. The target weights are upstream's 4-bit conversion of Qwen3.6-35B-A3B. The draft was trained by Inco AI against Qwen3.6-35B-A3B. The DFlash 2 post and the draft's card describe its training.

Evaluation

Setup

Measured on an M5 Pro (16-core GPU, 48 GB unified memory) with selected SPEED-Bench coding prompts, over HTTP, with reasoning on, with a 1,024-token output limit. Single-request figures are medians, and concurrent throughput is averaged over four waves. Decode counts reasoning tokens. The ratio in each cell is against the next-fastest engine we measured.

Results

Metric Qwen3.6-35B-A3B
Decode 路 short prompt 210 tok/s (1.7脳)
Prefill 路 32K prompt 2,011 tok/s (1.3脳)
Time to first token 路 32K prompt, uncached 17 s (1.3脳)
Cached time to first token 路 32K replay 123 ms (6.6脳)
Aggregate decode 路 4 concurrent short prompts 357 tok/s (2.0脳)
Aggregate decode 路 4 concurrent 32K prompts 236 tok/s (3.8脳)

Summary

Splash led on every measure at every prompt length we tested. The cached figure replays a prompt exactly. A real agent turn also pays to process the tokens it appends. These are end-to-end numbers for the complete package on the hardware above. The launch post has the full comparison against oMLX, Lily, uzu, and Ollama.

Technical specifications

Package contents 

target/      42 files   18.2 GiB   Qwen3.6-35B-A3B, 4-bit, one packed file per layer
draft/        7 files    0.5 GiB   DFlash 2 draft model
vision/       1 file     0.8 GiB   bf16 vision encoder (vision/model.bin)
tokenizer/    5 files              tokenizer and chat template
manifest.json                      provenance, geometry, and SHA-256 of every artifact
layout.json                        section-level map of every packed file

The packed weights in target/, draft/, and vision/ are fixed-layout binaries that Splash maps directly from disk, with StorageN=256 tiling and 16 KiB section alignment. The target keeps the upstream conversion's mixed precision (4-bit affine weights with group size 64, 8-bit expert routers) across its 40 hybrid layers (gated DeltaNet and attention), 256 experts with 8 routed per token plus a shared expert, and a 248,320-token vocabulary. The vision encoder is the target checkpoint's own bf16 encoder.

Component Source Revision
Target, tokenizer, vision mlx-community/Qwen3.6-35B-A3B-4bit 38740b847e4cb78f352aba30aa41c76e08e6eb46
Draft incoai/Qwen3.6-35B-A3B-DFlash2 8e713508f0bb02f03b5cb5cabbc8d9604f924be2

The draft is a six-layer DFlash 2 model (hidden size 2048, 32 query heads, 8 KV heads, 2048-token sliding-window attention, non-causal) that reads the target's hidden states at eight layers and proposes 7 tokens per step. The target verifies the block of 8 in one pass. Its projections are 4-bit affine (group size 64, StorageN=256); its norms, convolution kernels, and selector codebooks are bf16. The package's execution geometry, recorded in manifest.json, batches up to four sequences per decode step, prefills 2,048 tokens per step, and allocates KV cache in 32-token blocks.

The chat template is upstream's with one change: a system message that appears after the first turn is rendered in place instead of being rejected, which is what coding agents that inject instructions mid-conversation need.

Integrity

manifest.json records the upstream revisions above, the execution geometry, the converter's hashes, and the size and SHA-256 of every artifact. Splash pins an immutable commit of this repository plus the manifest's SHA-256, checks every artifact's SHA-256 before publishing the package into its local model directory, and re-checks sizes and alignment on every start.

Compute infrastructure

  • Hardware: Apple silicon, M3 or newer, with 36 GB of unified memory or more.
  • Software: Splash on macOS 26.4 or later. The kernels are Metal, compiled for this model's shapes and shipped precompiled.

Citation 

@misc{inco2026splash,
  title  = {{Splash: A Local Engine Built Around the Model}},
  author = {{Inco AI}},
  year   = {2026},
  month  = {September},
  url    = {https://inco.ai/blog/splash/}
}

Contact

contact@inco.ai, or an issue on https://github.com/incoai/splash.