Model card: sharp version

README.md

@@ -24,119 +24,41 @@ tags:
 **Qwen3.6-35B-A3B, packed for Splash on Apple silicon.**
 
 [Splash](https://github.com/incoai/splash) is Inco AI's open-source inference
-engine for Apple silicon, built around the model. Each model on its roster
-gets its own kernels, its own DFlash 2 draft, and a memory plan computed for
-the machine it runs on.
+engine for Apple silicon, built around the model. This package contains
+everything Splash needs to serve Qwen3.6-35B-A3B: the 4-bit target, its DFlash
+2 draft, the vision encoder, and the tokenizer. It is not a Transformers or
+MLX checkpoint and does not load anywhere but Splash.
 
-This package is what Splash maps into memory when you serve Qwen3.6-35B-A3B:
-the 4-bit target, its DFlash 2 draft, the vision encoder, and the tokenizer,
-laid out as fixed runtime artifacts. It is not a Transformers or MLX
-checkpoint and does not load in those frameworks.
-
-Qwen3.6-35B-A3B is the faster of the two launch models in Splash's benchmarks,
-a mixture-of-experts model with 35B total parameters and about 3B active per
-token. The other,
-[Qwen3.8-27B-Splash](https://huggingface.co/incoai/Qwen3.8-27B-Splash), is a
-dense 27B model.
+Qwen3.6-35B-A3B is the mixture-of-experts launch model, with 35B total
+parameters and about 3B active per token, and the faster of the two. The
+other, [Qwen3.8-27B-Splash](https://huggingface.co/incoai/Qwen3.8-27B-Splash),
+is a dense 27B model.
 
 [Engine](https://github.com/incoai/splash) ·
 [Launch post and benchmarks](https://inco.ai/blog/splash/) ·
 [DFlash 2](https://inco.ai/blog/dflash2/)
 
-## Model details
-
-### Description
-
-- **Developed by:** Inco AI. Qwen3.6-35B-A3B is by Alibaba's Qwen team.
-- **Model type:** a Splash package: the 4-bit Qwen3.6-35B-A3B target, its DFlash 2 draft
-  model, the vision encoder, and the tokenizer, as fixed-layout runtime
-  artifacts.
-- **Languages:** as Qwen3.6-35B-A3B.
-- **License:** Apache-2.0. Every upstream component is Apache-2.0 as well:
-  Qwen3.6-35B-A3B (Alibaba), its 4-bit conversion (mlx-community), and the DFlash 2 draft
-  (Inco AI).
-- **Packaged from:** [`mlx-community/Qwen3.6-35B-A3B-4bit`](https://huggingface.co/mlx-community/Qwen3.6-35B-A3B-4bit)
-  for the target, tokenizer, and vision encoder, and
-  [`incoai/Qwen3.6-35B-A3B-DFlash2`](https://huggingface.co/incoai/Qwen3.6-35B-A3B-DFlash2) for the draft, at the
-  revisions listed under Technical specifications.
-
-### Sources
-
-- **Engine:** <https://github.com/incoai/splash>
-- **Launch post and benchmarks:** <https://inco.ai/blog/splash/>
-- **Draft model:** <https://inco.ai/blog/dflash2/>
-- **Companion package:** [Qwen3.8-27B-Splash](https://huggingface.co/incoai/Qwen3.8-27B-Splash)
-
-## Uses
-
-### Direct use
-
-Serve Qwen3.6-35B-A3B locally with Splash: to coding agents (OpenCode, Claude
-Code, Codex CLI, Hermes Agent) and to any client of the OpenAI Chat
-Completions, Responses, or Anthropic Messages APIs. Text, images, and inline
-PDFs in. Streaming, tool calls, and JSON Schema output out.
-
-### Out-of-scope use
-
-- Loading it anywhere but Splash. It is not a Transformers, MLX, or GGUF
-  checkpoint. For other engines, use the upstream checkpoints.
-- Using the draft on its own. It is not a standalone language model.
-- Anything Qwen3.6-35B-A3B's own license and usage policy exclude.
-
-## Bias, risks, and limitations
-
-- The package inherits Qwen3.6-35B-A3B's biases, knowledge cutoff, and failure modes. See
-  the upstream model card.
-- The target is the upstream 4-bit conversion. We have not published a quality
-  comparison against the bf16 checkpoint.
-- The draft proposes tokens and the target verifies every one, so speculative
-  decoding changes the speed and not the output distribution.
-- Reasoning is on by default. Reasoning tokens count toward the output limit and
-  the time to the final answer.
-- Usable context is bounded by the memory left after the weights, so a 36 GB
-  Mac holds less context and fewer concurrent requests than a 48 GB one.
-- The server binds `127.0.0.1` and has no authentication. Put a proxy in front
-  of it before exposing it to a network.
-
-## How to get started
+## Quick start
 
-### Requirements
-
-Apple M3 or newer, macOS 26.4 or later, [Homebrew](https://brew.sh), and **36
-GB of unified memory, 48 GB or more recommended**. Splash checks available
-memory at startup and stops with a budget breakdown if the model does not fit.
-More memory means more context and more concurrent requests.
-
-### Quick start
+Apple M3 or newer, macOS 26.4 or later, [Homebrew](https://brew.sh), and 36 GB
+of unified memory (48 GB or more recommended).
 
 ```bash
 brew install incoai/tap/splash
 splash serve --model incoai/Qwen3.6-35B-A3B-Splash
 ```
 
-That is the complete setup. The first run downloads this package (20.9 GB),
-verifies it, computes a memory plan for this machine, and starts the server.
-Interrupted downloads resume. Later runs need no network.
-
-When it prints its `Ready` line, the server is up. Open:
-
-- Chat: <http://127.0.0.1:8000>
-- API: <http://127.0.0.1:8000/v1>
-
-To point a coding agent at it, run one of these in another terminal:
+The first run downloads this package (20.9 GB), verifies it, checks available
+memory, and starts serving on `127.0.0.1:8000`. When it prints its `Ready`
+line, open <http://127.0.0.1:8000> or attach an agent you already have
+installed from another terminal:
 
 ```bash
 splash opencode    # or: splash claude / splash codex / splash hermes
 ```
 
-This launches the agent you already have installed and points it at the
-serving model. Keep the server terminal open. **Ctrl+C stops the server.**
-
-### Use the API
-
-Splash serves OpenAI Chat Completions and Responses, and Anthropic Messages,
-all streaming, with tool calls, JSON Schema output, images, and inline PDFs
-(up to 20 pages and 10 MiB each).
+The API is OpenAI Chat Completions and Responses, and Anthropic Messages, with
+streaming, tool calls, JSON Schema output, images, and inline PDFs:
 
 ```bash
 curl http://127.0.0.1:8000/v1/chat/completions \
@@ -147,57 +69,18 @@ curl http://127.0.0.1:8000/v1/chat/completions \
   }'
 ```
 
-`model` is the package ID you served, exactly as passed to `--model`. Leave it
-out to address whatever is serving. `GET /v1/models` lists it.
-
-#### Reasoning
-
-Qwen3.6-35B-A3B reasons before it answers by default. Its reasoning is a
-switch, not a dial: on Chat Completions, `"reasoning_effort": "none"` turns it
-off, the other accepted levels (`minimal` through `max`) all render the same
-prompt, and anything else is rejected with a 400. Responses takes the same
-values in `reasoning.effort`. Anthropic Messages follows the `thinking` field
-and is off unless you send `{"type": "enabled"}` or `{"type": "adaptive"}`.
-Reasoning comes back as `reasoning_content` and is counted in
-`completion_tokens_details.reasoning_tokens`.
-
-The benchmarks below were run with reasoning on.
-
-#### Context and memory
-
-The model's native context window is 256K tokens. Splash grows its 8-bit KV cache as requests need it, within the memory left
-after the weights and the requests in flight, so the usable window depends on
-the machine and the workload. The settings are ceilings that leave room for
-other work on the same Mac:
-
-```bash
-splash serve --model incoai/Qwen3.6-35B-A3B-Splash --max-memory 28G    # cap Metal allocations
-splash serve --model incoai/Qwen3.6-35B-A3B-Splash --max-context 100K  # cap one request
-```
-
-`--max-memory` is a ceiling, not a reservation. The
-[README](https://github.com/incoai/splash#settings) lists every flag with its
-default.
-
-## Training details
-
-This package adds no training. The target weights are upstream's 4-bit
-conversion of Qwen3.6-35B-A3B. The draft was trained by Inco AI against
-Qwen3.6-35B-A3B. The [DFlash 2 post](https://inco.ai/blog/dflash2/) and the
-[draft's card](https://huggingface.co/incoai/Qwen3.6-35B-A3B-DFlash2) describe
-its training.
+Reasoning is on by default and is a switch, not a dial. `"reasoning_effort":
+"none"` turns it off, and reasoning comes back as `reasoning_content`.
 
-## Evaluation
+There is no config file. The only settings are ceilings such as `--max-memory`
+and `--max-context`, which the
+[README](https://github.com/incoai/splash#settings) lists with their defaults.
 
-### Setup
+## Performance
 
-Measured on an M5 Pro (16-core GPU, 48 GB unified memory) with selected
-SPEED-Bench coding prompts, over HTTP, with reasoning on, with a 1,024-token
-output limit. Single-request figures are medians, and concurrent throughput is
-averaged over four waves. Decode counts reasoning tokens. The ratio in each
-cell is against the next-fastest engine we measured.
-
-### Results
+Measured on an M5 Pro (16-core GPU, 48 GB) with selected SPEED-Bench coding
+prompts over HTTP, with a 1,024-token output limit and reasoning on. The ratio
+in each cell is against the next-fastest engine we measured.
 
 | Metric | Qwen3.6-35B-A3B |
 | --- | ---: |
@@ -208,68 +91,57 @@ cell is against the next-fastest engine we measured.
 | Aggregate decode · 4 concurrent short prompts | 357 tok/s (2.0×) |
 | Aggregate decode · 4 concurrent 32K prompts | 236 tok/s (3.8×) |
 
-### Summary
-
-Splash led on every measure at every prompt length we tested. The cached
-figure replays a prompt exactly. A real agent turn also pays to process the
-tokens it appends. These are end-to-end numbers for the complete package on
-the hardware above. The [launch post](https://inco.ai/blog/splash/) has the
-full comparison against oMLX, Lily, uzu, and Ollama.
-
-## Technical specifications
+Splash led on every measure at every prompt length we tested, and the lead
+grows with load. The cached figure replays the prompt exactly, so a real turn also pays for the
+tokens it adds. The [launch
+post](https://inco.ai/blog/splash/) has the method and the full comparison.
 
-### Package contents
+## Package contents
 
 ```
 target/      42 files   18.2 GiB   Qwen3.6-35B-A3B, 4-bit, one packed file per layer
 draft/        7 files    0.5 GiB   DFlash 2 draft model
-vision/       1 file     0.8 GiB   bf16 vision encoder (vision/model.bin)
+vision/       1 file     0.8 GiB   bf16 vision encoder
 tokenizer/    5 files              tokenizer and chat template
 manifest.json                      provenance, geometry, and SHA-256 of every artifact
 layout.json                        section-level map of every packed file
 ```
 
-The packed weights in `target/`, `draft/`, and `vision/` are fixed-layout
-binaries that Splash maps directly from disk, with `StorageN=256` tiling and
-16 KiB section alignment. The target keeps the
-upstream conversion's mixed precision (4-bit affine weights with group size
-64, 8-bit expert routers) across its 40 hybrid layers (gated DeltaNet and
-attention), 256 experts with 8 routed per token plus a shared expert, and a
-248,320-token vocabulary. The vision encoder is the target checkpoint's own
-bf16 encoder.
-
 | Component | Source | Revision |
 | --- | --- | --- |
 | Target, tokenizer, vision | [`mlx-community/Qwen3.6-35B-A3B-4bit`](https://huggingface.co/mlx-community/Qwen3.6-35B-A3B-4bit) | `38740b847e4cb78f352aba30aa41c76e08e6eb46` |
 | Draft | [`incoai/Qwen3.6-35B-A3B-DFlash2`](https://huggingface.co/incoai/Qwen3.6-35B-A3B-DFlash2) | `8e713508f0bb02f03b5cb5cabbc8d9604f924be2` |
 
-The draft is a six-layer DFlash 2 model (hidden size 2048, 32 query heads,
-8 KV heads, 2048-token sliding-window attention, non-causal) that reads the
-target's hidden states at eight layers and proposes 7 tokens per step. The
-target verifies the block of 8 in one pass. Its projections are 4-bit affine
-(group size 64, `StorageN=256`); its norms, convolution kernels, and selector
-codebooks are bf16. The package's execution
-geometry, recorded in `manifest.json`, batches up to four sequences per decode
-step, prefills 2,048 tokens per step, and allocates KV cache in 32-token blocks.
+The weights are fixed-layout binaries that Splash maps directly from disk,
+keeping the upstream conversion's mixed precision: 4-bit weights with 8-bit
+expert routers. The draft is a six-layer DFlash 2 model that reads the
+target's hidden states at eight layers and proposes 7 tokens per step, which
+the target verifies in one pass. The chat template is upstream's with one
+change: a system message after the first turn is rendered in place instead of
+rejected, which coding agents that inject instructions mid-conversation need.
 
-The chat template is upstream's with one change: a system message that
-appears after the first turn is rendered in place instead of being rejected,
-which is what coding agents that inject instructions mid-conversation need.
+`manifest.json` records the revisions above, the execution geometry, and the
+size and SHA-256 of every artifact. Splash pins an immutable commit of this
+repository, checks every artifact's SHA-256 before installing it, and
+re-checks sizes and alignment on every start.
 
-### Integrity
+## Limitations
 
-`manifest.json` records the upstream revisions above, the execution geometry,
-the converter's hashes, and the size and SHA-256 of every artifact. Splash pins
-an immutable commit of this repository plus the manifest's SHA-256, checks
-every artifact's SHA-256 before publishing the package into its local model
-directory, and re-checks sizes and alignment on every start.
+- The package inherits Qwen3.6-35B-A3B's biases, knowledge cutoff, and failure
+  modes.
+- The target is the upstream 4-bit conversion. We have not published a quality
+  comparison against the bf16 checkpoint. The target verifies every proposed
+  token, so speculation changes speed, not the output distribution.
+- Usable context is bounded by the memory left after the weights, so a 36 GB
+  Mac holds less context and fewer concurrent requests than a 48 GB one.
+- The server binds `127.0.0.1` and has no authentication. Put a proxy in front
+  of it before exposing it to a network.
 
-### Compute infrastructure
+## License
 
-- **Hardware:** Apple silicon, M3 or newer, with 36 GB of unified memory or
-  more.
-- **Software:** Splash on macOS 26.4 or later. The kernels are Metal, compiled
-  for this model's shapes and shipped precompiled.
+Apache-2.0. Every component is Apache-2.0 upstream as well: Qwen3.6-35B-A3B
+(Alibaba), its 4-bit conversion (mlx-community), and the DFlash 2 draft (Inco
+AI).
 
 ## Citation
 
@@ -282,7 +154,3 @@ directory, and re-checks sizes and alignment on every start.
   url    = {https://inco.ai/blog/splash/}
 }
 ```
-
-## Contact
-
-<contact@inco.ai>, or an issue on <https://github.com/incoai/splash>.