Ornith-1.5-9B for onw — entirely on the Intel NPU (text + image)

日本語 | English

ornith-ai/Ornith-1.5-9B converted for onw (俺のNPUがこんなに動くわけない - "there's no way my NPU runs this well"), the engine that runs LLMs entirely on the Intel NPU. This repo holds only the model; the engine is a separate download.

ornith-ai's Ornith-1.5 takes Qwen3.5-family models through continued pre-training, mid-training and post-training with a self-improvement loop aimed at coding agents, tool calls and long tasks (its card reports large gains over the base Qwen on SWE-bench Verified, Terminal-Bench, MCP-Atlas and more). The architecture is Qwen3.5 / 3.6's, so onw's Qwen conversion and runtime apply as they are. This 9B is dense, text + image.

Tool calls: OpenAI-style tools (parallel calls too). In thinking mode the thinking comes back as reasoning_content; send it back in the history and the model gets it (Ornith's template keeps earlier turns' thinking, which agents rely on). The original card recommends temperature 1.0, top_p 0.95, top_k 20, presence_penalty 1.5 for general use and temperature 0.6 for coding (onw defaults to greedy).

NPU 3720 (Core Ultra 9 285HX)
input	text + image
decode	4.9 tok/s (9-15 tok/s with prompt lookup)
prompt processing	image + question, 281 tokens: vision 0.5 s + prefill 8.0 s
download	5.3 GB
memory in use (working set after loading)	~11 GB
first start (NPU compile) / later	~10 min / ~20 s

Supported PCs: Intel Core Ultra with an NPU (series 1 / 2: Meteor Lake, Arrow Lake, Lunar Lake; series 3: Panther Lake), Windows 11 or Ubuntu 22.04+. The numbers above are measured on the test machine (NPU 3720) and vary with the NPU and memory.

Use

Install onw (the engine) if you have not yet - one line, see Install in onw. On Windows open "PowerShell" from the Start menu, paste this and press Enter (Ubuntu: the curl ... | bash line in the onw README).
```
irm https://huggingface.co/ryugyosoft/onw/resolve/main/install.ps1 | iex
```
In the onw window (from the onw icon in the task tray), Models tab: pick Ornith-1.5-9B and press Download (5.3 GB).
Server tab: pick it and press Load. The first load prepares it for the NPU (~10 min).
"Chat" to try it. Other apps can use it as an OpenAI-compatible API (base URL http://localhost:8000/v1).

Server tab Settings tab

from openai import OpenAI
c = OpenAI(base_url="http://localhost:8000/v1", api_key="none")
r = c.chat.completions.create(model="Ornith-1.5-9B-onw", messages=[{"role": "user", "content": "Hello"}])
print(r.choices[0].message.content)

Advanced: hf download ryugyosoft/Ornith-1.5-9B-onw and onw serve <folder> works too. Do not git clone without Git LFS (you would get pointer files instead of the weights; onw detects that and stops).

How it runs

32 layers (24 Gated DeltaNet + 8 gated attention) in 4 segments of 8 layers, group-128 INT4, plus an LM-head segment skipped for prompt blocks that need no logits.
DeltaNet: 1-token matrix form for decoding; 16-token prompt blocks in chunkwise-parallel form with an exact block-doubling inverse. The DeltaNet output is scaled by 1024 (fp16 subnormals) and the gated RMSNorm prescaled so it cannot overflow fp16.
Vision: the 27-layer ViT as one static graph for 512x512 input; MRoPE positions for image tokens are computed on the host.
Details: onw technical notes.

Files

file	what
`seg_S1.xml`, `seg_S16.xml` + `seg*.bin`	4 segments + LM head (one weights file per segment)
`vision.xml`	INT8 vision tower (512x512 -> 256 tokens)
`shared.bin`	INT8 LM head, INT4 token embedding (host lookup)
`engine.json`, tokenizer / config files	onw metadata, the original repo's configs

Quality

Group-128 INT4 segments, INT8 LM head, INT4 token embedding (host lookup), INT8 vision tower. Teacher-forced against the bf16 model on the CPU, all 10 tokens up to the end of the answer (<|im_end|>) match. Checked: an image (shapes and colours), two parallel tool calls and an answer built from their results, the thinking split.

License

MIT, same as the base model. The weights are re-quantized / restructured from it; no training.