ruwwww/qwen3.5-9b-ninfer

Qwen3.5-9B for NInfer

This model card is the version-controlled source for ruwwww/qwen3.5-9b-ninfer.

The repository contains Qwen3.5-9B converted to the native NInfer .ninfer artifact format. The artifact is intended only for NInfer; it is not a Transformers checkpoint, Safetensors distribution, or GGUF file.

Artifact

Field	Value
Filename	qwen3_5_9b.ninfer
Size	6,514,051,328 bytes (6.07 GiB)
SHA-256	5e823ea5b4df7c75c630cb5ff90017cf17769c68bbf9545640cf981af5ec7bd6
NInfer model ID	qwen3.5-9b
NInfer weights ID	groupwise-int
NInfer target key	qwen3_5_9b

The file contains the registered Text, Vision, MTP, proposal-head, tokenizer, chat-template, generation, and media-processor objects required by NInfer.

Verify a downloaded file with:

printf '%s  %s\n' \
  '5e823ea5b4df7c75c630cb5ff90017cf17769c68bbf9545640cf981af5ec7bd6' \
  'qwen3_5_9b.ninfer' | sha256sum --check

Source model

Qwen3.5-9B is a multimodal dense model with a mixed linear/full-attention text backbone:

Field	Value
Source repository	Qwen/Qwen3.5-9B
Model type	qwen3_5 (Qwen3_5ForConditionalGeneration)
Hidden size	4,096
Layers	32 (24 linear-attention, 8 full-attention at intervals of 4)
Attention heads	16 × head-dim 256 (4 KV heads)
Linear attention	16 key heads × 128, 32 value heads × 128, conv kernel 4
MLP intermediate	12,288
Vocab	248,320
Max context	262,144
MTP layers	1

Requirements

• NInfer revision 24bd04d or later, built from source;
• 64-bit Linux;
• NVIDIA GeForce RTX 5090 (sm_120a);
• CUDA Toolkit 13.1 or newer.

NInfer does not provide an install target or packaged binary. See the repository README for source-build dependencies.

Download and run

hf download ruwwww/qwen3.5-9b-ninfer \
  qwen3_5_9b.ninfer \
  --local-dir models

./build/apps/ninfer models/qwen3_5_9b.ninfer \
  --prompt "What is the capital of France?" \
  --max-context 4096 \
  --max-new 256

Serve it through the OpenAI-compatible HTTP endpoint:

./build/apps/ninfer-serve models/qwen3_5_9b.ninfer \
  --model-id qwen3.5-9b \
  --max-context 16384 \
  --spec mtp --draft-tokens 3

For images, videos, structured chat history, and HTTP serving, see the NInfer documentation.

Supported use

The artifact supports:

• text generation in thinking and non-thinking modes;
• image, multi-image, video, and mixed multimodal messages;
• MTP speculative decoding with draft windows from one to five;
• BF16 and INT8 group-64 KV cache;
• CUDA Graph decode and compatible-prefix reuse;
• the NInfer CLI;
• OpenAI Chat Completions and Anthropic Messages serving.

Verified behavior

The 9B artifact was verified against the source BF16 model (Hugging Face Transformers on an RTX 5090):

• Prefill activations match the reference top-10 logits exactly (byte-identical per step).
• Greedy decode tokens 0-7 are identical to the reference; later steps diverge only on near-tie logits within quantization drift, with both streams remaining coherent.
• Observed serving throughput on one RTX 5090 (greedy, short prompts): ~793 tok/s prefill and ~241 tok/s decode without speculation; with MTP=3, ~445-455 tok/s decode and ~84% draft acceptance.

Limits

• The artifact is accepted only by NInfer revision 24bd04d or later and the matching registered target.
• NInfer currently executes on one RTX 5090, one CUDA device, and one active request per Engine.
• It does not provide continuous batching, multi-GPU execution, CPU/GPU offload, or distributed serving.
• Context allocation is subject to GPU memory and the selected KV-cache type.
• NInfer does not execute generated tool calls.

Provenance

Field	Value
Source repository	Qwen/Qwen3.5-9B
Conversion recipe	qwen3_5_9b-v1
Converter repository	https://github.com/ruwwww/ninfer
Minimum runtime revision	24bd04d

License

This NInfer artifact is distributed under the Apache License 2.0. The source Qwen3.5-9B repository is also licensed under Apache-2.0. Users remain responsible for complying with the license and applicable laws.