Instructions to use yaanfpv/Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use yaanfpv/Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("yaanfpv/Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit")
config = load_config("yaanfpv/Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use yaanfpv/Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "yaanfpv/Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "yaanfpv/Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use yaanfpv/Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "yaanfpv/Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default yaanfpv/Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use yaanfpv/Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "yaanfpv/Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "yaanfpv/Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit

A 4-bit MLX (Apple Silicon) repackaging of LuffyTheFox's OmniClaw-Qwen3.5-9B-Claude-4.6-Opus-Uncensored-v2 text weights paired with a canonical Qwen3.5-VL vision tower from Jackrong's v2 safetensors release. Designed for image+text agentic workloads on Mac (LM Studio, vllm-swift with vision patches, mlx-vlm).

What this is

LuffyTheFox publishes OmniClaw as a GGUF only. To run it on MLX with vision, two things had to happen:

The text weights had to be recovered from the GGUF and merged with the canonical Qwen3.5-9B-VL vision tower (which OmniClaw inherits from upstream Qwen3.5).
The combined hybrid had to be quantized to 4-bit MLX with mlx-vlm conventions (vision tower kept BF16, language model quantized).

This repository is the result of that conversion process.

OmniClaw v2 lineage

Per LuffyTheFox's model card, OmniClaw v2 is itself a four-way merge built on Qwen3.5-9B. The text weights you see here trace back through the following chain (all credit to the upstream authors):

Qwen/Qwen3.5-9B  (base, Apache 2.0)
 │
 ├── Jackrong/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2  (reasoning distillation, weight 1.0)
 │     trained on Claude 4.6 Opus reasoning data via Unsloth
 │
 ├── HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive  (uncensoring / zero-refusal)
 │
 ├── Tesslate/OmniCoder-9B  (coding capability, weight 0.5)
 │
 └── nbeerbower/Qwen3.5-9B-Writing-DPO  (creative writing, weight 0.5)
        │
        ▼
LuffyTheFox/OmniClaw-Qwen3.5-9B-Claude-4.6-Opus-Uncensored-v2-GGUF
   (fp32 merge of Unsloth-trainable weights only, GGUF distribution)
        │
        ▼
This repository (MLX 4-bit + canonical Qwen3.5-VL vision tower)

The vision tower used here is the canonical Qwen3.5-9B-VL vision tower, sourced as safetensors from Jackrong/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2. Mmproj equivalence across the lineage was verified to be bit-identical, so the vision pipeline behaviour matches upstream Qwen3.5-VL.

Build process (full provenance)

Step	Source	Tool	Output
1. Pull text weights	`LuffyTheFox/OmniClaw-Qwen3.5-9B-Claude-4.6-Opus-Uncensored-v2-GGUF`	`dreamfast/ungguf` `convert-qwen35`	`OmniClaw-recovered-text-hf` (HF safetensors, language tensors only)
2. Pull vision tower	`Jackrong/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2`	direct download	`Jackrong-v2-safetensors` (HF safetensors, full VL)
3. Verify vision pedigree	mmproj GGUFs across the lineage	`compare_mmprojs.py`	confirmed bit-identical mmprojs (canonical Qwen3.5-VL vision tower)
4. Hybrid build	OmniClaw text (427 tensors) + Jackrong vision + MTP (348 tensors)	`build_hybrid.py`	`Vayaan-OmniClaw-hybrid-hf`
5. Apply config fixes	hybrid HF dir	manual (use_cache, mtp_num_hidden_layers, model_name, strip Unsloth metadata)	finalized HF dir
6. MLX 4-bit conversion	finalized HF dir	`mlx_vlm.convert` (group_size=64, bits=4, mode=affine)	this repository
7. Chat template	Froggeric Jinja (handles list-wrapped multimodal content)	`chat_template.jinja`	this repository

The hybrid is functionally a Qwen3.5-VL with OmniClaw text and canonical vision. Vision tower equivalence was verified via N-way mmproj comparison (Step 3); the canonical vision tower flows unchanged across the lineage.

Specs

Architecture: Qwen3.5-9B-VL (hybrid Mamba2 + Transformer, every-4th-layer full attention)
Quantization: 4-bit MLX (group_size=64, affine), language model only; vision tower + merger + LM head kept BF16
Context: 131072 tokens (model max)
Image processor: Qwen2VLImageProcessorFast (canonical, copied from upstream Qwen3.5-9B preprocessor)
Total disk size: ~5.6 GB (5.0 GB shard 1 + 0.57 GB shard 2)
Tokenizer: vocab_size=248320 (Qwen3.5 native), <|vision_start|>, <|image_pad|>, <|vision_end|>, <|video_pad|> special tokens
Chat template: froggeric's qwen3.5/chat_template-v9.jinja; supports list-wrapped multimodal content and preserves Qwen3.5's <think>...</think> reasoning blocks

Verified runtimes

Runtime	Status	Notes
LM Studio	works (text + vision)	reference behavior
`mlx-vlm` direct	should work	MLX-native runtime, same as LM Studio under the hood
`vllm-swift` (stock v0.6.0)	broken (vision returns hallucinated output)	upstream issue; see TheTom/vllm-swift#16
`vllm-swift` with PR #15	works (text + vision)	three patches: `mm_features` API drift, VLM-first load on `vision_config`, `[1, N]` token reshape
`llama.cpp`	not directly compatible	this is MLX safetensors, not GGUF; convert separately if you need GGUF

Known limitations

Long-context prefill on small Macs is constrained in vllm-swift. vllm-swift v0.6.0 force-disables chunked prefill (platform.py:113) and has no incremental prefill FFI, so a 40k+ prefill at 131k context can OOM 16 GB Macs. Tracking: TheTom/vllm-swift#18. LM Studio and mlx-vlm do not have this constraint.
TurboQuant unavailable. Qwen3.5's head_dim=256 Metal kernel (turbo_dequant_rotated_*_256_*) is missing from the v0.6.0 metallib bottle even though the source instantiates it. Tracking: TheTom/vllm-swift#17. Without TurboQuant, KV cache stays at fp16; on this hybrid architecture full-attention KV is only ~2 GB at 131k due to the 4:1 linear-to-full attention ratio, but compression would reduce that to ~500 MB.
Streaming usage info. stream_options.include_usage is not honored in vllm-swift streamed responses. Tracking: TheTom/vllm-swift#19.

Usage

LM Studio

Place this repo under ~/.lmstudio/models/yaanfpv/Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit/ (or use LM Studio's "Add Local Model"). Vision works out of the box.

mlx-vlm

pip install mlx-vlm
python -m mlx_vlm.generate \
  --model yaanfpv/Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit \
  --prompt "Describe this image." \
  --image https://example.com/photo.jpg \
  --max-tokens 200

vllm-swift (with PR #15 applied)

vllm-swift serve yaanfpv/Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit \
  --served-model-name vayaan-omniclaw \
  --max-model-len 32768 \
  --no-enable-prefix-caching \
  --max-num-seqs 1 \
  --host 127.0.0.1 --port 8080 \
  --trust-remote-code

Use a context size that fits your hardware until TheTom/vllm-swift#18 lands.

Notes on behavior

OmniClaw v2 is merged with HauhauCS's uncensoring component, so this model will engage with prompts that other Qwen3.5 derivatives may refuse. As with any LLM, outputs can still be inaccurate, biased, or otherwise wrong. Verify factual claims and apply moderation appropriate to your context when building user-facing applications.

Attribution

Direct sources used by this repository

Text weights (recovered from GGUF): LuffyTheFox/OmniClaw-Qwen3.5-9B-Claude-4.6-Opus-Uncensored-v2-GGUF (Apache 2.0)
Vision tower + MTP (donor safetensors): Jackrong/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2 (Apache 2.0)
Base architecture: Qwen/Qwen3.5-9B (Apache 2.0)

Components merged into OmniClaw v2 (full upstream credit)

LuffyTheFox's OmniClaw v2 is itself a merge of several finetunes. All credit for the underlying weights goes to:

Jackrong for Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2 (Apache 2.0): reasoning distillation backbone, merged at weight 1.0.
HauhauCS for Qwen3.5-9B-Uncensored-HauhauCS-Aggressive (Apache 2.0): uncensoring / zero-refusal characteristics.
Tesslate for OmniCoder-9B-GGUF (Apache 2.0): coding capability, merged at weight 0.5.
nbeerbower for Qwen3.5-9B-Writing-DPO (license not declared on the source repo at the time of writing; redistribution rights for this repo are inherited from LuffyTheFox's Apache 2.0 declaration on OmniClaw v2): creative writing capability, merged at weight 0.5.

Datasets behind the reasoning backbone (Jackrong's training pipeline)

Tooling and chat template

Chat template: froggeric/Qwen-Fixed-Chat-Templates qwen3.5/chat_template-v9.jinja. Handles list-wrapped multimodal content arrays correctly (the default Qwen3.5 chat template does not, which breaks vision in mlx-vlm and clients that send [{"type":"text",...},{"type":"image_url",...}] style content).
Conversion tooling: dreamfast/ungguf (convert-qwen35 for GGUF to safetensors recovery), mlx-vlm (final MLX 4-bit conversion)

Acknowledgments

Unsloth AI for making rapid Qwen3.5 fine-tuning practical, the Qwen team at Alibaba Cloud for the base model, and the broader open-source distillation community. Apple's mlx-swift and Awni Hannun and team for the MLX runtime that makes Apple Silicon inference of this model possible. TheTom for vllm-swift, the vLLM-on-Apple-Silicon serving layer (see also the open PR and issues that this repository's vllm-swift compatibility depends on).

Per Jackrong's request: a star on his repos is appreciated if this work is useful to you.

License

Apache License 2.0, inherited from all upstream sources. See the linked base models for full license text.

Maintainer

Repackaging by @YaanFPV. Issues and feedback welcome on the model's HF discussion tab.

Downloads last month: 22

Safetensors

Model size

2B params

Tensor type

BF16

U32

F32

MLX

Hardware compatibility

4-bit

Model tree for yaanfpv/Vayaan-Qwen3.5-9B-OmniClaw-VL-MLX-4bit

Base model

Qwen/Qwen3.5-9B-Base

Finetuned

Qwen/Qwen3.5-9B

Quantized

HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive

Quantized

(11)

this model