---
license: apache-2.0
library_name: coreml
base_model: Qwen/Qwen3-VL-2B-Instruct
tags:
  - coreml
  - apple-silicon
  - ane
  - on-device
  - qwen3-vl
  - vision-language
pipeline_tag: image-text-to-text
---

## Use it from Swift

<!-- swift-usage-begin -->
### Add the package

`Package.swift`:

```swift
.package(url: "https://github.com/john-rocky/CoreML-LLM", branch: "main"),

// In your target:
.product(name: "CoreMLLLM", package: "CoreML-LLM"),
```

Platforms: iOS 18+ / macOS 15+.

### Download + chat (one call)

```swift
import CoreMLLLM

let llm = try await CoreMLLLM.load(repo: "mlboydaisuke/qwen3-vl-2b-coreml")

let stream = try await llm.generate(
    [CoreMLLLM.Message(role: .user, content: "Hello!")],
    maxTokens: 256
)
for await chunk in stream { print(chunk, terminator: "") }
```

### With an image

```swift
import CoreGraphics

let cgImage: CGImage = ...   // your CGImage

let stream = try await llm.generate(
    [CoreMLLLM.Message(role: .user,
                       content: "What's in this image?")],
    image: cgImage,
    maxTokens: 256
)
for await chunk in stream { print(chunk, terminator: "") }
```
<!-- swift-usage-end -->



# Qwen3-VL 2B — Core ML (recurrent path, v1.4.0)

Core ML port of [`Qwen/Qwen3-VL-2B-Instruct`](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct) — text + vision, INT8 chunked, runs on iPhone A18 ANE.

> **Heads up:** for new work prefer the **stateful** variant at [`mlboydaisuke/qwen3-vl-2b-stateful-coreml`](https://huggingface.co/mlboydaisuke/qwen3-vl-2b-stateful-coreml) — same model, but KV cache lives inside ANE via `MLState` + `slice_update` so memory is 6× lower (264 MB vs 1.7 GB) and decode is 2× faster (24 vs 10 tok/s) on iPhone 17 Pro. This recurrent repo is kept for backward compatibility with the v1.4.0 runtime.

## Files

```
qwen3_vl_2b_decode_chunks/
├── chunk_0.mlpackage              # 353 MB — text path: embed + L0-6
├── chunk_1.mlpackage              # 353 MB — L7-13
├── chunk_2.mlpackage              # 353 MB — L14-20
├── chunk_3.mlpackage              # 353 MB — L21-27
├── chunk_head.mlpackage           # 311 MB — final_norm + lm_head + argmax
├── chunk_0_vision.mlpackage       # 353 MB — chunk_0 with DeepStack injection
├── prefill_chunk_{0..3}.mlpackage # 353 MB each — T=32 batched prefill bodies
├── prefill_chunk_0_vision.mlpackage  # vision-aware prefill chunk_0
└── embed_weight.bin               # 622 MB — raw fp16 embed (151936 × 2048)

qwen3_vl_2b_vision/
└── vision.mlpackage               # 406 MB — 448×448 → 196 tokens + 3 DeepStack taps
```

The vision encoder is loaded only when an image is in the prompt. DeepStack taps from vision layers 5/11/17 are injected into text layers 0/1/2 via `chunk_0_vision`.

## What this repo does NOT ship

- **No `model_config.json`** — Core ML serializes shapes into each `.mlpackage`. `coremltools` opens them without external config.
- **No tokenizer / processor** — fetch from the base model:

```python
from transformers import AutoTokenizer, AutoProcessor
tok  = AutoTokenizer.from_pretrained("Qwen/Qwen3-VL-2B-Instruct")
proc = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-2B-Instruct")
```

> **Vision preprocessing note:** Qwen3-VL uses `mean=std=0.5` (not the CLIP defaults), and the vision encoder expects `pixel_values (3, 2, 448, 448)` already pre-patchified — see [`conversion/build_qwen3_vl_2b_vision.py`](https://github.com/john-rocky/CoreML-LLM/blob/main/conversion/build_qwen3_vl_2b_vision.py) for the exact transform.

## Standalone usage (Python / Mac)

```python
import coremltools as ct, numpy as np
from huggingface_hub import snapshot_download

local = snapshot_download("mlboydaisuke/qwen3-vl-2b-coreml")
root  = f"{local}/qwen3_vl_2b_decode_chunks"

decode_chunks = [ct.models.MLModel(f"{root}/chunk_{i}.mlpackage") for i in range(4)]
head         = ct.models.MLModel(f"{root}/chunk_head.mlpackage")
embed        = np.memmap(f"{root}/embed_weight.bin",
                         dtype=np.float16, mode="r",
                         shape=(151936, 2048))
vision = ct.models.MLModel(
    f"{local}/qwen3_vl_2b_vision/vision.mlpackage")
```

For text-only prompts, skip the vision encoder and chain `chunk_0..3 → chunk_head` per step. For image prompts, run `vision.predict` once, swap `chunk_0` with `chunk_0_vision` and inject the 3 DeepStack tensors during the first token of each image span.

Reference loop: [`conversion/qwen3_vl_2b_parity.py`](https://github.com/john-rocky/CoreML-LLM/blob/main/conversion/qwen3_vl_2b_parity.py).

## iOS / Mac app

Swift runtime: [`Qwen3VL2BGenerator.swift`](https://github.com/john-rocky/CoreML-LLM/blob/main/Examples/CoreMLLLMChat/CoreMLLLMChat/Qwen3VL2BGenerator.swift). Pick **Qwen3-VL 2B (recurrent, v1.4.0)** in the model picker.

## Architecture

28-layer GQA text backbone + ViT vision tower.

- **Text:** hidden=2048, num_heads=16, num_kv=8, head_dim=128, vocab=151936, tie_embeddings=True, rope_theta=5e6, mRoPE section=[24,20,20] interleaved (collapses to standard 1D RoPE for text-only).
- **Vision:** 448×448 fixed, 196 tokens after spatial_merge=2, DeepStack taps at vision layers 5/11/17 → text layers 0/1/2.
- **Chunks:** 4 body chunks × 7 layers each.

## License

Apache 2.0 (inherits from the base model).

<!-- funnel:v1 -->

---

**More models in this format:** [Core ML Model Zoo](https://huggingface.co/collections/mlboydaisuke/core-ml-model-zoo-6a7078dc888e7b13efd35631) — 46 models, each with the recipe that produced it.

**Want a different model on-device?** [Open a request](https://github.com/john-rocky/on-device-requests) — free, open weights only; the export and its measured numbers get published publicly.

<!-- /funnel:v1 -->