---
license: apache-2.0
pipeline_tag: image-text-to-text
library_name: llama.cpp
tags: [qwen3-vl, spacemit, k1, k3, gguf, onnxruntime, moe]
---

# Qwen3-VL-30B-A3B for SpacemiT K1/K3

This is a SpacemiT deployment package for the Qwen3-VL 30B-A3B mixture-of-experts vision-language model. The upstream model is from the Qwen Team and is designed for multimodal understanding across text, images, and video, including visual question answering, document understanding, OCR-style reading, coding, and agent workflows. See the [Qwen3-VL Technical Report](https://arxiv.org/abs/2511.21631) and the [official Qwen3-VL announcement](https://qwen.ai/blog?id=qwen3-vl).

The package contains a Q4_1 GGUF text decoder and an ONNX vision encoder. The ONNX file is renamed to `qwen3_vl_vision.onnx` for a short, stable deployment name. The original Qwen3-VL model and this deployment package use the Apache-2.0 license; runtime dependency licenses remain with their upstream projects.

## Files

- `qwen3vl-30b-text-q4_1.gguf`: Q4_1 GGUF text/MoE decoder.
- `qwen3_vl_vision.onnx`: ONNX vision encoder.
- `configs/K1/config.json` and `configs/K3/config.json`: platform-specific SpaceMIT EP settings.

## Prerequisites

Install the [SpacemiT ONNX Runtime release](https://github.com/spacemit-com/onnxruntime/releases) and build or unpack the SMT-enabled [SpacemiT llama.cpp](https://github.com/spacemit-com/llama.cpp). Prebuilt RISC-V packages can be unpacked directly. To build llama.cpp from source, clone recursively, set `RISCV_ROOT_PATH` and `SPACEMIT_ORT_DIR`, then run:

```bash
bash build_spacemit.sh
```

On the board, set the runtime library path before starting the server:

```bash
export MODEL_DIR=/path/to/Qwen3-VL-30B-A3B-SpacemiT
export LLAMA_DIR=/path/to/llama.cpp-installed
export ORT_DIR=/path/to/spacemit-ort.riscv64.2.0.6
export LD_LIBRARY_PATH="$LLAMA_DIR/lib:$ORT_DIR/lib:${LD_LIBRARY_PATH:-}"
```

## Supported platforms and core affinity

K1 uses AI cores `0;1;2;3` and four threads. Use `configs/K1/config.json` and `-t 4`.

K3 uses AI cores `8;9;10;11;12;13;14;15` and eight threads. Use `configs/K3/config.json` and `-t 8`.

Do not interchange the configuration files: each `ep_config` pins the SpaceMIT Execution Provider to the correct board's AI cores.

## Run on board

K1:

```bash
"$LLAMA_DIR/bin/llama-server" \
  -m "$MODEL_DIR/qwen3vl-30b-text-q4_1.gguf" \
  --media-backend smt \
  --smt-config-dir "$MODEL_DIR/configs/K1" \
  -t 4 --host 0.0.0.0 --port 8080 --warmup
```

K3 uses the same command with `configs/K3` and `-t 8`.

Send an OpenAI-compatible `POST /v1/chat/completions` request containing an image data URL and a prompt such as `Describe the image content.`. Set `temperature` to `0`, use a small `max_tokens` value for a smoke test, and disable Qwen thinking with `chat_template_kwargs: {"enable_thinking": false}`.

Example request:

```bash
curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":[{"type":"text","text":"Describe the image content."},{"type":"image_url","image_url":{"url":"data:image/jpeg;base64,<BASE64_IMAGE>"}}]}],"max_tokens":64,"temperature":0,"chat_template_kwargs":{"enable_thinking":false}}'
```

The included `humanspeech.jpg` can be used for a basic end-to-end smoke test. Replace `<BASE64_IMAGE>` with its base64-encoded contents. The response should be HTTP 200 with a natural-language description; this is a functional check, not an accuracy benchmark.