Upload Qwen3-VL-30B-A3B package files

README.md

@@ -1,3 +1,69 @@
 ---
 license: apache-2.0
+pipeline_tag: image-text-to-text
+library_name: llama.cpp
+tags: [qwen3-vl, spacemit, k1, k3, gguf, onnxruntime, moe]
 ---
+
+# Qwen3-VL-30B-A3B for SpacemiT K1/K3
+
+This is a SpacemiT deployment package for the Qwen3-VL 30B-A3B mixture-of-experts vision-language model. The upstream model is from the Qwen Team and is designed for multimodal understanding across text, images, and video, including visual question answering, document understanding, OCR-style reading, coding, and agent workflows. See the [Qwen3-VL Technical Report](https://arxiv.org/abs/2511.21631) and the [official Qwen3-VL announcement](https://qwen.ai/blog?id=qwen3-vl).
+
+The package contains a Q4_1 GGUF text decoder and an ONNX vision encoder. The ONNX file is renamed to `qwen3_vl_vision.onnx` for a short, stable deployment name. The original Qwen3-VL model and this deployment package use the Apache-2.0 license; runtime dependency licenses remain with their upstream projects.
+
+## Files
+
+- `qwen3vl-30b-text-q4_1.gguf`: Q4_1 GGUF text/MoE decoder.
+- `qwen3_vl_vision.onnx`: ONNX vision encoder.
+- `configs/K1/config.json` and `configs/K3/config.json`: platform-specific SpaceMIT EP settings.
+
+## Prerequisites
+
+Install the [SpacemiT ONNX Runtime release](https://github.com/spacemit-com/onnxruntime/releases) and build or unpack the SMT-enabled [SpacemiT llama.cpp](https://github.com/spacemit-com/llama.cpp). Prebuilt RISC-V packages can be unpacked directly. To build llama.cpp from source, clone recursively, set `RISCV_ROOT_PATH` and `SPACEMIT_ORT_DIR`, then run:
+
+```bash
+bash build_spacemit.sh
+```
+
+On the board, set the runtime library path before starting the server:
+
+```bash
+export MODEL_DIR=/path/to/Qwen3-VL-30B-A3B-SpacemiT
+export LLAMA_DIR=/path/to/llama.cpp-installed
+export ORT_DIR=/path/to/spacemit-ort.riscv64.2.0.6
+export LD_LIBRARY_PATH="$LLAMA_DIR/lib:$ORT_DIR/lib:${LD_LIBRARY_PATH:-}"
+```
+
+## Supported platforms and core affinity
+
+K1 uses AI cores `0;1;2;3` and four threads. Use `configs/K1/config.json` and `-t 4`.
+
+K3 uses AI cores `8;9;10;11;12;13;14;15` and eight threads. Use `configs/K3/config.json` and `-t 8`.
+
+Do not interchange the configuration files: each `ep_config` pins the SpaceMIT Execution Provider to the correct board's AI cores.
+
+## Run on board
+
+K1:
+
+```bash
+"$LLAMA_DIR/bin/llama-server" \
+  -m "$MODEL_DIR/qwen3vl-30b-text-q4_1.gguf" \
+  --media-backend smt \
+  --smt-config-dir "$MODEL_DIR/configs/K1" \
+  -t 4 --host 0.0.0.0 --port 8080 --warmup
+```
+
+K3 uses the same command with `configs/K3` and `-t 8`.
+
+Send an OpenAI-compatible `POST /v1/chat/completions` request containing an image data URL and a prompt such as `Describe the image content.`. Set `temperature` to `0`, use a small `max_tokens` value for a smoke test, and disable Qwen thinking with `chat_template_kwargs: {"enable_thinking": false}`.
+
+Example request:
+
+```bash
+curl http://127.0.0.1:8080/v1/chat/completions \
+  -H 'Content-Type: application/json' \
+  -d '{"messages":[{"role":"user","content":[{"type":"text","text":"Describe the image content."},{"type":"image_url","image_url":{"url":"data:image/jpeg;base64,<BASE64_IMAGE>"}}]}],"max_tokens":64,"temperature":0,"chat_template_kwargs":{"enable_thinking":false}}'
+```
+
+The included `humanspeech.jpg` can be used for a basic end-to-end smoke test. Replace `<BASE64_IMAGE>` with its base64-encoded contents. The response should be HTTP 200 with a natural-language description; this is a functional check, not an accuracy benchmark.