Add opt-in vision flag, F16 projector, provenance and image smoke validation

.gitattributes

@@ -6,3 +6,5 @@ assets/qwen38-flash-ciru-strix-iu4.jpg filter=lfs diff=lfs merge=lfs -text
 *.png filter=lfs diff=lfs merge=lfs -text
 
 *.tar.gz filter=lfs diff=lfs merge=lfs -text
+
+*.mmproj filter=lfs diff=lfs merge=lfs -text

README.md

@@ -7,7 +7,7 @@ base_model:
   - Qwen/Qwen3.8-Flash-Next-FP8
 base_model_relation: quantized
 library_name: llama.cpp
-pipeline_tag: text-generation
+pipeline_tag: image-text-to-text
 inference: false
 tags:
   - qwen
@@ -28,6 +28,8 @@ tags:
   - ple
   - long-context
   - local-inference
+  - vision
+  - image-text-to-text
 ---
 
 [![Qwen3.8 Flash CIRU Strix IU4](https://huggingface.co/jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4/resolve/v3.0.0/assets/qwen38-flash-ciru-strix-iu4.jpg)](https://llm.ciru.ai/research)
@@ -36,7 +38,7 @@ tags:
 
 **V3 brings faster long-context serving and the qualified QSA conversation-isolation fixes to the Strix-only runner. Model weights are unchanged.** It adds parallel attention-cell selection, indexed decode attention, cached derived history, guarded PLE lookups and wider prefill. The general profile retains maximum MTP depth 6 and batch/microbatch 1024.
 
-Use the [v3 source archive](runtime/v3.0.0/ciru-runtime-v3.0.0-source.tar.gz) or [matching GitHub tag](https://github.com/ciru-ai/Qwen3.8-Flash-CIRU-STRIX-IU4/tree/v3.0.0). This text-only package requires the custom CIRU runtime, the target GGUF and all three `ple/` files. The `mtp/` head enables speculative decoding. Stock llama.cpp and Hugging Face hosted inference do not support this package.
+Use the [v3 source archive](runtime/v3.0.0/ciru-runtime-v3.0.0-source.tar.gz) or [matching GitHub tag](https://github.com/ciru-ai/Qwen3.8-Flash-CIRU-STRIX-IU4/tree/v3.0.0). This package supports optional image input with the projector and `--vision` launcher below, and requires the custom CIRU runtime, the target GGUF and all three `ple/` files. The `mtp/` head enables speculative decoding. Stock llama.cpp and Hugging Face hosted inference do not support this package.
 
 ## V3 serving comparison
 
@@ -166,7 +168,7 @@ Including the separate performance probe, the run took **3h 20m 46s**. This is a
 
 ## Download, build and run
 
-The unchanged model files total **135,962,881,135 bytes (126.625 GiB)**, excluding runtime and reports. The target is 73.945 GiB, the PLE payload 48.828 GiB and the Q8 MTP head 3.852 GiB. Allow at least 160 GiB for model storage/verification plus SDK and build space. Use fast NVMe and a Strix Halo machine with 128 GiB unified memory.
+The unchanged model files total **135,962,881,135 bytes (126.625 GiB)**, excluding runtime, reports, and the optional 904,004,320-byte (0.842 GiB) vision projector. The target is 73.945 GiB, the PLE payload 48.828 GiB and the Q8 MTP head 3.852 GiB. Allow at least 160 GiB for model storage/verification plus SDK and build space. Use fast NVMe and a Strix Halo machine with 128 GiB unified memory.
 
 On Ubuntu/Debian, install Git and a Python virtual environment, then download the model and build the source:
 
@@ -211,6 +213,78 @@ curl http://127.0.0.1:8080/v1/chat/completions \
 
 These sampling defaults follow the [Qwen model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next/blob/f5d08274bafd880402bd16f5e3e6c514136ec06c/README.md#best-practices). Ordinary chat can retain prefix caching; the benchmark's cold requests, fixed seed and output cap are measurement controls.
 
+## Enable vision
+
+Vision files and the new model-folder launcher are published on `main`; the historical `v3.0.0` tag remains unchanged. After following the build instructions above, return to the directory containing `model/` and `ciru-runtime-v3.0.0/`, then download the additional files:
+
+```bash
+hf download jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 \
+  --revision main run-server.sh 'vision/*' --local-dir ./model
+(cd model/vision && sha256sum -c SHA256SUMS)
+```
+
+The launcher stays text-only by default. To accept images, add **`--vision`**:
+
+```bash
+RUNTIME_DIR="$PWD/ciru-runtime-v3.0.0" \
+  BUILD_DIR="$PWD/ciru-runtime-v3.0.0/build-gfx1151-sdk" \
+  bash ./model/run-server.sh --vision
+```
+
+`ENABLE_VISION=1` is equivalent. The flag loads `vision/mmproj-Qwen3.8-Flash-F16.mmproj`; `MMPROJ=/path/to/projector.mmproj` overrides that path. The `.mmproj` file is a **GGUF-format vision projector**, with a filename extension chosen to keep Hub model-size detection focused on the language model. A missing projector produces a clear startup error. Direct runtime launches can pass `--mmproj ./model/vision/mmproj-Qwen3.8-Flash-F16.mmproj` alongside the normal model and PLE arguments.
+
+Existing users can download just the updated launcher and vision files:
+
+```bash
+hf download jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 \
+  run-server.sh 'vision/*' --local-dir ./model
+(cd model/vision && sha256sum -c SHA256SUMS)
+```
+
+Send an image through the server web UI or the OpenAI-compatible chat endpoint. For a local image:
+
+```bash
+python3 - <<'PYIMAGE'
+import base64, json, urllib.request
+from pathlib import Path
+image = base64.b64encode(Path("image.png").read_bytes()).decode()
+payload = {
+    "model": "Qwen3.8-Flash-CIRU-STRIX-IU4",
+    "messages": [{"role": "user", "content": [
+        {"type": "text", "text": "Describe this image."},
+        {"type": "image_url", "image_url": {"url": "data:image/png;base64," + image}}
+    ]}],
+    "chat_template_kwargs": {"enable_thinking": False},
+    "max_tokens": 512
+}
+request = urllib.request.Request(
+    "http://127.0.0.1:8080/v1/chat/completions",
+    data=json.dumps(payload).encode(),
+    headers={"Content-Type": "application/json"}
+)
+with urllib.request.urlopen(request, timeout=300) as response:
+    print(json.load(response)["choices"][0]["message"]["content"])
+PYIMAGE
+```
+
+The projector was converted to F16 directly from the pinned original Qwen checkpoint using the CIRU v3 converter. It contains 334 tensors. Vision uses additional memory for the projector, image encoding, and image tokens; the text-only performance figures above do not measure vision. [Projector provenance and validation](vision/README.md).
+
+### DeepSeek harness configuration
+
+When using this model through the DeepSeek harness, start the server with `--vision` and declare image input in the harness's this model model configuration using `input`:
+
+```yaml
+input:
+  - text
+  - image
+```
+
+Replace the this model entry's `inputModalities` field with the `input` list above. A user reported that the harness recognized this model image input with this configuration; the native DeepSeek model's `inputModalities` declaration did not work for their this model entry. This is a harness configuration requirement, in addition to loading the projector with `--vision`.
+
+### Image resolution budget
+
+The v3 runtime defaults to a maximum of **4,096 visual tokens per image**. Large images are resized to fit while preserving aspect ratio. Each separately submitted image or crop gets its own budget. To allow 8,192 tokens, append `--image-max-tokens 8192` to the vision launch above. `--image-min-tokens 1024` can also be tried for small-detail grounding. Higher budgets consume more memory and context and take longer to process; upscaling cannot recover missing detail. Any resizing performed by the harness before upload still applies.
+
 ## What ships in v3
 
 V3 combines the previously qualified QSA sequence isolation/indexer-copy fixes with the portable retained hybrid work: wider aligned QSA prefill, J32 expert tiling for supported prefill shapes, radix selection, indexed F16 attention, direct guarded PLE lookup, derived-history caching keyed by sequence and packed tiny F32 gathers. The F32 derived cache adds about 192 MiB at 256K. Hybrid external-GPU ownership, peer transfers, expert splitting, scheduler overlap and experimental compact masks are excluded.

run-server.sh

@@ -0,0 +1,134 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+package_root="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+repo_root="${RUNTIME_DIR:?Set RUNTIME_DIR to the CIRU v3 source checkout}"
+repo_root="$(cd "$repo_root" && pwd)"
+build_dir="${BUILD_DIR:-${repo_root}/build-gfx1151-sdk}"
+server_bin="${SERVER_BIN:-${build_dir}/bin/llama-server}"
+model_dir="${MODEL_DIR:-${package_root}}"
+model="${model_dir}/Qwen3.8-Flash-CIRU-STRIX-IU4.gguf"
+draft="${model_dir}/mtp/Qwen3.8-Flash-CIRU-STRIX-IU4-MTP-Q8_0.gguf"
+ple_dir="${model_dir}/ple"
+slot_dir="${SLOT_DIR:-${package_root}/slot-state}"
+
+# Launcher-only opt-in; remaining arguments are passed to llama-server.
+enable_vision="${ENABLE_VISION:-0}"
+server_args=()
+for arg in "$@"; do
+    case "$arg" in
+        --vision) enable_vision=1 ;;
+        *) server_args+=("$arg") ;;
+    esac
+done
+set -- "${server_args[@]}"
+if [[ "$enable_vision" != 0 && "$enable_vision" != 1 ]]; then
+    echo "ENABLE_VISION must be 0 or 1." >&2
+    exit 2
+fi
+mmproj="${MMPROJ:-${model_dir}/vision/mmproj-Qwen3.8-Flash-F16.mmproj}"
+if [[ "$enable_vision" == 1 && ! -f "$mmproj" ]]; then
+    echo "Vision is enabled but its projector is missing: $mmproj" >&2
+    echo "Download vision/mmproj-Qwen3.8-Flash-F16.mmproj from the model repository, or set MMPROJ." >&2
+    exit 2
+fi
+
+# The released shortlist has one shared row map and requires exactly one slot.
+# Include trailing CLI overrides so --parallel/-np cannot bypass this check.
+parallel_slots="${PARALLEL_SLOTS:-1}"
+extra_args=("$@")
+for ((i = 0; i < ${#extra_args[@]}; i++)); do
+    case "${extra_args[i]}" in
+        --parallel|-np)
+            parallel_slots="${extra_args[i+1]:-}"
+            ((i += 1))
+            ;;
+        --parallel=*) parallel_slots="${extra_args[i]#*=}" ;;
+        -np=*) parallel_slots="${extra_args[i]#*=}" ;;
+    esac
+done
+if [[ "${ENABLE_MTP:-1}" != "0" && "${parallel_slots}" != "1" ]]; then
+    echo "The CIRU MTP shortlist requires exactly one slot (--parallel 1)." >&2
+    echo "For parallel target-only serving, set ENABLE_MTP=0 and PARALLEL_SLOTS=2." >&2
+    echo "See docs/RUNNING.md: Parallel requests and unified KV cache." >&2
+    exit 2
+fi
+
+for required in "${server_bin}" "${model}" "${ple_dir}/ple.payload.bin" "${ple_dir}/ple.manifest.json" "${ple_dir}/ple.scale.bf16"; do
+    if [[ ! -e "${required}" ]]; then
+        echo "Required release file is missing: ${required}" >&2
+        exit 2
+    fi
+done
+
+mkdir -p "${slot_dir}"
+
+set -a
+# shellcheck source=/dev/null
+source "${repo_root}/profiles/strix-halo-production.env"
+set +a
+
+args=(
+    --model "${model}"
+    --alias Qwen3.8-Flash-CIRU-STRIX-IU4
+    --host "${HOST:-127.0.0.1}"
+    --port "${PORT:-8080}"
+    --jinja
+    --ple-sidecar "${ple_dir}"
+    --ple-cache-mib "${PLE_CACHE_MIB:-4096}"
+    --slot-save-path "${slot_dir}"
+    -ngl all
+    -sm none
+    --fit off
+    -c "${CONTEXT_SIZE:-262144}"
+    -b "${BATCH_SIZE:-1024}"
+    -ub "${UBATCH_SIZE:-1024}"
+    --parallel "${PARALLEL_SLOTS:-1}"
+    -t "${THREADS:-8}"
+    -tb "${BATCH_THREADS:-8}"
+    -ctk f16
+    -ctv f16
+    -fa on
+    --cont-batching
+    --cache-prompt
+    --cache-ram "${PROMPT_CACHE_MIB:-8192}"
+    --cache-idle-slots
+    --ctx-checkpoints "${CTX_CHECKPOINTS:-32}"
+    --checkpoint-min-step "${CHECKPOINT_MIN_STEP:-8192}"
+    --temp "${TEMPERATURE:-1.0}"
+    --top-p "${TOP_P:-0.95}"
+    --top-k "${TOP_K:-20}"
+    --min-p "${MIN_P:-0.0}"
+    --metrics
+    --slots
+)
+
+if [[ "$enable_vision" == 1 ]]; then
+    args+=(--mmproj "$mmproj")
+fi
+
+if [[ "${ENABLE_MTP:-1}" != "0" ]]; then
+    if [[ ! -f "${draft}" ]]; then
+        echo "MTP is enabled but the draft model is missing: ${draft}" >&2
+        exit 2
+    fi
+    args+=(
+        --spec-type draft-mtp
+        --spec-draft-model "${draft}"
+        --spec-draft-ngl all
+        --spec-draft-device "${DRAFT_DEVICE:-ROCm0}"
+        --spec-draft-type-k q8_0
+        --spec-draft-type-v q8_0
+        --spec-draft-threads "${DRAFT_THREADS:-8}"
+        --spec-draft-threads-batch "${DRAFT_BATCH_THREADS:-8}"
+        --spec-draft-n-max "${MTP_DEPTH:-6}"
+        --spec-draft-n-min 0
+        --spec-draft-p-min 0.0
+        --spec-draft-p-split 0.10
+    )
+fi
+
+unset GGML_HIP_GRAPH_EXEC_UPDATE CIRU_MTP_GPU_CONFIDENCE CIRU_MTP_GPU_ADAPTIVE CIRU_MTP_GPU_CONF_MIN CIRU_MOE_EXPERT_REUSE CIRU_MTP_TRACE CIRU_MTP_CONF_TRACE LD_PRELOAD
+# Backend discovery must not scan an unrelated working directory.
+cd -- "${repo_root}"
+exec "${server_bin}" "${args[@]}" "$@"

vision/README.md

@@ -0,0 +1,27 @@
+# Optional vision projector
+
+Enable image input with `bash ./model/run-server.sh --vision` and the same `RUNTIME_DIR` / `BUILD_DIR` used for text serving. `ENABLE_VISION=1` is equivalent. To use a different projector location, set `MMPROJ`.
+
+`mmproj-Qwen3.8-Flash-F16.mmproj` is a GGUF-format projector converted directly from [the pinned original Qwen checkpoint](https://huggingface.co/Qwen/Qwen3.8-Flash-Next/tree/f5d08274bafd880402bd16f5e3e6c514136ec06c) with the [CIRU v3 converter](https://github.com/ciru-ai/Qwen3.8-Flash-CIRU-STRIX-IU4/blob/v3.0.0/conversion/qwen4exp.py). Conversion produces 334 tensors, with `clip` architecture, `qwen3vl_merger` projector type, and projection dimension 2560 matching the language model.
+
+The `.mmproj` extension avoids confusing Hub model-size detection with a second standalone GGUF model. The runtime reads the GGUF header regardless of filename extension.
+
+- Precision: F16 conversion, with converter-selected F32 tensors retained.
+- File size: 904,004,320 bytes.
+- SHA-256: `db643482521c722ff1074afd5018c060ef6ce9b828421c7cfc27b2f235c2569b`.
+- License: the repository's existing Qwen Community License applies.
+
+Reproduction, using a local copy of the pinned source checkpoint:
+
+```bash
+python3 ciru-runtime/convert_hf_to_gguf.py /path/to/pinned-qwen-source \
+  --mmproj --outtype f16 \
+  --outfile mmproj-Qwen3.8-Flash-F16.mmproj
+```
+
+Verify the downloaded file from this directory with `sha256sum -c SHA256SUMS`. [Machine-readable provenance](provenance.json).
+
+## Validation
+
+On Ciru (Ryzen AI Max+ 395, gfx1151, 128 GB shared memory, NixOS/ROCm 10), the released launcher loaded this projector with the existing CIRU v3 runtime, a 262,144-token context, one slot, and MTP depth 6. Two separate OpenAI-compatible image requests returned `red square` and `Blue circle` for corresponding synthetic images. These are functional smoke tests, not a vision benchmark or a claim about OCR, detailed reasoning, or video quality. The text-only benchmark tables on the main card remain text-only measurements. [Validation record](validation.json).
+

vision/SHA256SUMS

@@ -0,0 +1 @@
+db643482521c722ff1074afd5018c060ef6ce9b828421c7cfc27b2f235c2569b  mmproj-Qwen3.8-Flash-F16.mmproj

vision/mmproj-Qwen3.8-Flash-F16.mmproj

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db643482521c722ff1074afd5018c060ef6ce9b828421c7cfc27b2f235c2569b
+size 904003840

vision/provenance.json

@@ -0,0 +1,8 @@
+{
+  "source": "Qwen/Qwen3.8-Flash-Next",
+  "source_revision": "f5d08274bafd880402bd16f5e3e6c514136ec06c",
+  "converter": "CIRU v3 conversion/qwen4exp.py Qwen4ExpVisionModel",
+  "outtype": "f16",
+  "sha256": "db643482521c722ff1074afd5018c060ef6ce9b828421c7cfc27b2f235c2569b",
+  "bytes": 904004320
+}

vision/validation.json

@@ -0,0 +1,11 @@
+{
+  "status": "PASS",
+  "vision": true,
+  "mtp": true,
+  "context": 262144,
+  "responses": [
+    "Red square",
+    "Blue circle"
+  ],
+  "scope": "Two synthetic image grounding smoke tests; not a vision quality benchmark"
+}