Qwen3.8-Flash-CIRU-STRIX-IU4 · v4.3.0

A Qwen3.8 Flash package for 128 GB AMD Strix Halo machines, with long-context text generation, an OpenAI-compatible API, a web UI and optional image input. V4 brings the reference prefill optimizations to the existing CIRU weights and CIRUPLE1 files. It requires the custom CIRU runtime; stock llama.cpp and Hugging Face hosted inference do not support this package.

The original fast-prefill breakthrough came from Halogen's creator, Peonist.ai (peonist-ai). pwilkin reproduced it in an open-source llama.cpp implementation and published the optimizations adapted here. CIRU contributed compatibility work and validation.

Performance and limits · Quality history · Install · Settings · Vision · Files · Earlier comparisons

v4.3 patch: speculative tuning

V4.3 makes the tested speculative-decoding settings the defaults: MTP depth 3 for IU4, depth 4 for Orca, and LLAMA_MTP_QSA_MIN_T=1 for both. Shorter draft batches reduce discarded speculative work; the lower threshold enables sparse draft attention for small batches. The corrected v4.2 inference binaries and existing model files are retained.

Model	Hermes mean score /100, passes 1 / 2	Full-score tasks, passes 1 / 2	Prompt tok/s	Generation tok/s
IU4	95.0 / 98.5	17/20 / 19/20	548.5	34.74
Orca	92.5 / 98.5	17/20 / 19/20	551.3	34.44

Hermes Agent 20: two passes per model, 80 completed scenario attempts and 449 model requests, a 32-turn allowance, production sampling and native xhigh thinking. Speeds are pooled across each model's two passes; generation includes reasoning. Both hosts are 128 GB Strix Halo / gfx1151, NixOS / ROCm 10; IU4 ran on Sozo and Orca on Ciru. These are observed tuned-configuration results, not a matched Hermes speedup over v4.2. Native verifier scores are retained; the detailed report records grading limitations and Orca's memory-persistence behavior.

Existing v4.2 users can update the launcher without rebuilding or downloading weights. MTP_DEPTH and LLAMA_MTP_QSA_MIN_T remain overridable; MTP_DEPTH=6 LLAMA_MTP_QSA_MIN_T=128 restores the previous speculative defaults. Earlier runtimes must first obtain the v4.2 attention correction. Context remains 262144, batch/microbatch 8192, with F16 target/draft KV and unchanged sampling and thinking.

Hermes task results and grading notes | v4.3 release

Launcher update, 16 September: the current main launcher checks the HIP library selected by the dynamic loader for the v4.2 indexed-attention correction before loading the model. It stops with upgrade instructions if an older or incomplete runtime is selected, and prints the actual server/library paths and library hash. Updating the model folder alone does not upgrade RUNTIME_DIR, BUILD_DIR or SERVER_BIN. This check was added after the immutable v4.3.0 tag; fetch the current launcher below to use it with an existing corrected v4.2/v4.3 installation.

For an existing corrected v4.2 or v4.3 installation, download and verify the current launcher:

hf download jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 \
  --revision main run-server.sh launcher-checksums.sha256 --local-dir ./model
(cd model && sha256sum -c launcher-checksums.sha256)
RUNTIME_DIR="$(realpath ./ciru-runtime-v4.2.0)" bash ./model/run-server.sh

Use your actual runtime/model paths. The new launcher applies the tuning even with the existing v4.2 profile.

Previous v4.2 patch: attention correctness

Upgrade the runtime; keep your existing model files. V4.2 fixes the fallback attention path that ignored selected keys in v4.0/v4.1 and could produce incorrect computation associated with the reproduced repetition incident. QSA3 stays enabled. Long thinking and production reasoning settings are unchanged.

The correction passed 48/48 CPU-reference checks, and both IU4 and Orca completed the frozen medium-effort incident replay with valid tool calls and no detected repeated long line. This is targeted regression evidence, not a new broad quality benchmark.

Measured decode tradeoff: one matched IU4 workload measured 25.54-25.55 -> 21.20-21.27 tok/s (-16.9%). The two mirrored comparisons were -17.04% and -16.71%, making the effect repeatable on that workload. Prefill remained within a 5% parity band. Only one prompt/seed and hardware/configuration were tested; this is not a universal 17% slowdown or a measured Orca speed result. Draft acceptance also fell, so the full difference cannot be attributed to the fallback kernel alone. The old engine performed incorrect attention.

We are investigating new GPU kernels and profiling the corrected attention path to increase performance further, with correctness and complete serving measurements required for any promotion.

v4.2 patch notes and measurement details · Release and downloads

Previous v4.1 patch

Faster PLE loading, with your existing model files. V4.1 enables 32 I/O workers by default; set GGML_QWEN4EXP_PLE_IO_WORKERS=16 for the previous setting. Non-Orca native 16k prefill improved 4.44%, and a quick matched MTP6 check found no TG loss. Release and downloads.

Historical V4 performance and limits

These measurements predate the v4.2 attention correction. The v4.2 cost screen is reported above.

Measured on Ciru's Ryzen AI Max+ 395 / Radeon 8060S / gfx1151, 128 GB unified memory, NixOS and stock TheRock ROCm 10. These are diagnostic observations with LLAMA_TRACE=1 and verbosity 4; production omits tracing. Each row is one observation, except the range across five different coding requests. This is not a clean repeated comparison with the older runtimes below.

V4 request	Input tokens	Prompt tok/s	Generation tok/s	First streamed piece (s)	Whole request (s)
Fixed token input, 128 output	4,096	631.04	14.61	6.49	15.19
Cold key recall	7,951	973.71	26.23	8.25	8.82
Five cold coding prompts	30,847–30,886	991.76–1,000.81	35.13–40.80	30.87–31.12	32.26–37.18
Cold key recall	65,295	947.89	32.56	69.00	69.77
Fixed token input, 128 output	16,384	1,003.33	14.43	16.45	25.25
Capacity, fixed input, 128 output	261,888	748.38	6.22	349.97	370.39

Cached requests are excluded. Three short coding requests measured 51.21–52.93 generation tok/s. Output content, speculative acceptance and context depth affect decode speed; the slower fixed-token and full-context results remain visible above.

Validation and unfinished work

• Serving: 15/15 requests completed; 5/5 recall and 3/3 cache/replay checks passed. Minimum available memory: 6.82 GiB.
• Capacity: the final row generated 128 tokens without truncation or cache reuse. Minimum available memory: 7.70 GiB. This measures capacity, not accuracy across that context.
• Numerics: 22/22 GPU cases passed, including typed Q4_1 expert/GLU/down paths and retained IQ4 regressions.
• Interfaces: relocated binaries/UI assets passed UI/chat checks and two target-only image smoke checks. V4 reuses that measured engine with corrected release, attribution and displayed-version metadata.

The eight-task coding screen scored 7/8 base and extended tests as standalone modules. One answer omitted from typing import List, supplied in its prompt. A separate rescore, prepending only each prompt's original top-level imports uniformly, scored 8/8 on the same answers. The standalone failure is retained; no answers were regenerated or selectively repaired. Five prompts were padded to about 30.8K tokens and three were short. This bounded nonthinking screen is not a full-suite quality result.

The performance gap remains unresolved. Some reference paths do not support these quantization types, and the gap to the author's different reference weights has not been causally explained. The MTP state/parameter repairs are implemented; how much further speed can transfer is still open. Vision with MTP and multi-slot MTP are not qualified.

Full qualification and failure history · Structured qualification · Raw evidence

Quality benchmark history

These scores used the same model artifacts on earlier runtimes. Each retains its full-dataset or sampled scope; they are not v4 reruns.

Benchmark	Historical score	Scope
HumanEval	160/164 (97.56%)	Full local-custom chat run
HumanEval+	155/164 (94.51%)	Full 164-task EvalPlus extended tests
ARC-Challenge	1,143/1,172 (97.53%)	Full EvalScope dataset
GPQA-Diamond	46/50 (92.0%)	Sampled subset
MMLU-Pro	61/70 (87.14%)	Five questions per subject
GSM8K	97/100 (97.0%)	Sampled subset
IFEval prompt strict	92/100 (92.0%)	Sampled subset
ToolEval Standard	115/138 points (83.33%)	69 local-custom cases
ToolEval Hard	23/30 points (76.67%)	15 local-custom cases

EvalScope wall time — 29 August H96 / MTP1 run

One request at a time, uncapped natural-EOS generation. These intervals cover 1,492 items and exclude setup and earlier failed attempts.

Dataset	Items	Wall time
ARC-Challenge	1,172	24m 42s
GPQA-Diamond	50	1h 33m 21s
MMLU-Pro	70	36m 37s
GSM8K	100	20m 19s
IFEval	100	22m 50s
Quality suite total	1,492	3h 17m 50s

Including the separate performance probe, the run took 3h 20m 46s. This is a different workload from the v3 mixed task panel. No full EvalScope wall-time measurement is available for v3 or v4.

Timestamp-derived records · Historical methodology

Download, build and run

The unchanged model files total 135,962,881,135 bytes (126.625 GiB), excluding runtime, reports, and the optional 904,003,840-byte (0.842 GiB) vision projector. The target is 73.945 GiB, the PLE payload 48.828 GiB and the Q8 MTP head 3.852 GiB. Allow at least 160 GiB for model storage/verification plus SDK and build space. Use fast NVMe and a Strix Halo machine with 128 GiB unified memory.

On Ubuntu/Debian, install Git and a Python virtual environment, then download the model and build the source:

sudo apt-get update
sudo apt-get install -y git python3-venv
python3 -m venv .venv-hf
.venv-hf/bin/python -m pip install -U huggingface_hub
. .venv-hf/bin/activate
hf download jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 \
  --revision v4.3.0 --local-dir ./model
(cd model && sha256sum -c checksums.sha256 && sha256sum -c v4.3.0-checksums.sha256)
git clone --branch v4.3.0 --single-branch \
  https://github.com/ciru-ai/Qwen3.8-Flash-CIRU-STRIX-IU4.git ciru-runtime-v4.3.0
cd ciru-runtime-v4.3.0
./scripts/ciru/setup-linux-amd.sh --install-host-deps
BUILD_DIR="$PWD/build-gfx1151-sdk" MODEL_DIR="$(realpath ../model)" \
  ./scripts/ciru/run-server.sh

The helper installs a private complete ROCm 10.0.0 SDK with gfx1151 libraries. Keep the SDK directory used by the build; the host needs a compatible AMD driver and access to /dev/kfd and its render node. V4 GPU qualification is NixOS/ROCm 10/gfx1151. The prior clean Ubuntu qualification belongs to v2.0.1, so the Ubuntu instructions are retained build guidance, not a new v4 Ubuntu test result. See platform/build instructions.

Existing v4.2 users can use the launcher-only update above. Earlier installations need the corrected runtime; keep the existing model files. The source archive is equivalent to the Git tag including executable modes and symlinks. The optional tested NixOS binary payload requires the recorded Nix store and ROCm SDK paths; use the source build for another installation.

The setup helper installs the private SDK and builds the runtime. Open http://127.0.0.1:8080 for the UI, or use /v1/chat/completions. Keep the complete matching executable/shared-library set and ui/ assets together.

Settings and sampling

Setting	V4.3 default	Override
Context	262,144 tokens	CONTEXT_SIZE
Batch / microbatch	8,192 / 8,192	BATCH_SIZE, UBATCH_SIZE
Target / draft KV	F16 / F16	Fixed in the release launcher
Speculative decoding	MTP, maximum depth 3	MTP_DEPTH; ENABLE_MTP=0 for target-only
Sparse draft threshold	1 token	LLAMA_MTP_QSA_MIN_T
Server slots	1	See parallel restrictions below
CPU threads	8	THREADS, BATCH_THREADS
Prompt RAM cache	1,024 MiB	PROMPT_CACHE_MIB
PLE cache	4,096 MiB	PLE_CACHE_MIB
PLE I/O workers	32	GGML_QWEN4EXP_PLE_IO_WORKERS=16 for the previous setting
Context checkpoints	32, minimum step 8,192 tokens	CTX_CHECKPOINTS, CHECKPOINT_MIN_STEP
Sampling	Temperature 1.0, top-p 0.95, top-k 20, min-p 0	TEMPERATURE, TOP_P, TOP_K, MIN_P or request parameters

Thinking follows the embedded template. Context shifting and unified KV are disabled. Larger batch/cache settings crossed the memory guard during qualification. Saved slots use slot-state/v4.3.0; old saves are not restored automatically, and prefixes without a matching MTP checkpoint safely reprocess.

MODEL_DIR, SERVER_BIN and BUILD_DIR select files/builds; HOST and PORT select the endpoint. UI_DIR selects the external UI; ENABLE_UI=0 runs API-only. Supported kernel flags accept explicit 0 overrides. MTP requires one slot; the launcher rejects multi-slot MTP. Target-only multi-slot serving is an experimental override for this engine. Parallel instructions.

For a nonthinking request with the evaluated sampling settings:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen3.8-Flash-CIRU-STRIX-IU4",
    "messages": [{"role": "user", "content": "Write a Python CSV validator with tests."}],
    "chat_template_kwargs": {"enable_thinking": false},
    "temperature": 0.7, "top_p": 0.8, "top_k": 20,
    "min_p": 0, "presence_penalty": 1.5, "cache_prompt": true
  }'

These settings follow the Qwen model card. Ordinary chat can reuse prompt cache; cold requests, fixed seeds and output caps in benchmarks are measurement controls.

Vision and harness setup

The optional projector contains 334 tensors converted to F16 from the pinned Qwen checkpoint using the CIRU v3 converter. Its .mmproj extension keeps Hub size detection focused on the language model; it is still a GGUF-format file. Projector provenance.

From the directory containing model/ and ciru-runtime-v4.3.0/, fetch the optional files if needed and launch image mode:

hf download jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 \
  --revision main run-server.sh 'vision/*' --local-dir ./model
(cd model/vision && sha256sum -c SHA256SUMS)

RUNTIME_DIR="$PWD/ciru-runtime-v4.3.0" \
  BUILD_DIR="$PWD/ciru-runtime-v4.3.0/build-gfx1151-sdk" \
  bash ./model/run-server.sh --vision

MTP stays enabled with vision in the current main model-directory launcher. Set ENABLE_MTP=0 for target-only generation, or ENABLE_MTP=1 to enable MTP explicitly. ENABLE_VISION=1 is equivalent to --vision; MMPROJ overrides the projector path. A missing projector fails before model load. The previous image smoke checks used target-only generation; vision with MTP has not been inference-qualified.

Existing installations and older versions

The versioned release archives and tags retain their original launchers. In those launchers, ENABLE_MTP=1 alone does not override the --vision guard. You can keep your existing runtime and pass the projector directly instead:

ENABLE_VISION=0 ENABLE_MTP=1 \
  MODEL_DIR="$(realpath ./model)" \
  RUNTIME_DIR="$(realpath ./ciru-runtime-v4.1.0)" \
  bash ./model/run-server.sh \
  --mmproj "$(realpath ./model/vision/mmproj-Qwen3.8-Flash-F16.mmproj)"

Use your installed runtime directory in RUNTIME_DIR. Do not add --vision to this older-launcher workaround. It enables image input through --mmproj while retaining the runtime's usual MTP arguments. For a launcher inside the runtime directory, use that launcher path instead; Orca needs its Orca launcher/model selection. This is a startup option, not a per-request switch.

For v4.1.0, downloading only run-server.sh from main as shown above installs the corrected launcher; no model-weight download or binary rebuild is needed. It runs against RUNTIME_DIR directly instead of delegating to the archived launcher. ENABLE_MTP=0 remains the opt-out.

V4.2 includes this launcher behavior in its versioned runtime and model-side launcher. The workaround above remains for older installations.

Send images through the UI or the chat endpoint. Use data URLs with the tested Nix binary, whose remote HTTPS fetching is disabled. Source builds can enable HTTPS when OpenSSL is found. Example using a local image.png:

python3 - <<'PYIMAGE'
import base64, json, urllib.request
from pathlib import Path
image = base64.b64encode(Path("image.png").read_bytes()).decode()
payload = {
    "model": "Qwen3.8-Flash-CIRU-STRIX-IU4",
    "messages": [{"role": "user", "content": [
        {"type": "text", "text": "Describe this image."},
        {"type": "image_url", "image_url": {"url": "data:image/png;base64," + image}}
    ]}],
    "chat_template_kwargs": {"enable_thinking": False},
    "max_tokens": 512
}
request = urllib.request.Request(
    "http://127.0.0.1:8080/v1/chat/completions",
    data=json.dumps(payload).encode(),
    headers={"Content-Type": "application/json"}
)
with urllib.request.urlopen(request, timeout=300) as response:
    print(json.load(response)["choices"][0]["message"]["content"])
PYIMAGE

Image resolution

The defaults for --image-max-tokens and --image-min-tokens follow the loaded vision model/backend. Set --image-max-tokens 4096 for an explicit ceiling matching the earlier v3 profile, or try 8192 for a larger budget. --image-min-tokens 1024 can help with small-detail grounding.

Large images are resized while preserving aspect ratio; each separately submitted image/crop has its own budget. Larger budgets use more memory/context and take longer. Upscaling cannot restore missing detail, and client-side resizing still applies. Text performance measurements do not cover image processing.

DeepSeek harness

Start with --vision and use input in place of inputModalities in the harness model entry:

input:
  - text
  - image

This configuration was reported working by an Orca user of the DeepSeek harness; it is retained as integration guidance.

Model files

V4 uses the same released model files. The target GGUF has 1,223 tensors: 144 Q4_1 routed-expert tensors, 328 Q5_K, 290 Q8_0, 48 Q5_1, 25 BF16 and 388 F32. The PLE payload preserves exact FP8 E4M3 weights with one BF16 scale. The IU4 model name is retained; it does not imply that every new runtime kernel uses native IU4 arithmetic.

Required component	Repository path	Purpose
Target model	Qwen3.8-Flash-CIRU-STRIX-IU4.gguf	Main model weights
PLE sidecars	ple/ple.manifest.json, ple/ple.payload.bin, ple/ple.scale.bf16	Complete CIRUPLE1 lookup storage; keep all three together
MTP draft	mtp/Qwen3.8-Flash-CIRU-STRIX-IU4-MTP-Q8_0.gguf	Speculative decoding for the default text profile
Optional vision projector	vision/mmproj-Qwen3.8-Flash-F16.mmproj	Image input with --vision; 904,003,840 bytes
Custom runtime	runtime/v4.2.0/ or the GitHub v4.2.0 tag	Matching source, launcher/profile and external UI

The engine preserves CIRUPLE1 compatibility and adds type-correct Q4_1 expert/GLU/down paths and MTP state/parameter repairs. Type and shape checks remain in place; incompatible quantization types do not enter IQ4-only paths, and whole-expert BF16 weight shadows are not introduced. See v4 qualification and source identity.

Model file tree · Weight checksums · V4 runtime/report checksums · GitHub release

Earlier runtime comparisons

Every result below belongs to its named runtime, protocol and workload. These panels were not rerun for v4. They provide bounded comparisons, not broad model-equivalence or leaderboard claims. Expand a section for the complete tables and methods.

V3 versus previous CIRU and Halo — serving speed, MTP2, quality and capacity

Serving speed

One Ryzen AI Max+ 395 / gfx1151 / 128 GB NixOS host; identical input token IDs, cold cache, 128 generated tokens, one slot and 262,144-token capacity. Nonthinking sampler: temperature 0.7, top-p 0.8, top-k 20, min-p 0, presence penalty 1.5, repeat penalty 1, frequency penalty 0 and seed 123; EOS honored.

Input tokens	Profile	Prompt tok/s	Generation tok/s	First streamed piece (s)	Whole request (s)
4,096	Previous CIRU	392.00	22.52	10.70	16.34
4,096	CIRU v3	455.65	24.60	9.25	14.41
4,096	Halo	381.49	35.30	11.09	14.69
65,536	Previous CIRU	284.49	13.33	230.46	239.99
65,536	CIRU v3	369.81	24.22	177.32	182.57
65,536	Halo	263.42	23.28	248.91	254.37

Against previous CIRU, v3 cut whole-request time 11.80% at 4K and 23.93% at 64K. At 64K, prefill rose 29.99% and generation 81.66%. First-piece time is the first streamed content-field event.

Previous CIRU used v2.0.1/MTP6/b2048/u512; v3 used the same weights with MTP6/b1024/u1024. Halo used unmodified commit 5f851647fe5ed795dfd6c0a3fba543114879e874, Vulkan, Unsloth UD-Q4_K_XL and the EasiiX Strix Q8 head, retaining its native KV, batch, thread, fitting and cache defaults. Different quantizations make this a serving-package comparison.

Halo's short-context depth screen measured 35.37 generation tok/s at depth 3, 31.4 at depth 2, 30.04 at depth 4, 25.91 at depth 6 and 29.31 with adaptive 6. Depth 3 was selected. Three clean v3 loads, two selected Halo loads and one previous-CIRU load are bounded evidence, not a confidence interval. Halo's generation advantage is retained.

The 2.79 GB Unsloth shared Q8 head requires borrowed main-model tensors. The pinned Halo loader failed on token_embd.weight; Unsloth's self-contained Q8 head failed on output_hc_norm.weight. Both attempts were recorded before using the compatible EasiiX head. Halo source and weights were unchanged.

Optional v3 MTP2

Input tokens	Optional v3 MTP 2 prompt tok/s	Generation tok/s	Whole request (s)
4,096	453.09	29.39	13.62
65,536	373.08	24.88	180.87

MTP2 helped the tested lower-acceptance long requests and gave a clearer 4K latency benefit. MTP6 remained the default because it was faster on the short coding panel. Target verification retains the full vocabulary at either depth.

Comparison and memory report · Structured results · Raw evidence

Quality and capacity

Profile	HumanEval base	EvalPlus extended tests	Recall at about 8K and 64K
Previous CIRU	20/20	20/20	Both keys and exact cached replay
CIRU v3	20/20	20/20	Both keys and exact cached replay
Halo	20/20	20/20	Both keys and exact cached replay

HumanEval tasks 0–19, EvalPlus v0.1.10, first samples without retries, a 4096-token cap, and truncations counted as failures; generated code ran in a filesystem/network sandbox. V3 also completed 261,888 input + 128 output at 257.44 PP / 18.00 TG in 1024.44 s. That was a CIRU-only capacity check, not full-context accuracy or a filled-context Halo comparison.

V3 passed 69 QSA mapping/state/guard cases, 33 actual ROCm operator reference cases and 30 batch allocator tests. A separate four-prefix diagnostic matched 15,892,480 F32 logits byte-for-byte against the previous runner. Radix selection can alter threshold ties and selected-list order; general bitwise equivalence was not claimed.

Ornith, Hermes and coding throughput — 9 September task panel

This panel measures complete tasks, including tool work, scoring and harness overhead. Arms ran sequentially on Ciru with one slot: previous CIRU v2.0.1 and v3 used MTP6; unchanged Halo Vulkan used MTP3.

Benchmark stage	Previous CIRU	CIRU v3	Halo MTP3
Complete mixed panel, after readiness	29m 17.11s	25m 18.32s	24m 34.43s
Additional model load	31.17s	31.15s	33.62s
Short scored stage	5m 20.33s	5m 20.85s	4m 35.60s
Hermes, two rounds combined	12m 28.01s	11m 37.57s	10m 08.12s
Shared 63K-token history seeding	3m 26.33s	2m 42.80s	3m 21.00s
Long hard stage, history already loaded	4m 43.92s	2m 41.90s	3m 22.96s
Total generated tokens	33,294	33,451	31,058

V3 took 13.59% less total time than previous CIRU, saving 3m 58.79s; Halo finished 43.89s sooner than v3. V3's long hard-stage time fell 42.98%. Output lengths differ, so this is a workload result rather than an equal-output decode comparison. Coding checks and other overhead also contribute to the full panel.

Cases comprised 22 short academic tasks, six Hermes scenarios repeated twice, eight hard tasks returning to shared long history, and 10 short/eight long coding health checks. Selection used earlier Ornith disagreements and failures rather than a representative dataset sample.

Profile	Short IFEval strict	Short GSM8K	Short HumanEval	Long IFEval strict	Long GSM8K	Long HumanEval
Previous CIRU	5/8	8/8	5/6	2/2	2/2	2/4
CIRU v3	5/8	8/8	5/6	2/2	2/2	2/4
Halo MTP3	6/8	8/8	5/6	2/2	2/2	3/4

Profile	Hermes native full passes	Native mean points / 100	Reviewed end states
Previous CIRU	7/12	80.83	11/12
CIRU v3	11/12	95.83	12/12
Halo MTP3	11/12	95.83	12/12

Native scores remain intact. Reviewed end states separately account for grader artifacts, including memory-case wording mismatches for previous CIRU and Halo. All arms passed the 10-task short and eight-task long coding checks on base and extended tests. Two Hermes repetitions do not establish a reliable failure rate.

Short-task generation was 38.44 / 39.09 / 43.97 tok/s for previous CIRU/v3/Halo; long hard-stage generation was 21.75 / 35.93 / 31.25 tok/s. Native short tasks used temperature 0, seed 15035, nonthinking, a 32,768-token output allowance, first samples and no repair. Hermes used temperature 0.6, top-p 0.95, top-k 20, thinking and full remaining context. History seeding was timed separately; interrupted or instrumented captures were excluded.

Short coding and MTP depth

Every profile passed 20/20 base and extended tests. Times sum API requests, excluding grading.

Profile	Generated tokens	Prompt tok/s	Generation tok/s	Sum of request times
Previous CIRU MTP6	3,179	148.53	53.33	75.49s
CIRU v3 MTP6	3,179	219.51	53.24	70.67s
CIRU v3 MTP2	3,212	226.25	39.63	91.57s
Halo MTP3	3,241	176.10	49.48	79.15s

Previous CIRU and v3 MTP6 produced identical token IDs on all 20 tasks. V3 cut request time 6.39%, principally through prefill; decode was effectively unchanged. MTP2 and Halo output lengths remain in the table. This supported v3's MTP6 default, not a universal optimum for all workloads or later engines.

Hard-panel report · Results JSON · Wall times CSV · Interactive research page

V3 implementation, shortlist and earlier release provenance

The v3 launcher used F16 target KV, Q8 draft KV, MTP6, b1024/u1024, eight threads and prefix caching. Its 32,768-row shortlist limited draft projection while retaining full-vocabulary target verification. The expected v3 startup marker was CIRU MTP shortlist enabled: 32768 / 248320 vocabulary rows; full target verification retained. MTP_DEPTH=2 selected the earlier optional setting; the draft attention window remained off and unqualified. These are historical details, not current launch instructions.

V3 combines the previously qualified QSA sequence isolation/indexer-copy fixes with the portable retained hybrid work: wider aligned QSA prefill, J32 expert tiling for supported prefill shapes, radix selection, indexed F16 attention, direct guarded PLE lookup, derived-history caching keyed by sequence and packed tiny F32 gathers. The F32 derived cache adds about 192 MiB at 256K. Hybrid external-GPU ownership, peer transfers, expert splitting, scheduler overlap and experimental compact masks are excluded.

The original READY package, source, evidence and checksum trees are included in the prior-package archive. Its v2.0.1 fixes ship as part of v3; no separate public v2.0.1 tag is claimed. Existing public v2.0 tags and weight identities remain unchanged. Historical v2.0 results remain available; their 42.3 tok/s short greedy probe and broader older quality suites are different workloads and were not rerun as full suites for v3.

The v3 Q4_1 matrix path expanded packed values into byte lanes for IU8 WMMA. It did not activate the separate native IU4/E3 bank path.

Model file tree · Weight checksums · V3 runtime/report checksums · Source identity · GitHub release

Lineage, license and contributors

Text lineage is Qwen/Qwen3.8-Flash-Next; PLE lineage is Qwen3.8-Flash-Next-FP8. Runtime lineage starts from ggml-org/llama.cpp. Model artifacts use the included Qwen Community License 1.0; runtime code retains MIT and component notices.

Credits include Qwen, ggml-org, Ryan Monsurate's MTP integration, AMD's ROCm ecosystem and the contributors in the source notices. Daniel Han's authorship of the imported QSA repair is preserved. Thanks to halo-box, EasiiX, Unsloth, Daniel Han Chen, Laurent Zuijdwijk and Agention AI for public models/runtimes, and the HumanEval and EvalPlus authors for evaluation tools. CIRU is an independent community research project; AMD and Qwen marks do not imply sponsorship.

Runtime provenance · Upstream README · Third-party notices