jcbtc commited on
Commit
4f881d4
verified
1 Parent(s): 88fd261

Document two-slot serving and unified KV validation; preserve v2.0 runtime

Browse files
Files changed (2) hide show
  1. README.md +19 -1
  2. v2.0-checksums.sha256 +1 -1
README.md CHANGED
@@ -171,7 +171,7 @@ cd ciru-runtime-v2.0
171
  ./scripts/ciru/setup-linux-amd.sh --install-host-deps
172
  ```
173
 
174
- The original archive and Hugging Face `v2.0` revision remain unchanged. Later documentation clarifications appear on `main`; use the [current running instructions](https://github.com/ciru-ai/Qwen3.8-Flash-CIRU-STRIX-IU4/blob/main/docs/RUNNING.md) for profile verification and draft-depth guidance.
175
 
176
  Benchmarked on NixOS with ROCm10. The runtime also passed an Ubuntu 24.04 container build and GPU smoke test.
177
 
@@ -231,6 +231,24 @@ Thinking defaults are temperature 1.0, top-p 0.95, top-k 20, min-p 0. For non-th
231
 
232
  `min_p=0` explicitly follows [Qwen's model-specific recommendation](https://huggingface.co/Qwen/Qwen3.8-Flash-Next/blob/f5d08274bafd880402bd16f5e3e6c514136ec06c/README.md#best-practices); v1.1 could inherit llama.cpp's `0.05`. This disables an additional probability filter and can change sampled output. We have not established that either value gives better answer quality in a controlled A/B test. Set `MIN_P=0.05` with the launcher to select the previous inherited value; request-level settings can override the server default.
233
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
234
  ## Historical task-quality results on the unchanged weights
235
 
236
  These earlier H96/depth-1 results were **not rerun as full suites on v2.0**. The current release evidence comprises the bounded BF16 panel, controlled coding probe, context sweeps, and scoped runtime correctness/build checks described above.
 
171
  ./scripts/ciru/setup-linux-amd.sh --install-host-deps
172
  ```
173
 
174
+ The original archive and Hugging Face `v2.0` revision remain unchanged. Later documentation clarifications and an early multi-slot MTP launcher check appear on GitHub `main`; the released inference code is unchanged. Use the [current running instructions](https://github.com/ciru-ai/Qwen3.8-Flash-CIRU-STRIX-IU4/blob/main/docs/RUNNING.md) for profile verification and draft-depth guidance.
175
 
176
  Benchmarked on NixOS with ROCm10. The runtime also passed an Ubuntu 24.04 container build and GPU smoke test.
177
 
 
231
 
232
  `min_p=0` explicitly follows [Qwen's model-specific recommendation](https://huggingface.co/Qwen/Qwen3.8-Flash-Next/blob/f5d08274bafd880402bd16f5e3e6c514136ec06c/README.md#best-practices); v1.1 could inherit llama.cpp's `0.05`. This disables an additional probability filter and can change sampled output. We have not established that either value gives better answer quality in a controlled A/B test. Set `MIN_P=0.05` with the launcher to select the previous inherited value; request-level settings can override the server default.
233
 
234
+ ### Parallel agents and unified KV
235
+
236
+ **The released MTP shortlist supports one slot only.** For two slots, disable MTP explicitly; otherwise the original v2.0 runtime asserts during initialization. The current GitHub `main` launcher now catches this before loading the model. The original `v2.0` tag and archive remain unchanged.
237
+
238
+ From either the original v2.0 runtime directory or the current repository:
239
+
240
+ ```bash
241
+ ENABLE_MTP=0 PARALLEL_SLOTS=2 CONTEXT_SIZE=524288 \
242
+ BUILD_DIR="$PWD/build-gfx1151-sdk" MODEL_DIR=/absolute/path/to/model \
243
+ ./scripts/ciru/run-server.sh --no-kv-unified
244
+ ```
245
+
246
+ With separate KV caches, `524288` is the total context allocation: **262144 per slot**, not 512K per agent. Inspect `/slots` for the actual limits.
247
+
248
+ The released CIRU/ROCm 10 runtime passed 16 target-only requests in our 2026-09-06 two-slot smoke: eight with separate KV and eight with unified KV, including overlapping short/long and streamed requests. This does not validate filled 512K contexts or every agent workload. The older upstream fragmented-KV fix is already included. A separate proposed HIP host-buffer patch is being held because it changed the MTP coding continuation during regression testing; this update does not claim that HIP issue is fixed.
249
+
250
+ [Two-slot instructions and upstream issue links](https://github.com/ciru-ai/Qwen3.8-Flash-CIRU-STRIX-IU4/blob/main/docs/RUNNING.md#parallel-requests-and-unified-kv-cache) 路 [Validation report](https://github.com/ciru-ai/Qwen3.8-Flash-CIRU-STRIX-IU4/blob/main/docs/PARALLEL_VALIDATION.md)
251
+
252
  ## Historical task-quality results on the unchanged weights
253
 
254
  These earlier H96/depth-1 results were **not rerun as full suites on v2.0**. The current release evidence comprises the bounded BF16 panel, controlled coding probe, context sweeps, and scoped runtime correctness/build checks described above.
v2.0-checksums.sha256 CHANGED
@@ -1,5 +1,5 @@
1
  c16fda8809ad10f0d25d2243f4f83e76c29b80e0ee1622a4357a6f7235df97b4 .gitattributes
2
- c1640924dd034fd2731875a737a8fb0c9d579d44eb6e3df39503133f5cfb2fde README.md
3
  3a46a7fd054305b2c1a931682015d93094312fb3b76c8ef83a2adb350b0f584c assets/qwen38-flash-ciru-strix-iu4.jpg
4
  e260f72cba14e4c4e523794345f7de001444626240ab620d96dc44ee51beee90 assets/v2-bf16-fidelity.png
5
  88dd940fda0790ec6f474dc0a99019703114df253758ee14136c9686e93ea404 assets/v2-context-sweep.png
 
1
  c16fda8809ad10f0d25d2243f4f83e76c29b80e0ee1622a4357a6f7235df97b4 .gitattributes
2
+ f369a1ea3134d72f96dbd9290752fb860334aea20d961eaeb761732bff4be0aa README.md
3
  3a46a7fd054305b2c1a931682015d93094312fb3b76c8ef83a2adb350b0f584c assets/qwen38-flash-ciru-strix-iu4.jpg
4
  e260f72cba14e4c4e523794345f7de001444626240ab620d96dc44ee51beee90 assets/v2-bf16-fidelity.png
5
  88dd940fda0790ec6f474dc0a99019703114df253758ee14136c9686e93ea404 assets/v2-context-sweep.png