Serving update: madeby561/vllm dark-devotion-...-mtpdcpfix image + VLLM_DCP_GLOBAL_TOPK + VLLM_DCP_SHARD_DRAFT + use_local_argmax_reduction; refresh perf (70-94 codegen, ~50@256k flat, 256k prefill no-wedge, ~200@C4, 474k pool)

README.md

@@ -74,7 +74,14 @@ Surviving experts are luke's NVFP4 weights bit-for-bit. **Saliency** `S_j = mean
 ```bash
 MODEL_DIR=/path/to/GLM-5.2-NVFP4-REAP-504B-term docker compose up -d   # OpenAI API on :5001, id "GLM-5.2"
 ```
-The included compose defaults to the **best config found: DCP4 + MTP5 + `use_index_cache`** → **~489k-token KV pool, ~80 tok/s** single-stream codegen (~30% MTP accept). `use_index_cache` (caches the DSA top-2048 sparse indices across decode steps) is what makes DCP4 fast — previously DCP4 was comm-bound ~40 tok/s on PCIe. For max short-context decode speed: `DCP_SIZE=1 MTP=1 MAX_MODEL_LEN=125000` (~125k ctx). Tested on `voipmonitor/vllm:glm52-v11-darkdevotion-...-cu132`.
+The included compose defaults to the **best config found: DCP4 + MTP5 + global-topk + DCP-sharded draft + `use_index_cache`**, on the patched serving image **[`madeby561/vllm:dark-devotion-…-mtpdcpfix`](https://hub.docker.com/r/madeby561/vllm)** — voipmonitor's `dark-devotion-…-dcpglobaltopk` base plus a one-file `deepseek_mtp.py` fix that lets `VLLM_DCP_SHARD_DRAFT` run the MTP draft DCP-parallel on GLM-5.2 (full write-up on the image page). Measured single-stream on 4× RTX PRO 6000 (PCIe, TP4 / DCP4 / MTP5):
+
+- **70–94 tok/s** codegen
+- **~50 tok/s at 256k context** — decode stays *flat* with depth (DSA sparse attention: 56 tok/s at 0 ctx → 49 at 256k)
+- **256k cold prefill: no wedge**; **~200 tok/s aggregate at concurrency 4**
+- **474k-token KV pool** @ util 0.95, `MAX_MODEL_LEN=300000`
+
+`VLLM_DCP_GLOBAL_TOPK=1` feeds the speculative draft the model's true global-topk attention target (faster *and* higher-fidelity than local-topk); `VLLM_DCP_SHARD_DRAFT=1` shards that draft across the DCP ranks; `use_index_cache` caches the DSA top-2048 sparse indices across decode steps (DCP4 was comm-bound ~40 tok/s on PCIe without it). For max short-context decode speed: `DCP_SIZE=1 MTP=1 MAX_MODEL_LEN=125000` (~125k ctx).
 
 ## Credits
 

docker-compose.yml

@@ -1,8 +1,9 @@
 # GLM-5.2-NVFP4-REAP (168 experts / ~504B) — vLLM serving on 4× 96 GB Blackwell GPUs
 # (e.g. RTX PRO 6000, SM120). Works for both the -504B (code-calib) and -504B-term variants.
 #
-# Best config (defaults below): DCP4 + MTP5 + use_index_cache
-#   → ~489k-token KV pool, ~80 tok/s single-stream codegen (~30% MTP accept).
+# Best config (defaults below): DCP4 + MTP5 + global-topk + DCP-sharded draft + use_index_cache
+#   → 474k-token KV pool; 70-94 tok/s codegen, ~50 tok/s @256k (flat at depth),
+#     256k prefill no-wedge, ~200 tok/s aggregate @ concurrency 4.
 #
 # Quickstart:
 #   MODEL_DIR=/path/to/GLM-5.2-NVFP4-REAP-504B-term docker compose up -d
@@ -10,12 +11,12 @@
 #
 # Tuning via env (no file edits):
 #   max decode speed, short ctx :  DCP_SIZE=1 MTP=1 MAX_MODEL_LEN=125000
-#   absolute max context        :  raise MAX_MODEL_LEN toward the KV pool (~489k @ DCP4)
+#   absolute max context        :  raise MAX_MODEL_LEN toward the KV pool (~474k @ DCP4)
 #   no speculative decode       :  MTP=0
 # Sampling: temperature 0.6, top_p 0.95, repetition_penalty 1.0 (do NOT exceed 1.0).
 services:
   glm52-reap:
-    image: ${IMAGE:-voipmonitor/vllm:glm52-v11-darkdevotion-vllma86f74e-b12x5b2e018-cu132-20260618}
+    image: ${IMAGE:-madeby561/vllm:dark-devotion-df8ad3b-b12x5af873a-dcpglobaltopk-cu132-20260621-mtpdcpfix}
     container_name: ${NAME:-glm52-reap}
     network_mode: host
     ipc: host
@@ -49,6 +50,8 @@ services:
       VLLM_USE_B12X_FP8_GEMM: "1"
       VLLM_USE_B12X_MOE: "1"
       VLLM_USE_B12X_SPARSE_INDEXER: "1"
+      VLLM_DCP_GLOBAL_TOPK: "1"
+      VLLM_DCP_SHARD_DRAFT: "1"
       VLLM_USE_V2_MODEL_RUNNER: "1"
       VLLM_ENABLE_PCIE_ALLREDUCE: "1"
       VLLM_PCIE_ALLREDUCE_BACKEND: b12x
@@ -87,7 +90,7 @@ services:
         fi
         SPEC_ARGS=()
         if [ "$${MTP}" = "1" ]; then
-          SPEC_CONFIG="$${SPEC_CONFIG:-$$(printf '{"model":"%s","method":"mtp","num_speculative_tokens":%s,"moe_backend":"b12x","draft_sample_method":"probabilistic"}' "$${MTP_MODEL}" "$${NUM_SPECULATIVE_TOKENS}")}"
+          SPEC_CONFIG="$${SPEC_CONFIG:-$$(printf '{"model":"%s","method":"mtp","num_speculative_tokens":%s,"moe_backend":"b12x","draft_sample_method":"probabilistic","use_local_argmax_reduction":true}' "$${MTP_MODEL}" "$${NUM_SPECULATIVE_TOKENS}")}"
           SPEC_ARGS=(--speculative-config "$${SPEC_CONFIG}")
         fi
         echo "[glm52-reap] TP$${TP_SIZE} DCP$${DCP_SIZE} MTP=$${MTP} ctx=$${MAX_MODEL_LEN} util=$${GPU_MEMORY_UTILIZATION}"