Update primary checkpoint to DSpark v2

Replace the v1 serving weights and configuration with the validated v2 Stage 2 step-5500 export from 3bd001d1a32539bd0afeeb07165c393c6adce29b. Update the model card with the v1/v2 acceptance-length and throughput results.

README.md

@@ -2,68 +2,172 @@
 license: other
 library_name: transformers
 pipeline_tag: text-generation
+base_model: RadixArk/Qwen3.8-27B-NVFP4
 tags:
 - speculative-decoding
 - dspark
-- dflash
 - specforge
 - sglang
+- qwen3.8
 inference: false
 ---
 
 # Qwen3.8-27B-DSpark
 
-A DSpark speculator for [Qwen/Qwen3.8-27B-FP8](https://huggingface.co/Qwen/Qwen3.8-27B-FP8). DSpark extends [DFlash](https://github.com/z-lab/dflash) with target-model auxiliary features and a confidence head that dynamically chooses the number of draft tokens. The model was trained with [SpecForge](https://github.com/sgl-project/SpecForge) and is served with [SGLang](https://github.com/sgl-project/sglang).
+A DSpark speculative-decoding draft model for Qwen3.8-27B target models, trained with [SpecForge](https://github.com/sgl-project/SpecForge) and served with [SGLang](https://github.com/sgl-project/sglang). This repository contains the v2 Stage 2 checkpoint and replaces the earlier v1 checkpoint.
 
-## Model specifications
+The checkpoint has been evaluated with both [RadixArk/Qwen3.8-27B-NVFP4](https://huggingface.co/RadixArk/Qwen3.8-27B-NVFP4) and [Qwen/Qwen3.8-27B-FP8](https://huggingface.co/Qwen/Qwen3.8-27B-FP8) targets. The acceptance-length evaluation below uses the NVFP4 target. The throughput evaluation uses the FP8 target.
 
-- Target model: [Qwen/Qwen3.8-27B-FP8](https://huggingface.co/Qwen/Qwen3.8-27B-FP8)
-- Checkpoint: `epoch_2_step_4166`
-- Draft parameters: 1,359,284,737 (1.36B)
+## Checkpoint
+
+- Version: DSpark v2, Stage 2 optimizer step 5,500
+- Draft parameters: 1,857,358,337 (1.86B)
 - Draft weight dtype: BF16
 - Hidden size: 5,120
-- Transformer layers: 5 full-attention layers
-- Attention: GQA with 40 query heads and 8 key/value heads
-- Target auxiliary feature layers: 4, 16, 28, 40, 52
-- Confidence head: vanilla Markov head, rank 256
-- DSpark block size: 7 draft tokens (verify width 8, including the target bonus token)
+- Transformer layers: five full-attention layers
+- Attention: GQA with 32 query heads and eight key/value heads
+- Target auxiliary feature layers: 5, 19, 33, 47, 61
+- Markov head: VanillaMarkov, rank 256
+- Training target width: 16 future positions
+- Serving gamma: seven draft proposals
+- Target verification width: eight tokens, including the target bonus token
 - Maximum position embeddings: 262,144
+- Positional configuration: YaRN factor 32 with original context 8,192
+- Training corpus: `RadixArk/Qwen3.8-27B-Regen-Mixture-v1@59c7cd36512cf9e8450158fb949295e16d8d0661`
+- v2 source revision: `RadixArk/Qwen3.8-27B-DSpark-v2@3bd001d1a32539bd0afeeb07165c393c6adce29b`
+- Replaced v1 revision: `RadixArk/Qwen3.8-27B-DSpark@85ef153be924f17ce4bf62726954eeaa4a73e854`
+
+The serving configuration uses `block_size=7`. The separate `training_block_size=16` records the supervision width used during training.
 
 ## Acceptance length
 
-Acceptance length is the mean number of tokens accepted per speculative verification step, including the target bonus token. Each workload value below is the unweighted mean of SGLang's per-request `spec_accept_length` values.
+The primary value follows the Z-lab proposal-weighted definition within each workload:
+
+```text
+proposal_weighted_accept_length = sum(completion_tokens) / sum(spec_verify_ct)
+```
 
-Setting: FP8 target and unquantized BF16 draft; DSpark block size 7; sampling temperature 0.6, top-k 20, top-p 0.95; thinking enabled; `max_new_tokens=2048`; seed 0; up to 128 prompts per workload, or the full set when smaller. The prompt construction and measurement loop follow `z-lab/dflash`'s SGLang benchmark path.
+The request-mean value first computes `completion_tokens / spec_verify_ct` for every request, then averages requests equally. Both metrics include the target bonus token in each speculative verification step.
 
-| Workload | Acceptance length |
-|---|---:|
-| HumanEval | 3.47 |
-| GSM8K | 4.57 |
-| MATH-500 | 4.08 |
-| LiveCodeBench | 2.94 |
-| MBPP | 3.67 |
-| AIME 2025 | 3.28 |
-| LBPP | 3.03 |
-| AIME 2026 | 3.07 |
-| MT-Bench | 3.10 |
-| Arena-Hard-v2 | 2.71 |
-| Alpaca | 2.95 |
+| Category | Workload | Prompts | v1 proposal-weighted | v2 proposal-weighted | v1 request-mean | v2 request-mean |
+|---|---|---:|---:|---:|---:|---:|
+| Code | HumanEval | 164 | 3.0437 | **3.8468** | 3.2586 | **4.1300** |
+| Code | MBPP | 257 | 3.2299 | **4.0603** | 3.4998 | **4.4809** |
+| Code | LiveCodeBench | 1,055 | 2.5915 | **3.3462** | 2.8404 | **3.7141** |
+| Code | BigCodeBench | 1,140 | 2.7752 | **3.4678** | 2.8139 | **3.5285** |
+| Math | GSM8K | 1,319 | 3.6030 | **4.5162** | 4.3379 | **5.4168** |
+| Math | MATH-500 | 500 | 3.2559 | **4.2267** | 3.8614 | **5.0496** |
+| Math | AIME 2025 | 30 | 2.9798 | **3.9401** | 3.1612 | **4.2022** |
+| Math | AMC23 | 40 | 3.2111 | **4.1572** | 3.6132 | **4.6945** |
+| Math | GSM-Symbolic | 2,048 | 3.4554 | **4.2716** | 4.1915 | **5.2818** |
+| Chat | MT-Bench | 80 | 2.6075 | **3.2860** | 2.9839 | **3.7579** |
+| Chat | Alpaca | 52,002 | 2.5659 | **3.2337** | 2.8673 | **3.6994** |
+| Chat | Arena-Hard-v2 | 750 | 2.5910 | **3.2536** | 2.7114 | **3.4283** |
+| Chat | IFEval | 541 | 2.9457 | **3.6628** | 2.9392 | **3.6253** |
+| Misc. | MMLU-Pro | 2,048 | 2.8345 | **3.5964** | 3.1140 | **4.0299** |
+| Misc. | GPQA-Diamond | 198 | 2.7634 | **3.5109** | 2.8615 | **3.6709** |
+| Misc. | LongBench-v2 | 503 | 3.2602 | **3.9268** | 3.4132 | **4.0937** |
+| Misc. | RULER-8K | 2,000 | 4.9585 | **6.3009** | 4.5204 | **5.5028** |
 
-The evaluation contains 1,164 requests. The mean across all requests is **3.39**, and the macro mean across the 11 workload means is **3.35**. Every response contained acceptance statistics, and recomputing `completion_tokens / spec_verify_ct` matched `spec_accept_length` exactly.
+| Aggregate | v1 proposal-weighted | v2 proposal-weighted | Change | v1 request-mean | v2 request-mean | Change |
+|---|---:|---:|---:|---:|---:|---:|
+| Request-count weighted, 64,675 prompts | 2.721143 | **3.428567** | **+26.00%** | 3.011715 | **3.862789** | **+28.26%** |
+| Workload macro, 17 workloads | 3.098368 | **3.917881** | **+26.45%** | 3.352293 | **4.253331** | **+26.88%** |
 
-## Serving with SGLang
+Acceptance-length protocol:
+
+- Target: `RadixArk/Qwen3.8-27B-NVFP4@554ebba9b5f1b79dc11246341960360e6ef05ef4`
+- v1 draft: `RadixArk/Qwen3.8-27B-DSpark@85ef153be924f17ce4bf62726954eeaa4a73e854`
+- v2 draft: `RadixArk/Qwen3.8-27B-DSpark-v2@3bd001d1a32539bd0afeeb07165c393c6adce29b`
+- Runtime: SGLang v0.5.17 at `eec794bce0808ae26cc1dcb84a56b65d2df82af5`
+- Hardware and topology: four NVIDIA GB300 GPUs, DP4 × TP1
+- Sampling: thinking enabled, temperature 1.0, top-p 0.95, top-k 20, seed 980406
+- Generation limit: 8,192 tokens; client concurrency: 128
+- DeepSpec context length: 32,768; extended-suite context length: 262,144
+- Prompt coverage: all source prompts for nine DeepSpec workloads; at most 2,048 prompts for each extended workload; 64,675 completed requests with no request errors per candidate
+
+The request-count-weighted aggregate weights each workload value by its prompt count. The workload macro gives each workload equal weight. Alpaca contributes 52,002 of the 64,675 prompts, so the two aggregates answer different weighting questions.
+
+## Throughput
+
+Throughput is total output tokens divided by end-to-end timed wall duration. Each DSpark cell is `output tok/s (speedup over autoregressive)`.
+
+### Concurrency 1
+
+| Workload | Autoregressive | DSpark v1 | DSpark v2 |
+|---|---:|---:|---:|
+| GSM8K | 94.2 | 238.6 (2.53×) | **297.3 (3.16×)** |
+| MATH-500 | 95.0 | 214.4 (2.26×) | **280.0 (2.95×)** |
+| HumanEval | 95.8 | 205.5 (2.14×) | **254.8 (2.66×)** |
+| MBPP | 93.8 | 208.6 (2.22×) | **261.6 (2.79×)** |
+| MT-Bench | 95.8 | 171.3 (1.79×) | **215.8 (2.25×)** |
+
+### Concurrency 8
 
-Use an SGLang version containing DSPARK support (the results above used the pinned main-branch commit shown in the evaluation setting):
+| Workload | Autoregressive | DSpark v1 | DSpark v2 |
+|---|---:|---:|---:|
+| GSM8K | 602.7 | 1,183.8 (1.96×) | **1,494.0 (2.48×)** |
+| MATH-500 | 635.2 | 1,208.2 (1.90×) | **1,575.1 (2.48×)** |
+| HumanEval | 667.9 | 1,159.2 (1.74×) | **1,435.1 (2.15×)** |
+| MBPP | 635.4 | 1,123.7 (1.77×) | **1,393.7 (2.19×)** |
+| MT-Bench | 647.9 | 958.4 (1.48×) | **1,195.5 (1.85×)** |
+
+### Concurrency 32
+
+| Workload | Autoregressive | DSpark v1 | DSpark v2 |
+|---|---:|---:|---:|
+| GSM8K | 1,298.5 | 1,934.2 (1.49×) | **2,268.5 (1.75×)** |
+| MATH-500 | 1,764.2 | 2,014.2 (1.14×) | **2,545.2 (1.44×)** |
+| HumanEval | 1,862.2 | 1,918.5 (1.03×) | **2,472.3 (1.33×)** |
+| MBPP | 1,738.4 | 1,926.3 (1.11×) | **2,413.1 (1.39×)** |
+| MT-Bench | 1,814.2 | 1,593.3 (0.88×) | **1,973.0 (1.09×)** |
+
+Throughput protocol:
+
+- Target: `Qwen/Qwen3.8-27B-FP8@017b9c7af6b5689d5dd426a76e0bc077eb5ca20a`
+- v1 and v2 revisions match the acceptance-length protocol above
+- Hardware and topology: one NVIDIA H200 per workload, TP1 × DP1
+- 128 prompts per cell, dataset shuffle seed 42, concurrency 1/8/32, `max_tokens=2048`, reasoning effort `xhigh`, temperature 1.0, top-p 0.95, top-k 20
+- Runtime: SGLang v0.5.17 based on `29481685462732237d80d86076d6563e1f658102`; FlashInfer target and draft attention; FP8 E4M3 KV cache; 32,768-token chunked and maximum prefill
+- DSpark serving: gamma 7, target verify width 8, one speculative step, block size 7, Mamba full-memory ratio 11.93, `extra_buffer` radix-cache strategy, float32 Mamba state
+- Autoregressive serving used `mem-fraction-static=0.85`. DSpark used 0.80 with expandable CUDA allocation segments while retaining every prefill and verification CUDA graph.
+
+The request protocol follows [`z-lab/dflash/dflash/benchmark.py`](https://github.com/z-lab/dflash/blob/07ebd93db9f472af339b644bb70221ad8428328a/dflash/benchmark.py). Model initialization, CUDA graph capture, and warmup are excluded from timed wall duration. Sampling is intentionally not request-seeded, matching the source benchmark, so completion-token totals can differ between v1 and v2.
+
+## Serving with SGLang
 
 ```bash
+PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
+SGLANG_RAGGED_VERIFY_MODE=static \
 sglang serve \
   --trust-remote-code \
   --model-path Qwen/Qwen3.8-27B-FP8 \
-  --tp-size 1 \
+  --kv-cache-dtype fp8_e4m3 \
+  --mem-fraction-static 0.80 \
+  --attention-backend flashinfer \
+  --chunked-prefill-size 32768 \
+  --max-prefill-tokens 32768 \
+  --reasoning-parser qwen3 \
+  --tool-call-parser qwen3_coder \
+  --mamba-full-memory-ratio 11.93 \
+  --mamba-radix-cache-strategy extra_buffer \
+  --mamba-ssm-dtype float32 \
+  --max-running-requests 48 \
   --speculative-algorithm DSPARK \
   --speculative-draft-model-path RadixArk/Qwen3.8-27B-DSpark \
-  --speculative-dspark-block-size 7 \
   --speculative-draft-model-quantization unquant \
-  --mamba-scheduler-strategy extra_buffer \
-  --attention-backend fa3
+  --speculative-draft-attention-backend flashinfer \
+  --speculative-dspark-block-size 7 \
+  --speculative-num-steps 1 \
+  --speculative-eagle-topk 1 \
+  --host 127.0.0.1 \
+  --port 30000
 ```
+
+## Artifact identity
+
+| File | Bytes | SHA-256 |
+|---|---:|---|
+| `model.safetensors` | 3,714,723,322 | `2aff025f45823b40ebe726b9dfa40302f3512bd9a11c3a7347de32a567acd9a7` |
+| `config.json` | 2,448 | `dd65fb1b01c2adea69512ff2990a79d58eb7fe2c7ea97375aa66f657a29a5bfd` |
+

config.json

@@ -4,11 +4,13 @@
   ],
   "attention_bias": false,
   "attention_dropout": 0.0,
+  "attention_mode": "gqa",
   "auto_map": {
     "AutoModel": "dspark.DSparkDraftModel"
   },
   "block_size": 7,
-  "bos_token_id": 248044,
+  "bos_token_id": null,
+  "confidence_head_alpha": 1.0,
   "confidence_head_with_markov": true,
   "dflash_config": {
     "attention_mode": "gqa",
@@ -17,24 +19,42 @@
     "enable_confidence_head": true,
     "markov_head_type": "vanilla",
     "markov_rank": 256,
-    "mask_token_id": 248077,
+    "mask_token_id": 248070,
     "projector_type": "dspark",
     "target_layer_ids": [
-      4,
-      16,
-      28,
-      40,
-      52
+      5,
+      19,
+      33,
+      47,
+      61
+    ]
+  },
+  "draft_vocab_size": 248320,
+  "dspark_config": {
+    "attention_mode": "gqa",
+    "confidence_head_alpha": 1.0,
+    "confidence_head_with_markov": true,
+    "enable_confidence_head": true,
+    "markov_head_type": "vanilla",
+    "markov_rank": 256,
+    "mask_token_id": 248070,
+    "projector_type": "dspark",
+    "target_layer_ids": [
+      5,
+      19,
+      33,
+      47,
+      61
     ]
   },
   "dtype": "bfloat16",
   "enable_confidence_head": true,
-  "eos_token_id": 248046,
+  "eos_token_id": 248044,
   "head_dim": 128,
   "hidden_act": "silu",
   "hidden_size": 5120,
   "initializer_range": 0.02,
-  "intermediate_size": 10240,
+  "intermediate_size": 17408,
   "layer_types": [
     "full_attention",
     "full_attention",
@@ -44,14 +64,16 @@
   ],
   "markov_head_type": "vanilla",
   "markov_rank": 256,
+  "mask_token_id": 248070,
   "max_position_embeddings": 262144,
   "max_window_layers": 5,
   "model_type": "qwen3",
-  "num_attention_heads": 40,
+  "num_attention_heads": 32,
   "num_hidden_layers": 5,
   "num_key_value_heads": 8,
   "num_target_layers": 64,
   "pad_token_id": 248044,
+  "projector_type": "dspark",
   "rms_norm_eps": 1e-06,
   "rope_parameters": {
     "beta_fast": 32.0,
@@ -61,9 +83,26 @@
     "rope_theta": 10000000,
     "rope_type": "yarn"
   },
+  "rope_scaling": {
+    "beta_fast": 32.0,
+    "beta_slow": 1.0,
+    "factor": 32.0,
+    "original_max_position_embeddings": 8192,
+    "rope_type": "yarn"
+  },
+  "rope_theta": 10000000,
   "sliding_window": null,
+  "target_hidden_size": null,
+  "target_layer_ids": [
+    5,
+    19,
+    33,
+    47,
+    61
+  ],
   "tie_word_embeddings": false,
-  "transformers_version": "5.12.1",
+  "training_block_size": 16,
+  "transformers_version": "5.8.1",
   "use_cache": true,
   "use_sliding_window": false,
   "vocab_size": 248320

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9d26d5e637551c244d543c67c790bd0947f360e005c569e5851a185ffe692786
-size 2718576122
+oid sha256:2aff025f45823b40ebe726b9dfa40302f3512bd9a11c3a7347de32a567acd9a7
+size 3714723322