Qwen3-ASR-1.7B DSpark speculator

A DSpark draft model for speculative decoding with both Qwen3-ASR-1.7B-hf and Qwen3-ASR-1.7B.

Model configuration

Setting	Value
Verifier	`Qwen/Qwen3-ASR-1.7B-hf`
Draft transformer layers	5
Hidden size	2,048
Intermediate size	6,144
Draft vocabulary	32,000, with target-vocabulary mappings
Target auxiliary hidden-state layers	2, 14, 25
DSpark block size	8
Speculative tokens per proposal	8
Markov head	Vanilla, rank 256
Confidence head	Included
Weight precision	BF16

Use the verifier's tokenizer and audio processor.

Inference

Install the latest version of vLLM from source.

Load the verifier with the DSpark head:

from vllm import LLM

llm = LLM(
    model="Qwen/Qwen3-ASR-1.7B-hf",
    dtype="bfloat16",
    max_model_len=4096,
    max_num_seqs=8,
    max_num_batched_tokens=8192,
    gpu_memory_utilization=0.4,
    enforce_eager=True,
    enable_prefix_caching=False,
    mm_processor_cache_gb=0,
    limit_mm_per_prompt={"audio": 1},
    generation_config="vllm",
    seed=42,
    speculative_config={
        "model": "mattymchen/Qwen3-ASR-1.7B-speculator.dspark",
        "method": "dspark",
        "num_speculative_tokens": 8,
    },
)

Transcribe audio with example.py:

python example.py sample.wav --draft .

For longer recordings, increase the context, output, and batch token budgets:

python example.py recording.wav --draft . \
  --max-model-len 32768 --max-tokens 4096 --max-num-batched-tokens 32768

--max-model-len includes the audio prompt and generated text; larger budgets may require more GPU memory.

Use --draft mattymchen/Qwen3-ASR-1.7B-speculator.dspark to load the head from Hugging Face, or --verifier Qwen/Qwen3-ASR-1.7B to select the original verifier format.