VibeVoice-ASR-Streaming-1.5B-AWQ-W4A16-ASYM-DFlash2

VibeVoice-ASR-Streaming-1.5B, AWQ W4A16 (asymmetric, groups of 128), with its DFlash 2 drafter bundled in drafter/: one download, and vibevoice.c decodes with speculative decoding -- the drafter proposes 8 tokens in one pass, the model checks them in one pass and keeps the ones it agrees with. The check is exact: every checked row is computed with the arithmetic of the model's own decode step, so the transcript is byte-for-byte the one without the drafter.

Use

Needs vibevoice.c with DFlash 2 support: branch dflash2 (PR #48), in the next release. A model directory's drafter/ is used without asking:

vv_cli --model ./VibeVoice-ASR-Streaming-1.5B-AWQ-W4A16-ASYM-DFlash2 --audio talk.wav                  # with the drafter
vv_cli --model ./VibeVoice-ASR-Streaming-1.5B-AWQ-W4A16-ASYM-DFlash2 --audio talk.wav --draft none     # plain decoding
vv_cli serve --model ./VibeVoice-ASR-Streaming-1.5B-AWQ-W4A16-ASYM-DFlash2 --slots 4                  # streaming sessions (WebSocket, SSE) too

Results

vibevoice.c 7d7d43e (branch dflash2), RTX 3090, greedy decoding, decode tokens per second:

	plain	drafted	speedup	tokens per block	same transcript
20 held-out clips, 8 rows	409 tok/s	866 tok/s	2.12x	3.53	20/20
2-minute file, 8 rows	416 tok/s	1047 tok/s	2.52x	4.03	yes
32-minute file, 8 rows	341 tok/s	722 tok/s	2.12x	4.03	yes

Streaming sessions (22 + 4 frames a chunk). Plain = the same model with --draft none; --draft-check exact.

Inside

The model: the files of Ar4ikov/VibeVoice-ASR-Streaming-1.5B-AWQ-W4A16-ASYM at revision bed7b5d5, unchanged (2.55 GB) -- its card has the quantization, the calibration and the WER.
drafter/: Ar4ikov/VibeVoice-ASR-Streaming-1.5B-DFlash2-Drafter-AWQ-W4A16-ASYM at revision 277f16a7 (0.23 GB): 5 Qwen3-style layers reading the model's layers 1/7/13/19/25, a candidate selector, a 32768-id draft vocabulary; its projections stored as INT4 (compressed-tensors pack-quantized). Its card has the architecture and the training.