---
license: other
library_name: sglang
base_model: LiquidAI/LFM2.5-VL-3B
base_model_relation: finetune
tags:
  - speculative-decoding
  - dspark
  - draft-model
  - lfm2
  - lfm2-vl
---

# LFM2.5-VL-3B DSpark draft (4 layers)

DSpark speculative-decoding draft model for
[LiquidAI/LFM2.5-VL-3B](https://huggingface.co/LiquidAI/LFM2.5-VL-3B), for use with
SGLang. 279.5M parameters, 4 layers, block size 9, Markov head rank 256, confidence
head enabled.

This is a drafter, not a standalone model: it only runs alongside the target above.

## Usage

```bash
python -m sglang.launch_server \
  --model-path LiquidAI/LFM2.5-VL-3B \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path tugot17/mango-9 \
  --speculative-draft-attention-backend flashinfer \
  --speculative-dspark-block-size 9 \
  --disable-radix-cache
```

Serving LFM2-VL with DSpark needs two delegating members on
`Lfm2VlForConditionalGeneration` (`lm_head` and `set_dflash_layers_to_capture`), since
the speculative stack introspects the top-level model while LFM2-VL keeps the decoder
under `language_model`. Until that lands upstream, use the branch in the PR linked
below.

## Measured performance

1xH100, bs=1, greedy, `--disable-radix-cache`, full
[MMSpec](https://github.com/killthefullmoon/MMSpec) `test` split
(600 samples / 723 turns / 100 per subset), base/spec/base bracket:

| subset | speedup | accept length |
|---|---|---|
| MMMU-Pro | 2.56x | 4.20 |
| COCO captions | 2.46x | 4.59 |
| CharXiv | 1.96x | 4.71 |
| multi-turn | 1.88x | 3.78 |
| TextVQA | 1.73x | 4.55 |
| GQA | 1.66x | 4.04 |
| **overall** | **2.08x** | **4.22** |

537 vs 258 tok/s overall. Accept length is `completion_tokens / spec_verify_ct`, i.e.
tokens emitted per target forward pass, bonus token included; MMSpec's MAT convention
is this minus one. The number above is a request-mean; the token-weighted cycle tau is
3.95.

At concurrency on the same hardware the speedup decays as the target verify becomes
compute-bound, while acceptance stays flat within 0.4%: 1.55x at batch 8, 1.32x at
batch 32, 1.18x at batch 64.

## Notes

- Block size 9 is the trained width and the best setting on GPU; block 8 is 4.7%
  slower end-to-end. On Metal the tradeoff reverses, because 8 rows exactly fill the
  verifier threadgroup.
- Speculative decoding preserves the target's output *distribution*, not a
  token-for-token match against a non-speculative run. Under verify the target
  processes a block of candidates in one pass, so reduction orders differ and
  near-ties can flip. Measured exact-text agreement against the non-speculative arm is
  388/723 turns on this split; draft-vs-draft determinism is the correctness gate and
  it holds.

## Related

- Target model: [LiquidAI/LFM2.5-VL-3B](https://huggingface.co/LiquidAI/LFM2.5-VL-3B)
- Text DSpark drafts:
  [1.2B](https://huggingface.co/tugot17/LFM2.5-1.2B-Instruct-DSpark-5L),
  [8B-A1B](https://huggingface.co/tugot17/LFM2.5-8B-A1B-DSpark-3L)