baranowskiadam commited on
Commit
f685cd1
·
verified ·
1 Parent(s): 3879c9f

Update DFlash recommendation

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -111,7 +111,7 @@ The full vLLM recipe is on the main [Laguna S 2.1 model card](https://huggingfac
111
  > INT4 support landed in vLLM with [vllm-project/vllm#47154](https://github.com/vllm-project/vllm/pull/47154). This checkpoint mixes INT4 and group-quantized INT8 experts, which earlier builds rejected in the Marlin WNA16 MoE path. Use a vLLM build that includes this fix.
112
 
113
  > [!NOTE]
114
- > **Optional: speculative decoding with DFlash.** Pair with the quantization-matched draft model [poolside/Laguna-S-2.1-DFlash-INT4](https://huggingface.co/poolside/Laguna-S-2.1-DFlash-INT4) by adding `--speculative-config '{"model":"poolside/Laguna-S-2.1-DFlash-INT4","num_speculative_tokens":15,"method":"dflash"}'` to the serve command.
115
 
116
  #### SGLang
117
 
 
111
  > INT4 support landed in vLLM with [vllm-project/vllm#47154](https://github.com/vllm-project/vllm/pull/47154). This checkpoint mixes INT4 and group-quantized INT8 experts, which earlier builds rejected in the Marlin WNA16 MoE path. Use a vLLM build that includes this fix.
112
 
113
  > [!NOTE]
114
+ > **Optional: speculative decoding with DFlash.** Pair with the quantization-matched draft model [poolside/Laguna-S-2.1-DFlash-INT4](https://huggingface.co/poolside/Laguna-S-2.1-DFlash-INT4) by adding `--speculative-config '{"model":"poolside/Laguna-S-2.1-DFlash-INT4","num_speculative_tokens":7,"method":"dflash"}'` to the serve command.
115
 
116
  #### SGLang
117