Remove vLLM NVFP4 tp>1 caveat
#3
by Nvidia-NetanelHaber - opened
README.md
CHANGED
|
@@ -250,8 +250,6 @@ Efficient Video Sampling: video-pruning-rate=0.5 drops 50% of redundant video to
|
|
| 250 |
#### Platform-Specific Notes
|
| 251 |
|
| 252 |
> **RTX Pro:** Due to a current bug with FlashInfer + RTX Pro, append: `--moe-backend triton`
|
| 253 |
-
|
| 254 |
-
> **NVFP4 + TP>1:** Due to a current bug with the TRTLLM_GEN MoE backend kernels on vLLM, when running with TP>1 on NVFP4, append: `--moe-backend flashinfer_cutlass`
|
| 255 |
|
| 256 |
##### vLLM on DGX Spark (aarch64 / ARM64)
|
| 257 |
|
|
|
|
| 250 |
#### Platform-Specific Notes
|
| 251 |
|
| 252 |
> **RTX Pro:** Due to a current bug with FlashInfer + RTX Pro, append: `--moe-backend triton`
|
|
|
|
|
|
|
| 253 |
|
| 254 |
##### vLLM on DGX Spark (aarch64 / ARM64)
|
| 255 |
|