Update README.md

README.md

@@ -21,9 +21,9 @@ Samples were drawn from a diverse mix of publicly available datasets spanning co
 
 ### How to Run
 
-I've had bad luck recently with NVFP4 and vLLM, so I've personally switched to sglang.
+If you experience NCCL hangs with P2P, make sure you have `iommu=pt` (and `amd_iommu=pt` on AMD platforms)  in your kernel command line.
 
-If someone has a working vLLM recipe, please share, but the below works fine for me on the latest sglang.
+#### SGLang
 
 Tested on 2x and 4x RTX Pro 6000 Blackwell.
 
@@ -55,4 +55,59 @@ python3 -m sglang.launch_server \
   --port 8000
 
 
+```
+
+#### vLLM
+
+(thanks to @zenmagnets)
+
+Set your Hugging Face cache and GPUs, then run (from project root with venv activated).
+```
+export CUDA_DEVICE_ORDER=PCI_BUS_ID
+export CUDA_VISIBLE_DEVICES=0,1
+export HF_HOME=/path/to/huggingface
+export HUGGINGFACE_HUB_CACHE=$HF_HOME/hub
+export VLLM_WORKER_MULTIPROC_METHOD=spawn
+export SAFETENSORS_FAST_GPU=1
+export VLLM_NVFP4_GEMM_BACKEND=cutlass
+export VLLM_USE_FLASHINFER_MOE_FP4=0
+export VLLM_DISABLE_PYNCCL=1
+export NCCL_IB_DISABLE=1
+export OMP_NUM_THREADS=8
+export VLLM_ALLOW_LONG_MAX_MODEL_LEN=1
+
+python -m vllm.entrypoints.openai.api_server \
+  --model lukealonso/MiniMax-M2.5-NVFP4 \
+  --download-dir $HUGGINGFACE_HUB_CACHE \
+  --host 0.0.0.0 \
+  --port 1235 \
+  --served-model-name MiniMax-M2.5-NVFP4 \
+  --trust-remote-code \
+  --tensor-parallel-size 2 \
+  --attention-backend FLASH_ATTN \
+  --gpu-memory-utilization 0.95 \
+  --max-model-len 190000 \
+  --max-num-batched-tokens 16384 \
+  --max-num-seqs 64 \
+  --disable-custom-all-reduce \
+  --enable-auto-tool-choice \
+  --tool-call-parser minimax_m2 \
+  --reasoning-parser minimax_m2_append_think
+```
+
+Dependencies
+
+Install in a Python 3.12 venv; use CUDA 12.x on the host.
+
+```
+Package	Version	Note
+vllm	0.15.1	OpenAI server + NVFP4 MoE
+torch	2.9.1+cu128	CUDA 12.8 build
+transformers	4.57.6	
+safetensors	0.7.0	
+nvidia-modelopt	0.41.0	NVFP4 / ModelOpt format
+flashinfer-python	0.6.1	Optional (we use FLASH_ATTN)
+nvidia-nccl-cu12	2.27.5	Multi-GPU
+nvidia-cutlass-dsl*	4.4.0.dev1	NVFP4 GEMM (script uses cutlass backend)
+System: CUDA 12.8, cuDNN 9.10.2 (or matching torch cuDNN). Driver must support your GPUs (e.g. Blackwell).
 ```