Remove duplicate vllm_deploy_guide.md from root (exists in docs/)

vllm_deploy_guide.md

@@ -1,193 +0,0 @@
-# MiniMax-M2.7-MXFP416 vLLM Deployment Guide (RDNA 4 / RX 9700)
-
-## Hardware Requirements
-
-- **GPU:** AMD RX 9700 (RDNA 4 / gfx12xx) — minimum 4x recommended
-- **Memory:** 128GB+ system RAM for 8-GPU setup
-- **OS:** Linux with ROCm 6.x
-- **Docker:** RDNA4-compatible vLLM image
-
-## Docker Image
-
-The only validated runtime for this model is [`tcclaviger/vllm22:latest`](https://hub.docker.com/r/tcclaviger/vllm22):
-
-```bash
-docker pull tcclaviger/vllm22:latest
-```
-
-This image includes:
-- Custom Triton attention kernels tuned for RDNA4 (significantly faster than ROCm attention at long context)
-- Fixed FP8 KV-cache quantization path (~2× throughput improvement)
-- Pre-tuned GEMM configs for RX 9700
-- MXFP4-16 kernels compiled for gfx12xx
-
-## System Setup
-
-### GPU Devices
-
-Make sure all GPUs are visible:
-
-```bash
-rocm-smi --showid
-# Should show: 0, 1, 2, 3, 4, 5, 6, 7
-```
-
-### Power Limit (Recommended)
-
-RDNA4 performs best with tuned power limits. Default is ~300W but 210W provides better sustained throughput on multi-GPU setups:
-
-```bash
-# Set per-GPU power limit
-for i in 0 1 2 3 4 5 6 7; do
-    rocm-smi --setpowerlimit $i 210
-done
-```
-
-> **Note:** At full power (300W) sustained speeds are lower due to thermal throttling. At 210W, sustained generation throughput is consistently higher under multi-user workloads.
-
-## Launching the Server
-
-### Single Container (8 GPUs)
-
-```bash
-docker run --name minimax-mxfp416 \
-  --rm --tty --ipc=host --shm-size=128g \
-  --device /dev/kfd:/dev/kfd \
-  --device /dev/dri/renderD128:/dev/dri/renderD128 \
-  --device /dev/dri/renderD129:/dev/dri/renderD129 \
-  --device /dev/dri/renderD130:/dev/dri/renderD130 \
-  --device /dev/dri/renderD132:/dev/dri/renderD132 \
-  --device /dev/dri/renderD137:/dev/dri/renderD137 \
-  --device /dev/dri/renderD138:/dev/dri/renderD138 \
-  --device /dev/dri/renderD139:/dev/dri/renderD139 \
-  --device /dev/dri/renderD140:/dev/dri/renderD140 \
-  -e HIP_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
-  -e ROCR_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
-  -e TRUST_REMOTE_CODE=1 \
-  -e PYTORCH_TUNABLEOP_ENABLED=1 \
-  -e PYTORCH_TUNABLEOP_TUNING=0 \
-  -e PYTORCH_TUNABLEOP_RECORD_UNTUNED=0 \
-  -e PYTORCH_ALLOC_CONF=expandable_segments:True \
-  -e PYTORCH_HIP_ALLOC_CONF=expandable_segments:True \
-  -e GPU_MAX_HW_QUEUES=1 \
-  -v /path/to/models:/app/models:ro \
-  -v /path/to/patches:/patches:ro \
-  -p 8000:8000 \
-  tcclaviger/vllm22:latest \
-  bash -c "cp /patches/vllm22_minimax_m2.py /app/vllm/vllm/model_executor/models/minimax_m2.py && \
-    pip install -q sentencepiece && \
-    exec vllm serve \
-      /app/models/MiniMax-M2.7-MXFP416 \
-      --served-model-name minimax-m2.7-mxfp416 \
-      --host 0.0.0.0 --port 8000 \
-      --trust-remote-code \
-      --tensor-parallel-size 8 \
-      --enable-expert-parallel \
-      --disable-cascade-attn \
-      --reasoning-parser minimax_m2 \
-      --enable-auto-tool-choice \
-      --tool-call-parser minimax_m2 \
-      --enable-prefix-caching \
-      --gpu-memory-utilization 0.93 \
-      --max-model-len 180000 \
-      --max-num-seqs 48 \
-      --max-num-batched-tokens 2048 \
-      --kv-cache-dtype fp8_e4m3 \
-      --attention-backend TRITON_ATTN \
-      --override-generation-config '{\"max_tokens\": 16384}'"
-```
-
-### Key vLLM Flags Explained
-
-| Flag | Value | Purpose |
-|------|-------|---------|
-| `--tensor-parallel-size` | 8 | Split model across 8 GPUs |
-| `--enable-expert-parallel` | | Enable expert-parallel distribution |
-| `--disable-cascade-attn` | | Disable cascade attention for MoE layers |
-| `--attention-backend` | TRITON_ATTN | Use Triton kernels (10× faster than ROCm on RDNA4) |
-| `--kv-cache-dtype` | fp8_e4m3 | FP8 KV cache (~50% memory savings) |
-| `--enable-prefix-caching` | | Cache common prefixes (93%+ hit rate observed) |
-| `--max-model-len` | 180000 | 180k context |
-| `--max-num-seqs` | 48 | Max concurrent sequences |
-| `--max-num-batched-tokens` | 2048 | Max tokens per batch |
-| `--gpu-memory-utilization` | 0.93 | Use 93% of GPU memory |
-
-> **Important:** The `--disable-cascade-attn` flag is required for MoE models. Without it, the model will produce incorrect outputs.
-
-### Running with Patches
-
-If you have custom model patches:
-
-```bash
--v /path/to/patches:/patches:ro \
-```
-
-The Docker entry point copies `vllm22_minimax_m2.py` to the vLLM model directory before launching. This adds MXFP4-16 support for MiniMax-M2.7.
-
-## Performance Notes
-
-### Observed Performance (4× RX 9700, 210W power limit)
-
-- **Generation throughput:** 50–80 tokens/s
-- **Prefill throughput:** 2000+ tokens/s (with prefix caching)
-- **Prefix cache hit rate:** ~93%
-- **KV cache usage:** 25–33% typical at 180k context
-- **Max concurrent users:** 4–5 at full 180k context
-
-### KV Cache Capacity
-
-With 8× RX 9700 and FP8 KV cache:
-- **KV cache memory:** 11.35 GiB
-- **KV cache tokens:** ~768K tokens
-- **Max context per request:** 180,000 tokens
-- **Max concurrent at 180k:** ~4 requests
-
-### Model Loading
-
-- **Weight loading time:** ~42 seconds
-- **Memory per GPU (TP8):** ~17.5 GiB
-- **Torch compile warmup:** ~37 seconds
-
-## Testing the Deployment
-
-```python
-from openai import OpenAI
-
-client = OpenAI(
-    base_url="http://localhost:8000/v1",
-    api_key="EMPTY",
-)
-
-completion = client.chat.completions.create(
-    model="minimax-m2.7-mxfp416",
-    messages=[
-        {"role": "system", "content": "You are a helpful assistant."},
-        {"role": "user", "content": "Explain what MXFP4 quantization is in one sentence."}
-    ],
-    temperature=1.0,
-    max_tokens=256,
-)
-
-print(completion.choices[0].message.content)
-```
-
-## Troubleshooting
-
-### "expandable_segments not supported"
-
-This warning is benign on ROCm. The model runs correctly despite the warning.
-
-### Low throughput at long context
-
-Ensure `TRITON_ATTN` backend is active. Default ROCm attention is 10× slower on RDNA4 at long context.
-
-### Thermal throttling
-
-If sustained throughput degrades over time, reduce power limit to 210W per GPU:
-```bash
-rocm-smi --setpowerlimit 0 210
-```
-
-### Model fails to load
-
-Ensure `--trust-remote-code` is set and the model path is correct. The custom model file (`vllm22_minimax_m2.py`) must be copied before vLLM loads the model.