add exemples for turboquant

README.md

@@ -91,12 +91,86 @@ This is recommended for latency-focused serving scenarios (not total throughput/
 
 ---
 
-### RTX 5090 Optimized Deployment
+## RTX 5090 Optimized Deployment
 
 Because of the increase model size from its previous versions (Qwen3-30B-A3B-Thinking-2507 and Qwen3-30B-A3B-Instruct-2507 were 30B not 35B) but also because of its mamba hybrid architecture and its native vision support (layers excluded from the quantization) the final model size is bigger (~ +5GiB) which make its inference by a RTX 5090 with only 32 GiB of RAM challenging.
 
 If you really want/need to, it is still possible by tuning a few parameters (and accepting a lower max model len/kv cache size and requests concurrency).
 
+### With turboquant (recommended)
+
+vLLM v0.21.0 landed support of TurbotQuant for hybrid models like Qwen3.5/3.6, drastically enhancing the available KV cache available on a limited VRAM.
+
+With a headless environment (no graphical env) you can push the memory usage to `0.95`. Without it is best to set it to `0.875` if you have graphical apps (like Zed) that use the GPU as well but you can try to push it to `0.9` beware that it might not leave the host enought.
+
+With `0.875`:
+```
+(EngineCore pid=139) INFO 05-15 15:37:22 [gpu_worker.py:462] Available KV cache memory: 1.59 GiB
+...
+(EngineCore pid=139) INFO 05-15 15:37:22 [kv_cache_utils.py:1871] Auto-fit max_model_len: reduced from 262144 to 210368 to fit in available GPU memory (1.59 GiB available for KV cache)
+(EngineCore pid=139) INFO 05-15 15:37:22 [kv_cache_utils.py:1710] GPU KV cache size: 210,368 tokens
+(EngineCore pid=139) INFO 05-15 15:37:22 [kv_cache_utils.py:1711] Maximum concurrency for 210,368 tokens per request: 1.00x
+```
+
+With `0.9`:
+```
+(EngineCore pid=139) INFO 05-15 15:21:12 [gpu_worker.py:462] Available KV cache memory: 2.69 GiB
+...
+(EngineCore pid=139) INFO 05-15 15:21:12 [kv_cache_utils.py:1863] Auto-fit max_model_len: full model context length 262144 fits in available GPU memory
+(EngineCore pid=139) INFO 05-15 15:21:12 [kv_cache_utils.py:1710] GPU KV cache size: 358,441 tokens
+(EngineCore pid=139) INFO 05-15 15:21:12 [kv_cache_utils.py:1711] Maximum concurrency for 262,144 tokens per request: 1.37x
+```
+
+#### Linux
+
+bash command:
+```bash
+docker run --name 'Qwen3.6-35B-A3B-NVFP4-KVTBQ' \
+  --runtime=nvidia --gpus 'all' --ipc=host \
+  -e 'HF_TOKEN' \
+  -v '/srv/cache:/root/.cache' \
+  -p '127.0.0.1:8000:8000' \
+  'vllm/vllm-openai:v0.21.0' \
+  'ig1/Qwen3.6-35B-A3B-NVFP4' \
+  --served-model-name 'Qwen3.6-35B-A3B' \
+  --reasoning-parser 'qwen3' \
+  --enable-auto-tool-choice \
+  --tool-call-parser 'qwen3_coder' \
+  --max-model-len 'auto' \
+  --limit-mm-per-prompt.video 0 \
+  --max-cudagraph-capture-size 32 \
+  --max-num-seqs 32 \
+  --max-num-batched-tokens 2048 \
+  --kv-cache-dtype 'turboquant_k8v4' \
+  --gpu-memory-utilization '0.875'
+```
+
+#### Windows with Docker and WSL
+
+powershell command:
+```powershell
+docker run --name 'Qwen3.6-35B-A3B-NVFP4-KVTBQ' `
+  --runtime=nvidia --gpus 'all' --ipc=host `
+  -e 'HF_TOKEN' `
+  -v 'E:\cache:/root/.cache' `
+  -p '127.0.0.1:8000:8000' `
+  'vllm/vllm-openai:v0.21.0' `
+  'ig1/Qwen3.6-35B-A3B-NVFP4' `
+  --served-model-name 'Qwen3.6-35B-A3B' `
+  --reasoning-parser 'qwen3' `
+  --enable-auto-tool-choice `
+  --tool-call-parser 'qwen3_coder' `
+  --max-model-len 'auto' `
+  --limit-mm-per-prompt.video 0 `
+  --max-cudagraph-capture-size 32 `
+  --max-num-seqs 32 `
+  --max-num-batched-tokens 2048 `
+  --kv-cache-dtype 'turboquant_k8v4' `
+  --gpu-memory-utilization '0.875'
+```
+
+### Without turboquant (old, not recommended)
+
 ```bash
 docker run --rm --name 'Qwen3.6-35B-A3B-NVFP4' \
   --runtime=nvidia --gpus 'all' --ipc=host \