Update README.md
Browse files
README.md
CHANGED
|
@@ -81,7 +81,7 @@ docker run --rm --gpus all --ipc=host --shm-size 32g \
|
|
| 81 |
--port 30000
|
| 82 |
```
|
| 83 |
|
| 84 |
-
On 80GB cards (H100 / H800), scale out to `--tp
|
| 85 |
|
| 86 |
**Client**
|
| 87 |
|
|
@@ -127,7 +127,7 @@ vllm serve "$MODEL_PATH" \
|
|
| 127 |
--port "$PORT" \
|
| 128 |
--trust-remote-code \
|
| 129 |
--served-model-name auto \
|
| 130 |
-
--tensor-parallel-size
|
| 131 |
--gpu-memory-utilization 0.85 \
|
| 132 |
--enable-prefix-caching \
|
| 133 |
--mamba-cache-mode align \
|
|
|
|
| 81 |
--port 30000
|
| 82 |
```
|
| 83 |
|
| 84 |
+
On 80GB cards (H100 / H800), scale out to `--tp 2`. The reasoning and tool-call parsers resolve automatically to `ling3` from the chat template; you can also set them explicitly with `--reasoning-parser ling3 --tool-call-parser ling3`.
|
| 85 |
|
| 86 |
**Client**
|
| 87 |
|
|
|
|
| 127 |
--port "$PORT" \
|
| 128 |
--trust-remote-code \
|
| 129 |
--served-model-name auto \
|
| 130 |
+
--tensor-parallel-size 2 \
|
| 131 |
--gpu-memory-utilization 0.85 \
|
| 132 |
--enable-prefix-caching \
|
| 133 |
--mamba-cache-mode align \
|