michael-qiu commited on
Commit
e98d882
·
verified ·
1 Parent(s): f48e5cc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -81,7 +81,7 @@ docker run --rm --gpus all --ipc=host --shm-size 32g \
81
  --port 30000
82
  ```
83
 
84
- On 80GB cards (H100 / H800), scale out to `--tp 8`. The reasoning and tool-call parsers resolve automatically to `ling3` from the chat template; you can also set them explicitly with `--reasoning-parser ling3 --tool-call-parser ling3`.
85
 
86
  **Client**
87
 
@@ -127,7 +127,7 @@ vllm serve "$MODEL_PATH" \
127
  --port "$PORT" \
128
  --trust-remote-code \
129
  --served-model-name auto \
130
- --tensor-parallel-size 4 \
131
  --gpu-memory-utilization 0.85 \
132
  --enable-prefix-caching \
133
  --mamba-cache-mode align \
 
81
  --port 30000
82
  ```
83
 
84
+ On 80GB cards (H100 / H800), scale out to `--tp 2`. The reasoning and tool-call parsers resolve automatically to `ling3` from the chat template; you can also set them explicitly with `--reasoning-parser ling3 --tool-call-parser ling3`.
85
 
86
  **Client**
87
 
 
127
  --port "$PORT" \
128
  --trust-remote-code \
129
  --served-model-name auto \
130
+ --tensor-parallel-size 2 \
131
  --gpu-memory-utilization 0.85 \
132
  --enable-prefix-caching \
133
  --mamba-cache-mode align \