Trim SGLang launch command to the verified minimal recipe
Browse filesTrim the SGLang launch command to the verified minimal recipe.
All removed flags were verified as unnecessary on the current `lmsysorg/sglang:dev-Ling-3.0-flash-VL` image: serving smoke plus full MMMU-Pro / GSM8K / bench_serving were measured without them. The MXFP4 MoE backend and the ling3 parsers are auto-selected; `--mem-fraction-static` uses the engine default with no measured perf/accuracy impact.
See the cookbook for the per-hardware verified matrix: https://docs.sglang.io/cookbook/autoregressive/InclusionAI/Ling-3.0-flash-VL
README.md
CHANGED
|
@@ -70,7 +70,6 @@ docker run --rm --gpus all --ipc=host --shm-size 32g \
|
|
| 70 |
--tp 2 \
|
| 71 |
--context-length 262144 \
|
| 72 |
--json-model-override-args '{"rope_scaling":{"rope_type":"yarn","factor":2.0,"rope_theta":6000000,"partial_rotary_factor":0.5,"original_max_position_embeddings":131072}}' \
|
| 73 |
-
--mem-fraction-static 0.85 \
|
| 74 |
--trust-remote-code \
|
| 75 |
--reasoning-parser auto \
|
| 76 |
--tool-call-parser auto \
|
|
@@ -78,7 +77,7 @@ docker run --rm --gpus all --ipc=host --shm-size 32g \
|
|
| 78 |
--port 30000
|
| 79 |
```
|
| 80 |
|
| 81 |
-
On 80GB cards (H100 / H800), scale out to `--tp 8`. The reasoning and tool-call parsers resolve automatically to `ling3` from the chat template; you can also set them explicitly with `--reasoning-parser ling3 --tool-call-parser ling3`.
|
| 82 |
|
| 83 |
**Client**
|
| 84 |
|
|
|
|
| 70 |
--tp 2 \
|
| 71 |
--context-length 262144 \
|
| 72 |
--json-model-override-args '{"rope_scaling":{"rope_type":"yarn","factor":2.0,"rope_theta":6000000,"partial_rotary_factor":0.5,"original_max_position_embeddings":131072}}' \
|
|
|
|
| 73 |
--trust-remote-code \
|
| 74 |
--reasoning-parser auto \
|
| 75 |
--tool-call-parser auto \
|
|
|
|
| 77 |
--port 30000
|
| 78 |
```
|
| 79 |
|
| 80 |
+
On 80GB cards (H100 / H800), scale out to `--tp 8 --ep 8` (plain `--tp 8` fails the 128×128 block-quantization divisibility check; `--ep 8` keeps experts whole per rank). The reasoning and tool-call parsers resolve automatically to `ling3` from the chat template; you can also set them explicitly with `--reasoning-parser ling3 --tool-call-parser ling3`.
|
| 81 |
|
| 82 |
**Client**
|
| 83 |
|