JustinTong commited on
Commit
eaea505
·
verified ·
1 Parent(s): 51de7e5

Trim SGLang launch command to the verified minimal recipe

Browse files

Trim the SGLang launch command to the verified minimal recipe.

All removed flags were verified as unnecessary on the current `lmsysorg/sglang:dev-Ling-3.0-flash-VL` image: serving smoke plus full MMMU-Pro / GSM8K / bench_serving were measured without them. The MXFP4 MoE backend and the ling3 parsers are auto-selected; `--mem-fraction-static` uses the engine default with no measured perf/accuracy impact.

See the cookbook for the per-hardware verified matrix: https://docs.sglang.io/cookbook/autoregressive/InclusionAI/Ling-3.0-flash-VL

Files changed (1) hide show
  1. README.md +1 -2
README.md CHANGED
@@ -70,7 +70,6 @@ docker run --rm --gpus all --ipc=host --shm-size 32g \
70
  --tp 2 \
71
  --context-length 262144 \
72
  --json-model-override-args '{"rope_scaling":{"rope_type":"yarn","factor":2.0,"rope_theta":6000000,"partial_rotary_factor":0.5,"original_max_position_embeddings":131072}}' \
73
- --mem-fraction-static 0.85 \
74
  --trust-remote-code \
75
  --reasoning-parser auto \
76
  --tool-call-parser auto \
@@ -78,7 +77,7 @@ docker run --rm --gpus all --ipc=host --shm-size 32g \
78
  --port 30000
79
  ```
80
 
81
- On 80GB cards (H100 / H800), scale out to `--tp 8`. The reasoning and tool-call parsers resolve automatically to `ling3` from the chat template; you can also set them explicitly with `--reasoning-parser ling3 --tool-call-parser ling3`.
82
 
83
  **Client**
84
 
 
70
  --tp 2 \
71
  --context-length 262144 \
72
  --json-model-override-args '{"rope_scaling":{"rope_type":"yarn","factor":2.0,"rope_theta":6000000,"partial_rotary_factor":0.5,"original_max_position_embeddings":131072}}' \
 
73
  --trust-remote-code \
74
  --reasoning-parser auto \
75
  --tool-call-parser auto \
 
77
  --port 30000
78
  ```
79
 
80
+ On 80GB cards (H100 / H800), scale out to `--tp 8 --ep 8` (plain `--tp 8` fails the 128×128 block-quantization divisibility check; `--ep 8` keeps experts whole per rank). The reasoning and tool-call parsers resolve automatically to `ling3` from the chat template; you can also set them explicitly with `--reasoning-parser ling3 --tool-call-parser ling3`.
81
 
82
  **Client**
83