Document verified vLLM 0.25.1 serving

Add the concise CUDA 13/GB10 serving configuration and the user-provided successful-run screenshot. Model weights and existing audit artifacts are unchanged.

README.md

@@ -27,6 +27,62 @@ tags:
 
 # Solar-Open2-148B-A15B-REAP-184E-NVFP4
 
+## Verified vLLM Serving / vLLM 서빙 확인
+
+This exact checkpoint was successfully served from a local model directory on
+an NVIDIA GB10. The working runtime is a custom source build based on
+**vLLM v0.25.1**, with Solar Open 2 support ported from
+[UpstageAI/vllm `v0.22.0-solar-open2`](https://github.com/UpstageAI/vllm/tree/v0.22.0-solar-open2).
+Stock vLLM v0.25.1 does not include this port.
+
+이 체크포인트를 로컬 모델 디렉터리에서 NVIDIA GB10으로 실제 구동했습니다.
+검증 환경은 Upstage Solar Open 2 지원을 옮긴
+**vLLM v0.25.1 커스텀 소스 빌드**이며, 순정 v0.25.1에는 이 포트가 포함되어
+있지 않습니다.
+
+| Component | Verified version / setting |
+|---|---|
+| vLLM | 0.25.1 + custom Solar Open 2 port |
+| PyTorch / CUDA | 2.11.0+cu130 / CUDA 13.0 |
+| Transformers | 5.14.1+solaropen2 |
+| FlashInfer | 0.6.13, JIT cache 0.6.13+cu130 |
+| Device | NVIDIA GB10, TP 1, max sequences 1 |
+
+Install the CUDA 13.0 JIT cache explicitly:
+
+```bash
+uv pip install "flashinfer-jit-cache==0.6.13+cu130" \
+  --index https://flashinfer.ai/whl/cu130
+```
+
+Core settings from the confirmed 65K-context launch:
+
+```bash
+MODEL_DIR=/path/to/Solar-Open2-148B-A15B-REAP-184E-NVFP4
+
+CUDA_HOME=/usr/local/cuda-13.0 vllm serve "$MODEL_DIR" \
+  --optimization-level 3 \
+  --max-model-len 65536 \
+  --max-num-seqs 1 \
+  --max-num-batched-tokens 4096 \
+  --gpu-memory-utilization 0.94 \
+  --tensor-parallel-size 1 \
+  --moe-backend flashinfer_b12x \
+  --attention-backend flashinfer \
+  --kv-cache-dtype fp8 \
+  --quantization compressed-tensors \
+  --enable-chunked-prefill \
+  --enable-prefix-caching \
+  --reasoning-parser solar_open2 \
+  --tool-call-parser solar_open2 \
+  --logits-processors vllm.v1.sample.logits_processor.solar_open2:SolarOpen2TemplateLogitsProcessor \
+  --default-chat-template-kwargs '{"think_render_option":"preserved"}' \
+  --enable-auto-tool-choice \
+  --language-model-only
+```
+
+![Solar Open 2 148B NVFP4 serving on NVIDIA GB10](assets/solar-open2-148b-nvfp4-gb10.jpg)
+
 ## English
 
 ## Built with Solar
@@ -116,9 +172,9 @@ Use the Upstage Transformers Solar Open 2 branch <code>v5.14.1-solar-open2</code
     uv pip install "git+https://github.com/UpstageAI/transformers.git@v5.14.1-solar-open2"
     uv pip install "llmcompressor==0.12.0" "compressed-tensors==0.17.1"
 
-Serving additionally requires a runtime that natively consumes compressed-tensors NVFP4A16 MoE weights and supports Solar Open 2 hybrid KDA/GQA. Use the Upstage vLLM Solar fork/revision <code>v0.22.0-solar-open2</code> as the architecture reference, not as a claim of prevalidated GB10 binaries. Solar reasoning and tool calls use <code>--reasoning-parser solar_open2</code>, <code>--tool-call-parser solar_open2</code>, and <code>--enable-auto-tool-choice</code>.
-
-For a first GB10 bring-up, use tensor parallel size 1, one sequence, eager execution, and a short context such as 4K. Confirm in engine logs that the packed weights are consumed directly rather than expanded to full precision before increasing context or concurrency.
+For serving, use the verified stack and launch command at the top of this card.
+The Upstage v0.22 branch is the architecture source for the custom v0.25.1
+port; stock vLLM v0.25.1 is not drop-in compatible.
 
 ## Limitations
 
@@ -232,9 +288,9 @@ Architecture 지원에는 Upstage Transformers Solar Open 2 branch <code>v5.14.1
     uv pip install "git+https://github.com/UpstageAI/transformers.git@v5.14.1-solar-open2"
     uv pip install "llmcompressor==0.12.0" "compressed-tensors==0.17.1"
 
-서빙 runtime은 compressed-tensors NVFP4A16 MoE weight를 직접 처리하고 Solar Open 2 hybrid KDA/GQA도 지원해야 합니다. Upstage vLLM Solar fork/revision <code>v0.22.0-solar-open2</code>는 architecture reference이며 GB10용 binary가 검증됐다는 의미가 아닙니다. Solar reasoning 및 tool call에는 <code>--reasoning-parser solar_open2</code>, <code>--tool-call-parser solar_open2</code>, <code>--enable-auto-tool-choice</code>를 사용합니다.
-
-GB10 첫 구동은 tensor parallel size 1, sequence 1개, eager execution, 4K 정도의 짧은 context로 시작하십시오. Context나 concurrency를 늘리기 전에 engine log에서 packed weight가 full precision으로 펼쳐지지 않고 직접 사용되는지 확인해야 합니다.
+서빙에는 카드 상단의 검증된 stack과 실행 명령을 사용하십시오. Upstage v0.22
+브랜치는 v0.25.1 커스텀 포트의 architecture source이며, 순정 vLLM v0.25.1과
+drop-in compatible하지 않습니다.
 
 ## 제한사항