Specify exact hardware alongside benchmark results
Browse files
README.md
CHANGED
|
@@ -129,7 +129,18 @@ curl http://127.0.0.1:8000/v1/chat/completions \
|
|
| 129 |
|
| 130 |
## Measured performance and validation
|
| 131 |
|
| 132 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 133 |
|
| 134 |
| Prompt tokens | Generated tokens | Decode tokens/sec | Time to first token (s) | End-to-end tokens/sec |
|
| 135 |
|---:|---:|---:|---:|---:|
|
|
|
|
| 129 |
|
| 130 |
## Measured performance and validation
|
| 131 |
|
| 132 |
+
Both benchmark runs below were measured on September 28, 2026 with the final release build on this workstation:
|
| 133 |
+
|
| 134 |
+
| Benchmark hardware / configuration | Tested setup |
|
| 135 |
+
|---|---|
|
| 136 |
+
| GPUs | **4 × NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition**, **96 GB VRAM each** (384 GB total nominal VRAM) |
|
| 137 |
+
| CPU | AMD Ryzen Threadripper PRO 7955WX |
|
| 138 |
+
| Host memory | 498 GiB usable RAM |
|
| 139 |
+
| OS / GPU driver | Linux / NVIDIA 595.91.07 |
|
| 140 |
+
| Serving runtime | Native vLLM 0.28.0 with the included Naive plugin; tensor parallelism **TP4** across all four GPUs |
|
| 141 |
+
| Workload | **One active request**, 256 generated tokens, temperature 0; maximum context 8,192 tokens |
|
| 142 |
+
|
| 143 |
+
The tokens/sec figures describe generation by the full model across this four-GPU workstation.
|
| 144 |
|
| 145 |
| Prompt tokens | Generated tokens | Decode tokens/sec | Time to first token (s) | End-to-end tokens/sec |
|
| 146 |
|---:|---:|---:|---:|---:|
|