WaveCut commited on
Commit
c14a6c6
·
verified ·
1 Parent(s): 51b3bbe

Document vLLM 0.25.1 production validation

Browse files
Files changed (1) hide show
  1. README.md +13 -3
README.md CHANGED
@@ -57,6 +57,7 @@ The production profile uses:
57
 
58
  ```bash
59
  vllm serve WaveCut/gemma-4-26B-A4B-it-heretic-W4A16-G64-RTN-vllm \
 
60
  --served-model-name "Gemma 4 26B Heretic" \
61
  --trust-remote-code \
62
  --dtype bfloat16 \
@@ -70,9 +71,18 @@ vllm serve WaveCut/gemma-4-26B-A4B-it-heretic-W4A16-G64-RTN-vllm \
70
  --tool-call-parser gemma4
71
  ```
72
 
73
- The historical RTX 3090 benchmark and harness are included under `bench/` and
74
- `scripts/`. A fresh vLLM 0.23.0 versus 0.25.1 text, vision, and long-context
75
- comparison will be added after the identical A/B run completes on AI Farm.
 
 
 
 
 
 
 
 
 
76
 
77
  ## Limitations
78
 
 
57
 
58
  ```bash
59
  vllm serve WaveCut/gemma-4-26B-A4B-it-heretic-W4A16-G64-RTN-vllm \
60
+ --revision 51b3bbe6b47f12a1841e3934ce24e767bb8dec17 \
61
  --served-model-name "Gemma 4 26B Heretic" \
62
  --trust-remote-code \
63
  --dtype bfloat16 \
 
71
  --tool-call-parser gemma4
72
  ```
73
 
74
+ The checkpoint was accepted for production on AI Farm with vLLM 0.25.1,
75
+ Torch 2.11.0+cu130, Transformers 5.8.0, and compressed-tensors 0.17.0.
76
+ On an RTX 3090, both vLLM 0.23.0 and 0.25.1 passed 13/13 text and vision case
77
+ groups plus streaming checks. The matrix included exact needle recovery at
78
+ 39,186 text tokens and 38,490 multimodal tokens, including concurrency 2.
79
+ Both runtimes also passed 24/24 production-shaped Russian quality probes;
80
+ median probe latency improved from 0.6385 s to 0.6125 s on vLLM 0.25.1.
81
+
82
+ The full environment, timing table, harness, and raw JSONL artifacts are in
83
+ the [AI Farm vLLM report](https://github.com/iamwavecut/aifarm-vllm-openai/blob/af3a2915246a73585c175355af387c23e7360238/results/20260716-vllm-023-vs-0251.md).
84
+ The historical RTX 3090 quantization benchmark and harness remain available
85
+ under `bench/` and `scripts/`.
86
 
87
  ## Limitations
88