File size: 8,498 Bytes
20523e7 | 1 | <svg xmlns="http://www.w3.org/2000/svg" width="1200" height="1100" viewBox="0 0 1200 1100" role="img" aria-labelledby="title desc"><title id="title">APUS-OpenJev vLLM response latency and historical comparison</title><desc id="desc">Provided 333-request run snapshot: P50 25.58, P95 222.36, P99 275.58 milliseconds; 333 successful, zero errors. Source screenshot, raw requests not independently verified. Historical systems have different timing boundaries and cannot support speedup ratios.</desc><rect width="1200" height="1100" rx="20" fill="#f6f8fc"/><g font-family="Arial, Helvetica, sans-serif"><text x="36" y="47" font-size="29" fill="#14243c" font-weight="700">Fast decision responses over vLLM HTTP.</text><text x="36" y="76" font-size="16" fill="#60708a" font-weight="400">9B · effort: high · vLLM HTTP API · 1× RTX PRO 6000 Blackwell 96GB</text><rect x="24" y="99" width="276" height="101" rx="8" fill="white" stroke="#dce3ee"/><text x="42" y="129" font-size="15" fill="#60708a" font-weight="400">P50 response</text><text x="42" y="176" font-size="30" fill="#087f73" font-weight="700">25.58 ms</text><rect x="316" y="99" width="276" height="101" rx="8" fill="white" stroke="#dce3ee"/><text x="334" y="129" font-size="15" fill="#60708a" font-weight="400">P95 response</text><text x="334" y="176" font-size="30" fill="#087f73" font-weight="700">222.36 ms</text><rect x="608" y="99" width="276" height="101" rx="8" fill="white" stroke="#dce3ee"/><text x="626" y="129" font-size="15" fill="#60708a" font-weight="400">P99 response</text><text x="626" y="176" font-size="30" fill="#087f73" font-weight="700">275.58 ms</text><rect x="900" y="99" width="276" height="101" rx="8" fill="white" stroke="#dce3ee"/><text x="918" y="129" font-size="15" fill="#60708a" font-weight="400">HTTP success</text><text x="918" y="176" font-size="30" fill="#087f73" font-weight="700">333 / 333</text><text x="36" y="228" font-size="17" fill="#14243c" font-weight="700">Error rate: 0% · full-response HTTP timing</text><text x="36" y="253" font-size="14" fill="#60708a" font-weight="400">4B · reported P50 / ratio-derived tails: P50 18.00 / P95 100.06 / P99 154.32 ms</text><rect x="24" y="277" width="1152" height="316" rx="8" fill="white" stroke="#dce3ee"/><text x="44" y="310" font-size="22" fill="#14243c" font-weight="700">P50 · typical response</text><text x="770" y="310" font-size="15" fill="#60708a" font-weight="400">Linear scale: 0–500 ms</text><line x1="356.0" y1="334" x2="356.0" y2="551" stroke="#e1e7ef"/><text x="350.0" y="575" font-size="12" fill="#60708a" font-weight="400">0</text><line x1="498.4" y1="334" x2="498.4" y2="551" stroke="#e1e7ef"/><text x="492.4" y="575" font-size="12" fill="#60708a" font-weight="400">100</text><line x1="640.8" y1="334" x2="640.8" y2="551" stroke="#e1e7ef"/><text x="634.8" y="575" font-size="12" fill="#60708a" font-weight="400">200</text><line x1="783.2" y1="334" x2="783.2" y2="551" stroke="#e1e7ef"/><text x="777.2" y="575" font-size="12" fill="#60708a" font-weight="400">300</text><line x1="925.6" y1="334" x2="925.6" y2="551" stroke="#e1e7ef"/><text x="919.6" y="575" font-size="12" fill="#60708a" font-weight="400">400</text><line x1="1068.0" y1="334" x2="1068.0" y2="551" stroke="#e1e7ef"/><text x="1062.0" y="575" font-size="12" fill="#60708a" font-weight="400">500</text><text x="44" y="364" font-size="16" fill="#14243c" font-weight="700">APUS-OpenJev 4B</text><text x="44" y="382" font-size="11" fill="#60708a" font-weight="400">vLLM HTTP / reported + ratio-derived</text><rect x="356" y="349" width="25.631999999999998" height="28" rx="3" fill="#245cdb"/><text x="391.632" y="369" font-size="16" fill="#245cdb" font-weight="700">18.00</text><text x="44" y="418" font-size="16" fill="#14243c" font-weight="700">APUS-OpenJev 9B · high</text><text x="44" y="436" font-size="11" fill="#60708a" font-weight="400">vLLM HTTP API / full response / snapshot</text><rect x="356" y="403" width="36.42592" height="28" rx="3" fill="#087f73"/><text x="402.42592" y="423" font-size="16" fill="#087f73" font-weight="700">25.58</text><text x="44" y="472" font-size="16" fill="#14243c" font-weight="700">Jev official API</text><text x="44" y="490" font-size="11" fill="#60708a" font-weight="400">TypeSafe HTTP / 240 pooled requests</text><rect x="356" y="457" width="399.8592" height="28" rx="3" fill="#8961a8"/><text x="765.8592" y="477" font-size="16" fill="#8961a8" font-weight="700">280.80</text><text x="44" y="526" font-size="16" fill="#14243c" font-weight="700">Laya typed / full input</text><text x="44" y="544" font-size="11" fill="#60708a" font-weight="400">Local system_one / three-run median</text><rect x="356" y="511" width="12.36032" height="28" rx="3" fill="#138b76"/><text x="378.36032" y="531" font-size="16" fill="#138b76" font-weight="700">8.68</text><rect x="24" y="610" width="1152" height="316" rx="8" fill="white" stroke="#dce3ee"/><text x="44" y="643" font-size="22" fill="#14243c" font-weight="700">P95 · slower tail</text><text x="770" y="643" font-size="15" fill="#60708a" font-weight="400">Linear scale: 0–500 ms</text><line x1="356.0" y1="667" x2="356.0" y2="884" stroke="#e1e7ef"/><text x="350.0" y="908" font-size="12" fill="#60708a" font-weight="400">0</text><line x1="498.4" y1="667" x2="498.4" y2="884" stroke="#e1e7ef"/><text x="492.4" y="908" font-size="12" fill="#60708a" font-weight="400">100</text><line x1="640.8" y1="667" x2="640.8" y2="884" stroke="#e1e7ef"/><text x="634.8" y="908" font-size="12" fill="#60708a" font-weight="400">200</text><line x1="783.2" y1="667" x2="783.2" y2="884" stroke="#e1e7ef"/><text x="777.2" y="908" font-size="12" fill="#60708a" font-weight="400">300</text><line x1="925.6" y1="667" x2="925.6" y2="884" stroke="#e1e7ef"/><text x="919.6" y="908" font-size="12" fill="#60708a" font-weight="400">400</text><line x1="1068.0" y1="667" x2="1068.0" y2="884" stroke="#e1e7ef"/><text x="1062.0" y="908" font-size="12" fill="#60708a" font-weight="400">500</text><text x="44" y="697" font-size="16" fill="#14243c" font-weight="700">APUS-OpenJev 4B</text><text x="44" y="715" font-size="11" fill="#60708a" font-weight="400">vLLM HTTP / reported + ratio-derived</text><rect x="356" y="682" width="142.488288" height="28" rx="3" fill="#245cdb"/><text x="508.488288" y="702" font-size="16" fill="#245cdb" font-weight="700">100.06</text><text x="44" y="751" font-size="16" fill="#14243c" font-weight="700">APUS-OpenJev 9B · high</text><text x="44" y="769" font-size="11" fill="#60708a" font-weight="400">vLLM HTTP API / full response / snapshot</text><rect x="356" y="736" width="316.64064" height="28" rx="3" fill="#087f73"/><text x="682.6406400000001" y="756" font-size="16" fill="#087f73" font-weight="700">222.36</text><text x="44" y="805" font-size="16" fill="#14243c" font-weight="700">Jev official API</text><text x="44" y="823" font-size="11" fill="#60708a" font-weight="400">TypeSafe HTTP / 240 pooled requests</text><rect x="356" y="790" width="536.22144" height="28" rx="3" fill="#8961a8"/><text x="902.22144" y="810" font-size="16" fill="#8961a8" font-weight="700">376.56</text><text x="44" y="859" font-size="16" fill="#14243c" font-weight="700">Laya typed / full input</text><text x="44" y="877" font-size="11" fill="#60708a" font-weight="400">Local system_one / three-run median</text><rect x="356" y="844" width="53.62784" height="28" rx="3" fill="#138b76"/><text x="419.62784" y="864" font-size="16" fill="#138b76" font-weight="700">37.66</text><text x="36" y="958" font-size="16" fill="#14243c" font-weight="700">Verified HTTP reference · concurrency 1 · three rounds × 1,040 requests</text><text x="36" y="984" font-size="15" fill="#60708a" font-weight="400">Median-of-round P50 / P95 / P99: 25.34 / 222.83 / 276.78 ms · 3,120 / 3,120 successful</text><text x="36" y="1020" font-size="15" fill="#60708a" font-weight="400">Different timing boundaries and workloads: these bars do not establish speedup ratios.</text><text x="36" y="1044" font-size="13" fill="#60708a" font-weight="400">TypeSafe: pooled 240 HTTP 200 calls; mean 302.65 ms; P99 830.45 ms. Includes public-network overhead.</text><text x="36" y="1067" font-size="13" fill="#60708a" font-weight="400">Same host: Google ping 31.97 ms (5); TypeSafe TCP 443 181.55 ms (10); TLS 184.57 ms (10 new connections).</text><text x="36" y="1086" font-size="13" fill="#60708a" font-weight="400">TypeSafe ICMP: no response. API: 240 consecutive calls. TCP/TLS probes separate; no per-request subtraction.</text></g></svg> |