brazley commited on
Commit
78286ad
·
verified ·
1 Parent(s): 2a90533

Add production-observed acceptance alongside the controlled benchmark — 82.44% / 3.473 accepted tokens/step over 20h of agent traffic, with temperature and harness caveats. Additive; the benchmark table is unchanged and correct.

Browse files
Files changed (1) hide show
  1. README.md +22 -0
README.md CHANGED
@@ -434,6 +434,28 @@ Stock Nemotron 3.5 Lightning NVFP4 reaches **94.72 tok/s** single-stream on this
434
 
435
  The drafter was trained against stock Lightning and runs here unmodified against a merged model. Acceptance sits within ~1.5 points of stock at every position and concurrency.
436
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
437
  This model's acceptance figures are gated by the shim's tensor-level verification, which refuses to serve on mismatch and refuses to certify when it has nothing comparable to check. The stock control's drafter loads through vLLM's own DSpark path, which carries no equivalent gate; its evidence of correct loading is behavioural.
438
 
439
  **Quantization Benchmark Scores:**
 
434
 
435
  The drafter was trained against stock Lightning and runs here unmodified against a merged model. Acceptance sits within ~1.5 points of stock at every position and concurrency.
436
 
437
+ **Observed in production, for contrast.** The table above is a controlled benchmark. Over 20 h
438
+ of continuous live agent traffic on a single DGX Spark GB10 — 91,458 drafts, 274,374 draft
439
+ tokens — the same engine and drafter measured **82.44% acceptance and 3.473 accepted
440
+ tokens/step**, per-position **92.8 / 80.9 / 73.7**.
441
+
442
+ The two figures are not in conflict; they are different workloads. Agent traffic — tool calls,
443
+ structured output, code — is far more predictable than open-ended generation, so the drafter
444
+ lands more of its proposals. **Read the table as the controlled figure and this as the
445
+ workload-dependent ceiling.**
446
+
447
+ Two caveats, because a number without its conditions invites the same mistake in reverse:
448
+
449
+ - **Temperature is uncontrolled here.** Callers used whatever sampling they used across those
450
+ 20 h. The table's rows state `t=0.0` and `t=1.0` explicitly; this row cannot, and some of
451
+ the gap is likely sampling rather than workload.
452
+ - **Same engine, same config.** A spec-off baseline taken on this deployment measured **79.73
453
+ tok/s** at c=1 against this card's **80.76** — 98.7% agreement, which is what licenses
454
+ comparing the two rows at all. Absolute *throughput* gains do not port across harnesses even
455
+ when the baseline does: the paired on/off delta on that harness was **+28.7%** at c=1 and
456
+ **+23.1%** at c=8, against this card's +13.2% / +12.4% measured on Nik's. Different prompts,
457
+ `max_tokens`, and cache state — the mechanism is the same, the percentage is not portable.
458
+
459
  This model's acceptance figures are gated by the shim's tensor-level verification, which refuses to serve on mismatch and refuses to certify when it has nothing comparable to check. The stock control's drafter loads through vLLM's own DSpark path, which carries no equivalent gate; its evidence of correct loading is behavioural.
460
 
461
  **Quantization Benchmark Scores:**