Any-to-Any
Transformers
Safetensors
NemotronH_Nano_Omni_Reasoning_V3
feature-extraction
multimodal
Merge
nvfp4
mixture-of-experts
speculative-decoding
custom_code
8-bit precision
modelopt
Instructions to use brazley/Brazley-Nemotron-3.5-Lightning-Omni with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use brazley/Brazley-Nemotron-3.5-Lightning-Omni with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("brazley/Brazley-Nemotron-3.5-Lightning-Omni", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Add production-observed acceptance alongside the controlled benchmark — 82.44% / 3.473 accepted tokens/step over 20h of agent traffic, with temperature and harness caveats. Additive; the benchmark table is unchanged and correct.
Browse files
README.md
CHANGED
|
@@ -434,6 +434,28 @@ Stock Nemotron 3.5 Lightning NVFP4 reaches **94.72 tok/s** single-stream on this
|
|
| 434 |
|
| 435 |
The drafter was trained against stock Lightning and runs here unmodified against a merged model. Acceptance sits within ~1.5 points of stock at every position and concurrency.
|
| 436 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 437 |
This model's acceptance figures are gated by the shim's tensor-level verification, which refuses to serve on mismatch and refuses to certify when it has nothing comparable to check. The stock control's drafter loads through vLLM's own DSpark path, which carries no equivalent gate; its evidence of correct loading is behavioural.
|
| 438 |
|
| 439 |
**Quantization Benchmark Scores:**
|
|
|
|
| 434 |
|
| 435 |
The drafter was trained against stock Lightning and runs here unmodified against a merged model. Acceptance sits within ~1.5 points of stock at every position and concurrency.
|
| 436 |
|
| 437 |
+
**Observed in production, for contrast.** The table above is a controlled benchmark. Over 20 h
|
| 438 |
+
of continuous live agent traffic on a single DGX Spark GB10 — 91,458 drafts, 274,374 draft
|
| 439 |
+
tokens — the same engine and drafter measured **82.44% acceptance and 3.473 accepted
|
| 440 |
+
tokens/step**, per-position **92.8 / 80.9 / 73.7**.
|
| 441 |
+
|
| 442 |
+
The two figures are not in conflict; they are different workloads. Agent traffic — tool calls,
|
| 443 |
+
structured output, code — is far more predictable than open-ended generation, so the drafter
|
| 444 |
+
lands more of its proposals. **Read the table as the controlled figure and this as the
|
| 445 |
+
workload-dependent ceiling.**
|
| 446 |
+
|
| 447 |
+
Two caveats, because a number without its conditions invites the same mistake in reverse:
|
| 448 |
+
|
| 449 |
+
- **Temperature is uncontrolled here.** Callers used whatever sampling they used across those
|
| 450 |
+
20 h. The table's rows state `t=0.0` and `t=1.0` explicitly; this row cannot, and some of
|
| 451 |
+
the gap is likely sampling rather than workload.
|
| 452 |
+
- **Same engine, same config.** A spec-off baseline taken on this deployment measured **79.73
|
| 453 |
+
tok/s** at c=1 against this card's **80.76** — 98.7% agreement, which is what licenses
|
| 454 |
+
comparing the two rows at all. Absolute *throughput* gains do not port across harnesses even
|
| 455 |
+
when the baseline does: the paired on/off delta on that harness was **+28.7%** at c=1 and
|
| 456 |
+
**+23.1%** at c=8, against this card's +13.2% / +12.4% measured on Nik's. Different prompts,
|
| 457 |
+
`max_tokens`, and cache state — the mechanism is the same, the percentage is not portable.
|
| 458 |
+
|
| 459 |
This model's acceptance figures are gated by the shim's tensor-level verification, which refuses to serve on mismatch and refuses to certify when it has nothing comparable to check. The stock control's drafter loads through vLLM's own DSpark path, which carries no equivalent gate; its evidence of correct loading is behavioural.
|
| 460 |
|
| 461 |
**Quantization Benchmark Scores:**
|