| --- |
| license: apache-2.0 |
| base_model: Agnes-AI/Agnes-3.0-Flash |
| base_model_relation: quantized |
| library_name: modelopt |
| pipeline_tag: image-text-to-text |
| tags: |
| - quantized |
| - nvfp4 |
| - fp8 |
| - modelopt |
| - custom_code |
| - tool-calling |
| --- |
| |
| # Agnes-3.0-Flash 32GB NVFP4/k=2 (candidate) |
|
|
| > **Hardware status:** This is a 32GB RTX Blackwell candidate profile. The packaged checkpoint has been validated on an RTX PRO 6000 96GB host; final RTX 5090D memory and throughput acceptance is still required. |
|
|
| This repository contains the mixed NVFP4/FP8 checkpoint intended for a single 32GB Blackwell card with a 262K configured context. |
|
|
| ## Quantization |
|
|
| - FFN main and parallel branches: NVFP4. |
| - Attention and recurrent QKV/QKVZ projections: FP8. |
| - Attention/recurrent output projections: NVFP4. |
| - Embedding, lm_head, MTP, vision linear/position/patch weights: FP8. |
| - Norms, biases, recurrent state: BF16/FP32. |
| - Main KV cache: NVFP4. MTP draft KV: FP8 E4M3. |
| |
| ## Runtime |
| |
| Use the custom patched SGLang runtime required by this checkpoint. The matching runtime patch archive is [`runtime-scripts-v2.tar.gz`](runtime-scripts-v2.tar.gz) (SHA256 `46d537da5a2ef47921fb85c659492274cc1f3dceaa35499d3dc8e773c89d2a16`). The validated profile is MTP k=2, 144 MiB FlashInfer workspace, max Mamba cache 1, prefill CUDA graph disabled, and ReplaySSM enabled. See [`docs/launch-config.json`](docs/launch-config.json) and [`docs/runtime-policy.json`](docs/runtime-policy.json). |
| |
| For retrieval-heavy long prompts, run [`docs/prepare_fact_anchor.py`](docs/prepare_fact_anchor.py) before generation. It appends [`docs/fact-anchor-template.txt`](docs/fact-anchor-template.txt)-equivalent tokens at the generation boundary while preserving the configured context limit. In a 260K probe this preserved all three access codes and the 57/18/7/32 inventory relation. |
| |
| ## Evidence and limits |
| |
| The combined FP8-vision/k=2/262K run used 261632 input tokens plus 56 output tokens. It produced correct retrieval facts, accepted 38/38 MTP drafts, and measured 106.68 tok/s over the short output. Startup-through-sequential-request NVML peak was 32044 MiB on an RTX PRO 6000 96GB host, leaving 724 MiB relative to a 32GiB budget. The required target gate is at least 500 MiB free; repeat this measurement on the RTX 5090D. |
| |
| Synthetic visual checks passed 3/3. A strict ten-item quality smoke passed 6/10, mostly due to formatting and reasoning-output expectations. Long-form relation behavior is prompt-sensitive; use the fact anchor for retrieval-heavy tasks. These results do not establish broad capability equivalence or 5090D performance. |
| |
| See [`docs/validation-k2-262k.json`](docs/validation-k2-262k.json), [`docs/5090D-acceptance-checklist.md`](docs/5090D-acceptance-checklist.md), and [`docs/checkpoint-manifest.json`](docs/checkpoint-manifest.json). |
| The checkpoint's embedded ModelOpt summary is reproduced in [`docs/quantization-metadata-summary.json`](docs/quantization-metadata-summary.json); KV cache precision is a runtime setting and is therefore not encoded in the weight metadata. |
| The release metadata/runtime bundle is covered by [`docs/release-manifest.json`](docs/release-manifest.json). |
| |
