XenonFear128/Agnes-3.0-Flash-32GB-NVFP4-k2

Agnes-3.0-Flash 32GB NVFP4/k=2 (candidate)

Hardware status: This is a 32GB RTX Blackwell candidate profile. The packaged checkpoint has been validated on an RTX PRO 6000 96GB host; final RTX 5090D memory and throughput acceptance is still required.

This repository contains the mixed NVFP4/FP8 checkpoint intended for a single 32GB Blackwell card with a 262K configured context.

Quantization

• FFN main and parallel branches: NVFP4.
• Attention and recurrent QKV/QKVZ projections: FP8.
• Attention/recurrent output projections: NVFP4.
• Embedding, lm_head, MTP, vision linear/position/patch weights: FP8.
• Norms, biases, recurrent state: BF16/FP32.
• Main KV cache: NVFP4. MTP draft KV: FP8 E4M3.

Runtime

Use the custom patched SGLang runtime required by this checkpoint. The matching runtime patch archive is runtime-scripts-v2.tar.gz (SHA256 46d537da5a2ef47921fb85c659492274cc1f3dceaa35499d3dc8e773c89d2a16). The validated profile is MTP k=2, 144 MiB FlashInfer workspace, max Mamba cache 1, prefill CUDA graph disabled, and ReplaySSM enabled. See docs/launch-config.json and docs/runtime-policy.json.

For retrieval-heavy long prompts, run docs/prepare_fact_anchor.py before generation. It appends docs/fact-anchor-template.txt-equivalent tokens at the generation boundary while preserving the configured context limit. In a 260K probe this preserved all three access codes and the 57/18/7/32 inventory relation.

Evidence and limits

The combined FP8-vision/k=2/262K run used 261632 input tokens plus 56 output tokens. It produced correct retrieval facts, accepted 38/38 MTP drafts, and measured 106.68 tok/s over the short output. Startup-through-sequential-request NVML peak was 32044 MiB on an RTX PRO 6000 96GB host, leaving 724 MiB relative to a 32GiB budget. The required target gate is at least 500 MiB free; repeat this measurement on the RTX 5090D.

Synthetic visual checks passed 3/3. A strict ten-item quality smoke passed 6/10, mostly due to formatting and reasoning-output expectations. Long-form relation behavior is prompt-sensitive; use the fact anchor for retrieval-heavy tasks. These results do not establish broad capability equivalence or 5090D performance.

See docs/validation-k2-262k.json, docs/5090D-acceptance-checklist.md, and docs/checkpoint-manifest.json. The checkpoint's embedded ModelOpt summary is reproduced in docs/quantization-metadata-summary.json; KV cache precision is a runtime setting and is therefore not encoded in the weight metadata. The release metadata/runtime bundle is covered by docs/release-manifest.json.