Model card: measured results, reproduction, known issues

README.md

@@ -0,0 +1,308 @@
+---
+license: apache-2.0
+base_model: Agnes-AI/Agnes-3.0-Flash
+base_model_relation: quantized
+pipeline_tag: image-text-to-text
+library_name: gguf
+tags:
+  - gguf
+  - llama.cpp
+  - rocm
+  - amd
+  - rocmfp4
+  - rocmfpx
+  - strix-halo
+  - amd-strix-halo
+  - gfx1151
+  - ryzen-ai-max
+  - ryzen-ai-max-395
+  - radeon-8060s
+  - mtp
+  - speculative-decoding
+  - reasoning
+  - multimodal
+  - vision
+  - agnes
+  - qwen3.5
+  - quantized
+  - imatrix
+---
+
+# Agnes-3.0-Flash Preview — ROCmFP4 **imatrix** for AMD Strix Halo (gfx1151)
+
+Importance-matrix-calibrated ROCmFP4 quantizations of
+**[Agnes-AI/Agnes-3.0-Flash](https://huggingface.co/Agnes-AI/Agnes-3.0-Flash)** (Preview, 33B dense, 262,144-token context,
+text + image). Companion to the standard build **[kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF)** — the same two 4-bit tiers,
+same fold, same built-in MTP head, same vision projector and the same prompt-cache patch; the only difference is how each
+4-bit block's scale was chosen.
+
+## What the imatrix changes
+
+ROCmFP4 has an importance-weighted quantizer path: with `--imatrix`, each block's scale is chosen by an exhaustive
+search that minimises error **weighted by how strongly the calibration activations use each weight**, instead of the
+unweighted default. It changes **which** scales are picked at the **same** bit width and tensor types — so it moves
+quality, not size, and per-token compute is identical.
+
+| | |
+| --- | --- |
+| calibration text | bartowski calibration_datav3.txt (the widely used community calibration set) |
+| computed on | BF16 GGUF, 129 chunks × 512 tokens, ROCm0 |
+| coverage | **558 entries = every matmul weight in the 72 trunk layers** (54 delta-rule layers × 8 + 18 global layers × 7); the output projection and the MTP block have none |
+| file | `Agnes-3.0-Flash-Preview.imatrix` (GGUF format), sha256 `eb66b219d8bb057b78b933e5b78042fc47e028415d626690883c8c8f72707a09` |
+
+## Measured effect
+
+Quality is graded against the **BF16 GGUF** on a **held-out** corpus (wikitext-2 *test*, `-c 2048`, 40 chunks × 1,023 scored tokens each — the second half of every window, less its first token — = 40,920), never on the imatrix calibration text. **KLD** is the per-token KL divergence of each quant's next-token distribution from BF16's on the same tokens — far more sensitive than perplexity, whose error bar here is wider than most of the differences. The calibration text and the grading text are different corpora.
+
+| Tier | Build | Size | KLD vs BF16 ↓ | Same top-1 ↑ | PPL (× BF16) | 99th-pct KLD |
+| --- | --- | ---: | ---: | ---: | ---: | ---: |
+| Q4_0_ROCMFP4_STRIX_LEAN | standard | 16.82 GiB | 0.0438 ± 0.0010 | 91.29 % | 6.4877 ± 0.0792 (×1.0250) | 0.4262 |
+| Q4_0_ROCMFP4_STRIX_LEAN | **imatrix** | 16.82 GiB | 0.0387 ± 0.0010 | 91.52 % | 6.4269 ± 0.0777 (×1.0154) | 0.3863 |
+| | *Δ imatrix* | | -11.6 % (3.5σ) | +0.23 pp | -0.9 % (0.5σ) | -9.3 % |
+| Q4_0_ROCMFP4_COHERENT | standard | 17.77 GiB | 0.0385 ± 0.0009 | 91.39 % | 6.4290 ± 0.0780 (×1.0157) | 0.3765 |
+| Q4_0_ROCMFP4_COHERENT | **imatrix** | 17.77 GiB | 0.0322 ± 0.0008 | 92.11 % | 6.4004 ± 0.0772 (×1.0112) | 0.3131 |
+| | *Δ imatrix* | | -16.4 % (5.4σ) | +0.72 pp | -0.4 % (0.3σ) | -16.8 % |
+
+σ = difference divided by the two runs' combined standard error. The two runs score the **same** tokens, so this is
+conservative (paired noise is smaller).
+
+**The imatrix measurably improves both tiers.** Mean KLD falls **-11.6 %** on STRIX_LEAN (3.5σ) and **-16.4 %** on COHERENT (5.4σ). The median moves further than the mean (-16.9 % / -18.3 %), so the gain is broad rather than a few repaired outliers, and the 99th-percentile tail also improves (-9.3 % / -16.8 %). Top-1 agreement with BF16 rises +0.23 / +0.72 percentage points.
+
+Two practical consequences: **imatrix STRIX_LEAN matches standard COHERENT's quality within noise** (KLD 0.0387 vs 0.0385, 0.2σ; PPL ratio ×1.0154 vs ×1.0157) in a file 5.3 % smaller, and **imatrix COHERENT is the lowest-KLD 4-bit build of this model we measured** (KLD 0.0322).
+
+**Where the quality numbers come from.** Measured directly on the files in this repo, against BF16 logits recomputed in the same session, and the STRIX_LEAN grade was run twice: all 40 per-chunk rows matched (fresh BF16 logits: `recipe/logs/R2_*`; grades: `R3_*`; repeat: `R4_*`). Today's 4-bit KLD values are 7.8–9.4 % higher than an earlier grading of the same weights on the same machine, before a reboot, while the 8-bit files reproduced their earlier output exactly; BF16's own logits also changed across that reboot, which is why every file here is graded against BF16 logits recomputed today. Those earlier numbers are not used. Grading again with a 16-thread CPU job running alongside (9.49 s per chunk, close to the earlier session's 9.66 s; today's quiet run took 7.53 s) gave today's rows exactly, so CPU load does not change the output.
+
+## Which file should I use?
+
+**Start with the imatrix `STRIX_LEAN`.** It matches the *standard* COHERENT's quality within noise (KLD 0.0387 vs 0.0385, 0.2σ) at STRIX_LEAN's size; with MTP, `STRIX_LEAN` decodes 13 % faster than `COHERENT` on ROCm0 (26.31 vs 23.30 tok/s; per-run ranges 25.56–26.42 and 23.29–23.66) and within 0.4 % of it on Vulkan0 (25.37 vs 25.46), and prefills 12 % faster than it on ROCm0 (260.8 vs 231.9 tok/s). Take the imatrix `COHERENT` when quality matters more than speed: it is the lowest-KLD 4-bit build we measured (0.0322, 17 % below the imatrix STRIX_LEAN). Decode speed differences under 8.3 % are treated as a tie: files that do identical work per token (each imatrix file and its standard twin) measured up to 8.3 % apart, because draft acceptance varies with the prompt. Prefill barely varies (twins within 2.0 %), so prefill gaps above 3.0 % are treated as real.
+
+| File | ftype | Size | BPW | KLD vs BF16 ↓² | Same top-1 ↑ | PPL (× BF16) | TG MTP ROCm0 | TG MTP Vulkan0 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | 106 | 16.82 GiB | 4.42 | 0.0387 ± 0.0010 | 91.52 % | 6.4269 ± 0.0777 (×1.0154) | 26.31 | 25.37 |
+| `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-COHERENT.gguf` | 102 | 17.77 GiB | 4.67 | 0.0322 ± 0.0008 | 92.11 % | 6.4004 ± 0.0772 (×1.0112) | 23.30 | 25.46 |
+| *BF16 reference* | 32 | 60.85 GiB³ | 16 | 0 | 100 % | 6.3296 ± 0.0762¹ | — | — |
+
+TG = decode tokens/s after a ~7.1K-token prompt, in-file MTP head, `--spec-draft-n-max 4`, default (non-strict) verification.
+¹ The BF16 PPL shown is the paired base every "×" ratio is computed against (averaged over the same scored tokens in the KL-divergence runs). The standalone BF16 run's own summary line reads 6.3314 ± 0.0763.
+² Quality columns: see *Where the quality numbers come from* below.
+³ BF16 conversion including the MTP layer (the graded BF16 trunk without it is 60.00 GiB); not published.
+
+The imatrix changes scale values, not tensor types or sizes: tensor by tensor, each imatrix file has the same names, types and byte sizes as its standard twin (the files differ by 224 / 224 bytes of header metadata), so the work per token is the same. Measured speed still differs because drafts were accepted at a different rate on the benchmark prompt — imatrix vs standard, n-max 4: STRIX_LEAN on ROCm0 26.31 vs 26.64 tok/s (acceptance 0.697 vs 0.724); STRIX_LEAN on Vulkan0 25.37 vs 27.31 tok/s (acceptance 0.684 vs 0.766); COHERENT on ROCm0 23.30 vs 24.76 tok/s (acceptance 0.618 vs 0.660); COHERENT on Vulkan0 25.46 vs 23.51 tok/s (acceptance 0.715 vs 0.653). The largest of these gaps, 8.3 %, is the spread both cards use to judge decode differences.
+
+Full speed tables (depth check, strict mode, both backends, workload range), prompt-cache and tool-calling results are on
+the [standard card](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF). The 8-bit tiers are not duplicated here; they are on the standard repo.
+
+## Quick start
+
+**1. Download**
+
+```bash
+hf download kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-imatrix-GGUF --local-dir ~/models/agnes
+```
+
+**2. Build `llama-server` with the prompt-cache patch** — ROCmFPX at the measured commit (ROCm and Vulkan
+prerequisites: the project's [build guide](https://github.com/charlie12345/ROCmFPX/blob/d3ca53726109b46fd5cd6fa42f688c707079b44e/docs/build.md))
+
+```bash
+git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
+git checkout d3ca53726109b46fd5cd6fa42f688c707079b44e
+git apply ~/models/agnes/recipe/patches/mtp-prompt-cache-fix.patch
+HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
+cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \
+  -DGGML_HIP=ON -DGGML_VULKAN=ON -DGPU_TARGETS=gfx1151 \
+  -DGGML_HIP_GRAPHS=ON -DGGML_HIP_NO_VMM=ON -DLLAMA_CURL=OFF
+cmake --build build --target llama-server -j
+```
+
+(The CMake options of the measured build are listed in [Reproduction](#reproduction).)
+
+`d3ca537` is also in the history of the official [ROCmFPX/ROCmFPX](https://github.com/ROCmFPX/ROCmFPX) repository; the
+patch was verified against `d3ca537` only.
+
+**3. Serve** — the MTP head is inside the model file, so there is no `--model-draft`:
+
+Text and tool calls, with MTP:
+
+```bash
+env LD_LIBRARY_PATH=$PWD/build/bin:/opt/rocm/lib HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
+build/bin/llama-server \
+  -m ~/models/agnes/Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf \
+  --spec-type draft-mtp \
+  --spec-draft-n-max 4 --spec-draft-n-min 0 --spec-draft-p-min 0.0 \
+  -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -dev ROCm0 \
+  -c 65536 --host 127.0.0.1 --port 8080
+```
+
+Images (`--mmproj`), **without** MTP:
+
+```bash
+env LD_LIBRARY_PATH=$PWD/build/bin:/opt/rocm/lib HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
+build/bin/llama-server \
+  -m ~/models/agnes/Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf \
+  --mmproj ~/models/agnes/mmproj-Agnes-3.0-Flash-Preview-BF16.gguf \
+  -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -dev ROCm0 \
+  -c 65536 --host 127.0.0.1 --port 8080
+```
+
+> ⛔ **Images:** with the MTP head loaded, image requests fail on this build ([measured](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF#vision)). For image input, drop the two `--spec-*` lines.
+
+(`LD_LIBRARY_PATH` avoids a soname clash on machines that also have a Vulkan-only llama.cpp build.) The benchmarks used
+the MTP command without `--mmproj`; the exact measured argv is in [Reproduction](#reproduction).
+
+| Flag | Why |
+| --- | --- |
+| `--spec-type draft-mtp` | Uses the MTP layer inside the model file; no second file to download or place. |
+| `--spec-draft-n-max 4` | llama.cpp's default is 16. On this model the measured optimum is about 4 — see [the table](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF#speculative-decoding-mtp). |
+| `--spec-draft-p-min 0.0` | The default 0.75 gate throttles drafting. |
+| `--jinja` | Already on by default in this build; keep it on — the thinking controls (`chat_template_kwargs`) and tool calling rely on the Jinja chat template. |
+| `-fit off` | Autofit reads `MemAvailable` on integrated GPUs and can silently shrink context or push tensors to CPU. |
+| `-cram <MiB>` | Not set above (default 8 GiB of host RAM for saved prompts). Set it on a shared box — see [Known issues](#known-issues-and-limits). |
+
+> ⚠️ **On an unpatched build the in-file head still works, but prompt caching turns off while it is loaded** — every
+> turn re-prefills the whole conversation ([measured](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF#prompt-caching-with-mtp)). For multi-turn / agentic use there,
+> drop the two `--spec-*` lines.
+
+Requires a llama.cpp build with ROCmFP4 / ROCmFPX tensor-type support; stock llama.cpp rejects these tensor types.
+
+## Reasoning controls
+
+⛔ **Do not pass `reasoning_effort: "high"`.**
+
+The upstream model card lists `high` as the default effort. **The chat template shipped with the checkpoint does not
+accept it.** It defaults to `xhigh` and accepts only `xhigh`, `medium`, `low`; anything else raises:
+
+```
+Unexpected reasoning effort high. Supported types are xhigh (default), medium, and low.
+```
+
+Verified by rendering the template with each value (`xhigh` / `medium` / `low` render; `high` raises; `medium` injects
+no instruction at all). Use:
+
+| goal | request field |
+| --- | --- |
+| default deep reasoning | nothing, or `"chat_template_kwargs": {"reasoning_effort": "xhigh"}` |
+| shorter thinking | `"chat_template_kwargs": {"reasoning_effort": "low"}` |
+| thinking off | `"chat_template_kwargs": {"enable_thinking": false}` |
+
+With thinking on and a small `max_tokens`, the whole budget can go to reasoning and `content` comes back empty — raise
+`max_tokens` before concluding the model is broken.
+
+⚠️ **`preserve_thinking` defaults to true** in this template: earlier turns' reasoning is re-rendered into the prompt
+(Qwen3.5's template drops it). Multi-turn and agentic contexts grow faster than you may expect; pass
+`"preserve_thinking": false` in `chat_template_kwargs` to get the Qwen3.5 behaviour.
+
+## MTP, prompt caching, tool calling, vision
+
+Same architecture, template, in-file MTP block (258 MiB in STRIX_LEAN,
+270 MiB in COHERENT) and projector as the standard build. The prompt-cache patch
+is included here too (`recipe/patches/`). The tool-calling suite ran on this repo's STRIX_LEAN — see
+[the standard card](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF#tool-calling).
+
+## How the parallel FFN was handled
+
+Agnes-3.0-Flash Preview is a Qwen3.5-style dense hybrid (72 layers: 54 gated-delta-rule + 18 global attention,
+interval 4) with one addition: **every layer runs a second, narrower SwiGLU (width 2048) in parallel with the
+main one (width 17408), and the two outputs are summed.** The reference implementation is literally
+
+```python
+y = down(act(gate(x)) * up(x)) + parallel_ffn(x)
+```
+
+Because it is a plain sum through the same activation, the branch folds **exactly** into one wider SwiGLU:
+`gate`/`up` concatenated along the output dimension and `down` along the input dimension, giving a stock `qwen35`
+MLP of width **19456**. That is the same transformation Agnes AI's own SGLang patch applies. The result loads on any
+llama.cpp/ROCmFPX build with `qwen35` support — **no Agnes-specific patch** (some other GGUFs of this model keep the
+branch as separate `ffn_*_par` tensors and require a patched fork).
+
+Three details that are easy to get wrong:
+
+1. **Rename `delta_attn.*` → `linear_attn.*`.** The converter reorders the 48 value heads into the tiled order ggml
+   expects *only* for tensors named `linear_attn.*`. Skipping the rename produces a model that loads and runs and is wrong.
+2. **The MTP layer has no parallel branch** (17408 wide), so its MLP is zero-padded to 19456. Zero
+   SwiGLU rows contribute exactly zero.
+3. Agnes's RMSNorm is one-centred, `x / rms(x) · (1 + w)`, so the converter's `+1` weight offset is correct; the gated
+   delta-rule norm is a plain scale and is correctly left alone.
+
+The fold boundary sits at column 17408 = 544 × 32 = 68 × 256, so no quantization block mixes main and parallel weights.
+
+**Verification (`recipe/verify_fold.py`, which re-derives every tensor from the source independently): PASS.**
+1305 output tensors, every one **bit-identical** to its source slice (216 folded,
+3 MTP-padded, 1086 passed through), and the folded MLP reproduces the reference
+forward pass to float32 rounding (max relative error: layer 0: 8.5e-10, layer 36: 5.4e-08, layer 71: 2.5e-08, MTP: 3.0e-08). The full log is in `recipe/verify_fold.log`.
+
+## Quantization methodology
+
+```bash
+llama-imatrix -m Agnes-3.0-Flash-Preview-BF16.gguf -f calibration_datav3.txt -o Agnes-3.0-Flash-Preview.imatrix \
+  -ngl 999 -c 512 -b 512 -fa on -dio -dev ROCm0
+llama-quantize --imatrix Agnes-3.0-Flash-Preview.imatrix --output-tensor-type q6_K \
+  Agnes-3.0-Flash-Preview-MTP-BF16.gguf Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0_ROCMFP4_STRIX_LEAN.gguf Q4_0_ROCMFP4_STRIX_LEAN 16
+llama-quantize --imatrix Agnes-3.0-Flash-Preview.imatrix --output-tensor-type q6_K --token-embedding-type q6_K \
+  Agnes-3.0-Flash-Preview-MTP-BF16.gguf Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0_ROCMFP4_COHERENT.gguf Q4_0_ROCMFP4_COHERENT 16
+```
+
+The imatrix was computed on the trunk-only BF16; the MTP layer is quantized from the BF16 that includes it
+(`Agnes-3.0-Flash-Preview-MTP-BF16.gguf`), with the unweighted path for its tensors.
+
+Receipts that the weighted path was actually taken, and that the shipped trunk is the graded one:
+
+| File | `output.weight` | `token_embd.weight` | tensors | trunk vs trunk-only build | MTP block | imatrix entries | bytes differ from standard |
+| --- | --- | --- | ---: | :---: | ---: | ---: | :---: |
+| `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | Q6_K | Q5_K | 972 | identical (957 tensors) | 258 MiB | 558 | yes |
+| `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-COHERENT.gguf` | Q6_K | Q6_K | 972 | identical (957 tensors) | 270 MiB | 558 | yes |
+
+## Reproduction
+
+```
+server   : https://github.com/charlie12345/ROCmFPX @ d3ca53726109b46fd5cd6fa42f688c707079b44e
+           + recipe/patches/mtp-prompt-cache-fix.patch (sha256 412a0173f60faa937018f0153efe9b5d9230c1606735502485e7e520fffb68dd), applied cleanly (git apply --check)
+           built target llama-server with the same CMake options as below; sha256 14e9f1299240c891478c820b8c20811ca5721044d2e63e23799eaa9be26b30bd
+tools    : same commit, unpatched (quantize / imatrix / perplexity; the patch touches only common/ and tools/server/)
+           build dir /opt/llama-rocm/rocmfpx-724/build-hipvk, Release, Unix Makefiles, GGML_HIP=ON GGML_VULKAN=ON
+           GGML_HIP_GRAPHS=ON GGML_HIP_NO_VMM=ON GGML_NATIVE=ON AMDGPU_TARGETS=gfx1151 LLAMA_CURL=OFF
+           (this CMake forwards AMDGPU_TARGETS to GPU_TARGETS, the spelling used in Quick start)
+           CMAKE_HIP_COMPILER=/opt/rocm-7.2.4/lib/llvm/bin/clang
+           sha256 llama-quantize   1559189968a30781f837e4d25ca3cdf26c8eb0351023675406535eddcbe0db13
+           sha256 llama-imatrix    bfdce5c21a01bdc8043d69335b74a7b68b8335ae989991a1e10e21d5faeaeb8e
+           sha256 llama-perplexity c2175925bb0c61df89ff787845d8e03a35403f684be52719ac2987b8ca636a53
+           sha256 llama-server     d077b98f3f04df4b482a5826c8922e39cda3aa8d2e2b2fbe1870f36d245cebe7   (unpatched; used for the unpatched control rows: prompt cache and vision)
+source   : Agnes-AI/Agnes-3.0-Flash  revision 891ce4f9ffb89b22888aa7fcc2bb2f3618867684
+           folded with recipe/fold_agnes.py (sha256 b1e6dbcf16bbac48fbc95b763efeb85efb21ce1d1fdfa825a6ae01187f044486)
+model    : Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf
+argv     : /opt/llama-rocm/rocmfpx-724-mtpcache/build-hipvk/bin/llama-server -m /mnt/models/agnes-3.0-flash/out-mtp-imat/Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0_ROCMFP4_STRIX_LEAN.gguf -dev ROCm0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-n-min 0 --spec-draft-p-min 0.0
+env      : LD_LIBRARY_PATH=<patched build>/bin:/opt/rocm-7.2.4/lib
+           HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1
+box      : aimax — AMD Ryzen AI Max+ 395 / Radeon 8060S (gfx1151), 124 GiB, GTT 131072 MiB,
+           kernel 6.17.6-061706-generic, ROCm 7.2.4
+power    : no platform_profile interface on this box; cpufreq governor=performance, EPP=performance;
+           GPU measured 84-85 W at 95-100% busy under load
+protocol : ~7.1K-token prompt, 256 generated tokens, temp 0 / top_k 1, ignore_eos, cache_prompt false,
+           1 warm-up + median of 3, no co-resident models (box iced)
+measured : 2026-09-16, by the pipeline in recipe/ (every raw number in recipe/results_summary.json and recipe/raw/)
+```
+
+## Files
+
+| File | Size | sha256 |
+| --- | ---: | --- |
+| `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | 16.82 GiB | `6aae32cf31652a509044288c31e2e38e5d4b3f8fb42e9531ef1498a48d1fc8cd` |
+| `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-COHERENT.gguf` | 17.77 GiB | `643d7a6f601826c7ee2ff3cc2b8092ef971d1dec1e6f1fff6c36ab6c4d875f30` |
+| `Agnes-3.0-Flash-Preview.imatrix` | 15.2 MiB | `eb66b219d8bb057b78b933e5b78042fc47e028415d626690883c8c8f72707a09` |
+| `mmproj-Agnes-3.0-Flash-Preview-BF16.gguf` | 0.87 GiB | `d35b469bce95e80dfa5928e7547089ceefe8e9bb77452349a3af02c5679d7026` |
+
+## Known issues and limits
+
+- Calibration text is general-purpose English/code; an imatrix built from your own domain can do better on that domain.
+- Graded on wikitext-2 *test*, a different corpus from the calibration text. Improvements on reasoning/code tasks were not measured.
+- The imatrix covers the 72 trunk layers only. The MTP block (`blk.72.*`) and the output projection were quantized without importance weights — `llama-quantize` logs `did not find weights` for each — because the calibration forward pass never runs them (`recipe/logs/`).
+- **Preview checkpoint.** Agnes AI's published benchmark results are for a different (production/API) checkpoint.
+- **`reasoning_effort: "high"` raises** in the shipped chat template — see [Reasoning controls](#reasoning-controls).
+- **Prompt caching with MTP needs the included server patch.** Without it the in-file head still drafts, but every turn re-prefills the whole conversation; for multi-turn use on an unpatched build, drop the `--spec-*` flags.
+- **A resumed turn can diverge from a cold run of the same prompt** (from one flipped greedy token onward) when it resumes from a checkpoint left by a prompt of a different length — a llama-server property, measured with and without a draft head (see [Prompt caching](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF#prompt-caching-with-mtp)).
+- **`llama-server`'s host-RAM prompt cache defaults to 8 GiB** (`-cram 8192`); each saved ~7K-token prompt costs ~0.7 GiB plus ~0.2 GiB per recurrent-state checkpoint. On a shared box, set `-cram` explicitly.
+
+## License and attribution
+
+Apache-2.0, inherited from the base model. Weights and architecture: **Agnes AI**. Calibration text:
+bartowski's `calibration_datav3`. ROCmFP4 / ROCmFPX: the ROCmFPX project. MTP prompt-cache patch, folding, imatrix,
+quantization and measurements: kingjones777.