README.md

---
license: apache-2.0
base_model: Agnes-AI/Agnes-3.0-Flash
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: gguf
tags:
  - gguf
  - llama.cpp
  - rocm
  - amd
  - rocmfp4
  - rocmfpx
  - strix-halo
  - amd-strix-halo
  - gfx1151
  - ryzen-ai-max
  - ryzen-ai-max-395
  - radeon-8060s
  - mtp
  - speculative-decoding
  - reasoning
  - multimodal
  - vision
  - agnes
  - qwen3.5
  - quantized
  - imatrix
---

# Agnes-3.0-Flash Preview — ROCmFP4 **imatrix** for AMD Strix Halo (gfx1151)

Importance-matrix-calibrated ROCmFP4 quantizations of
**[Agnes-AI/Agnes-3.0-Flash](https://huggingface.co/Agnes-AI/Agnes-3.0-Flash)** (Preview, 33B dense, 262,144-token context,
text + image). Companion to the standard build **[kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF)** — the same two 4-bit tiers,
same fold, same built-in MTP head, same vision projector and the same prompt-cache patch; the only difference is how each
4-bit block's scale was chosen.

## What the imatrix changes

ROCmFP4 has an importance-weighted quantizer path: with `--imatrix`, each block's scale is chosen by an exhaustive
search that minimises error **weighted by how strongly the calibration activations use each weight**, instead of the
unweighted default. It changes **which** scales are picked at the **same** bit width and tensor types — so it moves
quality, not size, and per-token compute is identical.

| | |
| --- | --- |
| calibration text | bartowski calibration_datav3.txt (the widely used community calibration set) |
| computed on | BF16 GGUF, 129 chunks × 512 tokens, ROCm0 |
| coverage | **558 entries = every matmul weight in the 72 trunk layers** (54 delta-rule layers × 8 + 18 global layers × 7); the output projection and the MTP block have none |
| file | `Agnes-3.0-Flash-Preview.imatrix` (GGUF format), sha256 `eb66b219d8bb057b78b933e5b78042fc47e028415d626690883c8c8f72707a09` |

## Measured effect

Quality is graded against the **BF16 GGUF** on a **held-out** corpus (wikitext-2 *test*, `-c 2048`, 40 chunks × 1,023 scored tokens each — the second half of every window, less its first token — = 40,920), never on the imatrix calibration text. **KLD** is the per-token KL divergence of each quant's next-token distribution from BF16's on the same tokens — far more sensitive than perplexity, whose error bar here is wider than most of the differences. The calibration text and the grading text are different corpora.

| Tier | Build | Size | KLD vs BF16 ↓ | Same top-1 ↑ | PPL (× BF16) | 99th-pct KLD |
| --- | --- | ---: | ---: | ---: | ---: | ---: |
| Q4_0_ROCMFP4_STRIX_LEAN | standard | 16.82 GiB | 0.0438 ± 0.0010 | 91.29 % | 6.4877 ± 0.0792 (×1.0250) | 0.4262 |
| Q4_0_ROCMFP4_STRIX_LEAN | **imatrix** | 16.82 GiB | 0.0387 ± 0.0010 | 91.52 % | 6.4269 ± 0.0777 (×1.0154) | 0.3863 |
| | *Δ imatrix* | | -11.6 % (3.5σ) | +0.23 pp | -0.9 % (0.5σ) | -9.3 % |
| Q4_0_ROCMFP4_COHERENT | standard | 17.77 GiB | 0.0385 ± 0.0009 | 91.39 % | 6.4290 ± 0.0780 (×1.0157) | 0.3765 |
| Q4_0_ROCMFP4_COHERENT | **imatrix** | 17.77 GiB | 0.0322 ± 0.0008 | 92.11 % | 6.4004 ± 0.0772 (×1.0112) | 0.3131 |
| | *Δ imatrix* | | -16.4 % (5.4σ) | +0.72 pp | -0.4 % (0.3σ) | -16.8 % |

σ = difference divided by the two runs' combined standard error. The two runs score the **same** tokens, so this is
conservative (paired noise is smaller).

**The imatrix measurably improves both tiers.** Mean KLD falls **-11.6 %** on STRIX_LEAN (3.5σ) and **-16.4 %** on COHERENT (5.4σ). The median moves further than the mean (-16.9 % / -18.3 %), so the gain is broad rather than a few repaired outliers, and the 99th-percentile tail also improves (-9.3 % / -16.8 %). Top-1 agreement with BF16 rises +0.23 / +0.72 percentage points.

Two practical consequences: **imatrix STRIX_LEAN matches standard COHERENT's quality within noise** (KLD 0.0387 vs 0.0385, 0.2σ; PPL ratio ×1.0154 vs ×1.0157) in a file 5.3 % smaller, and **imatrix COHERENT is the lowest-KLD 4-bit build of this model we measured** (KLD 0.0322).

**Where the quality numbers come from.** Measured directly on the files in this repo, against BF16 logits recomputed in the same session, and the STRIX_LEAN grade was run twice: all 40 per-chunk rows matched (fresh BF16 logits: `recipe/logs/R2_*`; grades: `R3_*`; repeat: `R4_*`). Today's 4-bit KLD values are 7.8–9.4 % higher than an earlier grading of the same weights on the same machine, before a reboot, while the 8-bit files reproduced their earlier output exactly; BF16's own logits also changed across that reboot, which is why every file here is graded against BF16 logits recomputed today. Those earlier numbers are not used. Grading again with a 16-thread CPU job running alongside (9.49 s per chunk, close to the earlier session's 9.66 s; today's quiet run took 7.53 s) gave today's rows exactly, so CPU load does not change the output.

## Which file should I use?

**Start with the imatrix `STRIX_LEAN`.** It matches the *standard* COHERENT's quality within noise (KLD 0.0387 vs 0.0385, 0.2σ) at STRIX_LEAN's size; with MTP, `STRIX_LEAN` decodes 13 % faster than `COHERENT` on ROCm0 (26.31 vs 23.30 tok/s; per-run ranges 25.56–26.42 and 23.29–23.66) and within 0.4 % of it on Vulkan0 (25.37 vs 25.46), and prefills 12 % faster than it on ROCm0 (260.8 vs 231.9 tok/s). Take the imatrix `COHERENT` when quality matters more than speed: it is the lowest-KLD 4-bit build we measured (0.0322, 17 % below the imatrix STRIX_LEAN). Decode speed differences under 8.3 % are treated as a tie: files that do identical work per token (each imatrix file and its standard twin) measured up to 8.3 % apart, because draft acceptance varies with the prompt. Prefill barely varies (twins within 2.0 %), so prefill gaps above 3.0 % are treated as real.

| File | ftype | Size | BPW | KLD vs BF16 ↓² | Same top-1 ↑ | PPL (× BF16) | TG MTP ROCm0 | TG MTP Vulkan0 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | 106 | 16.82 GiB | 4.42 | 0.0387 ± 0.0010 | 91.52 % | 6.4269 ± 0.0777 (×1.0154) | 26.31 | 25.37 |
| `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-COHERENT.gguf` | 102 | 17.77 GiB | 4.67 | 0.0322 ± 0.0008 | 92.11 % | 6.4004 ± 0.0772 (×1.0112) | 23.30 | 25.46 |
| *BF16 reference* | 32 | 60.85 GiB³ | 16 | 0 | 100 % | 6.3296 ± 0.0762¹ | — | — |

TG = decode tokens/s after a ~7.1K-token prompt, in-file MTP head, `--spec-draft-n-max 4`, default (non-strict) verification.
¹ The BF16 PPL shown is the paired base every "×" ratio is computed against (averaged over the same scored tokens in the KL-divergence runs). The standalone BF16 run's own summary line reads 6.3314 ± 0.0763.
² Quality columns: see *Where the quality numbers come from* below.
³ BF16 conversion including the MTP layer (the graded BF16 trunk without it is 60.00 GiB); not published.

The imatrix changes scale values, not tensor types or sizes: tensor by tensor, each imatrix file has the same names, types and byte sizes as its standard twin (the files differ by 224 / 224 bytes of header metadata), so the work per token is the same. Measured speed still differs because drafts were accepted at a different rate on the benchmark prompt — imatrix vs standard, n-max 4: STRIX_LEAN on ROCm0 26.31 vs 26.64 tok/s (acceptance 0.697 vs 0.724); STRIX_LEAN on Vulkan0 25.37 vs 27.31 tok/s (acceptance 0.684 vs 0.766); COHERENT on ROCm0 23.30 vs 24.76 tok/s (acceptance 0.618 vs 0.660); COHERENT on Vulkan0 25.46 vs 23.51 tok/s (acceptance 0.715 vs 0.653). The largest of these gaps, 8.3 %, is the spread both cards use to judge decode differences.

Full speed tables (depth check, strict mode, both backends, workload range), prompt-cache and tool-calling results are on
the [standard card](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF). The 8-bit tiers are not duplicated here; they are on the standard repo.

## Quick start

**1. Download**

```bash
hf download kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-imatrix-GGUF --local-dir ~/models/agnes
```

**2. Build `llama-server` with the prompt-cache patch** — ROCmFPX at the measured commit (ROCm and Vulkan
prerequisites: the project's [build guide](https://github.com/charlie12345/ROCmFPX/blob/d3ca53726109b46fd5cd6fa42f688c707079b44e/docs/build.md))

```bash
git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
git checkout d3ca53726109b46fd5cd6fa42f688c707079b44e
git apply ~/models/agnes/recipe/patches/mtp-prompt-cache-fix.patch
HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \
  -DGGML_HIP=ON -DGGML_VULKAN=ON -DGPU_TARGETS=gfx1151 \
  -DGGML_HIP_GRAPHS=ON -DGGML_HIP_NO_VMM=ON -DLLAMA_CURL=OFF
cmake --build build --target llama-server -j
```

(The CMake options of the measured build are listed in [Reproduction](#reproduction).)

`d3ca537` is also in the history of the official [ROCmFPX/ROCmFPX](https://github.com/ROCmFPX/ROCmFPX) repository; the
patch was verified against `d3ca537` only.

**3. Serve** — the MTP head is inside the model file, so there is no `--model-draft`:

Text and tool calls, with MTP:

```bash
env LD_LIBRARY_PATH=$PWD/build/bin:/opt/rocm/lib HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
build/bin/llama-server \
  -m ~/models/agnes/Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 4 --spec-draft-n-min 0 --spec-draft-p-min 0.0 \
  -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -dev ROCm0 \
  -c 65536 --host 127.0.0.1 --port 8080
```

Images (`--mmproj`), **without** MTP:

```bash
env LD_LIBRARY_PATH=$PWD/build/bin:/opt/rocm/lib HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
build/bin/llama-server \
  -m ~/models/agnes/Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf \
  --mmproj ~/models/agnes/mmproj-Agnes-3.0-Flash-Preview-BF16.gguf \
  -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -dev ROCm0 \
  -c 65536 --host 127.0.0.1 --port 8080
```

> ⛔ **Images:** with the MTP head loaded, image requests fail on this build ([measured](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF#vision)). For image input, drop the two `--spec-*` lines.

(`LD_LIBRARY_PATH` avoids a soname clash on machines that also have a Vulkan-only llama.cpp build.) The benchmarks used
the MTP command without `--mmproj`; the exact measured argv is in [Reproduction](#reproduction).

| Flag | Why |
| --- | --- |
| `--spec-type draft-mtp` | Uses the MTP layer inside the model file; no second file to download or place. |
| `--spec-draft-n-max 4` | llama.cpp's default is 16. On this model the measured optimum is about 4 — see [the table](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF#speculative-decoding-mtp). |
| `--spec-draft-p-min 0.0` | The default 0.75 gate throttles drafting. |
| `--jinja` | Already on by default in this build; keep it on — the thinking controls (`chat_template_kwargs`) and tool calling rely on the Jinja chat template. |
| `-fit off` | Autofit reads `MemAvailable` on integrated GPUs and can silently shrink context or push tensors to CPU. |
| `-cram <MiB>` | Not set above (default 8 GiB of host RAM for saved prompts). Set it on a shared box — see [Known issues](#known-issues-and-limits). |

> ⚠️ **On an unpatched build the in-file head still works, but prompt caching turns off while it is loaded** — every
> turn re-prefills the whole conversation ([measured](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF#prompt-caching-with-mtp)). For multi-turn / agentic use there,
> drop the two `--spec-*` lines.

Requires a llama.cpp build with ROCmFP4 / ROCmFPX tensor-type support; stock llama.cpp rejects these tensor types.

## Reasoning controls

⛔ **Do not pass `reasoning_effort: "high"`.**

The upstream model card lists `high` as the default effort. **The chat template shipped with the checkpoint does not
accept it.** It defaults to `xhigh` and accepts only `xhigh`, `medium`, `low`; anything else raises:

```
Unexpected reasoning effort high. Supported types are xhigh (default), medium, and low.
```

Verified by rendering the template with each value (`xhigh` / `medium` / `low` render; `high` raises; `medium` injects
no instruction at all). Use:

| goal | request field |
| --- | --- |
| default deep reasoning | nothing, or `"chat_template_kwargs": {"reasoning_effort": "xhigh"}` |
| shorter thinking | `"chat_template_kwargs": {"reasoning_effort": "low"}` |
| thinking off | `"chat_template_kwargs": {"enable_thinking": false}` |

With thinking on and a small `max_tokens`, the whole budget can go to reasoning and `content` comes back empty — raise
`max_tokens` before concluding the model is broken.

⚠️ **`preserve_thinking` defaults to true** in this template: earlier turns' reasoning is re-rendered into the prompt
(Qwen3.5's template drops it). Multi-turn and agentic contexts grow faster than you may expect; pass
`"preserve_thinking": false` in `chat_template_kwargs` to get the Qwen3.5 behaviour.

## MTP, prompt caching, tool calling, vision

Same architecture, template, in-file MTP block (258 MiB in STRIX_LEAN,
270 MiB in COHERENT) and projector as the standard build. The prompt-cache patch
is included here too (`recipe/patches/`). The tool-calling suite ran on this repo's STRIX_LEAN — see
[the standard card](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF#tool-calling).

## How the parallel FFN was handled

Agnes-3.0-Flash Preview is a Qwen3.5-style dense hybrid (72 layers: 54 gated-delta-rule + 18 global attention,
interval 4) with one addition: **every layer runs a second, narrower SwiGLU (width 2048) in parallel with the
main one (width 17408), and the two outputs are summed.** The reference implementation is literally

```python
y = down(act(gate(x)) * up(x)) + parallel_ffn(x)
```

Because it is a plain sum through the same activation, the branch folds **exactly** into one wider SwiGLU:
`gate`/`up` concatenated along the output dimension and `down` along the input dimension, giving a stock `qwen35`
MLP of width **19456**. That is the same transformation Agnes AI's own SGLang patch applies. The result loads on any
llama.cpp/ROCmFPX build with `qwen35` support — **no Agnes-specific patch** (some other GGUFs of this model keep the
branch as separate `ffn_*_par` tensors and require a patched fork).

Three details that are easy to get wrong:

1. **Rename `delta_attn.*` → `linear_attn.*`.** The converter reorders the 48 value heads into the tiled order ggml
   expects *only* for tensors named `linear_attn.*`. Skipping the rename produces a model that loads and runs and is wrong.
2. **The MTP layer has no parallel branch** (17408 wide), so its MLP is zero-padded to 19456. Zero
   SwiGLU rows contribute exactly zero.
3. Agnes's RMSNorm is one-centred, `x / rms(x) · (1 + w)`, so the converter's `+1` weight offset is correct; the gated
   delta-rule norm is a plain scale and is correctly left alone.

The fold boundary sits at column 17408 = 544 × 32 = 68 × 256, so no quantization block mixes main and parallel weights.

**Verification (`recipe/verify_fold.py`, which re-derives every tensor from the source independently): PASS.**
1305 output tensors, every one **bit-identical** to its source slice (216 folded,
3 MTP-padded, 1086 passed through), and the folded MLP reproduces the reference
forward pass to float32 rounding (max relative error: layer 0: 8.5e-10, layer 36: 5.4e-08, layer 71: 2.5e-08, MTP: 3.0e-08). The full log is in `recipe/verify_fold.log`.

## Quantization methodology

```bash
llama-imatrix -m Agnes-3.0-Flash-Preview-BF16.gguf -f calibration_datav3.txt -o Agnes-3.0-Flash-Preview.imatrix \
  -ngl 999 -c 512 -b 512 -fa on -dio -dev ROCm0
llama-quantize --imatrix Agnes-3.0-Flash-Preview.imatrix --output-tensor-type q6_K \
  Agnes-3.0-Flash-Preview-MTP-BF16.gguf Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0_ROCMFP4_STRIX_LEAN.gguf Q4_0_ROCMFP4_STRIX_LEAN 16
llama-quantize --imatrix Agnes-3.0-Flash-Preview.imatrix --output-tensor-type q6_K --token-embedding-type q6_K \
  Agnes-3.0-Flash-Preview-MTP-BF16.gguf Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0_ROCMFP4_COHERENT.gguf Q4_0_ROCMFP4_COHERENT 16
```

The imatrix was computed on the trunk-only BF16; the MTP layer is quantized from the BF16 that includes it
(`Agnes-3.0-Flash-Preview-MTP-BF16.gguf`), with the unweighted path for its tensors.

Receipts that the weighted path was actually taken, and that the shipped trunk is the graded one:

| File | `output.weight` | `token_embd.weight` | tensors | trunk vs trunk-only build | MTP block | imatrix entries | bytes differ from standard |
| --- | --- | --- | ---: | :---: | ---: | ---: | :---: |
| `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | Q6_K | Q5_K | 972 | identical (957 tensors) | 258 MiB | 558 | yes |
| `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-COHERENT.gguf` | Q6_K | Q6_K | 972 | identical (957 tensors) | 270 MiB | 558 | yes |

## Reproduction

```
server   : https://github.com/charlie12345/ROCmFPX @ d3ca53726109b46fd5cd6fa42f688c707079b44e
           + recipe/patches/mtp-prompt-cache-fix.patch (sha256 412a0173f60faa937018f0153efe9b5d9230c1606735502485e7e520fffb68dd), applied cleanly (git apply --check)
           built target llama-server with the same CMake options as below; sha256 14e9f1299240c891478c820b8c20811ca5721044d2e63e23799eaa9be26b30bd
tools    : same commit, unpatched (quantize / imatrix / perplexity; the patch touches only common/ and tools/server/)
           build dir /opt/llama-rocm/rocmfpx-724/build-hipvk, Release, Unix Makefiles, GGML_HIP=ON GGML_VULKAN=ON
           GGML_HIP_GRAPHS=ON GGML_HIP_NO_VMM=ON GGML_NATIVE=ON AMDGPU_TARGETS=gfx1151 LLAMA_CURL=OFF
           (this CMake forwards AMDGPU_TARGETS to GPU_TARGETS, the spelling used in Quick start)
           CMAKE_HIP_COMPILER=/opt/rocm-7.2.4/lib/llvm/bin/clang
           sha256 llama-quantize   1559189968a30781f837e4d25ca3cdf26c8eb0351023675406535eddcbe0db13
           sha256 llama-imatrix    bfdce5c21a01bdc8043d69335b74a7b68b8335ae989991a1e10e21d5faeaeb8e
           sha256 llama-perplexity c2175925bb0c61df89ff787845d8e03a35403f684be52719ac2987b8ca636a53
           sha256 llama-server     d077b98f3f04df4b482a5826c8922e39cda3aa8d2e2b2fbe1870f36d245cebe7   (unpatched; used for the unpatched control rows: prompt cache and vision)
source   : Agnes-AI/Agnes-3.0-Flash  revision 891ce4f9ffb89b22888aa7fcc2bb2f3618867684
           folded with recipe/fold_agnes.py (sha256 b1e6dbcf16bbac48fbc95b763efeb85efb21ce1d1fdfa825a6ae01187f044486)
model    : Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf
argv     : /opt/llama-rocm/rocmfpx-724-mtpcache/build-hipvk/bin/llama-server -m /mnt/models/agnes-3.0-flash/out-mtp-imat/Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0_ROCMFP4_STRIX_LEAN.gguf -dev ROCm0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-n-min 0 --spec-draft-p-min 0.0
env      : LD_LIBRARY_PATH=<patched build>/bin:/opt/rocm-7.2.4/lib
           HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1
box      : aimax — AMD Ryzen AI Max+ 395 / Radeon 8060S (gfx1151), 124 GiB, GTT 131072 MiB,
           kernel 6.17.6-061706-generic, ROCm 7.2.4
power    : no platform_profile interface on this box; cpufreq governor=performance, EPP=performance;
           GPU measured 84-85 W at 95-100% busy under load
protocol : ~7.1K-token prompt, 256 generated tokens, temp 0 / top_k 1, ignore_eos, cache_prompt false,
           1 warm-up + median of 3, no co-resident models (box iced)
measured : 2026-09-16, by the pipeline in recipe/ (every raw number in recipe/results_summary.json and recipe/raw/)
```

## Files

| File | Size | sha256 |
| --- | ---: | --- |
| `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | 16.82 GiB | `6aae32cf31652a509044288c31e2e38e5d4b3f8fb42e9531ef1498a48d1fc8cd` |
| `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-COHERENT.gguf` | 17.77 GiB | `643d7a6f601826c7ee2ff3cc2b8092ef971d1dec1e6f1fff6c36ab6c4d875f30` |
| `Agnes-3.0-Flash-Preview.imatrix` | 15.2 MiB | `eb66b219d8bb057b78b933e5b78042fc47e028415d626690883c8c8f72707a09` |
| `mmproj-Agnes-3.0-Flash-Preview-BF16.gguf` | 0.87 GiB | `d35b469bce95e80dfa5928e7547089ceefe8e9bb77452349a3af02c5679d7026` |

## Known issues and limits

- Calibration text is general-purpose English/code; an imatrix built from your own domain can do better on that domain.
- Graded on wikitext-2 *test*, a different corpus from the calibration text. Improvements on reasoning/code tasks were not measured.
- The imatrix covers the 72 trunk layers only. The MTP block (`blk.72.*`) and the output projection were quantized without importance weights — `llama-quantize` logs `did not find weights` for each — because the calibration forward pass never runs them (`recipe/logs/`).
- **Preview checkpoint.** Agnes AI's published benchmark results are for a different (production/API) checkpoint.
- **`reasoning_effort: "high"` raises** in the shipped chat template — see [Reasoning controls](#reasoning-controls).
- **Prompt caching with MTP needs the included server patch.** Without it the in-file head still drafts, but every turn re-prefills the whole conversation; for multi-turn use on an unpatched build, drop the `--spec-*` flags.
- **A resumed turn can diverge from a cold run of the same prompt** (from one flipped greedy token onward) when it resumes from a checkpoint left by a prompt of a different length — a llama-server property, measured with and without a draft head (see [Prompt caching](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF#prompt-caching-with-mtp)).
- **`llama-server`'s host-RAM prompt cache defaults to 8 GiB** (`-cram 8192`); each saved ~7K-token prompt costs ~0.7 GiB plus ~0.2 GiB per recurrent-state checkpoint. On a shared box, set `-cram` explicitly.

## License and attribution

Apache-2.0, inherited from the base model. Weights and architecture: **Agnes AI**. Calibration text:
bartowski's `calibration_datav3`. ROCmFP4 / ROCmFPX: the ROCmFPX project. MTP prompt-cache patch, folding, imatrix,
quantization and measurements: kingjones777.