--- license: apache-2.0 base_model: Agnes-AI/Agnes-3.0-Flash base_model_relation: quantized pipeline_tag: image-text-to-text library_name: gguf tags: - gguf - llama.cpp - rocm - amd - rocmfp4 - rocmfpx - strix-halo - amd-strix-halo - gfx1151 - ryzen-ai-max - ryzen-ai-max-395 - radeon-8060s - mtp - speculative-decoding - reasoning - multimodal - vision - agnes - qwen3.5 - quantized - imatrix --- # Agnes-3.0-Flash Preview — ROCmFP4 **imatrix** for AMD Strix Halo (gfx1151) Importance-matrix-calibrated ROCmFP4 quantizations of **[Agnes-AI/Agnes-3.0-Flash](https://huggingface.co/Agnes-AI/Agnes-3.0-Flash)** (Preview, 33B dense, 262,144-token context, text + image). Companion to the standard build **[kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF)** — the same two 4-bit tiers, same fold, same built-in MTP head, same vision projector and the same prompt-cache patch; the only difference is how each 4-bit block's scale was chosen. ## What the imatrix changes ROCmFP4 has an importance-weighted quantizer path: with `--imatrix`, each block's scale is chosen by an exhaustive search that minimises error **weighted by how strongly the calibration activations use each weight**, instead of the unweighted default. It changes **which** scales are picked at the **same** bit width and tensor types — so it moves quality, not size, and per-token compute is identical. | | | | --- | --- | | calibration text | bartowski calibration_datav3.txt (the widely used community calibration set) | | computed on | BF16 GGUF, 129 chunks × 512 tokens, ROCm0 | | coverage | **558 entries = every matmul weight in the 72 trunk layers** (54 delta-rule layers × 8 + 18 global layers × 7); the output projection and the MTP block have none | | file | `Agnes-3.0-Flash-Preview.imatrix` (GGUF format), sha256 `eb66b219d8bb057b78b933e5b78042fc47e028415d626690883c8c8f72707a09` | ## Measured effect Quality is graded against the **BF16 GGUF** on a **held-out** corpus (wikitext-2 *test*, `-c 2048`, 40 chunks × 1,023 scored tokens each — the second half of every window, less its first token — = 40,920), never on the imatrix calibration text. **KLD** is the per-token KL divergence of each quant's next-token distribution from BF16's on the same tokens — far more sensitive than perplexity, whose error bar here is wider than most of the differences. The calibration text and the grading text are different corpora. | Tier | Build | Size | KLD vs BF16 ↓ | Same top-1 ↑ | PPL (× BF16) | 99th-pct KLD | | --- | --- | ---: | ---: | ---: | ---: | ---: | | Q4_0_ROCMFP4_STRIX_LEAN | standard | 16.82 GiB | 0.0438 ± 0.0010 | 91.29 % | 6.4877 ± 0.0792 (×1.0250) | 0.4262 | | Q4_0_ROCMFP4_STRIX_LEAN | **imatrix** | 16.82 GiB | 0.0387 ± 0.0010 | 91.52 % | 6.4269 ± 0.0777 (×1.0154) | 0.3863 | | | *Δ imatrix* | | -11.6 % (3.5σ) | +0.23 pp | -0.9 % (0.5σ) | -9.3 % | | Q4_0_ROCMFP4_COHERENT | standard | 17.77 GiB | 0.0385 ± 0.0009 | 91.39 % | 6.4290 ± 0.0780 (×1.0157) | 0.3765 | | Q4_0_ROCMFP4_COHERENT | **imatrix** | 17.77 GiB | 0.0322 ± 0.0008 | 92.11 % | 6.4004 ± 0.0772 (×1.0112) | 0.3131 | | | *Δ imatrix* | | -16.4 % (5.4σ) | +0.72 pp | -0.4 % (0.3σ) | -16.8 % | σ = difference divided by the two runs' combined standard error. The two runs score the **same** tokens, so this is conservative (paired noise is smaller). **The imatrix measurably improves both tiers.** Mean KLD falls **-11.6 %** on STRIX_LEAN (3.5σ) and **-16.4 %** on COHERENT (5.4σ). The median moves further than the mean (-16.9 % / -18.3 %), so the gain is broad rather than a few repaired outliers, and the 99th-percentile tail also improves (-9.3 % / -16.8 %). Top-1 agreement with BF16 rises +0.23 / +0.72 percentage points. Two practical consequences: **imatrix STRIX_LEAN matches standard COHERENT's quality within noise** (KLD 0.0387 vs 0.0385, 0.2σ; PPL ratio ×1.0154 vs ×1.0157) in a file 5.3 % smaller, and **imatrix COHERENT is the lowest-KLD 4-bit build of this model we measured** (KLD 0.0322). **Where the quality numbers come from.** Measured directly on the files in this repo, against BF16 logits recomputed in the same session, and the STRIX_LEAN grade was run twice: all 40 per-chunk rows matched (fresh BF16 logits: `recipe/logs/R2_*`; grades: `R3_*`; repeat: `R4_*`). Today's 4-bit KLD values are 7.8–9.4 % higher than an earlier grading of the same weights on the same machine, before a reboot, while the 8-bit files reproduced their earlier output exactly; BF16's own logits also changed across that reboot, which is why every file here is graded against BF16 logits recomputed today. Those earlier numbers are not used. Grading again with a 16-thread CPU job running alongside (9.49 s per chunk, close to the earlier session's 9.66 s; today's quiet run took 7.53 s) gave today's rows exactly, so CPU load does not change the output. ## Which file should I use? **Start with the imatrix `STRIX_LEAN`.** It matches the *standard* COHERENT's quality within noise (KLD 0.0387 vs 0.0385, 0.2σ) at STRIX_LEAN's size; with MTP, `STRIX_LEAN` decodes 13 % faster than `COHERENT` on ROCm0 (26.31 vs 23.30 tok/s; per-run ranges 25.56–26.42 and 23.29–23.66) and within 0.4 % of it on Vulkan0 (25.37 vs 25.46), and prefills 12 % faster than it on ROCm0 (260.8 vs 231.9 tok/s). Take the imatrix `COHERENT` when quality matters more than speed: it is the lowest-KLD 4-bit build we measured (0.0322, 17 % below the imatrix STRIX_LEAN). Decode speed differences under 8.3 % are treated as a tie: files that do identical work per token (each imatrix file and its standard twin) measured up to 8.3 % apart, because draft acceptance varies with the prompt. Prefill barely varies (twins within 2.0 %), so prefill gaps above 3.0 % are treated as real. | File | ftype | Size | BPW | KLD vs BF16 ↓² | Same top-1 ↑ | PPL (× BF16) | TG MTP ROCm0 | TG MTP Vulkan0 | | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | | `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | 106 | 16.82 GiB | 4.42 | 0.0387 ± 0.0010 | 91.52 % | 6.4269 ± 0.0777 (×1.0154) | 26.31 | 25.37 | | `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-COHERENT.gguf` | 102 | 17.77 GiB | 4.67 | 0.0322 ± 0.0008 | 92.11 % | 6.4004 ± 0.0772 (×1.0112) | 23.30 | 25.46 | | *BF16 reference* | 32 | 60.85 GiB³ | 16 | 0 | 100 % | 6.3296 ± 0.0762¹ | — | — | TG = decode tokens/s after a ~7.1K-token prompt, in-file MTP head, `--spec-draft-n-max 4`, default (non-strict) verification. ¹ The BF16 PPL shown is the paired base every "×" ratio is computed against (averaged over the same scored tokens in the KL-divergence runs). The standalone BF16 run's own summary line reads 6.3314 ± 0.0763. ² Quality columns: see *Where the quality numbers come from* below. ³ BF16 conversion including the MTP layer (the graded BF16 trunk without it is 60.00 GiB); not published. The imatrix changes scale values, not tensor types or sizes: tensor by tensor, each imatrix file has the same names, types and byte sizes as its standard twin (the files differ by 224 / 224 bytes of header metadata), so the work per token is the same. Measured speed still differs because drafts were accepted at a different rate on the benchmark prompt — imatrix vs standard, n-max 4: STRIX_LEAN on ROCm0 26.31 vs 26.64 tok/s (acceptance 0.697 vs 0.724); STRIX_LEAN on Vulkan0 25.37 vs 27.31 tok/s (acceptance 0.684 vs 0.766); COHERENT on ROCm0 23.30 vs 24.76 tok/s (acceptance 0.618 vs 0.660); COHERENT on Vulkan0 25.46 vs 23.51 tok/s (acceptance 0.715 vs 0.653). The largest of these gaps, 8.3 %, is the spread both cards use to judge decode differences. Full speed tables (depth check, strict mode, both backends, workload range), prompt-cache and tool-calling results are on the [standard card](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF). The 8-bit tiers are not duplicated here; they are on the standard repo. ## Quick start **1. Download** ```bash hf download kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-imatrix-GGUF --local-dir ~/models/agnes ``` **2. Build `llama-server` with the prompt-cache patch** — ROCmFPX at the measured commit (ROCm and Vulkan prerequisites: the project's [build guide](https://github.com/charlie12345/ROCmFPX/blob/d3ca53726109b46fd5cd6fa42f688c707079b44e/docs/build.md)) ```bash git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX git checkout d3ca53726109b46fd5cd6fa42f688c707079b44e git apply ~/models/agnes/recipe/patches/mtp-prompt-cache-fix.patch HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \ cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \ -DGGML_HIP=ON -DGGML_VULKAN=ON -DGPU_TARGETS=gfx1151 \ -DGGML_HIP_GRAPHS=ON -DGGML_HIP_NO_VMM=ON -DLLAMA_CURL=OFF cmake --build build --target llama-server -j ``` (The CMake options of the measured build are listed in [Reproduction](#reproduction).) `d3ca537` is also in the history of the official [ROCmFPX/ROCmFPX](https://github.com/ROCmFPX/ROCmFPX) repository; the patch was verified against `d3ca537` only. **3. Serve** — the MTP head is inside the model file, so there is no `--model-draft`: Text and tool calls, with MTP: ```bash env LD_LIBRARY_PATH=$PWD/build/bin:/opt/rocm/lib HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \ build/bin/llama-server \ -m ~/models/agnes/Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf \ --spec-type draft-mtp \ --spec-draft-n-max 4 --spec-draft-n-min 0 --spec-draft-p-min 0.0 \ -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -dev ROCm0 \ -c 65536 --host 127.0.0.1 --port 8080 ``` Images (`--mmproj`), **without** MTP: ```bash env LD_LIBRARY_PATH=$PWD/build/bin:/opt/rocm/lib HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \ build/bin/llama-server \ -m ~/models/agnes/Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf \ --mmproj ~/models/agnes/mmproj-Agnes-3.0-Flash-Preview-BF16.gguf \ -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -dev ROCm0 \ -c 65536 --host 127.0.0.1 --port 8080 ``` > ⛔ **Images:** with the MTP head loaded, image requests fail on this build ([measured](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF#vision)). For image input, drop the two `--spec-*` lines. (`LD_LIBRARY_PATH` avoids a soname clash on machines that also have a Vulkan-only llama.cpp build.) The benchmarks used the MTP command without `--mmproj`; the exact measured argv is in [Reproduction](#reproduction). | Flag | Why | | --- | --- | | `--spec-type draft-mtp` | Uses the MTP layer inside the model file; no second file to download or place. | | `--spec-draft-n-max 4` | llama.cpp's default is 16. On this model the measured optimum is about 4 — see [the table](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF#speculative-decoding-mtp). | | `--spec-draft-p-min 0.0` | The default 0.75 gate throttles drafting. | | `--jinja` | Already on by default in this build; keep it on — the thinking controls (`chat_template_kwargs`) and tool calling rely on the Jinja chat template. | | `-fit off` | Autofit reads `MemAvailable` on integrated GPUs and can silently shrink context or push tensors to CPU. | | `-cram ` | Not set above (default 8 GiB of host RAM for saved prompts). Set it on a shared box — see [Known issues](#known-issues-and-limits). | > ⚠️ **On an unpatched build the in-file head still works, but prompt caching turns off while it is loaded** — every > turn re-prefills the whole conversation ([measured](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF#prompt-caching-with-mtp)). For multi-turn / agentic use there, > drop the two `--spec-*` lines. Requires a llama.cpp build with ROCmFP4 / ROCmFPX tensor-type support; stock llama.cpp rejects these tensor types. ## Reasoning controls ⛔ **Do not pass `reasoning_effort: "high"`.** The upstream model card lists `high` as the default effort. **The chat template shipped with the checkpoint does not accept it.** It defaults to `xhigh` and accepts only `xhigh`, `medium`, `low`; anything else raises: ``` Unexpected reasoning effort high. Supported types are xhigh (default), medium, and low. ``` Verified by rendering the template with each value (`xhigh` / `medium` / `low` render; `high` raises; `medium` injects no instruction at all). Use: | goal | request field | | --- | --- | | default deep reasoning | nothing, or `"chat_template_kwargs": {"reasoning_effort": "xhigh"}` | | shorter thinking | `"chat_template_kwargs": {"reasoning_effort": "low"}` | | thinking off | `"chat_template_kwargs": {"enable_thinking": false}` | With thinking on and a small `max_tokens`, the whole budget can go to reasoning and `content` comes back empty — raise `max_tokens` before concluding the model is broken. ⚠️ **`preserve_thinking` defaults to true** in this template: earlier turns' reasoning is re-rendered into the prompt (Qwen3.5's template drops it). Multi-turn and agentic contexts grow faster than you may expect; pass `"preserve_thinking": false` in `chat_template_kwargs` to get the Qwen3.5 behaviour. ## MTP, prompt caching, tool calling, vision Same architecture, template, in-file MTP block (258 MiB in STRIX_LEAN, 270 MiB in COHERENT) and projector as the standard build. The prompt-cache patch is included here too (`recipe/patches/`). The tool-calling suite ran on this repo's STRIX_LEAN — see [the standard card](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF#tool-calling). ## How the parallel FFN was handled Agnes-3.0-Flash Preview is a Qwen3.5-style dense hybrid (72 layers: 54 gated-delta-rule + 18 global attention, interval 4) with one addition: **every layer runs a second, narrower SwiGLU (width 2048) in parallel with the main one (width 17408), and the two outputs are summed.** The reference implementation is literally ```python y = down(act(gate(x)) * up(x)) + parallel_ffn(x) ``` Because it is a plain sum through the same activation, the branch folds **exactly** into one wider SwiGLU: `gate`/`up` concatenated along the output dimension and `down` along the input dimension, giving a stock `qwen35` MLP of width **19456**. That is the same transformation Agnes AI's own SGLang patch applies. The result loads on any llama.cpp/ROCmFPX build with `qwen35` support — **no Agnes-specific patch** (some other GGUFs of this model keep the branch as separate `ffn_*_par` tensors and require a patched fork). Three details that are easy to get wrong: 1. **Rename `delta_attn.*` → `linear_attn.*`.** The converter reorders the 48 value heads into the tiled order ggml expects *only* for tensors named `linear_attn.*`. Skipping the rename produces a model that loads and runs and is wrong. 2. **The MTP layer has no parallel branch** (17408 wide), so its MLP is zero-padded to 19456. Zero SwiGLU rows contribute exactly zero. 3. Agnes's RMSNorm is one-centred, `x / rms(x) · (1 + w)`, so the converter's `+1` weight offset is correct; the gated delta-rule norm is a plain scale and is correctly left alone. The fold boundary sits at column 17408 = 544 × 32 = 68 × 256, so no quantization block mixes main and parallel weights. **Verification (`recipe/verify_fold.py`, which re-derives every tensor from the source independently): PASS.** 1305 output tensors, every one **bit-identical** to its source slice (216 folded, 3 MTP-padded, 1086 passed through), and the folded MLP reproduces the reference forward pass to float32 rounding (max relative error: layer 0: 8.5e-10, layer 36: 5.4e-08, layer 71: 2.5e-08, MTP: 3.0e-08). The full log is in `recipe/verify_fold.log`. ## Quantization methodology ```bash llama-imatrix -m Agnes-3.0-Flash-Preview-BF16.gguf -f calibration_datav3.txt -o Agnes-3.0-Flash-Preview.imatrix \ -ngl 999 -c 512 -b 512 -fa on -dio -dev ROCm0 llama-quantize --imatrix Agnes-3.0-Flash-Preview.imatrix --output-tensor-type q6_K \ Agnes-3.0-Flash-Preview-MTP-BF16.gguf Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0_ROCMFP4_STRIX_LEAN.gguf Q4_0_ROCMFP4_STRIX_LEAN 16 llama-quantize --imatrix Agnes-3.0-Flash-Preview.imatrix --output-tensor-type q6_K --token-embedding-type q6_K \ Agnes-3.0-Flash-Preview-MTP-BF16.gguf Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0_ROCMFP4_COHERENT.gguf Q4_0_ROCMFP4_COHERENT 16 ``` The imatrix was computed on the trunk-only BF16; the MTP layer is quantized from the BF16 that includes it (`Agnes-3.0-Flash-Preview-MTP-BF16.gguf`), with the unweighted path for its tensors. Receipts that the weighted path was actually taken, and that the shipped trunk is the graded one: | File | `output.weight` | `token_embd.weight` | tensors | trunk vs trunk-only build | MTP block | imatrix entries | bytes differ from standard | | --- | --- | --- | ---: | :---: | ---: | ---: | :---: | | `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | Q6_K | Q5_K | 972 | identical (957 tensors) | 258 MiB | 558 | yes | | `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-COHERENT.gguf` | Q6_K | Q6_K | 972 | identical (957 tensors) | 270 MiB | 558 | yes | ## Reproduction ``` server : https://github.com/charlie12345/ROCmFPX @ d3ca53726109b46fd5cd6fa42f688c707079b44e + recipe/patches/mtp-prompt-cache-fix.patch (sha256 412a0173f60faa937018f0153efe9b5d9230c1606735502485e7e520fffb68dd), applied cleanly (git apply --check) built target llama-server with the same CMake options as below; sha256 14e9f1299240c891478c820b8c20811ca5721044d2e63e23799eaa9be26b30bd tools : same commit, unpatched (quantize / imatrix / perplexity; the patch touches only common/ and tools/server/) build dir /opt/llama-rocm/rocmfpx-724/build-hipvk, Release, Unix Makefiles, GGML_HIP=ON GGML_VULKAN=ON GGML_HIP_GRAPHS=ON GGML_HIP_NO_VMM=ON GGML_NATIVE=ON AMDGPU_TARGETS=gfx1151 LLAMA_CURL=OFF (this CMake forwards AMDGPU_TARGETS to GPU_TARGETS, the spelling used in Quick start) CMAKE_HIP_COMPILER=/opt/rocm-7.2.4/lib/llvm/bin/clang sha256 llama-quantize 1559189968a30781f837e4d25ca3cdf26c8eb0351023675406535eddcbe0db13 sha256 llama-imatrix bfdce5c21a01bdc8043d69335b74a7b68b8335ae989991a1e10e21d5faeaeb8e sha256 llama-perplexity c2175925bb0c61df89ff787845d8e03a35403f684be52719ac2987b8ca636a53 sha256 llama-server d077b98f3f04df4b482a5826c8922e39cda3aa8d2e2b2fbe1870f36d245cebe7 (unpatched; used for the unpatched control rows: prompt cache and vision) source : Agnes-AI/Agnes-3.0-Flash revision 891ce4f9ffb89b22888aa7fcc2bb2f3618867684 folded with recipe/fold_agnes.py (sha256 b1e6dbcf16bbac48fbc95b763efeb85efb21ce1d1fdfa825a6ae01187f044486) model : Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf argv : /opt/llama-rocm/rocmfpx-724-mtpcache/build-hipvk/bin/llama-server -m /mnt/models/agnes-3.0-flash/out-mtp-imat/Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0_ROCMFP4_STRIX_LEAN.gguf -dev ROCm0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-n-min 0 --spec-draft-p-min 0.0 env : LD_LIBRARY_PATH=/bin:/opt/rocm-7.2.4/lib HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 box : aimax — AMD Ryzen AI Max+ 395 / Radeon 8060S (gfx1151), 124 GiB, GTT 131072 MiB, kernel 6.17.6-061706-generic, ROCm 7.2.4 power : no platform_profile interface on this box; cpufreq governor=performance, EPP=performance; GPU measured 84-85 W at 95-100% busy under load protocol : ~7.1K-token prompt, 256 generated tokens, temp 0 / top_k 1, ignore_eos, cache_prompt false, 1 warm-up + median of 3, no co-resident models (box iced) measured : 2026-09-16, by the pipeline in recipe/ (every raw number in recipe/results_summary.json and recipe/raw/) ``` ## Files | File | Size | sha256 | | --- | ---: | --- | | `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | 16.82 GiB | `6aae32cf31652a509044288c31e2e38e5d4b3f8fb42e9531ef1498a48d1fc8cd` | | `Agnes-3.0-Flash-Preview-MTP-imatrix-Q4_0-ROCmFP4-COHERENT.gguf` | 17.77 GiB | `643d7a6f601826c7ee2ff3cc2b8092ef971d1dec1e6f1fff6c36ab6c4d875f30` | | `Agnes-3.0-Flash-Preview.imatrix` | 15.2 MiB | `eb66b219d8bb057b78b933e5b78042fc47e028415d626690883c8c8f72707a09` | | `mmproj-Agnes-3.0-Flash-Preview-BF16.gguf` | 0.87 GiB | `d35b469bce95e80dfa5928e7547089ceefe8e9bb77452349a3af02c5679d7026` | ## Known issues and limits - Calibration text is general-purpose English/code; an imatrix built from your own domain can do better on that domain. - Graded on wikitext-2 *test*, a different corpus from the calibration text. Improvements on reasoning/code tasks were not measured. - The imatrix covers the 72 trunk layers only. The MTP block (`blk.72.*`) and the output projection were quantized without importance weights — `llama-quantize` logs `did not find weights` for each — because the calibration forward pass never runs them (`recipe/logs/`). - **Preview checkpoint.** Agnes AI's published benchmark results are for a different (production/API) checkpoint. - **`reasoning_effort: "high"` raises** in the shipped chat template — see [Reasoning controls](#reasoning-controls). - **Prompt caching with MTP needs the included server patch.** Without it the in-file head still drafts, but every turn re-prefills the whole conversation; for multi-turn use on an unpatched build, drop the `--spec-*` flags. - **A resumed turn can diverge from a cold run of the same prompt** (from one flipped greedy token onward) when it resumes from a checkpoint left by a prompt of a different length — a llama-server property, measured with and without a draft head (see [Prompt caching](https://huggingface.co/kingjones777/Agnes-3.0-Flash-Preview-MTP-ROCmFP4-GGUF#prompt-caching-with-mtp)). - **`llama-server`'s host-RAM prompt cache defaults to 8 GiB** (`-cram 8192`); each saved ~7K-token prompt costs ~0.7 GiB plus ~0.2 GiB per recurrent-state checkpoint. On a shared box, set `-cram` explicitly. ## License and attribution Apache-2.0, inherited from the base model. Weights and architecture: **Agnes AI**. Calibration text: bartowski's `calibration_datav3`. ROCmFP4 / ROCmFPX: the ROCmFPX project. MTP prompt-cache patch, folding, imatrix, quantization and measurements: kingjones777.