---
license: apache-2.0
base_model: nex-agi/Nex-N2.5-mini
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: gguf
tags:
  - gguf
  - llama.cpp
  - rocm
  - amd
  - rocmfp4
  - rocmfpx
  - strix-halo
  - amd-strix-halo
  - gfx1151
  - ryzen-ai-max
  - ryzen-ai-max-395
  - radeon-8060s
  - moe
  - reasoning
  - multimodal
  - vision
  - nex
  - qwen3.5
  - quantized
---

# Nex-N2.5-mini — ROCmFP4 for AMD Strix Halo (gfx1151)

ROCmFP4 / ROCmFPX quantizations of **[nex-agi/Nex-N2.5-mini](https://huggingface.co/nex-agi/Nex-N2.5-mini)** —
35,107,181,936 parameters (BF16), 40-layer Qwen3.5 MoE (30 Gated DeltaNet linear-attention + 10 full-attention layers), 256 routed experts / 8 active, 262,144-token context, text + image — built and measured on an AMD Ryzen AI
Max+ 395 (Radeon 8060S, `gfx1151`). Upstream publishes no GGUF.

- Another public ROCmFP4 build of this model exists — [julianmb/Nex-N2.5-mini-ROCmFP4-GGUF](https://huggingface.co/julianmb/Nex-N2.5-mini-ROCmFP4-GGUF): its `Nex-N2.5-mini-ROCmFP4-STRIX_LEAN.gguf` stores `output.weight` as `Q4_0_ROCMFP4_FAST` and carries no imatrix metadata. Every tier here keeps `output.weight` at `Q6_K`, and the imatrix builds are a separate repo.
- **Vision projector included.**
- **No MTP head.** `mtp_num_hidden_layers: 1` is declared in `config.json`, but the checkpoint ships no `mtp.*` weights (the converted BF16 GGUF reads back 0 `nextn` tensors). There is no speculative
  decoding on these files.
- Importance-matrix builds of the same three 4-bit tiers: **[kingjones777/Nex-N2.5-mini-ROCmFP4-imatrix-GGUF](https://huggingface.co/kingjones777/Nex-N2.5-mini-ROCmFP4-imatrix-GGUF)**.

## Which file should I use?

Ryzen AI Max+ 395 (MAX-1), ROCm 7.2.4, unpatched `llama-server` at `d3ca537` (see [Quick start](#quick-start)), `-c 65536`, one request at a time (`--parallel 1`), greedy (`temp 0`, `top_k 1`), `ignore_eos` so every arm generates exactly 256 tokens after a code prompt of 7,094–7,102 tokens (the first 30,000 characters of `convert_hf_to_gguf.py` plus an instruction), a unique nonce per request and `cache_prompt: false` (`cache_n = 0` asserted on every timed request), 1 warm-up then the median of 3. Decode numbers are the server's own `predicted_per_second`. Box iced: no other model loaded.

| File | ftype | Size | BPW⁴ | KLD vs BF16 ↓² | Same top-1 ↑ | PPL (× BF16) | TG ROCm0 | TG Vulkan0 | PP ROCm0 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| `Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | 106 | 17.46 GiB | 4.32 | 0.1044 ± 0.0014 | 86.66 % | 6.4740 ± 0.0798 (×1.0393) | 63.94 | 68.16 | 1158 |
| `Nex-N2.5-mini-Q4_0-ROCmFP4-COHERENT.gguf` | 102 | 18.48 GiB | 4.58 | 0.0971 ± 0.0013 | 87.29 % | 6.5617 ± 0.0812 (×1.0534) | 61.74 | 68.00 | 1187 |
| `Nex-N2.5-mini-Q4_0-ROCmFP4-FAST.gguf` | 103 | 17.37 GiB | 4.30 | 0.1088 ± 0.0014 | 86.41 % | 6.5498 ± 0.0809 (×1.0515) | 63.31 | 68.62 | 1157 |
| *BF16 reference* | 32 | 64.61 GiB³ | 16.01 | 0 | 100 % | 6.2290 ± 0.0753¹ | — | — | — |

TG = decode tokens/s after the 7,094–7,102-token code prompt, no draft head. PP = prefill tokens/s on ROCm0.
¹ The BF16 PPL shown is the paired base every "×" ratio is computed against (averaged over the same scored tokens in the KL-divergence runs). The standalone BF16 run's own summary line reads 6.2303 ± 0.0754.
² Quality columns: see *Where the quality numbers come from* below.
³ BF16 conversion of the checkpoint; not published.
⁴ BPW as printed by `llama-quantize`: bits per weight over the 34,660,610,688 weights in each GGUF. The 35,107,181,936-parameter count above also includes the 446,571,248-weight vision tower, which ships in the projector file.

**Start with `STRIX_LEAN`; take `COHERENT` if quality matters more than speed.** `STRIX_LEAN` decodes 3.6 % faster than `COHERENT` on ROCm0 (63.94 vs 61.74 tok/s; per-run ranges 63.56–63.96 and 61.74–61.80) and within 0.2 % of it on Vulkan0 (68.16 vs 68.00), and prefills within 2.4 % of it on ROCm0 (1158 vs 1187 tok/s). `COHERENT`'s KLD is 7.1 % lower (3.8σ) for 1051 MiB more.

`FAST` does not buy a clear speed gain here: `FAST` decodes within 1.0 % of `STRIX_LEAN` on ROCm0 (63.31 vs 63.94 tok/s; per-run ranges 63.14–63.38 and 63.56–63.96) and within 0.7 % of it on Vulkan0 (68.62 vs 68.16), and prefills within 0.1 % of it on ROCm0 (1157 vs 1158 tok/s); its KLD is higher than `STRIX_LEAN`'s (0.1088 vs 0.1044, +4.2 %, 2.2σ).

KLD order (lower is closer to BF16): `COHERENT` 0.0971, `STRIX_LEAN` 0.1044 (+7.6 % vs `COHERENT`), `FAST` 0.1088 (+12.1 % vs `COHERENT`). `STRIX_LEAN` is 1051 MiB smaller than `COHERENT`; `FAST` is 91 MiB smaller than `STRIX_LEAN`. Speed gaps below 3.0 % (decode) and 3.0 % (prefill) are called a tie: the larger of 3 % and the widest gap measured between files that do identical work per token (each imatrix file and its standard twin: decode 1.0 %, prefill 1.1 %).

**The [imatrix build](https://huggingface.co/kingjones777/Nex-N2.5-mini-ROCmFP4-imatrix-GGUF) is measurably closer to BF16 at the same size:** `STRIX_LEAN` 0.1044 → 0.0852 (-18.5 %, 10.0σ); `COHERENT` 0.0971 → 0.0769 (-20.7 %, 11.3σ); `FAST` 0.1088 → 0.0890 (-18.2 %, 10.2σ).

Quality is graded against the **BF16 GGUF** (reference logits computed on the CPU) on a **held-out** corpus (wikitext-2 *test*, `-c 2048`, 40 chunks × 1,023 scored tokens each — the second half of every window, less its first token — = 40,920), never on the imatrix calibration text. **KLD** is the per-token KL divergence of each quant's next-token distribution from BF16's on the same tokens — far more sensitive than perplexity.

**Where the quality numbers come from.** Measured directly on these files, against BF16 logits computed **on the CPU** in the same session (first-window perplexity 5.6964; Vulkan0 gave 5.6953 for the same window). Every file was graded on ROCm0 (the columns above) and again on Vulkan0. The STRIX_LEAN grade was run twice on each backend and every per-chunk row matched: 40 of 40 on ROCm0, 40 of 40 on Vulkan0. Why not the GPU for the reference: see [Known issues](#known-issues-and-limits).

**Same files, same reference, graded on each backend** — KLD ROCm0 / Vulkan0: STRIX_LEAN 0.1044 / 0.1044 (0.0 %, 0.0σ); COHERENT 0.0971 / 0.0972 (+0.1 %, 0.0σ); FAST 0.1088 / 0.1088 (0.0 %, 0.0σ); imatrix STRIX_LEAN 0.0852 / 0.0836 (-1.9 %, 0.9σ); imatrix COHERENT 0.0769 / 0.0768 (-0.1 %, 0.1σ); imatrix FAST 0.0890 / 0.0891 (+0.2 %, 0.1σ). The two backends agree within noise on every file.

## Quick start

**1. Download**

```bash
hf download kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF --local-dir ~/models/nex
```

**2. Build `llama-server`** — ROCmFPX at the measured commit (ROCm and Vulkan
prerequisites: the project's [build guide](https://github.com/charlie12345/ROCmFPX/blob/d3ca537/docs/build.md)). No patch.

```bash
git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
git checkout d3ca537
HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \
  -DGGML_HIP=ON -DGGML_VULKAN=ON -DGPU_TARGETS=gfx1151 \
  -DGGML_HIP_GRAPHS=ON -DGGML_HIP_NO_VMM=ON -DLLAMA_CURL=OFF
cmake --build build --target llama-server -j
```

(The CMake options of the measured build are listed in [Reproduction](#reproduction).)

`d3ca537` is also in the history of the official [ROCmFPX/ROCmFPX](https://github.com/ROCmFPX/ROCmFPX) repository.

**3. Serve**

```bash
env LD_LIBRARY_PATH=$PWD/build/bin:/opt/rocm/lib HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
build/bin/llama-server \
  -m ~/models/nex/Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf \
  --mmproj ~/models/nex/mmproj-Nex-N2.5-mini-BF16.gguf \
  --chat-template-file ~/models/nex/chat_template_enable_thinking.jinja --reasoning off \
  -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -dev ROCm0 \
  -c 65536 --host 127.0.0.1 --port 8080
```

(`LD_LIBRARY_PATH` avoids a soname clash on machines that also have a Vulkan-only llama.cpp build.) The exact measured
argv is in [Reproduction](#reproduction).

**4. Call** — upstream sampling. Thinking is off unless the request sets `"enable_thinking": true` (as here; drop that
line for a direct answer):

```bash
curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "messages": [{"role": "user", "content": "Hello"}],
    "temperature": 0.7,
    "top_p": 0.95,
    "top_k": 40,
    "chat_template_kwargs": {"enable_thinking": true}
  }'
```

| Flag | Why |
| --- | --- |
| `--chat-template-file …/chat_template_enable_thinking.jinja` | The model's own template plus one line (see [Reasoning controls](#reasoning-controls)). Without it llama-server leaves the reasoning in `content` and thinking-on tool calls fail ([measured](#tool-calling)). |
| `--reasoning off` | Thinking stays off unless a request passes `"enable_thinking": true`. |
| `--jinja` | Already on by default in this build; keep it on — the reasoning controls (`chat_template_kwargs`) and tool calling rely on the Jinja chat template. |
| `-fit off` | Autofit reads `MemAvailable` on integrated GPUs and can silently shrink context or push tensors to CPU. |
| `-cram <MiB>` | Not set above (default 8 GiB of host RAM for saved prompts). Set it on a shared box — see [Known issues](#known-issues-and-limits). |
| `--mmproj` | Loads the 27-layer vision tower. Drop the flag for text-only. |

Requires a llama.cpp build with ROCmFP4 / ROCmFPX tensor-type support; stock llama.cpp rejects these tensor types.

## Reasoning controls

The model's own chat template switches thinking with `chat_template_kwargs.reasoning_effort` and ignores
`enable_thinking`:

| `reasoning_effort` | What the stock template emits |
| --- | --- |
| `"none"` | empty `<think>\n\n</think>` (no thinking) |
| `"high"` | opens `<think>\n` (always think) |
| `"medium"`, unset, or anything else | opens `<think>` and lets the model decide (adaptive; upstream default is `"medium"`) |

llama-server decides how to split reasoning from the answer by rendering the template with `enable_thinking` on and
off. With this template both renders are the same, so it never extracts the reasoning ([measured](#tool-calling)).
`chat_template_enable_thinking.jinja` in this repo is the model's `chat_template.jinja` (sha256
`f1753536417ee87cded4bc5017354eb2123c079d20c634403a3d808b2ba3cc5b`) with one line added at the top (file sha256 `9183c7ba8510fb9628edd2265a4cad8c02b3aecec3cbdf65620f47494e75836d`):

```jinja
{%- if reasoning_effort is not defined and enable_thinking is defined %}{%- set reasoning_effort = 'high' if enable_thinking else 'none' %}{%- endif %}
```

Serve it with `--chat-template-file` and `--reasoning off`.

Measured on the standard STRIX_LEAN file with the included template file and `--reasoning off` (greedy probes: a direct question, a one-word instruction and a tool request, each with the tool schema attached; tool suite: 3 passes):

| request | thinking | reasoning ends up in | measured |
| --- | --- | --- | --- |
| no `chat_template_kwargs` | off (the server default with `--reasoning off`) | — | 0 of 3 replies with think tags in `content` |
| `"enable_thinking": true` | on | `reasoning_content` | tool suite with thinking on: 19/21 (a pass requires no think tags in `content`) |
| `"enable_thinking": false` | off | — | 0 of 3 replies with think tags in `content` |
| `"reasoning_effort": "none"` | off | — | 0 of 3 replies with think tags in `content` |
| `"reasoning_effort": "high"` | on | **`content`** — do not use | 3 of 3 replies with think tags in `content` |
| `"reasoning_effort": "medium"` | adaptive | **`content`** — do not use | 3 of 3 replies with think tags in `content` |

So: switch thinking with `enable_thinking` only. Thinking-off tool checks: 21/21.

Upstream serving (SGLang) uses `--reasoning-parser qwen3 --tool-call-parser qwen3_coder`. Recommended sampling:
temperature 0.7, top_p 0.95, top_k 40.

Earlier assistant turns are re-rendered **with** their reasoning (contexts grow faster than with templates that drop
it). With thinking on and a small `max_tokens`, the whole budget can go to reasoning and `content` comes back empty —
raise `max_tokens` before concluding the model is broken.

Tool calls use the XML-style `<tool_call><function=…><parameter=…>` format, which llama.cpp parses natively
through the Jinja chat template (on by default).

## Speed

| File | Backend | Workload | Decode tok/s (min–max) | Prefill tok/s |
| --- | --- | --- | ---: | ---: |
| `Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | ROCm0 | code | 63.94 (63.56–63.96) | 1158 |
| `Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | Vulkan0 | code | 68.16 (68.10–68.21) | 1006 |
| `Nex-N2.5-mini-Q4_0-ROCmFP4-COHERENT.gguf` | ROCm0 | code | 61.74 (61.74–61.80) | 1187 |
| `Nex-N2.5-mini-Q4_0-ROCmFP4-COHERENT.gguf` | Vulkan0 | code | 68.00 (67.88–68.09) | 1001 |
| `Nex-N2.5-mini-Q4_0-ROCmFP4-FAST.gguf` | ROCm0 | code | 63.31 (63.14–63.38) | 1157 |
| `Nex-N2.5-mini-Q4_0-ROCmFP4-FAST.gguf` | Vulkan0 | code | 68.62 (68.60–68.67) | 999 |
| `Nex-N2.5-mini-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | ROCm0 | code | 63.29 (63.19–63.41) | 1145 |
| `Nex-N2.5-mini-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | Vulkan0 | code | 67.77 (67.74–67.86) | 995 |
| `Nex-N2.5-mini-imatrix-Q4_0-ROCmFP4-COHERENT.gguf` | ROCm0 | code | 61.56 (61.53–61.63) | 1184 |
| `Nex-N2.5-mini-imatrix-Q4_0-ROCmFP4-COHERENT.gguf` | Vulkan0 | code | 67.81 (67.77–67.85) | 999 |
| `Nex-N2.5-mini-imatrix-Q4_0-ROCmFP4-FAST.gguf` | ROCm0 | code | 63.03 (62.55–63.11) | 1154 |
| `Nex-N2.5-mini-imatrix-Q4_0-ROCmFP4-FAST.gguf` | Vulkan0 | code | 68.44 (68.42–68.56) | 994 |
| `Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | ROCm0 | prose | 63.06 (62.94–63.09) | 1139 |
| `Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | Vulkan0 | prose | 67.34 (67.31–67.35) | 978 |

`STRIX_LEAN`: ROCm0 decodes 6.2 % slower than Vulkan0 (63.94 vs 68.16 tok/s; ranges 63.56–63.96 / 68.10–68.21) and prefills 15.1 % faster than it (1158 vs 1006 tok/s). `COHERENT`: ROCm0 decodes 9.2 % slower than Vulkan0 (61.74 vs 68.00 tok/s; ranges 61.74–61.80 / 67.88–68.09) and prefills 18.5 % faster than it (1187 vs 1001 tok/s). `FAST`: ROCm0 decodes 7.7 % slower than Vulkan0 (63.31 vs 68.62 tok/s; ranges 63.14–63.38 / 68.60–68.67) and prefills 15.8 % faster than it (1157 vs 999 tok/s). STRIX_LEAN workload range on ROCm0: code 63.94 tok/s vs prose 63.06 (prose decodes within 1.4 % of code); Vulkan0 code 68.16 vs prose 67.34 (prose decodes within 1.2 % of code). Prompt lengths: code 7,094–7,102 tokens (the first 30,000 characters of `convert_hf_to_gguf.py` plus an instruction), prose 7,453–7,457 tokens (the first 34,000 characters of wikitext-2 *train* plus a writing instruction). Speed gaps below 3.0 % (decode) and 3.0 % (prefill) are called a tie: the larger of 3 % and the widest gap measured between files that do identical work per token (each imatrix file and its standard twin: decode 1.0 %, prefill 1.1 %).

## Prompt caching

Measured: pairs of requests that share a long code prefix and differ only in the closing instruction. The second
request of each pair runs warm (`cache_prompt: true`, resuming from what the first one left) and then cold
(`cache_prompt: false`), and the two replies are compared byte for byte. Every prompt is padded to one token length so
warm and cold see identical chunking.

| server | second-request prompt tokens reused | processed | warm reply = cold reply |
| --- | ---: | ---: | :---: |
| d3ca537, unpatched | **6,011** of 7,039 (all 5 pairs) | 1,028 | 5/5 |

In 5 request pairs sharing a long prefix at one fixed prompt length of 7,039 tokens, every second request resumed from the checkpoint the first one left 1,028 tokens before its end — **6,011 tokens reused (85 %), 1,028 processed** — median prefill **1.0 s instead of 6.2 s** cold (6.1× faster). Each warm reply was byte-identical to a cold run of the same prompt in **5/5** exchanges.

llama-server processes the last `n_ubatch` + 4 tokens of every prompt as two batches so it can checkpoint there (1,024 + 4 = 1,028 tokens with the `-ub 1024` used in these measurements, where the server default is `-ub 512`; [upstream PR #20288](https://github.com/ggml-org/llama.cpp/pull/20288)). A turn that resumes from a checkpoint left by a prompt of a *different* length therefore splits its tail differently from a cold run, and float rounding can flip a greedy token.

## Tool calling

The template emits the XML-style `<tool_call><function=…><parameter=…>` format, which llama.cpp parses natively
through the Jinja chat template (on by default). Suite run through `llama-server`, at the checkpoint's recommended
sampling (temperature 0.7, top-p 0.95, top-k 40):

**40/42** over three passes with the quick-start configuration, **6/14** with the stock template, run on `Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf`. Quick start = the included template file + `--reasoning off`, thinking switched with `enable_thinking`; stock = the model's own template, thinking switched with `reasoning_effort` (`high` / `none`). A check passes only with a native `tool_calls` entry carrying the right arguments and no raw XML or think tags left in `content`.

| check | quick start, thinking ON | quick start, thinking OFF | stock template, thinking ON | stock template, thinking OFF |
| --- | :---: | :---: | :---: | :---: |
| multi-arg | 3/3 | 3/3 | ❌ | ✅ |
| nested-object | 2/3 | 3/3 | ❌ | ❌ |
| enum | 3/3 | 3/3 | ❌ | ✅ |
| correct-decline | 3/3 | 3/3 | ❌ | ✅ |
| multi-turn | 3/3 | 3/3 | ❌ | ✅ |
| streaming | 3/3 | 3/3 | ❌ | ✅ |
| parallel | 2/3 | 3/3 | ❌ | ✅ |

**Stock chat template: 6/14.** Every thinking-on check failed. Re-run with the raw replies kept, 7 of 7 thinking-on replies carried the reasoning and a `</think>` in `content`, and 0 had any `reasoning_content`. llama-server builds its reasoning parser by rendering the template with `enable_thinking` on and off; this template ignores `enable_thinking` (it switches on `reasoning_effort`), so the parser finds no reasoning markers and extracts nothing. Server switches did not help (`--reasoning-format deepseek`: 3 of 3 replies still had reasoning in `content`; `--chat-template-kwargs` with `reasoning_effort`: 3 of 3 replies still had reasoning in `content`; `--reasoning on`: 3 of 3 replies still had reasoning in `content`). With thinking off, `nested-object` failed on an HTTP 500 — see [Known issues](#known-issues-and-limits).

**With the included `chat_template_enable_thinking.jinja` and `--reasoning off` (the quick start): 13/14, 13/14, 14/14 over three passes of the same suite (40/42)** — thinking off 21/21, thinking on 19/21; the misses were `nested-object` with thinking on (pass 1), `parallel` with thinking on (pass 2). A pass requires a native `tool_calls` entry with the right arguments and no think tags in `content`. Each check is a single sample at the recommended temperature 0.7.

## Vision

`mmproj-Nex-N2.5-mini-BF16.gguf` is the 27-layer vision tower (width 1152),
loaded with `--mmproj`. Its attention follows the server's `-fa` setting, so both settings were checked.

Probe: a synthetic 448×448 image with a red circle and a blue square (a model that ignores the image cannot name both), sent to `Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf` with `--mmproj`, temperature 0. Pass = the reply names every expected term (`red`, `blue`, `circle`, `square`).

| | `-fa on` | `-fa off` |
| --- | :---: | :---: |
| STRIX_LEAN + projector | ✅ 4/4 terms | ✅ 4/4 terms |

✅ **Images work with `-fa on` and `-fa off`.** The `-fa on` image test was repeated with the included template file and `--reasoning off` (the quick start): ✅ passed.

Reply (`-fa on`):

> The image shows two shapes: a red circle on the left and a blue square on the right.

## Memory

Measured footprint (drop in `MemAvailable`) of STRIX_LEAN with the vision projector, q8_0 KV cache, `-cram 512`, one slot, no draft head:

| context | after load | after one request (30,000-character code prompt) |
| ---: | ---: | ---: |
| 65,536 | 21.11 GiB | 21.29 GiB |
| 262,144 | 24.36 GiB | 24.52 GiB |

No row was decode-benchmarked beyond that one request. Nothing beyond these rows was measured.

## Quantization methodology

```bash
# 1. convert: text model and the vision projector (the checkpoint has no mtp.* tensors)
python convert_hf_to_gguf.py hf --outtype bf16 --model-name Nex-N2.5-mini --outfile Nex-N2.5-mini-BF16.gguf
python convert_hf_to_gguf.py hf --outtype bf16 --mmproj --model-name Nex-N2.5-mini --outfile mmproj-Nex-N2.5-mini-BF16.gguf

# 2. quantize from BF16 only; the LM head is forced up on every tier and read back by exact tensor name
llama-quantize --output-tensor-type q6_K                            Nex-N2.5-mini-BF16.gguf OUT Q4_0_ROCMFP4_STRIX_LEAN 16
llama-quantize --output-tensor-type q6_K --token-embedding-type q6_K Nex-N2.5-mini-BF16.gguf OUT Q4_0_ROCMFP4_COHERENT 16
llama-quantize --output-tensor-type q6_K                            Nex-N2.5-mini-BF16.gguf OUT Q4_0_ROCMFP4_FAST 16

# 3. BF16 reference logits on the CPU only (this build's ROCm0 path computes the BF16 MoE wrong — Known issues)
llama-perplexity -m Nex-N2.5-mini-BF16.gguf -f wikitext-2-raw/wiki.test.raw -c 2048 -b 2048 --chunks 40 --kl-divergence-base bf16.kld \
  -dev none -ngl 0 --no-op-offload -t 16
# 4. grade each shipped file against those logits, on each GPU backend
llama-perplexity -m OUT --kl-divergence-base bf16.kld --kl-divergence -c 2048 -b 2048 -ngl 999 -fa on -dio -dev ROCm0
llama-perplexity -m OUT --kl-divergence-base bf16.kld --kl-divergence -c 2048 -b 2048 -ngl 999 -fa on -dio -dev Vulkan0
```

Receipts (the built file is the receipt — exact tensor names, never a substring match; `recipe/logs/`):

| File | `output.weight` | `token_embd.weight` | tensors |
| --- | --- | --- | ---: |
| `Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | Q6_K | Q5_K | 733 |
| `Nex-N2.5-mini-Q4_0-ROCmFP4-COHERENT.gguf` | Q6_K | Q6_K | 733 |
| `Nex-N2.5-mini-Q4_0-ROCmFP4-FAST.gguf` | Q6_K | Q4_0_ROCMFP4_FAST | 733 |

`tie_word_embeddings` is false, so the output head is a real tensor and `--output-tensor-type q6_K` does real work.
All three tiers pin `output.weight` to `q6_K`; COHERENT also pins `token_embd.weight` to `q6_K`, while STRIX_LEAN and
FAST keep their tier's own embedding type (shown in the receipts).

## Reproduction

```
server   : https://github.com/charlie12345/ROCmFPX @ d3ca537
           unpatched; build dir /opt/llama-rocm/rocmfpx-724/build-hipvk, Release, Unix Makefiles, GGML_HIP=ON GGML_VULKAN=ON
           GGML_HIP_GRAPHS=ON GGML_HIP_NO_VMM=ON GGML_NATIVE=ON AMDGPU_TARGETS=gfx1151 LLAMA_CURL=OFF
           CMAKE_HIP_COMPILER=/opt/rocm-7.2.4/lib/llvm/bin/clang
           sha256 llama-quantize   1559189968a30781f837e4d25ca3cdf26c8eb0351023675406535eddcbe0db13
           sha256 llama-imatrix    bfdce5c21a01bdc8043d69335b74a7b68b8335ae989991a1e10e21d5faeaeb8e
           sha256 llama-perplexity c2175925bb0c61df89ff787845d8e03a35403f684be52719ac2987b8ca636a53
           sha256 llama-server     d077b98f3f04df4b482a5826c8922e39cda3aa8d2e2b2fbe1870f36d245cebe7
source   : nex-agi/Nex-N2.5-mini  revision 87420286149d9cce9bd46cd335ef9bda33c37c1b
model    : Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf (the argv below; every file was measured the same way)
argv     : /opt/llama-rocm/rocmfpx-724/build-hipvk/bin/llama-server -m /mnt/models/nex-n2.5-mini/out/Nex-N2.5-mini-Q4_0_ROCMFP4_STRIX_LEAN.gguf -dev ROCm0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui
template : the quick-start tool-suite and image rows add --chat-template-file chat_template_enable_thinking.jinja
           --reasoning off to this argv (recipe/pipeline/run_tools_roff.sh -> nex_tools_tpl.py; their server logs
           read "chat template, thinking = 0"); the speed rows use the stock template
env      : LD_LIBRARY_PATH=<build>/bin:/opt/rocm-7.2.4/lib
           HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1
box      : aimax — AMD Ryzen AI Max+ 395 / Radeon 8060S (gfx1151), 124 GiB, GTT 131072 MiB,
           kernel 6.17.6-061706-generic, ROCm 7.2.4
protocol : 7,094–7,102-token code prompt, 256 generated tokens, temp 0 / top_k 1, ignore_eos, cache_prompt false,
           1 warm-up + median of 3, no co-resident models (box iced)
measured : 2026-09-16 to 2026-09-17, by the pipeline in recipe/ (every raw number in recipe/results_summary.json and recipe/raw/)
```

## Files

| File | Size | sha256 |
| --- | ---: | --- |
| `Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | 17.46 GiB | `4437c3d8871acf04e4be8a9b773cf69697e5fc4b7b41fae41261f246ee5add9e` |
| `Nex-N2.5-mini-Q4_0-ROCmFP4-COHERENT.gguf` | 18.48 GiB | `43673ce7b05f6d3924f6268e7e3102bab8fc4f91771505ef5734beaddacaa2f1` |
| `Nex-N2.5-mini-Q4_0-ROCmFP4-FAST.gguf` | 17.37 GiB | `156b516ecfc0f8e400b165b7d42d959a0bd902814bb6949ca040c991494886d1` |
| `mmproj-Nex-N2.5-mini-BF16.gguf` | 0.84 GiB | `a80a82977e4787a838a612316c79a6533c1b098fcac19372cc78929120162a4e` |
| `chat_template_enable_thinking.jinja` | 7.7 KiB | `9183c7ba8510fb9628edd2265a4cad8c02b3aecec3cbdf65620f47494e75836d` |

`SHA256SUMS` covers every model file and the chat template file. `recipe/` holds the measurement pipeline (`recipe/pipeline/`), raw per-run
results (`recipe/raw/`), build and receipt logs (`recipe/logs/`), and `results_summary.json` with every measured value
on this card. Architecture facts (layer counts, vocabulary, vision depth) come from the checkpoint's `config.json` at
revision `87420286149d9cce9bd46cd335ef9bda33c37c1b`.

## Known issues and limits

- **Do not run the BF16 GGUF of this model on ROCm0 with this build.** `d3ca537` computes it wrong on that path: wikitext-2 perplexity 105.9 on ROCm0 vs 6.23 on the CPU (first window 139.1 vs 5.70; also wrong with `-fa off`). The 4-bit files are not affected — their ROCm0 grades are in the table — and the BF16 file is not published here; the quality reference was computed on the CPU instead.
- **llama-server rejects a tool call whose required arguments are not in the order the schema lists them** — HTTP 500, *The model produced output that does not match the expected peg-native format*: this build's parser for the XML tool-call format expects required arguments in definition order. On the standard STRIX_LEAN file with the stock template and thinking off, the `nested-object` request (three required arguments) hit it in the tool suite and in 2 of 4 repeats; the model had written a well-formed call with the arguments reordered. Be ready to retry on this error.
- **With the included template, do not set `reasoning_effort` to `high` or `medium`** — the reasoning goes back into `content` (6 of 6 probe replies). `enable_thinking: true` is the way to turn thinking on.
- **With thinking on, a very short answer can stay inside the think block.** Served as in the quick start, `Reply with the single word: ready` (no tools, `enable_thinking: true`, temperature 0) came back with the word in `reasoning_content` and an empty `content` on 2 of 2 FAST files tested. With thinking off the same request returned `ready` in `content`.
- **No MTP head.** The checkpoint's `config.json` declares `mtp_num_hidden_layers: 1`, but the weights contain **no** `mtp.*` tensors (1,026 tensors total). There is no multi-token-prediction head and no speculative decoding on these files — do not pass `--spec-type draft-mtp`.
- **The stock chat template ignores `enable_thinking`, and llama-server cannot separate its reasoning** (see [Tool calling](#tool-calling)). Serve with the included `chat_template_enable_thinking.jinja` and `--reasoning off`, and switch thinking per request with `enable_thinking` — see [Reasoning controls](#reasoning-controls).
- **Earlier assistant turns are re-rendered with their reasoning**, so multi-turn contexts grow faster than with templates that drop it.
- **`llama-server`'s host-RAM prompt cache defaults to 8 GiB** (`-cram 8192`). On a shared box, set `-cram` explicitly.
- Measured on Linux only (Ryzen AI Max+ 395, ROCm 7.2.4, unpatched `d3ca537`).
- **Not measured:** decode beyond a 7,457-token prompt (a 262,144-token context was loaded in the memory test, not benchmarked), long-context quality, video input, concurrency above 1, and task-level accuracy. Perplexity/KLD measure next-token fidelity to BF16 on prose, not reasoning or code correctness.

## License and attribution

Apache-2.0, inherited from the base model. Weights and architecture: **Nex-AGI**
([nex-agi/Nex-N2.5-mini](https://huggingface.co/nex-agi/Nex-N2.5-mini)). ROCmFP4 / ROCmFPX quantization format and
runtime: the ROCmFPX project. Quantization and measurements: kingjones777.