Instructions to use kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0
# Run inference directly in the terminal:
llama cli -hf kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0
# Run inference directly in the terminal:
llama cli -hf kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0
# Run inference directly in the terminal:
./llama-cli -hf kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0

Use Docker

docker model run hf.co/kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0

LM Studio
Jan

vLLM

How to use kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0

Ollama
How to use kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF with Ollama:
```
ollama run hf.co/kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0
```
Unsloth Desktop

How to use kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF with Docker Model Runner:
```
docker model run hf.co/kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0
```

Lemonade

How to use kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0

Run and chat with the model

lemonade run user.Nex-N2.5-mini-ROCmFP4-GGUF-Q4_0

List all available models

lemonade list

Hermes Agent

How to use kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF:Q4_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Nex-N2.5-mini-ROCmFP4-GGUF / recipe /logs /N6_bench.log

kingjones777

Add files using upload-large-folder tool

4294de3 verified 3 days ago

Raw

History Blame Contribute Delete

12.1 kB

	[2026-09-16T23:22:44Z] waiting for the quiet-box lock (no HF upload during speed runs)
	[2026-09-17T00:32:03Z] quiet-box lock held
	[2026-09-17T00:32:03Z] bench n-q106-rocm
	{"label": "n-q106-rocm", "model": "Nex-N2.5-mini-Q4_0_ROCMFP4_STRIX_LEAN.gguf", "draft": null, "nmax": null, "strict": false, "bin": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin", "dev": "ROCm0", "ctx": 65536, "workload": "code", "load_s": 22.0, "cmd": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin/llama-server -m /mnt/models/nex-n2.5-mini/out/Nex-N2.5-mini-Q4_0_ROCMFP4_STRIX_LEAN.gguf -dev ROCm0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui", "tg_median": 63.94, "tg_min": 63.56, "tg_max": 63.96, "pp_median": 1158.2, "prompt_n": 7098, "accept": null}
	[2026-09-17T00:33:06Z] bench n-q106-vk
	{"label": "n-q106-vk", "model": "Nex-N2.5-mini-Q4_0_ROCMFP4_STRIX_LEAN.gguf", "draft": null, "nmax": null, "strict": false, "bin": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin", "dev": "Vulkan0", "ctx": 65536, "workload": "code", "load_s": 4.0, "cmd": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin/llama-server -m /mnt/models/nex-n2.5-mini/out/Nex-N2.5-mini-Q4_0_ROCMFP4_STRIX_LEAN.gguf -dev Vulkan0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui", "tg_median": 68.16, "tg_min": 68.1, "tg_max": 68.21, "pp_median": 1006.0, "prompt_n": 7096, "accept": null}
	[2026-09-17T00:33:54Z] bench n-q102-rocm
	{"label": "n-q102-rocm", "model": "Nex-N2.5-mini-Q4_0_ROCMFP4_COHERENT.gguf", "draft": null, "nmax": null, "strict": false, "bin": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin", "dev": "ROCm0", "ctx": 65536, "workload": "code", "load_s": 24.0, "cmd": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin/llama-server -m /mnt/models/nex-n2.5-mini/out/Nex-N2.5-mini-Q4_0_ROCMFP4_COHERENT.gguf -dev ROCm0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui", "tg_median": 61.74, "tg_min": 61.74, "tg_max": 61.8, "pp_median": 1186.7, "prompt_n": 7100, "accept": null}
	[2026-09-17T00:34:58Z] bench n-q102-vk
	{"label": "n-q102-vk", "model": "Nex-N2.5-mini-Q4_0_ROCMFP4_COHERENT.gguf", "draft": null, "nmax": null, "strict": false, "bin": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin", "dev": "Vulkan0", "ctx": 65536, "workload": "code", "load_s": 22.0, "cmd": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin/llama-server -m /mnt/models/nex-n2.5-mini/out/Nex-N2.5-mini-Q4_0_ROCMFP4_COHERENT.gguf -dev Vulkan0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui", "tg_median": 68.0, "tg_min": 67.88, "tg_max": 68.09, "pp_median": 1001.4, "prompt_n": 7098, "accept": null}
	[2026-09-17T00:36:05Z] bench n-q103-rocm
	{"label": "n-q103-rocm", "model": "Nex-N2.5-mini-Q4_0_ROCMFP4_FAST.gguf", "draft": null, "nmax": null, "strict": false, "bin": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin", "dev": "ROCm0", "ctx": 65536, "workload": "code", "load_s": 22.0, "cmd": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin/llama-server -m /mnt/models/nex-n2.5-mini/out/Nex-N2.5-mini-Q4_0_ROCMFP4_FAST.gguf -dev ROCm0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui", "tg_median": 63.31, "tg_min": 63.14, "tg_max": 63.38, "pp_median": 1156.6, "prompt_n": 7100, "accept": null}
	[2026-09-17T00:37:08Z] bench n-q103-vk
	{"label": "n-q103-vk", "model": "Nex-N2.5-mini-Q4_0_ROCMFP4_FAST.gguf", "draft": null, "nmax": null, "strict": false, "bin": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin", "dev": "Vulkan0", "ctx": 65536, "workload": "code", "load_s": 20.0, "cmd": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin/llama-server -m /mnt/models/nex-n2.5-mini/out/Nex-N2.5-mini-Q4_0_ROCMFP4_FAST.gguf -dev Vulkan0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui", "tg_median": 68.62, "tg_min": 68.6, "tg_max": 68.67, "pp_median": 998.9, "prompt_n": 7098, "accept": null}
	[2026-09-17T00:38:12Z] bench n-q106i-rocm
	{"label": "n-q106i-rocm", "model": "Nex-N2.5-mini-imatrix-Q4_0_ROCMFP4_STRIX_LEAN.gguf", "draft": null, "nmax": null, "strict": false, "bin": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin", "dev": "ROCm0", "ctx": 65536, "workload": "code", "load_s": 22.0, "cmd": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin/llama-server -m /mnt/models/nex-n2.5-mini/out-imat/Nex-N2.5-mini-imatrix-Q4_0_ROCMFP4_STRIX_LEAN.gguf -dev ROCm0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui", "tg_median": 63.29, "tg_min": 63.19, "tg_max": 63.41, "pp_median": 1145.3, "prompt_n": 7100, "accept": null}
	[2026-09-17T00:39:16Z] bench n-q106i-vk
	{"label": "n-q106i-vk", "model": "Nex-N2.5-mini-imatrix-Q4_0_ROCMFP4_STRIX_LEAN.gguf", "draft": null, "nmax": null, "strict": false, "bin": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin", "dev": "Vulkan0", "ctx": 65536, "workload": "code", "load_s": 20.0, "cmd": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin/llama-server -m /mnt/models/nex-n2.5-mini/out-imat/Nex-N2.5-mini-imatrix-Q4_0_ROCMFP4_STRIX_LEAN.gguf -dev Vulkan0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui", "tg_median": 67.77, "tg_min": 67.74, "tg_max": 67.86, "pp_median": 995.0, "prompt_n": 7098, "accept": null}
	[2026-09-17T00:40:20Z] bench n-q102i-rocm
	{"label": "n-q102i-rocm", "model": "Nex-N2.5-mini-imatrix-Q4_0_ROCMFP4_COHERENT.gguf", "draft": null, "nmax": null, "strict": false, "bin": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin", "dev": "ROCm0", "ctx": 65536, "workload": "code", "load_s": 24.0, "cmd": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin/llama-server -m /mnt/models/nex-n2.5-mini/out-imat/Nex-N2.5-mini-imatrix-Q4_0_ROCMFP4_COHERENT.gguf -dev ROCm0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui", "tg_median": 61.56, "tg_min": 61.53, "tg_max": 61.63, "pp_median": 1183.6, "prompt_n": 7098, "accept": null}
	[2026-09-17T00:41:25Z] bench n-q102i-vk
	{"label": "n-q102i-vk", "model": "Nex-N2.5-mini-imatrix-Q4_0_ROCMFP4_COHERENT.gguf", "draft": null, "nmax": null, "strict": false, "bin": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin", "dev": "Vulkan0", "ctx": 65536, "workload": "code", "load_s": 22.0, "cmd": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin/llama-server -m /mnt/models/nex-n2.5-mini/out-imat/Nex-N2.5-mini-imatrix-Q4_0_ROCMFP4_COHERENT.gguf -dev Vulkan0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui", "tg_median": 67.81, "tg_min": 67.77, "tg_max": 67.85, "pp_median": 998.7, "prompt_n": 7097, "accept": null}
	[2026-09-17T00:42:32Z] bench n-q103i-rocm
	{"label": "n-q103i-rocm", "model": "Nex-N2.5-mini-imatrix-Q4_0_ROCMFP4_FAST.gguf", "draft": null, "nmax": null, "strict": false, "bin": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin", "dev": "ROCm0", "ctx": 65536, "workload": "code", "load_s": 22.0, "cmd": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin/llama-server -m /mnt/models/nex-n2.5-mini/out-imat/Nex-N2.5-mini-imatrix-Q4_0_ROCMFP4_FAST.gguf -dev ROCm0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui", "tg_median": 63.03, "tg_min": 62.55, "tg_max": 63.11, "pp_median": 1154.4, "prompt_n": 7101, "accept": null}
	[2026-09-17T00:43:35Z] bench n-q103i-vk
	{"label": "n-q103i-vk", "model": "Nex-N2.5-mini-imatrix-Q4_0_ROCMFP4_FAST.gguf", "draft": null, "nmax": null, "strict": false, "bin": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin", "dev": "Vulkan0", "ctx": 65536, "workload": "code", "load_s": 20.0, "cmd": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin/llama-server -m /mnt/models/nex-n2.5-mini/out-imat/Nex-N2.5-mini-imatrix-Q4_0_ROCMFP4_FAST.gguf -dev Vulkan0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui", "tg_median": 68.44, "tg_min": 68.42, "tg_max": 68.56, "pp_median": 993.9, "prompt_n": 7098, "accept": null}
	[2026-09-17T00:44:39Z] bench n-q106-rocm-prose
	{"label": "n-q106-rocm-prose", "model": "Nex-N2.5-mini-Q4_0_ROCMFP4_STRIX_LEAN.gguf", "draft": null, "nmax": null, "strict": false, "bin": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin", "dev": "ROCm0", "ctx": 65536, "workload": "prose", "load_s": 22.0, "cmd": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin/llama-server -m /mnt/models/nex-n2.5-mini/out/Nex-N2.5-mini-Q4_0_ROCMFP4_STRIX_LEAN.gguf -dev ROCm0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui", "tg_median": 63.06, "tg_min": 62.94, "tg_max": 63.09, "pp_median": 1139.1, "prompt_n": 7454, "accept": null}
	[2026-09-17T00:45:44Z] bench n-q106-vk-prose
	{"label": "n-q106-vk-prose", "model": "Nex-N2.5-mini-Q4_0_ROCMFP4_STRIX_LEAN.gguf", "draft": null, "nmax": null, "strict": false, "bin": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin", "dev": "Vulkan0", "ctx": 65536, "workload": "prose", "load_s": 22.0, "cmd": "/opt/llama-rocm/rocmfpx-724/build-hipvk/bin/llama-server -m /mnt/models/nex-n2.5-mini/out/Nex-N2.5-mini-Q4_0_ROCMFP4_STRIX_LEAN.gguf -dev Vulkan0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui", "tg_median": 67.34, "tg_min": 67.31, "tg_max": 67.35, "pp_median": 977.7, "prompt_n": 7454, "accept": null}
	[2026-09-17T00:46:53Z] cachegate n-c3-q106
	788d", "cold_sha": "a8545d39788d", "instr": "Write a function that returns every regular expression used in this file. ok ok ok", "warm_instr": "Summarise this file. (warm-up 3) ok ok ok ok"}
	{"variant": 4, "L": 7039, "warm_prompt_n": 1028, "warm_cache_n": 6011, "cold_prompt_n": 7039, "cold_cache_n": 0, "aligned": true, "warm_draft": [null, null], "cold_draft": [null, null], "identical": true, "first_diff_char": null, "warm_sha": "af97b2dde4e2", "cold_sha": "af97b2dde4e2", "instr": "Write a function that maps each class in this file to its base classes. ok", "warm_instr": "Summarise this file. (warm-up 4) ok ok ok ok"}
	{"label": "n-c3-q106", "gate": "cachegate3-aligned", "n": 5, "L": 7039, "expect_cache_n": 6011, "all_reused": true, "all_aligned": true, "identical": 5, "result": "PASS"}
	[2026-09-17T00:48:34Z] tools n-tools-q106
	alse enum: unit=fahrenheit
	PASS think=False correct-decline: content='391'
	PASS think=False multi-turn: final='Tokyo is currently 21°C with clear skies.'
	PASS think=False streaming: stream args={'city': 'Rome', 'unit': 'celsius'}
	PASS think=False parallel: calls=['lima', 'oslo']
	{"label": "n-tools-q106", "passed": 6, "total": 14, "detail": {"multi-arg\|think=True": false, "nested-object\|think=True": false, "enum\|think=True": false, "correct-decline\|think=True": false, "multi-turn\|think=True": false, "streaming\|think=True": false, "parallel\|think=True": false, "multi-arg\|think=False": true, "nested-object\|think=False": false, "enum\|think=False": true, "correct-decline\|think=False": true, "multi-turn\|think=False": true, "streaming\|think=False": true, "parallel\|think=False": true}}
	[2026-09-17T00:49:16Z] vision n-vision-q106-faon
	{"label": "n-vision-q106-faon", "fa": "on", "mtp": false, "expected": "red,blue,circle,square", "answer": "The image shows two shapes: a red circle on the left and a blue square on the right.", "hits": ["red", "blue", "circle", "square"], "error": null, "server_died": false, "server_log_errors": [], "result": "PASS"}
	[2026-09-17T00:49:42Z] vision n-vision-q106-faoff
	{"label": "n-vision-q106-faoff", "fa": "off", "mtp": false, "expected": "red,blue,circle,square", "answer": "The image shows two simple shapes:\n\n- A red circle on the left.\n- A blue square on the right.", "hits": ["red", "blue", "circle", "square"], "error": null, "server_died": false, "server_log_errors": [], "result": "PASS"}
	[2026-09-17T00:50:08Z] NEX_BENCH_DONE