NPU-Forge v0.1: permanent custom-model registry for FastFlowLM on Ryzen AI NPUs

README.md

@@ -0,0 +1,94 @@
+---
+license: mit
+tags:
+- amd
+- ryzen-ai
+- npu
+- xdna2
+- fastflowlm
+- strix-halo
+- tools
+pipeline_tag: text-generation
+---
+
+# NPU-Forge 🔥 — keep your custom models alive on AMD Ryzen AI NPUs
+
+**The problem nobody tells you about:** you convert a fine-tuned model to
+[FastFlowLM](https://github.com/FastFlowLM/FastFlowLM)'s Q4NX format, register
+it in `model_list.json`, it runs beautifully on your NPU… and then an FLM
+update ships and your custom model **silently disappears**. The model files
+survive on disk — but `model_list.json` lives in `C:\Program Files\flm\` and
+every update resets it, de-registering every custom entry you ever added.
+
+We hit this on a Strix Halo box: a fully converted Llama-3 fine-tune
+(Stheno 8B), complete with compiled `.xclbin` NPU kernels, sitting in
+`Documents\flm\models\` — invisible to `flm list` because an update had
+eaten its registration.
+
+**NPU-Forge fixes it permanently:**
+
+- **`registry.json`** — YOUR registry, in user space, owned by you. Every
+  custom model entry lives here forever. FLM updates can't touch it.
+- **`bin/register.js`** — idempotently merges your registry into FLM's
+  `model_list.json`. Run it any time a model "disappears" — takes one second,
+  backs up FLM's list (timestamped) before every write.
+- **`register-admin.bat`** — double-click version that self-elevates (the
+  one part that needs admin is writing into Program Files).
+
+## Usage
+
+1. Convert your fine-tune to Q4NX with the official
+   [FLM_Q4NX_Converter](https://github.com/FastFlowLM/FLM_Q4NX_Converter)
+   (GGUF in — any quant — Q4NX out). Supported families: LLaMA, Qwen 2/2.5/3/3.5,
+   Gemma 3, Phi-4, LFM2, GPT-OSS, and more.
+2. Put the model folder in `C:\Users\<you>\Documents\flm\models\<YourModel>\`
+   with the standard four files: `config.json`, `model.q4nx`,
+   `tokenizer.json`, `tokenizer_config.json`. (No admin needed — model
+   storage is user-space.)
+3. Add an entry to `registry.json` (copy `registry.example.json` and edit —
+   easiest is mirroring the entry of the same-architecture stock model from
+   FLM's own `model_list.json`, changing `name` to your folder name and
+   blanking `url`).
+4. Double-click `register-admin.bat`. Done: `flm run yourmodel-forge:8b`.
+5. FLM updated and your model vanished again? Double-click again. That's it.
+
+```json
+// registry.example.json — one custom llama3-family 8B model
+{
+  "models": {
+    "mymodel-forge": {
+      "8b": {
+        "name": "MyModel-8B-NPU2",
+        "url": "",
+        "file_url": "",
+        "default_context_length": 8192,
+        "max_prefill_len": 4096,
+        "files": ["config.json", "model.q4nx", "tokenizer.json", "tokenizer_config.json"],
+        "details": { "family": "llama3", "parameter_size": "8B", "quantization_level": "Q4_1" },
+        "footprint": 4.7
+      }
+    }
+  }
+}
+```
+
+## Facts we verified the hard way (Strix Halo, FLM v0.9.43)
+
+- Model **files** are user-space (`Documents\flm\models\`) — no admin needed.
+- The **registry** (`model_list.json`) is Program Files — admin only, and
+  reset by updates.
+- A user-level `Documents\flm\model_list.json` is **ignored** (we tested).
+- Unregistered model folders can't be run by folder name (we tested) —
+  registration is mandatory.
+- FLM compiles per-model NPU kernels (`.xclbin`) on first run; they live in
+  the model folder and survive alongside it.
+
+## Where this is going
+
+NPU-Forge is the first piece of a larger goal: **one-command fine-tune → NPU**.
+The full pipeline (LoRA fine-tune in the cloud → GGUF → Q4NX → registered and
+serving on your NPU, single command) is in active development, along with
+encoder-model (BERT-class) NPU deployment. Watch this repo.
+
+Built on a Ryzen AI MAX+ 395 (Strix Halo, XDNA2) as part of an ongoing
+project to make local NPUs a first-class home for personal AI.

bin/register.js

@@ -0,0 +1,45 @@
+#!/usr/bin/env node
+// register.js — merge npu-forge's own registry into FLM's model_list.json.
+// Forge entries live in npu-forge\registry.json (user-space, permanent);
+// FLM's list lives in Program Files and gets RESET by flm updates — so this
+// merge is idempotent and re-runnable any time a custom model "disappears."
+// Needs elevation (run via register-admin.bat). Always backs up first.
+'use strict';
+const fs = require('fs');
+const path = require('path');
+
+const FORGE = path.join(__dirname, '..');
+const FLM_LIST = 'C:\\Program Files\\flm\\model_list.json';
+const REGISTRY = path.join(FORGE, 'registry.json');
+
+function main() {
+  if (!fs.existsSync(REGISTRY)) { console.log('No registry.json — nothing to register.'); return; }
+  const reg = JSON.parse(fs.readFileSync(REGISTRY, 'utf8').replace(/^/, ''));
+  const ml = JSON.parse(fs.readFileSync(FLM_LIST, 'utf8').replace(/^/, ''));
+
+  // timestamped backup beside the forge registry (user-space, survives anything)
+  const stamp = new Date().toISOString().replace(/[:.]/g, '-');
+  fs.copyFileSync(FLM_LIST, path.join(FORGE, `model_list.backup-${stamp}.json`));
+
+  let added = 0, kept = 0;
+  for (const [key, sizes] of Object.entries(reg.models || {})) {
+    for (const [size, entry] of Object.entries(sizes)) {
+      ml.models[key] = ml.models[key] || {};
+      if (ml.models[key][size] && JSON.stringify(ml.models[key][size]) === JSON.stringify(entry)) { kept++; continue; }
+      ml.models[key][size] = entry;
+      added++;
+      console.log(`registered: ${key}:${size} -> ${entry.name}`);
+    }
+  }
+  try {
+    fs.writeFileSync(FLM_LIST, JSON.stringify(ml, null, 2));
+  } catch (e) {
+    console.error('\n[!] Could not write ' + FLM_LIST);
+    console.error('    This needs administrator rights — double-click register-admin.bat instead.');
+    process.exit(1);
+  }
+  console.log(`\nDone: ${added} entries written, ${kept} already current.`);
+  console.log('Verify with: flm list  (your models end in -forge)');
+}
+
+main();

register-admin.bat

@@ -0,0 +1,11 @@
+@echo off
+title NPU-Forge - register custom models (admin)
+net session >nul 2>&1
+if errorlevel 1 (
+  echo Requesting administrator rights...
+  powershell -Command "Start-Process cmd -ArgumentList '/k cd /d \"%~dp0\" ^&^& node bin\register.js' -Verb RunAs"
+  exit /b
+)
+cd /d "%~dp0"
+node "%~dp0bin\register.js"
+pause

registry.example.json

@@ -0,0 +1,30 @@
+{
+  "models": {
+    "mymodel-forge": {
+      "8b": {
+        "name": "MyModel-8B-NPU2",
+        "url": "",
+        "file_url": "",
+        "modified_at": "2026-06-10T00:00:00Z",
+        "size": 4700000000,
+        "flm_min_version": "0.9.38",
+        "default_context_length": 8192,
+        "max_prefill_len": 4096,
+        "files": [
+          "config.json",
+          "model.q4nx",
+          "tokenizer.json",
+          "tokenizer_config.json"
+        ],
+        "details": {
+          "family": "llama3",
+          "think": false,
+          "parameter_size": "8B",
+          "quantization_level": "Q4_1"
+        },
+        "label": [],
+        "footprint": 4.7
+      }
+    }
+  }
+}