Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+imatrix.dat filter=lfs diff=lfs merge=lfs -text
+Nex-N2-mini-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text

Modelfile

@@ -0,0 +1,32 @@
+# Ollama Modelfile for Nex-N2-mini-IQ3_XXS.
+# Requires Ollama 0.19 or newer (qwen35moe arch landed in llama.cpp PR #19468 on 2026-02-10).
+#
+# Usage:
+#   ollama create nex-n2-mini -f Modelfile
+#   ollama run nex-n2-mini "your prompt"
+
+FROM ./Nex-N2-mini-IQ3_XXS.gguf
+
+# Context window (4096 default; bump up to 262144 if you have GPU memory)
+PARAMETER num_ctx 4096
+
+# Offload everything to GPU. Drop if you have less than ~14 GB GPU memory.
+PARAMETER num_gpu 999
+
+# Leave 2 cores for the desktop on a typical 8-core laptop.
+PARAMETER num_thread 6
+
+# Single-user laptop: don't allocate multiple parallel KV caches.
+PARAMETER num_predict -1
+
+# Pin the model in RAM after first load (avoids re-mmap on each request).
+PARAMETER use_mlock true
+
+# Sampling defaults that match the model's training (Qwen 3.5 family).
+PARAMETER temperature 0.4
+PARAMETER top_p 0.95
+PARAMETER top_k 64
+
+# ChatML stop tokens (Nex-N2-mini uses Qwen 3.5 chat format).
+PARAMETER stop "<|im_end|>"
+PARAMETER stop "<|im_start|>"

Nex-N2-mini-IQ3_XXS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb18917549bf87c3ce6fc8294646c7f83cf333759782ecce89fde16fff971d66
+size 13623758848

README.md

@@ -1,3 +1,91 @@
 ---
 license: apache-2.0
+base_model: nex-agi/Nex-N2-mini
+base_model_relation: quantized
+library_name: gguf
+pipeline_tag: text-generation
+tags:
+  - gguf
+  - quantized
+  - imatrix
+  - iq3_xxs
+  - qwen3
+  - qwen35moe
+  - moe
+  - 35b
+  - 3b-active
+  - agentic
+  - tool-use
+  - reasoning
+language:
+  - en
 ---
+
+# Nex-N2-mini IQ3_XXS
+
+imatrix-calibrated IQ3_XXS of nex-n2-mini. ~13gb, fits in 15gb GPU memory with room left for context. mostly made it because i wanted to run this model on a laptop iGPU and the existing quants were all 16gb+.
+
+gets ~14 tok/s on CPU only on a ryzen 7 pro 7735u, more with vulkan offload.
+
+## you'll hit this if you quantize it yourself
+
+```
+missing tensor 'blk.39.nextn.eh_proj.weight'
+```
+
+took me forever to figure out. nex agi didnt release the MTP weights with the model, but the config says they're there, so the convert script writes "has MTP" into the GGUF header and then nothing loads it.
+
+two settings to patch:
+
+```
+qwen35moe.nextn_predict_layers: 1 → 0
+qwen35moe.block_count:          41 → 40
+```
+
+`patch_gguf.py` in the repo does it. 4-byte edits, doesnt shift anything else, takes 30s. way faster than re-converting from safetensors (8h).
+
+## using it
+
+needs a llama.cpp from after 2026-02-10 (when qwen35moe arch landed in PR #19468).
+
+LM Studio: drop the gguf in `~/.lmstudio/models/<you>/Nex-N2-mini-GGUF/`, load it. update the bundled llama.cpp runtime to 2.13+ if loading fails.
+
+Ollama 0.19+:
+```
+ollama create nex-n2-mini -f Modelfile
+ollama run nex-n2-mini "hi"
+```
+
+llama-cli (ChatML):
+```
+llama-cli -m Nex-N2-mini-IQ3_XXS.gguf -ngl 999 \
+  -p $'<|im_start|>system\nyou are helpful<|im_end|>\n<|im_start|>user\nhi<|im_end|>\n<|im_start|>assistant\n' \
+  -n 100
+```
+
+ollama before 0.19 wont work — too old to know the qwen35moe arch.
+
+## stuff to know
+
+- its a reasoning model so outputs have `<think>...</think>` blocks. handle that or strip it
+- no MTP speedup since the weights arent in the public release
+- text only. the base model's config.json has vision/video token slots but llama.cpp's qwen35moe converter is text-only (PR #19468 title literally says "no vision"). dont expect image support
+- Q3 means maybe 1-3% benchmark drop vs Q4. for chat/tool use i cant tell
+
+## how i made it
+
+```
+huggingface-cli download nex-agi/Nex-N2-mini --local-dir source
+python convert_hf_to_gguf.py source --outtype f16
+python patch_gguf.py source/*-F16.gguf   # whatever the convert script named it
+llama-imatrix -m <f16.gguf> -f calibration.txt -o imatrix.dat --chunks 50
+llama-quantize --imatrix imatrix.dat <f16.gguf> out.gguf IQ3_XXS
+```
+
+calibration text was Pride and Prejudice from gutenberg plus the nex-n2 README. imatrix-aware quantizer kept attention tensors at Q4_K and pushed expert FFN weights down to IQ3, ended up at 3.14 bpw avg.
+
+`imatrix.dat` is in the repo if you wanna re-quantize to something else.
+
+---
+
+credit to nex agi for the model, Qwen team for the base arch, ggml-org for llama.cpp. apache 2.0, same as the base.

imatrix.dat

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ffe3acb6b3ee36c7ae5c554a6b44c0d5645c126e39e37605ed3afa56484bb6f0
+size 192223904

patch_gguf.py

@@ -0,0 +1,104 @@
+#!/usr/bin/env python3
+"""Patch a fresh Nex-N2-mini GGUF to make it loadable by llama.cpp / Ollama / LM Studio.
+
+The convert script (convert_hf_to_gguf.py) reads `mtp_num_hidden_layers: 1` from
+config.json and writes `qwen35moe.nextn_predict_layers = 1` into the GGUF header.
+It also writes `qwen35moe.block_count = 41` (40 transformer + 1 MTP). But the
+public Nex-N2-mini safetensors don't include the MTP head weights, so llama.cpp's
+loader refuses with:
+
+    missing tensor 'blk.39.nextn.eh_proj.weight'
+
+This script flips two u32 metadata values in-place. Same length, no other bytes
+shifted, takes ~30 seconds vs. re-converting from safetensors (~8 hours).
+
+Usage:
+    python patch_gguf.py <path_to_gguf>
+
+Safe to run on either the F16 GGUF (from convert) or an already-quantized GGUF.
+Idempotent — running again is a no-op.
+"""
+from __future__ import annotations
+
+import struct
+import sys
+from pathlib import Path
+
+# GGUF KV value type sizes (for skipping)
+SCALAR_SIZE = {0: 1, 1: 1, 2: 2, 3: 2, 4: 4, 5: 4, 6: 4, 7: 1, 10: 8, 11: 8, 12: 8}
+TYPE_STRING = 8
+TYPE_ARRAY = 9
+TYPE_U32 = 4
+
+
+def skip_value(f, t):
+    if t in SCALAR_SIZE:
+        f.seek(SCALAR_SIZE[t], 1)
+    elif t == TYPE_STRING:
+        n, = struct.unpack('<Q', f.read(8))
+        f.seek(n, 1)
+    elif t == TYPE_ARRAY:
+        inner_t, = struct.unpack('<I', f.read(4))
+        cnt, = struct.unpack('<Q', f.read(8))
+        if inner_t in SCALAR_SIZE:
+            f.seek(SCALAR_SIZE[inner_t] * cnt, 1)
+        elif inner_t == TYPE_STRING:
+            for _ in range(cnt):
+                n, = struct.unpack('<Q', f.read(8))
+                f.seek(n, 1)
+        else:
+            raise ValueError(f"unsupported array inner type {inner_t}")
+    else:
+        raise ValueError(f"unsupported KV type {t}")
+
+
+def read_str(f):
+    n, = struct.unpack('<Q', f.read(8))
+    return f.read(n).decode('utf-8', errors='replace')
+
+
+def patch_u32(path: Path, key: str, new_value: int) -> bool:
+    """Find a u32 KV pair by key name and overwrite its value. Returns True if patched."""
+    with open(path, 'r+b') as f:
+        assert f.read(4) == b'GGUF', "not a GGUF file"
+        f.read(4)         # version
+        f.read(8)         # tensor count
+        nkv, = struct.unpack('<Q', f.read(8))
+        for _ in range(nkv):
+            n, = struct.unpack('<Q', f.read(8))
+            k = f.read(n).decode('utf-8', errors='replace')
+            t, = struct.unpack('<I', f.read(4))
+            if k == key:
+                if t != TYPE_U32:
+                    raise ValueError(f"{key} has type {t}, expected u32 (4)")
+                pos = f.tell()
+                old, = struct.unpack('<I', f.read(4))
+                if old == new_value:
+                    print(f"  {key} already = {new_value}, no change")
+                    return False
+                f.seek(pos)
+                f.write(struct.pack('<I', new_value))
+                print(f"  patched {key}: {old} -> {new_value}")
+                return True
+            skip_value(f, t)
+    raise ValueError(f"key not found: {key}")
+
+
+def main() -> int:
+    if len(sys.argv) != 2:
+        print(__doc__)
+        return 1
+    path = Path(sys.argv[1]).resolve()
+    if not path.exists():
+        print(f"ERROR: {path} does not exist")
+        return 1
+
+    print(f"patching {path}")
+    patch_u32(path, "qwen35moe.nextn_predict_layers", 0)
+    patch_u32(path, "qwen35moe.block_count", 40)
+    print("done. The GGUF should now load in any recent llama.cpp / LM Studio / Ollama 0.12.6+")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())