readme: add architecture + quant tables, keep voice

README.md

@@ -23,32 +23,41 @@ language:
 
 # Nex-N2-mini IQ3_XXS
 
-imatrix-calibrated IQ3_XXS of nex-n2-mini. ~13gb, fits in 15gb GPU memory with room left for context. mostly made it because i wanted to run this model on a laptop iGPU and the existing quants were all 16gb+.
+imatrix-calibrated IQ3_XXS of [nex-agi/Nex-N2-mini](https://huggingface.co/nex-agi/Nex-N2-mini). 13 GB, fits in 15 GB GPU memory with room for context. smallest quant of this model on the hub as of 2026-06-09.
 
-gets ~14 tok/s on CPU only on a ryzen 7 pro 7735u, more with vulkan offload.
+made it because i wanted to run nex-n2-mini on my laptop's AMD iGPU (15 GB GTT cap) and every existing quant was 14 GB+.
 
-## you'll hit this if you quantize it yourself
+gets **~14 tok/s on CPU only** (Ryzen 7 PRO 7735U, no GPU offload). vulkan offload pushes it higher.
 
-```
-missing tensor 'blk.39.nextn.eh_proj.weight'
-```
+## architecture
 
-took me forever to figure out. nex agi didnt release the MTP weights with the model, but the config says they're there, so the convert script writes "has MTP" into the GGUF header and then nothing loads it.
+| | |
+|---|---|
+| Base | Qwen3.5-35B-A3B-Base (post-trained by Nex AGI) |
+| Architecture | `qwen35moe` |
+| Total params | ~35B |
+| Active params | ~3B per token |
+| Experts | 256 total, 8 routed + 1 shared per token |
+| Hidden size | 2048 |
+| Trunk layers | 40 (MTP head not included — see below) |
+| Train context | 262144 |
+| Vocab | 248320 |
+| Vision | not in this GGUF (text-only — see "stuff to know") |
 
-two settings to patch:
+## file
 
-```
-qwen35moe.nextn_predict_layers: 1 → 0
-qwen35moe.block_count:          41 → 40
-```
-
-`patch_gguf.py` in the repo does it. 4-byte edits, doesnt shift anything else, takes 30s. way faster than re-converting from safetensors (8h).
+| file | quant | size | bpw | notes |
+|---|---|---:|---:|---|
+| `Nex-N2-mini-IQ3_XXS.gguf` | IQ3_XXS | 12.7 GB | 3.14 | attention kept at Q4_K, FFN experts pushed to IQ3 |
+| `imatrix.dat` | — | 183 MB | — | importance matrix, re-quantize from this if you want a different size |
+| `patch_gguf.py` | — | 3.5 KB | — | fixes the MTP load error (see below) |
+| `Modelfile` | — | 1 KB | — | for `ollama create` |
 
 ## using it
 
-needs a llama.cpp from after 2026-02-10 (when qwen35moe arch landed in PR #19468).
+needs a llama.cpp from after **2026-02-10** (when qwen35moe arch landed in [PR #19468](https://github.com/ggml-org/llama.cpp/pull/19468)).
 
-LM Studio: drop the gguf in `~/.lmstudio/models/<you>/Nex-N2-mini-GGUF/`, load it. update the bundled llama.cpp runtime to 2.13+ if loading fails.
+LM Studio: drop the gguf in `~/.lmstudio/models/<you>/Nex-N2-mini-GGUF/`, load it. update the bundled llama.cpp runtime to 2.13+ if it refuses to load.
 
 Ollama 0.19+:
 ```
@@ -65,27 +74,45 @@ llama-cli -m Nex-N2-mini-IQ3_XXS.gguf -ngl 999 \
 
 ollama before 0.19 wont work — too old to know the qwen35moe arch.
 
+## if you quantize it yourself
+
+you'll hit:
+```
+missing tensor 'blk.39.nextn.eh_proj.weight'
+```
+
+took me forever to figure out. nex agi didn't release the MTP draft head weights with the public release, but `config.json` claims they exist, so the convert script writes "has MTP" into the GGUF header and llama.cpp's loader then refuses because it can't find the tensor.
+
+two metadata values to flip:
+
+```
+qwen35moe.nextn_predict_layers: 1 → 0
+qwen35moe.block_count:          41 → 40
+```
+
+`patch_gguf.py` in this repo does it. 4-byte edits, idempotent, takes 30 seconds. way faster than re-converting from safetensors (8h on a laptop).
+
 ## stuff to know
 
-- its a reasoning model so outputs have `<think>...</think>` blocks. handle that or strip it
-- no MTP speedup since the weights arent in the public release
-- text only. the base model's config.json has vision/video token slots but llama.cpp's qwen35moe converter is text-only (PR #19468 title literally says "no vision"). dont expect image support
-- Q3 means maybe 1-3% benchmark drop vs Q4. for chat/tool use i cant tell
+- **reasoning model** — outputs contain `<think>...</think>` blocks. handle them in your wrapper or strip them
+- **no MTP speedup** — weights aren't in the public release. inference works fine, you just don't get the speculative-decoding bonus
+- **text only** — the base model's `config.json` has `vision_config` + image/video token slots, but llama.cpp's qwen35moe converter is text-only (PR #19468 literally titled "no vision"). if you want vision, look at quants that ship an `mmproj-*.gguf` alongside
+- **Q3 means ~1-3% benchmark drop vs Q4** — for chat and tool-calling i can't tell the difference. for code/math it'll be more noticeable
 
 ## how i made it
 
 ```
 huggingface-cli download nex-agi/Nex-N2-mini --local-dir source
 python convert_hf_to_gguf.py source --outtype f16
-python patch_gguf.py source/*-F16.gguf   # whatever the convert script named it
+python patch_gguf.py source/*-F16.gguf
 llama-imatrix -m <f16.gguf> -f calibration.txt -o imatrix.dat --chunks 50
-llama-quantize --imatrix imatrix.dat <f16.gguf> out.gguf IQ3_XXS
+llama-quantize --imatrix imatrix.dat <f16.gguf> Nex-N2-mini-IQ3_XXS.gguf IQ3_XXS
 ```
 
-calibration text was Pride and Prejudice from gutenberg plus the nex-n2 README. imatrix-aware quantizer kept attention tensors at Q4_K and pushed expert FFN weights down to IQ3, ended up at 3.14 bpw avg.
+calibration text = Pride and Prejudice from gutenberg + the nex-n2 README (~750 KB total, 50 chunks of 512 tokens). imatrix-aware quantizer kept attention tensors at Q4_K and pushed expert FFN weights down to IQ3 — ended up at 3.14 bpw avg.
 
-`imatrix.dat` is in the repo if you wanna re-quantize to something else.
+`imatrix.dat` is in the repo if you want to re-quantize to IQ2_S, Q4_K_S, or anything else without redoing the calibration pass.
 
 ---
 
-credit to nex agi for the model, Qwen team for the base arch, ggml-org for llama.cpp. apache 2.0, same as the base.
+base model © Nex AGI · base architecture © Qwen team · llama.cpp © ggml-org. apache 2.0, same as the base.