RUNNING.md — every step executed

RUNNING.md

@@ -0,0 +1,99 @@
+# Running Qwen3.5-4B-Instruct (Hob Forge Edition) — zero to first tool call
+
+Every step below was executed on real hardware before this shipped. Two paths: **A** for
+any machine with ollama (easiest), **B** for llama.cpp directly (most control). Numbers
+in the card's tables were measured with path B on a 12GB RTX 5070; an 8GB card runs
+everything here — see the memory table in the card.
+
+---
+
+## Path A — ollama (any OS, 5 minutes)
+
+1. Install ollama: https://ollama.com/download (one installer, all platforms).
+2. Pull and run this edition directly from HF:
+   ```bash
+   ollama run hf.co/Hob-forge/Qwen3.5-4B-Instruct-GGUF:Q4_K_M
+   ```
+   First run downloads 2.6GB. You're chatting when the `>>>` appears.
+3. **Thinking mode**: the model reasons out loud by default. For clean answers via the API:
+   ```bash
+   curl http://localhost:11434/api/chat -d '{
+     "model": "hf.co/Hob-forge/Qwen3.5-4B-Instruct-GGUF:Q4_K_M",
+     "messages": [{"role":"user","content":"Why is the sky blue? One sentence."}],
+     "think": false, "stream": false
+   }'
+   ```
+   Note `think` sits at the **top level** of the body — not inside `options`. (Scar #1:
+   we lost an afternoon to that once.)
+4. Context size: ollama defaults small. For the long context this arch is great at:
+   `ollama run … ` then `/set parameter num_ctx 16384` — or bake it into a Modelfile.
+   Check what you actually got: the server log prints the KV allocation.
+
+## Path B — llama.cpp (measured-numbers path)
+
+1. Get a **2026 build** — this is a hybrid-attention architecture; builds older than
+   ~March 2026 will fail with unknown-architecture errors (Scar #2: a December build
+   converted this model into a file that crashed *newer* runtimes — toolchain vintage
+   matters in both directions):
+   ```bash
+   git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
+   cmake -B build -DGGML_CUDA=ON && cmake --build build -j   # drop -DGGML_CUDA=ON for CPU
+   ```
+2. Download a quant (Q4_K_M recommended — see the card's which-file table):
+   ```bash
+   hf download Hob-forge/Qwen3.5-4B-Instruct-GGUF Qwen3.5-4B-Instruct-Q4_K_M.gguf --local-dir .
+   ```
+3. Chat, single-turn, GPU:
+   ```bash
+   ./build/bin/llama-cli -m Qwen3.5-4B-Instruct-Q4_K_M.gguf \
+     -st -p "Explain mmap in one paragraph." -ngl 99 -c 8192
+   ```
+   - `-st` (single-turn) matters: without it llama-cli may enter interactive conversation
+     mode and appear to "hang" waiting at a `>` prompt (Scar #3 — we watched a script wait
+     four hours for someone to type).
+   - 8GB card: this fits whole (`-ngl 99`). If you're sharing the GPU, `-ngl 20` splits
+     layers to CPU gracefully.
+4. Serve an OpenAI-compatible API:
+   ```bash
+   ./build/bin/llama-server -m Qwen3.5-4B-Instruct-Q4_K_M.gguf -ngl 99 -c 16384 --port 8080
+   ```
+
+## First tool call (the part most cards skip)
+
+The template supports native tool calling. Against llama-server:
+
+```bash
+curl http://localhost:8080/v1/chat/completions -d '{
+  "model": "qwen3.5-4b",
+  "messages": [{"role":"user","content":"What is 37.2% of 8412? Use the calculator."}],
+  "tools": [{"type":"function","function":{"name":"calculator",
+    "description":"Evaluate a math expression",
+    "parameters":{"type":"object","properties":{"expression":{"type":"string"}},
+    "required":["expression"]}}}]
+}'
+```
+
+Expected: a `tool_calls` entry with `{"expression":"8412*0.372"}`-style arguments. We ran
+exactly this before shipping. If you get prose instead of a tool call, your runtime is too
+old to render this template's tool block — see step B1.
+
+## Sampling that works (verified)
+
+| Mode | temp | top_p | top_k |
+|---|---|---|---|
+| Thinking (default) | 0.6 | 0.95 | 20 |
+| Non-thinking | 0.7 | 0.8 | 20 |
+
+## Troubleshooting (our scars, your shortcuts)
+
+| Symptom | Cause & fix |
+|---|---|
+| `unknown architecture` / load error | llama.cpp too old — build ≥ March 2026 (B1). |
+| `blk.32 … not found` on a self-converted file | You converted with a text-only load that dropped the MTP block — convert from the full snapshot, or use our files. |
+| Appears to hang at a `>` | Interactive mode — add `-st`, give `-p`. |
+| Painfully slow on CPU-only | The hybrid DeltaNet layers' CPU path is immature; this model *wants* a GPU. CPU works for testing, not serving. |
+| `<think>` text in answers | By design. `think:false` (ollama, top-level) / `enable_thinking=False` (transformers) / parse it out. |
+| IQ4_XS slower than Q4_K_M despite being smaller | Correct and measured (45 vs 138 t/s on RTX) — i-quant dequant cost. Use Q4_K_M unless the 200MB matters. |
+| Garbled/endless output | Check eos: template expects `<|im_end|>` — custom Modelfiles must include it as a stop. |
+
+*Something not covered? Open a discussion on the repo — we actually answer. — Hob Forge*