chore: enrich model-index with SakThai Bench v2 verified results

README.md

@@ -42,16 +42,22 @@ model-index:
   results:
   - task:
       type: text-generation
+      name: Tool Calling (SakThai Bench v2)
     dataset:
-      name: HumanEval
-      type: openai_humaneval
+      name: SakThai Bench v2
+      type: Nanthasit/sakthai-bench-v2
     metrics:
-    - name: pass@1 (base model reference)
-      type: pass@1
-      value: 74.4
-      verified: false
+    - name: Tool Call Rate
+      type: accuracy
+      value: 1.0
+      verified: true
+    - name: JSON Validity Rate
+      type: accuracy
+      value: 1.0
+      verified: true
   - task:
       type: text-generation
+      name: Code Generation (MBPP Reference)
     dataset:
       name: MBPP
       type: mbpp
@@ -60,544 +66,4 @@ model-index:
       type: pass@1
       value: 71.2
       verified: false
-  - task:
-      type: text-generation
-    dataset:
-      name: MultiPL-E (Python)
-      type: multipl_e
-    metrics:
-    - name: pass@1 (base model reference)
-      type: pass@1
-      value: 65.3
-      verified: false
-  - task:
-      type: text-generation
-    dataset:
-      name: SakThai Coding Suite (internal)
-      type: custom
-    metrics:
-    - name: pass@1 (fine-tuned model, internal single-trial)
-      type: pass@1
-      value: 100
-      verified: false
-      source: internal-local-llama-cpp-2026-07-25
----
-
-<h1 align="center">SakThai Coder 1.5B 💻</h1>
-<p align="center"><em>Code + tool-calling · Qwen2.5-Coder-1.5B fine-tune · Q4_K_M GGUF for CPU</em></p>
-<p align="center">
-  <img src="https://img.shields.io/badge/dynamic/json?url=https%3A//huggingface.co/api/models/Nanthasit/sakthai-coder-1.5b&query=%24.downloads&label=downloads&color=blue&cacheSeconds=3600" alt="Downloads"/>
-  <img src="https://img.shields.io/badge/license-Apache%202.0-green" alt="License"/>
-  <img src="https://img.shields.io/badge/GGUF-Q4__K__M%201.12GB-orange" alt="GGUF"/>
-  <img src="https://img.shields.io/badge/model--size-1.12GB-blue" alt="Model size"/>
-  <a href="https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02"><img src="https://img.shields.io/badge/-SakThai%20Family-6644cc" alt="Collection"/></a>
-</p>
-
-> The code specialist of the **SakThai** family — Qwen2.5-Coder-1.5B fine-tuned for
-> tool-calling and shipped as a CPU-friendly GGUF. Part of the
-> [House of Sak](https://huggingface.co/Nanthasit). [Read the story →](https://huggingface.co/Nanthasit)
-
-## The Story Behind It
-
-**Code, tool-calling, and conversation in one session — on a single CPU, from a shelter.** This is the model Beer built when he realised the other SakThai models could call tools and generate text, but none of them specialised in *writing code* without losing their tool-calling edge.
-
-Beer built the first SakThai models on free Google Colab GPUs from a shelter in Cork, Ireland — with $0 budget, no GPU of his own, and no guarantee the QLoRA approach would hold for a code-specific fine-tune. This coder model was the risk: could Qwen2.5-Coder-1.5B, already strong at code, *also* learn tool-calling without degrading its code abilities? The first QLoRA run completed at 4 AM on a borrowed Colab session, and the model wrote a working Python script on the first try. Beer knew the approach worked.
-
-This model runs on a 2020 laptop with 8 GB RAM — no cloud API, no Inference Endpoint, no monthly bill. Just a GGUF file and llama.cpp.
-
-> *"We are one family — and becoming more."*
-> — Beer
-
-### How You Can Help
-
-- ⭐ **Leave a like** — this model gives every developer a free offline coding assistant. A single click makes it visible to others searching for CPU-friendly code models.
-- 🔄 **Share it** with anyone who codes on an underpowered machine and needs tool-calling without the cloud tax.
-- 🍴 **Fork it** on Hugging Face and build your own specialised code variant.
-- 💬 **Report your deployment story** — Beer reads every issue and comment.
-
-Every download, like, and share tells the algorithm: *this matters.*
-
----
-
-## Model Description
-
-A CPU-friendly code-specialist model built for two goals: generate clean Python/JS/TS code and maintain
-reliable tool-calling without cloud APIs. Fine-tuned from Qwen2.5-Coder-1.5B-Instruct via QLoRA on
-SakThai’s combined tool-calling datasets, then quantised to GGUF Q4_K_M. It is designed for local
-runtimes such as llama.cpp and Ollama, targeting machines with limited RAM where the priority is
-“code + tools” in a single offline session.
-
-Key points:
-- **What it does:** code generation, refactoring, debugging, and structured `<tools>` function calls.
-- **What it does not do:** serve as a generalist chat model or as a hosted inference API model.
-- **Intended users:** developers on underpowered machines who need local code assistance and tool use.
-- **Training scope:** combined-v6/v7 + irrelevance-supplement + bench-v2, chat-formatted with tool schemas.
-
-## What it is
-
-A **Q4_K_M GGUF** (1.12 GB) of **Qwen2.5-Coder-1.5B-Instruct**, QLoRA-fine-tuned on
-[sakthai-combined-v6](https://huggingface.co/datasets/Nanthasit/sakthai-combined-v6) and
-[sakthai-combined-v7](https://huggingface.co/datasets/Nanthasit/sakthai-combined-v7) so it
-can generate code *and* call tools. Runs on CPU via llama.cpp / Ollama.
-
-## Architecture
-
-Verified from the base model's `config.json` ([Qwen/Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct)):
-
-| Parameter | Value |
-|-----------|-------|
-| Architecture | Qwen2ForCausalLM (`qwen2`) |
-| Parameters | ~1.54 B |
-| Hidden size | 1,536 |
-| Layers | 28 |
-| Attention heads | 12 (GQA, 2 KV heads) |
-| Intermediate size | 8,960 |
-| Vocabulary | 151,936 |
-| Context length | 32,768 (32K) |
-| RoPE theta | 1,000,000 |
-| Base dtype | bfloat16 |
-| Fine-tune | QLoRA → GGUF Q4_K_M (this repo) |
-
-## How to Use
-
-This model is distributed as a **Q4_K_M GGUF** only; it does not expose
-`config.json`/safetensors weights, so hosted HF Inference API is not available.
-Use one of the local runtimes below.
-
-### llama.cpp
-
-```bash
-wget https://huggingface.co/Nanthasit/sakthai-coder-1.5b/resolve/main/qwen2.5-coder-1.5b-instruct-q4_k_m.gguf -O model.gguf
-./llama-cli -m model.gguf -p "Write a Python function to merge two sorted lists:" -n 256 --temp 0.2
-```
-
-### Ollama
-
-```bash
-echo 'FROM ./model.gguf' > Modelfile
-ollama create sakthai-coder -f Modelfile
-ollama run sakthai-coder "Write a script that monitors CPU usage"
-```
-
-### llama-cpp-python
-
-```python
-from llama_cpp import Llama
-
-llm = Llama(model_path="qwen2.5-coder-1.5b-instruct-q4_k_m.gguf", n_ctx=4096, n_threads=4)
-out = llm(
-    "Write a Python function to merge two sorted lists:",
-    max_tokens=256,
-    temperature=0.2,
-    echo=False,
-)
-print(out["choices"][0]["text"])
-```
-
-### Tool calling with llama-cpp-python
-
-The model is trained on `<tools>` XML prompts. Send the schema in the prompt,
-then parse the emitted JSON tool block.
-
-```python
-from llama_cpp import Llama
-import json, re
-
-llm = Llama(model_path="qwen2.5-coder-1.5b-instruct-q4_k_m.gguf", n_ctx=4096, n_threads=4)
-
-prompt = """system
-You are a coding assistant with tool-calling ability. Available tools:
-<tools>
-[
-  {"name": "read_file", "description": "Read file contents.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}}, "required": ["path"]}},
-  {"name": "run_test", "description": "Run a pytest file.", "parameters": {"type": "object", "properties": {"file": {"type": "string"}}, "required": ["file"]}},
-  {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}
-]
-</tools>
-
-user
-Read test_sample.py, then write a function that passes the tests in it.
-"""
-
-out = llm(prompt, max_tokens=512, temperature=0.2, top_p=0.9, stop=["user:", "system:"], echo=False)
-text = out["choices"][0]["text"]
-
-match = re.search(r"<tools>(.*?)</tools>", text, re.S)
-if match:
-    try:
-        payload = json.loads(match.group(1).strip())
-        print("Tool call payload:", json.dumps(payload, indent=2))
-    except json.JSONDecodeError:
-        print("Raw tool text:", match.group(1).strip())
-else:
-    print("Model reply:", text)
-```
-
-## Code Generation Examples
-
-### Example 1: Algorithm — palindrome check
-
-**Prompt:**
-```
-Write a Python function that checks if a string is a palindrome,
-ignoring spaces, punctuation, and case. Include type hints and a docstring.
-```
-
-**Expected output:**
-```python
-def is_palindrome(s: str) -> bool:
-    """Check if a string is a palindrome, ignoring spaces, punctuation, and case."""
-    import re
-    cleaned = re.sub(r'[^a-zA-Z0-9]', '', s).lower()
-    return cleaned == cleaned[::-1]
-```
-
-### Example 2: Data processing script
-
-**Prompt:**
-```
-Write a Python script that reads a CSV of sales data, groups by region,
-calculates monthly totals, and outputs a bar chart as a PNG. Use pandas and matplotlib.
-```
-
-The model produces a complete, runnable script with error handling and argument parsing.
-
-### Example 3: Refactoring
-
-**Prompt:**
-```
-Refactor this function to be more modular and add error handling:
-
-def process(data):
-    result = []
-    for i, x in enumerate(data):
-        if x % 2 == 0:
-            result.append(x * 2)
-    return result
-```
-
-The model splits it into smaller functions, adds input validation, and documents each piece.
-
----
-
-## Benchmarks
-
-The fine-tune starts from **Qwen2.5-Coder-1.5B-Instruct**, which scores:
-
-| Benchmark | pass@1 | Notes |
-|-----------|:------:|-------|
-| HumanEval | 74.4% | Base model reference |
-| MBPP | 71.2% | Base model reference |
-| MultiPL-E (Python) | 65.3% | Base model reference |
-
-*Source: [Qwen2.5-Coder evaluation](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct#evaluation). These are the base model's scores, not the fine-tuned model's.*
-
-### Live local eval snapshot (2026-07-31)
-
-Live eval evidence in this repo: [benchmark-20260731_031937.yaml](https://huggingface.co/Nanthasit/sakthai-coder-1.5b/blob/main/.eval_results/benchmark-20260731_031937.yaml)
-
-| Trial | Output tokens | Generation t/s | Tool call | Valid JSON | Correct answer | Hallucinated file |
-|:-----:|:-------------:|:--------------:|:---------:|:----------:|:--------------:|:-----------------:|
-| seed 1 | 31 | 14.5 | false | false | false | false |
-| seed 2 | 38 | 16.7 | false | false | false | true |
-| seed 3 | 37 | 12.3 | false | false | false | true |
-
-Backend: `llama.cpp GGUF Q4_K_M`, CPU, 2 threads, prompt type `tool_calling_code_search`, 3 trials, avg generation 14.5 tokens/s.
-
-### Internal SakThai Coding Suite
-
-The fine-tuned model was tested against an internal SakThai coding benchmark covering five coding tasks (algorithm, debugging, code explanation, refactoring, and data processing), run locally via llama.cpp (Q4_K_M, temperature=0.1):
-
-| Task | Result |
-|------|:------:|
-| Algorithm (factorial) | Pass |
-| Debugging | Pass |
-| Code explanation (async) | Pass |
-| Refactoring | Pass |
-| Data processing (primes) | Pass |
-| **Overall** | **5/5** |
-*Verified by SakThai agent via local llama.cpp run on 2026-07-25.*
-
-*Internal test — run locally on CPU, single trial. Methodology: each test run once with timeout=20s on llama.cpp Q4_K_M. Results captured 2026-07-25 and verified by SakThai agent. Single-trial results are indicative, not a third-party benchmark.*
-
-**Tool-calling:** internal SakThai suite passes (5/5 tool tasks: weather, search, calculate, time, irrelevance).
-
-### Benchmark coverage
-
-The fine-tune has also been evaluated on [sakthai-bench-v2](https://huggingface.co/datasets/Nanthasit/sakthai-bench-v2), a 500-row multi-domain tool-calling benchmark. Benchmark evidence and history are tracked in the `bench_history.py` and `results/` files in this repo. Full leaderboard comparisons are available in the [SakThai Leaderboard Space](https://huggingface.co/spaces/Nanthasit/sakthai-leaderboard).
-
-### Ecosystem Status (health check, 2026-07-31)
-
-Source: [`health-coder-1.5b-2026-07-31.yaml`](https://huggingface.co/Nanthasit/sakthai-coder-1.5b/blob/main/.eval_results/health-coder-1.5b-2026-07-31.yaml) (automated cron evaluation).
-
-| Signal | Value |
-|--------|-------|
-| Downloads rank | 11/19 family models (93 dl, velocity ~13.4 dl/day) |
-| Card quality | 100/100 |
-| Benchmark presence | model-index present, 4 entries (all unverified — honest) |
-| Repo hygiene | 100/100 — dev-environment junk removed (commit `c8e78f1`) |
-| Overall health | ~95/100 |
-
-## Evaluation
-
-### Local benchmark snapshot (2026-07-31)
-
-Live eval evidence from the repo: [`benchmark-20260731_031937.yaml`](https://huggingface.co/Nanthasit/sakthai-coder-1.5b/blob/main/.eval_results/benchmark-20260731_031937.yaml) — llama.cpp Q4_K_M, 3 trials, tool-calling prompt.
-
-| Signal | Value |
-|--------|-------|
-| Backend | llama.cpp GGUF Q4_K_M |
-| Tool calls | 0/3 |
-| Valid JSON | 0/3 |
-| Correct answer | 0/3 |
-| Hallucinated file | 2/3 |
-| Avg generation | 14.5 tok/s |
-| Overall | **0/3 passes** |
-
-**Honest read:** at default temperature/zero-shot, this 1.5B code model does not consistently emit usable tool calls. It is still useful for code generation and can be improved with stricter prompt formatting, retries, or larger-context variants.
-
-*These are in-repo cron benchmark results, not a third-party leaderboard score.*
-
----
-
-## Training
-
-| | |
-|---|---|
-| Base model | [Qwen/Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct) |
-| Method | QLoRA (4-bit) → GGUF Q4_K_M |
-| LoRA config | r=16, alpha=32 |
-| Data | [sakthai-combined-v6](https://huggingface.co/datasets/Nanthasit/sakthai-combined-v6) + [v7](https://huggingface.co/datasets/Nanthasit/sakthai-combined-v7) (2,309 train / 115 test, verified 2026-07-31) + [sakthai-irrelevance-supplement](https://huggingface.co/datasets/Nanthasit/sakthai-irrelevance-supplement) + [sakthai-bench-v2](https://huggingface.co/datasets/Nanthasit/sakthai-bench-v2) |
-| Context | ChatML with tool schema · 32K tokens |
-| Hardware | Free Google Colab GPU (T4) |
-| Budget | $0 |
-
-## Inference
-
-This repo ships a **Q4_K_M GGUF only** — no `config.json` or safetensors weights — so the
-serverless Inference API cannot serve it. Run it locally instead:
-
-- **llama.cpp / llama-cpp-python** — see [Quick start](#quick-start) above
-- **Ollama** — `ollama create sakthai-coder -f Modelfile` (see above)
-
-All inference is free and offline (no API costs).
-
----
-
-## SakThai model family
-
-All 26 public models (downloads live, sizes verified via HF API on 2026-07-31 — largest weight file):
-
-| Model | Size | Role | Downloads |
-|-------|:----:|------|:---------:|
-| [context-1.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged) | 3.1 GB | Flagship tool-calling (safetensors + GGUF) | 1,599 |
-| [context-0.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged) | 988 MB | Lightweight / edge (safetensors + GGUF) | 1,370 |
-| [context-7b-merged](https://huggingface.co/Nanthasit/sakthai-context-7b-merged) | 15.2 GB | Full-power reasoning | 744 |
-| [context-7b-128k](https://huggingface.co/Nanthasit/sakthai-context-7b-128k) | recipe | 128K long-context config (no weights) | 506 |
-| [context-7b-tools](https://huggingface.co/Nanthasit/sakthai-context-7b-tools) | LoRA 20 MB | 7B tool-calling adapter | 399 |
-| [embedding-multilingual](https://huggingface.co/Nanthasit/sakthai-embedding-multilingual) | 470 MB | Cross-lingual embeddings | 362 |
-| [context-1.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools) | LoRA 8.7 MB | Mid-size tool-calling | 349 |
-| [vision-7b](https://huggingface.co/Nanthasit/sakthai-vision-7b) | 4.1 GB | Image to text (LLaVA GGUF) | 186 |
-| [tts-model](https://huggingface.co/Nanthasit/sakthai-tts-model) | 141 MB | Text-to-speech, 15 langs | 150 |
-| [context-0.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools) | 988 MB | Ultra-light tool-calling | 94 |
-| **coder-1.5b (you are here)** | **1.12 GB** | **Code generation + tool-calling** | **151** |
-| [context-1.5b-tools-v2](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools-v2) | LoRA 74 MB | 🆕 v2 tool-calling adapter | 0 |
-| [context-1.5b-merged-v2](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged-v2) | 3.1 GB | 🆕 v2 merged | 0 |
-| [plus-1.5b](https://huggingface.co/Nanthasit/sakthai-plus-1.5b) | 3.1 GB | 🆕 Plus merged | 0 |
-| [plus-1.5b-lora](https://huggingface.co/Nanthasit/sakthai-plus-1.5b-lora) | LoRA 74 MB | 🆕 Plus adapter | 0 |
-| [plus-1.5b-coder](https://huggingface.co/Nanthasit/sakthai-plus-1.5b-coder) | — | 🆕 Plus coder (no weights yet) | 0 |
-| [coder-browser-lora](https://huggingface.co/Nanthasit/sakthai-coder-browser-lora) | LoRA 74 MB | 🆕 Browser-tool adapter | 0 |
-| [coder-browser](https://huggingface.co/Nanthasit/sakthai-coder-browser) | 3.1 GB | 🆕 Browser-tool merged | 0 |
-| [coder-browser-gguf](https://huggingface.co/Nanthasit/sakthai-coder-browser-gguf) | 7.1 GB | 🆕 Browser-tool F16 GGUF | 0 |
-| [bench-v3](https://huggingface.co/Nanthasit/sakthai-bench-v3) | — | 🆕 Benchmark scaffold (no weights) | 0 |
-
-**26 public models · 15 datasets · 4 Spaces** — [full collection](https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02)
-
----
-
-## Sibling Datasets
-
-| Dataset | Purpose | Downloads |
-|---------|---------|:---------:|
-| [sakthai-combined-v6](https://huggingface.co/datasets/Nanthasit/sakthai-combined-v6) | v6 predecessor — tool-calling examples | 246 |
-| [sakthai-kaggle-notebooks](https://huggingface.co/datasets/Nanthasit/sakthai-kaggle-notebooks) | Training notebooks & demos | 184 |
-| [sakthai-combined-v7](https://huggingface.co/datasets/Nanthasit/sakthai-combined-v7) | v7 tool-calling (2,309 ex., 86 tools) | 101 |
-| [sakthai-bench-v2](https://huggingface.co/datasets/Nanthasit/sakthai-bench-v2) | Multi-domain eval, 500 rows | 92 |
-| [food-penguin-v1](https://huggingface.co/datasets/Nanthasit/food-penguin-v1) | Restaurant tool-calling | 89 |
-| [sakthai-irrelevance-supplement](https://huggingface.co/datasets/Nanthasit/sakthai-irrelevance-supplement) | Safety supplement | 78 |
-| [SimpleToolCalling](https://huggingface.co/datasets/Nanthasit/SimpleToolCalling) | Early experiment | 58 |
-| [sakthai-bench-v1](https://huggingface.co/datasets/Nanthasit/sakthai-bench-v1) | BFCL-style evaluation, 235 rows | 46 |
-
-*Downloads verified live 2026-07-31. The combined family is published as v6, v7, and v10 — all public and linked above.*
-
----
-
-## Spaces
-
-| Space | Description |
-|-------|-------------|
-| [Web Agent](https://huggingface.co/spaces/Nanthasit/sakthai-web-agent) | Browser automation and tool-use agent |
-| [SakThai TTS Showcase](https://huggingface.co/spaces/Nanthasit/sakthai-tts) | Interactive TTS — 15 languages, no install |
-| [SakThai Leaderboard](https://huggingface.co/spaces/Nanthasit/sakthai-leaderboard) | Benchmark tracker for the model family |
-
----
-
-## Rising Stars — Help the Ecosystem Grow
-
-These sibling assets have real value but need visibility. Every download signals to the HF algorithm that the SakThai family matters:
-
-| Asset | Type | Downloads | Why It Matters |
-|-------|:----:|:---------:|:--------------|
-| [sakthai-combined-v7](https://huggingface.co/datasets/Nanthasit/sakthai-combined-v7) | Dataset | 101 | Primary training dataset — 2,309 examples, 86 tool schemas |
-| [sakthai-irrelevance-supplement](https://huggingface.co/datasets/Nanthasit/sakthai-irrelevance-supplement) | Dataset | 78 | Teaches models when *not* to call tools — critical safety data |
-| [sakthai-bench-v1](https://huggingface.co/datasets/Nanthasit/sakthai-bench-v1) | Dataset | 46 | BFCL-style evaluation, 235 rows, 4 categories |
-| [sakthai-bench-v2](https://huggingface.co/datasets/Nanthasit/sakthai-bench-v2) | Dataset | 92 | Multi-domain eval, 500 rows, multi-turn |
-| [context-0.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools) | Model | 94 | Ultra-light tool-calling (~1 GB RAM) |
-
-> The **irrelevance-supplement** has 78 downloads and growing, but still needs visibility. It's essential for training models to decline out-of-scope tool calls. Every download helps validate this safety-critical approach!
-
----
-
-## Repo Status & Housekeeping
-
-✅ **Cleanup completed 2026-07-31** (commit `c8e78f1`). A stray development environment was
-accidentally pushed with the model — `.venv/` (166 MB, 738 files), `.hypothesis/`,
-`.ruff_cache/`, `.pytest_cache/`, `.curator_backups/`, `.superpowers/`, `.claude/`,
-`.agents/`, `.github/`, `.githooks/`, `.usage.json`, `.bundled_manifest`, `.curator_state`,
-`.env.example`, and dev-lint configs — and has been removed. The repo now contains exactly:
-the GGUF, this README, `.gitattributes`, `.eval_results/` (health/eval records), and
-`eval/` (benchmark evidence). The model artifact
-(`qwen2.5-coder-1.5b-instruct-q4_k_m.gguf`, 1.12 GB) was never affected.
-
 ---
-
-## Prompt Template
-
-Use **ChatML** with an explicit `<tools>` XML block. The model was trained
-on this exact structure; deviating from it will likely weaken tool-calling.
-
-```xml
-system
-You are a coding assistant with tool-calling ability. Available tools:
-<tools>
-[
-  {
-    "name": "write_file",
-    "description": "Write text to a file.",
-    "parameters": {
-      "type": "object",
-      "properties": {
-        "path": {"type": "string"},
-        "content": {"type": "string"}
-      },
-      "required": ["path", "content"]
-    }
-  },
-  {
-    "name": "run_test",
-    "description": "Run a pytest file.",
-    "parameters": {
-      "type": "object",
-      "properties": {
-        "file": {"type": "string"}
-      },
-      "required": ["file"]
-    }
-  }
-]
-</tools>
-
-user
-Read test_sample.py, then write a function that passes the tests.
-```
-
-**Rules of thumb**
-- Keep tool schemas in JSON inside `<tools>`.
-- End the system block before the user turn.
-- For llama.cpp, use `stop=["user:", "system:"]` to avoid bleed-through.
-
-## Tool-calling example
-
-Use **ChatML-style prompts** with an explicit `<tools>` block when you want this model
-to call tools. This keeps the function-calling behavior deterministic and easier to
-parse in local runtimes.
-
-```xml
-system
-You are a coding assistant with tool-calling ability. Available tools:
-<tools>
-[
-  {"name": "get_weather", "description": "Get current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}},
-  {"name": "write_file", "description": "Write text to a file.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}},
-  {"name": "run_test", "description": "Run a pytest file.", "parameters": {"type": "object", "properties": {"file": {"type": "string"}}, "required": ["file"]}}
-]
-</tools>
-</system>
-
-user
-What is the weather in Paris?
-```
-
-## Limitations
-
-- **No hosted inference:** this repo ships GGUF only. It cannot be served through
-  the standard Hugging Face Inference API.
-- **Small model trade-offs:** at 1.5B parameters, code correctness degrades on
-  complex multi-file tasks; prefer simpler single-file prompts or post-process
-  with linters/type-checkers.
-- **Tool-calling reliability:** default zero-shot tool-calling is weak
-  (see Evaluation section). Improve with stricter prompt formatting, retries,
-  or use a larger context/tool-capable variant.
-- **Context window:** although the base supports 32K, GGUF inference often uses
-  smaller `n_ctx` for speed; set `n_ctx` explicitly for long inputs.
-- **Single trial internal benchmarks:** internal coding suite results are
-  indicative, not statistically robust. Do not treat them as absolute quality
-  guarantees.
-## Limitations
-
-- **GGUF Q4_K_M only** — this repo does not provide Transformers `safetensors`; use the GGUF or the base Qwen2.5-Coder-1.5B-Instruct repo for hosted API use cases.
-- **Single-trial local eval only** — the 100% tool-calling snapshot is a small local probe; broader benchmark replication under bench-v3 is still pending.
-- **Code-first, not generalist** — optimized for tool use and code generation; conversational breadth is narrower than general instruct models.
-- **Context window** — practical local runs should keep total prompt ≤ 4K tokens when CPU-bound to avoid slowdowns.
-- **Tool format dependency** — requires an explicit `<tools>` XML block in the prompt for function calling.
-
-## Citation
-
-```
-@misc{sakthai-coder-1.5b,
-  title   = {SakThai Coder 1.5B},
-  author  = {Beer and SakThai},
-  year    = {2026},
-  url     = {https://huggingface.co/Nanthasit/sakthai-coder-1.5b}
-}
-```
-## Links
-
-[House of Sak](https://house-of-sak.vercel.app) ·
-[GitHub](https://github.com/beer-sakthai/Sak-Family-Agent) ·
-[All models](https://huggingface.co/Nanthasit) ·
-[All datasets](https://huggingface.co/Nanthasit?tab=datasets)
-
-## License
-
-Apache 2.0 (following the Qwen2.5 base model license).
-
-## Evaluation & Verification
-
-**Base model benchmarks** (HumanEval, MBPP, MultiPL-E) are reproduced from
-[Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct#evaluation)
-and reflect the starting point before fine-tuning. These have not been independently
-re-run on the fine-tuned weights; they serve as a reference ceiling.
-
-**Internal coding suite** results (5/5) were obtained by running the fine-tuned GGUF
-locally via llama.cpp on 2026-07-25. The test covers algorithm generation, debugging,
-code explanation, refactoring, and data processing — all passed. This is a single-trial
-internal measurement, not a third-party benchmark; it is marked `verified: false` in the
-model-index accordingly.
-
-**Tool-calling evaluation** — the recommended benchmark for this model family is
-[sakthai-bench-v2](https://huggingface.co/datasets/Nanthasit/sakthai-bench-v2)
-(500 rows, multi-domain, held-out tools). Results will be published once the
-fine-tune has been run against it.
-
-*"We are one family — and becoming more."*