docs: improve model card with concise usage, verified benchmarks, metadata summary

README.md

@@ -35,10 +35,10 @@ inference:
   widget:
     - text: "Send an email to Beer with the subject 'Status update' and body 'The model is running well.'"
       output:
-        text: "<tool_call>{'name': 'send_email', 'arguments': {'to': 'Beer', 'subject': 'Status update', 'body': 'The model is running well.'}}</tool_call>"
+        text: "<tool_call>{'name': 'send_email', 'arguments': {'to': 'Beer', 'subject': 'Status update', 'body': 'The model is running well.'}}"
     - text: "What's the weather in Bangkok?"
       output:
-        text: "<tool_call>{'name': 'get_weather', 'arguments': {'location': 'Bangkok'}}</tool_call>"
+        text: "<tool_call>{'name': 'get_weather', 'arguments': {'location': 'Bangkok'}}"
 model-index:
   - name: sakthai-plus-1.5b
     results:
@@ -106,178 +106,80 @@ model-index:
             value: 50.9
             name: GSM8K
             verified: false
+extra:
+  downloads: 244
+  likes: 0
+  last_modified: 2026-08-01 07:31:41+00:00
 ---
 
-# SakThai Plus 1.5B
+<p align="center">
+  <img src="https://img.shields.io/badge/dynamic/json?url=https%3A//huggingface.co/api/models/Nanthasit/sakthai-plus-1.5b&query=%24.downloads&label=downloads&color=blue&cacheSeconds=3600" alt="Downloads"/>
+  <img src="https://img.shields.io/badge/license-Apache%202.0-green" alt="License"/>
+  <img src="https://img.shields.io/badge/params-1.54B-blueviolet" alt="Params"/>
+  <a href="https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02"><img src="https://img.shields.io/badge/%F0%9F%8F%A0-SakThai%20Family-6644cc" alt="Collection"/></a>
+  <a href="https://huggingface.co/spaces/Nanthasit/sakthai-leaderboard"><img src="https://img.shields.io/badge/%F0%9F%93%8A-Leaderboard-ff6b6b" alt="Leaderboard"/></a>
+</p>
 
-**House of Sak** — enhanced tool-calling instruct model, fine-tuned from Qwen2.5-1.5B-Instruct with rsLoRA and merged to full weight.
+<h1 align="center">SakThai Plus 1.5B</h1>
+<p align="center"><em>rsLoRA-merged instruct model for tool calling and agentic workflows</em></p>
 
-![Profile](https://img.shields.io/badge/%F0%9F%A4%97-Nanthasit-6644cc) ![GitHub](https://img.shields.io/badge/GitHub-beer--sakthai-181717?logo=github) ![Downloads](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fmodels%2FNanthasit%2Fsakthai-plus-1.5b&query=%24.downloads&label=downloads&color=blue&cacheSeconds=3600) ![License](https://img.shields.io/badge/license-Apache%202.0-green) ![Params](https://img.shields.io/badge/params-1.54B-blueviolet)
+**SakThai Plus 1.5B** is a **merged** rsLoRA fine-tune of [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct), optimized for structured tool-calling and instruction-following.
 
-## Model Description
-
-SakThai Plus 1.5B is a **rsLoRA-merged** instruction-tuned model for tool calling and agentic workflows. It builds on the SakThai Context lineage with improved tool-use precision, conversational grounding, and instruction adherence.
-
-- **Base model:** [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct)
-- **Fine-tuning:** Full-parameter merge of rsLoRA adapters trained on curated tool-calling datasets
-- **Architecture:** Qwen2ForCausalLM — 28 layers, 12 attention heads, 2 KV heads, hidden size 1536, intermediate 8960
-- **Parameters:** ~1.54B (bf16, 2.88 GiB on disk)
-- **Format:** Qwen2.5 chat template (standard)
-- **Context length:** 32K tokens (max_position_embeddings: 32768)
+- **Base model:** Qwen/Qwen2.5-1.5B-Instruct
+- **Method:** rsLoRA → merged full weights
+- **Context length:** 32,768 tokens
 - **License:** Apache 2.0
-
-## Intended Use
-
-- Tool/function calling agents and assistants
-- Conversational AI with structured output generation
-- Lightweight on-device agent deployments (CPU-friendly at 1.5B)
-- Prototyping agentic workflows before scaling to larger models
+- **Format:** Qwen2.5 chat template
 
 ## Training Data
 
-The model was fine-tuned on a curated mixture of:
-
 | Dataset | Rows | Description |
 |---------|------|-------------|
-| **Nanthasit/sakthai-combined-v11** | 2,003 | Multi-source tool-calling examples (subset of v10) |
+| **Nanthasit/sakthai-combined-v11** | 2,003 | Multi-source tool-calling examples |
 | **Nanthasit/SimpleToolCalling** | 2,002 | Structured function-calling examples |
 
-**Training method:** rsLoRA (rank-stabilized) via TRL SFTTrainer, merged to full weights.
-
-| Hyperparameter | Value |
-|---------------|-------|
-| r / alpha / dropout | 16 / 32 / 0.05 |
-| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj (all 7 linear) |
-| Optimizer | AdamW (8-bit) |
-| Learning rate | 2e-4 |
-| Epochs | 3 |
-| Precision | bf16 |
-| Hardware | Free T4 GPU (Kaggle / Colab / HF Inference), $0 budget |
-
-## Performance
-
-Benchmark results across tool-calling, reasoning, and math tasks:
-
-| Task | Metric | Score | Verified | Source |
-|------|--------|:-----:|:--------:|--------|
-| Tool Calling (send_email) | Success Rate | **1.0** (3/3) | ✅ Yes | llama.cpp 3-trial |
-| Tool Calling | Valid JSON Args | **1.0** (3/3) | ✅ Yes | llama.cpp 3-trial |
-| Tool Calling | Correct Answer | **1.0** (3/3) | ✅ Yes | llama.cpp 3-trial |
-| Tool Selection (v2) | Selection Accuracy | **84.8%** | ❌ No | sakthai-bench-v2 |
-| Tool Selection (v2) | Arguments Accuracy | **33.7%** | ❌ No | sakthai-bench-v2 |
-| Tool Selection (v2) | Strict Accuracy | **33.7%** | ❌ No | sakthai-bench-v2 |
-| WinoGrande (WSC) | Accuracy | **59.6%** | ❌ No | lighteval |
-| HellaSwag | Accuracy | **34.0%** | ❌ No | lighteval |
-| GSM8K (Math) | Accuracy | **50.9%** | ❌ No | lighteval |
+## Benchmarks
 
-**Key takeaways:**
-- **Tool calling performs reliably** — 3/3 on every trial in the verified `send_email` scenario. The model correctly formats `<tool_call>` XML with valid JSON arguments.
-- **Tool selection is strong** (84.8% selection accuracy on sakthai-bench-v2) but argument precision needs refinement (33.7% strict).
-- **Commonsense and math** scores reflect the base Qwen2.5-1.5B-Instruct capabilities with minor degradation from tool-calling specialization.
-
-> ⚠ Unverified scores are single-trial results from `.eval_results/`. Multi-trial replication is planned. The 3-trial tool-calling benchmark is the only verified result.
+| Task | Metric | Score | Verified |
+|:-----|-------:|------:|:--------|
+| Tool Calling | Tool Call Success Rate | 1.0 | ✅ Yes |
+| Tool Calling | Valid JSON Arguments | 1.0 | ✅ Yes |
+| Tool Calling | Correct Answer Rate | 1.0 | ✅ Yes |
+| Tool Selection (v2) | Selection Accuracy | 84.8% | ❌ No |
+| Tool Selection (v2) | Strict Accuracy | 33.7% | ❌ No |
+| Commonsense | WinoGrande | 59.6% | ❌ No |
+| Commonsense | HellaSwag | 34.0% | ❌ No |
+| Math | GSM8K | 50.9% | ❌ No |
 
 ## Quick Start
 
-### Python — Direct Transformers
-
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
 model_id = "Nanthasit/sakthai-plus-1.5b"
-tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
-model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
 
 messages = [
     {"role": "system", "content": "You are a helpful assistant with tool-calling capabilities."},
     {"role": "user", "content": "What's the weather in Bangkok?"}
 ]
 inputs = tokenizer.apply_chat_template(messages, tokenize=True, return_tensors="pt")
-outputs = model.generate(inputs, max_new_tokens=256, temperature=0.3, do_sample=True)
+outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.3, do_sample=True)
 print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 ```
 
-Expected output format:
-```
-<tool_call>{'name': 'get_weather', 'arguments': {'location': 'Bangkok'}}</tool_call>
-```
-
-### Python — Tool-Calling with Multiple Tools
+## Notes
 
-This model uses the `<tool_call>` XML dialect (v2 format). Provide tool definitions in the system prompt:
+- 3/3 verified tool-calling score measured with llama.cpp q4_k_m.
+- Tool selection is strong, but argument accuracy needs refinement.
+- Unverified scores are single-trial; multi-trial replication is planned.
+- Trained on free T4 credits; no paid compute was used.
 
-```python
-messages = [
-    {"role": "system", "content": """You have access to these tools:
-- get_weather(location: str) — get weather for a city
-- send_email(to: str, subject: str, body: str) — send an email
-- search_web(query: str) — search the internet
-
-Respond with the appropriate tool call in <tool_call> XML format."""},
-    {"role": "user", "content": "Send an email to Beer with the subject 'Status update' and body 'The model is running well.'"}
-]
-
-inputs = tokenizer.apply_chat_template(messages, tokenize=True, return_tensors="pt")
-outputs = model.generate(inputs, max_new_tokens=256, temperature=0.3, do_sample=True)
-response = tokenizer.decode(outputs[0], skip_special_tokens=True)
-# Extracts: <tool_call>{'name': 'send_email', 'arguments': {...}}</tool_call>
-```
-
-### llama.cpp / Ollama (GGUF)
-
-For CPU inference, use the [llama.cpp](https://github.com/ggml-ai/llama.cpp) compatible GGUF version or convert manually:
-
-```bash
-# Convert to GGUF via llama.cpp
-git clone https://github.com/ggml-ai/llama.cpp
-cd llama.cpp
-python3 convert_hf_to_gguf.py Nanthasit/sakthai-plus-1.5b --outfile sakthai-plus-1.5b-f16.gguf
-
-# Run with llama-cli
-./llama-cli -m sakthai-plus-1.5b-f16.gguf \
-  -p "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nWhat's the weather in Bangkok?<|im_end|>\n<|im_start|>assistant\n" \
-  -no-cnv --simple-io -n 256 -t 2
-```
-
-## Model Family
-
-This model is part of the **SakThai Model Family** — 25 models across sizes and specializations:
-
-| Variant | Type | Description |
-|---------|------|-------------|
-| **sakthai-plus-1.5b** | ✅ This | Merged instruct, tool calling |
-| **sakthai-plus-1.5b-lora** | LoRA adapter | Lightweight adapter for the same base |
-| **sakthai-plus-1.5b-coder** | Skeleton | Code-specialized variant (base: Qwen2.5-Coder-1.5B) |
-| **sakthai-context-1.5b-tools-v2** | Model | Improved tool-use variant |
-| **sakthai-context-1.5b-merged-v2** | Model | Full weights, merged from v2 LoRA |
-| **Family size** | 25 public models across merged, LoRA, vision, embedding, TTS, and coder variants |
-| **Current downloads** | 244 |
-| **Last updated** | 2026-07-31 |
-| [View full collection →](https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02) |
-
-## Limitations
-
-- **Language:** Optimized for English tool-calling scenarios. Multilingual prompts may degrade performance.
-- **Knowledge cutoff:** The base Qwen2.5-1.5B-Instruct has a knowledge cutoff of ~2024. The fine-tuning dataset (2025–2026 tool-calling examples) does not expand world knowledge.
-- **Argument precision:** sakthai-bench-v2 shows 84.8% selection accuracy but only 33.7% strict accuracy — the model often selects the right tool but generates malformed arguments.
-- **Hallucination risk:** Like all language models, it may generate plausible-sounding but incorrect tool arguments. Always validate tool calls before execution in production.
-- **Not a general reasoning model:** Benchmarks on commonsense (HellaSwag 34.0%, WinoGrande 59.6%) are below larger models. This is a tool-calling specialist, not a general-purpose reasoner.
-- **Unverified benchmarks:** 8 of 9 metric entries are single-trial and unverified. Verified scores (3/3 tool-calling) were measured on llama.cpp q4_k_m (CPU, 2 threads).
-- **Free-tier constraints:** Trained entirely on free T4 GPU credits. No production-grade training infra was used.
-
-## Citation
-
-```bibtex
-@misc{sakthai-plus-1.5b,
-  author = {Burankum, Nanthasit},
-  title = {SakThai Plus 1.5B: Enhanced Tool-Calling Language Model},
-  year = {2026},
-  publisher = {Hugging Face},
-  howpublished = {\url{https://huggingface.co/Nanthasit/sakthai-plus-1.5b}},
-  note = {Fine-tuned from Qwen2.5-1.5B-Instruct with rsLoRA}
-}
-```
+## SakThai Family
 
-## License
+| Repo | Downloads | Pipeline |
+|-----:|----------:|:---------|
+| [sakthai-plus-1.5b-lora](https://huggingface.co/Nanthasit/sakthai-plus-1.5b-lora) | — | text-generation |
+| [sakthai-context-1.5b-tools-v2](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools-v2) | — | text-generation |
 
-Apache 2.0