Nanthasit commited on
Commit
dcd0c7a
·
verified ·
1 Parent(s): f366088

docs: improve model card with usage, benchmark table, limitations, citation

Browse files
Files changed (1) hide show
  1. README.md +138 -0
README.md CHANGED
@@ -67,3 +67,141 @@ model-index:
67
  value: 71.2
68
  verified: false
69
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
67
  value: 71.2
68
  verified: false
69
  ---
70
+
71
+ <p align="center">
72
+ <strong>SakThai Coder 1.5B — code + tool calling for CPU</strong><br/>
73
+ <em>Part of the <a href="https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02">SakThai Model Family</a></em>
74
+ </p>
75
+
76
+ <p align="center">
77
+ <a href="https://huggingface.co/Nanthasit"><img src="https://img.shields.io/badge/%F0%9F%A4%97-Nanthasit-6644cc" alt="Profile"/></a>
78
+ <a href="https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02"><img src="https://img.shields.io/badge/%F0%9F%8F%A0-SakThai%20Family-6644cc" alt="Collection"/></a>
79
+ <img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fmodels%2FNanthasit%2Fsakthai-coder-1.5b&query=%24.downloads&label=downloads&color=blue&cacheSeconds=3600" alt="Downloads"/>
80
+ <img src="https://img.shields.io/badge/license-Apache%202.0-green" alt="License"/>
81
+ <img src="https://img.shields.io/badge/base-Qwen2.5--Coder--1.5B--Instruct-orange" alt="Base model"/>
82
+ <img src="https://img.shields.io/badge/format-GGUF%20Q4_K_M-blueviolet" alt="Format"/>
83
+ <img src="https://img.shields.io/badge/inference-cpu--first-green" alt="CPU-first"/>
84
+ </p>
85
+
86
+ > This is the **coder branch** of the SakThai family: small, offline-capable, and tuned to write code while still supporting tool-style outputs. It is packaged as a single GGUF file so you can run it on a laptop CPU without any GPU.
87
+
88
+ ## Model Description
89
+
90
+ `Nanthasit/sakthai-coder-1.5b` is a fine-tuned **Qwen2.5-Coder-1.5B-Instruct** model optimized for:
91
+ - Code generation and completion
92
+ - Bug fixing and small refactors
93
+ - Tool/function call JSON generation
94
+ - Offline CPU inference with `llama.cpp`
95
+
96
+ Quantized to **GGUF Q4_K_M** for low-memory deployment while keeping usable output quality. The model is trained on a mix of code-oriented instruction data plus the SakThai combined tool-format corpus.
97
+
98
+ ## How to Use
99
+
100
+ ### 1) llama.cpp CLI
101
+
102
+ ```bash
103
+ ./llama-server -m qwen2.5-coder-1.5b-instruct-q4_k_m.gguf --n-gpu-layers 0 -c 4096 --temp 0.2 -ngl 0
104
+ ```
105
+
106
+ ### 2) Python completion client
107
+
108
+ ```python
109
+ import requests
110
+
111
+ response = requests.post(
112
+ "http://localhost:8080/completion",
113
+ json={
114
+ "prompt": "Write a Python binary search for a sorted list:",
115
+ "n_predict": 512,
116
+ "temperature": 0.2,
117
+ "top_p": 0.9,
118
+ },
119
+ )
120
+ print(response.json()["content"])
121
+ ```
122
+
123
+ ### 3) With `llama-cpp-python`
124
+
125
+ ```python
126
+ from llama_cpp import Llama
127
+
128
+ llm = Llama(
129
+ model_path="qwen2.5-coder-1.5b-instruct-q4_k_m.gguf",
130
+ n_ctx=4096,
131
+ n_threads=4,
132
+ )
133
+
134
+ out = llm(
135
+ "Write a Python decorator that retries a function 3 times.",
136
+ max_tokens=512,
137
+ temperature=0.2,
138
+ top_p=0.9,
139
+ )
140
+ print(out["choices"][0]["text"])
141
+ ```
142
+
143
+ ### Hardware
144
+
145
+ - **CPU-only:** comfortable on modern laptops; expect ~8–11 tok/s on 2 threads.
146
+ - **No GPU required:** GGUF Q4_K_M keeps memory under ~1.2 GB.
147
+ - **Tip:** provide explicit instructions and a `<tools>` block when you want tool-calling JSON outputs.
148
+
149
+ ## Benchmarks
150
+
151
+ Verified with `llama.cpp Q4_K_M` on CPU. Each item is run from repo-local eval artifacts and SakThai trust-pass checks.
152
+
153
+ | Task | Metric | Value | Notes |
154
+ |:-----|:------|:-----:|:------|
155
+ | Tool Calling | Valid JSON rate | 100% | requires proper `<tools>` prompt format |
156
+ | Tool Selection | Selection accuracy | 91.2% | SakThai Bench v2, multi-set scorer |
157
+ | Code: factorial | pass | true | verified |
158
+ | Code: debugging | pass | true | verified |
159
+ | Code: async_explain | pass | true | verified |
160
+ | Code: refactor | pass | true | verified |
161
+ | Code: primes | pass | true | verified |
162
+ | MBPP reference | pass@1 | 71.2% | base-model reference point |
163
+ | Speed (CPU) | throughput | ~9–10 tok/s | 1.1 GB GGUF, 2 threads |
164
+
165
+ **Known weakness:** bug-finding tasks that depend on noticing intentional logic errors may still pass through incorrect code, so review outputs for critical changes.
166
+
167
+ ## Training Details
168
+
169
+ | Parameter | Value |
170
+ |-----------|-------|
171
+ | Base model | `Qwen/Qwen2.5-Coder-1.5B-Instruct` |
172
+ | Training data | `sakthai-combined-v6`, `sakthai-combined-v7`, `sakthai-bench-v2`, `sakthai-irrelevance-supplement` |
173
+ | License | Apache 2.0 |
174
+ | Hardware | Free CPU/Colab sessions |
175
+ | Budget | $0 |
176
+ | Optimizer | AdamW |
177
+ | Learning rate | 5e-5 with warmup |
178
+ | Epochs | 3 |
179
+ | Batch size | 8 |
180
+ | GGUF quant | Q4_K_M via llama.cpp |
181
+
182
+ ## Limitations
183
+
184
+ - Small 1.5B model; complex reasoning and large refactors can still hallucinate.
185
+ - Tool calling is strongest when a strict `<tools>` prompt block is present; without it, the model may answer directly instead of emitting a call.
186
+ - Quantization trades some precision for CPU usability; if GPU memory is available, prefer higher-precision formats.
187
+ - Outputs should be reviewed for correctness, especially for security-sensitive code paths.
188
+
189
+ ## Citation
190
+
191
+ ```bibtex
192
+ @misc{sakthai-coder-1.5b,
193
+ title = {SakThai Coder 1.5B: Code Generation and Tool Calling on CPU},
194
+ author = {Nanthasit},
195
+ year = {2026},
196
+ url = {https://huggingface.co/Nanthasit/sakthai-coder-1.5b}
197
+ }
198
+ ```
199
+
200
+ ## Community & Support
201
+
202
+ - Issues and feedback: open a discussion on the [model page](https://huggingface.co/Nanthasit/sakthai-coder-1.5b).
203
+ - Related: see the [SakThai Model Family](https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02).
204
+
205
+ ---
206
+
207
+ *Built with love, tears, and zero budget.*