add README.md

.gitattributes

@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 tokenizer.json filter=lfs diff=lfs merge=lfs -text
+pathumma-thaillm-300.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,184 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+---
+
+![](pathumma-thaillm-300.png)
+
+# Pathumma-ThaiLLM-Think-3.0.0
+
+Post-trained Thai Large Language Model built upon the foundation model from the Thai national initiative [**ThaiLLM**](https://www.thaillm.or.th).
+
+This release applies multi-stage Supervised Fine-Tuning (SFT) to enhance:
+
+- Instruction following
+- Structured tool / function calling
+- Mathematical and coding competence
+- Multi-step analytical capability
+- Thai–English bilingual robustness
+
+---
+
+## Training Strategy
+
+Post-training is organized into **two stages**:
+
+- **Stage 1:** Instruction & Tool-Calling Alignment  
+- **Stage 2:** Reasoning Specialization  
+
+For selected corpora, only curated subsets were used to maintain domain balance.
+
+---
+
+## Stage 1: Instruction & Tool-Calling Alignment
+
+Focus areas:
+
+- Instruction compliance  
+- Structured tool-call formatting  
+- General Thai task robustness  
+- STEM-oriented instruction alignment  
+
+### Datasets
+
+| Dataset | Training Subset Size | Full Dataset Size | Domain | License |
+|----------|---------------------|------------------|----------|----------|
+| beyoru/ToolCall_synthetic_qwen3 | 60,000 | 60,000 | Tool | Apache-2.0 |
+| airesearch/WangchanX-FLAN-v6 | 2,000,000 | 13,619,450 | General | Mixed |
+| nvidia/OpenMathInstruct-2 | 1,000,000 | 14,000,000 | STEM | CC-BY-4.0 |
+| jdaddyalbs/playwright-mcp-toolcalling | 1,750 | 1,750 | Tool | MIT |
+| BitAgent/tool_calling | 551,000 | 551,000 | Tool | MIT |
+
+<br>
+
+## Stage 2: Reasoning Specialization
+
+Focus areas:
+
+- Multi-step mathematical analysis  
+- Code understanding and synthesis  
+- Structured analytical responses  
+- Tool-calling with explicit reasoning traces  
+- Thai reasoning distillation  
+
+### Datasets
+
+| Dataset | Training Subset Size | Full Dataset Size | Domain | License |
+|----------|---------------------|------------------|----------|----------|
+| nvidia/OpenMathReasoning | 500,000 | 4,920,000 | STEM | CC-BY-4.0 |
+| nvidia/OpenCodeReasoning | 585,000 | 585,000 | Coding | CC-BY-4.0 |
+| natolambert/GeneralThought-430K-filtered | 337,579 | 337,579 | General | MIT |
+| Jofthomas/hermes-function-calling-thinking-V1 | 3,570 | 3,570 | Tool | MIT |
+| open-thoughts/OpenThoughts3-1.2M | 1,200,000 | 1,200,000 | STEM | Apache-2.0 |
+| scb10x/typhoon-r1-sft-data | 23,851 | 23,851 | General | Custom |
+| iapp/Thai-R1-Distill-SFT | 10,000 | 10,000 | General | Custom |
+| nvidia/Nemotron-Post-Training-Dataset-v1 | 310,000 | 310,000 | Tool | CC-BY-4.0 |
+
+---
+
+> **Note:** For selected datasets, curated subsets were employed to ensure balanced domain representation.
+
+---
+
+## Methodology
+
+- Base model: ThaiLLM foundation model  
+- Training objective: Supervised Fine-Tuning (SFT)  
+- Two-stage curriculum design  
+- Domain-balanced optimization  
+- Tool-call schema alignment  
+- Thai reasoning distillation
+
+---
+
+## Compute Infrastructure
+
+Training was conducted on the LANTA high-performance computing cluster, utilizing 16 nodes (64×A100 40GB GPUs) for distributed large-scale post-training.
+## Capabilities
+
+- Thai instruction compliance  
+- Structured JSON tool invocation  
+- Mathematical problem solving  
+- Code generation and analysis  
+- Multi-step analytical tasks  
+- Thai–English bilingual support
+
+---
+
+## Limitations
+
+- May hallucinate if tool schema is incomplete  
+- Performance on long analytical chains may degrade without retrieval  
+- Domain coverage depends on included corpora  
+
+---
+<br>
+
+# Quickstart
+The code of Qwen3 has been in the latest Hugging Face `transformers` and we advise you to use the latest version of `transformers`.
+With `transformers<4.51.0`, you will encounter the following error:
+```
+KeyError: 'qwen3'
+```
+The following contains a code snippet illustrating how to use the model generate content based on given inputs. 
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+model_name = "nectec/pathumma-thaillm-8b-think-3.0.0"
+# load the tokenizer and the model
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto"
+)
+# prepare the model input
+prompt = "ทำไมวงกลมถึงมี 360 องศา"
+messages = [
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True,
+)
+model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
+# conduct text completion
+generated_ids = model.generate(
+    **model_inputs,
+    max_new_tokens=32768
+)
+output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() 
+# parsing thinking content
+try:
+    # rindex finding 151668 (</think>)
+    index = len(output_ids) - output_ids[::-1].index(151668)
+except ValueError:
+    index = 0
+thinking_content = tokenizer.decode(output_ids[:index], skip_special_tokens=True).strip("\n")
+content = tokenizer.decode(output_ids[index:], skip_special_tokens=True).strip("\n")
+print("thinking content:", thinking_content) # no opening <think> tag
+print("content:", content)
+```
+For deployment, you can use `vllm>=0.8.5` to create an OpenAI-compatible API endpoint:
+
+```shell
+vllm serve nectec/pathumma-thaillm-8b-think-3.0.0 \
+  --enforce-eager \
+  --no-enable-chunked-prefill \
+  --tool-call-parser hermes
+```
+
+For local use, applications such as Ollama, LMStudio, and llama.cpp have also supported.
+
+## About the Project
+
+Pathumma-ThaiLLM-Think-3.0.0 is part of ongoing research toward sovereign Thai large language models optimized for analytical and tool-augmented intelligence.
+
+# Contributor Contract
+**LLM Team** 
+<br>
+Piyawat Chuangkrud (piyawat@it.kmitl.ac.th)<br>
+Chanon Utupon (s6401001620165@email.kmutnb.ac.th)<br>
+Jessada Pranee (jessada.pran@kmutt.ac.th)<br>
+Arnon Saeoung (anon.saeoueng@gmail.com)<br>
+Chaianun Damrongrat (chaianun.damrongrat@nectec.or.th)<br>
+Sarawoot Kongyoung (sarawoot.kongyoung@nectec.or.th)