SKT AI LABS
SKT-ST-X-0-3B-V1
COMPACT MOE POWERHOUSE
3B Total Params • 1.1B Active • English & Hindi
A highly efficient Small Language Model (SLM) built on Mixtral MoE architecture for stability. Delivers intelligent responses with a tiny footprint.
🏗️ Model Architecture
| Total Parameters | ~3 Billion |
| Active Parameters | ~1.1 Billion (2 Experts/Token) |
| Architecture | Mixture of Experts (MoE) |
| Number of Experts | 4 |
| Context Length | 8K Tokens |
| Training Data | 40B Tokens (SKT-OMNI-CORPUS-2T) |
✨ Key Capabilities
- Bilingual Mastery: Fluent in both English and Hindi.
- Efficient Reasoning: Logical thinking and problem solving despite small size.
- Basic Coding: Python scripts, algorithms, and logic debugging.
- Creative Writing: Stories, poems, and roleplay with personality.
- Knowledge QA: Accurate general knowledge retrieval.
🛠️ Quick Start Guide
Installation
pip install transformers accelerate torch peft bitsandbytes
Basic Inference
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "sKT-Ai-Labs/SKT-ST-X-0-3B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
torch_dtype=torch.float16
)
prompt = "What is Quantum Physics?"
formatted = f"<|user|>\n{prompt}\n<|assistant|>\n"
inputs = tokenizer(formatted, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=100)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response.split("<|assistant|>")[-1].strip())
⚡ 4-bit Quantization (Low VRAM)
from transformers import BitsAndBytesConfig
quant_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_compute_dtype=torch.float16)
model = AutoModelForCausalLM.from_pretrained(
model_id,
quantization_config=quant_config,
device_map="auto"
)
❤️ Support Our Mission
🙏 Drop a Heart ❤ For Our Hard Work!
If you believe in the vision of Sovereign Indian AI, please show your support by dropping a heart below. Your encouragement fuels our journey!
Kindly follow us for more updates and contribute to our open-source journey!
📜 License & Citation
This model is released under the Apache-2.0 License.
@misc{SKT-ST-X-0-3B,
author = {SKT AI LABS, India},
title = {SKT-ST-X-0-3B: A Compact Mixture of Experts Model},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/sKT-Ai-Labs/SKT-ST-X-0-3B}
}
Made with ❤️ by SKT AI LABS
Support: support@sktailabs.in
- Downloads last month
- 44