Instructions to use toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot")
model = AutoModelForMultimodalLM.from_pretrained("toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot

SGLang

How to use toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot with Docker Model Runner:
```
docker model run hf.co/toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot

Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot is a fine-tuned conversational Large Language Model (LLM) for the Khasi language. Built upon Gemma 4 E2B IT QAT, the model is designed to provide fluent Khasi conversations, instruction following, translation, and reasoning while preserving cultural and linguistic context.

The model has been instruction-tuned using a large, curated Khasi dataset combined with high-quality reasoning examples, enabling it to perform both everyday conversational tasks and more complex problem-solving.

Model Description

The primary goal of this model is to improve AI support for the Khasi language while maintaining strong reasoning capabilities inherited through instruction tuning.

The model is suitable for:

General conversation
Khasi question answering
Instruction following
English ↔ Khasi translation
Educational assistance
Writing assistance
Logical reasoning
Cultural and language preservation

Training Dataset

The model was fine-tuned using:

Dataset: toiar/khasi-instruction-response-v2

Dataset Statistics

Category	Count
Total Instruction-Response Pairs	77,810
Khasi-Specific Data	72,810
External English Reasoning & General Data	5,000

The dataset is an expanded and improved version of the previous release, focusing on higher-quality instruction-response pairs, broader linguistic coverage, and improved reasoning performance.

It combines:

Khasi conversations
Instruction following
Translation pairs
Educational content
Cultural knowledge
General knowledge
Reasoning examples adapted from English-language sources

The reasoning examples are included to improve logical problem-solving while maintaining fluent Khasi responses.

Intended Uses

This model is intended for:

Khasi conversational assistants
AI chatbots
Language learning applications
Translation systems
Educational tools
Research on low-resource languages
Digital preservation of Khasi

Limitations

Although the model has been optimized for Khasi, it may still:

Produce incorrect or hallucinated information
Make reasoning mistakes on complex tasks
Struggle with highly specialized domains
Reflect biases present in the training data
Perform inconsistently on topics not represented in the dataset

Responses should be verified when factual accuracy is important.

Citation

If you use this model in your research or applications, please cite:

@misc{ri_gemma_e2b_it_qat_khasi_chatbot,
  title        = {Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot},
  author       = {Toiar},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot}}
}

Downloads last month: 59

Safetensors

Model size

5B params

Tensor type

BF16

Model tree for toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot

Base model

google/gemma-4-E2B

Finetuned

google/gemma-4-E2B-it

Finetuned

unsloth/gemma-4-E2B-it-qat-q4_0-unquantized

Finetuned

(6)

this model

Quantizations

1 model

toiar
/

Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot

You need to agree to share your contact information to access this model

Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot

Model Description

Training Dataset

Dataset Statistics

Intended Uses

Limitations

Citation

Model tree for toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot

Dataset used to train toiar/Ri-Gemma-E2B-IT-QAT-Khasi-Chatbot