Instructions to use TeluguHouseCollective/Gemma-2B-Telugu_Instruct_Finetuned with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TeluguHouseCollective/Gemma-2B-Telugu_Instruct_Finetuned with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TeluguHouseCollective/Gemma-2B-Telugu_Instruct_Finetuned")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TeluguHouseCollective/Gemma-2B-Telugu_Instruct_Finetuned")
model = AutoModelForCausalLM.from_pretrained("TeluguHouseCollective/Gemma-2B-Telugu_Instruct_Finetuned", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use TeluguHouseCollective/Gemma-2B-Telugu_Instruct_Finetuned with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TeluguHouseCollective/Gemma-2B-Telugu_Instruct_Finetuned"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TeluguHouseCollective/Gemma-2B-Telugu_Instruct_Finetuned",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/TeluguHouseCollective/Gemma-2B-Telugu_Instruct_Finetuned

SGLang

How to use TeluguHouseCollective/Gemma-2B-Telugu_Instruct_Finetuned with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TeluguHouseCollective/Gemma-2B-Telugu_Instruct_Finetuned" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TeluguHouseCollective/Gemma-2B-Telugu_Instruct_Finetuned",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TeluguHouseCollective/Gemma-2B-Telugu_Instruct_Finetuned" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TeluguHouseCollective/Gemma-2B-Telugu_Instruct_Finetuned",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Desktop
Docker Model Runner
How to use TeluguHouseCollective/Gemma-2B-Telugu_Instruct_Finetuned with Docker Model Runner:
```
docker model run hf.co/TeluguHouseCollective/Gemma-2B-Telugu_Instruct_Finetuned
```

Model Card for Model ID

Gemma 2B Model Finetuned on two Telugu Instruct Datasets:

Telugu-LLM-Labs/yahma_alpaca_cleaned_telugu_filtered_and_romanized
Telugu-LLM-Labs/teknium_GPTeacher_general_instruct_telugu_filtered_and_romanized

Model Details

Model Description

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

Developed by: Sai Teja Mummadi
Language(s) (NLP): English, Telugu (Original Script and Transliterated(Romanized))
Finetuned from model: google/gemma-2b

Uses

Text Generation, Telugu Chatbot, Telugu Text Generation

Downstream Use [optional]

Telugu Text Summarization, Further Finetuning on Telugu Datasets

Bias, Risks, and Limitations

Model is still under development, might need further finetuning on other datasets

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{}

### Input:
{}

### Response:
{}"""

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

model_name = "TeluguHouseCollective/Gemma-2B-Telugu_Instruct_Finetuned"

tokenizer = AutoTokenizer.from_pretrained(model_name, padding_side="right")
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16).to(device)

inputs = tokenizer(
[
    alpaca_prompt.format(
        "fibonacci series rayadaniki python program ivvu", # instruction
        "", # input
        "", # output - leave this blank for generation!
    )
], return_tensors = "pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 256, use_cache = True)
tokenizer.batch_decode(outputs)

Model Output was as following:

Response:\nfibonnaci rayadaniki python program ikkada vundi:\n\n\'\'\'\n
def fibonacci(n):\n    """\n    fibonacci series rayadaniki python program.\n    """\n
a = 0\n    b = 1\n    series = [a, b]\n
for i in range(2, n + 1):\n
series.append(a + b)\n
a, b = b, a + b\n
return series\n\n#
fibonacci series rayadaniki 10 vibhinna sankhyalanu rayandi\n
series = fibonacci(10)\nprint(series)\n\'\'\'\n\n
e program fibonacci series rayadaniki python language upayogistamdi
mariyu fibonacci(n) function upayoginchi fibonacci(n) sankhyanu
rayadaniki fibonacci(n) function upayogistamdi.
fibonacci(n) function yokka prarambha viluvanu 0 mariyu 1 set cheyadam dwara prarambhamavuthundi,
mariyu idi fibonacci(n) yokka prarambha viluvanu 0 mariyu 1 nundi n nundi 1 nundi 0 varaku prarambhama

Another input in telugu

inputs = tokenizer(
[
    alpaca_prompt.format(
        "ఆరోగ్యంగా ఉండాలి అంటే ఎం చేయాలి?", # instruction
        "", # input
        "", # output - leave this blank for generation!
    )
], return_tensors = "pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 256, use_cache = True)
tokenizer.batch_decode(outputs)

Model Output was as following:

### Response:
oka nirdishta anubhavanni batti, miru aaharam mariyu poshanalapai drishti pettavachu. kani, oka nirdishta anubhavanni batti, miru aaharam mariyu poshanalapai drishti pettavachu.

meeru aaharam mariyu poshanalapai drishti pettavachchu,
endukante idi mee aarogyanni meruguparachadamla sahayapaduthundi.
meeru aaharam mariyu poshanalapai drishti pettavachchu, endukante idi mee sarirak srama,
nidra mariyu manasika aarogyanni meruguparachadamla sahayapaduthundi.

meeru aaharam mariyu poshanalapai drishti pettavachchu,
endukante idi mee sarirak srama, nidra mariyu manasika aarogyanni meruguparachadamla sahayapaduthundi.
meeru aaharam mariyu poshanalapai drishti pettavachchu, endukante idi mee sarirak srama,
nidra mariyu manasika aarogyanni meruguparachad

Model Card Authors [optional]

Sai Teja Mummadi

Downloads last month: 23

Safetensors

Model size

3B params

Tensor type

F16

TeluguHouseCollective
/

Gemma-2B-Telugu_Instruct_Finetuned