Instructions to use KETI-NLP/keti-llama-7b-v0.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use KETI-NLP/keti-llama-7b-v0.1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="KETI-NLP/keti-llama-7b-v0.1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("KETI-NLP/keti-llama-7b-v0.1")
model = AutoModelForCausalLM.from_pretrained("KETI-NLP/keti-llama-7b-v0.1", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use KETI-NLP/keti-llama-7b-v0.1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "KETI-NLP/keti-llama-7b-v0.1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KETI-NLP/keti-llama-7b-v0.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/KETI-NLP/keti-llama-7b-v0.1

SGLang

How to use KETI-NLP/keti-llama-7b-v0.1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "KETI-NLP/keti-llama-7b-v0.1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KETI-NLP/keti-llama-7b-v0.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "KETI-NLP/keti-llama-7b-v0.1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KETI-NLP/keti-llama-7b-v0.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use KETI-NLP/keti-llama-7b-v0.1 with Docker Model Runner:
```
docker model run hf.co/KETI-NLP/keti-llama-7b-v0.1
```

KETI Llama 7B v0.1

KETI Llama 7B v0.1 is a long-context causal language model released under the Hugging Face repository KETI-AIR/keti-llama-7b-v0.1.

This checkpoint was produced from the local base model with the following training pipeline:

SFT on instruction data with 32K sequence packing.
DPO on preference data.
RL/GRPO on RL data.

The uploaded artifact is the merged Hugging Face model from:

outputs/llama-8b-keti-dpo-rl-merged

Model Details

Architecture: LlamaForCausalLM
Parameters: 8B-class
Context length in config: 131,072 tokens
Hidden size: 4096
Layers: 32
Attention heads: 32
KV heads: 8
Vocabulary size: 128,256
Recommended dtype: bfloat16

Evaluation

Evaluation timestamp: 20260604_202553

Category	Dataset	Version	Metric	Mode	Score
Core	core_average	-	naive_average	gen	27.77
Instruction Following	IFEval	353ae7	Prompt-level-strict-accuracy	gen	50.65
Math Calculation	aime2024	bc6078	accuracy	gen	16.67
Math Calculation	aime2025	5e9f4f	accuracy	gen	3.33
Math Calculation	math_prm800k_500	11c4b5	accuracy	gen	60.20
General Reasoning	bbh	-	naive_average	gen	11.87
General Reasoning	GPQA_diamond	5aeece	accuracy	gen	20.71
Knowledge	mmlu_pro	-	naive_average	gen	28.26
Code	openai_humaneval	dcae0e	humaneval_pass@1	gen	60.98
Code	lcb_code_generation	b5b6c5	pass@1	gen	6.00
Long Context Reasoning	leval	-	naive_average	gen	39.37
Long Context Reasoning	longbench	-	naive_average	gen	20.57
Long Context Reasoning	LongBenchv2	75fbba	accuracy	gen	24.85
Long Context Reasoning	keti_long_ctx_gutenberg	-	naive_average	gen	17.62

Quick Start

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "KETI-AIR/keti-llama-7b-v0.1"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {"role": "user", "content": "Explain why long-context reasoning is useful."}
]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=512,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

Intended Use

This model is intended for research and development on instruction following, code generation, mathematical reasoning, and long-context generation tasks.

Limitations

The model can generate incorrect, unsafe, or biased content. Users should evaluate the model for their own deployment setting and apply appropriate safety filters and human review where needed.

Training Framework

Transformers: 5.8.1
PyTorch: 2.11.0+cu130
Datasets: 4.8.5
Tokenizers: 0.22.2
TRL: 1.4.0

Citation

If you use this model, please cite the corresponding KETI-AIR release and the training/evaluation resources used in your work.

Downloads last month: 8

Safetensors

Model size

8B params

Tensor type

F32