Instructions to use zenlm/zen3-omni with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zenlm/zen3-omni with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="zenlm/zen3-omni")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("zenlm/zen3-omni")
model = AutoModelForMultimodalLM.from_pretrained("zenlm/zen3-omni", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use zenlm/zen3-omni with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zenlm/zen3-omni"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zenlm/zen3-omni",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/zenlm/zen3-omni

SGLang

How to use zenlm/zen3-omni with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zenlm/zen3-omni" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zenlm/zen3-omni",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zenlm/zen3-omni" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zenlm/zen3-omni",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use zenlm/zen3-omni with Docker Model Runner:
```
docker model run hf.co/zenlm/zen3-omni
```

zen3-omni

Commit History

docs: honest base attribution / canonical-name note

085a0b3
verified

zeekay commited on Jul 14

Add LICENSE and NOTICE files for license compliance

55ed0db

hanzo-dev commited on Jun 26

Add NOTICE: Copyright Zen Authors

78cb308
verified

zeekay commited on Jun 15

fix(readme): frontmatter:language

7c1a92a
verified

antjeworring commited on May 30

Update model card: fix YAML tags, add zen/zenlm branding

573986a
verified

zeekay commited on Feb 28

Update model card: Add omni/zenlm tags, add local usage example

33f26ff
verified

zeekay commited on Feb 28

Add model weights

513e530
verified

zeekay commited on Feb 26

Update Zen3 Omni model card — API-only

c5d1f5d
verified

zeekay commited on Feb 24

Update Zen3 Omni model card

bbf5566
verified

zeekay commited on Feb 24

Add model card

7bfcee7
verified

zeekay commited on Feb 24

initial commit

b6e06cb
verified

zeekay commited on Feb 24

Commit History

docs: honest base attribution / canonical-name note 085a0b3 verified

Add LICENSE and NOTICE files for license compliance 55ed0db

Add NOTICE: Copyright Zen Authors 78cb308 verified

fix(readme): frontmatter:language 7c1a92a verified

Update model card: fix YAML tags, add zen/zenlm branding 573986a verified

Update model card: Add omni/zenlm tags, add local usage example 33f26ff verified

Add model weights 513e530 verified

Update Zen3 Omni model card — API-only c5d1f5d verified

Update Zen3 Omni model card bbf5566 verified

Add model card 7bfcee7 verified

initial commit b6e06cb verified

docs: honest base attribution / canonical-name note

085a0b3
verified

Add LICENSE and NOTICE files for license compliance

55ed0db

Add NOTICE: Copyright Zen Authors

78cb308
verified

fix(readme): frontmatter:language

7c1a92a
verified

Update model card: fix YAML tags, add zen/zenlm branding

573986a
verified

Update model card: Add omni/zenlm tags, add local usage example

33f26ff
verified

Add model weights

513e530
verified

Update Zen3 Omni model card — API-only

c5d1f5d
verified

Update Zen3 Omni model card

bbf5566
verified

Add model card

7bfcee7
verified

initial commit

b6e06cb
verified