Instructions to use Hcompany/Holotron4-30B-A3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Hcompany/Holotron4-30B-A3B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Hcompany/Holotron4-30B-A3B", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Hcompany/Holotron4-30B-A3B", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Hcompany/Holotron4-30B-A3B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Hcompany/Holotron4-30B-A3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Hcompany/Holotron4-30B-A3B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Hcompany/Holotron4-30B-A3B

SGLang

How to use Hcompany/Holotron4-30B-A3B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Hcompany/Holotron4-30B-A3B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Hcompany/Holotron4-30B-A3B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Hcompany/Holotron4-30B-A3B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Hcompany/Holotron4-30B-A3B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Hcompany/Holotron4-30B-A3B with Docker Model Runner:
```
docker model run hf.co/Hcompany/Holotron4-30B-A3B
```

h-tonywu commited on 9 days ago

Commit

194dea2

verified ·

1 Parent(s): 989fe06

Add license files and update model card

Browse files

Files changed (1) hide show

README.md +15 -18

README.md CHANGED Viewed

@@ -16,7 +16,7 @@ tags:
 > *Holo4* family:
 >
 > - Holo4-27B: Qwen3.8 dense architecture. [BF16](https://huggingface.co/Hcompany/Holo4-27B), [FP8](https://huggingface.co/Hcompany/Holo4-27B-FP8), [NVFP4](https://huggingface.co/Hcompany/Holo4-27B-NVFP4), and [Q4 GGUF](https://huggingface.co/Hcompany/Holo4-27B-GGUF).
-> - Holo4-35B-A3B: Qwen3.5 MoE architecture. [BF16](https://huggingface.co/Hcompany/Holo4-35B-A3B), [FP8](https://huggingface.co/Hcompany/Holo4-35B-A3B-FP8), [NVFP4](https://huggingface.co/Hcompany/Holo4-35B-A3B-NVFP4), and [Q4 GGUF](https://huggingface.co/Hcompany/Holo4-35B-A3B-GGUF).
 > - Holotron4-30B-A3B: NemotronH Nano Omni architecture. [BF16](https://huggingface.co/Hcompany/Holotron4-30B-A3B) [current] and [FP8](https://huggingface.co/Hcompany/Holotron4-30B-A3B-FP8).
 [![H Models API](https://img.shields.io/badge/H_Models_API-181717?style=for-the-badge)](https://hub.hcompany.ai/models-api/introduction)
@@ -40,7 +40,7 @@ tags:
 <video src="https://assets.hcompanyprod.fr/holo4/eiffel-holo4.mp4" poster="https://assets.hcompanyprod.fr/holo4/eiffel-holo4-poster.jpg" controls muted loop playsinline width="100%"></video>
-*Holo4* uses FreeCAD to build a replica of the Eiffel Tower. More examples in the [blog post](https://hcompany.ai/newsroom/holo4).
 <details>
 <summary>Prompt</summary>
@@ -65,23 +65,20 @@ Refer to the documentation for more details about:
 ## Performance
-*Holotron4* improves over its Nemotron base model on GUI, MCP, API, and coding tasks. In the blog's evaluation, its OSWorld score rises from 21.0% to 76.3%, and its AutomationBench score rises from 19.4% to 35.6%.
-### Benchmark results
-![Holo4 benchmark comparison table](https://assets.hcompanyprod.fr/holo4/hf/holo4-benchmark-table.png)
-Read the [blog post](https://hcompany.ai/newsroom/holo4) for more tables about the benchmark results.
-### Pareto plots
-These plots compare benchmark scores with cost per task. On OSWorld 2.0, *Holo4*-27B scores 61.7% at $1.22 per task, and *Holo4*-35B-A3B scores 30.9% at $0.61 per task.
-![Cost versus score on OSWorld 2.0](https://assets.hcompanyprod.fr/holo4/hf/holo4-cost-osworld2.png)
-On AutomationBench, *Holo4*-27B scores 45.4% at $0.05 per task, and *Holo4*-35B-A3B scores 34.5% at $0.02 per task.
-![Cost versus score on AutomationBench](https://assets.hcompanyprod.fr/holo4/hf/holo4-cost-automationbench.png)
 ### Open-source evaluation traces

 > *Holo4* family:
 >
 > - Holo4-27B: Qwen3.8 dense architecture. [BF16](https://huggingface.co/Hcompany/Holo4-27B), [FP8](https://huggingface.co/Hcompany/Holo4-27B-FP8), [NVFP4](https://huggingface.co/Hcompany/Holo4-27B-NVFP4), and [Q4 GGUF](https://huggingface.co/Hcompany/Holo4-27B-GGUF).
+> - Holo4-35B-A3B: Qwen3.6 MoE architecture. [BF16](https://huggingface.co/Hcompany/Holo4-35B-A3B), [FP8](https://huggingface.co/Hcompany/Holo4-35B-A3B-FP8), [NVFP4](https://huggingface.co/Hcompany/Holo4-35B-A3B-NVFP4), and [Q4 GGUF](https://huggingface.co/Hcompany/Holo4-35B-A3B-GGUF).
 > - Holotron4-30B-A3B: NemotronH Nano Omni architecture. [BF16](https://huggingface.co/Hcompany/Holotron4-30B-A3B) [current] and [FP8](https://huggingface.co/Hcompany/Holotron4-30B-A3B-FP8).
 [![H Models API](https://img.shields.io/badge/H_Models_API-181717?style=for-the-badge)](https://hub.hcompany.ai/models-api/introduction)
 <video src="https://assets.hcompanyprod.fr/holo4/eiffel-holo4.mp4" poster="https://assets.hcompanyprod.fr/holo4/eiffel-holo4-poster.jpg" controls muted loop playsinline width="100%"></video>
+This demo shows *Holo4*-27B, not this model, using FreeCAD to build a replica of the Eiffel Tower. More examples in the [blog post](https://hcompany.ai/newsroom/holo4).
 <details>
 <summary>Prompt</summary>
 ## Performance
+*Holotron4* improves over its base model, Nemotron 3 Nano Omni, on GUI workflows and in environments with MCP tools, APIs, or code sandboxes. Gains are absolute percentage points.
+<table width="100%">
+<thead>
+<tr style="border-bottom: 2px solid rgb(230, 126, 34);"><th style="background-color: rgb(255, 248, 239); color: #1f2937;">Benchmark</th><th style="background-color: rgb(255, 248, 239); color: #1f2937;">Interface</th><th style="background-color: rgb(255, 248, 239); color: #1f2937;">Nemotron 3 Nano Omni</th><th style="background-color: rgb(255, 248, 239); color: #1f2937;"><em>Holotron4</em>-30B-A3B</th><th style="background-color: rgb(255, 248, 239); color: #1f2937;">Gain</th></tr>
+</thead>
+<tbody>
+<tr><td style="background-color: white; color: #1f2937;">OSWorld</td><td style="background-color: white; color: #1f2937;">GUI</td><td style="background-color: white; color: #1f2937;">21.0</td><td style="background-color: white; color: #1f2937;">76.3</td><td style="background-color: white; color: #1f2937;">+55.3</td></tr>
+<tr><td style="background-color: white; color: #1f2937;">OSWorld 2.0</td><td style="background-color: white; color: #1f2937;">GUI and code</td><td style="background-color: white; color: #1f2937;">0.2</td><td style="background-color: white; color: #1f2937;">7.9</td><td style="background-color: white; color: #1f2937;">+7.7</td></tr>
+<tr><td style="background-color: white; color: #1f2937;">AutomationBench</td><td style="background-color: white; color: #1f2937;">MCP</td><td style="background-color: white; color: #1f2937;">19.4</td><td style="background-color: white; color: #1f2937;">35.6</td><td style="background-color: white; color: #1f2937;">+16.2</td></tr>
+<tr><td style="background-color: white; color: #1f2937;">PinchBench</td><td style="background-color: white; color: #1f2937;">Terminal</td><td style="background-color: white; color: #1f2937;">84.7</td><td style="background-color: white; color: #1f2937;">88.6</td><td style="background-color: white; color: #1f2937;">+3.9</td></tr>
+<tr><td style="background-color: white; color: #1f2937;">ALE (Linux, code)</td><td style="background-color: white; color: #1f2937;">Terminal</td><td style="background-color: white; color: #1f2937;">0.6</td><td style="background-color: white; color: #1f2937;">8.5</td><td style="background-color: white; color: #1f2937;">+7.9</td></tr>
+</tbody>
+</table>
 ### Open-source evaluation traces