Instructions to use ibm-granite/granite-docling-258M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ibm-granite/granite-docling-258M with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="ibm-granite/granite-docling-258M")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ibm-granite/granite-docling-258M")
model = AutoModelForMultimodalLM.from_pretrained("ibm-granite/granite-docling-258M", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ibm-granite/granite-docling-258M with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ibm-granite/granite-docling-258M"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ibm-granite/granite-docling-258M",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/ibm-granite/granite-docling-258M

SGLang

How to use ibm-granite/granite-docling-258M with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ibm-granite/granite-docling-258M" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ibm-granite/granite-docling-258M",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ibm-granite/granite-docling-258M" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ibm-granite/granite-docling-258M",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use ibm-granite/granite-docling-258M with Docker Model Runner:
```
docker model run hf.co/ibm-granite/granite-docling-258M
```

output endless <ecel>

#42

by openmartin - opened Oct 17, 2025

Discussion

openmartin

Oct 17, 2025

I want to convert ibm financial statements to markdown, but the first page of "ibm-2q25-earnings-press-release.pdf" can not be processed.

<doctag><section_header_level_1><loc_155><loc_27><loc_347><loc_35>IBM RELEASES SECOND-QUARTER RESULTS</section_header_level_1>
<text><loc_50><loc_36><loc_452><loc_50>Results led by Software and Infrastructure revenue growth; Strong margin expansion; Raises outlook for profit and free cash flow</text>
<text><loc_44><loc_59><loc_439><loc_75>ARMONK, N.Y., July 23, 2025 . . . IBM (NYSE: IBM) today announced second-quarter 2025 earnings results.</text>
<text><loc_44><loc_75><loc_457><loc_125>"We once again exceeded expectations for revenue, profit and free cash flow in the quarter. IBM remains highly differentiated in the market because of our deep innovation and domain expertise, both crucial in helping clients deploy and scale AI. Our generative AI book of business continues to accelerate and now stands at more than $7.5 billion," said Arvind Krishna, IBM chairman, president and chief executive officer. "With our strong first-half performance, we are raising our full-year outlook for free cash flow, which we expect to exceed $13.5 billion."</text>
<section_header_level_1><loc_44><loc_130><loc_159><loc_138>Second-Quarter Highlights</section_header_level_1>
<unordered_list><list_item><loc_44><loc_139><loc_96><loc_147>• Revenue</list_item>
<list_item><loc_59><loc_147><loc_367><loc_155>-Revenue of $17.0 billion, up 8 percent, up 5 percent at constant currency</list_item>
<list_item><loc_59><loc_155><loc_343><loc_163>-Software revenue up 10 percent, up 8 percent at constant currency</list_item>
<list_item><loc_59><loc_163><loc_306><loc_171>-Consulting revenue up 3 percent, flat at constant currency</list_item>
<list_item><loc_59><loc_171><loc_369><loc_179>-Infrastructure revenue up 14 percent, up 11 percent at constant currency</list_item>
<list_item><loc_44><loc_182><loc_78><loc_190>• Profit</list_item>
<list_item><loc_59><loc_190><loc_416><loc_207>-Gross Profit Margin: GAAP: 58.8 percent, up 200 basis points; Operating (Non-GAAP): 60.1 percent, up 230 basis points</list_item>
<list_item><loc_59><loc_207><loc_434><loc_224>-Pre-Tax Income Margin: GAAP: 15.3 percent, up 120 basis points; Operating (Non-GAAP): 18.8 percent, up 110 basis points</list_item>
<list_item><loc_44><loc_226><loc_99><loc_234>• Cash Flow</list_item>
<list_item><loc_59><loc_234><loc_436><loc_242>-Year to date, net cash from operating activities of $6.1 billion; free cash flow of $4.8 billion</list_item>
</unordered_list>
<section_header_level_1><loc_162><loc_256><loc_374><loc_263>SECOND-QUARTER 2025 INCOME STATEMENT SUMMARY</section_header_level_1>
<otsl><loc_44><loc_267><loc_456><loc_343><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel><ecel>...

engrtipusultan

Oct 27, 2025

Hopefully someone from IBM team can look into it.

https://github.com/ggml-org/llama.cpp/issues/16678

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment