Instructions to use Qiskit/granite-3.2-8b-qiskit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Qiskit/granite-3.2-8b-qiskit with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Qiskit/granite-3.2-8b-qiskit")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Qiskit/granite-3.2-8b-qiskit")
model = AutoModelForCausalLM.from_pretrained("Qiskit/granite-3.2-8b-qiskit", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Qiskit/granite-3.2-8b-qiskit with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Qiskit/granite-3.2-8b-qiskit"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qiskit/granite-3.2-8b-qiskit",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Qiskit/granite-3.2-8b-qiskit

SGLang

How to use Qiskit/granite-3.2-8b-qiskit with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Qiskit/granite-3.2-8b-qiskit" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qiskit/granite-3.2-8b-qiskit",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Qiskit/granite-3.2-8b-qiskit" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qiskit/granite-3.2-8b-qiskit",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Qiskit/granite-3.2-8b-qiskit with Docker Model Runner:
```
docker model run hf.co/Qiskit/granite-3.2-8b-qiskit
```

ndupuis commited on Jun 2, 2025

Commit

a7e67e0

1 Parent(s): 3f13c0e

Update README

Browse files

Files changed (1) hide show

README.md +5 -5

README.md CHANGED Viewed

@@ -18,14 +18,14 @@ tags:
 ## Model Summary
-**granite-3.2-8b-qiskit** is a 8B parameter model extend pretrained and fine tuned on top of granite3.2-8b-base using Qiskit code and instruction data to improve capabilities at writing high-quality and non-deprecated Qiskit code. We used only data with the following licenses: Apache 2.0, MIT, the Unlicense, Mulan PSL Version 2, BSD-2, BSD-3, and Creative Commons Attribution 4.0.
 - **Developers:** IBM Quantum & IBM Research
 - **GitHub Repository:** Pending
 - **Related Papers:** [Qiskit Code Assistant: Training LLMs for
 generating Quantum Computing Code](https://arxiv.org/abs/2405.19495) and [Qiskit HumanEval: An Evaluation Benchmark For Quantum Code Generative Models](https://arxiv.org/abs/2406.14712)
-- **Release Date**: Pending
-- **License:** Pending.
 ## Usage
@@ -35,7 +35,7 @@ This model is designed for generating quantum computing code using Qiskit. Both
 ### Generation
-This is a simple example of how to use **granite-8b-qiskit** model.
 ```python
 import torch
@@ -69,7 +69,7 @@ for i in output:
 - **Data Collection and Filtering:** Our code data is sourced from a combination of publicly available datasets (e.g., Code available on <https://github.com>), and additional synthetic data generated at IBM Quantum. We exclude code that is older than 2023.
 - **Exact and Fuzzy Deduplication:** We use both exact and fuzzy deduplication to remove documents having (near) identical code content.
-- **HAP, PII, Malware Filtering:** We rely on the base model ibm-granite/granite-8b-code-base for HAP and malware filtering from the initial datasets used in the context of the base model. We also make sure to redact Personally Identifiable Information (PII) in our datasets by replacing PII content (e.g., names, email addresses, keys, passwords) with corresponding tokens (e.g., ⟨NAME⟩, ⟨EMAIL⟩, ⟨KEY⟩, ⟨PASSWORD⟩).
 ## Infrastructure

 ## Model Summary
+**granite-3.2-8b-qiskit** is a 8B parameter model extend pretrained and fine tuned on top of granite3.1-8b-base using Qiskit code and instruction data to improve capabilities at writing high-quality and non-deprecated Qiskit code. We used only data with the following licenses: Apache 2.0, MIT, the Unlicense, Mulan PSL Version 2, BSD-2, BSD-3, and Creative Commons Attribution 4.0.
 - **Developers:** IBM Quantum & IBM Research
 - **GitHub Repository:** Pending
 - **Related Papers:** [Qiskit Code Assistant: Training LLMs for
 generating Quantum Computing Code](https://arxiv.org/abs/2405.19495) and [Qiskit HumanEval: An Evaluation Benchmark For Quantum Code Generative Models](https://arxiv.org/abs/2406.14712)
+- **Release Date**: 06-03-2025
+- **License:** apache-2.0
 ## Usage
 ### Generation
+This is a simple example of how to use **granite-3.2-8b-qiskit** model.
 ```python
 import torch
 - **Data Collection and Filtering:** Our code data is sourced from a combination of publicly available datasets (e.g., Code available on <https://github.com>), and additional synthetic data generated at IBM Quantum. We exclude code that is older than 2023.
 - **Exact and Fuzzy Deduplication:** We use both exact and fuzzy deduplication to remove documents having (near) identical code content.
+- **HAP, PII, Malware Filtering:** We rely on the base model ibm-granite/granite-3.1-8b-code-base for HAP and malware filtering from the initial datasets used in the context of the base model. We also make sure to redact Personally Identifiable Information (PII) in our datasets by replacing PII content (e.g., names, email addresses, keys, passwords) with corresponding tokens (e.g., ⟨NAME⟩, ⟨EMAIL⟩, ⟨KEY⟩, ⟨PASSWORD⟩).
 ## Infrastructure