---
license: apache-2.0
base_model: ibm-granite/granite-4.1-8b
base_model_relation: quantized
tags:
- openvino
- int4
- awq
- intel
- arc
- igpu
- core-ultra
language:
- en
pipeline_tag: text-generation
---

# granite-4.1-8b — OpenVINO INT4 (channel-wise, symmetric, AWQ-calibrated)

OpenVINO IR conversion of [ibm-granite/granite-4.1-8b](https://huggingface.co/ibm-granite/granite-4.1-8b)
(dense 40-layer transformer, 128k context, Apache-2.0). To our knowledge the first OpenVINO IR
of this model. Quantized **INT4 symmetric channel-wise with AWQ + scale-estimation
calibration** (data-aware calibration, applied from the start).

Converted with `optimum-cli` (optimum-intel git-master, transformers 4.57.6):

```bash
optimum-cli export openvino -m ibm-granite/granite-4.1-8b \
  --task text-generation-with-past --weight-format int4 --sym --group-size -1 \
  --awq --scale-estimation --dataset wikitext2 \
  granite-4.1-8b-int4-cw-ov
```

## Related

Related artifacts: [granite-4.1-3b-int4-cw-ov](https://huggingface.co/HarmenWessels/granite-4.1-3b-int4-cw-ov)
(same recipe, smaller/faster) and [granite-4.1-3b-int8-ov](https://huggingface.co/HarmenWessels/granite-4.1-3b-int8-ov).

## Usage (OpenVINO GenAI)

```python
import openvino_genai as ov_genai

pipe = ov_genai.LLMPipeline("granite-4.1-8b-int4-cw-ov", "GPU", CACHE_DIR="./.ovcache")
print(pipe.generate("Write a Python function that merges overlapping intervals.",
                    max_new_tokens=256))
```

Tested end-to-end as an OpenAI-compatible Continue.dev backend via
[core-ultra-llm-server](https://github.com/harmenwessels/core-ultra-llm-server).

## Provenance

- Base model: `ibm-granite/granite-4.1-8b` (IBM, Apache-2.0, released 2026-04)
- Calibration: AWQ + scale estimation on wikitext2 (NNCF defaults)
- Conversion date: 2026-06-05
- No finetuning — weights are a direct quantization of the original