HarmenWessels/granite-4.1-3b-int4-cw-ov

granite-4.1-3b — OpenVINO INT4 (channel-wise, symmetric, AWQ-calibrated)

OpenVINO IR conversion of ibm-granite/granite-4.1-3b (dense 40-layer transformer, 128k context, Apache-2.0), quantized to INT4 symmetric channel-wise with AWQ + scale-estimation calibration — the fast recipe for Intel iGPUs, with data-aware calibration to preserve output quality.

Converted with optimum-cli (optimum-intel, transformers 4.57.6):

optimum-cli export openvino -m ibm-granite/granite-4.1-3b \
  --task text-generation-with-past --weight-format int4 --sym --group-size -1 \
  --awq --scale-estimation --dataset wikitext2 \
  granite-4.1-3b-int4-cw-ov

Note

An INT8 build is also available as the conservative choice.

Usage (OpenVINO GenAI)

import openvino_genai as ov_genai

pipe = ov_genai.LLMPipeline("granite-4.1-3b-int4-cw-ov", "GPU", CACHE_DIR="./.ovcache")
print(pipe.generate("Write a Python function that merges overlapping intervals.",
                    max_new_tokens=256))

Tested end-to-end as an OpenAI-compatible Continue.dev backend via core-ultra-llm-server.

Provenance

• Base model: ibm-granite/granite-4.1-3b (IBM, Apache-2.0, released 2026-04)
• Calibration: AWQ + scale estimation on wikitext2 (NNCF defaults)
• No finetuning — weights are a direct quantization of the original