---
license: gemma
base_model: google/gemma-4-E2B-it
tags:
  - onnx
  - openvino
  - onnxruntime-genai
  - mobius
  - int4
  - multimodal
---

# gemma-4-E2B-it — ONNX / OpenVINO (INT4)

INT4 export of [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it),
built with [mobius](https://github.com/onnxruntime/mobius) and quantized with
[Olive](https://github.com/microsoft/Olive) (K-Quant Q4_K_M + `MatMulNBitsToQDQ`).
Gemma 4 E2B is an any-to-any multimodal model, so this is a multi-component
package: `decoder`, `embedding`, `vision_encoder`, `audio_encoder`.

## Variants

| Folder | Format | Notes |
|---|---|---|
| `openvino-ir/` | **OpenVINO IR** (`.xml` + `.bin`) | INT4, per-component. **Recommended for OpenVINO.** Converts + compiles with OpenVINO **≥ 2026.3**. |
| `openvino/npu/` | ONNX (opset 24) | INT4, for ORT GenAI + OpenVINO EP. |

## `openvino-ir/` — the OpenVINO compilation artifact

Produced by converting the mobius ONNX with OpenVINO 2026.3
(`openvino.convert_model` → `openvino.save_model`). Verified: all four
components convert, and the decoder **compiles** (`Core().compile_model(...)`).

```python
import openvino as ov
core = ov.Core()
decoder = core.compile_model("openvino-ir/decoder/openvino_model.xml", "NPU")  # or "GPU" / "CPU"
# embedding / vision_encoder / audio_encoder load the same way
```

Requires **OpenVINO ≥ 2026.3** — earlier releases (≤ 2026.2) lack the ONNX
frontend translators for opset-24 `RMSNormalization` / `RotaryEmbedding`
(added by [openvinotoolkit/openvino#35545](https://github.com/openvinotoolkit/openvino/pull/35545)),
so conversion fails on them. The tokenizer / processor / `genai_config.json`
files are included for reference.

## Build pipeline

`MobiusBuilder(fp16, openvino EP)` → `OnnxKQuantQuantization(bits=4, block=32)`
→ `MatMulNBitsToQDQ` → (OpenVINO) `convert_model`.

The mobius `openvino` EP disables `SkipSimplifiedLayerNormalization` fusion and
`MatMulNBitsToQDQ` lowers INT4 weights to standard QDQ, so the exported graph
uses only ops the OpenVINO ONNX frontend supports (once opset-24 `RMSNormalization`
/ `RotaryEmbedding` are available, i.e. OpenVINO ≥ 2026.3).

Recipe: [microsoft/olive-recipes `google-gemma-4-E2B-it`](https://github.com/microsoft/olive-recipes/tree/main/google-gemma-4-E2B-it).

## License

Inherits the [Gemma license](https://ai.google.dev/gemma/terms).