File size: 2,397 Bytes
5d6166f c253cd1 5d6166f c253cd1 5d6166f c253cd1 5d6166f c253cd1 5d6166f c253cd1 5d6166f c253cd1 5d6166f c253cd1 5d6166f c253cd1 5d6166f c253cd1 5d6166f c253cd1 5d6166f c253cd1 5d6166f c253cd1 5d6166f c253cd1 5d6166f c253cd1 5d6166f c253cd1 5d6166f c253cd1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | ---
license: gemma
base_model: google/gemma-4-E2B-it
tags:
- onnx
- openvino
- onnxruntime-genai
- mobius
- int4
- multimodal
---
# gemma-4-E2B-it — ONNX / OpenVINO (INT4)
INT4 export of [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it),
built with [mobius](https://github.com/onnxruntime/mobius) and quantized with
[Olive](https://github.com/microsoft/Olive) (K-Quant Q4_K_M + `MatMulNBitsToQDQ`).
Gemma 4 E2B is an any-to-any multimodal model, so this is a multi-component
package: `decoder`, `embedding`, `vision_encoder`, `audio_encoder`.
## Variants
| Folder | Format | Notes |
|---|---|---|
| `openvino-ir/` | **OpenVINO IR** (`.xml` + `.bin`) | INT4, per-component. **Recommended for OpenVINO.** Converts + compiles with OpenVINO **≥ 2026.3**. |
| `openvino/npu/` | ONNX (opset 24) | INT4, for ORT GenAI + OpenVINO EP. |
## `openvino-ir/` — the OpenVINO compilation artifact
Produced by converting the mobius ONNX with OpenVINO 2026.3
(`openvino.convert_model` → `openvino.save_model`). Verified: all four
components convert, and the decoder **compiles** (`Core().compile_model(...)`).
```python
import openvino as ov
core = ov.Core()
decoder = core.compile_model("openvino-ir/decoder/openvino_model.xml", "NPU") # or "GPU" / "CPU"
# embedding / vision_encoder / audio_encoder load the same way
```
Requires **OpenVINO ≥ 2026.3** — earlier releases (≤ 2026.2) lack the ONNX
frontend translators for opset-24 `RMSNormalization` / `RotaryEmbedding`
(added by [openvinotoolkit/openvino#35545](https://github.com/openvinotoolkit/openvino/pull/35545)),
so conversion fails on them. The tokenizer / processor / `genai_config.json`
files are included for reference.
## Build pipeline
`MobiusBuilder(fp16, openvino EP)` → `OnnxKQuantQuantization(bits=4, block=32)`
→ `MatMulNBitsToQDQ` → (OpenVINO) `convert_model`.
The mobius `openvino` EP disables `SkipSimplifiedLayerNormalization` fusion and
`MatMulNBitsToQDQ` lowers INT4 weights to standard QDQ, so the exported graph
uses only ops the OpenVINO ONNX frontend supports (once opset-24 `RMSNormalization`
/ `RotaryEmbedding` are available, i.e. OpenVINO ≥ 2026.3).
Recipe: [microsoft/olive-recipes `google-gemma-4-E2B-it`](https://github.com/microsoft/olive-recipes/tree/main/google-gemma-4-E2B-it).
## License
Inherits the [Gemma license](https://ai.google.dev/gemma/terms).
|