license: gemma
base_model: google/gemma-4-E2B-it
tags:
- onnx
- openvino
- onnxruntime-genai
- mobius
- int4
- multimodal
gemma-4-E2B-it — ONNX / OpenVINO (INT4)
INT4 export of google/gemma-4-E2B-it,
built with mobius and quantized with
Olive (K-Quant Q4_K_M + MatMulNBitsToQDQ).
Gemma 4 E2B is an any-to-any multimodal model, so this is a multi-component
package: decoder, embedding, vision_encoder, audio_encoder.
Variants
| Folder | Format | Notes |
|---|---|---|
openvino-ir/ |
OpenVINO IR (.xml + .bin) |
INT4, per-component. Recommended for OpenVINO. Converts + compiles with OpenVINO ≥ 2026.3. |
openvino/npu/ |
ONNX (opset 24) | INT4, for ORT GenAI + OpenVINO EP. |
openvino-ir/ — the OpenVINO compilation artifact
Produced by converting the mobius ONNX with OpenVINO 2026.3
(openvino.convert_model → openvino.save_model). Verified: all four
components convert, and the decoder compiles (Core().compile_model(...)).
import openvino as ov
core = ov.Core()
decoder = core.compile_model("openvino-ir/decoder/openvino_model.xml", "NPU") # or "GPU" / "CPU"
# embedding / vision_encoder / audio_encoder load the same way
Requires OpenVINO ≥ 2026.3 — earlier releases (≤ 2026.2) lack the ONNX
frontend translators for opset-24 RMSNormalization / RotaryEmbedding
(added by openvinotoolkit/openvino#35545),
so conversion fails on them. The tokenizer / processor / genai_config.json
files are included for reference.
Build pipeline
MobiusBuilder(fp16, openvino EP) → OnnxKQuantQuantization(bits=4, block=32)
→ MatMulNBitsToQDQ → (OpenVINO) convert_model.
The mobius openvino EP disables SkipSimplifiedLayerNormalization fusion and
MatMulNBitsToQDQ lowers INT4 weights to standard QDQ, so the exported graph
uses only ops the OpenVINO ONNX frontend supports (once opset-24 RMSNormalization
/ RotaryEmbedding are available, i.e. OpenVINO ≥ 2026.3).
Recipe: microsoft/olive-recipes google-gemma-4-E2B-it.
License
Inherits the Gemma license.