Update README: add OpenVINO IR variant (OV 2026.3), path now works

README.md

@@ -10,60 +10,52 @@ tags:
   - multimodal
 ---
 
-# gemma-4-E2B-it — ONNX (OpenVINO, INT4)
-
-ONNX export of [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it),
-built with [mobius](https://github.com/onnxruntime/mobius) and INT4-quantized
-(K-Quant Q4_K_M) with [Olive](https://github.com/microsoft/Olive), packaged for
-[ONNX Runtime GenAI](https://github.com/microsoft/onnxruntime-genai) with the
-**OpenVINO execution provider**.
+# gemma-4-E2B-it — ONNX / OpenVINO (INT4)
 
+INT4 export of [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it),
+built with [mobius](https://github.com/onnxruntime/mobius) and quantized with
+[Olive](https://github.com/microsoft/Olive) (K-Quant Q4_K_M + `MatMulNBitsToQDQ`).
 Gemma 4 E2B is an any-to-any multimodal model, so this is a multi-component
-package (`decoder`, `embedding`, `vision_encoder`, `audio_encoder`) with a
-shared `genai_config.json`, tokenizer, and processor configs.
+package: `decoder`, `embedding`, `vision_encoder`, `audio_encoder`.
 
-## Variant
+## Variants
 
-| Folder | Precision | EP / Device |
+| Folder | Format | Notes |
 |---|---|---|
-| `openvino/npu` | INT4 (K-Quant body + embeddings) | OpenVINO EP, `device_type=NPU` |
+| `openvino-ir/` | **OpenVINO IR** (`.xml` + `.bin`) | INT4, per-component. **Recommended for OpenVINO.** Converts + compiles with OpenVINO **≥ 2026.3**. |
+| `openvino/npu/` | ONNX (opset 24) | INT4, for ORT GenAI + OpenVINO EP. |
 
-The `genai_config.json` sets `provider_options = [{"OpenVINO": {"device_type": "NPU"}}]`
-on every component. To target a different OpenVINO device, edit `device_type`
-to `GPU` or `CPU` — the ONNX graph is device-independent, no rebuild needed.
+## `openvino-ir/` — the OpenVINO compilation artifact
 
-## Pipeline
+Produced by converting the mobius ONNX with OpenVINO 2026.3
+(`openvino.convert_model` → `openvino.save_model`). Verified: all four
+components convert, and the decoder **compiles** (`Core().compile_model(...)`).
 
-`MobiusBuilder(fp16)` → `OnnxKQuantQuantization(bits=4, block_size=32)`.
-Recipe: [microsoft/olive-recipes `google-gemma-4-E2B-it`](https://github.com/microsoft/olive-recipes/tree/main/google-gemma-4-E2B-it).
+```python
+import openvino as ov
+core = ov.Core()
+decoder = core.compile_model("openvino-ir/decoder/openvino_model.xml", "NPU")  # or "GPU" / "CPU"
+# embedding / vision_encoder / audio_encoder load the same way
+```
 
-## ⚠️ Known limitation (please read before testing)
+Requires **OpenVINO ≥ 2026.3** — earlier releases (≤ 2026.2) lack the ONNX
+frontend translators for opset-24 `RMSNormalization` / `RotaryEmbedding`
+(added by [openvinotoolkit/openvino#35545](https://github.com/openvinotoolkit/openvino/pull/35545)),
+so conversion fails on them. The tokenizer / processor / `genai_config.json`
+files are included for reference.
 
-The mobius export uses **ONNX opset 24** ops — notably `ai.onnx::Attention`
-(plus `RMSNormalization`, `RotaryEmbedding`). As of `onnxruntime-openvino`
-**1.24** (OpenVINO 2024/2025 ONNX frontend), the OpenVINO EP **does not yet
-support the opset-24 `Attention` op** and **fails at session initialization**
-(it does not fall back to CPU):
+## Build pipeline
 
-```
-[OpenVINO-EP] Exception while Reading network:
-OpenVINO does not support the following ONNX operations: Attention
-```
+`MobiusBuilder(fp16, openvino EP)` → `OnnxKQuantQuantization(bits=4, block=32)`
+→ `MatMulNBitsToQDQ` → (OpenVINO) `convert_model`.
 
-So this model is expected to run only on an OpenVINO build whose ONNX
-frontend implements the opset-24 `Attention` op. If your OpenVINO / ORT
-version is newer and supports it, this package should load; otherwise it
-will need either a newer OpenVINO or a graph pass that decomposes `Attention`
-into OpenVINO-supported primitives.
+The mobius `openvino` EP disables `SkipSimplifiedLayerNormalization` fusion and
+`MatMulNBitsToQDQ` lowers INT4 weights to standard QDQ, so the exported graph
+uses only ops the OpenVINO ONNX frontend supports (once opset-24 `RMSNormalization`
+/ `RotaryEmbedding` are available, i.e. OpenVINO ≥ 2026.3).
 
-The model **does** load and run on the plain CPU EP (opset-24 ops are
-supported there), which can be used to sanity-check correctness while the
-OpenVINO opset gap is resolved.
+Recipe: [microsoft/olive-recipes `google-gemma-4-E2B-it`](https://github.com/microsoft/olive-recipes/tree/main/google-gemma-4-E2B-it).
 
-## Usage (ONNX Runtime GenAI, once OpenVINO supports opset-24 Attention)
+## License
 
-```python
-import onnxruntime_genai as og
-model = og.Model("openvino/npu")
-# ... standard ORT GenAI generation loop ...
-```
+Inherits the [Gemma license](https://ai.google.dev/gemma/terms).