Irfanuruchi/Nanbeige4.2-3B-CE-v1.0-OpenVINO-INT4

Nanbeige4.2-3B-CE v1.0 — OpenVINO INT4 NPU Candidate

INT4-targeted OpenVINO weight-compressed derivative of the validated stateful FP16 artifact.

Provenance

• Canonical source revision: v1.0
• Canonical source commit: 171d26c743c6f1f5e3fd49f88c0454448a28c39e
• Source OpenVINO artifact: Nanbeige4.2-3B-CE-v1.0-OpenVINO-FP16-Stateful

Compression

Created with NNCF 3.3.0 using:

• mode: INT4_SYM
• group size: 128
• ratio: 1.0

NNCF reported:

• 154/157 layers using symmetric INT4 group-size 128
• 2/157 layers using asymmetric per-channel INT8 fallback
• 1/157 layer left floating-point
• compressed openvino_model.bin: 2,645,700,572 bytes
• FP16 source openvino_model.bin: 8,339,601,804 bytes
• INT4/FP16 model-bin size ratio: 0.3172

The compressed stateful IR retains the 3-input / 1-output GenAI interface.

A separate stateless structural probe also confirmed that all 44 explicit key/value cache pairs survived compression: 90 inputs, 89 outputs, with all 88 past_key_values.* inputs and all 88 present.* outputs present.

Intel NPU validation status

Not hardware-validated for successful generation on the tested Intel AI Boost system.

Test machine had 32 GB system RAM and Intel(R) AI Boost.

Observed behavior:

• OpenVINO 2026.3.1: NPU compilation completed after applying the documented Level Zero allocation workaround, but generation hit a KV-cache output-name lookup failure.
• OpenVINO 2026.5 nightly (2026-09-03): the previous KV-name failure was no longer observed; NPU compilation completed, but execution failed with ZE_RESULT_ERROR_OUT_OF_HOST_MEMORY.
• A minimum-memory test with a 1-token prompt and very small static prompt/response limits still exhausted practical host-memory headroom during the specialized NPU LLM path.
• Plain direct stateful NPU compilation rejected the unbounded dynamic state shapes.
• A fully static explicit-KV compilation probe consumed essentially all available memory on the 32 GB test system and was stopped.

Therefore this package is published as an experimental NPU candidate / structurally validated artifact, not as production-proven Intel NPU support.

Important quality note

Runtime or structural validation is not a factual-quality benchmark. This release preserves the known limitations of the frozen Nanbeige4.2-3B-CE v1.0 checkpoint.

License

Apache License 2.0. See LICENSE.