aweussom/LFM2-1.2B-int4-cw-ov

LFM2-1.2B — OpenVINO INT4 channel-wise (Intel NPU build)

INT4 channel-wise symmetric OpenVINO export of LiquidAI/LFM2-1.2B, built for the Intel NPU. This build is NPU-only — see device notes below.

⚠️ NPU 3 only (Arrow Lake / Meteor Lake). On a Lunar Lake NPU 4 (Core Ultra 200V series) this build loads, runs at full speed, and emits word salad. A newer NPU driver is not expected to help — its sibling export was re-tested on two drivers and did not change. See NPU generation below.

Measured (Intel Core Ultra 9 285K NPU, driver 32.0.100.4778, genai 2026.3)

TTFT	878 ms
Decode	36.5 tok/s
Load (cold compile)	~25 s

Why channel-wise

Default group-quantized int4 exports crash the NPU driver compiler (StopLocationVerifierPass ... Found N duplicated names, a known vpux bug). Channel-wise symmetric avoids it:

optimum-cli export openvino --model LiquidAI/LFM2-1.2B \
  --weight-format int4 --group-size -1 --sym --ratio 1.0 --trust-remote-code <out>

Toolchain: OpenVINO 2026.1.0, optimum-intel 1.27.0, transformers 4.57.6.

Device notes — NPU only, and no good int8 exists (as of 2026-08)

• NPU generation matters, and NPU 4 is wrong for this model. On a Core Ultra 7 258V (Lunar Lake, DEVICE_ARCHITECTURE=4000) the model compiles, loads and decodes at full speed while producing word salad, byte-identical across OpenVINO 2026.3.0, 2026.3.1 and the 2026.5 nightly and both the plugin and the driver compiler. The same file is correct on the 285K's NPU 3 with identical software, and Intel's own LFM2.5 exports fail the same way on NPU 4, so this is the LFM2 family on NPU 4 rather than this particular export. Tracked upstream at openvinotoolkit/openvino#37322.

  Scope, so you can judge it: this build's NPU 4 runs were on driver 32.0.100.4778. It has not been re-probed on 32.0.100.5540 — the sibling LFM2.5-1.2B-Instruct-int4-cw-ov has, on the same machine, and its output was byte-identical on both drivers. So "a driver update does not fix this" is directly measured for the sibling and inferred here. If you have a Lunar Lake box and a newer driver, a contradicting result is welcome on the upstream issue. (Last checked 2026-09-01.)

• CPU/GPU: this build does not run (ScatterNDUpdate shape error in the short-conv layers — an optimum-intel 1.27 export artifact). For CPU/GPU, export with optimum-intel ≥2.1 + transformers 5.4 instead; those exports in turn fail on NPU (invalid unordered_map key at generate). No single export currently serves both device families.

• int8 on NPU is a trap for LFM2: int8-sym compiles and runs fast but generates garbage; int8-asym (Intel's own recipe) is coherent but ~1.4 tok/s. int4-cw is the only fast-and-correct NPU configuration.

Usage

import openvino_genai as og
pipe = og.LLMPipeline("LFM2-1.2B-int4-cw-ov", "NPU", MAX_PROMPT_LEN=4096)
print(pipe.generate("Hello!", og.GenerationConfig(max_new_tokens=64)))

Or with NoLlama: python nollama.py --model-dir ~/models/LFM2-1.2B-int4-cw-ov.

License

Inherits LiquidAI's LFM Open License v1.0 (see LICENSE): Apache-2.0-based, free use including commercial below US$10M annual revenue; above that, contact Liquid AI for a commercial license.