aweussom/Muse-Glimmer-30B-int4-ov

Muse Glimmer 30B — OpenVINO int4

INT4 OpenVINO export of Meta's Muse-Glimmer-30B: a 30B dense multimodal model built for local agentic use (text + images in, text out, 131k context, Apache 2.0). Full VLM export — language model, vision encoder and text embeddings — ~17 GB total.

Exported 2026-08-12 with optimum-intel main, one day after Glimmer support merged there (PR #1924, tracking issue #1927):

optimum-cli export openvino -m meta-models/Muse-Glimmer-30B out \
  --weight-format int4 --task image-text-to-text \
  --group-size 64 --group-size-fallback ignore

Requirements

• openvino 2026.x and optimum-intel from git main — no released transformers knows muse_glimmer (the checkpoint needs transformers 5.16.0.dev0+, installed from source after optimum-intel so its transformers<5.6 pin doesn't downgrade it)
• ~17 GB of device memory for weights, plus KV cache — a 24 GB GPU is comfortable; CPU inference works with 32 GB+ RAM

Usage (optimum-intel)

from optimum.intel.openvino import OVModelForVisualCausalLM
from transformers import AutoProcessor

model_id = "aweussom/Muse-Glimmer-30B-int4-ov"
model = OVModelForVisualCausalLM.from_pretrained(model_id, device="GPU")  # or CPU
processor = AutoProcessor.from_pretrained(model_id, padding_side="left")

See the PR #1924 description for a complete image-text-to-text inference example.

Export notes (hard-won)

• transformers 5.16.0.dev0's parallel shard loader (4 threads, no env knob) can crash with a native access violation on 32 GB machines while materializing the 60 GB BF16 checkpoint. Workaround: set transformers.core_model_loading.GLOBAL_WORKERS = 1 before loading. Export comfortably done on 128 GB RAM.
• Reproducible end-to-end via the scripts in aweussom/NoLlama.

Quantization: NNCF int4, group size 64 (--group-size-fallback ignore). Not affiliated with Meta; usage is governed by the upstream model's Apache 2.0 license and usage policy.