A native Apple-silicon conversion of meta-models/Muse-Glimmer-30B, quantized with oMLX's optimized mixed-precision oQ6 recipe for text-and-image inference with MLX-VLM.
## About this conversion This repository contains an **oMLX-optimized mixed-precision oQ6 conversion** of Muse Glimmer 30B, a dense agentic language model with a dedicated perception encoder. It preserves the upstream tokenizer, ATEM chat template, image processor configuration, generation configuration, licence, and usage policy. | Item | Value | | --- | --- | | Base model | [`meta-models/Muse-Glimmer-30B`](https://huggingface.co/meta-models/Muse-Glimmer-30B) | | Format | MLX safetensors | | Quantization | oQ6 affine mixed precision, group size 64 | | Mixed-precision overrides | 11 sensitive modules at 8-bit | | Conversion/runtime stack | MLX-VLM with MLX on Apple silicon | | Weight shards | 6 | | Weight size | 27.22 GB (25.35 GiB) | | Context length | 131,072 tokens | | Maximum visual tokens | 4,096 per image | | Architecture | `muse_glimmer` | The oQ recipe keeps the embedding, language-model head, and selected attention/MLP projections at 8-bit above the 6-bit base. Both `quantization` and `quantization_config` record every per-module override. > [!IMPORTANT] > Muse Glimmer support was validated against the official MLX-VLM source at commit [`5262cb6`](https://github.com/Blaizzy/mlx-vlm/commit/5262cb6a27c797c5c3daf64b17d757924c5c474f), reporting package version 0.6.12. An older MLX-VLM or oMLX bundle may report `Model type muse_glimmer not supported`; update to a build containing the upstream Muse implementation before loading this checkpoint. ## Apple-silicon performance This checkpoint was load-tested, text-generation tested, and benchmarked on: | Hardware | Configuration | | --- | --- | | Host | Mac Studio | | Chip | Apple M3 Ultra | | CPU | 32 cores (24 performance + 8 efficiency) | | Unified memory | 256 GB | | Runtime | MLX-VLM 0.6.12 source revision `5262cb6` with the oMLX MLX runtime | A warmed local text-only test produced: | Measurement | Result | | --- | ---: | | Decode (median) | **26.29 tokens/s** | | Individual decode runs | 26.32 / 26.28 / 26.29 tokens/s | | Reported peak memory | **29.13 GB** | | Timed runs | 3 × 256 generated tokens | | Warm-up | 256 generated tokens | | Prompt | 83 tokens after chat templating | The decode figure is the median of three greedy 256-token runs after a full 256-token Metal-kernel warm-up. It is a practical local reference, not a controlled cross-platform benchmark. Prompt length, images, context growth, sampler settings, memory pressure, thermal state, and runtime versions can materially change performance. Image prefill is not included in this decode benchmark. ## Runtime setup Until Muse Glimmer support reaches the MLX-VLM build supplied by your application, install the exact official source revision used for validation: ```bash python -m pip install \ "mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm.git@5262cb6a27c797c5c3daf64b17d757924c5c474f" ``` Download the checkpoint if a local copy is preferred: ```bash hf download Vontra/Muse-Glimmer-30B-oQ6 \ --local-dir ~/.omlx/models/Vontra/Muse-Glimmer-30B-oQ6 ``` ## Text generation ```python from mlx_vlm import generate, load model, processor = load("Vontra/Muse-Glimmer-30B-oQ6") tokenizer = processor.tokenizer if hasattr(processor, "tokenizer") else processor messages = [ {"role": "user", "content": "Explain unified memory on Apple silicon."} ] prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, reasoning_strength="medium", ) result = generate( model, processor, prompt, max_tokens=512, temperature=0.0, ) print(result.text) ``` Muse Glimmer supports `low`, `medium`, `high`, and `xhigh` reasoning strengths. Higher settings can spend more tokens reasoning before returning the final answer. ## Image and text generation The multimodal path was smoke-tested locally with a real image. Include an image content item so the ATEM template emits the required patch token: ```python from mlx_vlm import generate, load model, processor = load("Vontra/Muse-Glimmer-30B-oQ6") tokenizer = processor.tokenizer if hasattr(processor, "tokenizer") else processor messages = [ { "role": "user", "content": [ {"type": "image"}, {"type": "text", "text": "Describe this image in one sentence."}, ], } ] prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, ) result = generate( model, processor, prompt, image=["image.jpg"], max_tokens=256, temperature=0.0, ) print(result.text) ``` ## Architecture Muse Glimmer combines a dense causal transformer with a dedicated perception encoder for interleaved text and image input. | Architecture detail | Upstream value | | --- | ---: | | Total parameters | ~29.6B | | Transformer layers | 52 | | Hidden size | 6,656 | | Attention heads / KV heads | 32 / 2 | | Head dimension | 128 | | FFN intermediate size | 19,968 | | Attention pattern | local / local / local / global | | Sliding window | 2,048 tokens | | Vocabulary size | 202,048 | | Perception encoder | ~1.8B-parameter ViT-G/14, 50 layers | | Context length | 131,072 tokens | The model supports agentic task completion, tool use through the upstream ATEM protocol, controllable reasoning effort, multilingual input, failure recovery, and multimodal understanding. See the [original model card](https://huggingface.co/meta-models/Muse-Glimmer-30B) for upstream benchmarks, training details, intended uses, limitations, and safety guidance. ## Conversion and validation notes - Source weights: upstream BF16 checkpoint at revision `a4e59da52a7bc87ae7251dd5545c0dd437c44b68`. - Quantization mode: oQ6 affine mixed precision, group size 64, with 11 protected-module overrides at 8-bit. - The upstream tokenizer, ATEM chat template, processor configuration, generation configuration, licence, and usage policy are included. - All 2,272 converted tensors and all 6 indexed shards were checked locally. - The checkpoint was exercised through full text generation; the Muse architecture was also smoke-tested with image input. - Quantization can reduce output quality relative to BF16. Use a higher-precision variant when quality matters more than memory use. - This release does not include or claim support for the upstream speculative drafter. This is a community conversion, not an official Meta release. Validate quality, safety, and numerical behaviour on representative workloads before production use. ## Licence, usage policy, and attribution The upstream model is released under the **Apache License 2.0**. The upstream `LICENSE` and `USAGE_POLICY.md` files are included in this repository; use is subject to both the licence and the upstream usage policy. All model design, training, benchmark, and upstream documentation credit belongs to Meta and the original contributors. The MLX conversion, Apple-silicon validation, compatibility work, and model card are provided by [Vontra](https://huggingface.co/Vontra). ## Choose for your Mac [64GB Macs](https://huggingface.co/collections/Vontra/mlx-models-for-64gb-macs-6a9fefda17932216ec9ab457) · [128GB Macs](https://huggingface.co/collections/Vontra/mlx-models-for-128gb-macs-6a9ff0abd31bc9abbe7922d7) · [256GB Macs](https://huggingface.co/collections/Vontra/mlx-models-for-256gb-macs-6a9ff0ef9fed7c5bdca15e9b) Published peak memory: **29.13 GB**; estimated starting tier: **64GB**, leaving about **34 GB** nominal headroom. The collections use published M3 Studio peaks with at least 25% nominal headroom; fit on other Macs is an estimate, and full context is not guaranteed. Start with short context and one request. ### Runtime and evidence The exact tested oMLX application version is not recorded here; a library version is not an app version. The original performance tables retain their benchmark conditions and speed figures; this documentation update adds no new test results. ### Quick start and demo prompt ```bash hf download Vontra/Muse-Glimmer-30B-oQ6 --local-dir ./models/Muse-Glimmer-30B-oQ6 ``` Add the downloaded folder to oMLX model directories, refresh the list, and follow this card's architecture and MTP compatibility requirements before loading. Try this in a new chat with a 128-token output limit: ```text Explain why the sky looks blue in three short sentences. ``` This is a demo prompt to try, not a recorded successful run; a captured demonstration for this documentation update is not yet available. [Follow Vontra for new Apple Silicon releases and fixes.](https://huggingface.co/Vontra)