Add Mac memory chooser and reproducible demo prompt

README.md

@@ -18,22 +18,27 @@ tags:
   - 4-bit
 ---
 
+
 <p align="center">
   <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="92" alt="Hugging Face logo">
 </p>
 
+
 <p align="center">
   <img src="https://img.shields.io/badge/Meta-Muse_Glimmer-0467DF?style=for-the-badge&logo=meta&logoColor=white" alt="Meta Muse Glimmer">
   <img src="https://img.shields.io/badge/Apple_Silicon-MLX-000000?style=for-the-badge&logo=apple&logoColor=white" alt="Apple silicon MLX">
   <img src="https://img.shields.io/badge/Vontra-MLX_VLM-6E56CF?style=for-the-badge&logo=huggingface&logoColor=white" alt="Vontra MLX VLM">
 </p>
 
+
 <h1 align="center">Muse Glimmer 30B — MLX 4-bit</h1>
 
+
 <p align="center">
   A native Apple-silicon conversion of <a href="https://huggingface.co/meta-models/Muse-Glimmer-30B">meta-models/Muse-Glimmer-30B</a>, converted to uniform affine 4-bit MLX weights for text-and-image inference with MLX-VLM.
 </p>
 
+
 <p align="center">
   <a href="https://huggingface.co/meta-models/Muse-Glimmer-30B">Original model</a> ·
   <a href="https://github.com/Blaizzy/mlx-vlm">MLX-VLM</a> ·
@@ -41,10 +46,13 @@ tags:
   <a href="https://huggingface.co/Vontra">More Vontra conversions</a>
 </p>
 
+
 ## About this conversion
 
+
 This repository contains a **uniform MLX 4-bit conversion** of Muse Glimmer 30B, a dense agentic language model with a dedicated perception encoder. It preserves the upstream tokenizer, ATEM chat template, image processor configuration, generation configuration, licence, and usage policy.
 
+
 | Item | Value |
 | --- | --- |
 | Base model | [`meta-models/Muse-Glimmer-30B`](https://huggingface.co/meta-models/Muse-Glimmer-30B) |
@@ -58,15 +66,20 @@ This repository contains a **uniform MLX 4-bit conversion** of Muse Glimmer 30B,
 | Maximum visual tokens | 4,096 per image |
 | Architecture | `muse_glimmer` |
 
+
 Modules that MLX-VLM does not classify as quantizable remain at their source-compatible precision, so the effective whole-checkpoint bits-per-weight is higher than four. The `quantization` metadata records the actual eligible-module recipe.
 
+
 > [!IMPORTANT]
 > Muse Glimmer support was validated against the official MLX-VLM source at commit [`5262cb6`](https://github.com/Blaizzy/mlx-vlm/commit/5262cb6a27c797c5c3daf64b17d757924c5c474f), reporting package version 0.6.12. An older MLX-VLM or oMLX bundle may report `Model type muse_glimmer not supported`; update to a build containing the upstream Muse implementation before loading this checkpoint.
 
+
 ## Apple-silicon performance
 
+
 This checkpoint was load-tested, text-generation tested, and benchmarked on:
 
+
 | Hardware | Configuration |
 | --- | --- |
 | Host | Mac Studio |
@@ -75,8 +88,10 @@ This checkpoint was load-tested, text-generation tested, and benchmarked on:
 | Unified memory | 256 GB |
 | Runtime | MLX-VLM 0.6.12 source revision `5262cb6` with the oMLX MLX runtime |
 
+
 A warmed local text-only test produced:
 
+
 | Measurement | Result |
 | --- | ---: |
 | Decode (median) | **37.70 tokens/s** |
@@ -86,29 +101,38 @@ A warmed local text-only test produced:
 | Warm-up | 256 generated tokens |
 | Prompt | 83 tokens after chat templating |
 
+
 The decode figure is the median of three greedy 256-token runs after a full 256-token Metal-kernel warm-up. It is a practical local reference, not a controlled cross-platform benchmark. Prompt length, images, context growth, sampler settings, memory pressure, thermal state, and runtime versions can materially change performance. Image prefill is not included in this decode benchmark.
 
+
 ## Runtime setup
 
+
 Until Muse Glimmer support reaches the MLX-VLM build supplied by your application, install the exact official source revision used for validation:
 
+
 ```bash
 python -m pip install \
   "mlx-vlm @ git+https://github.com/Blaizzy/mlx-vlm.git@5262cb6a27c797c5c3daf64b17d757924c5c474f"
 ```
 
+
 Download the checkpoint if a local copy is preferred:
 
+
 ```bash
 hf download Vontra/Muse-Glimmer-30B-MLX-4bit \
   --local-dir ~/.omlx/models/Vontra/Muse-Glimmer-30B-MLX-4bit
 ```
 
+
 ## Text generation
 
+
 ```python
 from mlx_vlm import generate, load
 
+
 model, processor = load("Vontra/Muse-Glimmer-30B-MLX-4bit")
 tokenizer = processor.tokenizer if hasattr(processor, "tokenizer") else processor
 messages = [
@@ -130,15 +154,20 @@ result = generate(
 print(result.text)
 ```
 
+
 Muse Glimmer supports `low`, `medium`, `high`, and `xhigh` reasoning strengths. Higher settings can spend more tokens reasoning before returning the final answer.
 
+
 ## Image and text generation
 
+
 The multimodal path was smoke-tested locally with a real image. Include an image content item so the ATEM template emits the required patch token:
 
+
 ```python
 from mlx_vlm import generate, load
 
+
 model, processor = load("Vontra/Muse-Glimmer-30B-MLX-4bit")
 tokenizer = processor.tokenizer if hasattr(processor, "tokenizer") else processor
 messages = [
@@ -166,10 +195,13 @@ result = generate(
 print(result.text)
 ```
 
+
 ## Architecture
 
+
 Muse Glimmer combines a dense causal transformer with a dedicated perception encoder for interleaved text and image input.
 
+
 | Architecture detail | Upstream value |
 | --- | ---: |
 | Total parameters | ~29.6B |
@@ -184,10 +216,13 @@ Muse Glimmer combines a dense causal transformer with a dedicated perception enc
 | Perception encoder | ~1.8B-parameter ViT-G/14, 50 layers |
 | Context length | 131,072 tokens |
 
+
 The model supports agentic task completion, tool use through the upstream ATEM protocol, controllable reasoning effort, multilingual input, failure recovery, and multimodal understanding. See the [original model card](https://huggingface.co/meta-models/Muse-Glimmer-30B) for upstream benchmarks, training details, intended uses, limitations, and safety guidance.
 
+
 ## Conversion and validation notes
 
+
 - Source weights: upstream BF16 checkpoint at revision `a4e59da52a7bc87ae7251dd5545c0dd437c44b68`.
 - Quantization mode: affine, 4-bit, group size 64, without mixed-precision overrides.
 - The upstream tokenizer, ATEM chat template, processor configuration, generation configuration, licence, and usage policy are included.
@@ -196,10 +231,46 @@ The model supports agentic task completion, tool use through the upstream ATEM p
 - Quantization can reduce output quality relative to BF16. Use a higher-precision variant when quality matters more than memory use.
 - This release does not include or claim support for the upstream speculative drafter.
 
+
 This is a community conversion, not an official Meta release. Validate quality, safety, and numerical behaviour on representative workloads before production use.
 
+
 ## Licence, usage policy, and attribution
 
+
 The upstream model is released under the **Apache License 2.0**. The upstream `LICENSE` and `USAGE_POLICY.md` files are included in this repository; use is subject to both the licence and the upstream usage policy.
 
+
 All model design, training, benchmark, and upstream documentation credit belongs to Meta and the original contributors. The MLX conversion, Apple-silicon validation, compatibility work, and model card are provided by [Vontra](https://huggingface.co/Vontra).
+
+
+
+<!-- vontra-chooser-start -->
+## Choose for your Mac
+
+[64GB Macs](https://huggingface.co/collections/Vontra/mlx-models-for-64gb-macs-6a9fefda17932216ec9ab457) · [128GB Macs](https://huggingface.co/collections/Vontra/mlx-models-for-128gb-macs-6a9ff0abd31bc9abbe7922d7) · [256GB Macs](https://huggingface.co/collections/Vontra/mlx-models-for-256gb-macs-6a9ff0ef9fed7c5bdca15e9b)
+
+Published peak memory: **21.33 GB**; estimated starting tier: **64GB**, leaving about **42 GB** nominal headroom. The collections use published M3 Studio peaks with at least 25% nominal headroom; fit on other Macs is an estimate, and full context is not guaranteed. Start with short context and one request.
+
+### Runtime and evidence
+
+The exact tested oMLX application version is not recorded here; a library version is not an app version. The original performance tables retain their benchmark conditions and speed figures; this documentation update adds no new test results.
+
+### Quick start and demo prompt
+
+```bash
+hf download Vontra/Muse-Glimmer-30B-MLX-4bit --local-dir ./models/Muse-Glimmer-30B-MLX-4bit
+```
+
+Add the downloaded folder to oMLX model directories, refresh the list, and follow this card's architecture and MTP compatibility requirements before loading.
+
+Try this in a new chat with a 128-token output limit:
+
+```text
+Explain why the sky looks blue in three short sentences.
+```
+
+This is a demo prompt to try, not a recorded successful run; a captured demonstration for this documentation update is not yet available.
+
+[Follow Vontra for new Apple Silicon releases and fixes.](https://huggingface.co/Vontra)
+<!-- vontra-chooser-end -->