Instructions to use leonsarmiento/GRM-3.2-Sky-6bit-XL-mlx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use leonsarmiento/GRM-3.2-Sky-6bit-XL-mlx with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("leonsarmiento/GRM-3.2-Sky-6bit-XL-mlx")
config = load_config("leonsarmiento/GRM-3.2-Sky-6bit-XL-mlx")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use leonsarmiento/GRM-3.2-Sky-6bit-XL-mlx with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "leonsarmiento/GRM-3.2-Sky-6bit-XL-mlx"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "leonsarmiento/GRM-3.2-Sky-6bit-XL-mlx"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use leonsarmiento/GRM-3.2-Sky-6bit-XL-mlx with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "leonsarmiento/GRM-3.2-Sky-6bit-XL-mlx"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default leonsarmiento/GRM-3.2-Sky-6bit-XL-mlx

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use leonsarmiento/GRM-3.2-Sky-6bit-XL-mlx with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "leonsarmiento/GRM-3.2-Sky-6bit-XL-mlx"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "leonsarmiento/GRM-3.2-Sky-6bit-XL-mlx" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

leonsarmiento/GRM-3.2-Sky-6bit-XL-mlx

GRM-3.2-Sky by OrionLLM — a 35B-A3B sparse MoE fine-tuned from Ornith-1.0-35B for long-horizon agentic tasks and extremely difficult reasoning, quantized for Apple Silicon using the BaseQuant_XL 6/8-bit recipe.

Only ~3B parameters active per token — the decode speed of a small model with the capacity of a 35B one. GRM-3.2-Sky is OrionLLM's flagship: it targets failure modes like drift and loss of goal state over extended multi-step workflows, and posts 81.4 SWE-bench Verified, 87.7 LiveCodeBench v6, and 96.3 AIME26.

This is a full multimodal build — the vision tower is preserved.

About XL Quantization

BaseQuant_XL is a fully data-agnostic, static quantization. No calibration dataset, no sensitivity analysis, no importance matrix. Precision is allocated purely by architectural role — routing-critical layers get higher precision, bulk expert parameters get lower precision. The result is a transparent, faithful capture of the source model.

Data-dependent calibration quantizations (iMatrix, AWQ, GPTQ, oQ, oQ4e, etc.) use a calibration set to guide bit allocation. This can produce a skewed representation of the model: domains well-represented in the calibration data (English, popular topics, public or leaked benchmarks) are preserved better, while underrepresented domains (non-English languages, niche use cases, your own data) are preserved worse. XL avoids this trade-off entirely — it generalizes honestly because it is never fit to any particular data distribution.

Quickstart

pip install -U mlx-vlm

python -m mlx_vlm.generate --model leonsarmiento/GRM-3.2-Sky-6bit-XL-mlx --max-tokens 256 --temperature 0.6 --top-p 0.95 --prompt "Implement an LRU cache in Python with O(1) get/put."

Works with LM Studio — vision mmproj included. Thinking mode is on by default (emits <think>...</think>).

Quantization Strategy

BaseQuant_XL recipe — precision is allocated by layer importance, not applied uniformly:

Layers	Bits	Rationale
`mlp.gate` (router), `shared_expert_gate`, `lm_head`, `shared_expert`	bf16	Routing decisions and output projection — any quantization noise here causes expert misrouting or output degradation
`embed_tokens`, `self_attn`, `linear_attn`	8-bit	Every-token layers — near-lossless, attention quality preserved
`vision_tower`, `switch_mlp` (routed experts)	6-bit	Bulk parameters — 256 experts with only 8 active per token; redundancy absorbs quantization noise. 6-bit is the sweet spot for routed experts (higher bits can cause overthinking)

Bits per weight: ~6.8 · Total size: ~30 GB · Group size: 64

Notes specific to this build:

Source config declares an MTP (multi-token prediction) layer but ships no MTP weights — the phantom declaration is stripped in this build so engines don't mis-flag it.
Like its Ornith-1.0 parent, the source stores experts individually (256 per layer) — converted via a custom sanitize that merges them into stacked expert tensors.
Ships the qwen3.6-froggeric-v21.3 chat template — thinking on by default, preserve_thinking, <|think_on|>/<|think_off|> inline toggles, XML and JSON tool-call formats.

Recommended Inference Parameters

Parameter	Value
`temperature`	0.6 (general tasks) · 1.0 (reproducing reported benchmarks)
`top_p`	0.95
`top_k`	20
`presence_penalty`	1.1
`reasoning_parser`	qwen3
`tool_call_parser`	qwen3_xml

Thinking is on by default (<think>...</think> before the answer); with a reasoning parser enabled the chain-of-thought is returned in a separate reasoning_content field.

Model Overview

Property	Value
Architecture	Qwen3.5-family MoE (35B-A3B, Ornith-1.0 derivative) + native vision encoder
Parameters	35.1B total / ~3B active per token
Experts	256 (8 routed + 1 shared)
Attention	Hybrid — 30 linear_attn + 10 full attention (40 layers)
Modalities	text, image, video → text
Context window	262,144 tokens native
Thinking	`<think>...</think>` (default on)
Tool calling	XML (default) and JSON formats
License	Apache 2.0

Source Model Benchmarks (from OrionLLM)

Benchmark	GRM-3.2-Sky
MMLU-Pro	89.5
MMLU-Redux	96.9
GPQA Diamond	90.6
LiveCodeBench v6	87.7
HMMT Feb 26	86.4
AIME26	96.3
SWE-bench Verified	81.4
SWE-bench Pro	58.3
Terminal-Bench 2.1	66.3
DeepSWE	19.5