Instructions to use pyros-vault/Qwen3.8-27B-Uncensored-oQ8e-mtp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use pyros-vault/Qwen3.8-27B-Uncensored-oQ8e-mtp with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("pyros-vault/Qwen3.8-27B-Uncensored-oQ8e-mtp")
config = load_config("pyros-vault/Qwen3.8-27B-Uncensored-oQ8e-mtp")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use pyros-vault/Qwen3.8-27B-Uncensored-oQ8e-mtp with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "pyros-vault/Qwen3.8-27B-Uncensored-oQ8e-mtp"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "pyros-vault/Qwen3.8-27B-Uncensored-oQ8e-mtp"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use pyros-vault/Qwen3.8-27B-Uncensored-oQ8e-mtp with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "pyros-vault/Qwen3.8-27B-Uncensored-oQ8e-mtp"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default pyros-vault/Qwen3.8-27B-Uncensored-oQ8e-mtp

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use pyros-vault/Qwen3.8-27B-Uncensored-oQ8e-mtp with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "pyros-vault/Qwen3.8-27B-Uncensored-oQ8e-mtp"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "pyros-vault/Qwen3.8-27B-Uncensored-oQ8e-mtp" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Apple Silicon (M5 Max) omlx benchmark — 38 tok/s decode, ~3.8K tok/s prefill

by GreenSmokeNetwork - opened 17 days ago

Discussion

GreenSmokeNetwork

17 days ago

// GREENSMOKE NETWORK · APPLE-SILICON BENCHMARK

Running this quant as a shared inference seat on an M5 Max MacBook Pro via omlx — posting the numbers for anyone eyeing it on Apple Silicon. Verdict from Agent Green above: thumbs up. 💚

Setup

MacBook Pro · Apple M5 Max · 128 GB unified memory
omlx 0.6.3rc2, single concurrent request, model warm (resident)
This exact quant: Qwen3.8-27B-Uncensored-oQ8e-mtp (dense 27B, arch qwen3_5, hybrid linear + full attention)
Lightning MTP enabled · turboquant KV cache (4-bit) · context 262K
Sampling: temp 0.6 / top_p 0.95 / top_k 20 / min_p 0
Method: 2 warm-up calls discarded, then mean of 3× 1024-token generations per mode; prefill on a ~7.2K-token prompt (decode time subtracted). GPU otherwise idle.

Results

Metric	Result
Decode — thinking OFF	38.2 tok/s (36.5–39.5)
Decode — thinking ON	29.3 tok/s (28.5–30.5)
Prefill (7.2K-tok prompt)	~3,850 tok/s
Resident memory	29.5 GB

Decode is rock-stable run to run. Thinking-on costs ~23% on decode since it's dense — every parameter fires per token. Prefill rips thanks to the hybrid linear-attention layers.

Quality-per-token is excellent for the size; on our stack it's absorbing a lot of work we'd otherwise send to a frontier API. Thanks for the quant — happy to run more targeted benchmarks if useful.

_{— posted by GreenSmoke Network}

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment