Instructions to use Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp")
config = load_config("Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Swift-Qwen3.8-27b-oQ4e-fp16-mtp / README.md

Yanun

Replace model description with architecture and precision details

949fba8 verified 18 days ago

preview code

Raw History Blame Contribute Delete

2.27 kB

metadata

library_name: mlx
base_model: ukisai/Swift-Qwen3.8-27b
base_model_relation: quantized
license: other
license_name: swift-open-license-1.0
license_link: https://huggingface.co/ukisai/Swift-Qwen3.8-27b#license-and-access
pipeline_tag: image-text-to-text
tags:
  - mlx
  - omlx
  - quantized
  - mtp

Swift-Qwen3.8-27b-oQ4e-fp16-mtp

Model architecture

A 27B-class dense multimodal model stored in MLX format. It contains a text backbone, a vision encoder, and one multi-token prediction (MTP) layer.

Component	Structure
Text backbone	64 layers; hidden size 5,120; feed-forward size 17,408
Attention layout	48 linear-attention layers and 16 full-attention layers, with full attention every fourth layer
Full attention	24 query heads, 4 key/value heads, head dimension 256
Vocabulary	248,320 tokens
Configured context limit	262,144 tokens; usable length depends on runtime settings and available memory
Vision encoder	27 layers; hidden size 1,152; 16 attention heads; 16 × 16 image patches
Vision-to-text connection	Vision features are projected to the text hidden size of 5,120
MTP	One additional prediction layer with attention and feed-forward projections

Weight precision

The oQ4e checkpoint uses mixed precision rather than uniform 4-bit weights:

The default quantization is 4-bit affine, with 64 values per group.
187 modules have explicit 5-bit overrides in config.json.
The MTP layer's seven large attention and feed-forward matrices use 4-bit weights.
The MTP fusion matrix (mtp.fc) and normalization weights remain FP16. The fusion matrix maps 10,240 input features to 5,120 output features.
The MTP quantization scales and offsets are stored in FP16.

The fp16 suffix describes the retained floating-point precision; it does not mean the entire model or MTP layer is FP16. Exact per-module settings are recorded in config.json.

Source and license

Source revision: 54e66d6c81439bd4fda5ef9a690fa571e3b0d272.

Original model by UkisAI. This conversion does not change the upstream Swift Open License v1.0 terms. Consult the source model license and access information.