Instructions to use Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp")
config = load_config("Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Yanun/Swift-Qwen3.8-27b-oQ4e-fp16-mtp" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Swift-Qwen3.8-27b-oQ4e-fp16-mtp / README.md

Yanun

Replace model description with architecture and precision details

949fba8 verified 19 days ago

preview code

Raw History Blame Contribute Delete

2.27 kB

	---
	library_name: mlx
	base_model: ukisai/Swift-Qwen3.8-27b
	base_model_relation: quantized
	license: other
	license_name: swift-open-license-1.0
	license_link: https://huggingface.co/ukisai/Swift-Qwen3.8-27b#license-and-access
	pipeline_tag: image-text-to-text
	tags:
	- mlx
	- omlx
	- quantized
	- mtp
	---

	# Swift-Qwen3.8-27b-oQ4e-fp16-mtp

	## Model architecture

	A 27B-class dense multimodal model stored in MLX format. It contains a text backbone, a vision encoder, and one multi-token prediction (MTP) layer.

	\| Component \| Structure \|
	\|---\|---\|
	\| Text backbone \| 64 layers; hidden size 5,120; feed-forward size 17,408 \|
	\| Attention layout \| 48 linear-attention layers and 16 full-attention layers, with full attention every fourth layer \|
	\| Full attention \| 24 query heads, 4 key/value heads, head dimension 256 \|
	\| Vocabulary \| 248,320 tokens \|
	\| Configured context limit \| 262,144 tokens; usable length depends on runtime settings and available memory \|
	\| Vision encoder \| 27 layers; hidden size 1,152; 16 attention heads; 16 × 16 image patches \|
	\| Vision-to-text connection \| Vision features are projected to the text hidden size of 5,120 \|
	\| MTP \| One additional prediction layer with attention and feed-forward projections \|

	## Weight precision

	The oQ4e checkpoint uses mixed precision rather than uniform 4-bit weights:

	- The default quantization is 4-bit affine, with 64 values per group.
	- 187 modules have explicit 5-bit overrides in `config.json`.
	- The MTP layer's seven large attention and feed-forward matrices use 4-bit weights.
	- The MTP fusion matrix (`mtp.fc`) and normalization weights remain FP16. The fusion matrix maps 10,240 input features to 5,120 output features.
	- The MTP quantization scales and offsets are stored in FP16.

	The `fp16` suffix describes the retained floating-point precision; it does not mean the entire model or MTP layer is FP16. Exact per-module settings are recorded in `config.json`.

	## Source and license

	Source revision: `54e66d6c81439bd4fda5ef9a690fa571e3b0d272`.

	Original model by UkisAI. This conversion does not change the upstream Swift Open License v1.0 terms. Consult the [source model license and access information](https://huggingface.co/ukisai/Swift-Qwen3.8-27b#license-and-access).