Instructions to use ukisai/Swift-1.5-5bit-MLX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ukisai/Swift-1.5-5bit-MLX with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("ukisai/Swift-1.5-5bit-MLX")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use ukisai/Swift-1.5-5bit-MLX with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "ukisai/Swift-1.5-5bit-MLX"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "ukisai/Swift-1.5-5bit-MLX"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use ukisai/Swift-1.5-5bit-MLX with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "ukisai/Swift-1.5-5bit-MLX"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "ukisai/Swift-1.5-5bit-MLX"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "ukisai/Swift-1.5-5bit-MLX",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use ukisai/Swift-1.5-5bit-MLX with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "ukisai/Swift-1.5-5bit-MLX"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ukisai/Swift-1.5-5bit-MLX

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use ukisai/Swift-1.5-5bit-MLX with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "ukisai/Swift-1.5-5bit-MLX"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "ukisai/Swift-1.5-5bit-MLX" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Swift-1.5-5bit-MLX / USAGE.md

ukisai

initial release

e476358 11 days ago

preview code

Raw History Blame

4.96 kB

Swift 1.5 5-bit — complete MLX architecture

Use only a complete snapshot: all four shards, original index, tokenizer and processor/runtime files are required. The 19.28 GB tensor payload plus runtime, cache and OS must fit available memory. Do not load this full model on a 16 GiB Mac or raise system memory limits to conceal insufficient hardware.

Pin the snapshot and verify it

Use a new working directory. After installing the HF CLI, run hf auth login interactively if not already signed in with access to this private repository. The command below resolves current main once to a full commit and then uses only that pinned snapshot. For a repeat run, reuse the recorded commit. Do not use an incomplete historical upload or proceed after verification failure.

python3.12 -m venv .venv-swift5
source .venv-swift5/bin/activate
python -m pip install 'huggingface_hub==1.31.0'
SWIFT_MLX_REVISION="$(python -c 'from huggingface_hub import HfApi; print(HfApi().model_info("ukisai/Swift-1.5-5bit-MLX").sha)')"
printf 'Pinned model revision: %s\n' "$SWIFT_MLX_REVISION"
hf download ukisai/Swift-1.5-5bit-MLX --revision "$SWIFT_MLX_REVISION" --local-dir Swift-1.5-5bit-MLX
hf cache verify ukisai/Swift-1.5-5bit-MLX --revision "$SWIFT_MLX_REVISION" --local-dir Swift-1.5-5bit-MLX --fail-on-missing-files
python Swift-1.5-5bit-MLX/verify_release.py Swift-1.5-5bit-MLX

The included checker has no network or model-loading code. Supply its optional --manifest-sha256 argument from a trusted release plan to pin the manifest too. Without that trusted digest it checks consistency, not source authenticity. Stop if any file, checksum, index entry or payload-boundary check fails.

Install the patches in this order

git clone https://github.com/ml-explore/mlx-lm.git swift5-mlx-lm
git -C swift5-mlx-lm checkout --detach c69d1288440a0dc4e6401fc417098b07598dccd5
git -C swift5-mlx-lm apply --check ../Swift-1.5-5bit-MLX/compatibility/swift15-mlx-lm.patch
git -C swift5-mlx-lm apply ../Swift-1.5-5bit-MLX/compatibility/swift15-mlx-lm.patch
git -C swift5-mlx-lm apply --check ../Swift-1.5-5bit-MLX/compatibility/enable-5bit.patch
git -C swift5-mlx-lm apply ../Swift-1.5-5bit-MLX/compatibility/enable-5bit.patch

Apple Silicon:

python -m pip install 'mlx==0.32.2' 'transformers==5.14.1' 'huggingface_hub==1.31.0' 'pillow==12.3.0'
python -m pip install -e ./swift5-mlx-lm

Linux CPU, Python 3.12, glibc 2.35 or newer:

python -m pip install 'mlx[cpu]==0.32.2' 'transformers==5.14.1' 'huggingface_hub==1.31.0' 'pillow==12.3.0'
python -m pip install -e ./swift5-mlx-lm

The historical Linux environment records Hub 1.32.0. The 1.31.0 pin above was separately installed in the independent macOS audit, where 18 synthetic patch tests passed. These environments are not identical and the tests did not load the full 27B model. The upstream code MIT notice is included separately from weight licenses.

Text generation

import mlx.core as mx
from mlx_lm import generate, load
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load("Swift-1.5-5bit-MLX")
if mx.default_device() == mx.cpu:
    model.apply(lambda x: x.astype(mx.float32) if mx.issubdtype(x.dtype, mx.floating) else x)
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Reply with exactly: Hello from Swift."}],
    tokenize=False, add_generation_prompt=True, enable_thinking=False,
)
mx.random.seed(20260922)
print(generate(model, tokenizer, prompt=prompt, max_tokens=32, sampler=make_sampler(temp=0)))

The CPU branch changes only in-memory floating types; packed UINT32 weights and files are unchanged. The historical build reported a Linux BF16 accumulation issue. Its original diagnostic file was not published. The separately executed macOS CPU/Metal diagnostic uses synthetic tensors: summing 8,192 ones gives 256 on CPU BF16 and 8,192 with FP32; Metal BF16/FP32 also give 8,192. It is not a new Linux or full-model generation test.

The template supports reasoning_effort="low", "medium", and "xhigh". Template support does not establish generated quality for those modes. The explicit model.mtp_logits step and model.visual encoder have historical component evidence. Integrated image/video chat and speculative generation are not implemented; unsupported multimodal generation must not be reported as working.

Conversion provenance

Conversion is not part of installation. If separately authorized, use only the complete customized Swift BF16 source identified in QUANTIZATION_MANIFEST.json, with its 18 shards verified before conversion, the pinned patched converter, and affine / 5-bit / group size 64. Do not fill missing weights with base Qwen or another quantization. Existing quantized weights are unchanged by these packaging repairs.