Load Swift 1.5 with its complete MLX architecture
Use the included patch with the pinned official Apple MLX-LM revision. Unpatched text-only Qwen support does not preserve this checkpoint's complete parameter tree.
Use Python 3.12 in a new working directory. Install the HF CLI before using it,
and pin the complete model revision as well as the MLX-LM source revision.
This repository is private: after installing the CLI, run hf auth login
interactively if not already signed in with an account that has access.
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install 'huggingface_hub==1.31.0'
SWIFT_MLX_REVISION=d2140379e1fd593002c92fa552b2b37fb6eb1159
hf download ukisai/Swift-1.5-4bit-MLX --revision "$SWIFT_MLX_REVISION" --local-dir Swift-1.5-4bit-MLX
hf cache verify ukisai/Swift-1.5-4bit-MLX --revision "$SWIFT_MLX_REVISION" --local-dir Swift-1.5-4bit-MLX --fail-on-missing-files
git clone https://github.com/ml-explore/mlx-lm.git swift15-mlx-lm
git -C swift15-mlx-lm checkout --detach c69d1288440a0dc4e6401fc417098b07598dccd5
git -C swift15-mlx-lm apply --check ../Swift-1.5-4bit-MLX/compatibility/swift15-mlx-lm.patch
git -C swift15-mlx-lm apply ../Swift-1.5-4bit-MLX/compatibility/swift15-mlx-lm.patch
Stop after any missing-file or checksum failure. The checkpoint contains about 15.83 GB of tensor data, before runtime, cache and OS overhead. Do not force the full model onto a 16 GiB Mac or increase system memory limits. Full-model Apple generation remains unverified. The recorded small Metal samples are not a full run.
On Apple Silicon:
pip install 'mlx==0.32.2' 'transformers==5.14.1' 'huggingface_hub==1.31.0' pillow
pip install -e ./swift15-mlx-lm
On Linux CPU (Python 3.12 and glibc 2.35 or newer):
pip install 'mlx[cpu]==0.32.2' 'transformers==5.14.1' 'huggingface_hub==1.31.0' pillow
pip install -e ./swift15-mlx-lm
Text generation:
import mlx.core as mx
from mlx_lm import load, generate
model, tokenizer = load("Swift-1.5-4bit-MLX")
if mx.default_device() == mx.cpu:
model.apply(
lambda value: value.astype(mx.float32)
if mx.issubdtype(value.dtype, mx.floating) else value
)
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": "Say hello."}],
tokenize=False,
add_generation_prompt=True,
enable_thinking=False,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=32))
The Linux CPU branch promotes only in-memory floating parameters to FP32. Packed
4-bit weights and all files remain unchanged. This avoids the official MLX 0.32.2
Linux scalar BF16 quantized-matmul accumulation bug reproduced in
compatibility/cpu-quantized-matmul-diagnostic.json (8,192 exact ones summed to
256 in BF16, versus the correct 8,192 in FP32). The release's CPU generation,
MTP and vision smoke tests use this FP32 runtime. Apple Silicon inference does
not use this CPU workaround; full-model Apple Silicon execution was not tested.
The original chat template also accepts reasoning_effort="low", "medium",
and "xhigh"; this release validates the original low and xhigh formats.
This structural smoke test does not establish long-context or benchmark accuracy.
The patch implements an explicit MTP step (model.mtp_logits) and the vision
encoder (model.visual). Their weights are retained and the release validation
records their component execution. Speculative generation and integrated image/video
chat are not implemented. Unsupported multimodal generation calls raise an error.
Reproduce conversion only from the complete original Swift BF16 export identified
in QUANTIZATION_MANIFEST.json, after verifying its 18 shards and original assets:
mlx_lm.convert --hf-path /path/to/Swift-1.5-BF16 \
--mlx-path Swift-1.5-4bit-MLX \
--quantize --q-mode affine --q-bits 4 --q-group-size 64
The converter refuses an existing output directory. It uses official MLX-LM lazy loading, quantization, sharding, and saving; the patch supplies the complete model and strict parameter/asset mapping.