Add a verified Swift runtime installer and explicit server launcher (#2)

- Add a verified Swift runtime installer and explicit server launcher (47701522752d40e1a5d0c3ce2ee235a897cc08af)

QUICKSTART.md

@@ -0,0 +1,98 @@
+# Start Swift with the required MLX runtime
+
+`Received 501 parameters not in model: language_model.visual...` means the
+selected loader does not represent this checkpoint's vision parameter tree.
+For example, Homebrew's MLX-LM 0.31.3 runs from its own Python environment and
+does not acquire the Swift patches when you download this model. This error
+happens before generation; changing prompt-cache settings does not fix it.
+
+The helper below installs the same pinned architecture and cache patches used
+in [FULL_MAC_VALIDATION.md](FULL_MAC_VALIDATION.md). It creates a separate Python
+environment and an explicit launcher. Your cached weights can stay where they are.
+
+## 1. Use Python 3.12 on Apple Silicon
+
+```bash
+python3.12 --version
+```
+
+If that command is missing and you use Homebrew, install it with
+`brew install python@3.12`. Leave your existing Homebrew MLX-LM installation alone.
+Git is also required. The helper checks the Python version and native Apple
+Silicon platform before installing anything.
+
+## 2. Install the runtime using your existing model directory
+
+Stop your old server. Download only the small setup helper:
+
+```bash
+hf download ukisai/Swift-1.5-4bit-MLX swift_runtime.py --local-dir Swift-MLX-setup
+```
+
+Set `SWIFT_MODEL` to the complete model directory printed by your earlier
+`hf download` command. It can be an HF cache snapshot or a local download folder.
+This example uses the previously published snapshot in the default HF cache:
+
+```bash
+SWIFT_MODEL="$HOME/.cache/huggingface/hub/models--ukisai--Swift-1.5-4bit-MLX/snapshots/730aab9b0395b26f7d9cf4b0dfa6a4c788aff6fd"
+python3.12 Swift-MLX-setup/swift_runtime.py setup --model "$SWIFT_MODEL"
+```
+
+If you downloaded with `--local-dir Swift-1.5-4bit-MLX`, set
+`SWIFT_MODEL="$PWD/Swift-1.5-4bit-MLX"` instead. Use your actual path if you customized
+the HF cache location or downloaded a different complete revision.
+
+The helper verifies fixed SHA-256 hashes for the patches, fetches official
+MLX-LM revision `c69d1288440a0dc4e6401fc417098b07598dccd5`, applies the required
+architecture and cache patches, and installs pinned packages into
+`~/.local/share/swift15-mlx/4bit/.venv`. For 5-bit it also applies the existing
+5-bit support patch. It checks the selected architecture, server source and
+package versions before creating the launcher.
+
+Installation needs internet for source code and Python dependencies. It never
+downloads, converts, edits or re-quantizes the model weights. Setup checks local
+assets and indexed shard presence; use `check_download.py --hash` from
+[USAGE.md](USAGE.md) if you also need to verify every weight byte.
+
+## 3. Start with this command every time
+
+```bash
+"$HOME/.local/share/swift15-mlx/4bit/serve"
+```
+
+This starts the server at `http://127.0.0.1:8080` using the dedicated Python
+environment and your existing local model path. It enables offline Hub and
+Transformers operation. You do not need to activate a virtual environment.
+Keep using this launcher after closing and reopening Terminal; typing the bare
+`mlx_lm.server` command can select Homebrew again.
+
+To check the runtime without loading weights:
+
+```bash
+"$HOME/.local/share/swift15-mlx/4bit/serve" --check-only
+```
+
+Additional server arguments work, for example `serve --port 8081` using the same
+full launcher path. The tested cache defaults remain enabled. A previous
+`--prompt-cache-size 0` override disables reuse if you add it again.
+
+If the runtime directory already exists, use its launcher. The installer refuses
+to overwrite an existing directory; choose a new path with `--runtime-dir` if
+you need a separate installation, then use the launcher path it prints.
+Stop on an installation failure. A successful `--check-only` verifies the runtime
+and local file presence, not full-model inference or available memory.
+
+## Validation
+
+Both new 4-bit and 5-bit installations passed native Apple Silicon checks using
+existing tiny synthetic checkpoints containing text, vision and MTP entries.
+The actual launchers served two HTTP generation requests each while a conflicting
+`mlx_lm.server` and Python module were placed on the search path. Paths containing
+spaces worked. An unpatched loader was rejected before loading weights, and a
+repeat installation did not overwrite an existing runtime.
+
+[Recorded results](compatibility/launcher-validation.json) cover this installer
+and launch path. The earlier complete-model 48 GiB Mac tests remain in
+[FULL_MAC_VALIDATION.md](FULL_MAC_VALIDATION.md). No full-model rerun or hosted CI
+badge is claimed for this packaging update. Memory capacity, GUI integration,
+integrated image/video chat and speculative MTP limitations are unchanged.

README.md

@@ -34,6 +34,12 @@ converted with the official Apple MLX-LM converter using 4 bits and group size 6
 Swift 1.5 is UkisAI's reasoning-efficient Qwen3.8-27B derivative, focused on stronger
 long-horizon, agentic and coding performance while using fewer thinking tokens.
 
+**Starting from Homebrew or seeing `Received 501 parameters not in model`?**
+Use [QUICKSTART.md](QUICKSTART.md) to install the required runtime and create an
+explicit `serve` launcher. It reuses your existing model directory, including an
+HF cache snapshot. A bare `mlx_lm.server` command may select a separate Homebrew
+installation that lacks the Swift architecture and cache patches.
+
 **Runtime compatibility:** this complete checkpoint requires the supplied architecture patch
 and the pinned Python installation in [USAGE.md](USAGE.md). The tested unpatched
 MLX-LM 0.32.0 loader rejects 501 saved vision entries. Downloading the model does

SERVER_CACHE_UPDATE.md

@@ -1,5 +1,11 @@
 # Server cache update
 
+**Starting from Homebrew or seeing `Received 501 parameters not in model`?**
+Use [QUICKSTART.md](QUICKSTART.md) to install the required runtime and create an
+explicit `serve` launcher. It reuses your existing model directory, including an
+HF cache snapshot. A bare `mlx_lm.server` command may select a separate Homebrew
+installation that lacks the Swift architecture and cache patches.
+
 This update changes the pinned MLX-LM server, not the checkpoint. It is a Swift
 runtime patch based on official MLX-LM `c69d1288440a0dc4e6401fc417098b07598dccd5`, not an upstream release.
 The existing architecture patch remains required.

TROUBLESHOOTING.md

@@ -1,5 +1,11 @@
 # Check this download and its runtime
 
+**Starting from Homebrew or seeing `Received 501 parameters not in model`?**
+Use [QUICKSTART.md](QUICKSTART.md) to install the required runtime and create an
+explicit `serve` launcher. It reuses your existing model directory, including an
+HF cache snapshot. A bare `mlx_lm.server` command may select a separate Homebrew
+installation that lacks the Swift architecture and cache patches.
+
 This release needs **all 3 shards (15.83 GB (14.74 GiB))**, its index/config/tokenizer
 assets, and the supplied architecture patch. Downloading an HF repository does not install
 the MLX-LM patches into Python or into a GUI app's separate runtime.

UPLOAD_MANIFEST.json

@@ -34,25 +34,30 @@
       "bytes": 8230,
       "sha256": "9f061d15042d9368e2c6a406ee20071ba6d78ce8da4c3ca53bfc13f5c2f95dd1"
     },
+    {
+      "path": "QUICKSTART.md",
+      "bytes": 4671,
+      "sha256": "42046f33ae7e07154c89cacb32fe1569e4931dc82177406b03f5da91d28190cc"
+    },
     {
       "path": "README.md",
-      "bytes": 10407,
-      "sha256": "763ade1ad454b132b019879763b41b897cd324519a24fb16b85ee742d9736bce"
+      "bytes": 10794,
+      "sha256": "e30be618a534d318226726f04f7868242fc19550714dab1d617f48e7e3e35a45"
     },
     {
       "path": "SERVER_CACHE_UPDATE.md",
-      "bytes": 4950,
-      "sha256": "d85f05bc0bd339947b06af7d608b44898ac4b400ef680cfd6a4869910524eba1"
+      "bytes": 5337,
+      "sha256": "a6f648d3491eb4e9bd6f9c3291b38bc33aab1e64030e308b96a843f747b1cfc3"
     },
     {
       "path": "TROUBLESHOOTING.md",
-      "bytes": 5952,
-      "sha256": "ff92cf87f0bf35caf7bf8962be6eaaea0a132d781ee26fde83d8eae4e9b306d7"
+      "bytes": 6339,
+      "sha256": "8a0c5babebad8cc24ef5f3f7df3189806318d04dee723c145a67f5bc667343ae"
     },
     {
       "path": "USAGE.md",
-      "bytes": 8000,
-      "sha256": "8925823cdb938fcc5a597eef121486fbe2b3a593aa2753ec9028c46f9e3f51e2"
+      "bytes": 8387,
+      "sha256": "bbf3b30ad7cf5b6453ab485fbade6018cd736523f92e2de11560e5bb95c4bb39"
     },
     {
       "path": "chat_template.jinja",
@@ -169,6 +174,11 @@
       "bytes": 113,
       "sha256": "a40772b0d08d9f5ce659ee06cd5e5573ec0697073295d819a5dc9b6b1f4302bd"
     },
+    {
+      "path": "compatibility/launcher-validation.json",
+      "bytes": 1598,
+      "sha256": "30a08c065138836fbf9fa3c4b586e303742b5a7a155294177a15fc23e0efef50"
+    },
     {
       "path": "compatibility/mac-check/README.md",
       "bytes": 759,
@@ -314,6 +324,11 @@
       "bytes": 4738825,
       "sha256": "1cda6924169e8b83e5d6d7299baf8329ed1b1a182a481aa0e06344e0f5c00111"
     },
+    {
+      "path": "swift_runtime.py",
+      "bytes": 9029,
+      "sha256": "7a459033c6c419e3f7e8cabf59b303767d002039ef5e0a2e2999521951716cbf"
+    },
     {
       "path": "tokenizer.json",
       "bytes": 12809320,

USAGE.md

@@ -1,5 +1,11 @@
 # Load Swift 1.5 with its complete MLX architecture
 
+**Starting from Homebrew or seeing `Received 501 parameters not in model`?**
+Use [QUICKSTART.md](QUICKSTART.md) to install the required runtime and create an
+explicit `serve` launcher. It reuses your existing model directory, including an
+HF cache snapshot. A bare `mlx_lm.server` command may select a separate Homebrew
+installation that lacks the Swift architecture and cache patches.
+
 Use the included patch with the pinned official Apple MLX-LM revision. Unpatched
 text-only Qwen support does not preserve this checkpoint's complete parameter tree.
 

compatibility/launcher-validation.json

@@ -0,0 +1,56 @@
+{
+  "status": "PASS_LAUNCHER_INSTALL_AND_SYNTHETIC_HTTP",
+  "date": "2026-09-26",
+  "scope": "New isolated installation and real CLI launch on Apple Silicon with existing tiny synthetic 4-bit and 5-bit checkpoints. No full Swift weights loaded or requantized.",
+  "results": [
+    {
+      "bits": 4,
+      "clean_install": "PASS",
+      "runtime_identity": "PASS",
+      "path_shadowing_bypassed": "PASS",
+      "pythonpath_shadowing_bypassed": "PASS",
+      "paths_with_spaces": "PASS",
+      "fixture_tensor_entries": 190,
+      "vision_entries": 37,
+      "mtp_entries": 31,
+      "http_requests": [
+        {
+          "http_status": 200,
+          "completion_tokens": 4
+        },
+        {
+          "http_status": 200,
+          "completion_tokens": 4
+        }
+      ],
+      "worker_alive": true
+    },
+    {
+      "bits": 5,
+      "clean_install": "PASS",
+      "runtime_identity": "PASS",
+      "path_shadowing_bypassed": "PASS",
+      "pythonpath_shadowing_bypassed": "PASS",
+      "paths_with_spaces": "PASS",
+      "fixture_tensor_entries": 190,
+      "vision_entries": 37,
+      "mtp_entries": 31,
+      "http_requests": [
+        {
+          "http_status": 200,
+          "completion_tokens": 4
+        },
+        {
+          "http_status": 200,
+          "completion_tokens": 4
+        }
+      ],
+      "worker_alive": true
+    }
+  ],
+  "negative_checks": {
+    "stock_loader_rejected_before_loading": "PASS",
+    "existing_install_not_overwritten": "PASS"
+  },
+  "full_model_validation": "Previously recorded separately in FULL_MAC_VALIDATION.md; not rerun for this launcher."
+}

swift_runtime.py

@@ -0,0 +1,251 @@
+"""Install and launch the pinned Swift MLX runtime using an existing local model."""
+
+import argparse
+import hashlib
+import inspect
+import json
+import os
+import platform
+import shlex
+import shutil
+import subprocess
+import sys
+import urllib.request
+import venv
+from pathlib import Path
+
+BASE = "c69d1288440a0dc4e6401fc417098b07598dccd5"
+REVISIONS = {
+    4: "730aab9b0395b26f7d9cf4b0dfa6a4c788aff6fd",
+    5: "8aff72b145212e62c15146e41dcbef35ea5fa9ff",
+}
+PATCHES = {
+    "swift15-mlx-lm.patch": "f6f1d0bdafa45863bfbf93dac0398c481c993ea04fdf38b9bae98c643f89eaec",
+    "enable-5bit.patch": "b985961eac3035e05ca4c9f3a8b283c26e6dd69bab4d113997fc04fe2a4f99cc",
+    "swift15-server-cache.patch": "a7fbc0f0524ee7864d9f41a98a2e35acf0e63f53e369ee5b9a87ca12926a9eab",
+}
+MODEL_HASHES = {
+    4: "f559e4559946bfad4ce20b93906028449b76e007f42f36bb83d2c2c5f6cb95a9",
+    5: "ecda666d6d5f1059388a2ce4267d9ff8625398bb925d6c0840d0059ce17b676f",
+}
+SERVER_HASH = "791f8c1eb3431c4d24b4f9084f67b665c38e3d547ee4a307e52c5b2b6f12aba0"
+
+
+def digest(path):
+    return hashlib.sha256(Path(path).read_bytes()).hexdigest()
+
+
+def local_config(model):
+    config = json.loads((model / "config.json").read_text())
+    quant = config.get("quantization", {})
+    bits = quant.get("bits")
+    if bits not in REVISIONS or quant != {
+        "bits": bits,
+        "group_size": 64,
+        "mode": "affine",
+    }:
+        raise ValueError(
+            "Expected the Swift 1.5 affine 4-bit or 5-bit / group-size-64 checkpoint."
+        )
+    if config.get("language_model_only") is not False:
+        raise ValueError("Expected the complete Swift architecture configuration.")
+    for name in ("tokenizer.json", "tokenizer_config.json", "chat_template.jinja"):
+        if not (model / name).is_file():
+            raise ValueError(f"Missing local model asset: {name}")
+    index = json.loads((model / "model.safetensors.index.json").read_text())
+    shards = set(index["weight_map"].values())
+    if not shards:
+        raise ValueError("The checkpoint index is empty.")
+    for name in shards:
+        if Path(name).name != name or not name.endswith(".safetensors"):
+            raise ValueError("The checkpoint index contains an invalid shard path.")
+        if not (model / name).is_file():
+            raise ValueError(
+                f"Missing local weight shard: {name}; complete the existing download first."
+            )
+    return config, bits
+
+
+def runtime_check(config, bits):
+    from importlib.metadata import version
+
+    from mlx_lm import server
+    from mlx_lm.utils import _get_classes
+
+    cls, args = _get_classes(config)
+    args.from_dict(config)
+    if cls.__module__ != "mlx_lm.models.qwen3_5_full":
+        raise ValueError(
+            "Swift architecture patch is missing. Use the serve launcher created by setup."
+        )
+    if digest(inspect.getsourcefile(cls)) != MODEL_HASHES[bits]:
+        raise ValueError(
+            "The installed architecture differs from the pinned Swift implementation."
+        )
+    if digest(server.__file__) != SERVER_HASH:
+        raise ValueError("The tested server cache patch is missing or has changed.")
+    versions = {
+        name: version(name)
+        for name in ("mlx", "mlx-metal", "mlx-lm", "transformers", "huggingface_hub")
+    }
+    expected = {
+        "mlx": "0.32.2",
+        "mlx-metal": "0.32.2",
+        "mlx-lm": "0.32.0",
+        "transformers": "5.14.1",
+        "huggingface_hub": "1.31.0",
+    }
+    if versions != expected:
+        raise ValueError(
+            f"Installed package versions differ from the tested runtime: {versions}"
+        )
+    print(
+        json.dumps(
+            {
+                "status": "PASS_RUNTIME_IDENTITY",
+                "python": sys.executable,
+                "model_class": cls.__module__ + "." + cls.__name__,
+                "server": server.__file__,
+                "versions": versions,
+                "weights_loaded": False,
+            }
+        ),
+        flush=True,
+    )
+
+
+def setup(model, bits, destination):
+    if shutil.which("git") is None:
+        raise ValueError(
+            "Git is required. Install the macOS command-line tools, then retry."
+        )
+    if destination.exists():
+        raise ValueError(
+            f"Runtime directory already exists: {destination}. Use its serve launcher, or choose a new --runtime-dir."
+        )
+    destination.mkdir(parents=True)
+    source = destination / "mlx-lm"
+    envdir = destination / ".venv"
+    python = envdir / "bin/python"
+    patchdir = destination / "patches"
+    patchdir.mkdir()
+    patch_names = ["swift15-mlx-lm.patch"]
+    if bits == 5:
+        patch_names.append("enable-5bit.patch")
+    patch_names.append("swift15-server-cache.patch")
+    for name in patch_names:
+        url = f"https://huggingface.co/ukisai/Swift-1.5-{bits}bit-MLX/resolve/{REVISIONS[bits]}/compatibility/{name}"
+        with urllib.request.urlopen(url, timeout=60) as response:
+            data = response.read()
+        if hashlib.sha256(data).hexdigest() != PATCHES[name]:
+            raise ValueError(f"Patch checksum mismatch: {name}")
+        (patchdir / name).write_bytes(data)
+    subprocess.run(["git", "init", "--quiet", str(source)], check=True)
+    git = ["git", "-C", str(source)]
+    subprocess.run(
+        git + ["remote", "add", "origin", "https://github.com/ml-explore/mlx-lm.git"],
+        check=True,
+    )
+    subprocess.run(git + ["fetch", "--depth", "1", "origin", BASE], check=True)
+    subprocess.run(git + ["checkout", "--detach", BASE], check=True)
+    for name in patch_names:
+        subprocess.run(git + ["apply", "--check", str(patchdir / name)], check=True)
+        subprocess.run(git + ["apply", str(patchdir / name)], check=True)
+    venv.EnvBuilder(with_pip=True).create(envdir)
+    subprocess.run(
+        [
+            str(python),
+            "-I",
+            "-m",
+            "pip",
+            "install",
+            "mlx==0.32.2",
+            "mlx-metal==0.32.2",
+            "transformers==5.14.1",
+            "huggingface_hub==1.31.0",
+            "pillow==12.3.0",
+            "safetensors==0.8.0",
+            "-e",
+            str(source),
+        ],
+        check=True,
+    )
+    subprocess.run([str(python), "-I", "-m", "pip", "check"], check=True)
+    installed = destination / "swift_runtime.py"
+    shutil.copyfile(__file__, installed)
+    command = [str(python), "-I", str(installed), "start", "--model", str(model)]
+    subprocess.run(command + ["--check-only"], check=True)
+    launcher = destination / "serve"
+    launcher.write_text("#!/bin/sh\nexec " + shlex.join(command) + ' "$@"\n')
+    launcher.chmod(0o755)
+    print(
+        f"\nSetup complete. Start or restart with this exact command:\n{shlex.quote(str(launcher))}"
+    )
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    sub = parser.add_subparsers(dest="action", required=True)
+    install = sub.add_parser(
+        "setup",
+        help="Install into a new isolated directory; reuse existing model files.",
+    )
+    install.add_argument("--model", required=True, type=Path)
+    install.add_argument("--runtime-dir", type=Path)
+    start = sub.add_parser(
+        "start", help="Verify the selected Python runtime and serve a local checkpoint."
+    )
+    start.add_argument("--model", required=True, type=Path)
+    start.add_argument("--check-only", action="store_true")
+    args, extra = parser.parse_known_args()
+    if args.action == "setup" and extra:
+        parser.error("Unknown setup arguments: " + " ".join(extra))
+    if sys.version_info[:2] != (3, 12):
+        parser.error(
+            "Use python3.12 for this tested runtime. Homebrew's Python 3.14 environment is separate."
+        )
+    if platform.system() != "Darwin" or platform.machine() != "arm64":
+        parser.error("This installer and launcher target native Apple Silicon macOS.")
+    model = args.model.expanduser().resolve()
+    try:
+        config, bits = local_config(model)
+        if args.action == "setup":
+            target = (
+                args.runtime_dir
+                or Path.home() / ".local/share/swift15-mlx" / f"{bits}bit"
+            )
+            setup(model, bits, target.expanduser().resolve())
+            return
+        os.environ["HF_HUB_OFFLINE"] = "1"
+        os.environ["TRANSFORMERS_OFFLINE"] = "1"
+        runtime_check(config, bits)
+        if args.check_only:
+            return
+        os.execv(
+            sys.executable,
+            [
+                sys.executable,
+                "-I",
+                "-m",
+                "mlx_lm.server",
+                "--model",
+                str(model),
+                "--host",
+                "127.0.0.1",
+                "--port",
+                "8080",
+                *extra,
+            ],
+        )
+    except (
+        OSError,
+        ValueError,
+        KeyError,
+        ImportError,
+        subprocess.CalledProcessError,
+    ) as exc:
+        raise SystemExit(f"Swift runtime setup/start failed: {exc}") from None
+
+
+if __name__ == "__main__":
+    main()