Sorry, I just used here what GPT Sol 5.6 told me to do, so I do not understand

no problem at all 😊

i'll try to keep it simple then: the quantization you showed is uniform, meaning 6-bit is the fixed precision for all layers. oQ6 uses mixed precision, so 6-bit is the floor, but it calibrates to measure how sensitive each layer actually is, so some layers can use higher precision in order to minimize loss compared to the bf16 version.

Sorry, I forgot to thank you a few days ago. Understood. Thank You!
But I am still confused. While I quantized it said: Quantized model with 6.824 bits per weight.
I guess it's because I used q_mode="affine" so it must be similar to your method.

Anyway, I like this model. So far it seems to be the best for macOS. Running it on M4 Max 128GB. I refuse to use Chinese models out of principle. I tried Qwen 3.8 27B about which everybody is talking but in full precision it barely gets to 7-8 tokens/s. This quantized version gets to 33-34 tokens/s. It fails the car wash prompt, but I think Poolside has not pretrained the model for benchmaxxing. It is a very smart model.
I am attaching my complete procedure for running this model on a 128 GB M(x) Mac.

Laguna S 2.1 + Pool + SearXNG on Apple Container

This guide combines the Laguna MLX and SearXNG MCP recipes for an M4 Max Mac with 128 GB of unified memory.

Final architecture

Run the components in this split:

Do not put Laguna in Apple Container: MLX needs native access to Metal and unified memory. Do not put the stdio MCP bridge in the container either: Pool launches that bridge as a local child process. SearXNG is the only component that benefits from container isolation.

The request flow is:

Pool -> MLX-VLM -> Laguna S 2.1
  |
  +-> local stdio MCP bridge -> 127.0.0.1:8888 -> SearXNG container -> internet

Compatibility snapshot

This recipe is pinned as of 18 August 2026:

• Python 3.13 for Laguna/MLX.
• ==mlx0.32.0.==
• ==mlx-lm0.31.3==.
• ==mlx-vlm0.6.15.==
• ==transformers5.14.1==.
• Python 3.12 and ==mcp2.0.0 for the SearXNG bridge.==

==mlx-vlm0.6.7== was the first release with Laguna S support, but later releases contain Laguna tokenization and tool-call fixes. Use ==0.6.15== for the Pool/MCP workflow.

Before starting

You need:

• Apple Container installed and working.
• ==python3.13== and ==python3.12== available.
• At least roughly 350–360 GB free while the BF16 source and converted model coexist.
• A Hugging Face read token entered interactively with ==hf auth login==.
• A real project directory for Pool, for example ==/Users//Developer/your-project==.

Security note: an access token was pasted into the original Laguna recipe. It is deliberately omitted here. If it was a real token, revoke it in Hugging Face settings and create a replacement.

Part 1 — Install Laguna and MLX-VLM

1. Create the Laguna environment

python -m venv /Users/<username>/laguna-venv
source /Users/<username>/laguna-venv/bin/activate

python -m pip install --upgrade pip setuptools wheel
python -m pip install "mlx==0.32.0" "mlx-lm==0.31.3" "mlx-vlm==0.6.15" "transformers==5.14.1" "numpy==2.5.1" "huggingface-hub==1.24.0" "hf-xet==1.5.2" "safetensors==0.8.0"

python -m pip check
python -c 'import mlx, mlx_vlm, transformers; print("MLX environment OK")'

Keep the exact environment after the conversion; it will also serve the model.

2. Download the Laguna S 2.1 BF16 source

Authenticate interactively. Never paste the token into a script or guide.

source /Users/<username>/laguna-venv/bin/activate
hf auth login

Paste a Hugging Face read token when requested.
Download the checkpoint:

HF_XET_HIGH_PERFORMANCE=1 caffeinate -ims hf download poolside/Laguna-S-2.1 --local-dir /Users/<username>s/Laguna-S-2.1

Verify the 46 model shards and the index files:

find /Users/<username>/Laguna-S-2.1 -maxdepth 1 -type f -name 'model-*-of-00046.safetensors' | wc -l

Expected: ==46==.

test -s /Users/<username>/Laguna-S-2.1/config.json && test -s /Users/<username>/Laguna-S-2.1/model.safetensors.index.json && echo "BF16 source: OK"

3. Convert to Q6 group 64 with a BF16 spine

Raise the wired-memory limit after each reboot. This setting is temporary and returns to the macOS default after reboot.

sudo sysctl -w iogpu.wired_limit_mb=112000

Make sure the final and partial target directories do not already exist before starting a fresh conversion:

test ! -e /Users/<username>/Laguna-S-2.1-MLX-Q6-g64-BF16-spine
test ! -e /Users/<username>/Laguna-S-2.1-MLX-Q6-g64-BF16-spine.partial

Convert only the routed-expert projections. The remaining weights stay BF16:

python -u -c 'from mlx_vlm.convert import convert; convert(hf_path="/Users/<username>/Laguna-S-2.1", mlx_path="/Users/<username>/Laguna-S-2.1-MLX-Q6-g64-BF16-spine.partial", quantize=True, q_group_size=64, q_bits=6, q_mode="affine", dtype="bfloat16", trust_remote_code=True, quant_predicate=lambda path, module: {"group_size": 64, "bits": 6, "mode": "affine"} if path.endswith(("mlp.switch_mlp.gate_up_proj", "mlp.switch_mlp.down_proj")) else False)'

The expected final conversion message is approximately:

Quantized model with 6.824 bits per weight.

4. Validate before renaming

python -c 'import json; path="/Users/<username>/Laguna-S-2.1-MLX-Q6-g64-BF16-spine.partial"; weights=json.load(open(path+"/model.safetensors.index.json"))["weight_map"]; expected={f"language_model.model.layers.{layer}.mlp.switch_mlp.{projection}" for layer in range(1,48) for projection in ("gate_up_proj","down_proj")}; scales={name[:-7] for name in weights if name.endswith(".scales")}; biases={name[:-7] for name in weights if name.endswith(".biases")}; assert not scales.symmetric_difference(expected), f"Scales failed: missing={sorted(expected-scales)[:5]}, extra={sorted(scales-expected)[:5]}"; assert not biases.symmetric_difference(expected), f"Biases failed: missing={sorted(expected-biases)[:5]}, extra={sorted(biases-expected)[:5]}"; print("Validation PASS: exactly 94 routed-expert modules are quantized.")'

Only after the validation passes:

mv /Users/<username>/Laguna-S-2.1-MLX-Q6-g64-BF16-spine.partial /Users/<username>/Laguna-S-2.1-MLX-Q6-g64-BF16-spine
du -sh /Users/<username>/Laguna-S-2.1-MLX-Q6-g64-BF16-spine

Expected size: approximately ==93G==.

5. Smoke-test local generation

mlx_vlm.generate --model /Users/<username>/Laguna-S-2.1-MLX-Q6-g64-BF16-spine --trust-remote-code --prompt "Reply with exactly: Laguna MLX OK" --thinking-mode disabled --max-tokens 32 --temperature 0 --seed 0 --max-kv-size 8192 --prefill-step-size 512

Part 2 — Run SearXNG in Apple Container

6. Start the Apple Container services

container system start
container system status

7. Create persistent SearXNG directories and settings

mkdir -p /Users/<username>/searxng/config /Users/<username>/searxng/data

umask 077
SEARXNG_SECRET="$(openssl rand -hex 32)"

cat > /Users/<username>/searxng/config/settings.yml <<EOF
use_default_settings: true

general:
  debug: false
  instance_name: "<username> SearXNG"

search:
  safe_search: 0
  autocomplete: ""
  formats:
    - html
    - json

server:
  secret_key: "$SEARXNG_SECRET"
  limiter: false
  image_proxy: false

valkey:
  url: false
EOF

unset SEARXNG_SECRET

umask 022
chmod 600 /Users/<username>/searxng/config/settings.yml

==json== must remain enabled because the MCP bridge calls SearXNG's JSON search API. Disabling the limiter is suitable only because the host port is bound to loopback and the instance is private.

8. Pull and run SearXNG

container image pull docker.io/searxng/searxng:latest

container run --name searxng --detach --init --cpus 2 --memory 2G --dns 1.1.1.1 --dns 1.0.0.1 --publish 127.0.0.1:8888:8080 --volume /Users/<username>/searxng/config:/etc/searxng --volume /Users/<username>/searxng/data:/var/cache/searxng docker.io/searxng/searxng:latest

Using ==127.0.0.1:8888:8080==, rather than only ==8888:8080==, avoids exposing the private service to the LAN.

Inspect it:

container ls
container logs -n 100 searxng

9. Verify the JSON search API

curl -sS -G http://127.0.0.1:8888/search --data-urlencode 'q=Apple MLX documentation' --data-urlencode 'format=json' | python3 -m json.tool

Compact check:

curl -sS -G http://127.0.0.1:8888/search --data-urlencode 'q=Apple MLX documentation' --data-urlencode 'format=json' | python3 -c 'import json,sys; d=json.load(sys.stdin); results=d.get("results", []); print("results:", len(results));[print(r.get("title"), "->", r.get("url")) for r in results[:5]]'

Do not continue until this succeeds. A failure here belongs to Apple Container or SearXNG, not Pool or MCP.

Part 3 — Add a small SearXNG MCP bridge

10. Create a separate MCP environment

Do not install MCP into the Laguna environment.

deactivate
python -m venv /Users/<username>/searxng-mcp-venv
source /Users/<username>/searxng-mcp-venv/bin/activate
python -m pip install --upgrade pip setuptools wheel
python -m pip install "mcp==2.0.0"

python -m pip check

11. Create the MCP bridge

mkdir -p /Users/<username>/.local/lib/pool-tools

Create /Users//.local/lib/pool-tools/searxng_mcp.py with this content:

#!/usr/bin/env python

import json
import os
from typing import Any, Literal
from urllib.parse import urlencode
from urllib.request import Request, urlopen

from mcp.server import MCPServer


SEARXNG_URL = os.environ.get(
    "SEARXNG_URL", "http://127.0.0.1:8888"
).rstrip("/")

mcp = MCPServer(
    "searxng",
    instructions=(
        "Search the public internet through the user's local SearXNG instance. "
        "Use web_search for current, recent, version-specific, or source-backed "
        "information. Prefer primary sources whenever possible."
    ),
)


def _normalize_engines(value: Any) -> list[str]:
    if value is None:
        return []
    if isinstance(value, list):
        return [str(item) for item in value]
    return [str(value)]


@mcp.tool()
def web_search(
    query: str,
    max_results: int = 8,
    language: str = "all",
    time_range: Literal["", "day", "month", "year"] = "",
) -> dict[str, Any]:
    """Search the internet through the user's local SearXNG service.

    Args:
        query: Search query.
        max_results: Maximum unique results to return, from 1 through 20.
        language: SearXNG language code such as "en", "de", or "all".
        time_range: Optional recency filter: "day", "month", "year", or empty.
    """
    query = query.strip()
    if not query:
        raise ValueError("query must not be empty")

    max_results = max(1, min(int(max_results), 20))

    params: dict[str, str] = {
        "q": query,
        "format": "json",
        "safesearch": "0",
        "pageno": "1",
    }

    if language and language != "all":
        params["language"] = language

    if time_range:
        params["time_range"] = time_range

    request = Request(
        f"{SEARXNG_URL}/search?{urlencode(params)}",
        headers={
            "Accept": "application/json",
            "User-Agent": "pool-searxng-mcp/1.0",
        },
        method="GET",
    )

    try:
        with urlopen(request, timeout=20) as response:
            payload = json.load(response)
    except Exception as exc:
        raise RuntimeError(f"SearXNG request failed: {exc}") from exc

    results: list[dict[str, Any]] = []
    seen_urls: set[str] = set()

    for item in payload.get("results", []):
        url = str(item.get("url") or "").strip()
        if not url or url in seen_urls:
            continue

        seen_urls.add(url)
        snippet = str(item.get("content") or "").strip()
        if len(snippet) > 1500:
            snippet = snippet[:1497] + "..."

        results.append(
            {
                "title": str(item.get("title") or "").strip(),
                "url": url,
                "snippet": snippet,
                "engines": _normalize_engines(
                    item.get("engines") or item.get("engine")
                ),
                "score": item.get("score"),
                "published": (
                    item.get("publishedDate") or item.get("published_date")
                ),
            }
        )

        if len(results) >= max_results:
            break

    return {
        "query": query,
        "result_count": len(results),
        "results": results,
    }


if __name__ in {"__main__"}:
    mcp.run()

Then run:

chmod 755 /Users/<username>/.local/lib/pool-tools/searxng_mcp.py
python -m py_compile /Users/<username>/.local/lib/pool-tools/searxng_mcp.py
python -c 'from mcp.server import MCPServer; print("MCP import OK")'

python -c 'import runpy; runpy.run_path("/Users/<username>/.local/lib/pool-tools/searxng_mcp.py", run_name="syntax_test"); print("Import PASS")'

deactivate

Do not launch the bridge manually and wait for a prompt. In stdio mode, silence is normal: Pool owns its stdin and stdout and starts it when needed.

Part 4 — Install and configure Pool

12. Install Pool

curl -fsSL https://downloads.poolside.ai/pool/install.sh | sh
grep -qxF 'export PATH="$HOME/.local/bin:$PATH"' /Users/<username>/.zprofile || echo 'export PATH="$HOME/.local/bin:$PATH"' >> /Users/<username>/.zprofile
source /Users/<username>/.zprofile
pool --version

13. Register the SearXNG MCP server

Remove an older entry, if present:

pool mcp remove searxng 2>/dev/null || true

Register the corrected MCP v2 bridge:

pool mcp add --env SEARXNG_URL=http://127.0.0.1:8888 searxng -- /Users/<username>/searxng-mcp-venv/bin/python /Users/<username>/.local/lib/pool-tools/searxng_mcp.py

Inspect what Pool saved:

pool mcp list
pool mcp get searxng

There is no ==TRANSPORT=stdio== variable here. A command-based Pool MCP registration uses stdio automatically.

14. Create a Pool launcher for Laguna

Create ==/Users//.local/bin/pool-laguna==:

#!/bin/zsh
set -euo pipefail

export PATH="$HOME/.local/bin:$PATH"
export POOLSIDE_API_KEY="EMPTY"
export POOLSIDE_STANDALONE_BASE_URL="http://127.0.0.1:8080/v1"
export POOLSIDE_STANDALONE_MODEL="/Users/<username>/Laguna-S-2.1-MLX-Q6-g64-BF16-spine"
export POOLSIDE_STANDALONE_CONTEXT_LENGTH="65536"
export EDITOR="nano"

project_dir="${1:-$PWD}"
exec pool -C "$project_dir"

Make it executable:

chmod 755 /Users/<username>/.local/bin/pool-laguna

The ==/v1== suffix in ==POOLSIDE_STANDALONE_BASE_URL== is required. The context setting must agree with the MLX-VLM ==--max-kv-size== used below so Pool compacts before exceeding the server limit.

Part 5 — Start and verify the complete stack

15. Start the Laguna server in Terminal 1

source /Users/<username>/laguna-venv/bin/activate
sudo sysctl -w iogpu.wired_limit_mb=112000
caffeinate -ims mlx_vlm.server --model /Users/<username>/Laguna-S-2.1-MLX-Q6-g64-BF16-spine --host 127.0.0.1 --port 8080 --max-tokens 12288 --max-kv-size 65536 --prefill-step-size 512 --max-num-seqs 1 --enable-thinking --thinking-budget 4096 --trust-remote-code --log-level INFO

==--max-num-seqs 1== prevents concurrent requests from multiplying the memory requirement. Start with a 4096-token thinking budget; raise it to 8192 only for unusually difficult tasks.

16. Check the API in Terminal 2

Health and model discovery:

curl -sS http://127.0.0.1:8080/health | python3 -m json.tool
curl -sS http://127.0.0.1:8080/v1/models | python3 -m json.tool

Plain chat completion:

curl -sS http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
    "model": "/Users/<username>/Laguna-S-2.1-MLX-Q6-g64-BF16-spine",
    "messages": [
      {"role": "user", "content": "Reply with exactly: API OK"}
    ],
    "temperature": 0,
    "max_tokens": 128,
    "enable_thinking": false
  }' | python3 -m json.tool

17. Run the essential tool-calling preflight

This proves that the OpenAI-compatible server can return structured tool calls, which Pool needs for MCP:

curl -sS http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
    "model": "/Users/<username>/Laguna-S-2.1-MLX-Q6-g64-BF16-spine",
    "messages": [
      {"role": "user", "content": "Call get_status with no arguments."}
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_status",
          "description": "Return a test status.",
          "parameters": {
            "type": "object",
            "properties": {},
            "additionalProperties": false
          }
        }
      }
    ],
    "tool_choice": "required",
    "temperature": 0,
    "max_tokens": 128,
    "enable_thinking": false
  }' | python3 -c 'import json,sys; d=json.load(sys.stdin); calls=d["choices"][0]["message"].get("tool_calls"); print(json.dumps(calls, indent=2)); assert calls, d'

Expected: a non-empty ==tool_calls== array containing ==get_status==. Do not troubleshoot Pool until this succeeds.

18. Start Pool in Terminal 2

Replace the project path with the real directory you want Pool to work in:

pool-laguna /Users/<username>/Developer/test

Inside Pool, inspect MCP status:

/mcp

Then test the complete loop:

Use the SearXNG web_search tool to find the official Apple Container documentation. Return five results with titles and URLs.

Choose Always allow: web_search if Pool asks for permission and you want searches to run without repeated confirmation.

Daily startup and shutdown

Normal startup order

1. Start Apple Container services if needed:
container system status || container system start
2. Make sure SearXNG is running, then verify its HTTP endpoint:
container ls --all
If the searxng container is stopped:
container start searxng
Verify SearXNG after it is running:
curl -fsS http://127.0.0.1:8888/ >/dev/null && echo "SearXNG OK"
3. Start the Laguna server in Terminal 1 with the command from step 15.
4. Start Pool in Terminal 2:
pool-laguna /path/to/project
   The SearXNG MCP bridge has no separate startup step; Pool starts it automatically.

Normal shutdown

Exit Pool normally, then stop the Laguna server with ==Ctrl-C==. SearXNG may remain detached.

To stop SearXNG too:

container stop searxng

Troubleshooting

If Laguna runs out of memory

Keep ==--max-num-seqs 1== and reduce both limits together:

MLX-VLM: --max-kv-size 32768
Pool:    POOLSIDE_STANDALONE_CONTEXT_LENGTH=32768

Also keep the thinking budget at 4096. Do not increase the wired-memory limit further merely to hide memory pressure; macOS still needs headroom.

I stopped reading after the fourth sentence, but it still took like 15 years of scrolling to get to the reply button. What are you yelling about?