Instructions to use OsaurusAI/Laguna-S-2.1-JANG_2L with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OsaurusAI/Laguna-S-2.1-JANG_2L with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("OsaurusAI/Laguna-S-2.1-JANG_2L")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use OsaurusAI/Laguna-S-2.1-JANG_2L with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "OsaurusAI/Laguna-S-2.1-JANG_2L"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "OsaurusAI/Laguna-S-2.1-JANG_2L"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use OsaurusAI/Laguna-S-2.1-JANG_2L with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "OsaurusAI/Laguna-S-2.1-JANG_2L"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "OsaurusAI/Laguna-S-2.1-JANG_2L"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "OsaurusAI/Laguna-S-2.1-JANG_2L",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use OsaurusAI/Laguna-S-2.1-JANG_2L with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "OsaurusAI/Laguna-S-2.1-JANG_2L"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default OsaurusAI/Laguna-S-2.1-JANG_2L

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use OsaurusAI/Laguna-S-2.1-JANG_2L with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "OsaurusAI/Laguna-S-2.1-JANG_2L"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "OsaurusAI/Laguna-S-2.1-JANG_2L" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Laguna-S-2.1-JANG_2L / README.md

Osaurus-AI

Laguna S 2.1 JANG_2L — AWQ-protected JANG affine bundle

6129d11 verified 2 months ago

preview code

Raw History Blame Contribute Delete

4.04 kB

	---
	language:
	- en
	library_name: mlx
	license: other
	license_name: openmdw-1.1
	license_link: https://openmdw.ai/
	pipeline_tag: text-generation
	base_model: poolside/Laguna-S-2.1
	tags:
	- mlx
	- jang
	- jang-2l
	- awq
	- quantized
	- apple-silicon
	- laguna
	- moe
	- agentic-coding
	---

	<p align="center"><a href="https://osaurus.ai"><img src="./osaurus-x-banner.png" alt="Osaurus AI"></a></p>

	# OsaurusAI/Laguna-S-2.1-JANG_2L

	JANG_2L JANG affine bundle of [poolside/Laguna-S-2.1](https://huggingface.co/poolside/Laguna-S-2.1) — 118B-parameter / ~8B-active MoE for agentic coding and long-horizon work, quantized for Apple Silicon with AWQ-protected routed experts.

	Everything quality-critical stays high precision: attention (including the softplus output gate), the shared expert path, router, and norms. The 256-routed-expert bulk (~97% of parameters) carries the low bits, protected by activation-aware (AWQ) scales selected by measured
	quantization error on real expert weights and real calibration activations — the search always includes a no-AWQ baseline, so scales ship only because they beat it (2.4% lower reconstruction error at this bit width).

	## Bundle

	\| Field \| Value \|
	\|---\|---\|
	\| Source \| `poolside/Laguna-S-2.1` @ `a50e85e` (BF16, 235 GB) \|
	\| Architecture \| `laguna` / `LagunaForCausalLM` — 48 layers (12 global + 36 SWA w512), GQA 8 KV heads, 256 routed experts top-10 + 1 shared, dense layer 0, 1M context \|
	\| On-disk size \| 44.3 GB (10 shards) \|
	\| Routed experts \| 2-bit gate/up, 3-bit down affine, group 64, AWQ input scales folded \|
	\| Attention q/k/v/o + g_proj \| 8-bit affine \|
	\| Shared expert / dense FFN \| 6-bit affine \|
	\| Embeddings / lm_head \| 6-bit / 8-bit affine \|
	\| Router, e_score bias, norms \| fp16 passthrough (routing bit-identical to unfolded math) \|
	\| Modality \| text-only (verified from tensor index — no vision/audio/video weights) \|

	## Measured performance (M5 Max, 128 GB)

	\| Metric \| Value \|
	\|---\|---\|
	\| Decode \| ~48 tok/s greedy \|
	\| Load time \| 3.0 s \|
	\| Long-context cache parity \| teacher-forced top-1 agreement 1.000 / 1.000 across a 2,913-token pass (pre / post the 512 sliding window) \|
	\| BF16 vs quantized \| greedy smoke + chat-with-thinking verified coherent \|

	Memory note: budget bundle size + ~10 GB. The reference Python runtime auto-sets the Metal wired limit (`min(bundle x 1.2 + 8 GB, 118 GB)`) — without a wired limit, decode throughput roughly halves on 64 GB+ working sets.

	## Chat protocol

	- GLM-style think tags. Thinking is ON by default (vendor `default_chat_template_kwargs.enable_thinking=true`); pass `enable_thinking=False` to disable. The generation prompt ends `<assistant><think>` (thinking) or `<assistant></think>` (off).
	- Stop tokens: `eos_token_id = [2, 24]` — id 24 is end-of-turn and MUST be in the stop set.
	- The template emits its own leading `〈\|EOS\|〉` (= bos id 2): do not prepend another BOS.
	- Tool calls: `<tool_call>name<arg_key>k</arg_key><arg_value>v</arg_value></tool_call>` (GLM-4.7-compatible parsing).
	- Sampling defaults (vendor): temperature 1.0, top_p 1.0, top_k 20.

	## Run it

	[Osaurus](https://osaurus.ai) loads this bundle natively. Python reference runtime (mixed-precision affine aware):

	```bash
	pip install mlx mlx-lm transformers
	python -m jang_tools.laguna.runtime --src ./Laguna-S-2.1-JANG_2L \
	--prompt 'def fibonacci(n):' --max-new 64
	```

	`config.json[quantization]` carries per-module `{bits, group_size, mode}` overrides — loaders must honor per-module bits (a single top-level width mis-dequantizes the low-bit experts).

	## Source model quality (poolside, BF16 base)

	Per the [upstream model card](https://huggingface.co/poolside/Laguna-S-2.1): Terminal-Bench 2.1 70.2%, SWE-bench Multilingual 78.5%, SWE-Bench Pro 59.4% — competitive with much larger frontier MoEs. These figures are for the BF16 source model.

	---

	Quantized and verified by Jinho Jang (eric@osaurus.ai) with the JANG toolchain. License: [OpenMDW-1.1](https://openmdw.ai/) (inherited from the source model).