Instructions to use JANGQ-AI/Holo3-35B-A3B-JANGTQ with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use JANGQ-AI/Holo3-35B-A3B-JANGTQ with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("JANGQ-AI/Holo3-35B-A3B-JANGTQ")
config = load_config("JANGQ-AI/Holo3-35B-A3B-JANGTQ")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use JANGQ-AI/Holo3-35B-A3B-JANGTQ with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "JANGQ-AI/Holo3-35B-A3B-JANGTQ"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "JANGQ-AI/Holo3-35B-A3B-JANGTQ"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use JANGQ-AI/Holo3-35B-A3B-JANGTQ with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "JANGQ-AI/Holo3-35B-A3B-JANGTQ"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default JANGQ-AI/Holo3-35B-A3B-JANGTQ

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use JANGQ-AI/Holo3-35B-A3B-JANGTQ with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "JANGQ-AI/Holo3-35B-A3B-JANGTQ"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "JANGQ-AI/Holo3-35B-A3B-JANGTQ" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Holo3-35B-A3B-JANGTQ / README.md

jangq

Add Holo3-35B-A3B JANGTQ2 (MLX) + VL tower + README

8cae8cf verified 5 months ago

preview code

Raw

History Blame

8.47 kB

	---
	language:
	- en
	library_name: mlx
	license: apache-2.0
	pipeline_tag: image-text-to-text
	base_model: Hcompany/Holo3-35B-A3B
	tags:
	- quantized
	- apple-silicon
	- mlx
	- qwen3_5_moe
	- moe
	- vision
	- hybrid-attention
	- gated-deltanet
	- turboquant
	- jangtq
	- jangtq2
	- agent
	- computer-use
	- gui-agents
	---

	<p align="center">
	<a href="https://osaurus.ai"><img src="./osaurus-x-banner.png" alt="Osaurus AI"></a>
	</p>

	<h3 align="center">Holo3 35B-A3B — JANGTQ2 (MLX)</h3>
	<p align="center">TurboQuant codebook quantization of H Company's Holo3 GUI-agent VLM — routed experts at 2-bit via Lloyd-Max codebooks + Hadamard rotation, attention / embed / shared-expert / lm_head at 8-bit affine, vision tower preserved.</p>

	<p align="center">
	<a href="https://osaurus.ai"><img src="https://img.shields.io/badge/Web-osaurus.ai-blue" alt="Website"></a>
	<a href="https://huggingface.co/JANGQ-AI"><img src="https://img.shields.io/badge/HF-JANGQ--AI-yellow?logo=huggingface" alt="JANGQ-AI"></a>
	<a href="https://huggingface.co/Hcompany/Holo3-35B-A3B"><img src="https://img.shields.io/badge/Base-Holo3--35B--A3B-orange?logo=huggingface" alt="Holo3"></a>
	</p>

	---

	## Model Details

	\| Property \| Value \|
	\|---\|---\|
	\| Base model \| [`Hcompany/Holo3-35B-A3B`](https://huggingface.co/Hcompany/Holo3-35B-A3B) (finetune of `Qwen/Qwen3.5-35B-A3B`) \|
	\| Parameters (source) \| 35 B total, ~3 B active per token \|
	\| Architecture \| `qwen3_5_moe` — 40 decoder layers: 30 `Gated DeltaNet` (linear attn) + 10 full attention, 256 routed experts + 1 always-on shared expert \|
	\| Quantization format \| `weight_format: mxtq` — routed experts via TurboQuant codebook (2-bit), everything else affine 8-bit or fp16 passthrough \|
	\| Routed-expert storage \| `.tq_packed` (uint32) + `.tq_norms` (fp16) + `.tq_bits` (uint8); codebook + Hadamard signs re-derived deterministically at load \|
	\| Package size on disk \| 11.63 GB across 12 shards \|
	\| Shipped tensors \| 1,930 total (1,597 language-model + 333 vision tower + 120 routed-expert TQ triples) \|
	\| Vocab \| 248,320 \|
	\| Context (position embeddings) \| 262,144 native \|
	\| Vision tower \| 27-layer ViT (hidden 1152, patch 16), preserved in fp16 \|
	\| Chat format \| Qwen `im_start`/`im_end` with `<think>` reasoning toggle; Holo3 XML tool-call grammar \|
	\| Use case \| GUI / computer-use agent (desktop, web, mobile) — designed for screenshot → action loops \|

	### Quantization details, per tensor category

	\| Category \| Bits \| Group / codebook \| Notes \|
	\|---\|---\|---\|---\|
	\| Routed-expert MLP (`mlp.switch_mlp.gate_proj`, `up_proj`, `down_proj`) \| 2 (JANGTQ) \| 2² Lloyd-Max centroids + Hadamard rotation \| `.tq_packed` + `.tq_norms` + `.tq_bits` triples \|
	\| Embedding (`embed_tokens`), `lm_head` \| 8 (affine) \| group 64 \| MLX-native `QuantizedLinear` \|
	\| Full-attention projections (`q_proj`, `k_proj`, `v_proj`, `o_proj`) \| 8 (affine) \| group 64 \| Gate-doubled q_proj for `attn_output_gate` \|
	\| Linear-attention projections (`in_proj_qkv`, `in_proj_z`, `in_proj_b`, `in_proj_a`, `out_proj`) \| 8 (affine) \| group 64 \| Gated DeltaNet \|
	\| Shared-expert MLP (`gate_proj`, `up_proj`, `down_proj`) \| 8 (affine) \| group 64 \| Always active per token \|
	\| Router (`mlp.gate`) \| fp16 passthrough \| — \| Precision-critical \|
	\| Shared-expert gate (`shared_expert_gate`) \| fp16 passthrough \| — \| sigmoid scalar gate \|
	\| Norms (`_layernorm`, `_norm`), `A_log`, `dt_bias`, `conv1d` \| fp16 passthrough \| — \| Un-quantized \|
	\| Vision tower (333 tensors) \| fp16 passthrough \| — \| `patch_embed.proj` axes pre-transposed to MLX layout \|

	JANGTQ ("TurboQuant") stores routed-expert weights as indices into a small Lloyd-Max codebook with a per-row norm, after a randomized Hadamard rotation that concentrates the distribution so quantization error is uniform. At inference, the input is rotated once per layer (cheap fused Metal kernel) and dot products happen against the codebook centroids directly, so we never dequantize back to affine. Compared to affine 2-bit at the same bit budget, this gives better quality and faster decode on the routed-expert MLP path.

	---

	## Usage

	JANGTQ requires our custom loader — stock `mlx_lm.load()` can't parse `.tq_packed` tensors. You need `jang-tools` (free, public): <https://github.com/jjang-ai/jangq>.

	```bash
	pip install mlx mlx-lm mlx-vlm
	git clone https://github.com/jjang-ai/jangq && pip install -e ./jangq/jang-tools
	```

	### Text

	```python
	from jang_tools.load_jangtq import load_jangtq_model
	from mlx_lm import generate

	model, tokenizer = load_jangtq_model("JANGQ-AI/Holo3-35B-A3B-JANGTQ2")
	print(generate(model, tokenizer,
	prompt="The capital of France is",
	max_tokens=64))
	```

	### Image (VLM) — the intended use

	Holo3 is a GUI agent: give it a screenshot and it localizes UI elements and plans actions.

	```python
	from jang_tools.load_jangtq_vlm import load_jangtq_vlm_model
	from mlx_vlm import generate
	from mlx_vlm.prompt_utils import apply_chat_template
	from mlx_vlm.utils import load_config

	path = "JANGQ-AI/Holo3-35B-A3B-JANGTQ2"
	model, processor = load_jangtq_vlm_model(path)
	config = load_config(path)

	prompt = apply_chat_template(
	processor, config,
	"Look at this desktop screenshot. Where should I click to open settings?",
	num_images=1,
	)
	print(generate(model, processor, prompt, image="path/to/screenshot.png",
	max_tokens=256))
	```

	### Reasoning toggle

	```python
	msgs = [{"role": "user", "content": "What is 17 × 23?"}]
	# Reasoning OFF — pre-closed <think></think> block
	prompt = tokenizer.apply_chat_template(msgs, add_generation_prompt=True,
	enable_thinking=False)
	# Reasoning ON — model fills the <think> block
	prompt = tokenizer.apply_chat_template(msgs, add_generation_prompt=True,
	enable_thinking=True)
	```

	Pass `enable_thinking` as a direct kwarg (the `chat_template_kwargs={...}` form only propagates on some tokenizer versions).

	### Tool calls — Holo3 XML format

	Holo3 emits tool calls in a custom XML grammar (not JSON). Pass `tools=[...]` to the tokenizer's chat template; the model responds in this shape:

	```xml
	<tool_call>
	<function=click>
	<parameter=x>
	512
	</parameter>
	<parameter=y>
	384
	</parameter>
	</function>
	</tool_call>
	```

	Parse with a simple XML splitter on `<tool_call>`. See H Company's [quickstart](https://hub.hcompany.ai/quickstart) for a full agent harness example.

	### Video

	The base model supports video via `transformers` and the bundle preserves `video_preprocessor_config.json`. `mlx-vlm` 0.4.4's `prepare_inputs` has no video path yet for `qwen3_5_moe` — for video, use upstream `transformers`.

	---

	## Hardware notes

	`11.63 GB` on disk; expect ~12–14 GB resident after load, plus KV cache.

	\| Mac unified RAM \| Works? \| Notes \|
	\|---\|---\|---\|
	\| 16 GB \| ✅ text-only \| Image inference will be tight at long context \|
	\| 24 GB \| ✅ comfortable \| 32 k+ context, image inference OK \|
	\| 32 GB \| ✅ \| 100 k context viable, comfortable VL \|
	\| 64 GB+ \| ✅ headroom \| 262 k native context \|

	---

	## Upstream benchmarks

	These are the base-model numbers for `Hcompany/Holo3-35B-A3B`, not evaluations of this JANGTQ2 quant:

	\| Benchmark \| Score \|
	\|---\|---\|
	\| OSWorld-Verified (computer use) \| 77.8 % — SOTA at 3 B active \|
	\| WebArena (web navigation) \| State-of-the-art (see upstream card) \|
	\| ScreenSpot-Pro (UI localization) \| Top-tier (see upstream card) \|
	\| OSWorld-G (visual grounding) \| Top-tier (see upstream card) \|
	\| H Corporate Benchmark (486 enterprise tasks) \| Outperforms larger competitors \|

	Independent JANGTQ-quant evaluation is tracked in the jang-tools repo and will land in future README revisions.

	---

	## Citation

	```bibtex
	@misc{hai2025holo3modelfamily,
	title = {Holo3 - Open Foundation Models for Navigation and Computer Use Agents},
	author = {H Company},
	year = {2026},
	url = {https://huggingface.co/Hcompany/Holo3-35B-A3B}
	}
	```

	## License

	[Apache 2.0](https://huggingface.co/Hcompany/Holo3-35B-A3B/blob/main/LICENSE) — inherits from the base model.

	---

	<p align="center">
	Packaged on Apple Silicon with <a href="https://github.com/jjang-ai/jangq">jang-tools</a> (mlx-lm 0.31.2) by Jinho Jang (<a href="mailto:eric@jangq.ai">eric@jangq.ai</a>).<br>
	© 2026 Osaurus AI — <a href="https://osaurus.ai">osaurus.ai</a>
	</p>