---
language:
- en
library_name: mlx
license: mit
pipeline_tag: image-text-to-text
base_model: zai-org/GLM-5.3-Flash
tags:
- mlx
- jang
- jangtq
- quantized
- apple-silicon
- vision
- video
- reasoning
- thinking
- agent
- tool-use
- glm5_next
- moe
- gptq
- imatrix
---

<p align="center">
  <img src="./jangq-logo.png" alt="JANGQ" width="220">
  &nbsp;&nbsp;&nbsp;
  <img src="./vmlx-logo.png" alt="vMLX" width="90">
</p>

> ⚠️ **Runtime not released yet.** This bundle uses **JANGTQ v2**, a new routed-expert format, on a new
> architecture (`glm5_next`: KDA linear attention + MLA/DSA hybrid + mHC). No released vMLX or Osaurus build can load
> it today. They refuse it at load time rather than producing wrong output. Access is gated until runtime support
> ships. The numbers below were measured with the internal vMLX build this bundle was made for.

# JANGQ-AI/GLM-5.3-Flash-JANGTQ2

**GLM-5.3-Flash for 128 GB Macs.** Same size as our previous affine release, with **2.7x lower median KL**, **+5.1
points top-1**, and faster decode.

A JANGTQ v2 bundle of [zai-org/GLM-5.3-Flash](https://huggingface.co/zai-org/GLM-5.3-Flash): a 300B-class MoE (288
routed experts, top-8 + shared expert) with KDA linear attention, sparse attention, and vision + video towers.
- **Routed experts**: JANGTQ v2 at 2-3 bits. Codebook-quantized with per-row scales and a blockwise Hadamard rotation,
  calibrated with GPTQ + a per-expert importance matrix. Bits are placed per layer by measurement.
- **Everything else**: 8-bit (MXFP8 where the source weights sit on an FP8-MX grid, affine 8-bit elsewhere) or full
  precision.
- **Vision tower**: kept in bf16.

This replaces `JANGQ-AI/GLM-5.3-Flash-JANG` and `-JANG-MTP`.

## Quality vs the official FP8 release
15,830 teacher-forced positions on held-out prompts, top-128 renormalized KL.

| Bundle | Size | median KL ↓ | mean KL ↓ | p90 / p95 / p99 ↓ | top-1 ↑ | top-5 ↑ | top-10 ↑ |
|---|---|---|---|---|---|---|---|
| **GLM-5.3-Flash-JANGTQ2** | **95.89 GiB** | **0.0323** | **0.351** | **0.89 / 1.76 / 4.53** | **83.7%** | **97.0%** | **98.5%** |
| GLM-5.3-Flash-JANG (affine, previous release) | 95.35 GiB | 0.0882 | 0.528 | 1.50 / 2.55 / 5.67 | 78.6% | 94.6% | 96.8% |
| orcarouter GLM-5.3-Flash-MLX `2bit-lite` ¹ | 95.4 GiB | 0.2122 | 0.83 | — | 71.4% | 90.8% | 94.3% |

¹ Measured earlier on the same 20 prompts (15,850 positions), loading its shipped quantized weights natively.

## Agentic fidelity vs the bf16 model itself
Scored against the full bf16 model, run layer by layer from disk. 72 held-out tool-use conversations (25,867
positions; tools and phrasings disjoint from calibration), including 114 "call a tool or answer?" decision points.

| | **JANGTQ2** | affine JANG (previous) |
|---|---|---|
| median KL vs bf16 | **0.145** | 0.595 |
| top-1 agreement with bf16 | **68.7%** | 54.9% |
| tool-call decisions: same choice as bf16 | **50 / 50** | 50 / 50 |
| median P(`<tool_call>`) at call points (bf16: 0.998) | **0.999** | 0.981 |
| lowest P(`<tool_call>`) at a call point | **0.987** | 0.784 |
| answer decisions: same next token as bf16 | **89.1%** | 50.0% |
| decisions flipped call ↔ answer vs bf16 | **0** | 3 |

These are synthetic agent transcripts, so absolute KL is high for every bundle; read the columns against each other.
The calibration set includes agent-style conversations from the same generator (different tools and wording), so
part of this gain is in-distribution. The FP8 table above is independent of that.

## Live behavior (served, temperature 0)

| suite | **JANGTQ2** | affine JANG (previous) |
|---|---|---|
| 48-case tool-use eval (required / auto × thinking on / off) | **48 / 48** | 48 / 48 |
| 16 behavior probes: math, follow-up, tool round trip, efforts low/high/max, image, video | **16 / 16** | 16 / 16 |
| same 16 probes after a server restart (SSD prefix-cache restore) | **16 / 16** | — |

Both bundles saturate these suites, so they check behavior rather than rank the two.

## Speed (M5 Max 128 GB, served, interleaved A/B/A/B, fresh prompts, median of 3)

| | **JANGTQ2** | affine JANG (previous) |
|---|---|---|
| decode (tok/s) | **27.87 / 27.88** | 26.81 / 26.85 |
| prefill, ~5.2k-token prompt (tok/s) | 348 / 390 | 421 / 416 |
| peak memory while serving | 96.7 GiB | 96.1 GiB |

Two interleaved runs per bundle, same runtime build, each run the median of 3 probes on a never-seen prompt.
Decode is **~4% faster** than the affine bundle; long-prompt prefill is currently ~12% slower.

## What's in the bundle
- **Vision + video**: full bf16 vision tower + the consolidated image/video processor config.
- **No MTP**: layer 45 is omitted; its bytes went into expert precision.
- **Thinking + agentic**:
  - Thinking is ON by default (the template opens `<think>`).
  - Reasoning efforts are **`low` / `high` / `max`** (default `max`). There is no `medium`: the template renders any
    other value as Max.
  - `clear_thinking=false` preserves thinking in history.
- **Tool calls**: GLM's XML dialect (`<tool_call>name<arg_key>…</arg_key><arg_value>…</arg_value></tool_call>`),
  declared as `tool_parser: glm_xml_args`; tool results render as `<|observation|>`. Hermes-style JSON parsers will
  not work.
- **Self-describing**:
  - `config.json` carries the JANGTQ v2 format block (codebook, packing, rotation, method) and a per-module
    `quantization` map (126 expert projections, 147 MXFP8, 214 affine 8-bit).
  - `jang_config.json` records calibration and per-layer expert bits.
  - Raw evaluation results are in `evaluation/`.
- **Memory**: 95.89 GiB weights, 96.7 GiB peak while serving. Fixed-size linear-attention state +
  compressed-latent KV (~6 KB/token), so long contexts do not balloon memory.
- Every shard is alignment-safe (zero-copy memory mapping).

## Serving contract
- Sampling: `temperature=1.0, top_p=0.95` (vendor defaults), no repetition penalty
- EOS: `[154820, 154827, 154829]` · context: 1M native
- Reasoning: `reasoning_effort` chat-template kwarg (`low` / `high` / `max`), default `max`
- Thinking off: the template always opens `<think>`. Runtimes must close it in GLM's native form, `<think></think>`
  with no whitespace; an R1-style `\n</think>\n\n` measurably degrades thinking-off tool decisions.

## Build details
- Source: `zai-org/GLM-5.3-Flash-BF16` @ `a5b45eb`
- Calibration:
  - 600k tokens (web / code / multi-turn chat incl. tool transcripts / math), referenced to the official FP8 release
  - plus a bf16 agentic capture (448 GLM-template tool conversations), with evaluation prompts held out
- Experts: JANGTQ v2 (odd-cubic codebook, fp16 per-row scale, Hadamard-32 rotation), GPTQ on all 42 MoE layers with a
  per-expert importance matrix, bit allocation measured per layer (gate/up 2-3 bit, down 2-3 bit)

Quantized and validated by **Jinho Jang** — eric@jangq.ai