Run JANG models in MLX Studio / vMLX

⚠️ Runtime: JANGHT needs a vMLX (Python) build that is not released yet. The bundle was built and validated on a vMLX development build that adds the JANGT trellis experts (decode and prefill kernels) for this family. vMLX 1.6.77 and older refuse to load it (they stop with an error; they never produce wrong output). Use it with the first vMLX release that lists JANGHT support. The previous JANGH2 weights are in this repo's history at commit 54fab36 (revision="54fab3629aed6b8a2e5e481c3a8885bbe094c820").

JANGQ-AI/Naive-N0.5-Flash-JANGHT2.5

Naive-N0.5-Flash for 128 GB Macs: 575 GiB of bf16 weights in 91.31 GiB. 4.6 GiB smaller than our JANGH2 release and closer to the original model: 29% less extra loss on the real next token.

A JANGHT bundle of NaiveAI/Naive-N0.5-Flash: a 309B MoE (15.5B active, 256 routed experts, top-8) for coding and agentic work, with a native 1M-token context built from sliding-window attention and sparse attention. Text only.

• Routed experts: JANGHT, a per-layer mix of two formats, chosen by measurement:
  • JANGT: a trellis code at 2, 2.5 or 3 bits per weight (Hadamard-128 rotation, 12-bit state, up to 8 outlier columns per layer kept in fp16). The 2.5-bit variant alternates 6- and 4-bit transitions, so a layer can sit between 2 and 3 bits.
  • JANGH: our Hadamard-32 codebook format at 2, 3 or 4 bits, rounded with GPTQ.
• Everything else: affine 8-bit, or kept in the source precision (routers in fp32, norms, the sparse-attention indexer).
• Name: JANGHT + 2.5 bits per weight averaged over the whole model (91.31 GiB × 8 / 308.9B source parameters).

This repository was JANGQ-AI/Naive-N0.5-Flash-JANGH2 until 2026-10-10. Old links redirect here.

Quality vs the bf16 model

124 held-out sequences (coding, cybersecurity, agentic, general, Chinese, science, academic, long documents and 96 multi-turn tool conversations), 133,881 teacher-forced positions, scored against the full bf16 model run layer by layer from disk. Top-128 renormalized KL. The official FP8 rows use NaiveAI's own deployment format (FP8 routed experts, everything else bf16, ~315 GB): the practical ceiling for any compressed version of this model.

	size	ΔNLL vs bf16 ↓	median KL ↓	mean KL ↓	p90 / p99 ↓	top-1 ↑	top-5 ↑
official FP8 weights	~315 GB	+0.180	0.165	0.870	2.45 / 9.59	62.2%	81.3%
official FP8 W8A8 (NaiveAI's deployment)	~315 GB	+0.186	0.173	0.880	2.49 / 9.65	61.9%	81.0%
Naive-N0.5-Flash-JANGHT2.5	91.31 GiB	+0.271	0.256	1.012	2.83 / 10.15	58.4%	79.2%
Naive-N0.5-Flash-JANGH2 (previous release)	95.89 GiB	+0.380	0.270	1.034	2.89 / 10.43	57.7%	78.5%

ΔNLL is the extra loss on the token that actually comes next (nats per token, ±0.006). It is the main number for this model; agreement with bf16 has a low ceiling here (next section). Perplexity ratio vs bf16: 1.31 (JANGH2 1.46, official FP8 1.20).

Why no quantization gets close to 100% agreement here

Naive-N0.5-Flash picks 8 of 256 experts per token in every layer, and the 8th and 9th candidates are often almost tied. A change far smaller than any quantization error flips some of those choices, and the flips compound. Running the same bf16 weights in fp32 instead of bf16 arithmetic already gives only 79% top-1 agreement. That is why even NaiveAI's 8-bit deployment sits at 62% here, and why ΔNLL, which has no such floor, leads the table.

Earlier comparison: noise ceiling and a plain MLX quant (from the JANGH2 release)

68,370 teacher-forced positions on 28 held-out prompts against the bf16 model, top-128 renormalized KL. A different set from the table above (no multi-turn tool conversations), and JANGHT2.5 was not run on it.

	Size	median KL ↓	mean KL ↓	p90 / p95 / p99 ↓	top-1 ↑	top-5 ↑	top-10 ↑
Ceiling: bf16 + noise 0.001 on one tensor	575 GiB	0.060	0.749	2.23 / 4.26 / 9.62	71.6%	85.8%	89.1%
Naive-N0.5-Flash-JANGH2	95.89 GiB	0.170	1.008	3.11 / 5.37 / 10.50	64.4%	81.5%	85.6%
MLX affine RTN, same size ¹	95.83 GiB	1.552	2.976	8.04 / 11.06 / 16.78	48.2%	64.1%	68.6%

¹ Stock MLX affine quantization of the same source (experts 2-3 bits chosen by measurement, non-experts affine 8-bit, no calibration), built as a control; not published. Served, it passed 34 / 48 tool-use cases and 11 / 20 behavior probes, against 48 / 48 and 20 / 20 for the JANGH bundles.

Tool-use fidelity on that release's tool set (96 held-out conversations on unseen tools, 208 decision points, template-position decisions):

	ceiling	JANGH2	MLX affine RTN
tool-call points: model starts a tool call (bf16: 100 of 112)	97	101	83
tool-call points: decisions flipped call ↔ answer vs bf16	9	7	27
answer points: same next token as bf16	91.7%	88.5%	55.2%
median KL, whole conversations	0.292	0.395	3.112
top-1 agreement, whole conversations	55.1%	51.3%	22.7%

By domain, ΔNLL vs bf16:

domain	positions	official FP8 W8A8	JANGHT2.5	JANGH2
tool conversations	65,511	+0.151	+0.188	+0.210
long documents	10,990	+0.317	+0.234	+0.528
agentic, long prompts	10,190	+0.248	+0.356	+0.594
coding, long prompts	9,390	+0.191	+0.293	+0.315
coding	8,249	+0.197	+0.256	+0.343
agentic	7,411	+0.175	+0.359	+0.491
cybersecurity	6,653	+0.134	+0.173	+0.328
general	5,311	+0.179	+0.547	+0.673
science	3,895	+0.077	+0.381	+0.697
Chinese	3,158	+0.143	+0.511	+0.609
academic multiple choice	3,123	+0.284	+0.461	+0.606

JANGHT2.5 beats JANGH2 in every domain. The calibration set is weighted toward coding, tool use and cybersecurity; general, Chinese and science text lose the most.

Tool-use decisions vs the bf16 model

The 96 held-out tool conversations use tools that never appear in calibration. Decisions are scored where the model itself makes them: in its native output layout, the token after </think> and the blank line.

	official FP8 W8A8	JANGHT2.5	JANGH2
median P(<tool_call>) at tool-call points (bf16: 0.847)	0.850	0.863	0.872
tool-call points: same decision as bf16 (n = 112)	94.6%	91.1%	91.1%
answer points: same next token as bf16 (n = 96)	86.5%	89.6%	87.5%
answer points: false tool calls	0	1	0
ΔNLL vs bf16 on these conversations	+0.159	+0.189	+0.220

Live behavior (served, temperature 0)

suite	JANGHT2.5
48-case tool-use eval (required / auto × thinking on / off)	48 / 48
20 behavior probes: arithmetic, follow-ups with reasoning history, tool round trips, chained tool calls, multi-line tool arguments, efforts low/high/max, a 5,173-token needle prompt	20 / 20
reasoning efforts low / high / max render differently; default = max	pass
6-turn conversation with tool calls (~12k characters per turn), prompt cache reused every turn, thinking off and on	pass
prompt cache: cached and uncached answers byte-identical; three long contexts cycled through a small cache budget (evictions), answers unchanged	pass
after a server restart, a 5,173-token prompt restored from the SSD cache (5,172 tokens, 1.0 s to first token), same answer	pass

JANGH2 passed both suites too; they check behavior and separate a working bundle from a broken one, not more.

Speed (M5 Max 128 GB)

Served, one request at a time; two interleaved runs per bundle (A, B, A, B), a fresh server for each run started after a 3-minute cool-down, each run the median of 3 probes on a never-seen prompt. Same runtime build for both bundles.

	JANGHT2.5	JANGH2
decode (tok/s)	49.1 / 49.0	47.7 / 47.5
prefill, ~5.1k-token prompt (tok/s)	952 / 950	839 / 819
time to first token, ~5.1k-token prompt	5.4 s / 5.4 s	6.2 s / 6.4 s

Decode is ~3% faster and long-prompt prefill ~15% faster than JANGH2, in a 4.6 GiB smaller bundle.

What's in the bundle

• Text only. The source model has no vision, audio or video components. No speculative drafter.

• Experts (47 MoE layers):

  projection	JANGT 2	JANGT 2.5	JANGT 3	JANGH 2	JANGH 3	JANGH 4
  gate / up	20 layers	22	—	5	—	—
  down	2	8	1	10	25	1

• Thinking + agentic:

  • Thinking is ON by default. The model opens <think> itself; with thinking off the template appends <think></think>.
  • Reasoning efforts are low / high / max (default max). The template renders any other value as Max.
  • Reasoning is kept in history by default.

• Tool calls: Qwen3-Coder-style XML (<tool_call> / <function=name> / <parameter=key>value</parameter>), declared as tool_parser: xml_function; tool results use the tool role and render as <tool_response>. Hermes-style JSON parsers will not work.

• Self-describing:

  • config.json carries the per-module quantization map: 95 JANGT expert projections ("mode": "jangt", "kbits"), 46 JANGH ("mode": "jangtq2", "bits"), 197 affine 8-bit. The jangt block holds the trellis parameters and the per-layer outlier columns; the jangtq block (version 2) holds the JANGH codebooks.
  • jang_config.json records capabilities, calibration, the allocation plan and the format of every expert unit.

• Every shard is alignment-safe (zero-copy memory mapping). 18 shards.

Serving contract

• Sampling: temperature=1.0, top_p=0.95 (vendor defaults), no repetition penalty
• EOS: 151645 · context: 1M native
• Reasoning: reasoning_effort chat-template kwarg (low / high / max), default max; declared as reasoning_parser: think_xml
• Output layout: the model writes </think>, then a blank line, then the answer or the <tool_call>.
• The routers must stay in fp32 and the sparse-attention indexer unquantized, as shipped.
• JANGH units keep their original on-disk identifiers (jangtq block, "mode": "jangtq2", *.tq2_packed / *.tq2_scales). They are not loadable by, and must not be routed to, JANGTQ v1 loaders.

Build details

• Source: NaiveAI/Naive-N0.5-Flash @ 0235b3b (bf16, 575 GiB)
• Calibration: 618k tokens rendered with the model's own chat template (coding, tool conversations, cybersecurity, agentic, general, Chinese, science, academic), referenced to the bf16 model; evaluation prompts, tools and every tenth calibration document held out
• JANGT units: column-RMS scaling and a least-squares column refit on down projections; unit-gain row scales
• JANGH units: GPTQ on the full per-expert input covariance; 31 units carried over unchanged from JANGH2, 10 down projections re-quantized
• Allocation: per projection and layer, the format and width with the lowest measured cost on held-out rows, each layer's error weighted by its measured effect on the output distribution

Quantized and validated by Jinho Jang — eric@jangq.ai