Brooooooklyn/Qwen-AgentWorld-35B-A3B-nvfp4-mlx

Qwen-AgentWorld-35B-A3B — Unsloth NVFP4 + E4M3 FP8 weights (MLX, DGX)

Brooooooklyn/Qwen-AgentWorld-35B-A3B-nvfp4-mlx is an MLX mixed-weight-format quantization of Qwen/Qwen-AgentWorld-35B-A3B, prepared for experimental NVIDIA CUDA inference on Linux aarch64. Qwen-AgentWorld is a language world model for agentic environment simulation. It uses the 40-layer Qwen3.5 MoE architecture with 256 experts, top-8 routing, hybrid linear/full attention, and an MTP-capable config. The published source is language-model-only (language_model_only: true): it contains no vision tensors and no mtp.* tensor subtree.

This model is part of the Qwen Unsloth tensor-class recipe for MLX on macOS and DGX collection.

The source was the all-BF16 checkpoint at revision a6a5fe5200fae283a5fd1247711563d397e7c2a0.

Update — re-converted 2026-09-01

Expert FFN tensors (switch_mlp, shared_expert) are NOT lifted. The norm they read also drives the softmax router and the sigmoid shared-expert gate, and neither is scale-invariant, so folding a scalar into it would move routing. Those blocks therefore carry the same annihilation as the previous revision -- measured 0.9501% of expert blocks, worst layers.0.mlp.switch_mlp.gate_proj at 41.44%. This is a known open limitation, not a regression.

Weight bytes changed; the tensor-class map, shard layout and tensor counts are unchanged. Earlier revisions remain in this repository's history.

Quantization recipe

This is a data-free, weight-only MLX storage port of the Unsloth Qwen3.6 NVFP4 recipe:

• tensors assigned NVFP4 by the recipe are stored as NVFP4, 4-bit with group size 16;
• tensors assigned FP8 by the recipe are stored as raw E4M3 FP8 weight bytes with one BF16 dequantization scale per output channel;
• every excluded tensor stays BF16.

The NVFP4 class uses MLX weight-only quantized matmul with BF16/A16 activations. For the FP8 class, mlx-node reconstructs each weight to BF16 once at load and then uses ordinary A16 matmul or gather-matmul. The serialized fp8_e4m3 form is Uint8 [..., N, K] weight plus BF16 [..., N, 1] scale; it is not MLX mxfp8 and it is not native W8A8 execution.

No imatrix, calibration dataset, AWQ-style pre-scaling, activation calibration, NVFP4 global scale, or FP8 KV-cache calibration was used. This artifact preserves the recipe's tensor-class boundaries and weight storage formats under mlx-node's A16 runtime; it does not claim numerical or performance parity with Unsloth's calibrated W4A4/W8A8 execution.

Tensor class	Stored format
Routed expert switch_mlp.{gate,up,down}_proj, layers 0–31	NVFP4 4/16
Shared expert {gate,up,down}_proj, layers 0–31	NVFP4 4/16
The same routed/shared FFN projections, layers 32–39	E4M3 FP8 weight + per-output BF16 scale
Full-attention {q,k,v,o}_proj	E4M3 FP8 weight + per-output BF16 scale
Linear-attention in_proj_qkv, in_proj_z, out_proj	E4M3 FP8 weight + per-output BF16 scale
lm_head	E4M3 FP8 weight + per-output BF16 scale
Embeddings; router mlp.gate and shared_expert_gate; in_proj_a/b; GDN state, convolution, and norm tensors; all other norms	BF16
MTP tensors	Not present in the published source checkpoint
Vision tower and merger tensors	Not present in the language-model-only source checkpoint

The allocation contains 192 NVFP4 modules and 179 E4M3 FP8 modules. The top-level config is nvfp4, 4-bit, group size 16, so the 192 low-class modules inherit that default. The config carries 179 explicit fp8_e4m3 overrides with bits: 8 and group_size: null. The final eight FFN layers intentionally use the higher class; this is the regular, accuracy-oriented 35B recipe rather than the all-FFN-FP4 "Fast" variant.

Target and usage

The canonical native target is aarch64-unknown-linux-gnu: Linux aarch64 with glibc and NVIDIA CUDA 13.0. mlx-node currently validates this experimental, inference-only path on NVIDIA GB10 / DGX Spark (sm_121). It is not a generic CUDA or x86_64 artifact.

At mlx-node 0.0.8, CUDA has no published prebuilt native npm binary. Build mlx-node from source on the DGX host:

git clone --branch v0.0.8 https://github.com/mlx-node/mlx-node.git
cd mlx-node
git submodule update --init --recursive
yarn install
yarn build

Paged attention is Metal-only in this release. Set both eager-mode variables for DGX inference:

MLX_QWEN35_FORCE_EAGER=1 \
MLX_QWEN35_PAGED_OVERRIDE=0 \
  yarn oxnode your-script.ts

For example, your-script.ts can load a locally downloaded copy:

import { loadSession } from '@mlx-node/lm';

const session = await loadSession('./Qwen-AgentWorld-35B-A3B-nvfp4-mlx');
const result = await session.send('Predict the next state after an agent opens a terminal.');
console.log(result.text);

This checkpoint requires the @mlx-node/lm and @mlx-node/core 0.0.8 source tree or a newer release that explicitly supports the same Linux target and serialized modes.

Reproduction

Converter: mlx-node at or after PR #131, which made the tuned MX weight encoders and the NVFP4 power-of-two lift unconditional. v0.0.8 reproduces the earlier revision of this repository, not the current weights. The reproducible invocation from the mlx-node repository root was:

mlx convert \
  --input .cache/models/qwen-agentworld-35b-a3b \
  --output .cache/models/qwen-agentworld-35b-a3b-unsloth-nvfp4-fp8-dgx-mlx-fresh \
  --model-type qwen3_5_moe \
  --dtype bfloat16 \
  --quantize \
  --q-recipe unsloth \
  --q-mode nvfp4

The -fresh suffix is only the local conversion directory; the canonical Hub repository is the ID shown at the top of this card.

The resulting five-shard SafeTensors index contains 1,104 tensor entries and reports metadata.total_size = 23,891,200,256 bytes. Its tensor dtypes are 541 BF16, 371 U8, and 192 U32 entries, with 371 scale sidecars and no quantization bias sidecars.

Validation

Static validation confirmed identical quantization and quantization_config blocks, exact index-to-shard closure, 192 inherited NVFP4 groups, 179 complete fp8_e4m3 groups, the expected storage dtypes and shapes, and BF16 preservation for protected tensors. The artifact has zero vision tensors and no MTP tensor subtree, matching the pinned language-only source.

The fixed one-token mlx-node load-and-generate smoke test exited successfully with finishReason = "length", numTokens = 1, text = "OK", and rawText = "OK". This one-token text smoke does not validate model quality, long-context behavior, or tool use.

Benchmark

macOS A16 fallback only — these are not DGX/CUDA throughput results.

The values below are medians from three fresh child processes, each loading the checkpoint and generating a deterministic 512-token completion on an Apple M5 Max with 128 GiB of unified memory, Darwin 25.5.0/arm64, Node 24.13.1, and @mlx-node/lm, @mlx-node/core, and @mlx-node/core-darwin-arm64 0.0.8. The run used zero warmups, a 60-second cooldown, temperature 0, reasoning effort none, and the same 106-token prompt. Every sample generated all 512 tokens and ended with finishReason = "length".

Metric	macOS A16 fallback median (not DGX/CUDA)
Load time	79,722.779 ms
Time to first token	3,014.820 ms
Prefill throughput	35.160 tokens/s
Decode throughput	61.244 tokens/s
Generation wall time	11,735.035 ms
Total wall time	91,457.940 ms

The prompt and all per-run samples are recorded in benchmark.json. Load time varied strongly because the weights were read from external storage and OS page-cache state differed between fresh processes; treat that median as specific to this run. This fallback benchmark did not exercise DGX, CUDA, or native W4A4/W8A8 execution, and it must not be used to infer model quality, memory requirements, or parity with upstream execution.

License and attribution

The source model card declares the Apache-2.0 license. Model capability and training credit belong to the Qwen Team. The tensor-class recipe is credited to Unsloth. This repository converts the pinned BF16 source weights into the mixed NVFP4/plain-E4M3 MLX representation described above.