README.md

metadata

library_name: mlx
license: apache-2.0
license_link: >-
  https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/995ad96eacd98c81ed38be0c5b274b04031597b0/LICENSE
base_model:
  - nvidia/Qwen3.6-35B-A3B-NVFP4
  - N8Programs/Qwen3.6-35B-A3B-AntiLoop
pipeline_tag: image-text-to-text
tags:
  - mlx
  - mlx-vlm
  - image-text-to-text
  - qwen3.6
  - moe
  - modelopt
  - quantized
  - nvfp4
  - fp4
  - fp8
  - lora
  - merged
  - antidoom

Qwen3.6-35B-A3B-AntiLoop-NVFP4 for MLX-VLM

This is the Apple-silicon MLX-VLM conversion of N8Programs/Qwen3.6-35B-A3B-AntiLoop-NVFP4. It supports both text and vision inputs, including Qwen thinking controls.

The conversion does not re-quantize the mixed NVIDIA ModelOpt language checkpoint. Its FP8 and NVFP4 payloads and scales are re-expressed for MLX's native MXFP8/NVFP4 kernels, while the source tensor-level scales are applied by the included runtime. Activations remain in the model dtype. The vision tower is the original BF16 data, byte-for-byte rather than quantized.

• 1,808 tensors across 42 safetensors shards
• 130 scaled MXFP8 dense modules
• 121 scaled NVFP4 dense modules
• 120 scaled NVFP4 expert projections
• 333 BF16 vision tensors (893,142,496 tensor-data bytes)

The checkpoint was converted and smoke-tested with mlx==0.31.2, mlx-lm==0.31.3, and mlx-vlm==0.6.4.

Use with MLX-VLM

Install MLX-VLM and download the repository:

pip install -U "mlx-vlm==0.6.4"
hf download mlx-community/Qwen3.6-35B-A3B-AntiLoop-NVFP4 \
  --local-dir Qwen3.6-35B-A3B-AntiLoop-NVFP4
cd Qwen3.6-35B-A3B-AntiLoop-NVFP4

Image + text generation:

python run_mlx_vlm.py \
  --model . \
  --trust-remote-code \
  --image /path/to/image.png \
  --prompt "Describe this image." \
  --max-tokens 256

Text-only generation with thinking enabled:

python run_mlx_vlm.py \
  --model . \
  --trust-remote-code \
  --enable-thinking \
  --prompt "Solve: 27 * 43" \
  --max-tokens 512

For programmatic loading from the downloaded repository:

from mlx_vlm_model_file_loader import load

model, processor = load(".")

MLX-VLM 0.6.4 does not yet natively consult a model-local vlm_model_file. run_mlx_vlm.py installs that single lookup inside the current process without modifying the installed package. The local runtime is executed only when --trust-remote-code (or trust_remote_code=True) is explicitly enabled. Review the included Python files before trusting them.

An HTTP server can be started similarly:

python run_mlx_vlm_server.py \
  --model . \
  --trust-remote-code \
  --enable-thinking

Original model card

This is a mixed-precision NVIDIA ModelOpt deployment checkpoint for Qwen3.6-35B-A3B-AntiLoop, a narrow fine-tune intended to recover from pathological self-verification and enumeration loops while preserving ordinary long-form reasoning.

No PEFT adapter is required at inference time. The MLX conversion retains the upstream multimodal architecture, tokenizer, chat template, and 262,144-token native context configuration. It does not include the source checkpoint's MTP draft weights.

Training data

The exact 178 masked supervised targets used for the final AntiLoop training round are published in the Qwen3.6-35B-A3B-AntiLoop-SFT dataset. The dataset preserves each loss_start_char boundary so the pathological loop prefix remains conditioning context rather than a supervised target. It intentionally excludes the separately generated KL-regularization anchors.

Training procedure

The AntiLoop adapter was trained on the 178 masked supervised targets using a standard supervised fine-tuning procedure, but regularized via KL-loss on separately generated non-loop anchors from the base model on everyday prompts.

Benchmark results

LoopHard

LoopHard is our held-out set of 285 enumeration prompts designed to elicit futile recall, recounting, and self-verification loops. The primary metric is judged loops: whether the model's reasoning trace remains stuck in a futile cycle when generation ends.

Model	Judged loops	Loop rate
NVIDIA NVFP4	72 / 285	25.26%
AntiLoop NVFP4	10 / 285	3.51%
NVIDIA NVFP4 + presence_penalty=1.5	30 / 285	10.53%
AntiLoop NVFP4 + presence_penalty=1.5	1 / 285	0.35%

The matched presence_penalty=1.5 comparison converted all 30 control loops to clean completions while introducing one different loop. Exact two-sided McNemar p = 2.98e-8.

Generation used thinking mode, temperature=0.7, top_p=0.95, top_k=20, a 6,144-token completion limit, and concurrency 24. The two presence_penalty=1.5 arms used the exact original and AntiLoop NVFP4 checkpoints with the same vLLM build and serving configuration: TP1, FP8 KV cache, FlashInfer attention, Marlin NVFP4 MoE, and MTP speculative decoding with three draft tokens.

LoopHard is judged by GLM-5.2 using a convergence-aware rubric: systematic reasoning and verification that reaches a conclusion are not loops, and a trace that notices its own circling and exits is classified as recovered. The calibration set contained 42 manually labeled traces. Across three judge runs, accuracy was 88.1%, 92.9%, and 95.2%; all three runs identified all 17 labeled loops, with 2–5 false positives among the 25 non-loop traces.

The 285 prompts, metadata, and GLM-5.2 evaluation code are published in the LoopHard dataset on Hugging Face.

Capability preservation

The capability checks below compare the same round-2 AntiLoop adapter against its FP8 reference model under a matched runtime-LoRA setup. These runs used the default presence penalty and should not be interpreted as evaluations of the exact mixed-precision artifact at presence_penalty=1.5.

GPQA Diamond

Model	Accuracy
Qwen3.6-35B-A3B official model card	86.0%
FP8 reference, our matched harness	167 / 198 (84.34%)
AntiLoop FP8, our matched harness	166 / 198 (83.84%)

The official-model-card number is included for context and was not produced by our harness.

Our GPQA run used thinking mode, paired per-question seeds, temperature=0.7, top_p=0.95, top_k=20, MTP3 speculative decoding, a 65,536-token reasoning budget, and 4,096 tokens of answer headroom. The difference was not significant.

Source for the published 86.0% result: Qwen/Qwen3.6-35B-A3B model card.

GSM8K

Model	Accuracy
FP8 reference	1273 / 1319 (96.51%)
AntiLoop FP8	1270 / 1319 (96.29%)

The GSM8K run used the exact 1,319-example openai/gsm8k main test split, thinking mode, paired seeds, temperature=0.7, top_p=0.95, top_k=20, MTP3, an 8,192-token reasoning budget and 1,024 tokens of answer headroom. The difference was not significant.

Taken together, the matched GPQA and GSM8K results show no material or statistically detectable capability loss at these sample sizes. They do not establish equivalence across other tasks, modalities, or sampling settings.

Original NVIDIA ModelOpt usage

Use a recent vLLM build with ModelOpt mixed-precision support:

vllm serve N8Programs/Qwen3.6-35B-A3B-AntiLoop-NVFP4 \
  --quantization modelopt \
  --trust-remote-code \
  --max-model-len 262144 \
  --kv-cache-dtype fp8 \
  --reasoning-parser qwen3

MTP speculative decoding can be enabled on a compatible build with:

--speculative-config '{"method":"mtp","num_speculative_tokens":3}'

For the measured LoopHard setting, send the following sampling parameters:

{
  "temperature": 0.7,
  "top_p": 0.95,
  "top_k": 20,
  "presence_penalty": 1.5
}

The capability-preservation results above used the default presence penalty; presence_penalty=1.5 has not yet been evaluated on GPQA or GSM8K.

Follow the nvidia/Qwen3.6-35B-A3B-NVFP4 model card for deployment requirements and the Qwen/Qwen3.6-35B-A3B model card for chat templating, thinking controls, multimodal inputs, and base-model limitations.

Limitations

• This is a narrow behavioral fine-tune, not a general alignment or safety model.
• The MLX conversion does not include an MTP speculative drafter.
• LoopHard is a task-specific, judge-based benchmark; its loop rate should not be interpreted as a general safety, truthfulness, or factuality score.
• The GPQA and GSM8K checks used runtime LoRA on an FP8 base, not this exact mixed-precision artifact.
• Capability preservation has not been tested at presence_penalty=1.5.
• Fixed-scale FP8 re-quantization approximates the exact BF16 LoRA merge; small adapter updates can round away or clip at the original E4M3 range.
• Runtime validation used a 65,536-token configured context, not the full native 262,144-token context.
• Multimodal generation quality has not been evaluated on this artifact.
• Outputs may still be incorrect, overconfident, repetitive, biased, toxic, or unsafe.

License

Apache 2.0, following both the underlying Qwen checkpoint and NVIDIA's quantized derivative. See the pinned Qwen license, the Qwen model card, and the NVIDIA ModelOpt checkpoint card.

(co-written with GPT-5.6-Sol)