Ship lossless text-only MLX-LM package

Remove the vision tower and mlx-vlm runtime. Preserve all 41 mixed FP8/NVFP4 language shards exactly, use the model-local MLX-LM kernels, and document the zero-requantization representation and text-only runtime.

README.md

@@ -5,11 +5,10 @@ license_link: https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/995ad96eacd98c81e
 base_model:
   - nvidia/Qwen3.6-35B-A3B-NVFP4
   - N8Programs/Qwen3.6-35B-A3B-AntiLoop
-pipeline_tag: image-text-to-text
+pipeline_tag: text-generation
 tags:
   - mlx
-  - mlx-vlm
-  - image-text-to-text
+  - mlx-lm
   - qwen3.6
   - moe
   - modelopt
@@ -22,86 +21,72 @@ tags:
   - antidoom
 ---
 
-# Qwen3.6-35B-A3B-AntiLoop-NVFP4 for MLX-VLM
+# Qwen3.6-35B-A3B-AntiLoop-NVFP4 for MLX-LM
 
 ![AntiLoop looping rate versus GPQA capability](assets/looping_vs_gpqa.png)
 
 ![LoopHard judged-loop rates across four inference settings](assets/loophard_four_settings.png)
 
-This is the Apple-silicon MLX-VLM conversion of
+This is the **text-only, lossless-weight MLX-LM conversion** of
 [`N8Programs/Qwen3.6-35B-A3B-AntiLoop-NVFP4`](https://huggingface.co/N8Programs/Qwen3.6-35B-A3B-AntiLoop-NVFP4).
-It supports both text and vision inputs, including Qwen thinking controls.
+It deliberately omits the vision tower so the mixed NVIDIA ModelOpt weights
+can be preserved without a second quantization pass or an installed-package
+patch.
 
-The conversion does not re-quantize the mixed NVIDIA ModelOpt language
-checkpoint. Its FP8 and NVFP4 payloads and scales are re-expressed for MLX's
-native MXFP8/NVFP4 kernels, while the source tensor-level scales are applied by
-the included runtime. Activations remain in the model dtype. The vision tower
-is the original BF16 data, byte-for-byte rather than quantized.
+## Lossless MLX representation
 
-- 1,808 tensors across 42 safetensors shards
-- 130 scaled MXFP8 dense modules
-- 121 scaled NVFP4 dense modules
-- 120 scaled NVFP4 expert projections
-- 333 BF16 vision tensors (893,142,496 tensor-data bytes)
+All source quantized weight payloads and scales are retained:
 
-The checkpoint was converted and smoke-tested with `mlx==0.31.2`,
-`mlx-lm==0.31.3`, and `mlx-vlm==0.6.4`.
+- ModelOpt FP8 E4M3 bytes are packed directly into MLX `uint32` tensors with
+  unit E8M0 carrier scales; the original FP32 tensor scale is applied after
+  MLX's native MXFP8 matrix multiplication.
+- ModelOpt NVFP4 E2M1 nibbles and E4M3 block-scale bytes remain bit-identical;
+  the original FP32 tensor or per-expert scale is applied after MLX's native
+  NVFP4 matrix multiplication.
+- BF16 and FP32 language-model tensors remain in their source precision.
+- No weight tensor is dequantized or requantized during conversion.
 
-## Use with MLX-VLM
+The package contains 1,475 tensors across 41 language shards: 130 scaled MXFP8
+dense modules, 121 scaled NVFP4 dense modules, and 120 scaled NVFP4 expert
+projections. `weight_bytes_requantized=false` is recorded in
+`mlx_conversion_manifest.json`.
 
-Install MLX-VLM and download the repository:
+“Lossless” here means no additional error is introduced into the source
+quantized **weights**. Activations remain in the model dtype instead of using
+the source checkpoint's stored activation scales, so execution is not expected
+to be bit-identical to NVIDIA ModelOpt inference.
 
-```bash
-pip install -U "mlx-vlm==0.6.4"
-hf download mlx-community/Qwen3.6-35B-A3B-AntiLoop-NVFP4 \
-  --local-dir Qwen3.6-35B-A3B-AntiLoop-NVFP4
-cd Qwen3.6-35B-A3B-AntiLoop-NVFP4
-```
+## Use with MLX-LM
 
-Image + text generation:
+Install the tested runtime:
 
 ```bash
-python run_mlx_vlm.py \
-  --model . \
-  --trust-remote-code \
-  --image /path/to/image.png \
-  --prompt "Describe this image." \
-  --max-tokens 256
+pip install -U "mlx-lm==0.31.3"
 ```
 
-Text-only generation with thinking enabled:
+Generate with an unmodified MLX-LM installation:
 
 ```bash
-python run_mlx_vlm.py \
-  --model . \
+python -m mlx_lm generate \
+  --model mlx-community/Qwen3.6-35B-A3B-AntiLoop-NVFP4 \
   --trust-remote-code \
-  --enable-thinking \
   --prompt "Solve: 27 * 43" \
   --max-tokens 512
 ```
 
-For programmatic loading from the downloaded repository:
-
-```python
-from mlx_vlm_model_file_loader import load
-
-model, processor = load(".")
-```
-
-MLX-VLM 0.6.4 does not yet natively consult a model-local
-`vlm_model_file`. `run_mlx_vlm.py` installs that single lookup inside the
-current process without modifying the installed package. The local runtime is
-executed only when `--trust-remote-code` (or `trust_remote_code=True`) is
-explicitly enabled. Review the included Python files before trusting them.
+The packaged interactive defaults are `temperature=0.6`, `top_p=0.95`, and
+`top_k=20`. The Qwen chat template retains its `enable_thinking` control, so
+clients such as LM Studio can expose the usual thinking toggle.
 
-An HTTP server can be started similarly:
+MLX-LM natively loads the included
+`modeling_mlx_qwen36_modelopt_hybrid.py` through its model-local `model_file`
+architecture hook; the loader does not patch or modify the installed package.
+Review that runtime file before loading this repository. In MLX-LM 0.31.3,
+`--trust-remote-code` controls tokenizer loading rather than the `model_file`
+hook; it is included above for tokenizer compatibility.
 
-```bash
-python run_mlx_vlm_server.py \
-  --model . \
-  --trust-remote-code \
-  --enable-thinking
-```
+This repository is intentionally text-only. It does not contain the 333 BF16
+vision tensors, image processor, or video processor from the upstream model.
 
 ## Original model card
 
@@ -110,10 +95,10 @@ This is a mixed-precision NVIDIA ModelOpt deployment checkpoint for
 a narrow fine-tune intended to recover from pathological self-verification and
 enumeration loops while preserving ordinary long-form reasoning.
 
-No PEFT adapter is required at inference time. The MLX conversion retains the
-upstream multimodal architecture, tokenizer, chat template, and 262,144-token
-native context configuration. It does not include the source checkpoint's MTP
-draft weights.
+No PEFT adapter is required at inference time. This MLX-LM derivative retains
+the language architecture, tokenizer, chat template, and 262,144-token native
+context configuration. It does not package the vision tower or MTP draft
+weights.
 
 ## Training data
 
@@ -250,17 +235,19 @@ limitations.
 ## Limitations
 
 - This is a narrow behavioral fine-tune, not a general alignment or safety model.
-- The MLX conversion does not include an MTP speculative drafter.
+- This MLX-LM artifact is text-only; image and video inputs are unsupported.
+- The conversion preserves quantized weight values, but uses model-dtype
+  activations rather than the source ModelOpt activation scales.
 - LoopHard is a task-specific, judge-based benchmark; its loop rate should not
   be interpreted as a general safety, truthfulness, or factuality score.
 - The GPQA and GSM8K checks used runtime LoRA on an FP8 base, not this exact
   mixed-precision artifact.
 - Capability preservation has not been tested at `presence_penalty=1.5`.
-- Fixed-scale FP8 re-quantization approximates the exact BF16 LoRA merge; small
-  adapter updates can round away or clip at the original E4M3 range.
+- The source checkpoint's fixed-scale FP8 merge approximates the exact BF16
+  LoRA merge; small adapter updates can round away or clip at the original E4M3
+  range. This MLX conversion does not add another weight-quantization pass.
 - Runtime validation used a 65,536-token configured context, not the full native
   262,144-token context.
-- Multimodal generation quality has not been evaluated on this artifact.
 - Outputs may still be incorrect, overconfident, repetitive, biased, toxic, or
   unsafe.
 

config.json

@@ -3,7 +3,6 @@
     "Qwen3_5MoeForConditionalGeneration"
   ],
   "dtype": "bfloat16",
-  "image_token_id": 248056,
   "mlx_hybrid_format": {
     "activations": "model_dtype_weight_only_quantized_matmul",
     "format": "modelopt_fp8_nvfp4_v1",
@@ -1380,12 +1379,6 @@
           "quant_algo": "W4A16_NVFP4"
         }
       }
-    },
-    "vision": {
-      "storage": "bf16_exact_source_weights",
-      "tensor_count": 333,
-      "tensor_data_bytes": 893142496,
-      "weight_bytes_requantized": false
     }
   },
   "mlx_modelopt_quantization": {
@@ -1855,26 +1848,5 @@
     "vocab_size": 248320
   },
   "tie_word_embeddings": false,
-  "transformers_version": "5.7.0.dev0",
-  "video_token_id": 248057,
-  "vision_config": {
-    "deepstack_visual_indexes": [],
-    "depth": 27,
-    "dtype": "bfloat16",
-    "hidden_act": "gelu_pytorch_tanh",
-    "hidden_size": 1152,
-    "in_channels": 3,
-    "initializer_range": 0.02,
-    "intermediate_size": 4304,
-    "model_type": "qwen3_5_moe_vision",
-    "num_heads": 16,
-    "num_position_embeddings": 2304,
-    "out_hidden_size": 2048,
-    "patch_size": 16,
-    "spatial_merge_size": 2,
-    "temporal_patch_size": 2
-  },
-  "vision_end_token_id": 248054,
-  "vision_start_token_id": 248053,
-  "vlm_model_file": "modeling_mlx_vlm_qwen36_modelopt_hybrid.py"
+  "transformers_version": "5.7.0.dev0"
 }

generation_config.json

@@ -6,7 +6,7 @@
         248044
     ],
     "pad_token_id": 248044,
-    "temperature": 1.0,
+  "temperature": 0.6,
     "top_k": 20,
     "top_p": 0.95
 }

mlx_conversion_manifest.json

@@ -1,15 +1,15 @@
 {
   "converter": "convert_qwen36_modelopt_hybrid_to_mlx.py",
+  "conversion_lossless_from_source_quantized_weights": true,
   "created_at": "2026-07-10T03:37:44.758817+00:00",
   "input_scales_dropped": 30971,
-  "multimodal_upgrade_at": "2026-07-10T04:02:39.296349+00:00",
   "norm_weights_shifted": true,
   "num_experts": 256,
   "num_layers": 40,
   "output": "mlx-community/Qwen3.6-35B-A3B-AntiLoop-NVFP4",
-  "output_shards": 42,
-  "output_tensor_count": 1808,
-  "output_total_size_bytes": 21758152396,
+  "output_shards": 41,
+  "output_tensor_count": 1475,
+  "output_total_size_bytes": 20865009900,
   "quantized_module_counts": {
     "scaled_mxfp8": 130,
     "scaled_nvfp4": 121,
@@ -18,10 +18,7 @@
   "runtime_file": "modeling_mlx_qwen36_modelopt_hybrid.py",
   "source": "N8Programs/Qwen3.6-35B-A3B-AntiLoop-NVFP4@1fc377564024dce4e8e7f2bdc04d34cd869f928f",
   "source_tensor_count": 124468,
-  "vision_shard": "model-vision.safetensors",
-  "vision_tensor_count": 333,
-  "vision_tensor_data_bytes": 893142496,
-  "vision_weight_bytes_requantized": false,
-  "vlm_runtime_file": "modeling_mlx_vlm_qwen36_modelopt_hybrid.py",
+  "text_only": true,
+  "vision_weights_included": false,
   "weight_bytes_requantized": false
 }

mlx_vlm_model_file_loader.py

@@ -1,121 +0,0 @@
-"""Opt-in loader for model-local MLX-VLM architecture files.
-
-MLX-VLM 0.6.4 does not yet consult ``vlm_model_file`` in a model config.
-This module adds that one lookup in-process.  It never edits the installed
-``mlx_vlm`` package, and it only executes model-local code when the caller has
-explicitly passed ``trust_remote_code=True``.
-"""
-
-from __future__ import annotations
-
-import importlib.util
-import sys
-from contextvars import ContextVar
-from pathlib import Path
-from types import ModuleType
-from typing import Any
-
-
-_ACTIVE_MODEL_PATH: ContextVar[Path | None] = ContextVar(
-    "mlx_vlm_model_file_path", default=None
-)
-_ACTIVE_TRUST_REMOTE_CODE: ContextVar[bool] = ContextVar(
-    "mlx_vlm_model_file_trust", default=False
-)
-_MODULE_CACHE: dict[Path, ModuleType] = {}
-
-
-def _load_local_module(model_path: Path, filename: str) -> ModuleType:
-    root = model_path.resolve()
-    module_path = (root / filename).resolve()
-
-    try:
-        module_path.relative_to(root)
-    except ValueError as exc:
-        raise ValueError(
-            f"vlm_model_file must stay inside the model directory: {filename!r}"
-        ) from exc
-
-    if module_path.suffix != ".py" or not module_path.is_file():
-        raise FileNotFoundError(f"Model-local MLX-VLM runtime not found: {module_path}")
-
-    cached = _MODULE_CACHE.get(module_path)
-    if cached is not None:
-        return cached
-
-    module_name = f"mlx_vlm_remote_{abs(hash(str(module_path))):x}"
-    spec = importlib.util.spec_from_file_location(module_name, module_path)
-    if spec is None or spec.loader is None:
-        raise ImportError(f"Could not import model-local runtime: {module_path}")
-
-    module = importlib.util.module_from_spec(spec)
-    sys.modules[module_name] = module
-    try:
-        spec.loader.exec_module(module)
-    except Exception:
-        sys.modules.pop(module_name, None)
-        raise
-
-    _MODULE_CACHE[module_path] = module
-    return module
-
-
-def install() -> None:
-    """Install the model-file lookup into the current Python process."""
-
-    import mlx_vlm.utils as utils
-
-    if getattr(utils.get_model_and_args, "_model_file_loader_installed", False):
-        return
-
-    original_get_model_and_args = utils.get_model_and_args
-    original_load_model = utils.load_model
-
-    def get_model_and_args(config: dict, *args: Any, **kwargs: Any):
-        filename = config.get("vlm_model_file")
-        model_path = kwargs.get("model_path") or _ACTIVE_MODEL_PATH.get()
-
-        # Calls made later by processor setup do not carry a model path in
-        # MLX-VLM 0.6.4; let its native Qwen implementation handle those.
-        if not filename or model_path is None:
-            return original_get_model_and_args(config, *args, **kwargs)
-
-        trusted = bool(
-            kwargs.get("trust_remote_code", False)
-            or _ACTIVE_TRUST_REMOTE_CODE.get()
-        )
-        if not trusted:
-            raise PermissionError(
-                "This checkpoint includes a model-local MLX-VLM runtime. "
-                "Re-run with --trust-remote-code (or trust_remote_code=True)."
-            )
-
-        module = _load_local_module(Path(model_path), str(filename))
-        return module, f"model-local:{filename}"
-
-    def load_model(model_path: Path, lazy: bool = False, **kwargs: Any):
-        path_token = _ACTIVE_MODEL_PATH.set(Path(model_path))
-        trust_token = _ACTIVE_TRUST_REMOTE_CODE.set(
-            bool(kwargs.get("trust_remote_code", False))
-        )
-        try:
-            return original_load_model(model_path, lazy=lazy, **kwargs)
-        finally:
-            _ACTIVE_TRUST_REMOTE_CODE.reset(trust_token)
-            _ACTIVE_MODEL_PATH.reset(path_token)
-
-    get_model_and_args._model_file_loader_installed = True  # type: ignore[attr-defined]
-    load_model._model_file_loader_installed = True  # type: ignore[attr-defined]
-    utils.get_model_and_args = get_model_and_args
-    utils.load_model = load_model
-
-
-def load(path_or_hf_repo: str, **kwargs: Any):
-    """Programmatic convenience wrapper around :func:`mlx_vlm.load`."""
-
-    install()
-    kwargs.setdefault("trust_remote_code", True)
-    from mlx_vlm import load as mlx_vlm_load
-
-    return mlx_vlm_load(path_or_hf_repo, **kwargs)
-

model-vision.safetensors

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:7949c801ab3672ddc5ba569a65a17eb836a7cddd121cefbff7deacbeeb2d25c9
-size 893179784

model.safetensors.index.json

@@ -1,6 +1,6 @@
 {
   "metadata": {
-    "total_size": 21758152396
+    "total_size": 20865009900
   },
   "weight_map": {
     "language_model.lm_head.global_scale": "model-00001-of-00041.safetensors",
@@ -1477,339 +1477,6 @@
     "language_model.model.layers.9.mlp.switch_mlp.up_proj.scales": "model-00011-of-00041.safetensors",
     "language_model.model.layers.9.mlp.switch_mlp.up_proj.weight": "model-00011-of-00041.safetensors",
     "language_model.model.layers.9.post_attention_layernorm.weight": "model-00011-of-00041.safetensors",
-    "language_model.model.norm.weight": "model-00001-of-00041.safetensors",
-    "model.visual.blocks.0.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.0.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.0.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.0.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.0.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.0.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.0.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.0.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.0.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.0.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.0.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.0.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.1.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.1.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.1.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.1.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.1.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.1.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.1.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.1.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.1.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.1.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.1.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.1.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.10.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.10.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.10.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.10.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.10.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.10.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.10.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.10.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.10.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.10.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.10.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.10.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.11.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.11.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.11.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.11.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.11.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.11.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.11.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.11.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.11.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.11.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.11.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.11.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.12.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.12.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.12.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.12.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.12.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.12.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.12.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.12.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.12.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.12.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.12.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.12.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.13.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.13.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.13.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.13.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.13.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.13.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.13.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.13.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.13.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.13.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.13.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.13.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.14.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.14.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.14.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.14.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.14.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.14.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.14.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.14.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.14.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.14.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.14.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.14.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.15.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.15.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.15.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.15.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.15.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.15.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.15.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.15.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.15.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.15.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.15.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.15.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.16.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.16.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.16.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.16.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.16.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.16.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.16.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.16.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.16.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.16.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.16.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.16.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.17.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.17.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.17.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.17.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.17.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.17.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.17.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.17.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.17.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.17.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.17.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.17.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.18.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.18.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.18.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.18.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.18.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.18.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.18.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.18.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.18.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.18.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.18.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.18.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.19.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.19.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.19.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.19.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.19.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.19.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.19.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.19.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.19.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.19.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.19.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.19.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.2.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.2.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.2.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.2.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.2.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.2.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.2.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.2.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.2.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.2.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.2.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.2.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.20.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.20.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.20.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.20.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.20.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.20.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.20.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.20.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.20.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.20.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.20.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.20.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.21.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.21.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.21.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.21.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.21.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.21.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.21.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.21.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.21.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.21.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.21.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.21.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.22.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.22.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.22.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.22.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.22.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.22.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.22.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.22.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.22.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.22.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.22.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.22.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.23.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.23.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.23.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.23.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.23.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.23.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.23.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.23.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.23.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.23.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.23.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.23.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.24.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.24.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.24.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.24.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.24.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.24.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.24.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.24.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.24.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.24.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.24.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.24.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.25.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.25.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.25.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.25.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.25.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.25.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.25.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.25.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.25.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.25.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.25.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.25.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.26.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.26.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.26.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.26.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.26.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.26.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.26.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.26.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.26.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.26.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.26.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.26.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.3.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.3.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.3.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.3.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.3.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.3.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.3.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.3.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.3.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.3.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.3.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.3.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.4.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.4.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.4.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.4.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.4.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.4.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.4.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.4.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.4.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.4.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.4.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.4.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.5.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.5.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.5.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.5.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.5.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.5.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.5.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.5.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.5.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.5.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.5.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.5.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.6.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.6.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.6.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.6.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.6.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.6.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.6.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.6.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.6.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.6.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.6.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.6.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.7.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.7.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.7.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.7.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.7.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.7.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.7.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.7.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.7.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.7.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.7.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.7.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.8.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.8.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.8.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.8.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.8.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.8.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.8.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.8.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.8.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.8.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.8.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.8.norm2.weight": "model-vision.safetensors",
-    "model.visual.blocks.9.attn.proj.bias": "model-vision.safetensors",
-    "model.visual.blocks.9.attn.proj.weight": "model-vision.safetensors",
-    "model.visual.blocks.9.attn.qkv.bias": "model-vision.safetensors",
-    "model.visual.blocks.9.attn.qkv.weight": "model-vision.safetensors",
-    "model.visual.blocks.9.mlp.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.blocks.9.mlp.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.blocks.9.mlp.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.blocks.9.mlp.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.blocks.9.norm1.bias": "model-vision.safetensors",
-    "model.visual.blocks.9.norm1.weight": "model-vision.safetensors",
-    "model.visual.blocks.9.norm2.bias": "model-vision.safetensors",
-    "model.visual.blocks.9.norm2.weight": "model-vision.safetensors",
-    "model.visual.merger.linear_fc1.bias": "model-vision.safetensors",
-    "model.visual.merger.linear_fc1.weight": "model-vision.safetensors",
-    "model.visual.merger.linear_fc2.bias": "model-vision.safetensors",
-    "model.visual.merger.linear_fc2.weight": "model-vision.safetensors",
-    "model.visual.merger.norm.bias": "model-vision.safetensors",
-    "model.visual.merger.norm.weight": "model-vision.safetensors",
-    "model.visual.patch_embed.proj.bias": "model-vision.safetensors",
-    "model.visual.patch_embed.proj.weight": "model-vision.safetensors",
-    "model.visual.pos_embed.weight": "model-vision.safetensors"
+    "language_model.model.norm.weight": "model-00001-of-00041.safetensors"
   }
 }

modeling_mlx_vlm_qwen36_modelopt_hybrid.py

@@ -1,217 +0,0 @@
-"""MLX-VLM runtime for Qwen3.6 ModelOpt hybrid FP8/NVFP4 checkpoints.
-
-The language model keeps the lossless ModelOpt-to-MLX representation used by
-``modeling_mlx_qwen36_modelopt_hybrid.py``.  The vision tower remains in its
-original BF16 representation and is delegated to MLX-VLM's native Qwen3.5 MoE
-vision implementation.
-
-This module intentionally exports the same public symbols as an MLX-VLM model
-package so a model-local loader can select it without changing ``model_type``.
-"""
-
-from dataclasses import dataclass, field
-from typing import Dict
-
-import mlx.core as mx
-import mlx.nn as nn
-from mlx.utils import tree_flatten, tree_unflatten
-
-from mlx_vlm.models.qwen3_5_moe import LanguageModel, TextConfig, VisionConfig
-from mlx_vlm.models.qwen3_5_moe import Model as BaseModel
-from mlx_vlm.models.qwen3_5_moe import ModelConfig as BaseModelConfig
-from mlx_vlm.models.qwen3_5_moe import VisionModel
-from mlx_vlm.models.switch_layers import SwitchLinear
-
-
-@dataclass
-class ModelConfig(BaseModelConfig):
-    mlx_modelopt_quantization: Dict[str, str] = field(default_factory=dict)
-
-
-class ScaledQuantizedLinear(nn.Module):
-    """Weight-quantized dense linear with a ModelOpt tensor scale."""
-
-    def __init__(
-        self,
-        input_dims: int,
-        output_dims: int,
-        *,
-        group_size: int,
-        bits: int,
-        mode: str,
-        bias: bool = False,
-    ):
-        super().__init__()
-        if input_dims % group_size:
-            raise ValueError(
-                f"input_dims={input_dims} is not divisible by group_size={group_size}"
-            )
-        if (input_dims * bits) % 32:
-            raise ValueError(
-                f"input_dims={input_dims}, bits={bits} cannot be packed into uint32"
-            )
-
-        self.group_size = group_size
-        self.bits = bits
-        self.mode = mode
-        self.weight = mx.zeros(
-            (output_dims, input_dims * bits // 32), dtype=mx.uint32
-        )
-        self.scales = mx.zeros(
-            (output_dims, input_dims // group_size), dtype=mx.uint8
-        )
-        self.global_scale = mx.ones((), dtype=mx.float32)
-        if bias:
-            self.bias = mx.zeros((output_dims,))
-        self.freeze()
-
-    @classmethod
-    def from_linear(cls, linear: nn.Module, kind: str):
-        output_dims, input_dims = linear.weight.shape
-        has_bias = linear.get("bias") is not None
-        if kind == "scaled_mxfp8":
-            params = dict(group_size=32, bits=8, mode="mxfp8")
-        elif kind == "scaled_nvfp4":
-            params = dict(group_size=16, bits=4, mode="nvfp4")
-        else:
-            raise ValueError(f"Unsupported dense quantization kind: {kind}")
-        return cls(input_dims, output_dims, bias=has_bias, **params)
-
-    def __call__(self, x):
-        y = mx.quantized_matmul(
-            x,
-            self["weight"],
-            self["scales"],
-            transpose=True,
-            group_size=self.group_size,
-            bits=self.bits,
-            mode=self.mode,
-        )
-        y = y * self["global_scale"].astype(y.dtype)
-        if "bias" in self:
-            y = y + self["bias"]
-        return y
-
-
-class ScaledNVFP4SwitchLinear(nn.Module):
-    """Expert linear using MLX gather_qmm and per-expert tensor scales."""
-
-    group_size = 16
-    bits = 4
-    mode = "nvfp4"
-
-    def __init__(
-        self,
-        input_dims: int,
-        output_dims: int,
-        num_experts: int,
-        *,
-        bias: bool = False,
-    ):
-        super().__init__()
-        if input_dims % self.group_size:
-            raise ValueError(
-                f"input_dims={input_dims} is not divisible by {self.group_size}"
-            )
-        self.weight = mx.zeros(
-            (num_experts, output_dims, input_dims * self.bits // 32),
-            dtype=mx.uint32,
-        )
-        self.scales = mx.zeros(
-            (num_experts, output_dims, input_dims // self.group_size),
-            dtype=mx.uint8,
-        )
-        self.global_scales = mx.ones((num_experts,), dtype=mx.float32)
-        if bias:
-            self.bias = mx.zeros((num_experts, output_dims))
-        self.freeze()
-
-    @classmethod
-    def from_switch_linear(cls, linear: SwitchLinear):
-        num_experts, output_dims, input_dims = linear.weight.shape
-        has_bias = linear.get("bias") is not None
-        return cls(input_dims, output_dims, num_experts, bias=has_bias)
-
-    @property
-    def input_dims(self):
-        return self.scales.shape[2] * self.group_size
-
-    @property
-    def output_dims(self):
-        return self.weight.shape[1]
-
-    @property
-    def num_experts(self):
-        return self.weight.shape[0]
-
-    def __call__(self, x, indices, sorted_indices=False):
-        y = mx.gather_qmm(
-            x,
-            self["weight"],
-            self["scales"],
-            rhs_indices=indices,
-            transpose=True,
-            group_size=self.group_size,
-            bits=self.bits,
-            mode=self.mode,
-            sorted_indices=sorted_indices,
-        )
-        scale = self["global_scales"][indices].astype(y.dtype)[..., None, None]
-        y = y * scale
-        if "bias" in self:
-            y = y + mx.expand_dims(self["bias"][indices], -2)
-        return y
-
-
-def _replace_quantized_modules(model: nn.Module, quantization: Dict[str, str]):
-    leaves = dict(
-        tree_flatten(model.leaf_modules(), is_leaf=lambda m: isinstance(m, nn.Module))
-    )
-    missing = sorted(set(quantization) - set(leaves))
-    if missing:
-        preview = "\n  ".join(missing[:20])
-        raise ValueError(f"Quantized module paths are absent from the model:\n  {preview}")
-
-    for path, kind in quantization.items():
-        module = leaves[path]
-        if kind in ("scaled_mxfp8", "scaled_nvfp4"):
-            if not isinstance(module, nn.Linear):
-                raise TypeError(f"{path} is {type(module).__name__}, expected Linear")
-            leaves[path] = ScaledQuantizedLinear.from_linear(module, kind)
-        elif kind == "scaled_nvfp4_switch":
-            if not isinstance(module, SwitchLinear):
-                raise TypeError(
-                    f"{path} is {type(module).__name__}, expected SwitchLinear"
-                )
-            leaves[path] = ScaledNVFP4SwitchLinear.from_switch_linear(module)
-        else:
-            raise ValueError(f"Unknown quantization kind {kind!r} for {path}")
-
-    model.update_modules(tree_unflatten(list(leaves.items())))
-
-
-class Model(BaseModel):
-    def __init__(self, config: ModelConfig):
-        super().__init__(config)
-        _replace_quantized_modules(self, config.mlx_modelopt_quantization)
-
-    def sanitize(self, weights):
-        """Map only raw source keys; converted language keys are already sanitized."""
-        sanitized = {}
-        for key, value in weights.items():
-            if "mtp." in key:
-                continue
-            if key.startswith("model.language_model.visual"):
-                key = key.replace(
-                    "model.language_model.visual", "vision_tower", 1
-                )
-            elif key.startswith("model.language_model"):
-                key = key.replace(
-                    "model.language_model", "language_model.model", 1
-                )
-            elif key.startswith("model.visual"):
-                key = key.replace("model.visual", "vision_tower", 1)
-            elif key.startswith("lm_head"):
-                key = key.replace("lm_head", "language_model.lm_head", 1)
-            sanitized[key] = value
-        return sanitized

preprocessor_config.json

@@ -1,21 +0,0 @@
-{
-    "size": {
-        "longest_edge": 16777216,
-        "shortest_edge": 65536
-    },
-    "patch_size": 16,
-    "temporal_patch_size": 2,
-    "merge_size": 2,
-    "image_mean": [
-        0.5,
-        0.5,
-        0.5
-    ],
-    "image_std": [
-        0.5,
-        0.5,
-        0.5
-    ],
-    "processor_class": "Qwen3VLProcessor",
-    "image_processor_type": "Qwen2VLImageProcessorFast"
-}

run_mlx_vlm.py

@@ -1,12 +0,0 @@
-"""Run the MLX-VLM generation CLI with model-local runtime support."""
-
-from mlx_vlm_model_file_loader import install
-
-install()
-
-from mlx_vlm.generate.cli import main  # noqa: E402
-
-
-if __name__ == "__main__":
-    main()
-

run_mlx_vlm_server.py

@@ -1,12 +0,0 @@
-"""Run the MLX-VLM HTTP server with model-local runtime support."""
-
-from mlx_vlm_model_file_loader import install
-
-install()
-
-from mlx_vlm.server.cli import main  # noqa: E402
-
-
-if __name__ == "__main__":
-    main()
-

video_preprocessor_config.json

@@ -1,21 +0,0 @@
-{
-    "size": {
-        "longest_edge": 25165824,
-        "shortest_edge": 4096
-    },
-    "patch_size": 16,
-    "temporal_patch_size": 2,
-    "merge_size": 2,
-    "image_mean": [
-        0.5,
-        0.5,
-        0.5
-    ],
-    "image_std": [
-        0.5,
-        0.5,
-        0.5
-    ],
-    "processor_class": "Qwen3VLProcessor",
-    "video_processor_type": "Qwen3VLVideoProcessor"
-}