Instructions to use AutomatosX/AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AutomatosX/AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8 with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("AutomatosX/AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8")
config = load_config("AutomatosX/AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use AutomatosX/AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8 with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AutomatosX/AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use AutomatosX/AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8 with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AutomatosX/AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use AutomatosX/AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8 with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "AutomatosX/AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

AutomatosX commited on 6 days ago

Commit

6518640

verified ·

1 Parent(s): 9c1d960

Publish AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8

Browse files

Files changed (21) hide show

.gitattributes +1 -0
README.md +175 -0
axquant_manifest.json +379 -0
axquant_plan.json +0 -0
axquant_quantizer_execution.json +2537 -0
axquant_runtime.json +45 -0
axquant_source_binding.json +32 -0
chat_template.jinja +120 -0
chat_template.json +3 -0
config.json +2624 -0
generation_config.json +14 -0
merges.txt +0 -0
model-00001-of-00002.safetensors +3 -0
model-00002-of-00002.safetensors +3 -0
model.safetensors.index.json +1010 -0
preprocessor_config.json +21 -0
processor_config.json +50 -0
tokenizer.json +3 -0
tokenizer_config.json +31 -0
video_preprocessor_config.json +21 -0
vocab.json +0 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md ADDED Viewed

	@@ -0,0 +1,175 @@

+---
+license: apache-2.0
+library_name: mlx
+base_model: Qwen/Qwen3-VL-8B-Instruct
+base_model_relation: quantized
+pipeline_tag: image-text-to-text
+tags:
+- mlx
+- apple-silicon
+- quantized
+- mixed-precision
+- axquant
+- axq
+- development
+- qwen3-vl
+- MXFP8
+- MXFP8
+- vision
+---
+# AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8 — 9.31 BPW measured main
+An **AXQuant (AXQ)** mixed-precision MLX checkpoint for Apple Silicon, converted directly from
+the BF16 source model. The language path is quantized while the vision tower are preserved at BF16 in the checkpoint (or a bound sidecar when present).
+> **Development evidence — not a certified AXQuant release.** This package has conversion and
+> artifact-integrity records, but it does not publish measured quality, long-context, kernel-speed,
+> or MTP-speed evidence. Do not interpret the AXQ product label as a benchmark claim.
+## Model details
+| Property | Value |
+| --- | --- |
+| Base model | [Qwen/Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct/tree/0c351dd01ed87e9c1b53cbc748cba10e6187ff3b) |
+| Source revision | `0c351dd01ed87e9c1b53cbc748cba10e6187ff3b` |
+| Product family | `qwen3-vl` |
+| Source architecture | `Qwen3VLForConditionalGeneration` (dense); text path optimized |
+| Main-model parameters | 8.77B logical parameters |
+| Quantizer | AXQuant `1.9.0` |
+| Hub budget class | `MXFP8` |
+| AXQuant base precision class | `16p0bpw` |
+| Planned storage-adjusted BPW | 9.9573 |
+| Measured main-model BPW | 9.3100 |
+| Measured total BPW | **9.3100** |
+| Safetensors weight size | 10.20 GB |
+| Approximate complete download | 10.22 GB |
+| Configured maximum context | 262,144 tokens; practical limits depend on unified memory |
+| Primary MLX runtime | MLX-VLM |
+| AX Engine native execution | Not established; no validated native manifest is included |
+| MTP present | `False` |
+| Vision present | `True` |
+| Audio present | `False` |
+This repository contains MLX Safetensors. It does **not** contain PyTorch or GGUF weights.
+## Choosing an AXQ pack
+AXQ names describe a **storage-budget product class**, not one uniform precision applied to every
+tensor. Protected tensors remain at higher precision, so the exact measured BPW is authoritative.
+In particular, a `6bit`-named mixed plan may retain `4bit` as its base precision while selecting
+6-bit, 8-bit, or BF16 for other tensors to meet an approximately 6-BPW total budget. Protection
+floors can also raise a `4bit`-named pack close to (or above) a `6bit` budget on small or heavily
+protected models. When that collapse happens, AutomatosX does **not** publish a separate
+misleading `4bit` sibling for that base.
+| Sibling | Intended trade-off |
+| --- | --- |
+| [4bit sibling](https://huggingface.co/AutomatosX/AX-Qwen3-VL-8B-Instruct-MLX-AXQ-4bit) | Lower-storage AXQ budget; check its exact BPW |
+| [6bit sibling](https://huggingface.co/AutomatosX/AX-Qwen3-VL-8B-Instruct-MLX-AXQ-6bit) | Higher average precision near the 6-BPW budget |
+See the [AutomatosX collections](https://huggingface.co/AutomatosX/collections)
+for the family catalog, or the [complete index](https://huggingface.co/collections/AutomatosX/automatosx-mlx-model-catalog).
+## Download
+```bash
+python -m pip install -U huggingface_hub
+hf download AutomatosX/AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8 --local-dir ./AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8
+```
+Allow at least 10.22 GB of free disk space. Pin the resulting Hub commit in reproducible
+deployments rather than relying indefinitely on `main`.
+## Run with MLX-VLM
+```bash
+python -m pip install -U mlx-vlm
+python -m mlx_vlm.generate \
+  --model AutomatosX/AX-Qwen3-VL-8B-Instruct-MLX-AXQ-MXFP8 \
+  --image ./image.png \
+  --prompt "Describe this image." \
+  --max-tokens 128 \
+  --temperature 0.0
+```
+The protected vision tower and AXQ language decoder are loaded together by MLX-VLM. The artifact
+records MLX `0.32.1`; runtime QA is reported separately from model-quality claims.
+## AX Engine status
+This package does **not** include a validated native `model-manifest.json`, so AX Engine execution
+is not established by this release. The AX Engine fields in `axquant_runtime.json` describe the
+intended compatibility contract, not observed runtime evidence. Use the architecture-specific MLX
+runtime path above. The artifact records AX Engine version
+`7.5.7`, but version discovery alone is not a runtime check.
+## Quantization layout
+| Main-weight precision | Parameters | Share |
+| --- | ---: | ---: |
+| `8bit` | 7.57B | 86.32% |
+| `bf16` | 1.20B | 13.68% |
+- Quantization methods: `affine, bf16`.
+- Group sizes used by quantized assignments: `32`.
+- MTP sidecar: not included.
+- Vision sidecar: not included.
+- Vision weights: protected BF16 in main shards.
+- Optimization scope: `text-path`.
+- Support tier: `convertible`.
+BF16 sidecars, when present, are included in total download size. Their presence does not by itself
+establish MTP acceleration or vision-language quality.
+## Evidence and validation status
+| Check | Status |
+| --- | --- |
+| Planning evidence | `architecture_prior` |
+| Calibration | none; the allocation is based on architecture priors |
+| Quantizer execution | 253/253 recorded module conversions succeeded; 0 fallbacks |
+| AX Engine native manifest | not included |
+| Quality versus BF16 or uniform baselines | Not published; no quality-retention claim |
+| MTP acceptance and speed | not measured; no MTP speedup claim |
+| AX Engine kernel evidence | `unmeasured` |
+| Vision-language quality | Not evaluated or claimed; vision tensors are preserved at BF16 |
+| Speech-recognition quality | Not applicable |
+| Long-context quality | 262,144-token capacity is config metadata, not a validated claim |
+| Release certification | **Not certified**; formal AXQuant M0-M8 gates are not closed |
+## Intended use and limitations
+- Intended for local development and evaluation on Apple Silicon with MLX-compatible runtimes.
+- No minimum unified-memory figure is claimed; loadability depends on model size, context length,
+  KV-cache policy, runtime buffers, and other processes using unified memory.
+- Architecture-prior allocation is not measured sensitivity. It must not be presented as measured
+  model quality.
+- Vision weights are preserved at BF16, but this release does not claim validated VLM quality.
+- The configured context window can require substantially more memory as the KV cache grows.
+- AX Engine execution is not established because this package has no validated native manifest.
+- Upstream capabilities, limitations, biases, and responsible-use guidance still apply.
+## Provenance and audit files
+- [`axquant_manifest.json`](axquant_manifest.json): package identity, byte accounting, runtime
+  contract, software versions, and file checksums.
+- [`axquant_plan.json`](axquant_plan.json): per-tensor precision decisions and planning evidence.
+- [`axquant_quantizer_execution.json`](axquant_quantizer_execution.json): conversion coverage and
+  fallback records.
+- [`axquant_runtime.json`](axquant_runtime.json): declared AX Engine and MLX compatibility metadata; runtime checks remain separate evidence.
+All published provenance uses repository-relative paths. Local source paths are stripped before
+publication. The checkpoint was converted from BF16 rather than re-quantized from an OptiQ
+artifact. If an OptiQ repository is published separately, it uses a different quantizer and
+should not be assumed to have identical BPW or quality.
+## License
+The checkpoint follows the upstream model license where applicable (often Apache License 2.0). See
+the [Qwen/Qwen3-VL-8B-Instruct model card](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct/tree/0c351dd01ed87e9c1b53cbc748cba10e6187ff3b) for license terms, model
+limitations, and responsible-use guidance.

axquant_manifest.json ADDED Viewed

	@@ -0,0 +1,379 @@

+{
+  "axquant_version": "1.9.0",
+  "calibration": null,
+  "created_at": "2026-10-04T01:42:07.390053Z",
+  "effective_bpw": 9.957347609436535,
+  "files": [
+    {
+      "path": ".cache/huggingface/.gitignore",
+      "sha256": "684888c0ebb17f374298b65ee2807526c066094c701bcc7ebbe1c1095f494fc1",
+      "size_bytes": 1
+    },
+    {
+      "path": ".cache/huggingface/CACHEDIR.TAG",
+      "sha256": "f6572428f6d5e1575e73a1502895a8731f10757dfbb634909c6e154b849af91d",
+      "size_bytes": 191
+    },
+    {
+      "path": ".cache/huggingface/download/.gitattributes.lock",
+      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "size_bytes": 0
+    },
+    {
+      "path": ".cache/huggingface/download/.gitattributes.metadata",
+      "sha256": "31f1b4bec6d8a4ca5f2a15e69c8db00b4ca41812b67866fef9d3d2a7dea9183f",
+      "size_bytes": 100
+    },
+    {
+      "path": ".cache/huggingface/download/README.md.lock",
+      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "size_bytes": 0
+    },
+    {
+      "path": ".cache/huggingface/download/README.md.metadata",
+      "sha256": "2bc038e781cfce4bba3d2cf988ff151991412f4aa7cc9c4f68115bfd65f25650",
+      "size_bytes": 100
+    },
+    {
+      "path": ".cache/huggingface/download/chat_template.json.lock",
+      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "size_bytes": 0
+    },
+    {
+      "path": ".cache/huggingface/download/chat_template.json.metadata",
+      "sha256": "49bfbbab1aa9536dff6bf67058ba17a1f83d2cda3df6a586311e73e7e4416813",
+      "size_bytes": 100
+    },
+    {
+      "path": ".cache/huggingface/download/config.json.lock",
+      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "size_bytes": 0
+    },
+    {
+      "path": ".cache/huggingface/download/config.json.metadata",
+      "sha256": "d09661f3b050a928873a61830328cc3f14478e70c3b70aa7505e44b00dbfd5f2",
+      "size_bytes": 101
+    },
+    {
+      "path": ".cache/huggingface/download/generation_config.json.lock",
+      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "size_bytes": 0
+    },
+    {
+      "path": ".cache/huggingface/download/generation_config.json.metadata",
+      "sha256": "b38b45a108e9739edda7984a3f1ab51d6016fbef4aba5558f205611c6c33d117",
+      "size_bytes": 99
+    },
+    {
+      "path": ".cache/huggingface/download/merges.txt.lock",
+      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "size_bytes": 0
+    },
+    {
+      "path": ".cache/huggingface/download/merges.txt.metadata",
+      "sha256": "0413531eb0c39bc40f53d3c826a6d32ac3338eda3e450c5fc9f61370bca1c896",
+      "size_bytes": 100
+    },
+    {
+      "path": ".cache/huggingface/download/model-00001-of-00004.safetensors.lock",
+      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "size_bytes": 0
+    },
+    {
+      "path": ".cache/huggingface/download/model-00001-of-00004.safetensors.metadata",
+      "sha256": "b5a1116b246628f975ab6eab6dd154153df176f9e904a4e5a74b6a8f88ea0882",
+      "size_bytes": 124
+    },
+    {
+      "path": ".cache/huggingface/download/model-00002-of-00004.safetensors.lock",
+      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "size_bytes": 0
+    },
+    {
+      "path": ".cache/huggingface/download/model-00002-of-00004.safetensors.metadata",
+      "sha256": "02bfcd753be1d48555dda7b53ae0ea2a9333ab3565c0cf281fa542fa5c330001",
+      "size_bytes": 124
+    },
+    {
+      "path": ".cache/huggingface/download/model-00003-of-00004.safetensors.lock",
+      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "size_bytes": 0
+    },
+    {
+      "path": ".cache/huggingface/download/model-00003-of-00004.safetensors.metadata",
+      "sha256": "02373ef5c289fecd28bed21fc1e72b66ccd60826e4b88f4d1e7276593ee151fb",
+      "size_bytes": 124
+    },
+    {
+      "path": ".cache/huggingface/download/model-00004-of-00004.safetensors.lock",
+      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "size_bytes": 0
+    },
+    {
+      "path": ".cache/huggingface/download/model-00004-of-00004.safetensors.metadata",
+      "sha256": "4123a164b98867b126d49a7cc2d8257d65fcf23a7160705a42f9dc1b5cb6d3ac",
+      "size_bytes": 124
+    },
+    {
+      "path": ".cache/huggingface/download/model.safetensors.index.json.lock",
+      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "size_bytes": 0
+    },
+    {
+      "path": ".cache/huggingface/download/model.safetensors.index.json.metadata",
+      "sha256": "05f6355c1f711e90ebc2a6d6deb63f400ccb306348448b8d008b4a847fd50001",
+      "size_bytes": 100
+    },
+    {
+      "path": ".cache/huggingface/download/preprocessor_config.json.lock",
+      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "size_bytes": 0
+    },
+    {
+      "path": ".cache/huggingface/download/preprocessor_config.json.metadata",
+      "sha256": "70625e7938e7468dee59ad74f181b34c8093db6a559589a42e0d4bc95e2be418",
+      "size_bytes": 101
+    },
+    {
+      "path": ".cache/huggingface/download/tokenizer.json.lock",
+      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "size_bytes": 0
+    },
+    {
+      "path": ".cache/huggingface/download/tokenizer.json.metadata",
+      "sha256": "1277f94a4c6330b4d905f0feb5a3076f5d71e81d9ab0534fc3395ed1d5adcb87",
+      "size_bytes": 100
+    },
+    {
+      "path": ".cache/huggingface/download/tokenizer_config.json.lock",
+      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "size_bytes": 0
+    },
+    {
+      "path": ".cache/huggingface/download/tokenizer_config.json.metadata",
+      "sha256": "b84c4b091d6862c64e46275a63bcc39690c5155b362f4b8ebd544bfa9e0a9bd9",
+      "size_bytes": 100
+    },
+    {
+      "path": ".cache/huggingface/download/video_preprocessor_config.json.lock",
+      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "size_bytes": 0
+    },
+    {
+      "path": ".cache/huggingface/download/video_preprocessor_config.json.metadata",
+      "sha256": "a303c3c73e08d28bedcfdbb5d22c27fa02eb73365e1454b49e61f508a574e0ad",
+      "size_bytes": 100
+    },
+    {
+      "path": ".cache/huggingface/download/vocab.json.lock",
+      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+      "size_bytes": 0
+    },
+    {
+      "path": ".cache/huggingface/download/vocab.json.metadata",
+      "sha256": "0ba9bc4043bc5f0fee8420355f2dd35ad71876ff6e293c1a40a87b0121b9c5a4",
+      "size_bytes": 100
+    },
+    {
+      "path": ".cache/huggingface/trees/0c351dd01ed87e9c1b53cbc748cba10e6187ff3b.json",
+      "sha256": "0b69c6246b0cb01b5e2b26f2efa02fcfb5203a32f321c494fd2217ce6e2c8921",
+      "size_bytes": 2582
+    },
+    {
+      "path": "README.md",
+      "sha256": "5a49a26e6cfd3b3b500d84da74cf596d5ce6b6872cc65600cf20e2e72e1c6a40",
+      "size_bytes": 7776
+    },
+    {
+      "path": "axquant_plan.json",
+      "sha256": "05f046633e51afae71bb13091d954f26c307a21cc097410e550265706327f836",
+      "size_bytes": 683120
+    },
+    {
+      "path": "axquant_quantizer_execution.json",
+      "sha256": "3bf798f8bf9f768f035eb6aa079227ab95d0699e5f1079e4f86b0dd34bd9633a",
+      "size_bytes": 65761
+    },
+    {
+      "path": "axquant_runtime.json",
+      "sha256": "11a754b53f68caac8f5e67e980b55d8dd9cbf9647c156f09661be93210f54146",
+      "size_bytes": 1252
+    },
+    {
+      "path": "axquant_source_binding.json",
+      "sha256": "7b58c04d48a5138721e8de34527ad31e94ac5a2d4ca2f68303b89b884e584ce1",
+      "size_bytes": 986
+    },
+    {
+      "path": "chat_template.jinja",
+      "sha256": "3636d0f0bd6bef02654cdffdc447b79cb2cef8ab02cc75267345946291a489e4",
+      "size_bytes": 5292
+    },
+    {
+      "path": "chat_template.json",
+      "sha256": "5c72a170d2a4a1a3bc5adad2e689ae28138a9700e5b8c96c0266331e86c0acce",
+      "size_bytes": 5499
+    },
+    {
+      "path": "config.json",
+      "sha256": "f21859567110d8a3ea49c3f170534aac747c0d837af2aca6f7ab679c5e8d16a7",
+      "size_bytes": 78864
+    },
+    {
+      "path": "generation_config.json",
+      "sha256": "8469742d1fce0de951c8909b26a2c0c0d8490837ce476efb114da9e0cefc4d44",
+      "size_bytes": 269
+    },
+    {
+      "path": "merges.txt",
+      "sha256": "599bab54075088774b1733fde865d5bd747cbcc7a547c5bc12610e874e26f5e3",
+      "size_bytes": 1671839
+    },
+    {
+      "path": "model-00001-of-00002.safetensors",
+      "sha256": "5eff1c5fbc271c231b5781f294efd5b5939a2791030e7f1b4c38cde255d061e8",
+      "size_bytes": 5324446910
+    },
+    {
+      "path": "model-00002-of-00002.safetensors",
+      "sha256": "6e9cf9b2160316c76a0498db01a5145aebdfcb59510ac24177ea2329e604aad2",
+      "size_bytes": 4878330370
+    },
+    {
+      "path": "model.safetensors.index.json",
+      "sha256": "34438f9032774adac94d97ffcf0d57f1ecaab15661717d7cc9cd057a4e37f5c0",
+      "size_bytes": 96111
+    },
+    {
+      "path": "preprocessor_config.json",
+      "sha256": "27225450ac9c6529872ee1924fcb0962ff5634834f817040f444118116f4e516",
+      "size_bytes": 390
+    },
+    {
+      "path": "processor_config.json",
+      "sha256": "45fc17c8dd2474af6b493b52483c26c0584b0082d368c480f9fa611e73070040",
+      "size_bytes": 991
+    },
+    {
+      "path": "tokenizer.json",
+      "sha256": "be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506",
+      "size_bytes": 11422650
+    },
+    {
+      "path": "tokenizer_config.json",
+      "sha256": "7ce86d685d3fb775073b9b1c0f576dd89b7943194c771dd2573cc10e5cae4a48",
+      "size_bytes": 734
+    },
+    {
+      "path": "video_preprocessor_config.json",
+      "sha256": "7768af27c1fafa9cc9011c1dc20067e03f8915e03b63504550e11d5066986d13",
+      "size_bytes": 385
+    },
+    {
+      "path": "vocab.json",
+      "sha256": "ca10d7e9fb3ed18575dd1e277a2579c16d108e32f27439684afa0e10b1440910",
+      "size_bytes": 2776833
+    }
+  ],
+  "format": "mlx",
+  "logical_parameters": 8767123696,
+  "main_logical_parameters": 8767123696,
+  "main_weight_file_size_bytes": 10202777280,
+  "measured_main_bpw": 9.310033834385175,
+  "measured_total_bpw": 9.310033834385175,
+  "mtp_acceptance_retention": null,
+  "mtp_distribution": {},
+  "mtp_measured_speedup": null,
+  "mtp_policy": {
+    "candidate_bits": [
+      8,
+      16
+    ],
+    "min_bits": 8,
+    "mode": "protected",
+    "optimize_for_acceptance": true,
+    "preserve_external_sidecar": true,
+    "protect_norms": true,
+    "protect_output_head": true
+  },
+  "mtp_present": false,
+  "mtp_weight_file_size_bytes": 0,
+  "plan_sha256": "d72ae9af557116a27c51338f841c72f2d9b8d22c8ced3c7b4c73fc1ca52c87f8",
+  "profile": "agent-coding",
+  "protected_weight_file_size_bytes": 0,
+  "quantizer": "axquant",
+  "runtime": {
+    "ax_engine": {
+      "decode_kernel": null,
+      "fused_mtp": null,
+      "kernel_evidence": "unmeasured",
+      "model_manifest": "model-manifest.json",
+      "preferred_group_size": 32
+    },
+    "compatible_runtimes": [],
+    "created_at": "2026-10-04T01:42:02.276883Z",
+    "kv_cache": null,
+    "memory_policy": {
+      "expert_stream": "off",
+      "kv_cache_precision": "runtime-default",
+      "mtp_buffers": "not-required",
+      "prefix_cache": "runtime-managed",
+      "unified_memory_safety_margin": "benchmark-required"
+    },
+    "mtp": {
+      "acceptance_retention": null,
+      "detected": false,
+      "draft_tokens": null,
+      "enabled_by_default": false,
+      "head_precision": null,
+      "measured_speedup": null,
+      "optimized": false,
+      "recommended_temperature_max": null,
+      "sidecar_file": null,
+      "verification_mode": null
+    },
+    "optimization_scope": "text-path",
+    "primary_runtime": {
+      "compatibility_level": "A",
+      "manifest": "config.json",
+      "mtp_support": "none",
+      "name": "mlx-vlm",
+      "notes": [
+        "MLX-VLM loads the protected modality tower and AXQ language decoder."
+      ],
+      "standard_inference": true,
+      "standard_mlx_weights": true,
+      "support_level": "standard-inference"
+    },
+    "schema_version": "axquant.runtime.v1"
+  },
+  "schema_version": "axquant.artifact.v2",
+  "software_versions": {
+    "ax_engine": "7.5.7",
+    "axquant": "1.9.0",
+    "mlx": "0.32.1",
+    "mlx_lm": "0.31.3",
+    "pydantic": "2.13.4",
+    "python": "3.12.13",
+    "safetensors": "0.8.0"
+  },
+  "source_model": {
+    "architecture": "Qwen3VLForConditionalGeneration",
+    "format": "mlx",
+    "local_path": null,
+    "model_id": "Qwen/Qwen3-VL-8B-Instruct",
+    "revision": "0c351dd01ed87e9c1b53cbc748cba10e6187ff3b"
+  },
+  "target_class": "16p0bpw",
+  "weight_distribution": {
+    "8bit": {
+      "fraction": 0.8632360557947807,
+      "parameters": 7568097280
+    },
+    "bf16": {
+      "fraction": 0.1367639442052193,
+      "parameters": 1199026416
+    }
+  },
+  "weight_file_size_bytes": 10202777280
+}

axquant_plan.json ADDED Viewed

The diff for this file is too large to render. See raw diff

axquant_quantizer_execution.json ADDED Viewed

	@@ -0,0 +1,2537 @@

+{
+  "created_at": "2026-10-04T01:42:02.252943Z",
+  "plan_sha256": "d72ae9af557116a27c51338f841c72f2d9b8d22c8ced3c7b4c73fc1ca52c87f8",
+  "records": [
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.embed_tokens",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.0.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.0.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.0.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.0.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.0.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.0.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.0.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.1.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.1.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.1.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.1.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.1.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.1.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.1.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.2.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.2.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.2.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.2.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.2.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.2.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.2.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.3.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.3.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.3.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.3.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.3.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.3.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.3.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.4.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.4.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.4.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.4.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.4.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.4.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.4.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.5.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.5.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.5.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.5.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.5.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.5.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.5.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.6.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.6.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.6.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.6.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.6.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.6.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.6.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.7.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.7.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.7.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.7.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.7.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.7.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.7.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.8.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.8.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.8.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.8.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.8.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.8.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.8.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.9.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.9.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.9.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.9.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.9.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.10.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.10.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.10.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.10.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.10.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.10.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.10.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.11.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.11.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.11.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.11.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.11.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.11.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.11.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.12.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.12.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.12.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.12.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.12.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.12.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.12.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.13.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.13.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.13.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.13.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.13.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.13.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.13.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.14.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.14.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.14.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.14.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.14.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.14.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.14.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.15.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.15.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.15.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.15.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.15.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.15.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.15.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.16.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.16.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.16.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.16.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.16.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.16.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.16.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.17.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.17.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.17.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.17.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.17.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.17.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.17.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.18.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.18.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.18.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.18.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.18.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.18.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.18.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.19.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.19.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.19.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.19.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.19.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.19.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.19.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.20.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.20.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.20.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.20.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.20.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.20.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.20.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.21.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.21.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.21.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.21.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.21.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.21.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.21.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.22.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.22.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.22.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.22.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.9.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.9.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.22.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.22.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.22.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.23.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.23.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.23.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.23.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.23.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.23.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.23.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.24.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.24.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.24.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.24.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.24.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.24.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.24.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.25.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.25.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.25.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.25.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.25.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.25.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.25.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.26.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.26.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.26.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.26.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.26.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.26.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.26.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.27.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.27.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.27.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.27.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.27.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.27.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.27.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.28.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.28.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.28.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.28.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.28.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.28.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.28.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.29.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.29.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.29.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.29.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.29.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.29.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.29.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.30.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.30.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.30.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.30.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.30.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.30.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.30.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.31.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.31.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.31.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.31.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.31.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.31.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.31.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.32.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.32.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.32.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.32.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.32.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.32.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.32.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.33.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.33.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.33.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.33.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.33.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.33.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.33.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.34.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.34.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.34.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.34.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.34.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.34.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.34.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.35.self_attn.o_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.35.self_attn.q_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.35.mlp.down_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.35.mlp.gate_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.35.mlp.up_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.35.self_attn.k_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    },
+    {
+      "bits": 8,
+      "fallback": false,
+      "group_size": 32,
+      "metadata": {},
+      "method": "affine",
+      "module_path": "model.language_model.layers.35.self_attn.v_proj",
+      "note": "mlx-vlm affine packing",
+      "success": true
+    }
+  ],
+  "schema_version": "axquant.quantizer-execution.v2"
+}

axquant_runtime.json ADDED Viewed

	@@ -0,0 +1,45 @@

+{
+  "ax_engine": {
+    "decode_kernel": null,
+    "fused_mtp": null,
+    "kernel_evidence": "unmeasured",
+    "model_manifest": "model-manifest.json",
+    "preferred_group_size": 32
+  },
+  "compatible_runtimes": [],
+  "created_at": "2026-10-04T01:42:02.276883Z",
+  "kv_cache": null,
+  "memory_policy": {
+    "expert_stream": "off",
+    "kv_cache_precision": "runtime-default",
+    "mtp_buffers": "not-required",
+    "prefix_cache": "runtime-managed",
+    "unified_memory_safety_margin": "benchmark-required"
+  },
+  "mtp": {
+    "acceptance_retention": null,
+    "detected": false,
+    "draft_tokens": null,
+    "enabled_by_default": false,
+    "head_precision": null,
+    "measured_speedup": null,
+    "optimized": false,
+    "recommended_temperature_max": null,
+    "sidecar_file": null,
+    "verification_mode": null
+  },
+  "optimization_scope": "text-path",
+  "primary_runtime": {
+    "compatibility_level": "A",
+    "manifest": "config.json",
+    "mtp_support": "none",
+    "name": "mlx-vlm",
+    "notes": [
+      "MLX-VLM loads the protected modality tower and AXQ language decoder."
+    ],
+    "standard_inference": true,
+    "standard_mlx_weights": true,
+    "support_level": "standard-inference"
+  },
+  "schema_version": "axquant.runtime.v1"
+}

axquant_source_binding.json ADDED Viewed

	@@ -0,0 +1,32 @@

+{
+  "config_sha256": "5cd452860dc1e9c29dd71cc3cef7f39b338b7a40793f7a260655c2d3568f3661",
+  "created_at": "2026-10-04T01:41:56.854284Z",
+  "index_sha256": "520b2e05079402e9468a8701d03d1154d14b2599593afb6effa7fb60c1bff070",
+  "members": [
+    {
+      "path": "model-00001-of-00004.safetensors",
+      "size_bytes": 4902275944
+    },
+    {
+      "path": "model-00002-of-00004.safetensors",
+      "size_bytes": 4915962496
+    },
+    {
+      "path": "model-00003-of-00004.safetensors",
+      "size_bytes": 4999831048
+    },
+    {
+      "path": "model-00004-of-00004.safetensors",
+      "size_bytes": 2716270024
+    }
+  ],
+  "plan_sha256": "d72ae9af557116a27c51338f841c72f2d9b8d22c8ced3c7b4c73fc1ca52c87f8",
+  "schema_version": "axquant.source-plan-binding.v1",
+  "source_model": {
+    "architecture": "Qwen3VLForConditionalGeneration",
+    "format": "mlx",
+    "local_path": null,
+    "model_id": "Qwen/Qwen3-VL-8B-Instruct",
+    "revision": "0c351dd01ed87e9c1b53cbc748cba10e6187ff3b"
+  }
+}

chat_template.jinja ADDED Viewed

	@@ -0,0 +1,120 @@

+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {%- if messages[0].content is string %}
+            {{- messages[0].content }}
+        {%- else %}
+            {%- for content in messages[0].content %}
+                {%- if 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' }}
+        {%- if messages[0].content is string %}
+            {{- messages[0].content }}
+        {%- else %}
+            {%- for content in messages[0].content %}
+                {%- if 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- for message in messages %}
+    {%- if message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content in message.content %}
+                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}
+                    <|vision_start|><|image_pad|><|vision_end|>
+                {%- elif content.type == 'video' or 'video' in content %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}
+                    <|vision_start|><|video_pad|><|vision_end|>
+                {%- elif 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content_item in message.content %}
+                {%- if 'text' in content_item %}
+                    {{- content_item.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and message.content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content in message.content %}
+                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}
+                    <|vision_start|><|image_pad|><|vision_end|>
+                {%- elif content.type == 'video' or 'video' in content %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}
+                    <|vision_start|><|video_pad|><|vision_end|>
+                {%- elif 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

chat_template.json ADDED Viewed

	@@ -0,0 +1,3 @@

+{
+    "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0].role == 'system' %}\n        {%- if messages[0].content is string %}\n            {{- messages[0].content }}\n        {%- else %}\n            {%- for content in messages[0].content %}\n                {%- if 'text' in content %}\n                    {{- content.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '\\n\\n' }}\n    {%- endif %}\n    {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0].role == 'system' %}\n        {{- '<|im_start|>system\\n' }}\n        {%- if messages[0].content is string %}\n            {{- messages[0].content }}\n        {%- else %}\n            {%- for content in messages[0].content %}\n                {%- if 'text' in content %}\n                    {{- content.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- set image_count = namespace(value=0) %}\n{%- set video_count = namespace(value=0) %}\n{%- for message in messages %}\n    {%- if message.role == \"user\" %}\n        {{- '<|im_start|>' + message.role + '\\n' }}\n        {%- if message.content is string %}\n            {{- message.content }}\n        {%- else %}\n            {%- for content in message.content %}\n                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}\n                    {%- set image_count.value = image_count.value + 1 %}\n                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}\n                    <|vision_start|><|image_pad|><|vision_end|>\n                {%- elif content.type == 'video' or 'video' in content %}\n                    {%- set video_count.value = video_count.value + 1 %}\n                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}\n                    <|vision_start|><|video_pad|><|vision_end|>\n                {%- elif 'text' in content %}\n                    {{- content.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role + '\\n' }}\n        {%- if message.content is string %}\n            {{- message.content }}\n        {%- else %}\n            {%- for content_item in message.content %}\n                {%- if 'text' in content_item %}\n                    {{- content_item.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {%- if message.tool_calls %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if (loop.first and message.content) or (not loop.first) %}\n                    {{- '\\n' }}\n                {%- endif %}\n                {%- if tool_call.function %}\n                    {%- set tool_call = tool_call.function %}\n                {%- endif %}\n                {{- '<tool_call>\\n{\"name\": \"' }}\n                {{- tool_call.name }}\n                {{- '\", \"arguments\": ' }}\n                {%- if tool_call.arguments is string %}\n                    {{- tool_call.arguments }}\n                {%- else %}\n                    {{- tool_call.arguments | tojson }}\n                {%- endif %}\n                {{- '}\\n</tool_call>' }}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {%- if message.content is string %}\n            {{- message.content }}\n        {%- else %}\n            {%- for content in message.content %}\n                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}\n                    {%- set image_count.value = image_count.value + 1 %}\n                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}\n                    <|vision_start|><|image_pad|><|vision_end|>\n                {%- elif content.type == 'video' or 'video' in content %}\n                    {%- set video_count.value = video_count.value + 1 %}\n                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}\n                    <|vision_start|><|video_pad|><|vision_end|>\n                {%- elif 'text' in content %}\n                    {{- content.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n"
+}

config.json ADDED Viewed

	@@ -0,0 +1,2624 @@

+{
+    "architectures": [
+        "Qwen3VLForConditionalGeneration"
+    ],
+    "do_sample": true,
+    "eos_token_id": [
+        151645,
+        151643
+    ],
+    "generation_config": {
+        "bos_token_id": 151643,
+        "pad_token_id": 151643,
+        "do_sample": true,
+        "eos_token_id": [
+            151645,
+            151643
+        ],
+        "top_k": 20,
+        "top_p": 0.8,
+        "repetition_penalty": 1.0,
+        "temperature": 0.7,
+        "transformers_version": "4.56.0"
+    },
+    "image_token_id": 151655,
+    "model_type": "qwen3_vl",
+    "quantization": {
+        "group_size": 32,
+        "bits": 8,
+        "mode": "mxfp8",
+        "language_model.model.embed_tokens": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.0.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.0.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.0.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.0.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.0.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.0.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.0.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.1.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.1.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.1.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.1.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.1.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.1.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.1.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.2.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.2.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.2.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.2.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.2.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.2.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.2.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.3.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.3.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.3.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.3.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.3.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.3.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.3.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.4.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.4.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.4.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.4.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.4.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.4.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.4.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.5.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.5.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.5.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.5.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.5.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.5.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.5.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.6.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.6.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.6.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.6.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.6.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.6.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.6.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.7.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.7.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.7.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.7.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.7.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.7.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.7.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.8.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.8.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.8.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.8.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.8.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.8.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.8.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.9.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.9.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.9.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.9.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.9.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.9.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.9.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.10.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.10.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.10.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.10.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.10.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.10.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.10.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.11.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.11.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.11.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.11.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.11.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.11.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.11.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.12.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.12.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.12.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.12.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.12.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.12.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.12.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.13.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.13.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.13.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.13.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.13.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.13.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.13.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.14.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.14.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.14.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.14.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.14.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.14.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.14.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.15.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.15.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.15.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.15.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.15.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.15.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.15.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.16.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.16.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.16.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.16.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.16.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.16.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.16.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.17.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.17.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.17.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.17.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.17.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.17.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.17.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.18.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.18.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.18.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.18.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.18.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.18.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.18.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.19.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.19.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.19.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.19.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.19.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.19.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.19.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.20.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.20.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.20.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.20.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.20.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.20.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.20.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.21.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.21.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.21.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.21.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.21.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.21.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.21.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.22.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.22.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.22.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.22.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.22.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.22.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.22.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.23.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.23.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.23.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.23.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.23.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.23.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.23.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.24.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.24.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.24.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.24.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.24.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.24.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.24.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.25.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.25.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.25.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.25.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.25.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.25.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.25.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.26.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.26.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.26.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.26.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.26.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.26.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.26.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.27.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.27.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.27.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.27.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.27.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.27.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.27.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.28.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.28.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.28.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.28.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.28.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.28.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.28.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.29.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.29.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.29.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.29.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.29.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.29.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.29.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.30.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.30.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.30.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.30.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.30.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.30.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.30.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.31.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.31.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.31.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.31.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.31.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.31.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.31.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.32.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.32.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.32.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.32.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.32.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.32.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.32.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.33.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.33.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.33.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.33.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.33.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.33.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.33.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.34.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.34.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.34.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.34.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.34.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.34.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.34.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.35.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.35.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.35.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.35.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.35.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.35.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.35.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        }
+    },
+    "quantization_config": {
+        "group_size": 32,
+        "bits": 8,
+        "mode": "mxfp8",
+        "language_model.model.embed_tokens": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.0.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.0.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.0.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.0.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.0.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.0.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.0.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.1.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.1.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.1.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.1.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.1.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.1.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.1.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.2.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.2.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.2.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.2.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.2.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.2.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.2.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.3.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.3.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.3.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.3.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.3.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.3.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.3.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.4.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.4.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.4.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.4.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.4.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.4.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.4.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.5.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.5.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.5.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.5.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.5.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.5.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.5.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.6.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.6.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.6.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.6.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.6.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.6.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.6.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.7.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.7.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.7.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.7.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.7.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.7.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.7.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.8.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.8.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.8.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.8.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.8.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.8.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.8.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.9.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.9.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.9.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.9.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.9.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.9.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.9.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.10.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.10.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.10.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.10.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.10.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.10.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.10.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.11.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.11.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.11.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.11.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.11.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.11.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.11.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.12.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.12.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.12.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.12.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.12.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.12.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.12.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.13.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.13.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.13.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.13.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.13.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.13.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.13.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.14.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.14.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.14.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.14.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.14.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.14.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.14.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.15.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.15.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.15.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.15.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.15.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.15.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.15.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.16.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.16.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.16.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.16.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.16.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.16.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.16.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.17.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.17.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.17.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.17.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.17.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.17.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.17.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.18.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.18.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.18.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.18.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.18.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.18.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.18.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.19.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.19.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.19.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.19.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.19.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.19.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.19.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.20.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.20.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.20.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.20.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.20.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.20.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.20.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.21.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.21.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.21.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.21.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.21.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.21.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.21.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.22.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.22.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.22.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.22.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.22.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.22.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.22.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.23.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.23.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.23.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.23.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.23.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.23.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.23.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.24.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.24.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.24.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.24.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.24.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.24.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.24.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.25.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.25.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.25.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.25.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.25.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.25.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.25.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.26.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.26.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.26.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.26.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.26.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.26.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.26.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.27.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.27.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.27.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.27.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.27.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.27.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.27.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.28.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.28.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.28.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.28.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.28.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.28.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.28.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.29.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.29.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.29.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.29.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.29.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.29.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.29.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.30.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.30.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.30.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.30.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.30.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.30.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.30.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.31.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.31.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.31.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.31.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.31.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.31.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.31.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.32.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.32.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.32.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.32.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.32.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.32.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.32.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.33.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.33.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.33.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.33.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.33.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.33.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.33.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.34.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.34.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.34.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.34.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.34.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.34.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.34.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.35.self_attn.q_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.35.self_attn.k_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.35.self_attn.v_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.35.self_attn.o_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.35.mlp.gate_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.35.mlp.up_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        },
+        "language_model.model.layers.35.mlp.down_proj": {
+            "group_size": 32,
+            "bits": 8,
+            "mode": "mxfp8"
+        }
+    },
+    "temperature": 0.7,
+    "text_config": {
+        "attention_bias": false,
+        "attention_dropout": 0.0,
+        "bos_token_id": 151643,
+        "dtype": "bfloat16",
+        "eos_token_id": 151645,
+        "head_dim": 128,
+        "hidden_act": "silu",
+        "hidden_size": 4096,
+        "initializer_range": 0.02,
+        "intermediate_size": 12288,
+        "max_position_embeddings": 262144,
+        "model_type": "qwen3_vl_text",
+        "num_attention_heads": 32,
+        "num_hidden_layers": 36,
+        "num_key_value_heads": 8,
+        "rms_norm_eps": 1e-06,
+        "rope_scaling": {
+            "mrope_interleaved": true,
+            "mrope_section": [
+                24,
+                20,
+                20
+            ],
+            "rope_type": "default"
+        },
+        "rope_theta": 5000000,
+        "use_cache": true,
+        "vocab_size": 151936
+    },
+    "tie_word_embeddings": false,
+    "top_k": 20,
+    "top_p": 0.8,
+    "transformers_version": "4.57.0.dev0",
+    "video_token_id": 151656,
+    "vision_config": {
+        "deepstack_visual_indexes": [
+            8,
+            16,
+            24
+        ],
+        "depth": 27,
+        "hidden_act": "gelu_pytorch_tanh",
+        "hidden_size": 1152,
+        "in_channels": 3,
+        "initializer_range": 0.02,
+        "intermediate_size": 4304,
+        "model_type": "qwen3_vl",
+        "num_heads": 16,
+        "num_position_embeddings": 2304,
+        "out_hidden_size": 4096,
+        "patch_size": 16,
+        "spatial_merge_size": 2,
+        "temporal_patch_size": 2
+    },
+    "vision_end_token_id": 151653,
+    "vision_start_token_id": 151652
+}

generation_config.json ADDED Viewed

	@@ -0,0 +1,14 @@

+{
+    "bos_token_id": 151643,
+    "pad_token_id": 151643,
+    "do_sample": true,
+    "eos_token_id": [
+        151645,
+        151643
+    ],
+    "top_k": 20,
+    "top_p": 0.8,
+    "repetition_penalty": 1.0,
+    "temperature": 0.7,
+    "transformers_version": "4.56.0"
+}

merges.txt ADDED Viewed

The diff for this file is too large to render. See raw diff

model-00001-of-00002.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5eff1c5fbc271c231b5781f294efd5b5939a2791030e7f1b4c38cde255d061e8
+size 5324446910

model-00002-of-00002.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6e9cf9b2160316c76a0498db01a5145aebdfcb59510ac24177ea2329e604aad2
+size 4878330370

model.safetensors.index.json ADDED Viewed

	@@ -0,0 +1,1010 @@

+{
+    "metadata": {
+        "total_size": 10202653152
+    },
+    "weight_map": {
+        "language_model.lm_head.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.embed_tokens.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.embed_tokens.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.17.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.17.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.17.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.17.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.17.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.17.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.17.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.17.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.17.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.17.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.17.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.17.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.17.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.18.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.18.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.18.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.18.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.18.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.18.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.18.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.18.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.18.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.18.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.18.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.18.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.18.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.18.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.18.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.18.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.18.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.18.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.19.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.2.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.2.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.2.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.2.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.2.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.2.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.2.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.2.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.2.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.20.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.20.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.20.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.20.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.20.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.20.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.20.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.20.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.20.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.20.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.20.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.20.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.20.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.20.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.20.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.20.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.20.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.20.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.21.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.22.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.23.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.24.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.25.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.26.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.27.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.28.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.29.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.3.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.3.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.3.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.3.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.3.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.3.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.3.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.3.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.3.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.30.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.30.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.30.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.30.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.30.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.30.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.30.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.30.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.30.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.30.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.30.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.30.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.30.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.30.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.30.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.30.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.30.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.30.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.31.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.32.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.33.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.34.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.input_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.mlp.down_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.mlp.gate_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.mlp.up_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.self_attn.k_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.self_attn.o_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.self_attn.q_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.self_attn.v_proj.scales": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.35.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+        "language_model.model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.4.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.4.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.4.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.4.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.4.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.4.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.4.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.4.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.4.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
+        "language_model.model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+        "language_model.model.norm.weight": "model-00002-of-00002.safetensors",
+        "vision_tower.blocks.0.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.0.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.0.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.0.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.0.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.0.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.0.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.0.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.0.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.0.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.0.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.0.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.1.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.1.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.1.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.1.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.1.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.1.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.1.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.1.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.1.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.1.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.1.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.1.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.10.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.10.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.10.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.10.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.10.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.10.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.10.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.10.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.10.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.10.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.10.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.10.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.11.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.11.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.11.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.11.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.11.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.11.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.11.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.11.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.11.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.11.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.11.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.11.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.12.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.12.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.12.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.12.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.12.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.12.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.12.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.12.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.12.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.12.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.12.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.12.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.13.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.13.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.13.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.13.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.13.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.13.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.13.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.13.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.13.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.13.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.13.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.13.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.14.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.14.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.14.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.14.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.14.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.14.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.14.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.14.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.14.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.14.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.14.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.14.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.15.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.15.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.15.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.15.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.15.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.15.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.15.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.15.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.15.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.15.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.15.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.15.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.16.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.16.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.16.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.16.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.16.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.16.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.16.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.16.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.16.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.16.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.16.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.16.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.17.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.17.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.17.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.17.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.17.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.17.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.17.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.17.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.17.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.17.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.17.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.17.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.18.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.18.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.18.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.18.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.18.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.18.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.18.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.18.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.18.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.18.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.18.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.18.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.19.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.19.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.19.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.19.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.19.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.19.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.19.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.19.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.19.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.19.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.19.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.19.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.2.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.2.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.2.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.2.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.2.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.2.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.2.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.2.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.2.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.2.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.2.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.2.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.20.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.20.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.20.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.20.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.20.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.20.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.20.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.20.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.20.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.20.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.20.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.20.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.21.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.21.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.21.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.21.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.21.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.21.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.21.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.21.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.21.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.21.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.21.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.21.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.22.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.22.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.22.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.22.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.22.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.22.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.22.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.22.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.22.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.22.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.22.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.22.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.23.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.23.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.23.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.23.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.23.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.23.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.23.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.23.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.23.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.23.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.23.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.23.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.24.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.24.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.24.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.24.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.24.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.24.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.24.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.24.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.24.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.24.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.24.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.24.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.25.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.25.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.25.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.25.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.25.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.25.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.25.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.25.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.25.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.25.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.25.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.25.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.26.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.26.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.26.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.26.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.26.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.26.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.26.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.26.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.26.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.26.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.26.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.26.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.3.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.3.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.3.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.3.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.3.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.3.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.3.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.3.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.3.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.3.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.3.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.3.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.4.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.4.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.4.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.4.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.4.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.4.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.4.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.4.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.4.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.4.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.4.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.4.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.5.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.5.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.5.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.5.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.5.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.5.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.5.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.5.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.5.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.5.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.5.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.5.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.6.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.6.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.6.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.6.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.6.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.6.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.6.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.6.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.6.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.6.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.6.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.6.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.7.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.7.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.7.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.7.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.7.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.7.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.7.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.7.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.7.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.7.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.7.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.7.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.8.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.8.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.8.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.8.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.8.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.8.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.8.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.8.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.8.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.8.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.8.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.8.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.9.attn.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.9.attn.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.9.attn.qkv.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.9.attn.qkv.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.9.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.9.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.9.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.9.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.9.norm1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.9.norm1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.9.norm2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.blocks.9.norm2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.0.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.0.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.0.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.0.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.0.norm.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.0.norm.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.1.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.1.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.1.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.1.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.1.norm.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.1.norm.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.2.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.2.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.2.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.2.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.2.norm.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.deepstack_merger_list.2.norm.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.merger.linear_fc1.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.merger.linear_fc1.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.merger.linear_fc2.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.merger.linear_fc2.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.merger.norm.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.merger.norm.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.patch_embed.proj.bias": "model-00001-of-00002.safetensors",
+        "vision_tower.patch_embed.proj.weight": "model-00001-of-00002.safetensors",
+        "vision_tower.pos_embed.weight": "model-00001-of-00002.safetensors"
+    }
+}

preprocessor_config.json ADDED Viewed

	@@ -0,0 +1,21 @@

+{
+    "size": {
+        "longest_edge": 16777216,
+        "shortest_edge": 65536
+    },
+    "patch_size": 16,
+    "temporal_patch_size": 2,
+    "merge_size": 2,
+    "image_mean": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "image_std": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "processor_class": "Qwen3VLProcessor",
+    "image_processor_type": "Qwen2VLImageProcessorFast"
+}

processor_config.json ADDED Viewed

	@@ -0,0 +1,50 @@

+{
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_processor_type": "Qwen3VLImageProcessor",
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "max_pixels": 16777216,
+    "merge_size": 2,
+    "min_pixels": 65536,
+    "patch_size": 16,
+    "rescale_factor": 0.00392156862745098,
+    "temporal_patch_size": 2
+  },
+  "processor_class": "Qwen3VLProcessor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "fps": 2.0,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "max_frames": 768,
+    "max_pixels": 25165824,
+    "merge_size": 2,
+    "min_frames": 4,
+    "min_pixels": 4096,
+    "patch_size": 16,
+    "rescale_factor": 0.00392156862745098,
+    "temporal_patch_size": 2,
+    "video_processor_type": "Qwen3VLVideoProcessor"
+  }
+}

tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
+size 11422650

tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,31 @@

+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "pad_token": "<|endoftext|>",
+  "processor_class": "Qwen3VLProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

video_preprocessor_config.json ADDED Viewed

	@@ -0,0 +1,21 @@

+{
+    "size": {
+        "longest_edge": 25165824,
+        "shortest_edge": 4096
+    },
+    "patch_size": 16,
+    "temporal_patch_size": 2,
+    "merge_size": 2,
+    "image_mean": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "image_std": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "processor_class": "Qwen3VLProcessor",
+    "video_processor_type": "Qwen3VLVideoProcessor"
+}

vocab.json ADDED Viewed

The diff for this file is too large to render. See raw diff