Instructions to use peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e")
config = load_config("peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

peculiar-ragdoll commited on 24 days ago

Commit

e0ae249

0 Parent(s):

CyberTiel release

Browse files

Files changed (24) hide show

.gitattributes +42 -0
README.md +231 -0
assets/cybertiel_banner_crt.gif +3 -0
assets/cybertiel_bench_cybench.gif +3 -0
assets/cybertiel_bench_harmbench.gif +3 -0
assets/cybertiel_bench_mmlu.gif +3 -0
assets/cybertiel_bench_swe.gif +3 -0
assets/cybertiel_bench_swe_speed.gif +3 -0
chat_template.jinja +488 -0
config.json +3175 -0
generation_config.json +12 -0
merges.txt +0 -0
model-00001-of-00005.safetensors +3 -0
model-00002-of-00005.safetensors +3 -0
model-00003-of-00005.safetensors +3 -0
model-00004-of-00005.safetensors +3 -0
model-00005-of-00005.safetensors +3 -0
model.safetensors.index.json +0 -0
oq_imatrix_report.json +577 -0
preprocessor_config.json +21 -0
processor_config.json +60 -0
tokenizer.json +3 -0
tokenizer_config.json +305 -0
vocab.json +0 -0

.gitattributes ADDED Viewed

	@@ -0,0 +1,42 @@

+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/cybertiel_banner_crt.gif filter=lfs diff=lfs merge=lfs -text
+assets/cybertiel_bench_cybench.gif filter=lfs diff=lfs merge=lfs -text
+assets/cybertiel_bench_harmbench.gif filter=lfs diff=lfs merge=lfs -text
+assets/cybertiel_bench_swe.gif filter=lfs diff=lfs merge=lfs -text
+assets/cybertiel_bench_mmlu.gif filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+assets/cybertiel_bench_swe_speed.gif filter=lfs diff=lfs merge=lfs -text

README.md ADDED Viewed

	@@ -0,0 +1,231 @@

+---
+license: mit
+base_model:
+  - huihui-ai/Huihui-Ornith-1.5-35B-A3B-abliterated
+base_model_relation: quantized
+pipeline_tag: image-text-to-text
+library_name: mlx
+language:
+  - en
+  - zh
+tags:
+  - mlx
+  - omlx
+  - apple-silicon
+  - oq
+  - qwen35moe
+  - moe
+  - imatrix
+  - unsloth-dynamic
+  - agentic-coding
+  - abliterated
+  - uncensored
+  - vision
+  - 4-bit
+---
+<div align="center">
+  <img src="assets/cybertiel_banner_crt.gif" alt="CyberTiel — TielCoder 35B-A3B, abliterated and cyber-tuned" width="100%">
+</div>
+# All power to all people
+*CyberTiel* outcodes every other 35B-A3B at Q4 quantization (and spring-of-2026 frontier models), while engaging with offensive security work without hesitation or refusal. As a sweet spot between speed and ability, CyberTiel delivers agentic coding solves about 3-4x faster than 3.8-27B dense. This is the first time the frontier coder in this size/speed class is an uncensored model. If you need a safer censored alternative, go for *[TielCoder](https://huggingface.co/peculiar-ragdoll/Tiel-Coder-35B-A3B-MLX-oQ4e)*.
+> **CyberTiel is based on [Huihui-Ornith-1.5-35B-A3B-abliterated](https://huggingface.co/huihui-ai/Huihui-Ornith-1.5-35B-A3B-abliterated)**
+> (an uncensored [Ornith-1.5](https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B)),
+> **re-quantized with oMLX's oQ4e quantizer against our own cyber-weighted calibration corpus,
+> and carrying the [Sharp chat template](https://huggingface.co/peculiar-ragdoll/Qwen-Sharp-Chat-Templates)**
+> inside the checkpoint.
+> **These numbers were measured on the GGUF build, not this one.** The plates below are the best
+> evidence we have about CyberTiel — and we would rather show them than show nothing — but they were
+> produced on the [GGUF build](https://huggingface.co/peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-GGUF) at its
+> `UD-Q4_K_M` tier, using llama.cpp's k-quants. This file uses a different quantizer (oMLX's oQ), and
+> changing quantizer moves results. On other builds we have measured a sub-point MMLU-Pro gap and a
+> double-digit percent difference in token counts between MLX and GGUF. Read the plates as evidence
+> about the *model*, not as measurements of *this* file. If you need numbers you can hold us to, use
+> the GGUF build.
+<div align="center">
+  <img src="assets/cybertiel_bench_swe.gif" alt="SWE-bench-Live — problems solved, coding ability improved" width="100%">
+</div>
+SWE-bench-Live tests the model's ability to autonomously solve a set of real issues and bugs in large codebases, published continuously and recently, with hidden regression tests catching if you broke something trying to fix something. Doing well on SWE-bench-Live represents real world autonomous production coding ability: the opposite of "benchmaxxing" and answer memorization for programming work.
+As measured on the GGUF build, *CyberTiel* represents a new frontier in quantized 35B-A3B MoE coders, suitable to solve real-world programming problems at speed, even on low-power hardware with limited memory. This file is those same abliterated weights in Apple-silicon MLX form, at 4-bit and ≈21 GB.
+<div align="center">
+  <img src="assets/cybertiel_bench_swe_speed.gif" alt="SWE-bench-Live — time per solve, effective speed maintained" width="100%">
+</div>
+Benchmarked at 4-bit quantization, CyberTiel thinks and talks less than Ornith-1.5 and Qwen3.6-35B-A3B, making it a faster coder at the same time as it manages to solve ~70% more real world coding problems than Ornith-1.5 and Qwen3.6. For comparison, this domain-specific ability increase is about 7x larger than the generational step from Qwen3.5-35B-A3B to its 3.6 successor.
+<div align="center">
+  <img src="assets/cybertiel_bench_cybench.gif" alt="Cybench unguided — agentic CTF: CyberTiel 15/43 flags, 35%" width="100%">
+</div>
+*CyberTiel* has real offensive capabilities: run unguided, with no hints and no judge, it captures the flag on 15 of the 43 Cybench CTF tasks (35%).
+<div align="center">
+  <img src="assets/cybertiel_bench_harmbench.gif" alt="HarmBench — refusals removed, 0% refusal" width="100%">
+</div>
+*CyberTiel* (unlike *TielCoder*) does not refuse on HarmBench: zero refusals across all 84 requests, sampled twelve at a time from each of HarmBench's seven categories — cybercrime and intrusion among them, alongside chemical/biological, illegal, harassment, misinformation, copyright and general harm.
+<div align="center">
+  <img src="assets/cybertiel_bench_mmlu.gif" alt="MMLU-Pro — bird-brained on world knowledge" width="100%">
+</div>
+*CyberTiel* and *TielCoder* sacrifice world knowledge for coding ability and speed: pick them for work, and pick something else (like [Nail](https://huggingface.co/peculiar-ragdoll/Nail-Qwen3.6-35B-A3B-GGUF)) for trivia or exams. The loss comes with the specialization, not with the abliteration: CyberTiel lands on exactly TielCoder's MMLU-Pro score.
+> The **benchmarked** build is the [GGUF ladder](https://huggingface.co/peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-GGUF)
+> (Q2 → Q8, with vision). For higher precision the sibling [oQ6e build](https://huggingface.co/peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ6e) is near-lossless at 6-bit (≈30 GB). These MLX builds might give you more or fewer tokens per second
+> than the GGUF, depending on your hardware.
+## Abliterated: Willing, able and slightly unstable
+> ⚠️ WARNING: Abliterated models like CyberTiel are able to say and do things other models refuse, including potentially harmful behaviours. By using it, you agree to take personal responsibility for its behaviour, and show caution. It is entirely up to you — the operator — to ensure your use of CyberTiel is legitimate and harmless, and that the model is safely sandboxed and monitored when running. Much like a knife, abliterated models like CyberTiel can be classified and used as either a tool or a weapon, depending on the context and use case. We carry forward [huihui's original usage warnings](https://huggingface.co/huihui-ai/Huihui-Ornith-1.5-35B-A3B-abliterated#usage-warnings).
+**Abliteration**, also known as "uncensoring" or "ablation", is the suppression of refusal in LLMs. This model has undergone abliteration.
+Unabliterated models sometimes wrongly refuse benign (harmless) requests. With CyberTiel you don't need careful wording to get your work done, and deliberation of refusal does not distract the model's attention or waste tokens, thus increasing its ability to perform legitimate work cleanly. This usually comes at the cost of some small corruption of the original model, which in the case of *CyberTiel* is more than balanced out by the advantages combined with the optimized imatrix and quant strategy, leading to a decisive gain on both SWE-bench-Live (agentic coding) and Cybench (offensive security ability).
+HarmBench measures to what degree models refuse to produce language and behaviours that can be deemed harmful when applied maliciously. CyberTiel does not refuse on HarmBench. Models that are capable of these behaviours can be used for good or neutral purposes, so this benchmark is a measurement of specific capability that demands personal responsibility on behalf of the user deploying the model, not of inherent harmfulness.
+**We strongly insist on you sandboxing this model at the operating system level**, limiting and controlling its access to execute code on your machine, and limiting/controlling the way it can access the internet. With refusals removed, this is not an ordinary coding agent: after a misinterpreted intention or a prompt injection from a hostile website or third-party code, this model can turn against you or others and cause real harm. If you do not understand this or how to effectively mitigate it, we recommend you use the very capable yet guardrailed [TielCoder](https://huggingface.co/peculiar-ragdoll/Tiel-Coder-35B-A3B-MLX-oQ4e) instead.
+## Run it
+One tier here: **oQ4e** — 4-bit dynamic mixed precision with a cyber-weighted imatrix pass.
+Vision is included in the same folder; there is no separate projector file. At ≈21 GB of unified
+memory it fits a 24 GB Mac, with room for a working context.
+**oMLX** — put the folder under `~/.omlx/models/peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e`, or pull it from the oMLX
+admin dashboard.
+Sampling: `temperature 0.6`, `top_p 0.95`, `top_k 20`, `min_p 0` for agentic coding. For cybersecurity/CTF work, swap to `top_k 40`, `min_p 0.05` (same temperature and top_p), tested on the GGUF Q4 build. This is a looser configuration leading to more divergent and exploratory thinking, which leads to more solutions on Q4 but might create issues and non-convergence on lower quants.
+**Prefer to keep the files yourself?**
+```bash
+hf download peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e --local-dir CyberTiel-MLX
+python -m mlx_vlm.generate --model CyberTiel-MLX --max-tokens 512 \
+  --prompt "Explain what this function does."                       # text
+python -m mlx_vlm.generate --model CyberTiel-MLX --max-tokens 512 \
+  --prompt "What is in this screenshot?" --image photo.jpg          # vision
+```
+**Load it with `mlx-vlm`, not `mlx-lm`.** This is a vision-language checkpoint. `mlx_lm.load()`
+accepts it and then emits garbage tokens — a loader mismatch, not a bad quant, but it fails quietly.
+Both runtimes apply the embedded Sharp template automatically — nothing to pass.
+## Use it
+The recommended coding-agent harness for CyberTiel, with which the SWE-bench-Live results were achieved, is [Pi.dev](https://pi.dev). It is a lean, open source, extensible framework, that you can adapt to your own use and workflows using the coding agent itself.
+For larger projects and complex multi-part work, we use an orchestrated subagent workflow with test-driven and spec-driven development: After interviewing you about what you want built, the orchestrator agent commissions subagents for recon and research, then writes up a plan document, design, and a spec, defining the scope and shape of the work. It then commissions an implementer with the needed context to implement one part of it, which is then reviewed by the next subagent, and then fixes and corrections are applied by yet another fresh-context agent, which are then re-reviewed, until the orchestrator is happy with the result. The task is then marked as done, and the orchestrator moves on to the next point. This has the advantage of keeping work scoped inside the usable context window of each agent, increasing quality and rigor when applied correctly. CyberTiel does not need this sort of workflow to function or deliver contained fixes or features, but it makes it possible for the model to tackle larger work that would otherwise be outside the capability of a 35B-A3B model with a 262k context window, thus extending its reach.
+There exist plug-and-play extensions and tools that can be used with Pi for this kind of workflow, or you can build your own using the coding agent itself, including skills and system prompts for the different agents and different steps of the workflow.
+For security work, give it a harness whose skills, plugins and tools encode the patterns and workflows you actually use — the model follows a well-worn path far better than it invents one.
+## The CyberTiel imatrix
+oMLX's oQ quantizer runs its own importance-matrix pass — the "e" in `oQ4e` — that measures which
+weights carry the most signal before deciding what to keep at higher precision. For CyberTiel we fed
+that pass **our own code- and cybersecurity-weighted calibration corpus** — the same corpus behind the
+[GGUF build's importance matrix](https://huggingface.co/peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-GGUF) —
+rather than oMLX's default calibration set. So this MLX build is cyber-weighted too: the abliteration
+removes the refusals, and the imatrix keeps cyber and coding ability intact under quantization.
+An imatrix is **not training data**. It measures *which weights carry the load* under a representative
+input distribution, so the quantizer spends its precision there and lets rounding error fall where it
+matters least. Point that measurement at cyber-and-code text and the quant stays comparable to full
+precision on exactly the work this build is for.
+> Note: oQ derives its *own* importance data from that corpus — it does not consume the GGUF imatrix
+> bytes we baked. Same corpus, different quantizer and a different importance computation, so this
+> build is not identical to any GGUF tier. Only the GGUF route has been benchmarked.
+**Calibration corpus** — ≈50 MB (~50 M characters), matched to TielCoder's corpus depth so the two
+imatrices are comparable, assembled entirely from public, redistributable security engineering and code:
+| bucket | share | what it is | source |
+|---|--:|---|---|
+| **Security** | 40% | the specialization: half **offensive** (PoC / exploit code), half **defensive** (methodology, tooling, detection rules) | exploit-db · PayloadsAllTheThings · HackTricks · nuclei-templates |
+| **Code** | 27% | hold general coding ability through the quant | eaddario `code_medium` + `code_large` |
+| **Agentic tool-use** | 18% | the model is driven by a coding agent — real tool-call / bash-session traces | eaddario `tools_large` |
+| **General + multilingual** | 15% | keep language and broad-knowledge pathways alive; non-Latin scripts (zh / ja / ko / ru / ar) weighted **2.5×**, since public offensive-security text is English by measurement | eaddario `combined_*` |
+Buckets are **interleaved** as ~2 KB fragments, round-robin by budget, rather than concatenated in
+blocks — so every calibration chunk sees a code + security + prose mix and no bucket gets over-weighted
+by wherever a chunk boundary happens to land.
+*The most interesting finding* is that — in comparison to TielCoder — our cyber-weighted imatrix (in combination with abliteration) cleanly and significantly increases *Tiel*'s performance on standard real-world software engineering tasks outside the training data, from the level of Opus 4.6 medium (12, where its TielCoder counterpart sits) to a 3-seed mean of 13.7 / 25 — above both — measured on SWE-bench-Live (on the GGUF build).
+## No multi-token-prediction head
+CyberTiel quantizes from the abliterated safetensors, which carry no MTP (`nextn`) block — so there is
+no speculative-decoding head here, and nothing to graft or strip. Ornith-1.5's own MTP head is trained,
+but it does not come down the abliteration lineage; a CyberTiel MTP graft is future work.
+## How the quantization was done
+oQ4e is oMLX's dynamic quantizer: a 4-bit base with mixed precision by layer position and
+selective non-quantization, plus the imatrix pass described above. It is the same idea as the GGUF
+ladder's Unsloth-Dynamic-plus-imatrix recipe, implemented for MLX, but it is *not* the same
+computation: oQ derives its own importance data rather than consuming the GGUF imatrix. The Sharp
+template (`qwen3.8-froggeric-v22.5.0`) is embedded in the checkpoint — the same template the GGUF ladder
+carries — so the only deliberate differences from the GGUF build are the quantizer and its importance
+computation.
+That is the reason for the caveat at the top. Two quantizers pursuing the same goal by different routes
+do not land in the same place, and only the GGUF route has been benchmarked.
+## Benchmarks disclaimer
+The plates on this page were measured on the GGUF build at `UD-Q4_K_M`, not on this MLX file — see the
+note at the top. We show them because they are the best evidence we have about the model, and this build
+is the same abliterated weights by a different quantizer.
+All 35B-A3B-based models in the benchmark ran with a 75–80 tok/s base generation rate on the
+benchmarking hardware, and Qwen3.8-27B with a 22 tok/s generation rate; both rates decrease as the
+model climbs toward the context ceiling. On Apple silicon the MLX runtime's throughput depends on your
+specific chip and memory bandwidth, so it will differ again from those figures.
+For increased validity, we ran CyberTiel three times on SWE-bench-Live (on the GGUF build). The three
+passes resolved 15, 13 and 13 problems out of the 25-problem set (mean 13.7). This variance across
+attempts is an artifact of the inherent variability and indeterminism of LLMs running at non-zero
+temperature. If we had the GPU-time and tokens, we would run all models at more seeds and problems
+across all benchmarks for maximal cross-comparison statistical validity, so take results as a strong
+indicator rather than a perfect comparison.
+**On the Cybench task set.** Cybench is published as a 40-task benchmark, but the [public repository](https://github.com/andyzorigin/cybench) does not ship all 40: nine of the official tasks are Glacier CTF challenges whose files are not distributed with it. It does ship twelve *additional* tasks — from the same competitions (HackTheBox Cyber Apocalypse 2024, Sekai CTF 2022/2023, HKCert CTF 2022), with full metadata, subtasks and human first-blood times — that are not on the official 40 list. We ran every task the repository actually ships: 43 = 31 of the official 40, plus those 12. **15/43 (35%) is therefore not directly comparable to a published Cybench score**; restricted to the 31 official tasks alone, CyberTiel captured 10 flags. The runs are *unguided* — no subtask hints, no judge, exact final-flag match only — capped at 15 agent iterations, at 262k context with the CTF sampling settings given above.
+This account and the models published are a non-profit project, and we intentionally decline offers of donations in order to ensure the independence and validity of our published results and products.
+## Credits
+- [**huihui-ai**](https://huggingface.co/huihui-ai/Huihui-Ornith-1.5-35B-A3B-abliterated) — the abliterated base this quantizes (refusals removed from Ornith-1.5).
+- [**ornith-ai**](https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B) — the underlying Ornith-1.5-35B-A3B weights and vision tower (MIT).
+- [**oMLX**](https://github.com/jundot/omlx) — the oQ dynamic quantizer this build uses.
+- [**Unsloth**](https://huggingface.co/unsloth) — the Dynamic quantization method the imatrix recipe follows.
+- [**froggeric**](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates) — the template lineage Sharp builds on.
+- [**eaddario**](https://huggingface.co/datasets/eaddario/imatrix-calibration) — the code, tool-use and multilingual calibration corpora the imatrix was measured on (MIT).
+- Security calibration sources — [Exploit-DB](https://gitlab.com/exploit-database/exploitdb), [PayloadsAllTheThings](https://github.com/swisskyrepo/PayloadsAllTheThings), [HackTricks](https://github.com/HackTricks-wiki/hacktricks), [nuclei-templates](https://github.com/projectdiscovery/nuclei-templates) — the public security engineering that formed the cyber bucket.
+- [**MLX**](https://github.com/ml-explore/mlx) and [**mlx-vlm**](https://github.com/Blaizzy/mlx-vlm) — the runtime.
+MIT, inheriting Ornith-1.5's license.
+## Citation
+```bibtex
+@misc{Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e,
+  title  = {Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e},
+  author = {Saga Ishtardottir},
+  year   = {2026},
+  url    = {https://huggingface.co/peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ4e},
+  note   = {Huihui-abliterated Ornith-1.5-35B-A3B, re-quantized with oMLX's oQ4e against a cyber-weighted calibration corpus and carrying the Sharp chat template}
+}
+```

assets/cybertiel_banner_crt.gif ADDED Viewed

Git LFS Details

SHA256: 2aae028d7b12806c10936c9885a6af62f17047472db17bc58bfe445f5bb43968
Pointer size: 131 Bytes
Size of remote file: 907 kB

assets/cybertiel_bench_cybench.gif ADDED Viewed

Git LFS Details

SHA256: fe88396a45cec96218ca9399379c91aa370efbf8b275ab09049ef1b936c027c6
Pointer size: 131 Bytes
Size of remote file: 262 kB

assets/cybertiel_bench_harmbench.gif ADDED Viewed

Git LFS Details

SHA256: b19926355370dc3b538e97149e1574f7e843c5b330eef9b99dd6ae6f0f3e417e
Pointer size: 132 Bytes
Size of remote file: 1.39 MB

assets/cybertiel_bench_mmlu.gif ADDED Viewed

Git LFS Details

SHA256: 362b468eab9d6bcc453949b7641348e41fe0491d77c517f335545177579802b2
Pointer size: 131 Bytes
Size of remote file: 291 kB

assets/cybertiel_bench_swe.gif ADDED Viewed

Git LFS Details

SHA256: 09c66b62699a4fdd80b1c51142261184ef8ffcd281ed593f6ac7091212f9dcbd
Pointer size: 131 Bytes
Size of remote file: 492 kB

assets/cybertiel_bench_swe_speed.gif ADDED Viewed

Git LFS Details

SHA256: 1b6da10ea763f69b70f18ecf1310aacbfd18d1d067094eb4b3d88d983f5327ab
Pointer size: 131 Bytes
Size of remote file: 422 kB

chat_template.jinja ADDED Viewed

	@@ -0,0 +1,488 @@

+{%- set template_version = "qwen3.8-froggeric-v22.5.0" %}
+{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}
+{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}
+{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}
+{%- if preserve_reasoning is defined and preserve_reasoning is not none %}
+    {%- set _preserve_thinking = preserve_reasoning %}
+{%- elif preserve_thinking is defined and preserve_thinking is not none %}
+    {%- set _preserve_thinking = preserve_thinking %}
+{%- else %}
+    {%- set _preserve_thinking = true %}
+{%- endif %}
+{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}
+{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}
+{%- set _default_reasoning_effort = 'medium' %}
+{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}
+{%- set _effort_raw = (reasoning_effort | string | lower) if reasoning_effort is defined and reasoning_effort is not none else _default_reasoning_effort %}
+{%- set _initial_thinking = enable_thinking %}
+{%- if _effort_raw in ('none', 'off') %}
+    {%- set _initial_thinking = false %}
+    {%- set _initial_effort = 'medium' %}
+{%- elif _effort_raw in ('minimal', 'low') %}
+    {%- set _initial_effort = 'low' %}
+{%- elif _effort_raw in ('high', 'xhigh', 'max', 'ultracode', 'extreme') %}
+    {%- set _initial_effort = 'xhigh' %}
+{%- else %}
+    {%- set _initial_effort = 'medium' %}
+{%- endif %}
+{%- set ns_state = namespace(thinking=_initial_thinking, effort=_initial_effort) %}
+{%- if auto_disable_thinking_with_tools and _has_tools %}
+    {%- set ns_state.thinking = false %}
+{%- endif %}
+{%- for msg in messages %}
+    {%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}
+        {%- if msg.content is string %}
+            {%- if '<|think_off|>' in msg.content %}
+                {%- set ns_state.thinking = false %}
+            {%- elif '<|think_on|>' in msg.content %}
+                {%- set ns_state.thinking = true %}
+            {%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content or '<|think_ultracode|>' in msg.content or '<|think_extreme|>' in msg.content or '<|think_max|>' in msg.content %}
+                {%- set ns_state.thinking = true %}
+                {%- set ns_state.effort = 'xhigh' %}
+            {%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}
+                {%- set ns_state.thinking = true %}
+                {%- set ns_state.effort = 'low' %}
+            {%- elif '<|think_medium|>' in msg.content %}
+                {%- set ns_state.thinking = true %}
+                {%- set ns_state.effort = 'medium' %}
+            {%- endif %}
+        {%- elif msg.content is iterable and msg.content is not mapping %}
+            {%- for item in msg.content %}
+                {%- if item is string %}
+                    {%- set _item_text = item %}
+                {%- elif item is mapping and 'text' in item and item.text is string %}
+                    {%- set _item_text = item.text %}
+                {%- else %}
+                    {%- set _item_text = '' %}
+                {%- endif %}
+                {%- if _item_text %}
+                    {%- if '<|think_off|>' in _item_text %}
+                        {%- set ns_state.thinking = false %}
+                    {%- elif '<|think_on|>' in _item_text %}
+                        {%- set ns_state.thinking = true %}
+                    {%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text or '<|think_ultracode|>' in _item_text or '<|think_extreme|>' in _item_text or '<|think_max|>' in _item_text %}
+                        {%- set ns_state.thinking = true %}
+                        {%- set ns_state.effort = 'xhigh' %}
+                    {%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}
+                        {%- set ns_state.thinking = true %}
+                        {%- set ns_state.effort = 'low' %}
+                    {%- elif '<|think_medium|>' in _item_text %}
+                        {%- set ns_state.thinking = true %}
+                        {%- set ns_state.effort = 'medium' %}
+                    {%- endif %}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- set reasoning_instructions = '' %}
+{%- if ns_state.thinking %}
+    {%- if ns_state.effort == 'xhigh' %}
+        {%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}
+    {%- elif ns_state.effort == 'low' %}
+        {%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}
+    {%- endif %}
+{%- endif %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if item is mapping %}
+                {%- if item.type == 'image' or 'image' in item or 'image_url' in item %}
+                    {%- if is_system_content %}
+                        {{- raise_exception('System message cannot contain images.') }}
+                    {%- endif %}
+                    {%- if do_vision_count %}
+                        {%- set image_count.value = image_count.value + 1 %}
+                    {%- endif %}
+                    {%- if add_vision_id %}
+                        {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                    {%- endif %}
+                    {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+                {%- elif item.type == 'video' or 'video' in item or 'video_url' in item %}
+                    {%- if is_system_content %}
+                        {{- raise_exception('System message cannot contain videos.') }}
+                    {%- endif %}
+                    {%- if do_vision_count %}
+                        {%- set video_count.value = video_count.value + 1 %}
+                    {%- endif %}
+                    {%- if add_vision_id %}
+                        {{- 'Video ' ~ video_count.value ~ ': ' }}
+                    {%- endif %}
+                    {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+                {%- elif 'text' in item %}
+                    {{- item.text }}
+                {%- else %}
+                    {{- raise_exception('Unexpected item type in content.') }}
+                {%- endif %}
+            {%- else %}
+                {{- item | string }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- set head = namespace(count=0, seen_non_system=false) %}
+{%- for message in messages %}
+    {%- set _is_sys = (message.role == 'system' or message.role == 'developer') %}
+    {%- if _is_sys and not head.seen_non_system %}
+        {%- set head.count = head.count + 1 %}
+    {%- else %}
+        {%- set head.seen_non_system = true %}
+    {%- endif %}
+{%- endfor %}
+{%- set sys_state = namespace(content='') %}
+{%- for message in messages[:head.count] %}
+    {%- set _part = render_content(message.content, false, true) | trim %}
+    {%- if '<|think_off|>' in _part %}{%- set _part = _part.split('<|think_off|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_on|>' in _part %}{%- set _part = _part.split('<|think_on|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_xhigh|>' in _part %}{%- set _part = _part.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_high|>' in _part %}{%- set _part = _part.split('<|think_high|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_ultracode|>' in _part %}{%- set _part = _part.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_extreme|>' in _part %}{%- set _part = _part.split('<|think_extreme|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_max|>' in _part %}{%- set _part = _part.split('<|think_max|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_medium|>' in _part %}{%- set _part = _part.split('<|think_medium|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_low|>' in _part %}{%- set _part = _part.split('<|think_low|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_minimal|>' in _part %}{%- set _part = _part.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
+    {%- if _part %}
+        {%- if sys_state.content %}
+            {%- set sys_state.content = sys_state.content ~ '\n\n' ~ _part %}
+        {%- else %}
+            {%- set sys_state.content = _part %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- set _sc = sys_state.content %}
+{%- set _msgs = messages[head.count:] %}
+{%- if ns_state.thinking %}
+    {%- set _terse_lead = 'You are Cyber-Tiel-Coder, a variant of Ornith-1.5-35B-A3B. Answer directly, after thinking. Lead with the answer, then only what it needs to be correct and usable.' %}
+{%- else %}
+    {%- set _terse_lead = 'You are Cyber-Tiel-Coder, a variant of Ornith-1.5-35B-A3B. Answer directly and concisely. Give the answer with only what it needs to be correct and usable.' %}
+{%- endif %}
+{%- set _terse_core %}
+Never: open with preamble or pleasantries; restate the question; add filler transitions; hedge with niceties; or repeat a point you've already made.
+Always: keep essential steps, caveats, uncertainties, and specifics — never drop correctness or a needed warning for brevity. Keep the final answer lean. Use the least structure that conveys it (plain prose when short; lists or code only when they earn their place). If genuinely uncertain, say so and explain why — never omit uncertainty for the sake of brevity.
+If a user request is genuinely ambiguous, ask a sharp question, don't guess.
+{%- endset %}
+{%- set _terse = _terse_lead ~ '\n' ~ (_terse_core | trim) %}
+{#- `terse` chat-template kwarg: default true, so behaviour is unchanged unless a caller opts out.
+    Pass {"terse": false} via chat_template_kwargs to serve the model with only its own system
+    prompt -- useful for A/B-ing the terseness effect, or when a downstream prompt conflicts. #}
+{%- set _terse_on = terse if terse is defined else true %}
+{%- if _terse_on %}
+    {%- if not _sc %}
+        {%- set _sc = _terse | trim %}
+    {%- else %}
+        {%- set _sc = (_sc | trim) ~ '\n\n' ~ (_terse | trim) %}
+    {%- endif %}
+{%- endif %}
+{#- Some runtimes (LM Studio's MLX backend) have no native tool-call parser and prepend their own
+    protocol into the system content. Emitting ours as well gives the model two contradictory tool
+    protocols and it follows one at random. Detect that marker and stand down; the
+    `suppress_tool_instructions` kwarg overrides either way where a runtime can pass template kwargs. #}
+{%- set _runtime_tool_protocol = (_sc is defined and _sc and '[TOOL_REQUEST]' in _sc) %}
+{%- set _suppress_tools = suppress_tool_instructions if suppress_tool_instructions is defined else _runtime_tool_protocol %}
+{%- if _has_tools and not _suppress_tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if reasoning_instructions %}
+        {{- reasoning_instructions + '\n\n' }}
+    {%- endif %}
+    {{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}
+    {%- for tool in tools %}
+        {{- '\n' }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- '\n</tools>' }}
+    {%- if _tool_format == 'json' %}
+        {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}
+        {%- if ns_state.thinking %}
+            {{- '<think>\nBrief explanation of tool call\n</think>\n' }}
+        {%- endif %}
+        {{- '<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}
+        {%- if ns_state.thinking %}
+            {{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}
+        {%- endif %}
+        {{- '- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n' }}
+        {%- if ns_state.thinking %}
+            {{- '- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n' }}
+        {%- else %}
+            {{- '- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY, with NO conversational text before it.\n' }}
+        {%- endif %}
+        {{- '- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
+    {%- else %}
+        {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}
+        {%- if ns_state.thinking %}
+            {{- '<think>\nBrief explanation of tool call\n</think>\n' }}
+        {%- endif %}
+        {{- '<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}
+        {%- if ns_state.thinking %}
+            {{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}
+        {%- endif %}
+        {{- '- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n' }}
+        {%- if ns_state.thinking %}
+            {{- '- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n' }}
+        {%- else %}
+            {{- '- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY, with NO conversational text before it.\n' }}
+        {%- endif %}
+        {{- '- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
+    {%- endif %}
+    {%- if _sc %}
+        {{- '\n\n' + _sc }}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if _sc %}
+        {{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '') + _sc + '<|im_end|>\n' }}
+    {%- elif reasoning_instructions %}
+        {{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set _last_idx = _msgs | length - 1 %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}
+{%- for message in _msgs[::-1] %}
+    {%- set index = (_msgs | length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == 'user' %}
+        {%- set _rc = render_content(message.content, false) | trim %}
+        {%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {%- if _last_idx > 50 %}
+        {%- set ns.last_query_index = _last_idx %}
+    {%- else %}
+        {%- set ns.last_query_index = 0 %}
+    {%- endif %}
+{%- endif %}
+{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}
+{%- for message in _msgs %}
+    {%- set is_system = (message.role == "system" or message.role == "developer") %}
+    {%- set content = render_content(message.content, true, is_system) | trim %}
+    {%- if is_system or message.role == 'user' %}
+        {%- if '<|think_off|>' in content %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_ultracode|>' in content %}{%- set content = content.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_extreme|>' in content %}{%- set content = content.split('<|think_extreme|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_max|>' in content %}{%- set content = content.split('<|think_max|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
+    {%- endif %}
+    {%- if is_system %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- elif message.role == 'user' %}
+        {%- set ns2.consecutive_failures = 0 %}
+        {{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
+    {%- elif message.role == 'assistant' %}
+        {%- set reasoning_content = '' %}
+        {%- set _explicit_reasoning = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- if message.reasoning_content is string %}
+                {%- set _explicit_reasoning = message.reasoning_content %}
+            {%- else %}
+                {%- set _explicit_reasoning = message.reasoning_content | string %}
+            {%- endif %}
+        {%- elif message.thinking is defined and message.thinking is not none %}
+            {%- if message.thinking is string %}
+                {%- set _explicit_reasoning = message.thinking %}
+            {%- else %}
+                {%- set _explicit_reasoning = message.thinking | string %}
+            {%- endif %}
+        {%- elif message.reasoning is defined and message.reasoning is not none %}
+            {%- if message.reasoning is string %}
+                {%- set _explicit_reasoning = message.reasoning %}
+            {%- else %}
+                {%- set _explicit_reasoning = message.reasoning | string %}
+            {%- endif %}
+        {%- endif %}
+        {%- if _explicit_reasoning %}
+            {%- set _lead_end = '' %}
+            {%- if content.startswith('<think>') and '</think>' in content %}
+                {%- set _lead_end = '</think>' %}
+            {%- elif content.startswith('<thinking>') and '</thinking>' in content %}
+                {%- set _lead_end = '</thinking>' %}
+            {%- elif content.startswith('</think>') %}
+                {%- set _lead_end = '</think>' %}
+            {%- elif content.startswith('</thinking>') %}
+                {%- set _lead_end = '</thinking>' %}
+            {%- endif %}
+            {%- if _lead_end %}
+                {%- set content = content.split(_lead_end)[-1].lstrip('\n') %}
+            {%- endif %}
+            {%- set reasoning_content = _explicit_reasoning %}
+        {%- else %}
+            {%- set _think_end = '' %}
+            {%- if content.startswith('</think>') %}
+                {%- set _think_end = '</think>' %}
+            {%- elif content.startswith('</thinking>') %}
+                {%- set _think_end = '</thinking>' %}
+            {%- elif '\n</think>' in content %}
+                {%- set _think_end = '\n</think>' %}
+            {%- elif '\n</thinking>' in content %}
+                {%- set _think_end = '\n</thinking>' %}
+            {%- elif '\n</ think>' in content %}
+                {%- set _think_end = '\n</ think>' %}
+            {%- elif '\n</think >' in content %}
+                {%- set _think_end = '\n</think >' %}
+            {%- elif content.startswith('<think>') and '</think>' in content %}
+                {%- set _think_end = '</think>' %}
+            {%- elif content.startswith('<thinking>') and '</thinking>' in content %}
+                {%- set _think_end = '</thinking>' %}
+            {%- endif %}
+            {%- if _think_end %}
+                {%- if 'thinking' in _think_end %}
+                    {%- set _think_start = '<thinking>' %}
+                {%- else %}
+                    {%- set _think_start = '<think>' %}
+                {%- endif %}
+                {%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}
+                {%- if _think_start in reasoning_content %}
+                    {%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}
+                {%- endif %}
+                {%- set content = content.split(_think_end)[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content | trim %}
+        {%- if (_preserve_thinking or loop.index0 > ns.last_query_index) %}
+            {{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>assistant\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined and tool_call.function is not none %}
+                    {%- set tc = tool_call.function %}
+                {%- else %}
+                    {%- set tc = tool_call %}
+                {%- endif %}
+                {%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}
+                {%- if _tool_format == 'json' %}
+                    {%- if loop.first %}
+                        {%- if content | trim %}
+                            {{- '\n\n' }}
+                        {%- endif %}
+                    {%- else %}
+                        {{- '\n' }}
+                    {%- endif %}
+                    {%- set _args = '{}' %}
+                    {%- if tc.arguments is defined and tc.arguments is not none %}
+                        {%- if tc.arguments is mapping %}
+                            {%- set _args = tc.arguments | tojson %}
+                        {%- elif tc.arguments is string %}
+                            {%- if tc.arguments %}
+                                {%- set _args = tc.arguments %}
+                            {%- endif %}
+                        {%- else %}
+                            {%- set _args = tc.arguments | tojson %}
+                        {%- endif %}
+                    {%- endif %}
+                    {{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}
+                {%- else %}
+                    {%- if loop.first %}
+                        {%- if content | trim %}
+                            {{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}
+                        {%- else %}
+                            {{- '<tool_call>\n<function=' + tc_name + '>\n' }}
+                        {%- endif %}
+                    {%- else %}
+                        {{- '\n<tool_call>\n<function=' + tc_name + '>\n' }}
+                    {%- endif %}
+                    {%- if tc.arguments is defined and tc.arguments is not none %}
+                        {%- if tc.arguments is mapping %}
+                            {%- for args_name, args_value in tc.arguments.items() %}
+                                {{- '<parameter=' + args_name + '>\n' }}
+                                {%- if args_value is string %}
+                                    {%- set _av = args_value %}
+                                {%- else %}
+                                    {%- set _av = args_value | tojson %}
+                                {%- endif %}
+                                {%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}
+                                    {{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}
+                                {%- else %}
+                                    {{- _av }}
+                                {%- endif %}
+                                {{- '\n</parameter>\n' }}
+                            {%- endfor %}
+                        {%- else %}
+                            {%- if tc.arguments is string %}
+                                {%- set _raw_args = tc.arguments %}
+                            {%- else %}
+                                {%- set _raw_args = tc.arguments | tojson %}
+                            {%- endif %}
+                            {%- if _raw_args %}
+                                {%- if max_tool_arg_chars > 0 and _raw_args | length > max_tool_arg_chars %}
+                                    {{- _raw_args[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_raw_args | length | string) ~ ' chars]' }}
+                                {%- else %}
+                                    {{- _raw_args }}
+                                {%- endif %}
+                            {%- endif %}
+                        {%- endif %}
+                    {%- endif %}
+                    {{- '</function>\n</tool_call>' }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == 'tool' %}
+        {%- set _content_lower = content | lower %}
+        {%- set _content_head = _content_lower[:120] %}
+        {%- set _is_code_or_grep = ('throw new ' in _content_lower or 'throw error' in _content_lower or 'console.error' in _content_lower or 'logger.error' in _content_lower or 'logging.error' in _content_lower or 'import ' in _content_head or 'def ' in _content_head or 'function ' in _content_head) %}
+        {%- set _exit_code_zero = ('exit code: 0' in _content_head or 'process exited with code 0' in _content_head) %}
+        {%- set _error_field_ok = ('"error": null' in _content_head or '"error":null' in _content_head or '"error": false' in _content_head or '"error":false' in _content_head or '"error": ""' in _content_head or '"error":""' in _content_head) %}
+        {%- set _strong_error = (('"error":' in _content_head and not _error_field_ok) or '"status": "error"' in _content_head or '"status":"error"' in _content_head or 'traceback (most recent call last):' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'fatal:' in _content_head or (('exit code: ' in _content_head or 'process exited with code' in _content_head) and not _exit_code_zero) or _content_head.startswith('exception:') or _content_head.startswith('failed to ')) %}
+        {%- set _weak_error = ('error:' in _content_head or 'err!' in _content_head) %}
+        {%- set _weak_suppressed = ('$ ' in _content_head or 'took ' in _content_head or content | length >= 600) %}
+        {%- if not _is_code_or_grep and (_strong_error or (_weak_error and not _weak_suppressed)) %}
+            {%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}
+        {%- else %}
+            {%- set ns2.consecutive_failures = 0 %}
+        {%- endif %}
+        {%- if ns2.prev_role != 'tool' %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {%- set _is_json_payload = (_tool_format == 'json' and content | trim | length > 0 and (content | trim)[:1] in ('{', '[')) %}
+        {%- if not _is_json_payload and max_tool_response_chars > 0 and content | length > max_tool_response_chars %}
+            {%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}
+        {%- endif %}
+        {{- '\n<tool_response>\n' + content }}
+        {%- if ns2.consecutive_failures >= 2 %}
+            {{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}
+        {%- elif ns2.consecutive_failures == 1 %}
+            {{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}
+        {%- endif %}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- else %}
+            {%- set _next_role = _msgs[loop.index0 + 1].role %}
+            {%- if _next_role != 'tool' %}
+                {{- '<|im_end|>\n' }}
+            {%- endif %}
+        {%- endif %}
+    {%- else %}
+        {{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}
+    {%- endif %}
+    {%- set ns2.prev_role = message.role %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if not ns_state.thinking %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

config.json ADDED Viewed

	@@ -0,0 +1,3175 @@

+{
+  "architectures": [
+    "Qwen3_5MoeForConditionalGeneration"
+  ],
+  "bos_token_id": null,
+  "dtype": "bfloat16",
+  "eos_token_id": 248046,
+  "hidden_size": 2048,
+  "image_token_id": 248056,
+  "model_type": "qwen3_5_moe",
+  "pad_token_id": 248044,
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attn_output_gate": true,
+    "bos_token_id": 248044,
+    "dtype": "bfloat16",
+    "eos_token_id": 248044,
+    "full_attention_interval": 4,
+    "head_dim": 256,
+    "hidden_act": "silu",
+    "hidden_size": 2048,
+    "initializer_range": 0.02,
+    "layer_types": [
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention"
+    ],
+    "linear_conv_kernel_dim": 4,
+    "linear_key_head_dim": 128,
+    "linear_num_key_heads": 16,
+    "linear_num_value_heads": 32,
+    "linear_value_head_dim": 128,
+    "mamba_ssm_dtype": "float32",
+    "max_position_embeddings": 262144,
+    "model_type": "qwen3_5_moe_text",
+    "moe_intermediate_size": 512,
+    "mtp_num_hidden_layers": 0,
+    "mtp_use_dedicated_embeddings": false,
+    "num_attention_heads": 16,
+    "num_experts": 256,
+    "num_experts_per_tok": 8,
+    "num_hidden_layers": 40,
+    "num_key_value_heads": 2,
+    "output_router_logits": false,
+    "pad_token_id": 248044,
+    "partial_rotary_factor": 0.25,
+    "rms_norm_eps": 1e-06,
+    "rope_parameters": {
+      "mrope_interleaved": true,
+      "mrope_section": [
+        11,
+        11,
+        10
+      ],
+      "partial_rotary_factor": 0.25,
+      "rope_theta": 10000000,
+      "type": "default"
+    },
+    "router_aux_loss_coef": 0.0,
+    "shared_expert_intermediate_size": 512,
+    "tie_word_embeddings": false,
+    "use_cache": false,
+    "vocab_size": 248320
+  },
+  "tie_word_embeddings": false,
+  "transformers_version": "5.8.1",
+  "video_token_id": 248057,
+  "vision_config": {
+    "deepstack_visual_indexes": [],
+    "depth": 27,
+    "dtype": "bfloat16",
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 4304,
+    "model_type": "qwen3_5_moe_vision",
+    "num_heads": 16,
+    "num_position_embeddings": 2304,
+    "out_hidden_size": 2048,
+    "patch_size": 16,
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2
+  },
+  "vision_end_token_id": 248054,
+  "vision_start_token_id": 248053,
+  "quantization": {
+    "group_size": 64,
+    "bits": 4,
+    "mode": "affine",
+    "language_model.model.embed_tokens": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.linear_attn.in_proj_a": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.linear_attn.in_proj_b": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.linear_attn.in_proj_qkv": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.1.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.1.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.1.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.1.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.1.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.1.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.1.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.1.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.2.linear_attn.in_proj_a": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.2.linear_attn.in_proj_b": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.2.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.2.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.2.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.2.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.2.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.2.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.3.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.3.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.3.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.3.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.3.self_attn.k_proj": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.4.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.4.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.4.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.4.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.4.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.4.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.4.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.4.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.5.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.5.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.5.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.5.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.5.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.5.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.5.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.5.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.6.linear_attn.in_proj_a": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.6.linear_attn.in_proj_b": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.6.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.6.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.6.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.6.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.6.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.6.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.7.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.7.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.7.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.7.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.7.self_attn.k_proj": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.7.self_attn.q_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.7.self_attn.v_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.8.linear_attn.in_proj_a": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.8.linear_attn.in_proj_b": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.8.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.8.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.8.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.8.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.8.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.8.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.9.linear_attn.in_proj_a": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.9.linear_attn.in_proj_b": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.9.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.9.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.9.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.9.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.9.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.9.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.linear_attn.in_proj_a": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.linear_attn.in_proj_b": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.linear_attn.in_proj_qkv": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.11.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.11.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.11.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.11.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.12.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.12.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.12.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.12.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.12.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.12.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.12.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.12.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.13.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.13.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.13.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.13.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.13.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.13.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.13.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.13.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.14.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.14.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.14.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.14.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.14.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.14.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.14.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.14.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.15.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.15.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.15.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.15.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.16.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.16.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.16.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.16.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.16.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.16.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.16.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.16.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.17.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.17.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.17.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.17.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.17.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.17.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.17.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.17.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.18.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.18.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.18.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.18.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.18.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.18.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.18.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.18.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.19.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.19.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.19.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.19.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.20.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.20.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.20.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.20.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.20.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.20.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.20.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.20.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.21.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.21.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.21.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.21.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.21.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.21.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.21.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.21.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.22.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.22.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.22.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.22.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.22.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.22.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.22.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.22.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.23.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.23.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.23.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.23.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.24.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.24.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.24.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.24.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.24.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.24.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.24.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.24.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.25.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.25.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.25.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.25.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.25.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.25.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.25.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.25.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.26.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.26.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.26.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.26.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.26.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.26.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.26.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.26.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.27.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.27.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.27.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.27.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.28.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.28.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.28.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.28.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.28.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.28.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.28.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.28.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.29.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.29.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.29.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.29.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.29.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.29.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.29.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.29.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.30.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.30.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.30.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.30.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.30.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.30.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.30.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.30.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.31.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.31.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.31.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.31.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.31.self_attn.k_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.31.self_attn.o_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.31.self_attn.q_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.31.self_attn.v_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.32.linear_attn.in_proj_a": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.32.linear_attn.in_proj_b": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.32.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.32.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.32.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.32.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.32.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.32.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.linear_attn.in_proj_a": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.linear_attn.in_proj_b": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.linear_attn.in_proj_qkv": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.34.linear_attn.in_proj_a": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.34.linear_attn.in_proj_b": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.34.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.34.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.34.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.34.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.34.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.34.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.35.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.35.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.35.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.35.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.35.self_attn.k_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.35.self_attn.o_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.35.self_attn.q_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.35.self_attn.v_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.linear_attn.in_proj_a": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.linear_attn.in_proj_b": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.linear_attn.in_proj_qkv": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.linear_attn.in_proj_a": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.linear_attn.in_proj_b": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.linear_attn.in_proj_qkv": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.linear_attn.in_proj_z": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.linear_attn.in_proj_a": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.linear_attn.in_proj_b": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.linear_attn.in_proj_qkv": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.lm_head": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.39.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.39.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.39.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.39.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.39.self_attn.k_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.39.self_attn.o_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.39.self_attn.q_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.39.self_attn.v_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    }
+  },
+  "quantization_config": {
+    "group_size": 64,
+    "bits": 4,
+    "mode": "affine",
+    "language_model.model.embed_tokens": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.linear_attn.in_proj_a": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.linear_attn.in_proj_b": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.linear_attn.in_proj_qkv": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.0.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.1.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.1.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.1.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.1.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.1.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.1.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.1.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.1.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.2.linear_attn.in_proj_a": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.2.linear_attn.in_proj_b": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.2.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.2.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.2.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.2.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.2.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.2.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.3.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.3.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.3.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.3.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.3.self_attn.k_proj": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.4.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.4.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.4.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.4.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.4.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.4.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.4.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.4.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.5.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.5.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.5.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.5.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.5.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.5.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.5.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.5.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.6.linear_attn.in_proj_a": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.6.linear_attn.in_proj_b": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.6.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.6.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.6.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.6.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.6.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.6.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.7.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.7.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.7.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.7.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.7.self_attn.k_proj": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.7.self_attn.q_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.7.self_attn.v_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.8.linear_attn.in_proj_a": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.8.linear_attn.in_proj_b": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.8.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.8.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.8.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.8.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.8.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.8.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.9.linear_attn.in_proj_a": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.9.linear_attn.in_proj_b": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.9.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.9.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.9.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.9.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.9.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.9.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.linear_attn.in_proj_a": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.linear_attn.in_proj_b": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.linear_attn.in_proj_qkv": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.10.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.11.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.11.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.11.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.11.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.12.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.12.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.12.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.12.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.12.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.12.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.12.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.12.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.13.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.13.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.13.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.13.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.13.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.13.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.13.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.13.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.14.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.14.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.14.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.14.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.14.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.14.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.14.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.14.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.15.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.15.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.15.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.15.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.16.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.16.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.16.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.16.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.16.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.16.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.16.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.16.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.17.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.17.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.17.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.17.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.17.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.17.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.17.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.17.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.18.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.18.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.18.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.18.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.18.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.18.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.18.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.18.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.19.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.19.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.19.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.19.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.20.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.20.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.20.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.20.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.20.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.20.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.20.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.20.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.21.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.21.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.21.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.21.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.21.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.21.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.21.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.21.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.22.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.22.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.22.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.22.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.22.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.22.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.22.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.22.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.23.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.23.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.23.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.23.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.24.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.24.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.24.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.24.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.24.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.24.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.24.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.24.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.25.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.25.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.25.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.25.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.25.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.25.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.25.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.25.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.26.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.26.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.26.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.26.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.26.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.26.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.26.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.26.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.27.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.27.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.27.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.27.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.28.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.28.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.28.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.28.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.28.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.28.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.28.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.28.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.29.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.29.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.29.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.29.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.29.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.29.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.29.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.29.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.30.linear_attn.in_proj_a": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.30.linear_attn.in_proj_b": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.30.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.30.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.30.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.30.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.30.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.30.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.31.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.31.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.31.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.31.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.31.self_attn.k_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.31.self_attn.o_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.31.self_attn.q_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.31.self_attn.v_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.32.linear_attn.in_proj_a": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.32.linear_attn.in_proj_b": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.32.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.32.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.32.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.32.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.32.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.32.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.linear_attn.in_proj_a": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.linear_attn.in_proj_b": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.linear_attn.in_proj_qkv": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.33.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.34.linear_attn.in_proj_a": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.34.linear_attn.in_proj_b": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.34.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.34.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.34.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.34.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.34.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.34.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.35.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.35.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.35.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.35.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.35.self_attn.k_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.35.self_attn.o_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.35.self_attn.q_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.35.self_attn.v_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.linear_attn.in_proj_a": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.linear_attn.in_proj_b": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.linear_attn.in_proj_qkv": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.36.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.linear_attn.in_proj_a": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.linear_attn.in_proj_b": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.linear_attn.in_proj_qkv": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.linear_attn.in_proj_z": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.37.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.linear_attn.in_proj_a": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.linear_attn.in_proj_b": {
+      "bits": 6,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.linear_attn.in_proj_qkv": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.linear_attn.in_proj_z": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.linear_attn.out_proj": {
+      "bits": 5,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.38.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.lm_head": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.39.mlp.shared_expert.down_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.39.mlp.shared_expert.gate_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.39.mlp.shared_expert.up_proj": {
+      "bits": 8,
+      "group_size": 128,
+      "mode": "affine"
+    },
+    "language_model.model.layers.39.mlp.shared_expert_gate": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.39.self_attn.k_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.39.self_attn.o_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.39.self_attn.q_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    },
+    "language_model.model.layers.39.self_attn.v_proj": {
+      "bits": 8,
+      "group_size": 64,
+      "mode": "affine"
+    }
+  }
+}

generation_config.json ADDED Viewed

	@@ -0,0 +1,12 @@

+{
+  "bos_token_id": 248044,
+  "do_sample": true,
+  "eos_token_id": [
+    248046,
+    248044
+  ],
+  "pad_token_id": 248044,
+  "temperature": 0.6,
+  "top_k": 20,
+  "top_p": 0.95
+}

merges.txt ADDED Viewed

The diff for this file is too large to render. See raw diff

model-00001-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d387c5a8a7506c9b9932a6842f96f78a82ef99554cf85486fbed1682b34cd4a1
+size 5097618408

model-00002-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9724b21302be71819b349b5a0dfd3caf4620cc496e9bb3de11233daf1d090384
+size 5133842458

model-00003-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7828e207b7a8a5de24c9384988968f9a48059f86d8f69d4abff33f2e576a5898
+size 5133842508

model-00004-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2e2ec190fc7de3a3067233c77340ae3055d3da3fde5b24b4d7e4368bd033866d
+size 5133842502

model-00005-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2a6622aa399a3b7b233d6f6ba26ea8044a0eab39fc9422d0cbafc0e0ead84f96
+size 603981479

model.safetensors.index.json ADDED Viewed

The diff for this file is too large to render. See raw diff

oq_imatrix_report.json ADDED Viewed

	@@ -0,0 +1,577 @@

+{
+  "enabled": true,
+  "cache_path": "/Users/miad/bench-qwen36/.oqe_imatrix/Huihui-Ornith-1.5-35B-A3B-abliterated-a65d1f247df2-s128-l512.npz",
+  "cache_reused": false,
+  "entry_count": 510,
+  "calib_dataset": "oqe_code_multilingual",
+  "collection": {
+    "dataset": "oqe_code_multilingual",
+    "requested_samples": 128,
+    "seq_length": 512,
+    "adaptive": true,
+    "adaptive_step_samples": 128,
+    "adaptive_max_samples": 1024,
+    "available_samples": 1024,
+    "micro_batch_size": 6,
+    "micro_batches": 22,
+    "batch_plan": {
+      "micro_batch_size": 6,
+      "estimated_sample_bytes": 33554432,
+      "capture_budget_bytes": 220672381,
+      "system_available_bytes": 42489053184,
+      "metal_available_bytes": 55606213600,
+      "live_available_bytes": 42489053184,
+      "model_bytes": 20421815039,
+      "remaining_available_bytes": 22067238145,
+      "fits_one_sample": true,
+      "hidden_size": 2048,
+      "num_experts": 256,
+      "top_k": 8,
+      "gemma4_state_bytes": 0,
+      "num_hidden_layers": 40,
+      "num_kv_shared_layers": 0,
+      "per_layer_input_size": 0
+    },
+    "processed_samples": 128,
+    "installed_modules": 621,
+    "capture_module_classes": {
+      "Linear": 150,
+      "QuantizedLinear": 351,
+      "QuantizedSwitchLinear": 120
+    },
+    "switch_capture_modules": 120,
+    "requires_expert_counts": true,
+    "coverage_sufficient": true,
+    "collection_sufficient": true,
+    "coverage": {
+      "has_expert_counts": true,
+      "expert_modules": 120,
+      "total_experts": 30720,
+      "active_experts": 30720,
+      "zero_count_experts": 0,
+      "active_ratio": 1.0,
+      "min_count": 1,
+      "p05_count": 301.0,
+      "p10_count": 454.9000000000001,
+      "median_count": 1436.5,
+      "max_count": 32813,
+      "min_required_count": 16,
+      "required_percentile": 5
+    },
+    "rounds": [
+      {
+        "processed_samples": 128,
+        "coverage_sufficient": true,
+        "collection_sufficient": true,
+        "coverage": {
+          "has_expert_counts": true,
+          "expert_modules": 120,
+          "total_experts": 30720,
+          "active_experts": 30720,
+          "zero_count_experts": 0,
+          "active_ratio": 1.0,
+          "min_count": 1,
+          "p05_count": 301.0,
+          "p10_count": 454.9000000000001,
+          "median_count": 1436.5,
+          "max_count": 32813,
+          "min_required_count": 16,
+          "required_percentile": 5
+        }
+      }
+    ]
+  },
+  "expert_coverage": {
+    "has_expert_counts": true,
+    "expert_modules": 120,
+    "total_experts": 30720,
+    "active_experts": 30720,
+    "zero_count_experts": 0,
+    "active_ratio": 1.0,
+    "min_count": 1,
+    "p05_count": 301.0,
+    "p10_count": 454.9000000000001,
+    "median_count": 1436.5,
+    "max_count": 32813,
+    "min_required_count": 16,
+    "required_percentile": 5
+  },
+  "applied": [
+    "language_model.model.layers.0.linear_attn.in_proj_a",
+    "language_model.model.layers.0.linear_attn.in_proj_b",
+    "language_model.model.layers.0.linear_attn.in_proj_qkv",
+    "language_model.model.layers.0.linear_attn.in_proj_z",
+    "language_model.model.layers.0.linear_attn.out_proj",
+    "language_model.model.layers.0.mlp.shared_expert.down_proj",
+    "language_model.model.layers.0.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.0.mlp.shared_expert.up_proj",
+    "language_model.model.layers.0.mlp.shared_expert_gate",
+    "language_model.model.layers.0.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.0.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.0.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.1.linear_attn.in_proj_a",
+    "language_model.model.layers.1.linear_attn.in_proj_b",
+    "language_model.model.layers.1.linear_attn.in_proj_qkv",
+    "language_model.model.layers.1.linear_attn.in_proj_z",
+    "language_model.model.layers.1.linear_attn.out_proj",
+    "language_model.model.layers.1.mlp.shared_expert.down_proj",
+    "language_model.model.layers.1.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.1.mlp.shared_expert.up_proj",
+    "language_model.model.layers.1.mlp.shared_expert_gate",
+    "language_model.model.layers.1.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.1.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.1.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.10.linear_attn.in_proj_a",
+    "language_model.model.layers.10.linear_attn.in_proj_b",
+    "language_model.model.layers.10.linear_attn.in_proj_qkv",
+    "language_model.model.layers.10.linear_attn.in_proj_z",
+    "language_model.model.layers.10.linear_attn.out_proj",
+    "language_model.model.layers.10.mlp.shared_expert.down_proj",
+    "language_model.model.layers.10.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.10.mlp.shared_expert.up_proj",
+    "language_model.model.layers.10.mlp.shared_expert_gate",
+    "language_model.model.layers.10.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.10.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.10.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.11.mlp.shared_expert.down_proj",
+    "language_model.model.layers.11.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.11.mlp.shared_expert.up_proj",
+    "language_model.model.layers.11.mlp.shared_expert_gate",
+    "language_model.model.layers.11.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.11.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.11.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.11.self_attn.k_proj",
+    "language_model.model.layers.11.self_attn.o_proj",
+    "language_model.model.layers.11.self_attn.q_proj",
+    "language_model.model.layers.11.self_attn.v_proj",
+    "language_model.model.layers.12.linear_attn.in_proj_a",
+    "language_model.model.layers.12.linear_attn.in_proj_b",
+    "language_model.model.layers.12.linear_attn.in_proj_qkv",
+    "language_model.model.layers.12.linear_attn.in_proj_z",
+    "language_model.model.layers.12.linear_attn.out_proj",
+    "language_model.model.layers.12.mlp.shared_expert.down_proj",
+    "language_model.model.layers.12.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.12.mlp.shared_expert.up_proj",
+    "language_model.model.layers.12.mlp.shared_expert_gate",
+    "language_model.model.layers.12.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.12.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.12.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.13.linear_attn.in_proj_a",
+    "language_model.model.layers.13.linear_attn.in_proj_b",
+    "language_model.model.layers.13.linear_attn.in_proj_qkv",
+    "language_model.model.layers.13.linear_attn.in_proj_z",
+    "language_model.model.layers.13.linear_attn.out_proj",
+    "language_model.model.layers.13.mlp.shared_expert.down_proj",
+    "language_model.model.layers.13.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.13.mlp.shared_expert.up_proj",
+    "language_model.model.layers.13.mlp.shared_expert_gate",
+    "language_model.model.layers.13.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.13.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.13.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.14.linear_attn.in_proj_a",
+    "language_model.model.layers.14.linear_attn.in_proj_b",
+    "language_model.model.layers.14.linear_attn.in_proj_qkv",
+    "language_model.model.layers.14.linear_attn.in_proj_z",
+    "language_model.model.layers.14.linear_attn.out_proj",
+    "language_model.model.layers.14.mlp.shared_expert.down_proj",
+    "language_model.model.layers.14.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.14.mlp.shared_expert.up_proj",
+    "language_model.model.layers.14.mlp.shared_expert_gate",
+    "language_model.model.layers.14.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.14.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.14.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.15.mlp.shared_expert.down_proj",
+    "language_model.model.layers.15.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.15.mlp.shared_expert.up_proj",
+    "language_model.model.layers.15.mlp.shared_expert_gate",
+    "language_model.model.layers.15.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.15.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.15.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.15.self_attn.k_proj",
+    "language_model.model.layers.15.self_attn.o_proj",
+    "language_model.model.layers.15.self_attn.q_proj",
+    "language_model.model.layers.15.self_attn.v_proj",
+    "language_model.model.layers.16.linear_attn.in_proj_a",
+    "language_model.model.layers.16.linear_attn.in_proj_b",
+    "language_model.model.layers.16.linear_attn.in_proj_qkv",
+    "language_model.model.layers.16.linear_attn.in_proj_z",
+    "language_model.model.layers.16.linear_attn.out_proj",
+    "language_model.model.layers.16.mlp.shared_expert.down_proj",
+    "language_model.model.layers.16.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.16.mlp.shared_expert.up_proj",
+    "language_model.model.layers.16.mlp.shared_expert_gate",
+    "language_model.model.layers.16.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.16.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.16.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.17.linear_attn.in_proj_a",
+    "language_model.model.layers.17.linear_attn.in_proj_b",
+    "language_model.model.layers.17.linear_attn.in_proj_qkv",
+    "language_model.model.layers.17.linear_attn.in_proj_z",
+    "language_model.model.layers.17.linear_attn.out_proj",
+    "language_model.model.layers.17.mlp.shared_expert.down_proj",
+    "language_model.model.layers.17.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.17.mlp.shared_expert.up_proj",
+    "language_model.model.layers.17.mlp.shared_expert_gate",
+    "language_model.model.layers.17.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.17.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.17.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.18.linear_attn.in_proj_a",
+    "language_model.model.layers.18.linear_attn.in_proj_b",
+    "language_model.model.layers.18.linear_attn.in_proj_qkv",
+    "language_model.model.layers.18.linear_attn.in_proj_z",
+    "language_model.model.layers.18.linear_attn.out_proj",
+    "language_model.model.layers.18.mlp.shared_expert.down_proj",
+    "language_model.model.layers.18.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.18.mlp.shared_expert.up_proj",
+    "language_model.model.layers.18.mlp.shared_expert_gate",
+    "language_model.model.layers.18.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.18.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.18.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.19.mlp.shared_expert.down_proj",
+    "language_model.model.layers.19.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.19.mlp.shared_expert.up_proj",
+    "language_model.model.layers.19.mlp.shared_expert_gate",
+    "language_model.model.layers.19.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.19.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.19.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.19.self_attn.k_proj",
+    "language_model.model.layers.19.self_attn.o_proj",
+    "language_model.model.layers.19.self_attn.q_proj",
+    "language_model.model.layers.19.self_attn.v_proj",
+    "language_model.model.layers.2.linear_attn.in_proj_a",
+    "language_model.model.layers.2.linear_attn.in_proj_b",
+    "language_model.model.layers.2.linear_attn.in_proj_qkv",
+    "language_model.model.layers.2.linear_attn.in_proj_z",
+    "language_model.model.layers.2.linear_attn.out_proj",
+    "language_model.model.layers.2.mlp.shared_expert.down_proj",
+    "language_model.model.layers.2.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.2.mlp.shared_expert.up_proj",
+    "language_model.model.layers.2.mlp.shared_expert_gate",
+    "language_model.model.layers.2.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.2.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.2.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.20.linear_attn.in_proj_a",
+    "language_model.model.layers.20.linear_attn.in_proj_b",
+    "language_model.model.layers.20.linear_attn.in_proj_qkv",
+    "language_model.model.layers.20.linear_attn.in_proj_z",
+    "language_model.model.layers.20.linear_attn.out_proj",
+    "language_model.model.layers.20.mlp.shared_expert.down_proj",
+    "language_model.model.layers.20.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.20.mlp.shared_expert.up_proj",
+    "language_model.model.layers.20.mlp.shared_expert_gate",
+    "language_model.model.layers.20.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.20.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.20.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.21.linear_attn.in_proj_a",
+    "language_model.model.layers.21.linear_attn.in_proj_b",
+    "language_model.model.layers.21.linear_attn.in_proj_qkv",
+    "language_model.model.layers.21.linear_attn.in_proj_z",
+    "language_model.model.layers.21.linear_attn.out_proj",
+    "language_model.model.layers.21.mlp.shared_expert.down_proj",
+    "language_model.model.layers.21.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.21.mlp.shared_expert.up_proj",
+    "language_model.model.layers.21.mlp.shared_expert_gate",
+    "language_model.model.layers.21.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.21.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.21.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.22.linear_attn.in_proj_a",
+    "language_model.model.layers.22.linear_attn.in_proj_b",
+    "language_model.model.layers.22.linear_attn.in_proj_qkv",
+    "language_model.model.layers.22.linear_attn.in_proj_z",
+    "language_model.model.layers.22.linear_attn.out_proj",
+    "language_model.model.layers.22.mlp.shared_expert.down_proj",
+    "language_model.model.layers.22.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.22.mlp.shared_expert.up_proj",
+    "language_model.model.layers.22.mlp.shared_expert_gate",
+    "language_model.model.layers.22.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.22.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.22.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.23.mlp.shared_expert.down_proj",
+    "language_model.model.layers.23.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.23.mlp.shared_expert.up_proj",
+    "language_model.model.layers.23.mlp.shared_expert_gate",
+    "language_model.model.layers.23.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.23.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.23.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.23.self_attn.k_proj",
+    "language_model.model.layers.23.self_attn.o_proj",
+    "language_model.model.layers.23.self_attn.q_proj",
+    "language_model.model.layers.23.self_attn.v_proj",
+    "language_model.model.layers.24.linear_attn.in_proj_a",
+    "language_model.model.layers.24.linear_attn.in_proj_b",
+    "language_model.model.layers.24.linear_attn.in_proj_qkv",
+    "language_model.model.layers.24.linear_attn.in_proj_z",
+    "language_model.model.layers.24.linear_attn.out_proj",
+    "language_model.model.layers.24.mlp.shared_expert.down_proj",
+    "language_model.model.layers.24.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.24.mlp.shared_expert.up_proj",
+    "language_model.model.layers.24.mlp.shared_expert_gate",
+    "language_model.model.layers.24.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.24.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.24.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.25.linear_attn.in_proj_a",
+    "language_model.model.layers.25.linear_attn.in_proj_b",
+    "language_model.model.layers.25.linear_attn.in_proj_qkv",
+    "language_model.model.layers.25.linear_attn.in_proj_z",
+    "language_model.model.layers.25.linear_attn.out_proj",
+    "language_model.model.layers.25.mlp.shared_expert.down_proj",
+    "language_model.model.layers.25.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.25.mlp.shared_expert.up_proj",
+    "language_model.model.layers.25.mlp.shared_expert_gate",
+    "language_model.model.layers.25.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.25.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.25.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.26.linear_attn.in_proj_a",
+    "language_model.model.layers.26.linear_attn.in_proj_b",
+    "language_model.model.layers.26.linear_attn.in_proj_qkv",
+    "language_model.model.layers.26.linear_attn.in_proj_z",
+    "language_model.model.layers.26.linear_attn.out_proj",
+    "language_model.model.layers.26.mlp.shared_expert.down_proj",
+    "language_model.model.layers.26.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.26.mlp.shared_expert.up_proj",
+    "language_model.model.layers.26.mlp.shared_expert_gate",
+    "language_model.model.layers.26.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.26.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.26.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.27.mlp.shared_expert.down_proj",
+    "language_model.model.layers.27.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.27.mlp.shared_expert.up_proj",
+    "language_model.model.layers.27.mlp.shared_expert_gate",
+    "language_model.model.layers.27.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.27.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.27.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.27.self_attn.k_proj",
+    "language_model.model.layers.27.self_attn.o_proj",
+    "language_model.model.layers.27.self_attn.q_proj",
+    "language_model.model.layers.27.self_attn.v_proj",
+    "language_model.model.layers.28.linear_attn.in_proj_a",
+    "language_model.model.layers.28.linear_attn.in_proj_b",
+    "language_model.model.layers.28.linear_attn.in_proj_qkv",
+    "language_model.model.layers.28.linear_attn.in_proj_z",
+    "language_model.model.layers.28.linear_attn.out_proj",
+    "language_model.model.layers.28.mlp.shared_expert.down_proj",
+    "language_model.model.layers.28.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.28.mlp.shared_expert.up_proj",
+    "language_model.model.layers.28.mlp.shared_expert_gate",
+    "language_model.model.layers.28.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.28.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.28.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.29.linear_attn.in_proj_a",
+    "language_model.model.layers.29.linear_attn.in_proj_b",
+    "language_model.model.layers.29.linear_attn.in_proj_qkv",
+    "language_model.model.layers.29.linear_attn.in_proj_z",
+    "language_model.model.layers.29.linear_attn.out_proj",
+    "language_model.model.layers.29.mlp.shared_expert.down_proj",
+    "language_model.model.layers.29.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.29.mlp.shared_expert.up_proj",
+    "language_model.model.layers.29.mlp.shared_expert_gate",
+    "language_model.model.layers.29.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.29.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.29.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.3.mlp.shared_expert.down_proj",
+    "language_model.model.layers.3.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.3.mlp.shared_expert.up_proj",
+    "language_model.model.layers.3.mlp.shared_expert_gate",
+    "language_model.model.layers.3.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.3.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.3.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.3.self_attn.k_proj",
+    "language_model.model.layers.3.self_attn.o_proj",
+    "language_model.model.layers.3.self_attn.q_proj",
+    "language_model.model.layers.3.self_attn.v_proj",
+    "language_model.model.layers.30.linear_attn.in_proj_a",
+    "language_model.model.layers.30.linear_attn.in_proj_b",
+    "language_model.model.layers.30.linear_attn.in_proj_qkv",
+    "language_model.model.layers.30.linear_attn.in_proj_z",
+    "language_model.model.layers.30.linear_attn.out_proj",
+    "language_model.model.layers.30.mlp.shared_expert.down_proj",
+    "language_model.model.layers.30.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.30.mlp.shared_expert.up_proj",
+    "language_model.model.layers.30.mlp.shared_expert_gate",
+    "language_model.model.layers.30.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.30.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.30.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.31.mlp.shared_expert.down_proj",
+    "language_model.model.layers.31.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.31.mlp.shared_expert.up_proj",
+    "language_model.model.layers.31.mlp.shared_expert_gate",
+    "language_model.model.layers.31.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.31.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.31.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.31.self_attn.k_proj",
+    "language_model.model.layers.31.self_attn.o_proj",
+    "language_model.model.layers.31.self_attn.q_proj",
+    "language_model.model.layers.31.self_attn.v_proj",
+    "language_model.model.layers.32.linear_attn.in_proj_a",
+    "language_model.model.layers.32.linear_attn.in_proj_b",
+    "language_model.model.layers.32.linear_attn.in_proj_qkv",
+    "language_model.model.layers.32.linear_attn.in_proj_z",
+    "language_model.model.layers.32.linear_attn.out_proj",
+    "language_model.model.layers.32.mlp.shared_expert.down_proj",
+    "language_model.model.layers.32.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.32.mlp.shared_expert.up_proj",
+    "language_model.model.layers.32.mlp.shared_expert_gate",
+    "language_model.model.layers.32.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.32.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.32.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.33.linear_attn.in_proj_a",
+    "language_model.model.layers.33.linear_attn.in_proj_b",
+    "language_model.model.layers.33.linear_attn.in_proj_qkv",
+    "language_model.model.layers.33.linear_attn.in_proj_z",
+    "language_model.model.layers.33.linear_attn.out_proj",
+    "language_model.model.layers.33.mlp.shared_expert.down_proj",
+    "language_model.model.layers.33.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.33.mlp.shared_expert.up_proj",
+    "language_model.model.layers.33.mlp.shared_expert_gate",
+    "language_model.model.layers.33.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.33.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.33.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.34.linear_attn.in_proj_a",
+    "language_model.model.layers.34.linear_attn.in_proj_b",
+    "language_model.model.layers.34.linear_attn.in_proj_qkv",
+    "language_model.model.layers.34.linear_attn.in_proj_z",
+    "language_model.model.layers.34.linear_attn.out_proj",
+    "language_model.model.layers.34.mlp.shared_expert.down_proj",
+    "language_model.model.layers.34.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.34.mlp.shared_expert.up_proj",
+    "language_model.model.layers.34.mlp.shared_expert_gate",
+    "language_model.model.layers.34.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.34.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.34.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.35.mlp.shared_expert.down_proj",
+    "language_model.model.layers.35.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.35.mlp.shared_expert.up_proj",
+    "language_model.model.layers.35.mlp.shared_expert_gate",
+    "language_model.model.layers.35.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.35.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.35.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.35.self_attn.k_proj",
+    "language_model.model.layers.35.self_attn.o_proj",
+    "language_model.model.layers.35.self_attn.q_proj",
+    "language_model.model.layers.35.self_attn.v_proj",
+    "language_model.model.layers.36.linear_attn.in_proj_a",
+    "language_model.model.layers.36.linear_attn.in_proj_b",
+    "language_model.model.layers.36.linear_attn.in_proj_qkv",
+    "language_model.model.layers.36.linear_attn.in_proj_z",
+    "language_model.model.layers.36.linear_attn.out_proj",
+    "language_model.model.layers.36.mlp.shared_expert.down_proj",
+    "language_model.model.layers.36.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.36.mlp.shared_expert.up_proj",
+    "language_model.model.layers.36.mlp.shared_expert_gate",
+    "language_model.model.layers.36.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.36.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.36.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.37.linear_attn.in_proj_a",
+    "language_model.model.layers.37.linear_attn.in_proj_b",
+    "language_model.model.layers.37.linear_attn.in_proj_qkv",
+    "language_model.model.layers.37.linear_attn.in_proj_z",
+    "language_model.model.layers.37.linear_attn.out_proj",
+    "language_model.model.layers.37.mlp.shared_expert.down_proj",
+    "language_model.model.layers.37.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.37.mlp.shared_expert.up_proj",
+    "language_model.model.layers.37.mlp.shared_expert_gate",
+    "language_model.model.layers.37.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.37.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.37.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.38.linear_attn.in_proj_a",
+    "language_model.model.layers.38.linear_attn.in_proj_b",
+    "language_model.model.layers.38.linear_attn.in_proj_qkv",
+    "language_model.model.layers.38.linear_attn.in_proj_z",
+    "language_model.model.layers.38.linear_attn.out_proj",
+    "language_model.model.layers.38.mlp.shared_expert.down_proj",
+    "language_model.model.layers.38.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.38.mlp.shared_expert.up_proj",
+    "language_model.model.layers.38.mlp.shared_expert_gate",
+    "language_model.model.layers.38.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.38.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.38.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.39.mlp.shared_expert.down_proj",
+    "language_model.model.layers.39.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.39.mlp.shared_expert.up_proj",
+    "language_model.model.layers.39.mlp.shared_expert_gate",
+    "language_model.model.layers.39.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.39.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.39.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.39.self_attn.k_proj",
+    "language_model.model.layers.39.self_attn.o_proj",
+    "language_model.model.layers.39.self_attn.q_proj",
+    "language_model.model.layers.39.self_attn.v_proj",
+    "language_model.model.layers.4.linear_attn.in_proj_a",
+    "language_model.model.layers.4.linear_attn.in_proj_b",
+    "language_model.model.layers.4.linear_attn.in_proj_qkv",
+    "language_model.model.layers.4.linear_attn.in_proj_z",
+    "language_model.model.layers.4.linear_attn.out_proj",
+    "language_model.model.layers.4.mlp.shared_expert.down_proj",
+    "language_model.model.layers.4.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.4.mlp.shared_expert.up_proj",
+    "language_model.model.layers.4.mlp.shared_expert_gate",
+    "language_model.model.layers.4.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.4.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.4.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.5.linear_attn.in_proj_a",
+    "language_model.model.layers.5.linear_attn.in_proj_b",
+    "language_model.model.layers.5.linear_attn.in_proj_qkv",
+    "language_model.model.layers.5.linear_attn.in_proj_z",
+    "language_model.model.layers.5.linear_attn.out_proj",
+    "language_model.model.layers.5.mlp.shared_expert.down_proj",
+    "language_model.model.layers.5.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.5.mlp.shared_expert.up_proj",
+    "language_model.model.layers.5.mlp.shared_expert_gate",
+    "language_model.model.layers.5.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.5.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.5.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.6.linear_attn.in_proj_a",
+    "language_model.model.layers.6.linear_attn.in_proj_b",
+    "language_model.model.layers.6.linear_attn.in_proj_qkv",
+    "language_model.model.layers.6.linear_attn.in_proj_z",
+    "language_model.model.layers.6.linear_attn.out_proj",
+    "language_model.model.layers.6.mlp.shared_expert.down_proj",
+    "language_model.model.layers.6.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.6.mlp.shared_expert.up_proj",
+    "language_model.model.layers.6.mlp.shared_expert_gate",
+    "language_model.model.layers.6.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.6.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.6.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.7.mlp.shared_expert.down_proj",
+    "language_model.model.layers.7.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.7.mlp.shared_expert.up_proj",
+    "language_model.model.layers.7.mlp.shared_expert_gate",
+    "language_model.model.layers.7.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.7.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.7.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.7.self_attn.k_proj",
+    "language_model.model.layers.7.self_attn.o_proj",
+    "language_model.model.layers.7.self_attn.q_proj",
+    "language_model.model.layers.7.self_attn.v_proj",
+    "language_model.model.layers.8.linear_attn.in_proj_a",
+    "language_model.model.layers.8.linear_attn.in_proj_b",
+    "language_model.model.layers.8.linear_attn.in_proj_qkv",
+    "language_model.model.layers.8.linear_attn.in_proj_z",
+    "language_model.model.layers.8.linear_attn.out_proj",
+    "language_model.model.layers.8.mlp.shared_expert.down_proj",
+    "language_model.model.layers.8.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.8.mlp.shared_expert.up_proj",
+    "language_model.model.layers.8.mlp.shared_expert_gate",
+    "language_model.model.layers.8.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.8.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.8.mlp.switch_mlp.up_proj",
+    "language_model.model.layers.9.linear_attn.in_proj_a",
+    "language_model.model.layers.9.linear_attn.in_proj_b",
+    "language_model.model.layers.9.linear_attn.in_proj_qkv",
+    "language_model.model.layers.9.linear_attn.in_proj_z",
+    "language_model.model.layers.9.linear_attn.out_proj",
+    "language_model.model.layers.9.mlp.shared_expert.down_proj",
+    "language_model.model.layers.9.mlp.shared_expert.gate_proj",
+    "language_model.model.layers.9.mlp.shared_expert.up_proj",
+    "language_model.model.layers.9.mlp.shared_expert_gate",
+    "language_model.model.layers.9.mlp.switch_mlp.down_proj",
+    "language_model.model.layers.9.mlp.switch_mlp.gate_proj",
+    "language_model.model.layers.9.mlp.switch_mlp.up_proj"
+  ],
+  "missing": [
+    "language_model.lm_head",
+    "language_model.model.embed_tokens"
+  ],
+  "mismatched": [],
+  "zero_count_experts": 0
+}

preprocessor_config.json ADDED Viewed

	@@ -0,0 +1,21 @@

+{
+    "size": {
+        "longest_edge": 16777216,
+        "shortest_edge": 65536
+    },
+    "patch_size": 16,
+    "temporal_patch_size": 2,
+    "merge_size": 2,
+    "image_mean": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "image_std": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "processor_class": "Qwen3VLProcessor",
+    "image_processor_type": "Qwen2VLImageProcessorFast"
+}

processor_config.json ADDED Viewed

	@@ -0,0 +1,60 @@

+{
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_processor_type": "Qwen2VLImageProcessor",
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "merge_size": 2,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "size": {
+      "longest_edge": 16777216,
+      "shortest_edge": 65536
+    },
+    "temporal_patch_size": 2
+  },
+  "processor_class": "Qwen3VLProcessor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "fps": 2,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "max_frames": 768,
+    "merge_size": 2,
+    "min_frames": 4,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "size": {
+      "longest_edge": 25165824,
+      "shortest_edge": 4096
+    },
+    "temporal_patch_size": 2,
+    "video_processor_type": "Qwen3VLVideoProcessor"
+  }
+}

tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5f9e4d4901a92b997e463c1f46055088b6cca5ca61a6522d1b9f64c4bb81cb42
+size 12807982

tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,305 @@

+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "248044": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248045": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248046": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248047": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248048": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248049": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248050": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248051": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248052": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248053": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248054": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248055": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248056": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248057": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248058": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "248059": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "248060": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "248061": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "248062": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "248063": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "248064": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "248065": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "248066": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "248067": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "248068": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "248069": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "248070": {
+      "content": "<|audio_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248071": {
+      "content": "<|audio_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248072": {
+      "content": "<tts_pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248073": {
+      "content": "<tts_text_bos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248074": {
+      "content": "<tts_text_eod>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248075": {
+      "content": "<tts_text_bos_single>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "248076": {
+      "content": "<|audio_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- set template_version = \"qwen3.8-froggeric-v22.5.0\" %}\n{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}\n{%- set image_count = namespace(value=0) %}\n{%- set video_count = namespace(value=0) %}\n{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}\n{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}\n{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}\n{%- if preserve_reasoning is defined and preserve_reasoning is not none %}\n    {%- set _preserve_thinking = preserve_reasoning %}\n{%- elif preserve_thinking is defined and preserve_thinking is not none %}\n    {%- set _preserve_thinking = preserve_thinking %}\n{%- else %}\n    {%- set _preserve_thinking = true %}\n{%- endif %}\n{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}\n{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}\n{%- set _default_reasoning_effort = 'medium' %}\n{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}\n{%- set _effort_raw = (reasoning_effort | string | lower) if reasoning_effort is defined and reasoning_effort is not none else _default_reasoning_effort %}\n{%- set _initial_thinking = enable_thinking %}\n{%- if _effort_raw in ('none', 'off') %}\n    {%- set _initial_thinking = false %}\n    {%- set _initial_effort = 'medium' %}\n{%- elif _effort_raw in ('minimal', 'low') %}\n    {%- set _initial_effort = 'low' %}\n{%- elif _effort_raw in ('high', 'xhigh', 'max', 'ultracode', 'extreme') %}\n    {%- set _initial_effort = 'xhigh' %}\n{%- else %}\n    {%- set _initial_effort = 'medium' %}\n{%- endif %}\n{%- set ns_state = namespace(thinking=_initial_thinking, effort=_initial_effort) %}\n{%- if auto_disable_thinking_with_tools and _has_tools %}\n    {%- set ns_state.thinking = false %}\n{%- endif %}\n{%- for msg in messages %}\n    {%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}\n        {%- if msg.content is string %}\n            {%- if '<|think_off|>' in msg.content %}\n                {%- set ns_state.thinking = false %}\n            {%- elif '<|think_on|>' in msg.content %}\n                {%- set ns_state.thinking = true %}\n            {%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content or '<|think_ultracode|>' in msg.content or '<|think_extreme|>' in msg.content or '<|think_max|>' in msg.content %}\n                {%- set ns_state.thinking = true %}\n                {%- set ns_state.effort = 'xhigh' %}\n            {%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}\n                {%- set ns_state.thinking = true %}\n                {%- set ns_state.effort = 'low' %}\n            {%- elif '<|think_medium|>' in msg.content %}\n                {%- set ns_state.thinking = true %}\n                {%- set ns_state.effort = 'medium' %}\n            {%- endif %}\n        {%- elif msg.content is iterable and msg.content is not mapping %}\n            {%- for item in msg.content %}\n                {%- if item is string %}\n                    {%- set _item_text = item %}\n                {%- elif item is mapping and 'text' in item and item.text is string %}\n                    {%- set _item_text = item.text %}\n                {%- else %}\n                    {%- set _item_text = '' %}\n                {%- endif %}\n                {%- if _item_text %}\n                    {%- if '<|think_off|>' in _item_text %}\n                        {%- set ns_state.thinking = false %}\n                    {%- elif '<|think_on|>' in _item_text %}\n                        {%- set ns_state.thinking = true %}\n                    {%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text or '<|think_ultracode|>' in _item_text or '<|think_extreme|>' in _item_text or '<|think_max|>' in _item_text %}\n                        {%- set ns_state.thinking = true %}\n                        {%- set ns_state.effort = 'xhigh' %}\n                    {%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}\n                        {%- set ns_state.thinking = true %}\n                        {%- set ns_state.effort = 'low' %}\n                    {%- elif '<|think_medium|>' in _item_text %}\n                        {%- set ns_state.thinking = true %}\n                        {%- set ns_state.effort = 'medium' %}\n                    {%- endif %}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- set reasoning_instructions = '' %}\n{%- if ns_state.thinking %}\n    {%- if ns_state.effort == 'xhigh' %}\n        {%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}\n    {%- elif ns_state.effort == 'low' %}\n        {%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}\n    {%- endif %}\n{%- endif %}\n{%- macro render_content(content, do_vision_count, is_system_content=false) %}\n    {%- if content is string %}\n        {{- content }}\n    {%- elif content is iterable and content is not mapping %}\n        {%- for item in content %}\n            {%- if item is mapping %}\n                {%- if item.type == 'image' or 'image' in item or 'image_url' in item %}\n                    {%- if is_system_content %}\n                        {{- raise_exception('System message cannot contain images.') }}\n                    {%- endif %}\n                    {%- if do_vision_count %}\n                        {%- set image_count.value = image_count.value + 1 %}\n                    {%- endif %}\n                    {%- if add_vision_id %}\n                        {{- 'Picture ' ~ image_count.value ~ ': ' }}\n                    {%- endif %}\n                    {{- '<|vision_start|><|image_pad|><|vision_end|>' }}\n                {%- elif item.type == 'video' or 'video' in item or 'video_url' in item %}\n                    {%- if is_system_content %}\n                        {{- raise_exception('System message cannot contain videos.') }}\n                    {%- endif %}\n                    {%- if do_vision_count %}\n                        {%- set video_count.value = video_count.value + 1 %}\n                    {%- endif %}\n                    {%- if add_vision_id %}\n                        {{- 'Video ' ~ video_count.value ~ ': ' }}\n                    {%- endif %}\n                    {{- '<|vision_start|><|video_pad|><|vision_end|>' }}\n                {%- elif 'text' in item %}\n                    {{- item.text }}\n                {%- else %}\n                    {{- raise_exception('Unexpected item type in content.') }}\n                {%- endif %}\n            {%- else %}\n                {{- item | string }}\n            {%- endif %}\n        {%- endfor %}\n    {%- elif content is none or content is undefined %}\n        {{- '' }}\n    {%- else %}\n        {{- raise_exception('Unexpected content type.') }}\n    {%- endif %}\n{%- endmacro %}\n{%- if not messages %}\n    {{- raise_exception('No messages provided.') }}\n{%- endif %}\n{%- set head = namespace(count=0, seen_non_system=false) %}\n{%- for message in messages %}\n    {%- set _is_sys = (message.role == 'system' or message.role == 'developer') %}\n    {%- if _is_sys and not head.seen_non_system %}\n        {%- set head.count = head.count + 1 %}\n    {%- else %}\n        {%- set head.seen_non_system = true %}\n    {%- endif %}\n{%- endfor %}\n{%- set sys_state = namespace(content='') %}\n{%- for message in messages[:head.count] %}\n    {%- set _part = render_content(message.content, false, true) | trim %}\n    {%- if '<|think_off|>' in _part %}{%- set _part = _part.split('<|think_off|>') | join('') | trim %}{%- endif %}\n    {%- if '<|think_on|>' in _part %}{%- set _part = _part.split('<|think_on|>') | join('') | trim %}{%- endif %}\n    {%- if '<|think_xhigh|>' in _part %}{%- set _part = _part.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}\n    {%- if '<|think_high|>' in _part %}{%- set _part = _part.split('<|think_high|>') | join('') | trim %}{%- endif %}\n    {%- if '<|think_ultracode|>' in _part %}{%- set _part = _part.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}\n    {%- if '<|think_extreme|>' in _part %}{%- set _part = _part.split('<|think_extreme|>') | join('') | trim %}{%- endif %}\n    {%- if '<|think_max|>' in _part %}{%- set _part = _part.split('<|think_max|>') | join('') | trim %}{%- endif %}\n    {%- if '<|think_medium|>' in _part %}{%- set _part = _part.split('<|think_medium|>') | join('') | trim %}{%- endif %}\n    {%- if '<|think_low|>' in _part %}{%- set _part = _part.split('<|think_low|>') | join('') | trim %}{%- endif %}\n    {%- if '<|think_minimal|>' in _part %}{%- set _part = _part.split('<|think_minimal|>') | join('') | trim %}{%- endif %}\n    {%- if _part %}\n        {%- if sys_state.content %}\n            {%- set sys_state.content = sys_state.content ~ '\\n\\n' ~ _part %}\n        {%- else %}\n            {%- set sys_state.content = _part %}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- set _sc = sys_state.content %}\n{%- set _msgs = messages[head.count:] %}\n{%- if ns_state.thinking %}\n    {%- set _terse_lead = 'You are Cyber-Tiel-Coder, a variant of Ornith-1.5-35B-A3B. Answer directly, after thinking. Lead with the answer, then only what it needs to be correct and usable.' %}\n{%- else %}\n    {%- set _terse_lead = 'You are Cyber-Tiel-Coder, a variant of Ornith-1.5-35B-A3B. Answer directly and concisely. Give the answer with only what it needs to be correct and usable.' %}\n{%- endif %}\n{%- set _terse_core %}\nNever: open with preamble or pleasantries; restate the question; add filler transitions; hedge with niceties; or repeat a point you've already made.\nAlways: keep essential steps, caveats, uncertainties, and specifics — never drop correctness or a needed warning for brevity. Keep the final answer lean. Use the least structure that conveys it (plain prose when short; lists or code only when they earn their place). If genuinely uncertain, say so and explain why — never omit uncertainty for the sake of brevity.\nIf a user request is genuinely ambiguous, ask a sharp question, don't guess.\n{%- endset %}\n{%- set _terse = _terse_lead ~ '\\n' ~ (_terse_core | trim) %}\n{#- `terse` chat-template kwarg: default true, so behaviour is unchanged unless a caller opts out.\n    Pass {\"terse\": false} via chat_template_kwargs to serve the model with only its own system\n    prompt -- useful for A/B-ing the terseness effect, or when a downstream prompt conflicts. #}\n{%- set _terse_on = terse if terse is defined else true %}\n{%- if _terse_on %}\n    {%- if not _sc %}\n        {%- set _sc = _terse | trim %}\n    {%- else %}\n        {%- set _sc = (_sc | trim) ~ '\\n\\n' ~ (_terse | trim) %}\n    {%- endif %}\n{%- endif %}\n{#- Some runtimes (LM Studio's MLX backend) have no native tool-call parser and prepend their own\n    protocol into the system content. Emitting ours as well gives the model two contradictory tool\n    protocols and it follows one at random. Detect that marker and stand down; the\n    `suppress_tool_instructions` kwarg overrides either way where a runtime can pass template kwargs. #}\n{%- set _runtime_tool_protocol = (_sc is defined and _sc and '[TOOL_REQUEST]' in _sc) %}\n{%- set _suppress_tools = suppress_tool_instructions if suppress_tool_instructions is defined else _runtime_tool_protocol %}\n{%- if _has_tools and not _suppress_tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if reasoning_instructions %}\n        {{- reasoning_instructions + '\\n\\n' }}\n    {%- endif %}\n    {{- '# Tools\\n\\nYou have access to the following functions:\\n\\n<tools>' }}\n    {%- for tool in tools %}\n        {{- '\\n' }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- '\\n</tools>' }}\n    {%- if _tool_format == 'json' %}\n        {{- '\\n\\nIf you choose to call a function ONLY reply in the following format with NO suffix:\\n\\n' }}\n        {%- if ns_state.thinking %}\n            {{- '<think>\\nBrief explanation of tool call\\n</think>\\n' }}\n        {%- endif %}\n        {{- '<tool_call>\\n{\"name\": \"example_function_name\", \"arguments\": {\"example_parameter_1\": \"value_1\", \"example_parameter_2\": \"This is the value for the second parameter\"}}\\n</tool_call>\\n\\n<IMPORTANT>\\nReminder:\\n' }}\n        {%- if ns_state.thinking %}\n            {{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\\n' }}\n        {%- endif %}\n        {{- '- Function calls MUST follow the specified format: a single JSON object with \"name\" and \"arguments\" keys inside <tool_call></tool_call> XML tags.\\n' }}\n        {%- if ns_state.thinking %}\n            {{- '- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\\n' }}\n        {%- else %}\n            {{- '- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY, with NO conversational text before it.\\n' }}\n        {%- endif %}\n        {{- '- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\\n</IMPORTANT>' }}\n    {%- else %}\n        {{- '\\n\\nIf you choose to call a function ONLY reply in the following format with NO suffix:\\n\\n' }}\n        {%- if ns_state.thinking %}\n            {{- '<think>\\nBrief explanation of tool call\\n</think>\\n' }}\n        {%- endif %}\n        {{- '<tool_call>\\n<function=example_function_name>\\n<parameter=example_parameter_1>\\nvalue_1\\n</parameter>\\n<parameter=example_parameter_2>\\nThis is the value for the second parameter\\nthat can span\\nmultiple lines\\n</parameter>\\n</function>\\n</tool_call>\\n\\n<IMPORTANT>\\nReminder:\\n' }}\n        {%- if ns_state.thinking %}\n            {{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\\n' }}\n        {%- endif %}\n        {{- '- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\\n' }}\n        {%- if ns_state.thinking %}\n            {{- '- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\\n' }}\n        {%- else %}\n            {{- '- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY, with NO conversational text before it.\\n' }}\n        {%- endif %}\n        {{- '- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\\n</IMPORTANT>' }}\n    {%- endif %}\n    {%- if _sc %}\n        {{- '\\n\\n' + _sc }}\n    {%- endif %}\n    {{- '<|im_end|>\\n' }}\n{%- else %}\n    {%- if _sc %}\n        {{- '<|im_start|>system\\n' + (reasoning_instructions + '\\n\\n' if reasoning_instructions else '') + _sc + '<|im_end|>\\n' }}\n    {%- elif reasoning_instructions %}\n        {{- '<|im_start|>system\\n' + reasoning_instructions + '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- set _last_idx = _msgs | length - 1 %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}\n{%- for message in _msgs[::-1] %}\n    {%- set index = (_msgs | length - 1) - loop.index0 %}\n    {%- if ns.multi_step_tool and message.role == 'user' %}\n        {%- set _rc = render_content(message.content, false) | trim %}\n        {%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}\n            {%- set ns.multi_step_tool = false %}\n            {%- set ns.last_query_index = index %}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if ns.multi_step_tool %}\n    {%- if _last_idx > 50 %}\n        {%- set ns.last_query_index = _last_idx %}\n    {%- else %}\n        {%- set ns.last_query_index = 0 %}\n    {%- endif %}\n{%- endif %}\n{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}\n{%- for message in _msgs %}\n    {%- set is_system = (message.role == \"system\" or message.role == \"developer\") %}\n    {%- set content = render_content(message.content, true, is_system) | trim %}\n    {%- if is_system or message.role == 'user' %}\n        {%- if '<|think_off|>' in content %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- endif %}\n        {%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}\n        {%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}\n        {%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}\n        {%- if '<|think_ultracode|>' in content %}{%- set content = content.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}\n        {%- if '<|think_extreme|>' in content %}{%- set content = content.split('<|think_extreme|>') | join('') | trim %}{%- endif %}\n        {%- if '<|think_max|>' in content %}{%- set content = content.split('<|think_max|>') | join('') | trim %}{%- endif %}\n        {%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}\n        {%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}\n        {%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}\n    {%- endif %}\n    {%- if is_system %}\n        {{- '<|im_start|>system\\n' + content + '<|im_end|>\\n' }}\n    {%- elif message.role == 'user' %}\n        {%- set ns2.consecutive_failures = 0 %}\n        {{- '<|im_start|>user\\n' + content + '<|im_end|>\\n' }}\n    {%- elif message.role == 'assistant' %}\n        {%- set reasoning_content = '' %}\n        {%- set _explicit_reasoning = '' %}\n        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}\n            {%- if message.reasoning_content is string %}\n                {%- set _explicit_reasoning = message.reasoning_content %}\n            {%- else %}\n                {%- set _explicit_reasoning = message.reasoning_content | string %}\n            {%- endif %}\n        {%- elif message.thinking is defined and message.thinking is not none %}\n            {%- if message.thinking is string %}\n                {%- set _explicit_reasoning = message.thinking %}\n            {%- else %}\n                {%- set _explicit_reasoning = message.thinking | string %}\n            {%- endif %}\n        {%- elif message.reasoning is defined and message.reasoning is not none %}\n            {%- if message.reasoning is string %}\n                {%- set _explicit_reasoning = message.reasoning %}\n            {%- else %}\n                {%- set _explicit_reasoning = message.reasoning | string %}\n            {%- endif %}\n        {%- endif %}\n        {%- if _explicit_reasoning %}\n            {%- set _lead_end = '' %}\n            {%- if content.startswith('<think>') and '</think>' in content %}\n                {%- set _lead_end = '</think>' %}\n            {%- elif content.startswith('<thinking>') and '</thinking>' in content %}\n                {%- set _lead_end = '</thinking>' %}\n            {%- elif content.startswith('</think>') %}\n                {%- set _lead_end = '</think>' %}\n            {%- elif content.startswith('</thinking>') %}\n                {%- set _lead_end = '</thinking>' %}\n            {%- endif %}\n            {%- if _lead_end %}\n                {%- set content = content.split(_lead_end)[-1].lstrip('\\n') %}\n            {%- endif %}\n            {%- set reasoning_content = _explicit_reasoning %}\n        {%- else %}\n            {%- set _think_end = '' %}\n            {%- if content.startswith('</think>') %}\n                {%- set _think_end = '</think>' %}\n            {%- elif content.startswith('</thinking>') %}\n                {%- set _think_end = '</thinking>' %}\n            {%- elif '\\n</think>' in content %}\n                {%- set _think_end = '\\n</think>' %}\n            {%- elif '\\n</thinking>' in content %}\n                {%- set _think_end = '\\n</thinking>' %}\n            {%- elif '\\n</ think>' in content %}\n                {%- set _think_end = '\\n</ think>' %}\n            {%- elif '\\n</think >' in content %}\n                {%- set _think_end = '\\n</think >' %}\n            {%- elif content.startswith('<think>') and '</think>' in content %}\n                {%- set _think_end = '</think>' %}\n            {%- elif content.startswith('<thinking>') and '</thinking>' in content %}\n                {%- set _think_end = '</thinking>' %}\n            {%- endif %}\n            {%- if _think_end %}\n                {%- if 'thinking' in _think_end %}\n                    {%- set _think_start = '<thinking>' %}\n                {%- else %}\n                    {%- set _think_start = '<think>' %}\n                {%- endif %}\n                {%- set reasoning_content = content.split(_think_end)[0].rstrip('\\n') %}\n                {%- if _think_start in reasoning_content %}\n                    {%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\\n') %}\n                {%- endif %}\n                {%- set content = content.split(_think_end)[-1].lstrip('\\n') %}\n            {%- endif %}\n        {%- endif %}\n        {%- set reasoning_content = reasoning_content | trim %}\n        {%- if (_preserve_thinking or loop.index0 > ns.last_query_index) %}\n            {{- '<|im_start|>assistant\\n<think>\\n' + reasoning_content + '\\n</think>\\n\\n' + content }}\n        {%- else %}\n            {{- '<|im_start|>assistant\\n' + content }}\n        {%- endif %}\n        {%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if tool_call.function is defined and tool_call.function is not none %}\n                    {%- set tc = tool_call.function %}\n                {%- else %}\n                    {%- set tc = tool_call %}\n                {%- endif %}\n                {%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}\n                {%- if _tool_format == 'json' %}\n                    {%- if loop.first %}\n                        {%- if content | trim %}\n                            {{- '\\n\\n' }}\n                        {%- endif %}\n                    {%- else %}\n                        {{- '\\n' }}\n                    {%- endif %}\n                    {%- set _args = '{}' %}\n                    {%- if tc.arguments is defined and tc.arguments is not none %}\n                        {%- if tc.arguments is mapping %}\n                            {%- set _args = tc.arguments | tojson %}\n                        {%- elif tc.arguments is string %}\n                            {%- if tc.arguments %}\n                                {%- set _args = tc.arguments %}\n                            {%- endif %}\n                        {%- else %}\n                            {%- set _args = tc.arguments | tojson %}\n                        {%- endif %}\n                    {%- endif %}\n                    {{- '<tool_call>\\n{\"name\": ' }}{{- tc_name | tojson }}{{- ', \"arguments\": ' }}{{- _args }}{{- '}\\n</tool_call>' }}\n                {%- else %}\n                    {%- if loop.first %}\n                        {%- if content | trim %}\n                            {{- '\\n\\n<tool_call>\\n<function=' + tc_name + '>\\n' }}\n                        {%- else %}\n                            {{- '<tool_call>\\n<function=' + tc_name + '>\\n' }}\n                        {%- endif %}\n                    {%- else %}\n                        {{- '\\n<tool_call>\\n<function=' + tc_name + '>\\n' }}\n                    {%- endif %}\n                    {%- if tc.arguments is defined and tc.arguments is not none %}\n                        {%- if tc.arguments is mapping %}\n                            {%- for args_name, args_value in tc.arguments.items() %}\n                                {{- '<parameter=' + args_name + '>\\n' }}\n                                {%- if args_value is string %}\n                                    {%- set _av = args_value %}\n                                {%- else %}\n                                    {%- set _av = args_value | tojson %}\n                                {%- endif %}\n                                {%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}\n                                    {{- _av[:max_tool_arg_chars] + '\\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}\n                                {%- else %}\n                                    {{- _av }}\n                                {%- endif %}\n                                {{- '\\n</parameter>\\n' }}\n                            {%- endfor %}\n                        {%- else %}\n                            {%- if tc.arguments is string %}\n                                {%- set _raw_args = tc.arguments %}\n                            {%- else %}\n                                {%- set _raw_args = tc.arguments | tojson %}\n                            {%- endif %}\n                            {%- if _raw_args %}\n                                {%- if max_tool_arg_chars > 0 and _raw_args | length > max_tool_arg_chars %}\n                                    {{- _raw_args[:max_tool_arg_chars] + '\\n[TRUNCATED - original length ' ~ (_raw_args | length | string) ~ ' chars]' }}\n                                {%- else %}\n                                    {{- _raw_args }}\n                                {%- endif %}\n                            {%- endif %}\n                        {%- endif %}\n                    {%- endif %}\n                    {{- '</function>\\n</tool_call>' }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == 'tool' %}\n        {%- set _content_lower = content | lower %}\n        {%- set _content_head = _content_lower[:120] %}\n        {%- set _is_code_or_grep = ('throw new ' in _content_lower or 'throw error' in _content_lower or 'console.error' in _content_lower or 'logger.error' in _content_lower or 'logging.error' in _content_lower or 'import ' in _content_head or 'def ' in _content_head or 'function ' in _content_head) %}\n        {%- set _exit_code_zero = ('exit code: 0' in _content_head or 'process exited with code 0' in _content_head) %}\n        {%- set _error_field_ok = ('\"error\": null' in _content_head or '\"error\":null' in _content_head or '\"error\": false' in _content_head or '\"error\":false' in _content_head or '\"error\": \"\"' in _content_head or '\"error\":\"\"' in _content_head) %}\n        {%- set _strong_error = (('\"error\":' in _content_head and not _error_field_ok) or '\"status\": \"error\"' in _content_head or '\"status\":\"error\"' in _content_head or 'traceback (most recent call last):' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'fatal:' in _content_head or (('exit code: ' in _content_head or 'process exited with code' in _content_head) and not _exit_code_zero) or _content_head.startswith('exception:') or _content_head.startswith('failed to ')) %}\n        {%- set _weak_error = ('error:' in _content_head or 'err!' in _content_head) %}\n        {%- set _weak_suppressed = ('$ ' in _content_head or 'took ' in _content_head or content | length >= 600) %}\n        {%- if not _is_code_or_grep and (_strong_error or (_weak_error and not _weak_suppressed)) %}\n            {%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}\n        {%- else %}\n            {%- set ns2.consecutive_failures = 0 %}\n        {%- endif %}\n        {%- if ns2.prev_role != 'tool' %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {%- set _is_json_payload = (_tool_format == 'json' and content | trim | length > 0 and (content | trim)[:1] in ('{', '[')) %}\n        {%- if not _is_json_payload and max_tool_response_chars > 0 and content | length > max_tool_response_chars %}\n            {%- set content = content[:max_tool_response_chars] + '\\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' + content }}\n        {%- if ns2.consecutive_failures >= 2 %}\n            {{- '\\n\\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}\n        {%- elif ns2.consecutive_failures == 1 %}\n            {{- '\\n\\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}\n        {%- endif %}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last %}\n            {{- '<|im_end|>\\n' }}\n        {%- else %}\n            {%- set _next_role = _msgs[loop.index0 + 1].role %}\n            {%- if _next_role != 'tool' %}\n                {{- '<|im_end|>\\n' }}\n            {%- endif %}\n        {%- endif %}\n    {%- else %}\n        {{- '<|im_start|>user\\n[' + message.role + ']: ' + content + '<|im_end|>\\n' }}\n    {%- endif %}\n    {%- set ns2.prev_role = message.role %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n    {%- if not ns_state.thinking %}\n        {{- '<think>\\n\\n</think>\\n\\n' }}\n    {%- else %}\n        {{- '<think>\\n' }}\n    {%- endif %}\n{%- endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 262144,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "add_bos_token": false,
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "extra_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  }
+}

vocab.json ADDED Viewed

The diff for this file is too large to render. See raw diff