Release NeoHorse-Jev-4B

Co-authored-by: AlexDL <AlexDL@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,38 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/jev-snake-demo.gif filter=lfs diff=lfs merge=lfs -text
+tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+assets/jev-six-demo-grid.gif filter=lfs diff=lfs merge=lfs -text

DEPLOYMENT.md

@@ -0,0 +1,398 @@
+# Deployment and API Guide
+
+This guide covers installation and text and image inference with the `neohorse_decision` runtime. See the [README](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B) for the model overview and quickstart, or the [backend guide](https://github.com/TokenRhythm/NeoHorse/blob/main/jev/infer/README.md) for the separate vLLM and SGLang adapters.
+
+## 1. Installation
+
+Download the complete release from the [Hugging Face model repository](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B/tree/main) and install its bundled runtime. Loading the model requires `backbone/`, `tokenizer/`, `pointer_head.safetensors`, and `model_manifest.json`. Image examples are also included in the release.
+
+The recorded environment is Linux, Python 3.12, PyTorch 2.8.0, Transformers 5.17.0, Triton 3.7.1, and flash-linear-attention 0.5.2, with a CUDA GPU that supports BF16. Version details are in [environment.json](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B/blob/main/environment.json). The following commands assume these ML dependencies are already installed in an isolated environment:
+
+```bash
+export MODEL_DIR="/path/to/NeoHorse-Jev-4B"
+cd "$MODEL_DIR"
+
+python -m pip install --no-deps dist/neohorse_decision-1.0.0-py3-none-any.whl
+python -m pip install 'fastapi==0.141.1' 'uvicorn==0.53.0' 'starlette==1.6.0' 'httpx==0.28.1' 'pillow==12.3.0'
+```
+
+`--no-deps` does not install ML dependencies such as Torch. When installing or upgrading Torch, check whether dependency resolution changes the Triton version. Avoid mixing in Transformers or TorchVision packages from other environments.
+
+`backbone/` contains both language and vision parameters and occupies approximately 9.08 GB. The separate decision head occupies approximately 5.25 MB. Actual GPU memory usage also depends on the input and runtime settings. Load the complete directory with the matching runtime.
+
+### Install from Bundled Source
+
+With the native ML dependencies above already installed, run this from the root of the downloaded model bundle:
+
+```bash
+python -m pip install --no-deps ./package
+```
+
+`MODEL_DIR` still points to the complete model bundle downloaded from Hugging Face or ModelScope. The source installation replaces the wheel installation step above.
+
+### Model Composition
+
+The unified multimodal backbone contains the language model, vision encoder, and merger (`Qwen3_5Model`, with `language_model` and `visual` components). Backbone weights use BF16; the separate pointer head uses FP32. Keep weights, tokenizer, configuration, and runtime from the same release together. Release provenance is recorded in [model_manifest.json](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B/blob/main/model_manifest.json).
+
+## 2. Local Text Inference
+
+The CLI can read the example request included in the model release:
+
+```bash
+CUDA_VISIBLE_DEVICES=0 neohorse-decision predict \
+  --model-dir "$MODEL_DIR" --request "$MODEL_DIR/example_request.json"
+```
+
+Change the GPU index to match your allocation. The runtime does not schedule GPU resources across a cluster.
+
+### Python Decision Examples
+
+**Provide a state and get a yes/no probability, a selected candidate, or a rating.** The examples below use the same user message to demonstrate the three decision modes.
+
+Load the model once, then reuse `engine` and `state`:
+
+```python
+import os
+
+from neohorse_decision import DecisionEngine
+
+engine = DecisionEngine(os.environ["MODEL_DIR"])
+state = "I was charged twice for the same order. Please refund the extra charge today."
+```
+
+All output numbers below are illustrative, not measured results. Actual values depend on the model's predictions.
+
+#### Noul: Is It True?
+
+**Is the user requesting a refund?** Return the probability of "yes", `P(true)`.
+
+```python
+result = engine.predict({
+    "state": state,
+    "questions": {
+        "refund": {
+            "type": "noul",
+            "instructions": "Is the user requesting a refund?",
+        },
+    },
+})
+print(result["answers"]["refund"]["noul"])
+```
+
+Illustrative output: `0.97` means the model assigns a 97% probability to the user requesting a refund. Your application can use this to enter a refund workflow.
+
+#### Choice: Which One?
+
+**Which team should handle this message?** Select from the candidates and return each candidate's probability.
+
+```python
+result = engine.predict({
+    "state": state,
+    "questions": {
+        "team": {
+            "type": "choice",
+            "instructions": "Which team should handle this message?",
+            "criteria": {
+                "billing": "Billing, charges, or refunds",
+                "technical": "Product failures or technical issues",
+                "other": "Other matters",
+            },
+        },
+    },
+})
+print(result["answers"]["team"]["choice"])
+print(result["answers"]["team"]["probabilities"])
+```
+
+Illustrative output:
+
+```text
+billing
+{'billing': 0.96, 'technical': 0.03, 'other': 0.01}
+```
+
+Read `billing` to route the message to the billing team.
+
+#### Score: To What Degree?
+
+**How urgent is the request?** Rate it against the levels you define. Levels start at `0`, and the result is their probability-weighted expected value.
+
+```python
+result = engine.predict({
+    "state": state,
+    "questions": {
+        "urgency": {
+            "type": "score",
+            "instructions": "How soon does the user want this resolved?",
+            "criteria": ["Can wait", "This week", "Today"],
+        },
+    },
+})
+print(result["answers"]["urgency"]["score"])
+```
+
+Illustrative output: `1.9` is close to level `2` ("Today"), which your application can use to raise the request's priority.
+
+Save the four Python blocks above, in order, as `quickstart.py`, then run:
+
+```bash
+CUDA_VISIBLE_DEVICES=0 python quickstart.py
+```
+
+To make all three decisions in one text request, place `refund`, `team`, and `urgency` in the same `questions` dictionary. One request returns three answers. Set decision thresholds using data from your own tasks.
+
+
+## 3. Start the HTTP Service
+
+```bash
+CUDA_VISIBLE_DEVICES=0 neohorse-decision serve --model-dir "$MODEL_DIR" --port 8080
+```
+
+The service binds to `127.0.0.1` by default. Check readiness from another terminal:
+
+```bash
+curl -sS http://127.0.0.1:8080/health
+```
+
+| Endpoint | Purpose |
+| --- | --- |
+| `POST /v1/decision` | Native decision API |
+| `POST /v1/systemone` | Response structure containing `model`, `answers`, and `usage` |
+| `GET /health` | Readiness status and `input_modalities` |
+
+To enable authentication, set `NEOHORSE_API_KEY` in the service environment before starting it. Clients send `Authorization: Bearer <API_KEY>`. For external access, use a TLS gateway and limit concurrency and request body sizes. Only the endpoints and protocol scope described here are supported; `/v1/models` is not provided.
+
+## 4. Text Requests
+
+A request contains `model`, `state`, and `questions`. Each question has an application-defined key, a `type`, `instructions`, and `criteria` where required.
+
+| Type | `criteria` | Meaning |
+| --- | --- | --- |
+| `noul` | Optional | Determine whether the condition is true |
+| `choice` | Dictionary of candidate keys and descriptions | Select from the candidates |
+| `score` | List of rating descriptions ordered from lowest to highest | Compute the rating distribution and expected value |
+
+This request makes three decisions about the same user message:
+
+```bash
+curl -sS http://127.0.0.1:8080/v1/systemone \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "model": "NeoHorse-Jev-4B",
+    "state": "I was charged twice for the same order. Please refund the extra charge today.",
+    "questions": {
+      "refund": {"type": "noul", "instructions": "Is the user requesting a refund?"},
+      "team": {
+        "type": "choice",
+        "instructions": "Which team should handle this message?",
+        "criteria": {"billing": "Billing, charges, or refunds", "technical": "Product failures or technical issues", "other": "Other matters"}
+      },
+      "urgency": {
+        "type": "score",
+        "instructions": "How soon does the user want this resolved?",
+        "criteria": ["Can wait", "This week", "Today"]
+      }
+    }
+  }'
+```
+
+This example assumes a local service without authentication. If authentication is enabled, add `-H "Authorization: Bearer $NEOHORSE_API_KEY"`.
+
+Use `NeoHorse-Jev-4B` or `TokenRhythm/NeoHorse-Jev-4B` as the model name. Both HTTP endpoints also accept `neohorse-jev`, `NeoHorse-JEV-4B`, and `TokenRhythm/NeoHorse-JEV-4B`. Responses use the canonical name `NeoHorse-Jev-4B`.
+
+## 5. Responses
+
+Each question's result is available at `answers.<question_key>`:
+
+| Type | Main fields |
+| --- | --- |
+| Choice | `type`, `choice`, `probabilities`, `confidence` |
+| Noul | `type`, `noul`; the native API also retains yes/no `probabilities` |
+| Score | `type`, `score`, `legend`, `probabilities`, `confidence` |
+
+`noul` is the probability that the condition is true. Score levels are indexed from `0`; `score` is their probability-weighted expected value and can be fractional. `legend` maps level indices to their descriptions.
+
+| Endpoint | Top-level structure and token counts |
+| --- | --- |
+| `/v1/decision` | `model`, `answers`, `input_tokens`; image requests also return `image_tokens` |
+| `/v1/systemone` | `model`, `answers`, `usage`; token counts are in `usage.input_tokens` and `usage.output_tokens`, with `usage.image_tokens` for image requests |
+
+Both HTTP endpoints use the same weights, encoding, and probabilities. The text Python interface, `DecisionEngine.predict`, does not currently return `confidence`.
+
+`input_tokens` counts encoded input tokens. The shared state in a text request with multiple questions is counted once, so this is not the total number of tokens processed by the GPU after expanding the questions into separate branches. `output_tokens` is the response JSON's local tokenizer count; it does not indicate autoregressive generation. For image requests, `input_tokens` already includes image tokens and vision start/end markers. Do not add `image_tokens` again. Text responses do not include an `image_tokens` field.
+
+### Understanding confidence
+
+`confidence` is a local distribution statistic, not a calibrated probability of correctness:
+
+- Choice: `(max(p) - 1/K) / (1 - 1/K)`, where `K` is the number of candidates. A single candidate returns `1`.
+- Score: `1 - sum_i p_i * abs(i - argmax(p)) / (L - 1)`, where `L` is the number of rating levels.
+
+Values are clamped to `[0, 1]`. Validate application thresholds on independent data. Both endpoints return `X-NeoHorse-Confidence: local-distribution-statistic-v1`. The System One-style endpoint also returns `X-NeoHorse-Usage: local-tokenizer-not-jev-billing`.
+
+## 6. Image Requests
+
+Both HTTP endpoints accept an optional top-level `image` field containing a base64 data URL for a PNG, JPEG, or WebP image. Each image request supports one static image and one Noul, Choice, or Score question. Omit `image` when no image is provided; do not send `null`. External URLs and server file paths are not accepted.
+
+### Python Image Decisions
+
+**Provide a page screenshot and a task goal to determine whether the task succeeded, what state the page is in, or how far the task has progressed.** The image and text jointly inform the decision, with Noul, Choice, or Score outputs.
+
+This is a standalone image example. Save your screenshot as `screenshot.png`, then save the four Python blocks in this section, in order, as `multimodal_quickstart.py`. Load the model once and reuse the same image for all three calls, with **one question per request**. The output values below are illustrative, not measured results.
+
+```python
+import os
+
+from PIL import Image
+from neohorse_decision.vision import VisionDecisionEngine
+
+vision_engine = VisionDecisionEngine(os.environ["MODEL_DIR"])
+with Image.open("screenshot.png") as source:
+    screenshot = source.convert("RGB")
+state = "Goal: submit the form. Assess the current page screenshot."
+```
+
+#### Noul: Was the Form Submitted Successfully?
+
+```python
+result = vision_engine.predict({
+    "model": "NeoHorse-Jev-4B",
+    "state": state,
+    "questions": {
+        "submitted": {
+            "type": "noul",
+            "instructions": "Does the screenshot clearly show that the form was submitted successfully?",
+        },
+    },
+}, screenshot)
+print(result["answers"]["submitted"]["noul"])
+```
+
+For example, `0.97` means the model assigns a 97% probability to the screenshot showing a successful submission. Your workflow can use this to decide whether to move to the next step.
+
+#### Choice: What State Is the Page In?
+
+```python
+result = vision_engine.predict({
+    "model": "NeoHorse-Jev-4B",
+    "state": state,
+    "questions": {
+        "page_status": {
+            "type": "choice",
+            "instructions": "Which page state does the screenshot show?",
+            "criteria": {
+                "success": "Submission succeeded",
+                "error": "Submission failed or an error is shown",
+                "processing": "Submission or loading is in progress",
+                "unknown": "Cannot determine the submission status from the screenshot",
+            },
+        },
+    },
+}, screenshot)
+print(result["answers"]["page_status"]["choice"])
+print(result["answers"]["page_status"]["probabilities"])
+```
+
+For example, the result may be `success` alongside each candidate's probability. Route the next step according to the selected state.
+
+#### Score: How Far Has the Task Progressed?
+
+```python
+result = vision_engine.predict({
+    "model": "NeoHorse-Jev-4B",
+    "state": state,
+    "questions": {
+        "completion": {
+            "type": "score",
+            "instructions": "How far has the form submission task progressed, based on the screenshot?",
+            "criteria": ["Submission has not started", "Submission is in progress", "Submission clearly succeeded"],
+        },
+    },
+}, screenshot)
+print(result["answers"]["completion"]["score"])
+```
+
+For example, `1.9` is close to level `2` ("Submission clearly succeeded"). Actual results depend on the input image.
+
+```bash
+CUDA_VISIBLE_DEVICES=0 python multimodal_quickstart.py
+```
+
+Image requests currently support **one static image + text + one question**. The HTTP image formats are PNG, JPEG, and WebP. Multiple images, video, and audio are not supported.
+
+For local image inference, run:
+
+```bash
+CUDA_VISIBLE_DEVICES=0 python "$MODEL_DIR/vision/example.py" \
+  --model-dir "$MODEL_DIR" --image /path/to/image.png \
+  --request "$MODEL_DIR/vision/example_request.json"
+```
+
+The Python interface is `neohorse_decision.vision.VisionDecisionEngine`, called as `engine.predict(request, pil_image)`. HTTP text and image requests share the same backbone, decision head, and GPU lock, without dynamic batching. The result is a structured decision distribution. Video and multiple-image interfaces are not provided. For the separate vLLM and SGLang adapters, including text and single-image requests, see the [backend deployment guide](https://github.com/TokenRhythm/NeoHorse/blob/main/jev/infer/README.md).
+
+### HTTP Image Requests
+
+For the same question about whether the screenshot shows a successful submission, save this as `screenshot_request.json`:
+
+```json
+{
+  "model": "NeoHorse-Jev-4B",
+  "state": "Goal: submit the form. Assess the current page screenshot.",
+  "questions": {
+    "submitted": {
+      "type": "noul",
+      "instructions": "Does the screenshot clearly show that the form was submitted successfully?"
+    }
+  }
+}
+```
+
+Once the service is running, use the bundled client to read the local screenshot and send the request:
+
+```bash
+python "$MODEL_DIR/vision/http_example.py" \
+  --image screenshot.png \
+  --request screenshot_request.json \
+  --base-url http://127.0.0.1:8080 \
+  --endpoint systemone
+```
+
+The client encodes the image as a base64 data URL in the top-level `image` field. In the response, `answers.submitted.noul` is the probability of a successful submission. Set `--endpoint` to `decision` to use the native endpoint. When authentication is enabled, the client reads the key from the `NEOHORSE_API_KEY` environment variable.
+
+If `--request` is omitted, the bundled client asks for the image's dominant color by default.
+
+## 7. Request Limits and Error Handling
+
+| Item | Default limit |
+| --- | --- |
+| Text `state` | 2,048 tokens |
+| Each question branch | 8,192 tokens |
+| Questions per text request | 16 |
+| Total tokens after expanding a text request into question branches | 32,768 |
+| Text HTTP request body; JSON fields other than `image` in an image request | 1 MiB |
+| Image HTTP request body | 8 MiB |
+| Image file after base64 decoding | 4 MiB |
+| Image pixel count | 4,194,304 |
+| Minimum/maximum image preprocessing area budget | 65,536 / 1,048,576 pixels |
+| Image tokens | 1,024 |
+| Total encoded length of an image request | 12,288 tokens |
+| Score levels on `/v1/systemone` | 2–10 |
+
+Inputs that exceed these limits are rejected without silent truncation. These are deployment protection limits. Check task performance and GPU memory usage before changing them.
+
+| HTTP status | Meaning and action |
+| --- | --- |
+| `401` | Authentication failed; check the Bearer token |
+| `413` | The request body or text fields exceed size limits |
+| `422` | Invalid JSON or fields, unknown model, or unsupported image format, dimensions, token count, multiple questions, animation, or other input constraint violations |
+| `429` | The GPU worker for native `/v1/decision` is busy |
+| `529` | The GPU worker for `/v1/systemone` is busy |
+
+Busy responses include `Retry-After: 1`. Clients should back off and retry. There is no separate quota-based rate limiter; control high concurrency at the gateway and test it for your deployment.
+
+## 8. Limitations
+
+- **Decisions can be wrong.** Valid structure and normalized probabilities do not guarantee correct judgments. Missing evidence, candidate descriptions, candidate order, and domain shifts can all affect results.
+- **Validate probabilities for your application.** NLL, Brier, and ECE calibration results have not been reported. Set thresholds on an independent dataset.
+- **Scope claims to measured evidence.** Comprehensive evaluations of multilingual inputs, long inputs, and computational isolation between questions are not yet available. Multiple questions in one request do not imply a single shared forward pass.
+- **Applications enforce execution constraints.** Tool permissions, business rules, and action validation remain the application's responsibility. The current materials do not provide latency, GPU memory, or cost comparisons under a common timing protocol.

README.md

@@ -0,0 +1,584 @@
+---
+license: apache-2.0
+pipeline_tag: text-classification
+library_name: pytorch
+base_model:
+  - TokenRhythm/NeoHorse-1-4B
+base_model_relation: finetune
+tags:
+  - agentic
+  - decision-model
+  - typed-decisions
+  - structured-prediction
+  - non-generative
+  - multimodal
+  - vision-language
+---
+
+<div align="center">
+  <h1>NeoHorse-Jev-4B</h1>
+  <p><b>Prefill-only decisions for agent workflows.</b></p>
+</div>
+
+<div align="center">
+  <a href="https://github.com/TokenRhythm/NeoHorse"><img alt="GitHub" src="https://img.shields.io/badge/GitHub-NeoHorse-181717?logo=github&logoColor=white"></a>
+  <a href="https://huggingface.co/collections/TokenRhythm/neohorse-jev"><img alt="Hugging Face" src="https://img.shields.io/badge/Hugging%20Face-Models-FFD21E?logo=huggingface&logoColor=000000"></a>
+  <a href="https://www.modelscope.cn/models/TokenRhythm/NeoHorse-Jev-4B"><img alt="ModelScope" src="https://img.shields.io/badge/ModelScope-Models-624AFF?logo=modelscope&logoColor=white"></a>
+  <a href="https://tokenrhythm.ai/"><img alt="Company" src="https://img.shields.io/badge/Company-TokenRhythm-F97316?logo=homeassistant&logoColor=white"></a>
+  <a href="https://x.com/opensquilla"><img alt="Twitter / X" src="https://img.shields.io/badge/Twitter%20%2F%20X-OpenSquilla-111827?logo=x&logoColor=white"></a>
+  <a href="https://github.com/TokenRhythm/NeoHorse/blob/main/jev/LICENSE"><img alt="License: Apache-2.0" src="https://img.shields.io/badge/License-Apache--2.0-64748B"></a>
+</div>
+
+<div align="center"><a href="https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B/blob/main/DEPLOYMENT.md">Deployment</a></div>
+
+## Introduction
+
+We introduce **NeoHorse-Jev-4B**, a **4B structured decision model** from TokenRhythm, built on [NeoHorse-1-4B](https://huggingface.co/TokenRhythm/NeoHorse-1-4B). Given a state and questions defined by your application, it predicts decisions and their probabilities for routing requests, selecting tools, checking conditions, and rating outcomes.
+
+The model uses **prefill-only inference** with three decision types: **Choice**, **Noul**, and **Score**. It predicts directly over the answers you define, without autoregressive text generation.
+
+**NeoHorse-Jev-4B scores 77.70 on the six-group text aggregate below, the highest among the four open-weight decision models with complete results in this comparison.** It also achieves **83.26% mean accuracy** across Nimble, VitaminC, and MASSIVE, **11.50 percentage points** above the NeoHorse-1-4B baseline.
+
+- **Application-defined decisions.** Define candidate actions, yes/no questions, or ordered rating levels. Text requests can include multiple questions.
+- **Probabilities for application logic.** Use candidate distributions, yes/no probabilities, and expected ratings to drive routing rules and thresholds.
+- **Local deployment.** Run with vLLM, SGLang, or the native Python, CLI, and HTTP runtime. Optional image requests combine a single image with text.
+
+## Decision Demos
+
+![NeoHorse-Jev-4B six-demo grid](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B/resolve/main/assets/jev-six-demo-grid.gif)
+
+**Six decision demos:** Tetris, Snake, robot manipulation, Mahjong, four-player bomb arena, and autonomous driving (left to right, top to bottom). Each panel preserves the original replay and decision displays and loops independently.
+
+## Evaluation
+
+Results updated **September 24, 2026**. These are our evaluations under the protocols described below. Text accuracy, image understanding, and interactive games are reported separately.
+
+### Text Decision Benchmarks
+
+All component scores are on a 0–100 scale; higher is better. NeoHorse-Jev-4B uses the **vLLM** results for JevBench, Kev, and OpenJev in this table; Nimble, VitaminC, and MASSIVE retain the original fixed-subset evaluation results.
+
+| Model | JevBench | Kev | OpenJev text | Nimble | VitaminC | MASSIVE | AVG |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| [Open-Jev-9B](https://huggingface.co/ZefanCai/Open-Jev-9B) | **77.13** | 77.87 | **65.39** | <ins>80.50</ins> | 68.28 | 84.86 | <ins>75.67</ins> |
+| [Kev-4B](https://huggingface.co/jaredpalmer/kev-4b) | 73.71 | <ins>81.47</ins> | 54.75 | 73.40 | 76.46 | **85.71** | 74.25 |
+| [Laya English](https://huggingface.co/convaiinnovations/laya) | 55.82 | 61.30 | 40.07 | 45.04 | **78.63** | 68.57 | 58.24 |
+| [Laya Typed Decisions](https://huggingface.co/convaiinnovations/laya-typed-decisions) | -- | -- | -- | 48.94 | <ins>78.30</ins> | 65.43 | -- |
+| **[NeoHorse-1-4B](https://huggingface.co/TokenRhythm/NeoHorse-1-4B)** | -- | -- | -- | 69.15 | 63.27 | 82.86 | -- |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)** | <ins>75.73</ins> | **81.92** | <ins>58.74</ins> | **87.23** | 77.13 | <ins>85.43</ins> | **77.70** |
+
+**Bold scores** mark the best result and <ins>underlined scores</ins> the second-best among the listed open-weight entries. `--` means no result is available. NeoHorse-1-4B is a base-model reference; missing groups are not filled with zeros or results from a different checkpoint.
+
+**AVG** is our equal-weight mean of the six displayed group scores, calculated before rounding the final aggregate. It is not pooled per-example accuracy or an official combined leaderboard. Only models with all six groups are ranked by this aggregate; images and games do not enter it.
+
+NeoHorse-Jev-4B leads the tested open-weight entries on **Kev (81.92)** and **Nimble (87.23)**. Open-Jev-9B scores higher on JevBench and OpenJev's static text tasks; Kev-4B scores slightly higher on MASSIVE, and the Laya checkpoints score higher on VitaminC. The aggregate advantage therefore reflects the balance across tasks, rather than a win on every benchmark.
+
+<details>
+<summary>Benchmark scope, sample counts, and aggregation</summary>
+
+| Benchmark group | Evaluated scope | Score used in the overview |
+| --- | --- | --- |
+| JevBench | Public set of 231 examples | Official family-macro score |
+| Kev | Development and test splits of decision-v7, transfer-v4, and transfer-v9; 6,436 input records in total | Equal-weight mean of the six clean-accuracy scores; a record may contain multiple decisions |
+| OpenJev text | Static text tasks: NLI, multiple-choice reranking, and fixed-candidate GSM8K | Equal-weight mean of 19 task scores; the two MNLI splits are averaged first |
+| Nimble | 282 examples selected from 324, keeping related case groups intact; 116 Choice, 112 Noul, 54 Score | Per-example exact decision accuracy, including exact rating-level matches |
+| VitaminC-dev | 599 examples from the upstream Nimble sampling pipeline | Three-way evidence/claim classification accuracy |
+| MASSIVE-en | 350 English test examples from the upstream Nimble sampling pipeline | Classification accuracy across 18 assistant scenarios; not intent/slot or multilingual evaluation |
+
+For Nimble, VitaminC, and MASSIVE, selected IDs and records were frozen before model comparison. Reference answers are used for scoring, not as model input. Upstream VitaminC/MASSIVE sampling uses seed `20260918` and complete case groups. Local selection uses a 384-token state limit, a 2,048-token packed decision limit, and at most 26 candidates; the base-model letter-logit prompt allows 4,096 tokens. These are subset-selection rules for these three datasets, not universal limits for all benchmarks or deployment. Nimble falls from 324 to 282 examples after length and whole-group filtering; the other two subsets pass unchanged.
+
+</details>
+
+<details>
+<summary>Nimble, VitaminC, and MASSIVE: three-benchmark means</summary>
+
+| Model | Three-benchmark mean accuracy (%) |
+| --- | ---: |
+| [Open-Jev-9B](https://huggingface.co/ZefanCai/Open-Jev-9B) | 77.88 |
+| [Kev-4B](https://huggingface.co/jaredpalmer/kev-4b) | <ins>78.53</ins> |
+| [Laya English](https://huggingface.co/convaiinnovations/laya) | 64.08 |
+| [Laya Typed Decisions](https://huggingface.co/convaiinnovations/laya-typed-decisions) | 64.22 |
+| **[NeoHorse-1-4B](https://huggingface.co/TokenRhythm/NeoHorse-1-4B)** | 71.76 |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)** | **83.26** |
+
+This mean weights Nimble, VitaminC, and MASSIVE equally, rather than pooling their 1,231 examples. The three-benchmark means retain the original evaluation report, which averages unrounded accuracies; recomputing from the two-decimal component scores can differ by 0.01. For example, Kev is reported as 78.53. The separately defined AVG above uses the six displayed group scores.
+
+</details>
+
+<details>
+<summary>Detailed text comparisons: JevBench, Kev, and OpenJev</summary>
+
+**JevBench.** NeoHorse-Jev-4B has 75.32% per-example accuracy and 100% valid output format on the public 231 examples. Its 75.73 family-macro score weights families, rather than individual examples.
+
+| Model | adequacy | adversarial | ambiguous | extraction | fact | intent |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| [Open-Jev-9B](https://huggingface.co/ZefanCai/Open-Jev-9B) | **83.33** | **100.00** | 42.86 | 91.67 | **100.00** | **100.00** |
+| [Kev-4B](https://huggingface.co/jaredpalmer/kev-4b) | 66.67 | **100.00** | <ins>57.14</ins> | <ins>95.83</ins> | **100.00** | <ins>95.83</ins> |
+| [Laya English](https://huggingface.co/convaiinnovations/laya) | 66.67 | <ins>50.00</ins> | 14.29 | 83.33 | <ins>83.33</ins> | 83.33 |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)** | <ins>75.00</ins> | **100.00** | **71.43** | **100.00** | **100.00** | **100.00** |
+
+| Model | judge_hard | long_policy | multi_hop | ordinal | policy | probability |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| [Open-Jev-9B](https://huggingface.co/ZefanCai/Open-Jev-9B) | **76.47** | **47.37** | **66.67** | **100.00** | **100.00** | **60.00** |
+| [Kev-4B](https://huggingface.co/jaredpalmer/kev-4b) | <ins>52.94</ins> | <ins>21.05</ins> | <ins>55.56</ins> | **100.00** | <ins>91.67</ins> | <ins>50.00</ins> |
+| [Laya English](https://huggingface.co/convaiinnovations/laya) | 41.18 | <ins>21.05</ins> | 33.33 | <ins>91.67</ins> | 83.33 | <ins>50.00</ins> |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)** | <ins>52.94</ins> | **47.37** | <ins>55.56</ins> | **100.00** | **100.00** | 40.00 |
+
+| Model | routing | routing_hard | temporal_numeric | tool_selection | tradeoff | trap |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| [Open-Jev-9B](https://huggingface.co/ZefanCai/Open-Jev-9B) | <ins>66.67</ins> | **100.00** | <ins>20.00</ins> | **100.00** | <ins>33.33</ins> | **100.00** |
+| [Kev-4B](https://huggingface.co/jaredpalmer/kev-4b) | **100.00** | **100.00** | 6.67 | **100.00** | <ins>33.33</ins> | **100.00** |
+| [Laya English](https://huggingface.co/convaiinnovations/laya) | 50.00 | <ins>20.00</ins> | **33.33** | **100.00** | **100.00** | 0.00 |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)** | **100.00** | **100.00** | 0.00 | **100.00** | <ins>33.33</ins> | <ins>87.50</ins> |
+
+Examples: adequacy: 12; adversarial: 6; ambiguous: 7; extraction: 24; fact: 12; intent: 24; judge_hard: 17; long_policy: 19; multi_hop: 18; ordinal: 12; policy: 12; probability: 10; routing: 12; routing_hard: 5; temporal_numeric: 15; tool_selection: 12; tradeoff: 6; trap: 8.
+
+**Kev.** Clean accuracy (%) by suite; record counts differ from decision counts.
+
+| Model | decision-v7 / development | decision-v7 / test | transfer-v4 / development | transfer-v4 / test | transfer-v9 / development | transfer-v9 / test |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| [Open-Jev-9B](https://huggingface.co/ZefanCai/Open-Jev-9B) | 80.30 | 77.00 | 77.44 | 83.54 | 72.47 | <ins>76.48</ins> |
+| [Kev-4B](https://huggingface.co/jaredpalmer/kev-4b) | **87.18** | **87.08** | <ins>79.73</ins> | <ins>83.69</ins> | <ins>74.76</ins> | 76.39 |
+| [Laya English](https://huggingface.co/convaiinnovations/laya) | 66.22 | 65.50 | 65.09 | 65.55 | 52.39 | 53.06 |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)** | <ins>86.23</ins> | <ins>86.58</ins> | **81.71** | **84.60** | **75.53** | **76.86** |
+
+Input records: decision-v7 / development: 1,204; decision-v7 / test: 1,176; transfer-v4 / development: 764; transfer-v4 / test: 764; transfer-v9 / development: 1,264; transfer-v9 / test: 1,264.
+
+**OpenJev static text.** NLI classification accuracy (%):
+
+| Model | scitail | anli_r1 | anli_r2 | anli_r3 | wanli | control |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| [Open-Jev-9B](https://huggingface.co/ZefanCai/Open-Jev-9B) | 79.16 | **74.00** | **66.30** | **59.42** | **67.10** | **67.58** |
+| [Kev-4B](https://huggingface.co/jaredpalmer/kev-4b) | <ins>84.81</ins> | 65.60 | 54.30 | 52.25 | 63.50 | 64.35 |
+| [Laya English](https://huggingface.co/convaiinnovations/laya) | 74.84 | 48.40 | 38.80 | 34.33 | 53.00 | 37.64 |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)** | **87.02** | <ins>67.20</ins> | <ins>56.20</ins> | <ins>53.42</ins> | <ins>65.74</ins> | <ins>65.96</ins> |
+
+| Model | MNLI / validation_matched | MNLI / validation_mismatched |
+| --- | ---: | ---: |
+| [Open-Jev-9B](https://huggingface.co/ZefanCai/Open-Jev-9B) | 80.64 | 80.54 |
+| [Kev-4B](https://huggingface.co/jaredpalmer/kev-4b) | **89.17** | <ins>89.35</ins> |
+| [Laya English](https://huggingface.co/convaiinnovations/laya) | 63.28 | 64.27 |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)** | <ins>88.95</ins> | **89.46** |
+
+Examples: scitail: 2,126; anli_r1: 1,000; anli_r2: 1,000; anli_r3: 1,200; wanli: 5,000; control: 805; MNLI / validation_matched: 9,815; MNLI / validation_mismatched: 9,832.
+
+Multiple-choice rerank accuracy:
+
+| Model | arc_easy | arc_challenge | winogrande | gsm8k_mc4 | gsm8k_mc10 | gpqa |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| [Open-Jev-9B](https://huggingface.co/ZefanCai/Open-Jev-9B) | **95.71** | **87.29** | **66.30** | **52.69** | **35.71** | **37.37** |
+| [Kev-4B](https://huggingface.co/jaredpalmer/kev-4b) | 75.42 | 66.89 | 58.33 | 38.59 | 20.77 | 33.84 |
+| [Laya English](https://huggingface.co/convaiinnovations/laya) | 40.19 | 33.36 | 49.64 | 24.26 | 8.49 | 26.77 |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)** | <ins>88.93</ins> | <ins>78.50</ins> | <ins>60.69</ins> | <ins>41.77</ins> | <ins>21.83</ins> | <ins>34.34</ins> |
+
+| Model | gpqa_fewshot | chess | hellaswag | mmlu | mmlu_fewshot |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| [Open-Jev-9B](https://huggingface.co/ZefanCai/Open-Jev-9B) | **39.90** | **52.40** | **54.09** | **66.80** | **65.05** |
+| [Kev-4B](https://huggingface.co/jaredpalmer/kev-4b) | <ins>34.85</ins> | 19.20 | 18.92 | 52.29 | 57.50 |
+| [Laya English](https://huggingface.co/convaiinnovations/laya) | 22.22 | <ins>29.60</ins> | 28.44 | 29.96 | 26.58 |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)** | 34.34 | 22.00 | <ins>34.95</ins> | <ins>59.19</ins> | <ins>60.20</ins> |
+
+Examples: arc_easy: 2,376; arc_challenge: 1,172; winogrande: 1,267; gsm8k_mc4: 1,319; gsm8k_mc10: 1,319; gpqa: 198; gpqa_fewshot: 198; chess: 500; hellaswag: 10,042; mmlu: 14,042; mmlu_fewshot: 14,042.
+
+**GSM8K with frozen candidates (200 examples).** The main metric is `nli_rerank@4`:
+
+| Model | nli_rerank@4 | nli_rerank_margin@4 |
+| --- | ---: | ---: |
+| [Open-Jev-9B](https://huggingface.co/ZefanCai/Open-Jev-9B) | **95.00** | **95.00** |
+| [Kev-4B](https://huggingface.co/jaredpalmer/kev-4b) | 89.50 | 89.50 |
+| [Laya English](https://huggingface.co/convaiinnovations/laya) | 91.00 | 91.50 |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)** | <ins>94.50</ins> | <ins>94.00</ins> |
+
+Examples: nli_rerank@4: 200; nli_rerank_margin@4: 200.
+
+The shared candidate set has 93.00% greedy accuracy, 93.50% majority-vote accuracy, and a 97.00% oracle@4 ceiling. These are properties of the same candidate pool, not separate generations by each decision model. The 19-task aggregate uses the main rerank score, not the auxiliary margin score.
+
+</details>
+
+### Image and Text Evaluation
+
+On **Image-NLI**, NeoHorse-Jev-4B reaches **60.65% accuracy over 8,000 examples** with vLLM; the native runtime gives 60.66%. The task evaluates statements against an image and text context.
+
+| Model | Overall | `vqa_answer` | `vqa_answer_neg` | `vqa_disagree` | `vqa_spatial` | `vqa_yesno` |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)**<br>vLLM | 60.65 | 75.94 | 68.79 | 52.88 | 59.95 | 48.53 |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)**<br>Native | 60.66 | 75.94 | 68.63 | 52.98 | 60.03 | 48.37 |
+
+Examples: Overall: 8,000; `vqa_answer`: 1376; `vqa_answer_neg`: 644; `vqa_disagree`: 1040; `vqa_spatial`: 3648; `vqa_yesno`: 1292.
+
+The comparison report contains no Image-NLI results for Kev-4B, Open-Jev-9B, or Laya, so this is a capability measurement without a cross-model ranking. The image assets were reconstructed and frozen locally; this does not claim reproduction of the upstream author's unavailable original image assets.
+
+<details>
+<summary>Doom with image input: all 11 candidate configurations</summary>
+
+| Model | `action` | `danger` | `pixels` | `pixels_pct` | `pixels_sym` | `precise` |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)**<br>vLLM | 1.00 | 1.40 | 9.20 | 7.40 | 8.40 | 16.00 |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)**<br>Native | 1.00 | 1.40 | 11.80 | 8.80 | 10.00 | 15.60 |
+
+| Model | `should` | `thirds` | `where` | `where_closest` | `where_plain` |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)**<br>vLLM | 1.00 | 12.80 | 1.40 | 1.40 | 6.00 |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)**<br>Native | 1.00 | 10.60 | 1.40 | 1.40 | 8.40 |
+
+Each configuration runs for five episodes; scores are mean kills. Five author configurations (`pixels`, `pixels_sym`, `precise`, `thirds`, `where_closest`) use the completed reruns; the six unchanged configurations retain their valid results. The report supplies no image-interface results for the comparison models. All configurations are listed because candidate wording substantially affects the outcome.
+
+</details>
+
+### Interactive Decision Tasks
+
+The September 24 results include the completed game reruns and corrected Minecraft action execution. The tables below report the specified candidate configurations separately and use vLLM for NeoHorse-Jev-4B unless another backend is named. **Text-state Doom, Flappy, and Minecraft results are not image-input evaluations.** Game scores use their own units and are excluded from the text aggregate.
+
+<details>
+<summary>Cross-model game results: Doom, Flappy, and Minecraft</summary>
+
+**Doom with text state — mean kills, five episodes per configuration.**
+
+| Model | `position` (author configuration) | `position_none` (includes no-enemy condition) | `aligned_state` (aligned target and tolerance) |
+| --- | ---: | ---: | ---: |
+| [Open-Jev-9B](https://huggingface.co/ZefanCai/Open-Jev-9B) | 11.20 | 7.80 | 18.60 |
+| [Kev-4B](https://huggingface.co/jaredpalmer/kev-4b) | 1.40 | 10.40 | 14.20 |
+| [Laya English](https://huggingface.co/convaiinnovations/laya) | 1.00 | 1.60 | 1.00 |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)** | 1.40 | 10.60 | 14.40 |
+
+`aligned_state` changes target definition and tolerance wording, so it is a different decision policy from the author's `position` configuration. The reported environment controls are random: 1.00 and oracle: 16.60 mean kills; five-episode outcomes should not be read as a precise ranking.
+
+**Flappy — mean pipes cleared.** Both NeoHorse-Jev backends are shown because real-time outcomes depend on the deployment path. No best/second-best markers are applied to this timing-dependent table.
+
+| Model | sign (author) | position (author) | action (real-time) | action (wait for model) | position_v (real-time) | position_v (wait for model) |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| [Open-Jev-9B](https://huggingface.co/ZefanCai/Open-Jev-9B) | 28.00 | 23.50 | 0.60 | 0.70 | 8.95 | 48.00 |
+| [Kev-4B](https://huggingface.co/jaredpalmer/kev-4b) | 28.00 | 2.67 | 0.00 | 0.10 | 26.80 | 48.00 |
+| [Laya English](https://huggingface.co/convaiinnovations/laya) | 19.00 | 0.50 | 0.00 | 0.00 | 0.00 | 0.00 |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)**<br>vLLM | 27.67 | 27.67 | 0.05 | 0.05 | 15.35 | 48.00 |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)**<br>Native | 28.00 | 27.83 | 0.00 | 0.05 | 42.00 | 48.00 |
+
+Settings: sign (author) and position (author) — 6 episodes, 900-frame cap, 15 FPS; action (real-time) and position_v (real-time) — 20 episodes, 1500-frame cap, 30 FPS; action (wait for model) and position_v (wait for model) — 20 episodes, 1500-frame cap, 0 FPS.
+
+`FPS = 0` waits for every model's response. Flappy outcomes are not a controlled cross-model speed benchmark.
+
+**Real Minecraft — success rate (%), ten episodes and at most 60 decision steps per strategy.**
+
+| Model | flat/action | flat/state | chain |
+| --- | ---: | ---: | ---: |
+| [Open-Jev-9B](https://huggingface.co/ZefanCai/Open-Jev-9B) | 0.00 | 80.00 | 100.00 |
+| [Kev-4B](https://huggingface.co/jaredpalmer/kev-4b) | 0.00 | 60.00 | 90.00 |
+| [Laya English](https://huggingface.co/convaiinnovations/laya) | 0.00 | 0.00 | 20.00 |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)** | 20.00 | 50.00 | 90.00 |
+
+These use the corrected action cancellation and pathfinding-failure handling, with a 240-second action timeout. `flat/action` is a separate ten-episode run; `flat/state` and `chain` share the corrected execution setup. Native NeoHorse-Jev results are 30.00%, 40.00%, and 90.00%, respectively. The environment oracle itself reaches 70–100% across model runs, while random scores 0%, so environment variation remains relevant.
+
+**Simulated Minecraft — success rate (%), ten episodes and at most 60 decision steps per strategy.**
+
+| Model | flat/action | chain |
+| --- | ---: | ---: |
+| [Open-Jev-9B](https://huggingface.co/ZefanCai/Open-Jev-9B) | 0.00 | 100.00 |
+| [Kev-4B](https://huggingface.co/jaredpalmer/kev-4b) | 0.00 | 100.00 |
+| [Laya English](https://huggingface.co/convaiinnovations/laya) | 0.00 | 0.00 |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)** | 0.00 | 100.00 |
+
+The simulated environment's oracle reaches 100% and random scores 0%. Simulated and real Minecraft are different settings and should not be averaged together.
+
+</details>
+
+## Download Model
+
+| Model | Download Links | Parameters | Base Model |
+| --- | --- | --- | --- |
+| **[NeoHorse-Jev-4B](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)** | [🤗 Hugging Face](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B)<br>[🤖 ModelScope](https://www.modelscope.cn/models/TokenRhythm/NeoHorse-Jev-4B) | ~4B | [NeoHorse-1-4B](https://huggingface.co/TokenRhythm/NeoHorse-1-4B) |
+
+Download the complete model bundle, including the backbone, tokenizer, separate decision head, and matching runtime wheel. The [GitHub source repository](https://github.com/TokenRhythm/NeoHorse/tree/main/jev) provides the inference source, backend adapters, and examples.
+
+<details>
+<summary>Model details</summary>
+
+| Field | Value |
+| --- | --- |
+| Parameters | Approximately 4B |
+| Base model | NeoHorse-1-4B |
+| Input | Text, or a single image with text |
+| Decision types | Choice, Noul, Score |
+| Inference | Prefill-only |
+| License | Apache-2.0 |
+
+</details>
+
+## Deployment
+
+Choose [vLLM](#vllm), [SGLang](#sglang), or the [native runtime](#native-runtime). Each path requires the complete model bundle from Hugging Face or ModelScope.
+
+Use a separate, existing environment for each backend. The adapters and example requests are maintained in the [GitHub source repository](https://github.com/TokenRhythm/NeoHorse/tree/main/jev). If you have only downloaded the model bundle, obtain the source first:
+
+```bash
+git clone https://github.com/TokenRhythm/NeoHorse.git
+cd NeoHorse/jev
+```
+
+Run the commands below from the `jev/` directory of the cloned NeoHorse repository and replace `/path/to/model` with the complete model directory downloaded from Hugging Face or ModelScope.
+
+### vLLM
+
+Use an existing **vLLM 0.28.0** environment.
+
+```bash
+# Start the server and keep this terminal running
+CUDA_VISIBLE_DEVICES=0 python infer/vllm/launch.py \
+  --bundle /path/to/model --port 30000
+
+# Once ready, run inference from another terminal
+python infer/vllm/infer.py \
+  --bundle /path/to/model \
+  --url http://127.0.0.1:30000 \
+  --request infer/request.json
+```
+
+### SGLang
+
+Use an existing **SGLang 0.5.17** environment.
+
+```bash
+# Start the server and keep this terminal running
+CUDA_VISIBLE_DEVICES=0 python infer/sglang/launch.py \
+  --bundle /path/to/model --port 30000
+
+# Once ready, run inference from another terminal
+python infer/sglang/infer.py \
+  --bundle /path/to/model \
+  --url http://127.0.0.1:30000 \
+  --request infer/request.json
+```
+
+The sample request is included. Results are printed to the terminal; read `answers.move.choice` and `answers.move.probabilities`. Both backends also support a single image combined with text. See [infer/README.md](https://github.com/TokenRhythm/NeoHorse/blob/main/jev/infer/README.md) for text and image examples, dependency setup, and input limits.
+
+### Native Runtime
+
+The `neohorse_decision` package provides local Python and CLI inference, plus HTTP services for text and image decisions. Expand the walkthrough for installation and examples of all three decision types, or see the [Deployment](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B/blob/main/DEPLOYMENT.md) guide for the complete API reference.
+
+<details>
+<summary>Installation and usage examples</summary>
+
+#### 1. Download the Complete Model Release
+
+Download the complete release from [Hugging Face](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B/tree/main) or [ModelScope](https://www.modelscope.cn/models/TokenRhythm/NeoHorse-Jev-4B), then set its local path:
+
+```bash
+export MODEL_DIR="/path/to/NeoHorse-Jev-4B"
+cd "$MODEL_DIR"
+```
+
+The complete model bundle contains:
+
+| File or directory | Purpose |
+| --- | --- |
+| `backbone/` | Unified multimodal backbone; language and vision parameters share safetensors shards and an index |
+| `tokenizer/` | Matching tokenizer |
+| `pointer_head.safetensors` | Separate decision head |
+| `model_manifest.json` | Model composition and provenance |
+| `dist/`, `package/` | Runtime wheel and source |
+| `example_request.json` | Example request covering all three decision types |
+| `vision/` | Local image inference and HTTP image client examples |
+
+Use the matching `neohorse_decision` package for the native runtime. The vLLM and SGLang adapters are in the [GitHub source repository](https://github.com/TokenRhythm/NeoHorse/tree/main/jev/infer); see [backend deployment](#vllm) above. All three paths require the complete model directory, including the separate decision head.
+
+#### 2. Install the Runtime
+
+The recorded test environment is **Linux, Python 3.12, PyTorch 2.8.0, Transformers 5.17.0, Triton 3.7.1, and flash-linear-attention 0.5.2**, with a CUDA GPU that supports BF16. See [environment.json](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B/blob/main/environment.json) and [DEPLOYMENT.md](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B/blob/main/DEPLOYMENT.md) for details.
+
+The following commands assume these ML dependencies are already installed in an isolated environment and GPU 0 has been allocated to your workload:
+
+```bash
+python -m pip install --no-deps dist/neohorse_decision-1.0.0-py3-none-any.whl
+python -m pip install 'fastapi==0.141.1' 'uvicorn==0.53.0' 'starlette==1.6.0' 'httpx==0.28.1' 'pillow==12.3.0'
+
+CUDA_VISIBLE_DEVICES=0 neohorse-decision predict --model-dir . --request example_request.json
+```
+
+`--no-deps` installs the bundled wheel into an already prepared environment; it does not install the ML dependencies listed above. The unified backbone weights occupy approximately 9.08 GB. Actual GPU memory usage also depends on input and runtime settings. **Download the complete model repository and install its bundled runtime.**
+
+#### Decision Types
+
+| Type | Input | Output | Typical use cases |
+| --- | --- | --- | --- |
+| **Choice** | An ordered dictionary of candidate keys and descriptions | Selected candidate and full candidate probability distribution | Request routing, tool selection, action selection |
+| **Noul** | A yes/no question | Probability that the statement is true, `P(true)` | Condition checks, filtering, workflow gates |
+| **Score** | Rating levels ordered from lowest to highest | Probability distribution over levels and the expected rating | Quality assessment, severity, priority |
+
+Score levels are indexed from `0`, and the expected rating can be fractional. These use cases describe the interface; performance should be validated on your target tasks.
+
+#### 3. Python Examples
+
+**Provide a state and get a yes/no probability, a selected candidate, or a rating.** The examples below use the same user message to demonstrate the three decision modes.
+
+Load the model once, then reuse `engine` and `state`:
+
+```python
+import os
+
+from neohorse_decision import DecisionEngine
+
+engine = DecisionEngine(os.environ["MODEL_DIR"])
+state = "I was charged twice for the same order. Please refund the extra charge today."
+```
+
+All output numbers below are illustrative, not measured results. Actual values depend on the model's predictions.
+
+##### Noul: Is It True?
+
+**Is the user requesting a refund?** Return the probability of "yes", `P(true)`.
+
+```python
+result = engine.predict({
+    "state": state,
+    "questions": {
+        "refund": {
+            "type": "noul",
+            "instructions": "Is the user requesting a refund?",
+        },
+    },
+})
+print(result["answers"]["refund"]["noul"])
+```
+
+Illustrative output: `0.97` means the model assigns a 97% probability to the user requesting a refund. Your application can use this to enter a refund workflow.
+
+##### Choice: Which One?
+
+**Which team should handle this message?** Select from the candidates and return each candidate's probability.
+
+```python
+result = engine.predict({
+    "state": state,
+    "questions": {
+        "team": {
+            "type": "choice",
+            "instructions": "Which team should handle this message?",
+            "criteria": {
+                "billing": "Billing, charges, or refunds",
+                "technical": "Product failures or technical issues",
+                "other": "Other matters",
+            },
+        },
+    },
+})
+print(result["answers"]["team"]["choice"])
+print(result["answers"]["team"]["probabilities"])
+```
+
+Illustrative output:
+
+```text
+billing
+{'billing': 0.96, 'technical': 0.03, 'other': 0.01}
+```
+
+Read `billing` to route the message to the billing team.
+
+##### Score: To What Degree?
+
+**How urgent is the request?** Rate it against the levels you define. Levels start at `0`, and the result is their probability-weighted expected value.
+
+```python
+result = engine.predict({
+    "state": state,
+    "questions": {
+        "urgency": {
+            "type": "score",
+            "instructions": "How soon does the user want this resolved?",
+            "criteria": ["Can wait", "This week", "Today"],
+        },
+    },
+})
+print(result["answers"]["urgency"]["score"])
+```
+
+Illustrative output: `1.9` is close to level `2` ("Today"), which your application can use to raise the request's priority.
+
+Save the four Python blocks above, in order, as `quickstart.py`, then run:
+
+```bash
+CUDA_VISIBLE_DEVICES=0 python quickstart.py
+```
+
+To make all three decisions in one text request, place `refund`, `team`, and `urgency` in the same `questions` dictionary. One request returns three answers. Set decision thresholds using data from your own tasks.
+
+#### 4. Image and Text Decisions
+
+Save a page screenshot as `screenshot.png`. This Choice example identifies the page's current state:
+
+```python
+import os
+
+from PIL import Image
+from neohorse_decision.vision import VisionDecisionEngine
+
+vision_engine = VisionDecisionEngine(os.environ["MODEL_DIR"])
+with Image.open("screenshot.png") as source:
+    screenshot = source.convert("RGB")
+
+result = vision_engine.predict({
+    "model": "NeoHorse-Jev-4B",
+    "state": "Goal: submit the form. Assess the current page screenshot.",
+    "questions": {
+        "page_status": {
+            "type": "choice",
+            "instructions": "Which page state does the screenshot show?",
+            "criteria": {
+                "success": "Submission succeeded",
+                "error": "Submission failed or an error is shown",
+                "processing": "Submission or loading is in progress",
+                "unknown": "Cannot determine the submission status from the screenshot",
+            },
+        },
+    },
+}, screenshot)
+print(result["answers"]["page_status"]["choice"])
+print(result["answers"]["page_status"]["probabilities"])
+```
+
+Image requests also support Noul and Score, with one image and one question per request. Examples for all three modes and HTTP image requests are in the [image usage guide](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B/blob/main/DEPLOYMENT.md#6-image-requests).
+
+#### 5. HTTP Service
+
+Start the service:
+
+```bash
+CUDA_VISIBLE_DEVICES=0 neohorse-decision serve --model-dir "$MODEL_DIR" --port 8080
+```
+
+From another terminal, send the same Noul question:
+
+```bash
+curl -sS http://127.0.0.1:8080/v1/systemone \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"NeoHorse-Jev-4B","state":"I was charged twice for the same order. Please refund the extra charge today.","questions":{"refund":{"type":"noul","instructions":"Is the user requesting a refund?"}}}'
+```
+
+The service binds to `127.0.0.1` by default. For external access, enable Bearer authentication with `NEOHORSE_API_KEY` and use a TLS gateway. The native endpoint is `/v1/decision`, the System One-style endpoint is `/v1/systemone`, and `/health` reports readiness.
+
+See the [deployment and API guide](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B/blob/main/DEPLOYMENT.md) for request formats, response fields, default limits, and error handling.
+
+#### Install from Source
+
+With the ML dependencies above already installed, run this from the root of the downloaded model bundle:
+
+```bash
+python -m pip install --no-deps ./package
+```
+
+`MODEL_DIR` still points to the complete model bundle downloaded from Hugging Face or ModelScope. Inference source is in `package/src/neohorse_decision/`; image clients and local image examples are in `vision/`.
+
+</details>
+
+## Limitations
+
+- **Decisions can be wrong.** Valid structure and normalized probabilities do not guarantee correct judgments. Missing evidence, candidate descriptions, candidate order, and domain shifts can all affect results.
+- **Validate probabilities for your application.** NLL, Brier, and ECE calibration results have not been reported. Set thresholds on an independent dataset.
+- **Scope claims to measured evidence.** Comprehensive evaluations of multilingual inputs, long inputs, and computational isolation between questions are not yet available. Multiple questions in one request do not imply a single shared forward pass.
+- **Applications enforce execution constraints.** Tool permissions, business rules, and action validation remain the application's responsibility. The current materials do not provide latency, GPU memory, or cost comparisons under a common timing protocol.
+
+## License and Acknowledgments
+
+NeoHorse-Jev-4B is released under **Apache License 2.0**. It is derived from NeoHorse-1-4B, whose upstream base is Qwen3.5-4B. Bundled third-party runtime components retain their licenses and attribution. Preserve the relevant copyright, license, and modification notices when redistributing.
+
+We thank Jared Palmer for open-sourcing [Kev](https://github.com/jaredpalmer/kev). Parts of this project's decision inference code are adapted from Kev.
+
+For questions or bug reports, use the [NeoHorse issue tracker](https://github.com/TokenRhythm/NeoHorse/issues).

SHA256SUMS

@@ -0,0 +1,43 @@
+76fdab375b201f5f42d82a06794bdb015d9e98bbe083ada46740b67a26e443bd  DEPLOYMENT.md
+30e9fa2b5abc577cddfc7aa17821226561e53ca22807444d55561273dbf888a0  README.md
+e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  assets/.gitkeep
+9d0b6dfa93abc37c50872b9e568b1c06c73ca867bb4592f92429691cca83bebc  assets/jev-six-demo-grid.gif
+3e5547987b4976248251215d90ae234772988a2d3099bdc77c48dc70ba117b0b  assets/jev-snake-demo.gif
+833ef256e6b87f27d0f10cbb06ef51e34b2687922ac1bfabc3bc016166d847ff  backbone/config.json
+7d1adbb748ff60a91b3b6ffba1ff70bfcab855ff2b8cf5e33f9c6bc1ff13cb7e  backbone/model-00001-of-00003.safetensors
+c37c278c3977b16b7358421a10b5805615a0f393f19f56b540644dded4d0e6c3  backbone/model-00002-of-00003.safetensors
+475b9a4b012cc888da1d6575746d2b329a944a6107f8e20b1dcb83b116bf1b1d  backbone/model-00003-of-00003.safetensors
+5c881b4f2ae4b9600d2a7d68a84d4a0dc3c62e6974c601dd281c9e61a3c3da7c  backbone/model.safetensors.index.json
+27225450ac9c6529872ee1924fcb0962ff5634834f817040f444118116f4e516  backbone/preprocessor_config.json
+8a8254e1f597af2d400e9df2758d9a6491c3351a92e2c6e70e46eeca780a1e7f  config.json
+7401f1f3a041bd9c05662aaf38ad894e68e0103685a2fd3ed23169c3aff64bfc  dist/neohorse_decision-1.0.0-py3-none-any.whl
+4d5790e2ace3cf8bfc26fbc23c973e2ef1c796911c407d5a75adc48ed4b1c32c  environment.json
+86095b334c58f2c0817e46c53d241ac6fb2503b386e2cb90174480626ddf360b  example_request.json
+8a8254e1f597af2d400e9df2758d9a6491c3351a92e2c6e70e46eeca780a1e7f  model_manifest.json
+d6feb31981fc445e3e9e5a18e1ece4909bdec9f4ef293d57dd547e93dcd0127f  package/pyproject.toml
+257213078c9c179fbde33ea8defd63174cd36207bb72a725ea7845b581c30cba  package/src/neohorse_decision/__init__.py
+d8283d38f51f2cd88017039dc944d3d44a60e56e871090e077cd23d7e1c11804  package/src/neohorse_decision/_inference.py
+f93060bd0f1875daaee413754eeb3a4757205bdb9e4e7a36b4efc379a027d43f  package/src/neohorse_decision/_vendor/LICENSE
+9bdb2a943d547c457a5614c5ab138b19753798e59f94ed92f3d1c07384510703  package/src/neohorse_decision/_vendor/NOTICE.md
+7b86e4fe30867d00f45c09e3a9930e7406838c934abd338fb027126aa0eb1f89  package/src/neohorse_decision/_vendor/__init__.py
+cff5be248612012ae49d7e3fd4c75c220c94d5432114a661175a68cf3237cdc1  package/src/neohorse_decision/_vendor/model.py
+e78ee8f180660e2ddb057b208b581f9e1cdca024ea50d87c631286c187691370  package/src/neohorse_decision/_vendor/schema.py
+a3daeb60ce7211e067737fd7345e7b2d20a04403a0424d310a9c38eb0096b1cc  package/src/neohorse_decision/cli.py
+3b6b3f36f830c654ffa161ef222a39b9e766039cfa6d7612add9a4c70fb88b76  package/src/neohorse_decision/engine.py
+69c64508c024b1217f732c078e74ca7d838e95614d428359698a2f053fddfd41  package/src/neohorse_decision/image_input.py
+bd49282fd749d4550bf83190abdda9fdeb906a86ae2addc9da2c758d92a91286  package/src/neohorse_decision/server.py
+bce2de084b017bb2e5615ea27ffc226e1373cc6410df553a556b88c200c7633c  package/src/neohorse_decision/systemone.py
+3777920631dd0a62d79a4428c06089a898676e45b9468f2dbf430cf109f8e506  package/src/neohorse_decision/vision.py
+467ae48977b5c4bf87dd1db29021199e0c3401c52c5b57a1eccde679c0bade26  pointer_head.safetensors
+a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715  tokenizer/chat_template.jinja
+06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523  tokenizer/tokenizer.json
+bee8eba30f0eb4af73c0fe2cd06d0f89b657d7819941c438157ec42f7c80ea87  tokenizer/tokenizer_config.json
+50cbab8a892c5f2993b8c7351a99182507472def3b1374558308605d99b86b32  vision/LICENSE
+4683bbfa3df4e757e85ce205c4c078a5c2e7521f8f50fc6d596fd7f65e6fa198  vision/README.md
+8328816e999c1fec44601f38e9063505b67f12541736f136e53f94f5a10fb783  vision/__init__.py
+72d2920650ee35d0814aeb3b7450431f56b132a2d75d72ff9a0a9df6ea0203f6  vision/base_vision_provenance.json
+cd7e84e77580254ad9e898a962f1e09c55170714f2a7b81d7025047e60200aa7  vision/example.py
+371c4962ed7901a4629058eacf3984993b1c871ff10c49e2cc8b0f52485fec6b  vision/example_request.json
+f0a8d8182bb654ea829bf37415b1b4ef72272546f69dda2d8f9cfbef921131c4  vision/http_example.py
+b376c208ba3647173ab029f52e34aac5a783460bd06a643db401a1422fd02422  vision/predictor.py
+60127324a1cab22c6e8f1314efcb7ea736dc038420c63f34e4ee527da1e62363  vision/verification.json

backbone/config.json

@@ -0,0 +1,112 @@
+{
+  "architectures": [
+    "Qwen3_5Model"
+  ],
+  "image_token_id": 248056,
+  "model_type": "qwen3_5",
+  "text_config": {
+    "architectures": [
+      "Qwen3_5TextModel"
+    ],
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attn_output_gate": true,
+    "bos_token_id": null,
+    "dtype": "bfloat16",
+    "eos_token_id": 248044,
+    "full_attention_interval": 4,
+    "head_dim": 256,
+    "hidden_act": "silu",
+    "hidden_size": 2560,
+    "initializer_range": 0.02,
+    "intermediate_size": 9216,
+    "layer_types": [
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention"
+    ],
+    "linear_conv_kernel_dim": 4,
+    "linear_key_head_dim": 128,
+    "linear_num_key_heads": 16,
+    "linear_num_value_heads": 32,
+    "linear_value_head_dim": 128,
+    "mamba_ssm_dtype": "float32",
+    "max_position_embeddings": 262144,
+    "mlp_only_layers": [],
+    "model_type": "qwen3_5_text",
+    "modification_notice": "Modified by TokenRhythm: language-model weights fine-tuned from Qwen/Qwen3.5-4B; repackaged for text-only inference by changing config and tensor key prefixes. Tensor values are unchanged by repackaging. Original model: Copyright 2026 Alibaba Cloud.",
+    "mtp_num_hidden_layers": 1,
+    "mtp_use_dedicated_embeddings": false,
+    "num_attention_heads": 16,
+    "num_hidden_layers": 32,
+    "num_key_value_heads": 4,
+    "pad_token_id": null,
+    "partial_rotary_factor": 0.25,
+    "rms_norm_eps": 1e-06,
+    "rope_parameters": {
+      "mrope_interleaved": true,
+      "mrope_section": [
+        11,
+        11,
+        10
+      ],
+      "partial_rotary_factor": 0.25,
+      "rope_theta": 10000000,
+      "rope_type": "default"
+    },
+    "tie_word_embeddings": true,
+    "transformers_version": "5.17.0",
+    "use_cache": true,
+    "vocab_size": 248320
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.17.0",
+  "video_token_id": 248057,
+  "vision_config": {
+    "deepstack_visual_indexes": [],
+    "depth": 24,
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1024,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 4096,
+    "model_type": "qwen3_5",
+    "num_heads": 16,
+    "num_position_embeddings": 2304,
+    "out_hidden_size": 2560,
+    "patch_size": 16,
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2
+  },
+  "vision_end_token_id": 248054,
+  "vision_start_token_id": 248053
+}

backbone/model-00001-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7d1adbb748ff60a91b3b6ffba1ff70bfcab855ff2b8cf5e33f9c6bc1ff13cb7e
+size 3991297968

backbone/model-00002-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c37c278c3977b16b7358421a10b5805615a0f393f19f56b540644dded4d0e6c3
+size 3968952928

backbone/model-00003-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:475b9a4b012cc888da1d6575746d2b329a944a6107f8e20b1dcb83b116bf1b1d
+size 1118364688

backbone/model.safetensors.index.json

@@ -0,0 +1,730 @@
+{
+  "metadata": {
+    "total_size": 9078531072
+  },
+  "weight_map": {
+    "language_model.embed_tokens.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.0.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.0.linear_attn.A_log": "model-00001-of-00003.safetensors",
+    "language_model.layers.0.linear_attn.conv1d.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.0.linear_attn.dt_bias": "model-00001-of-00003.safetensors",
+    "language_model.layers.0.linear_attn.in_proj_a.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.0.linear_attn.in_proj_b.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.0.linear_attn.in_proj_qkv.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.0.linear_attn.in_proj_z.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.0.linear_attn.norm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.0.linear_attn.out_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.0.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.0.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.0.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.0.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.1.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.1.linear_attn.A_log": "model-00001-of-00003.safetensors",
+    "language_model.layers.1.linear_attn.conv1d.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.1.linear_attn.dt_bias": "model-00001-of-00003.safetensors",
+    "language_model.layers.1.linear_attn.in_proj_a.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.1.linear_attn.in_proj_b.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.1.linear_attn.in_proj_qkv.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.1.linear_attn.in_proj_z.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.1.linear_attn.norm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.1.linear_attn.out_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.1.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.1.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.1.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.1.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.10.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.10.linear_attn.A_log": "model-00001-of-00003.safetensors",
+    "language_model.layers.10.linear_attn.conv1d.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.10.linear_attn.dt_bias": "model-00001-of-00003.safetensors",
+    "language_model.layers.10.linear_attn.in_proj_a.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.10.linear_attn.in_proj_b.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.10.linear_attn.in_proj_qkv.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.10.linear_attn.in_proj_z.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.10.linear_attn.norm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.10.linear_attn.out_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.10.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.10.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.10.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.10.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.11.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.11.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.11.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.11.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.11.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.11.self_attn.k_norm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.11.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.11.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.11.self_attn.q_norm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.11.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.11.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.12.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.12.linear_attn.A_log": "model-00001-of-00003.safetensors",
+    "language_model.layers.12.linear_attn.conv1d.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.12.linear_attn.dt_bias": "model-00001-of-00003.safetensors",
+    "language_model.layers.12.linear_attn.in_proj_a.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.12.linear_attn.in_proj_b.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.12.linear_attn.in_proj_qkv.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.12.linear_attn.in_proj_z.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.12.linear_attn.norm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.12.linear_attn.out_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.12.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.12.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.12.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.12.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.13.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.13.linear_attn.A_log": "model-00001-of-00003.safetensors",
+    "language_model.layers.13.linear_attn.conv1d.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.13.linear_attn.dt_bias": "model-00001-of-00003.safetensors",
+    "language_model.layers.13.linear_attn.in_proj_a.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.13.linear_attn.in_proj_b.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.13.linear_attn.in_proj_qkv.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.13.linear_attn.in_proj_z.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.13.linear_attn.norm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.13.linear_attn.out_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.13.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.13.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.13.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.13.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.14.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.14.linear_attn.A_log": "model-00001-of-00003.safetensors",
+    "language_model.layers.14.linear_attn.conv1d.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.14.linear_attn.dt_bias": "model-00001-of-00003.safetensors",
+    "language_model.layers.14.linear_attn.in_proj_a.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.14.linear_attn.in_proj_b.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.14.linear_attn.in_proj_qkv.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.14.linear_attn.in_proj_z.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.14.linear_attn.norm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.14.linear_attn.out_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.14.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.14.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.14.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.14.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.15.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.15.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.15.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.15.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.15.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.15.self_attn.k_norm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.15.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.15.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.15.self_attn.q_norm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.15.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.15.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.16.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.16.linear_attn.A_log": "model-00001-of-00003.safetensors",
+    "language_model.layers.16.linear_attn.conv1d.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.16.linear_attn.dt_bias": "model-00001-of-00003.safetensors",
+    "language_model.layers.16.linear_attn.in_proj_a.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.16.linear_attn.in_proj_b.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.16.linear_attn.in_proj_qkv.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.16.linear_attn.in_proj_z.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.16.linear_attn.norm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.16.linear_attn.out_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.16.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.16.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.16.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.16.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.17.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.17.linear_attn.A_log": "model-00001-of-00003.safetensors",
+    "language_model.layers.17.linear_attn.conv1d.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.17.linear_attn.dt_bias": "model-00001-of-00003.safetensors",
+    "language_model.layers.17.linear_attn.in_proj_a.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.17.linear_attn.in_proj_b.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.17.linear_attn.in_proj_qkv.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.17.linear_attn.in_proj_z.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.17.linear_attn.norm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.17.linear_attn.out_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.17.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.17.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.17.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.17.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.18.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.18.linear_attn.A_log": "model-00001-of-00003.safetensors",
+    "language_model.layers.18.linear_attn.conv1d.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.18.linear_attn.dt_bias": "model-00001-of-00003.safetensors",
+    "language_model.layers.18.linear_attn.in_proj_a.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.18.linear_attn.in_proj_b.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.18.linear_attn.in_proj_qkv.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.18.linear_attn.in_proj_z.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.18.linear_attn.norm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.18.linear_attn.out_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.18.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.18.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.18.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.18.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.19.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.19.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.19.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.19.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.19.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.19.self_attn.k_norm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.19.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.19.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.19.self_attn.q_norm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.19.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.19.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.2.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.2.linear_attn.A_log": "model-00001-of-00003.safetensors",
+    "language_model.layers.2.linear_attn.conv1d.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.2.linear_attn.dt_bias": "model-00001-of-00003.safetensors",
+    "language_model.layers.2.linear_attn.in_proj_a.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.2.linear_attn.in_proj_b.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.2.linear_attn.in_proj_qkv.weight": "model-00001-of-00003.safetensors",
+    "language_model.layers.2.linear_attn.in_proj_z.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.2.linear_attn.norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.2.linear_attn.out_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.2.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.2.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.2.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.2.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.20.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.20.linear_attn.A_log": "model-00002-of-00003.safetensors",
+    "language_model.layers.20.linear_attn.conv1d.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.20.linear_attn.dt_bias": "model-00002-of-00003.safetensors",
+    "language_model.layers.20.linear_attn.in_proj_a.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.20.linear_attn.in_proj_b.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.20.linear_attn.in_proj_qkv.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.20.linear_attn.in_proj_z.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.20.linear_attn.norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.20.linear_attn.out_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.20.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.20.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.20.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.20.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.21.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.21.linear_attn.A_log": "model-00002-of-00003.safetensors",
+    "language_model.layers.21.linear_attn.conv1d.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.21.linear_attn.dt_bias": "model-00002-of-00003.safetensors",
+    "language_model.layers.21.linear_attn.in_proj_a.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.21.linear_attn.in_proj_b.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.21.linear_attn.in_proj_qkv.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.21.linear_attn.in_proj_z.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.21.linear_attn.norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.21.linear_attn.out_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.21.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.21.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.21.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.21.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.22.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.22.linear_attn.A_log": "model-00002-of-00003.safetensors",
+    "language_model.layers.22.linear_attn.conv1d.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.22.linear_attn.dt_bias": "model-00002-of-00003.safetensors",
+    "language_model.layers.22.linear_attn.in_proj_a.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.22.linear_attn.in_proj_b.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.22.linear_attn.in_proj_qkv.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.22.linear_attn.in_proj_z.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.22.linear_attn.norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.22.linear_attn.out_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.22.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.22.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.22.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.22.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.23.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.23.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.23.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.23.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.23.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.23.self_attn.k_norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.23.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.23.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.23.self_attn.q_norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.23.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.23.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.24.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.24.linear_attn.A_log": "model-00002-of-00003.safetensors",
+    "language_model.layers.24.linear_attn.conv1d.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.24.linear_attn.dt_bias": "model-00002-of-00003.safetensors",
+    "language_model.layers.24.linear_attn.in_proj_a.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.24.linear_attn.in_proj_b.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.24.linear_attn.in_proj_qkv.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.24.linear_attn.in_proj_z.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.24.linear_attn.norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.24.linear_attn.out_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.24.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.24.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.24.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.24.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.25.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.25.linear_attn.A_log": "model-00002-of-00003.safetensors",
+    "language_model.layers.25.linear_attn.conv1d.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.25.linear_attn.dt_bias": "model-00002-of-00003.safetensors",
+    "language_model.layers.25.linear_attn.in_proj_a.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.25.linear_attn.in_proj_b.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.25.linear_attn.in_proj_qkv.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.25.linear_attn.in_proj_z.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.25.linear_attn.norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.25.linear_attn.out_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.25.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.25.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.25.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.25.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.26.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.26.linear_attn.A_log": "model-00002-of-00003.safetensors",
+    "language_model.layers.26.linear_attn.conv1d.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.26.linear_attn.dt_bias": "model-00002-of-00003.safetensors",
+    "language_model.layers.26.linear_attn.in_proj_a.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.26.linear_attn.in_proj_b.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.26.linear_attn.in_proj_qkv.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.26.linear_attn.in_proj_z.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.26.linear_attn.norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.26.linear_attn.out_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.26.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.26.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.26.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.26.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.27.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.27.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.27.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.27.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.27.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.27.self_attn.k_norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.27.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.27.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.27.self_attn.q_norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.27.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.27.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.28.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.28.linear_attn.A_log": "model-00002-of-00003.safetensors",
+    "language_model.layers.28.linear_attn.conv1d.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.28.linear_attn.dt_bias": "model-00002-of-00003.safetensors",
+    "language_model.layers.28.linear_attn.in_proj_a.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.28.linear_attn.in_proj_b.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.28.linear_attn.in_proj_qkv.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.28.linear_attn.in_proj_z.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.28.linear_attn.norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.28.linear_attn.out_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.28.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.28.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.28.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.28.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.29.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.29.linear_attn.A_log": "model-00002-of-00003.safetensors",
+    "language_model.layers.29.linear_attn.conv1d.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.29.linear_attn.dt_bias": "model-00002-of-00003.safetensors",
+    "language_model.layers.29.linear_attn.in_proj_a.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.29.linear_attn.in_proj_b.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.29.linear_attn.in_proj_qkv.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.29.linear_attn.in_proj_z.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.29.linear_attn.norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.29.linear_attn.out_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.29.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.29.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.29.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.29.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.3.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.3.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.3.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.3.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.3.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.3.self_attn.k_norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.3.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.3.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.3.self_attn.q_norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.3.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.3.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.30.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.30.linear_attn.A_log": "model-00002-of-00003.safetensors",
+    "language_model.layers.30.linear_attn.conv1d.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.30.linear_attn.dt_bias": "model-00002-of-00003.safetensors",
+    "language_model.layers.30.linear_attn.in_proj_a.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.30.linear_attn.in_proj_b.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.30.linear_attn.in_proj_qkv.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.30.linear_attn.in_proj_z.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.30.linear_attn.norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.30.linear_attn.out_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.30.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.30.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.30.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.30.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.31.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.31.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.31.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.31.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.31.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.31.self_attn.k_norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.31.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.31.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.31.self_attn.q_norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.31.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.31.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.4.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.4.linear_attn.A_log": "model-00002-of-00003.safetensors",
+    "language_model.layers.4.linear_attn.conv1d.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.4.linear_attn.dt_bias": "model-00002-of-00003.safetensors",
+    "language_model.layers.4.linear_attn.in_proj_a.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.4.linear_attn.in_proj_b.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.4.linear_attn.in_proj_qkv.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.4.linear_attn.in_proj_z.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.4.linear_attn.norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.4.linear_attn.out_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.4.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.4.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.4.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.4.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.5.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.5.linear_attn.A_log": "model-00002-of-00003.safetensors",
+    "language_model.layers.5.linear_attn.conv1d.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.5.linear_attn.dt_bias": "model-00002-of-00003.safetensors",
+    "language_model.layers.5.linear_attn.in_proj_a.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.5.linear_attn.in_proj_b.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.5.linear_attn.in_proj_qkv.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.5.linear_attn.in_proj_z.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.5.linear_attn.norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.5.linear_attn.out_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.5.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.5.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.5.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.5.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.6.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.6.linear_attn.A_log": "model-00002-of-00003.safetensors",
+    "language_model.layers.6.linear_attn.conv1d.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.6.linear_attn.dt_bias": "model-00002-of-00003.safetensors",
+    "language_model.layers.6.linear_attn.in_proj_a.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.6.linear_attn.in_proj_b.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.6.linear_attn.in_proj_qkv.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.6.linear_attn.in_proj_z.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.6.linear_attn.norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.6.linear_attn.out_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.6.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.6.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.6.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.6.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.7.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.7.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.7.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.7.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.7.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.7.self_attn.k_norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.7.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.7.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.7.self_attn.q_norm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.7.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.7.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.8.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.8.linear_attn.A_log": "model-00002-of-00003.safetensors",
+    "language_model.layers.8.linear_attn.conv1d.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.8.linear_attn.dt_bias": "model-00002-of-00003.safetensors",
+    "language_model.layers.8.linear_attn.in_proj_a.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.8.linear_attn.in_proj_b.weight": "model-00002-of-00003.safetensors",
+    "language_model.layers.8.linear_attn.in_proj_qkv.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.8.linear_attn.in_proj_z.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.8.linear_attn.norm.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.8.linear_attn.out_proj.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.8.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.8.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.8.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.8.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.9.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.9.linear_attn.A_log": "model-00003-of-00003.safetensors",
+    "language_model.layers.9.linear_attn.conv1d.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.9.linear_attn.dt_bias": "model-00003-of-00003.safetensors",
+    "language_model.layers.9.linear_attn.in_proj_a.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.9.linear_attn.in_proj_b.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.9.linear_attn.in_proj_qkv.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.9.linear_attn.in_proj_z.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.9.linear_attn.norm.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.9.linear_attn.out_proj.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.9.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.9.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.9.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "language_model.layers.9.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "language_model.norm.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.0.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.0.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.0.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.0.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.0.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.0.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.0.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.0.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.0.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.0.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.0.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.0.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.1.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.1.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.1.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.1.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.1.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.1.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.1.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.1.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.1.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.1.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.1.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.1.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.10.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.10.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.10.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.10.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.10.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.10.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.10.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.10.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.10.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.10.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.10.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.10.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.11.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.11.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.11.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.11.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.11.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.11.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.11.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.11.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.11.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.11.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.11.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.11.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.12.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.12.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.12.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.12.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.12.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.12.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.12.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.12.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.12.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.12.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.12.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.12.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.13.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.13.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.13.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.13.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.13.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.13.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.13.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.13.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.13.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.13.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.13.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.13.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.14.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.14.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.14.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.14.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.14.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.14.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.14.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.14.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.14.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.14.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.14.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.14.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.15.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.15.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.15.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.15.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.15.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.15.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.15.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.15.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.15.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.15.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.15.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.15.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.16.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.16.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.16.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.16.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.16.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.16.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.16.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.16.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.16.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.16.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.16.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.16.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.17.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.17.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.17.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.17.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.17.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.17.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.17.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.17.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.17.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.17.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.17.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.17.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.18.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.18.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.18.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.18.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.18.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.18.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.18.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.18.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.18.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.18.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.18.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.18.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.19.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.19.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.19.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.19.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.19.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.19.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.19.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.19.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.19.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.19.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.19.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.19.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.2.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.2.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.2.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.2.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.2.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.2.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.2.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.2.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.2.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.2.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.2.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.2.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.20.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.20.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.20.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.20.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.20.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.20.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.20.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.20.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.20.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.20.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.20.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.20.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.21.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.21.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.21.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.21.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.21.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.21.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.21.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.21.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.21.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.21.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.21.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.21.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.22.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.22.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.22.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.22.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.22.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.22.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.22.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.22.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.22.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.22.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.22.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.22.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.23.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.23.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.23.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.23.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.23.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.23.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.23.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.23.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.23.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.23.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.23.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.23.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.3.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.3.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.3.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.3.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.3.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.3.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.3.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.3.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.3.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.3.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.3.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.3.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.4.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.4.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.4.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.4.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.4.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.4.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.4.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.4.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.4.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.4.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.4.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.4.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.5.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.5.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.5.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.5.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.5.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.5.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.5.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.5.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.5.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.5.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.5.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.5.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.6.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.6.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.6.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.6.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.6.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.6.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.6.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.6.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.6.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.6.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.6.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.6.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.7.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.7.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.7.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.7.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.7.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.7.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.7.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.7.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.7.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.7.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.7.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.7.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.8.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.8.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.8.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.8.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.8.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.8.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.8.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.8.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.8.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.8.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.8.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.8.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.9.attn.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.9.attn.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.9.attn.qkv.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.9.attn.qkv.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.9.mlp.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.9.mlp.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.9.mlp.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.9.mlp.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.9.norm1.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.9.norm1.weight": "model-00003-of-00003.safetensors",
+    "visual.blocks.9.norm2.bias": "model-00003-of-00003.safetensors",
+    "visual.blocks.9.norm2.weight": "model-00003-of-00003.safetensors",
+    "visual.merger.linear_fc1.bias": "model-00003-of-00003.safetensors",
+    "visual.merger.linear_fc1.weight": "model-00003-of-00003.safetensors",
+    "visual.merger.linear_fc2.bias": "model-00003-of-00003.safetensors",
+    "visual.merger.linear_fc2.weight": "model-00003-of-00003.safetensors",
+    "visual.merger.norm.bias": "model-00003-of-00003.safetensors",
+    "visual.merger.norm.weight": "model-00003-of-00003.safetensors",
+    "visual.patch_embed.proj.bias": "model-00003-of-00003.safetensors",
+    "visual.patch_embed.proj.weight": "model-00003-of-00003.safetensors",
+    "visual.pos_embed.weight": "model-00003-of-00003.safetensors"
+  }
+}

backbone/preprocessor_config.json

@@ -0,0 +1,21 @@
+{
+    "size": {
+        "longest_edge": 16777216,
+        "shortest_edge": 65536
+    },
+    "patch_size": 16,
+    "temporal_patch_size": 2,
+    "merge_size": 2,
+    "image_mean": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "image_std": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "processor_class": "Qwen3VLProcessor",
+    "image_processor_type": "Qwen2VLImageProcessorFast"
+}

config.json

@@ -0,0 +1,22 @@
+{
+  "name": "NeoHorse-Jev-4B",
+  "format": "unified multimodal backbone + independent decision pointer head; NOT chat causal LM",
+  "backbone_class": "Qwen3_5Model",
+  "backbone_dtype": "bfloat16",
+  "head_dtype": "float32",
+  "merge_dtype": "float32",
+  "lora_scale": 1.0,
+  "head_dim": 256,
+  "option_isolation": false,
+  "hybrid": true,
+  "temperature": 1.0,
+  "runtime_origin": "https://github.com/jaredpalmer/kev",
+  "created_unix": 1790132795.85603,
+  "bundle_version": "1.0.0",
+  "vision_source": "Qwen/Qwen3.5-4B",
+  "vision_tensors": 297,
+  "vision_finetuning": false,
+  "text_tensors_unchanged": true,
+  "pointer_head_unchanged": true,
+  "http_image_field": "image: inline PNG/JPEG/WebP base64 data URL"
+}

environment.json

@@ -0,0 +1,17 @@
+{
+  "packages": {
+    "torch": "2.8.0",
+    "transformers": "5.17.0",
+    "safetensors": "0.8.0",
+    "pydantic": "2.13.5",
+    "peft": "0.21.0",
+    "triton": "3.7.1",
+    "flash-linear-attention": "0.5.2",
+    "tokenizers": "0.23.2",
+    "huggingface-hub": "1.32.0",
+    "einops": "0.8.2"
+  },
+  "python": "3.12.10 (main, Apr  9 2025, 08:55:05) [GCC 11.4.0]",
+  "cuda": "12.8",
+  "gpu": "NVIDIA H20"
+}

example_request.json

@@ -0,0 +1,26 @@
+{
+  "state": "The corridor ahead is blocked. The left route is clear.",
+  "questions": {
+    "move": {
+      "type": "choice",
+      "instructions": "Choose the safe route.",
+      "criteria": {
+        "left": "Take the clear route.",
+        "forward": "Hit the blockage."
+      }
+    },
+    "blocked": {
+      "type": "noul",
+      "instructions": "Is the corridor ahead blocked?"
+    },
+    "risk": {
+      "type": "score",
+      "instructions": "Risk of moving forward.",
+      "criteria": [
+        "low",
+        "medium",
+        "high"
+      ]
+    }
+  }
+}

model_manifest.json

@@ -0,0 +1,22 @@
+{
+  "name": "NeoHorse-Jev-4B",
+  "format": "unified multimodal backbone + independent decision pointer head; NOT chat causal LM",
+  "backbone_class": "Qwen3_5Model",
+  "backbone_dtype": "bfloat16",
+  "head_dtype": "float32",
+  "merge_dtype": "float32",
+  "lora_scale": 1.0,
+  "head_dim": 256,
+  "option_isolation": false,
+  "hybrid": true,
+  "temperature": 1.0,
+  "runtime_origin": "https://github.com/jaredpalmer/kev",
+  "created_unix": 1790132795.85603,
+  "bundle_version": "1.0.0",
+  "vision_source": "Qwen/Qwen3.5-4B",
+  "vision_tensors": 297,
+  "vision_finetuning": false,
+  "text_tensors_unchanged": true,
+  "pointer_head_unchanged": true,
+  "http_image_field": "image: inline PNG/JPEG/WebP base64 data URL"
+}

package/pyproject.toml

@@ -0,0 +1,24 @@
+[build-system]
+requires = ["setuptools>=69", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "neohorse-decision"
+version = "1.0.0"
+description = "NeoHorse-JEV prefill-only decision inference"
+requires-python = ">=3.11"
+dependencies = ["torch==2.8.0", "transformers==5.17.0", "safetensors==0.8.0", "pydantic==2.13.5"]
+
+[project.optional-dependencies]
+vision = ["pillow==12.3.0"]
+serve = ["fastapi==0.141.1", "uvicorn==0.53.0", "starlette==1.6.0", "pillow==12.3.0"]
+test = ["httpx==0.28.1"]
+
+[project.scripts]
+neohorse-decision = "neohorse_decision.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+neohorse_decision = ["_vendor/LICENSE", "_vendor/NOTICE.md"]

package/src/neohorse_decision/__init__.py

@@ -0,0 +1,5 @@
+from importlib.metadata import version as _package_version
+from .engine import DecisionEngine
+
+__version__ = _package_version('neohorse-decision')
+__all__ = ['DecisionEngine']

package/src/neohorse_decision/_inference.py

@@ -0,0 +1,70 @@
+"""Offline loader for the merged NeoHorse decision model (not a chat LM)."""
+import json
+from pathlib import Path
+
+import torch
+from safetensors.torch import load_file
+from transformers import AutoModel, AutoTokenizer
+from ._vendor.model import DecisionModel, PointerHead
+from ._vendor.schema import SystemOneRequest, to_record
+
+
+def load_bundle(bundle, device='cuda'):
+    root = Path(bundle)
+    meta = json.loads((root / 'model_manifest.json').read_text())
+    torch.set_num_threads(4)
+    torch.backends.cuda.matmul.allow_tf32 = False
+    torch.backends.cudnn.allow_tf32 = False
+    torch.backends.cuda.enable_flash_sdp(True)
+    torch.backends.cuda.enable_mem_efficient_sdp(True)
+    tok = AutoTokenizer.from_pretrained(root / 'tokenizer', local_files_only=True)
+    backbone, info = AutoModel.from_pretrained(
+        root / 'backbone', dtype=torch.bfloat16, attn_implementation='sdpa',
+        local_files_only=True, output_loading_info=True)
+    assert not info.get('missing_keys') and not info.get('unexpected_keys'), info
+    assert not info.get('mismatched_keys') and not info.get('error_msgs'), info
+    assert type(backbone).__name__ == meta['backbone_class']
+    # Bypass the training constructor: it loads the original base checkpoint.
+    # This bundle instead supplies the complete, already merged backbone.
+    model = DecisionModel.__new__(DecisionModel)
+    torch.nn.Module.__init__(model)
+    # Some Transformers modules are kept FP32 during from_pretrained even when
+    # dtype=BF16. The evaluated serving loader explicitly cast the WHOLE merged
+    # backbone after merging, so reproduce that cast after reload as well.
+    backbone = backbone.to(device=device, dtype=torch.bfloat16)
+    if type(backbone).__name__ == 'Qwen3_5Model':
+        model.multimodal = backbone
+        model.lm = backbone.language_model
+    else:
+        model.lm = backbone
+    text_config = model.lm.config
+    model.head = PointerHead(text_config.hidden_size, dp=meta['head_dim'])
+    model.head.load_state_dict(load_file(str(root / 'pointer_head.safetensors')))
+    model.head = model.head.to(device=device, dtype=torch.float32)
+    model.pad_id = tok.pad_token_id if tok.pad_token_id is not None else 0
+    model.hybrid = 'linear_attention' in set(getattr(text_config, 'layer_types', None) or [])
+    model.option_isolation = meta['option_isolation']
+    model.device = device
+    assert model.hybrid == meta['hybrid']
+    assert not (model.hybrid and model.option_isolation)
+    model.eval()
+    return tok, model
+
+
+def predict(tok, model, request, max_state=2048, max_branch=8192):
+    rec, meta = to_record(SystemOneRequest(**request))
+    enc = model.encode(tok, rec, strict=True, max_state=max_state, max_branch=max_branch)
+    with torch.inference_mode():
+        ps = [p.tolist() for p in model.probs(enc)]
+    answers = {}
+    for p, m in zip(ps, meta):
+        if m['type'] == 'choice':
+            answers[m['id']] = dict(type='choice', choice=m['keys'][max(range(len(p)), key=p.__getitem__)],
+                                    probabilities=dict(zip(m['keys'], p)))
+        elif m['type'] == 'noul':
+            answers[m['id']] = dict(type='noul', noul=p[1], probabilities={'false': p[0], 'true': p[1]})
+        else:
+            answers[m['id']] = dict(type='score', score=sum(i * v for i, v in enumerate(p)),
+                                    legend=m['legend'], probabilities={str(i): v for i, v in enumerate(p)})
+    return dict(answers=answers, input_tokens=len(enc['ids']))
+

package/src/neohorse_decision/_vendor/LICENSE

@@ -0,0 +1,203 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2026 Jared Palmer
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+

package/src/neohorse_decision/_vendor/NOTICE.md

@@ -0,0 +1,11 @@
+# Third-party attribution
+
+`model.py` and `schema.py` derive from Jared Palmer's Kev (https://github.com/jaredpalmer/kev), Copyright 2026 Jared Palmer, Apache-2.0; see LICENSE.
+
+The starting files were copied from the local runtime used for this checkpoint. Original file hashes are recorded in the accompanying model_manifest.json. The default API model identifier in `schema.py` was changed to `neohorse-jev`.
+
+NeoHorse modifications add `NEOHORSE_SHAPE_BUCKET` as the primary environment variable, retaining `KEV_SHAPE_BUCKET` as a lower-priority compatibility alias (default 64; 1 disables MPS sequence padding). Unused date-preprocessing helpers were removed, and stale comments and the package description were refreshed. Decision encoding, probability readout, and model weights are unchanged.
+
+The package does not fetch or import an external Kev distribution. Vendored code is versioned with this package, not automatically updated from upstream.
+
+This notice does not assign a new license to the model weights or training data.

package/src/neohorse_decision/_vendor/__init__.py

@@ -0,0 +1 @@
+"""Bundled decision primitives; third-party attribution is recorded in NOTICE.md."""

package/src/neohorse_decision/_vendor/model.py

@@ -0,0 +1,300 @@
+"""Decision model: causal LM backbone + block-causal branch mask + pointer readout."""
+# Modified for NeoHorse: environment naming with a legacy alias; refreshed comments.
+import math, os, re
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+# Reuse existing rarely-used Qwen special tokens as delimiters (state, q, opt, /opt, decide) so no
+# embedding rows need to be added/trained; LoRA adapts their meaning.
+SPECIAL = ["<|fim_prefix|>", "<|fim_middle|>", "<|box_start|>", "<|box_end|>", "<|fim_suffix|>"]
+MAX_STATE, MAX_BRANCH = 384, 1024
+
+
+def load_tokenizer(name, revision=None):
+    return AutoTokenizer.from_pretrained(name, revision=revision)
+
+
+_SPECIAL_RE = re.compile(r"<\|([A-Za-z0-9_]+)\|>")
+
+
+def user_tokens(tok, text):
+    """Tokenize caller-supplied text so it can never produce delimiter/control tokens (option boundaries are unforgeable).
+    The fast tokenizer ignores split_special_tokens, so `<|name|>` is rewritten to `<¦name¦>` before tokenizing."""
+    return tok(_SPECIAL_RE.sub(r"<¦\1¦>", text), add_special_tokens=False).input_ids
+
+
+OPT_NONE, OPT_DECIDE = -1, -2   # values of enc["opt"]: instruction/state tokens, and the <decide> token
+
+
+def encode(tok, rec, max_state=MAX_STATE, max_branch=MAX_BRANCH, strict=False, option_isolation=False):
+    """Pack one record: [<state> ...] then per-question [<q> instr <opt> o </opt>... <decide>].
+
+    Returns ids, seg (0 = state, k = question k), pos (branch positions restart after state),
+    decide_idx [Q], opt_idx [Q][K] (index of </opt> token for each option), opt (per-token option index within its
+    question: OPT_NONE for state/instruction, 0..K-1 for option spans, OPT_DECIDE for <decide>).
+
+    option_isolation=True: every option span is its own sub-branch (it sees state + instruction + itself only), all
+    option spans share the same position ids, and <decide> sits at one fixed position after the longest span. Then the
+    per-option representations and <decide>'s attention over them are permutation-invariant by construction.
+    """
+    state_tokens = user_tokens(tok, rec["state"])
+    if strict and len(state_tokens) + 1 > max_state:
+        raise ValueError(f"state exceeds {max_state} tokens: {len(state_tokens) + 1}")
+    S = [tok.convert_tokens_to_ids(SPECIAL[0])] + state_tokens[: max_state - 1]
+    ids, seg, pos, opt = list(S), [0] * len(S), list(range(len(S))), [OPT_NONE] * len(S)
+    q_id, o_id, c_id, d_id = (tok.convert_tokens_to_ids(t) for t in SPECIAL[1:])
+    decide_idx, opt_idx = [], []
+    for k, q in enumerate(rec["questions"], start=1):
+        instr = [q_id] + user_tokens(tok, q["instr"])
+        spans = [[o_id] + user_tokens(tok, o) + [c_id] for o in q["options"]]
+        br = instr + [t for sp in spans for t in sp] + [d_id]
+        if len(br) > max_branch - len(S):
+            raise ValueError(f"branch too long: {len(br)}")
+        base = len(ids); p0 = len(S)
+        br_opt = [OPT_NONE] * len(instr) + [j for j, sp in enumerate(spans) for _ in sp] + [OPT_DECIDE]
+        if option_isolation:
+            longest = max(len(sp) for sp in spans)
+            br_pos = list(range(p0, p0 + len(instr))) + [p0 + len(instr) + i for sp in spans for i in range(len(sp))] + [p0 + len(instr) + longest]
+        else:
+            br_pos = list(range(p0, p0 + len(br)))
+        ends, cursor = [], len(instr)
+        for sp in spans:
+            cursor += len(sp); ends.append(cursor - 1)
+        ids += br; seg += [k] * len(br); pos += br_pos; opt += br_opt
+        decide_idx.append(base + len(br) - 1); opt_idx.append([base + e for e in ends])
+    return {"ids": ids, "seg": seg, "pos": pos, "opt": opt, "option_isolation": option_isolation, "decide_idx": decide_idx, "opt_idx": opt_idx,
+            "labels": [q["label"] for q in rec["questions"]], "state_truncated": len(state_tokens) + 1 > max_state}
+
+
+def branch_mask(seg, device, dtype=torch.float32):
+    """attend(i,j) iff j<=i and (seg[j]==0 or seg[j]==seg[i]). Returns additive [1,1,L,L]."""
+    return branch_mask_batch([seg], device, dtype)
+
+
+def branch_mask_batch(segs, device, dtype=torch.float32, opts=None, length=None):
+    """Batched block-causal mask, additive [B,1,L,L], right-padded to the longest sequence.
+
+    Padded key positions are masked for every query; padded query rows keep the diagonal so no row is fully
+    masked (finfo.min, not -inf, so softmax stays finite either way). Real tokens never see pads because pads sit
+    after them (causal) and belong to no segment (-1).
+
+    opts (option isolation): within a question, an option-span token may attend to state, the instruction, and its own
+    span only; <decide> attends to everything in its question. Instruction tokens never see option spans (causal)."""
+    L = max(max(len(s) for s in segs), length or 0)
+    s = torch.full((len(segs), L), -1, device=device)
+    for b, seg in enumerate(segs):
+        s[b, : len(seg)] = torch.tensor(seg, device=device)
+    causal = torch.tril(torch.ones(L, L, dtype=torch.bool, device=device))
+    same = (s[:, None, :] == s[:, :, None]) | (s[:, None, :] == 0)
+    valid_key = (s != -1)[:, None, :]
+    allow = causal[None] & same & valid_key
+    if opts is not None:
+        o = torch.full((len(segs), L), OPT_NONE, device=device)
+        for b, op in enumerate(opts):
+            o[b, : len(op)] = torch.tensor(op, device=device)
+        key_is_option = (o[:, None, :] >= 0)
+        query_is_decide = (o[:, :, None] == OPT_DECIDE)
+        same_option = o[:, None, :] == o[:, :, None]
+        allow = allow & (~key_is_option | query_is_decide | same_option)
+    allow = allow | torch.eye(L, dtype=torch.bool, device=device)[None]
+    return torch.zeros(len(segs), L, L, dtype=dtype, device=device).masked_fill(~allow, torch.finfo(dtype).min)[:, None]
+
+
+def rows_of(enc):
+    """Split a packed encoding into its state and per-question branch rows.
+
+    Returns (state_ids, state_pos, rows) with rows[k] = {"ids", "pos", "decide", "opts"}: the branch tokens of question
+    k with their (already state-continuing) positions, and the readout offsets *within the branch*. Feeding
+    state + rows[k] as one causal row is equivalent to the packed block-causal form for that question, on any
+    architecture: the row contains exactly the tokens question k may attend to, in the same positions."""
+    seg = enc["seg"]; Ls = seg.count(0)
+    rows, start = [], Ls
+    for k, (d, oi) in enumerate(zip(enc["decide_idx"], enc["opt_idx"]), start=1):
+        end = d + 1                                    # <decide> is the last token of its branch
+        if seg[start] != k or seg[end - 1] != k: raise ValueError("branch layout mismatch")
+        rows.append({"ids": enc["ids"][start:end], "pos": enc["pos"][start:end], "decide": d - start, "opts": [o - start for o in oi]})
+        start = end
+    return enc["ids"][:Ls], enc["pos"][:Ls], rows
+
+
+class PointerHead(nn.Module):
+    def __init__(self, d, dp=256):
+        """dp = pointer dimension (head capacity knob)."""
+        super().__init__()
+        self.q, self.k = nn.Linear(d, dp), nn.Linear(d, dp)
+        self.scale = 1 / math.sqrt(dp)
+
+    def forward(self, h_decide, h_opts):  # [d], [K,d] -> logits [K]
+        return (self.k(h_opts) @ self.q(h_decide)) * self.scale
+
+
+class DecisionModel(nn.Module):
+    def __init__(self, name, tok, device, lora=None, revision=None, attn=None, head_dim=256, option_isolation=False, special_embeddings=False, lora_targets="all", dtype=torch.float32):
+        super().__init__()
+        # backbone only (no vocab head): we never generate text.
+        # eager on MPS/CPU (known-good with our float 4D mask); SDPA on CUDA (accepts arbitrary additive masks).
+        attn = attn or ("sdpa" if str(device).startswith("cuda") else "eager")
+        # dtype: fp32 for training and exact evaluation; bf16 is a serving option for large backbones (8B on a 32 GB Mac)
+        self.lm = AutoModelForCausalLM.from_pretrained(name, revision=revision, dtype=dtype, attn_implementation=attn).model
+        self.pad_id = tok.pad_token_id if tok.pad_token_id is not None else 0
+        # hybrid backbones (Qwen3.5: Gated DeltaNet layers, recurrent) cannot honour the block-causal mask, so every
+        # question runs as its own causal row continuing from the state (rows_of). Attention-only backbones keep the
+        # packed form; each question can attend only to its state and its own branch.
+        cfg = self.lm.config
+        self.hybrid = "linear_attention" in set(getattr(cfg, "layer_types", None) or [])
+        if self.hybrid and option_isolation: raise ValueError("option_isolation needs the packed mask; not available on hybrid backbones")
+        self.option_isolation = option_isolation
+        if lora:
+            from peft import LoraConfig, get_peft_model
+            extra = {"trainable_token_indices": {"embed_tokens": [tok.convert_tokens_to_ids(t) for t in SPECIAL]}} if special_embeddings else {}
+            targets = {"all": ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
+                       "dense": ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],   # "all" minus the DeltaNet projections on hybrids (retention ablation)
+                       "attn": ["q_proj", "k_proj", "v_proj", "o_proj"], "qv": ["q_proj", "v_proj"]}[lora_targets]
+            if self.hybrid and lora_targets in ("all", "attn"):
+                # Gated DeltaNet projections (transformers 5 names, verified on Qwen3_5TextModel); the mixer's out_proj too
+                targets = targets + ["in_proj_qkv", "in_proj_z", "in_proj_a", "in_proj_b", "out_proj"]
+            cfg = LoraConfig(task_type="FEATURE_EXTRACTION", r=lora, lora_alpha=2 * lora, lora_dropout=0.05, target_modules=targets, **extra)
+            self.lm = get_peft_model(self.lm, cfg)
+        self.head = PointerHead(self.lm.config.hidden_size, dp=head_dim)
+        self.device = device
+        self.to(device)
+
+    def encode(self, tok, rec, **kw):
+        """encode() with this model's option-isolation setting; use this from serving/eval code."""
+        return encode(tok, rec, option_isolation=self.option_isolation, **kw)
+
+    def hidden(self, enc):
+        return self.hidden_batch([enc])[0, : len(enc["ids"])]
+
+    # MPS sequence padding; 1 disables. The NeoHorse name takes precedence over the legacy alias.
+    SHAPE_BUCKET = int(os.environ.get("NEOHORSE_SHAPE_BUCKET", os.environ.get("KEV_SHAPE_BUCKET", "64")))
+
+    def hidden_batch(self, encs):
+        """[B, L_max, d] hidden states for a right-padded batch of encoded records. Pads are masked keys and sit after every
+        real token, so padding never changes a real token's hidden state (parity measured exact)."""
+        L = max(len(e["ids"]) for e in encs)
+        if str(self.device) == "mps" and not self.training: L = -(-L // self.SHAPE_BUCKET) * self.SHAPE_BUCKET
+        ids = torch.full((len(encs), L), self.pad_id, device=self.device)
+        pos = torch.zeros((len(encs), L), dtype=torch.long, device=self.device)
+        for b, e in enumerate(encs):
+            ids[b, : len(e["ids"])] = torch.tensor(e["ids"], device=self.device)
+            pos[b, : len(e["pos"])] = torch.tensor(e["pos"], device=self.device)
+        isolate = any(e.get("option_isolation") for e in encs)
+        if isolate and not all(e.get("option_isolation") for e in encs):
+            raise ValueError("cannot mix option-isolated and plain encodings in one batch")
+        lm_dtype = next(self.lm.parameters()).dtype
+        mask = branch_mask_batch([e["seg"] for e in encs], self.device, dtype=lm_dtype, opts=[e["opt"] for e in encs] if isolate else None, length=L)
+        return self.lm(input_ids=ids, position_ids=pos, attention_mask=mask).last_hidden_state.float()   # head stays fp32
+
+    def _readout(self, h, enc):
+        return [self.head(h[d], h[torch.tensor(oi, device=self.device)]) for d, oi in zip(enc["decide_idx"], enc["opt_idx"])]
+
+    def forward_rows_batch(self, encs):
+        """Row form: every question of every record is one causal row = state tokens + its branch tokens, right-padded
+        into a single batch. Returns the same nested logits as forward_batch. Exact isolation by construction (rows are
+        independent); the state is recomputed per row (Q x state tokens), which training accepts; serving uses the
+        prefix cache instead."""
+        rows, owners = [], []
+        for b, e in enumerate(encs):
+            S, Sp, brs = rows_of(e)
+            for r in brs:
+                rows.append((S + r["ids"], Sp + r["pos"], len(S) + r["decide"], [len(S) + o for o in r["opts"]])); owners.append(b)
+        L = max(len(ids) for ids, *_ in rows)
+        if str(self.device) == "mps" and not self.training: L = -(-L // self.SHAPE_BUCKET) * self.SHAPE_BUCKET
+        ids = torch.full((len(rows), L), self.pad_id, device=self.device)
+        pos = torch.zeros((len(rows), L), dtype=torch.long, device=self.device)
+        att = torch.zeros((len(rows), L), dtype=torch.long, device=self.device)
+        for i, (rid, rpos, _, _) in enumerate(rows):
+            ids[i, : len(rid)] = torch.tensor(rid, device=self.device); pos[i, : len(rpos)] = torch.tensor(rpos, device=self.device); att[i, : len(rid)] = 1
+        h = self.lm(input_ids=ids, position_ids=pos, attention_mask=att).last_hidden_state.float()
+        out = [[] for _ in encs]
+        for i, (b, (_, _, d, oi)) in enumerate(zip(owners, rows)):
+            out[b].append(self.head(h[i, d], h[i, torch.tensor(oi, device=self.device)]))
+        return out
+
+    def forward(self, enc):
+        """Returns list of logits tensors, one per question."""
+        if self.hybrid: return self.forward_rows_batch([enc])[0]
+        return self._readout(self.hidden(enc), enc)
+
+    def forward_batch(self, encs):
+        """List (per record) of lists (per question) of logits, from one padded forward pass."""
+        if self.hybrid: return self.forward_rows_batch(encs)
+        hs = self.hidden_batch(encs)
+        return [self._readout(hs[b], e) for b, e in enumerate(encs)]
+
+    @torch.no_grad()
+    def probs(self, enc):
+        return [F.softmax(z, -1).cpu() for z in self.forward(enc)]
+
+    # --- state-prefix reuse (serving): the state is encoded once, question branches attend to its cached keys/values.
+    # Exact by construction: branch tokens never attend to each other across questions (block-causal mask) and the state
+    # never sees the branches (causal), so the state's hidden states and KV are identical with or without the branches.
+
+    def _branch_rows_from_prefix(self, enc, cache):
+        """Hybrid serving: replicate the cached state once per question and run the branches as causal rows (exactly the
+        forward_rows_batch layout, minus the recomputed state). The cache is consumed (replicated, then extended)."""
+        S, Sp, rows = rows_of(enc); Q = len(rows)
+        cache.reorder_cache(torch.zeros(Q, dtype=torch.long, device=self.device))
+        W = max(len(r["ids"]) for r in rows)
+        if str(self.device) == "mps": W = -(-W // self.SHAPE_BUCKET) * self.SHAPE_BUCKET
+        ids = torch.full((Q, W), self.pad_id, device=self.device); pos = torch.zeros((Q, W), dtype=torch.long, device=self.device)
+        att = torch.zeros((Q, len(S) + W), dtype=torch.long, device=self.device)
+        for i, r in enumerate(rows):
+            ids[i, : len(r["ids"])] = torch.tensor(r["ids"], device=self.device); pos[i, : len(r["pos"])] = torch.tensor(r["pos"], device=self.device); att[i, : len(S) + len(r["ids"])] = 1
+        h = self.lm(input_ids=ids, position_ids=pos, attention_mask=att, past_key_values=cache, use_cache=True).last_hidden_state.float()
+        return [F.softmax(self.head(h[i, r["decide"]], h[i, torch.tensor(r["opts"], device=self.device)]), -1).cpu() for i, r in enumerate(rows)]
+
+    @torch.no_grad()
+    def prefix(self, enc):
+        """Run the state tokens only. Returns (n_state_tokens, kv cache, state hidden states [Ls, d])."""
+        from transformers import DynamicCache
+        Ls = enc["seg"].count(0)
+        ids = torch.tensor([enc["ids"][:Ls]], device=self.device); pos = torch.tensor([enc["pos"][:Ls]], device=self.device)
+        # the cache must know the layer types (hybrid backbones keep recurrent + conv states per DeltaNet layer)
+        out = self.lm(input_ids=ids, position_ids=pos, past_key_values=DynamicCache(config=self.lm.config), use_cache=True)
+        return Ls, out.past_key_values, out.last_hidden_state[0].float()
+
+    @torch.no_grad()
+    def probs_and_prefix(self, enc):
+        """One full pass that also returns the state prefix (KV cropped to the state, state hidden states): a cache miss
+        costs a single forward pass, not two."""
+        from transformers import DynamicCache
+        Ls = enc["seg"].count(0)
+        if self.hybrid:
+            # recurrent layers cannot be cropped back to the state, so a hybrid miss is state pass + branch rows (the
+            # state pass is kept as the reusable prefix by running it twice? no: copy the cache before consuming it)
+            Ls, cache, h_state = self.prefix(enc)
+            import copy
+            return self._branch_rows_from_prefix(enc, copy.deepcopy(cache)), (Ls, cache, h_state)
+        ids = torch.tensor([enc["ids"]], device=self.device); pos = torch.tensor([enc["pos"]], device=self.device)
+        dt = next(self.lm.parameters()).dtype
+        mask = branch_mask_batch([enc["seg"]], self.device, dtype=dt, opts=[enc["opt"]] if enc.get("option_isolation") else None)
+        out = self.lm(input_ids=ids, position_ids=pos, attention_mask=mask, past_key_values=DynamicCache(config=self.lm.config), use_cache=True)
+        h = out.last_hidden_state[0].float()
+        out.past_key_values.crop(-(len(enc["ids"]) - Ls))     # keep the state only (negative = drop that many trailing tokens; positive form deprecated in transformers 5)
+        return [F.softmax(z, -1).cpu() for z in self._readout(h, enc)], (Ls, out.past_key_values, h[:Ls].clone())
+
+    @torch.no_grad()
+    def probs_with_prefix(self, enc, prefix):
+        """probs() for a record whose state tokens equal the cached prefix's; only the branches run. The cache is cropped
+        back to the state afterwards so it can be reused."""
+        Ls, cache, h_state = prefix
+        if enc["seg"].count(0) != Ls: raise ValueError("prefix does not match this record's state")
+        if self.hybrid:
+            import copy
+            return self._branch_rows_from_prefix(enc, copy.deepcopy(cache))   # the stored prefix stays pristine
+        ids = torch.tensor([enc["ids"][Ls:]], device=self.device); pos = torch.tensor([enc["pos"][Ls:]], device=self.device)
+        dt = next(self.lm.parameters()).dtype
+        mask = branch_mask_batch([enc["seg"]], self.device, dtype=dt, opts=[enc["opt"]] if enc.get("option_isolation") else None)[:, :, Ls:, :]
+        try:
+            out = self.lm(input_ids=ids, position_ids=pos, past_key_values=cache, attention_mask=mask, use_cache=True)
+            h = torch.cat([h_state, out.last_hidden_state[0].float()], 0)
+        finally:
+            cache.crop(-(len(enc["ids"]) - Ls))
+        return [F.softmax(z, -1).cpu() for z in self._readout(h, enc)]
+
+    def trainable_parameters(self):
+        return [p for p in self.parameters() if p.requires_grad]

package/src/neohorse_decision/_vendor/schema.py

@@ -0,0 +1,112 @@
+"""TypeSafe-compatible request/response shapes (POST /v1/systemone) mapped onto the single pointer primitive.
+
+Noul   -> 2 options [false, true];            answer = p(true)
+Choice -> options 'name' or 'name: desc';      answer = argmax, probabilities by name, confidence
+Score  -> options = ordered level descriptions; answer = expected level, legend, probabilities by index
+"""
+# Modified for NeoHorse: default model identifier; unused date preprocessing removed.
+from typing import Any, Literal, Union
+from pydantic import BaseModel, Field, model_validator
+
+JSONContent = Union[str, dict, list, int, float, bool, None]
+MAX_OPTIONS = 255
+
+
+class Noul(BaseModel):
+    type: Literal["noul"]
+    instructions: JSONContent
+    criteria: dict[str, JSONContent] | None = None
+
+
+class Choice(BaseModel):
+    type: Literal["choice"]
+    instructions: JSONContent
+    criteria: dict[str, JSONContent]
+
+    @model_validator(mode="after")
+    def _check(self):
+        if not 1 <= len(self.criteria) <= MAX_OPTIONS: raise ValueError(f"criteria must have 1..{MAX_OPTIONS} options")
+        return self
+
+
+class Score(BaseModel):
+    type: Literal["score"]
+    instructions: JSONContent
+    criteria: list[JSONContent] = Field(min_length=2, max_length=MAX_OPTIONS)
+
+
+Question = Union[Noul, Choice, Score]
+
+
+class SystemOneRequest(BaseModel):
+    state: JSONContent
+    model: str = "neohorse-jev"
+    questions: dict[str, Question] = Field(min_length=1)
+
+
+def render(v: JSONContent, indent: int = 0) -> str:
+    """Flatten str | object | array into text the model sees. Field names are kept as labels."""
+    pad = "  " * indent
+    if v is None: return ""
+    if isinstance(v, (str, int, float, bool)): return str(v)
+    if isinstance(v, list): return "\n".join(f"{pad}- {render(x, indent + 1).lstrip()}" for x in v)
+    return "\n".join(f"{pad}{k}:\n{render(x, indent + 1)}" if isinstance(x, (dict, list)) else f"{pad}{k}: {render(x)}" for k, x in v.items())
+
+
+def option_text(name: str, desc: JSONContent) -> str:
+    return name if desc is None or desc == "" else f"{name}: {render(desc)}"
+
+
+def to_record(req: SystemOneRequest):
+    """-> internal record for encode(), plus per-question metadata to map probabilities back."""
+    qs, meta = [], []
+    for qid, q in req.questions.items():
+        instr = render(q.instructions)
+        if q.type == "noul":
+            c = q.criteria or {}
+            opts = [option_text("no", c.get("false")), option_text("yes", c.get("true"))]
+            meta.append({"id": qid, "type": "noul"})
+        elif q.type == "choice":
+            opts = [option_text(k, v) for k, v in q.criteria.items()]
+            meta.append({"id": qid, "type": "choice", "keys": list(q.criteria.keys())})
+        else:
+            opts = [render(x) for x in q.criteria]
+            meta.append({"id": qid, "type": "score", "legend": {str(i): render(x) for i, x in enumerate(q.criteria)}})
+        qs.append({"instr": instr, "options": opts, "label": 0})
+    return {"state": render(req.state), "questions": qs}, meta
+
+
+def choice_confidence(p: list[float]) -> float:
+    K = len(p)
+    return 1.0 if K == 1 else (max(p) - 1 / K) / (1 - 1 / K)
+
+
+def score_confidence(p: list[float]) -> float:
+    """Approximation of TypeSafe's 'distance from the modal level' statistic (exact formula unpublished):
+    1 - E|level - mode| / (L - 1)."""
+    L = len(p); mode = max(range(L), key=lambda i: p[i])
+    return 1.0 - sum(pi * abs(i - mode) for i, pi in enumerate(p)) / (L - 1)
+
+
+def r2(x: float) -> float:
+    return round(float(x), 2)
+
+
+def to_answers(probs: list[list[float]], meta: list[dict]) -> dict[str, Any]:
+    out = {}
+    for p, m in zip(probs, meta):
+        if m["type"] == "noul":
+            out[m["id"]] = {"type": "noul", "noul": r2(p[1])}
+        elif m["type"] == "choice":
+            dist = {k: r2(v) for k, v in zip(m["keys"], p)}
+            out[m["id"]] = {"type": "choice", "choice": m["keys"][max(range(len(p)), key=lambda i: p[i])], "confidence": r2(choice_confidence(p)), "probabilities": dist}
+        else:
+            score = sum(i * pi for i, pi in enumerate(p))
+            out[m["id"]] = {"type": "score", "score": r2(score), "legend": m["legend"], "probabilities": {str(i): r2(v) for i, v in enumerate(p)}, "confidence": r2(score_confidence(p))}
+    return out
+
+
+def output_tokens(tok, answers: dict) -> int:
+    """Billing-style figure: tokens of the serialised answers. Not a measure of generation (there is none)."""
+    import json
+    return len(tok(json.dumps(answers), add_special_tokens=False).input_ids)

package/src/neohorse_decision/cli.py

@@ -0,0 +1,25 @@
+import argparse
+import json
+import os
+from pathlib import Path
+from .engine import DecisionEngine
+
+
+def main():
+    p = argparse.ArgumentParser()
+    p.add_argument('command', choices=['predict', 'serve'])
+    p.add_argument('--model-dir', required=True)
+    p.add_argument('--device', default='cuda')
+    p.add_argument('--request')
+    p.add_argument('--host', default='127.0.0.1')
+    p.add_argument('--port', default=8080, type=int)
+    a = p.parse_args()
+    if a.command == 'predict' and not a.request:
+        p.error('predict requires --request')
+    engine = DecisionEngine(a.model_dir, a.device)
+    if a.command == 'predict':
+        print(json.dumps(engine.predict(json.loads(Path(a.request).read_text())), ensure_ascii=False, indent=2))
+    else:
+        import uvicorn
+        from .server import create_app
+        uvicorn.run(create_app(engine, os.environ.get('NEOHORSE_API_KEY')), host=a.host, port=a.port, workers=1)

package/src/neohorse_decision/engine.py

@@ -0,0 +1,46 @@
+import json
+import threading
+from ._inference import load_bundle, predict
+from ._vendor.schema import SystemOneRequest, to_record
+from .systemone import MODEL_ID, MODEL_NAMES
+
+
+class BusyError(RuntimeError):
+    pass
+
+
+class DecisionEngine:
+    """One serialized GPU worker. Requests are never silently truncated."""
+    model_id = MODEL_ID
+
+    def __init__(self, model_dir, device='cuda', max_state=2048, max_branch=8192,
+                 max_questions=16, max_tokens=32768):
+        self.model_dir, self.device = model_dir, device
+        self.tokenizer, self.model = load_bundle(model_dir, device)
+        self.max_state, self.max_branch = max_state, max_branch
+        self.max_questions, self.max_tokens = max_questions, max_tokens
+        self._lock = threading.Lock()
+
+    def predict(self, request):
+        if len(json.dumps(request, ensure_ascii=False).encode()) > 1024 * 1024:
+            raise ValueError('Request exceeds 1 MiB')
+        req = SystemOneRequest(**request)
+        if req.model not in MODEL_NAMES:
+            raise ValueError('Unknown model; use ' + self.model_id)
+        if len(req.questions) > self.max_questions:
+            raise ValueError('Too many questions')
+        if not self._lock.acquire(blocking=False):
+            raise BusyError('Model busy; retry later')
+        try:
+            rec, _ = to_record(req)
+            enc = self.model.encode(self.tokenizer, rec, strict=True,
+                                    max_state=self.max_state, max_branch=self.max_branch)
+            # Hybrid execution repeats the state in each independent question row.
+            state_length = sum(s == 0 for s in enc['seg'])
+            effective_tokens = len(enc['ids']) + (len(req.questions) - 1) * state_length
+            if effective_tokens > self.max_tokens:
+                raise ValueError('Request exceeds total expanded token budget')
+            result = predict(self.tokenizer, self.model, request, self.max_state, self.max_branch)
+            return dict(model=self.model_id, **result)
+        finally:
+            self._lock.release()

package/src/neohorse_decision/image_input.py

@@ -0,0 +1,41 @@
+"""Bounded inline image decoding. Never fetch URLs or open caller-supplied paths."""
+import base64
+import binascii
+from io import BytesIO
+import warnings
+
+MAX_IMAGE_BYTES = 4 * 1024 * 1024
+MAX_PIXELS = 4 * 1024 * 1024
+MIME_FORMAT = {'image/png': 'PNG', 'image/jpeg': 'JPEG', 'image/webp': 'WEBP'}
+
+def decode_image(value):
+    from PIL import Image, UnidentifiedImageError
+    if not isinstance(value, str) or not value.startswith('data:') or ',' not in value:
+        raise ValueError('image must be a data:image/png|jpeg|webp;base64,... string; URLs and paths are not supported')
+    header, encoded = value.split(',', 1)
+    mime = header[5:].removesuffix(';base64')
+    if header != 'data:' + mime + ';base64' or mime not in MIME_FORMAT:
+        raise ValueError('image supports only base64 PNG, JPEG or WebP data URLs')
+    if len(encoded) > 4 * ((MAX_IMAGE_BYTES + 2) // 3):
+        raise ValueError('Encoded image exceeds 4 MiB decoded-byte limit')
+    try:
+        data = base64.b64decode(encoded, validate=True)
+    except (binascii.Error, ValueError) as exc:
+        raise ValueError('Invalid image base64') from exc
+    if not data or len(data) > MAX_IMAGE_BYTES:
+        raise ValueError('Image must contain 1 byte to 4 MiB')
+    try:
+        with warnings.catch_warnings():
+            warnings.simplefilter('error', Image.DecompressionBombWarning)
+            with Image.open(BytesIO(data)) as source:
+                if source.format != MIME_FORMAT[mime]:
+                    raise ValueError('Image MIME type does not match decoded format')
+                if source.width * source.height > MAX_PIXELS:
+                    raise ValueError('Image exceeds 4 megapixels')
+                if getattr(source, 'n_frames', 1) != 1:
+                    raise ValueError('Animated/multi-frame images are not supported')
+                source.load()
+                return source.convert('RGB')
+    except (UnidentifiedImageError, OSError, Image.DecompressionBombError,
+            Image.DecompressionBombWarning) as exc:
+        raise ValueError('Invalid, truncated or oversized image') from exc

package/src/neohorse_decision/server.py

@@ -0,0 +1,90 @@
+import hmac
+import json
+import threading
+from fastapi import FastAPI, HTTPException, Request
+from fastapi.responses import JSONResponse
+from pydantic import ValidationError
+from starlette.concurrency import run_in_threadpool
+from . import __version__
+from .engine import BusyError
+from .systemone import validate_request, format_response, with_confidence
+from .image_input import decode_image
+
+
+def create_app(engine, api_key=None, *, vision_engine=None):
+    app = FastAPI(title='NeoHorse Decision', version=__version__)
+    if vision_engine is None and hasattr(getattr(engine, 'model', None), 'multimodal'):
+        from .vision import VisionDecisionEngine
+        vision_engine = VisionDecisionEngine(engine.model_dir, engine.device, text_engine=engine)
+    image_lock = threading.Lock()
+
+    def image_predict(value, encoded):
+        if not image_lock.acquire(blocking=False):
+            raise BusyError('Image worker busy')
+        try:
+            image = decode_image(encoded)
+            try:
+                return vision_engine.predict(value, image)
+            finally:
+                image.close()
+        finally:
+            image_lock.release()
+
+    @app.get('/health')
+    def health():
+        return {'status': 'ready', 'model': engine.model_id,
+                'input_modalities': ['text', 'image'] if vision_engine is not None else ['text']}
+
+    async def handle(request: Request, systemone=False):
+        if api_key and not hmac.compare_digest(request.headers.get('authorization', ''), 'Bearer ' + api_key):
+            raise HTTPException(401, 'Invalid bearer token')
+        body = bytearray()
+        async for chunk in request.stream():
+            body.extend(chunk)
+            if len(body) > 8 * 1024 * 1024:
+                raise HTTPException(413, 'Request exceeds 8 MiB')
+        try:
+            try:
+                value = json.loads(body)
+            except (ValueError, UnicodeError, RecursionError):
+                if len(body) > 1024 * 1024:
+                    raise HTTPException(413, 'Text request exceeds 1 MiB')
+                raise ValueError('Invalid JSON')
+            if not isinstance(value, dict):
+                raise ValueError('Request must be an object')
+            if any(k in value for k in ('images', 'image_url', 'image_base64', 'video')):
+                raise ValueError('Use the single top-level image data URL field')
+            has_image = 'image' in value
+            encoded = value.pop('image', None)
+            if (not has_image and len(body) > 1024 * 1024) or len(json.dumps(value, ensure_ascii=False).encode()) > 1024 * 1024:
+                raise HTTPException(413, 'Text request fields exceed 1 MiB')
+            if has_image:
+                if vision_engine is None:
+                    raise ValueError('This model has no enabled vision adapter')
+                if not isinstance(value.get('questions'), dict) or len(value['questions']) != 1:
+                    raise ValueError('Image requests require exactly one question')
+            if systemone:
+                value = validate_request(value)
+            if has_image:
+                result = await run_in_threadpool(image_predict, value, encoded)
+            else:
+                result = await run_in_threadpool(engine.predict, value)
+            if systemone:
+                result = format_response(result, engine.tokenizer)
+                return JSONResponse(result, headers={
+                    'X-NeoHorse-Confidence': 'local-distribution-statistic-v1',
+                    'X-NeoHorse-Usage': 'local-tokenizer-not-jev-billing'})
+            return JSONResponse(with_confidence(result), headers={
+                'X-NeoHorse-Confidence': 'local-distribution-statistic-v1'})
+        except BusyError:
+            return JSONResponse({'detail': 'Model busy; retry later'}, status_code=529 if systemone else 429, headers={'Retry-After': '1'})
+        except (ValueError, TypeError, ValidationError) as e:
+            raise HTTPException(422, str(e)) from e
+    @app.post('/v1/decision')
+    async def decision(request: Request):
+        return await handle(request)
+
+    @app.post('/v1/systemone')
+    async def systemone(request: Request):
+        return await handle(request, systemone=True)
+    return app

package/src/neohorse_decision/systemone.py

@@ -0,0 +1,56 @@
+"""System One wire adapter; not official Jev calibration or billing."""
+import json
+from ._vendor.schema import SystemOneRequest, choice_confidence, score_confidence
+
+MODEL_ID = 'NeoHorse-Jev-4B'
+MODEL_NAMES = {MODEL_ID, 'TokenRhythm/' + MODEL_ID, 'neohorse-jev',
+               'NeoHorse-JEV-4B', 'TokenRhythm/NeoHorse-JEV-4B'}
+
+def validate_request(value):
+    if not isinstance(value.get('model'), str) or value['model'] not in MODEL_NAMES:
+        raise ValueError('model is required; use NeoHorse-Jev-4B (not a Jev model alias)')
+    if not isinstance(value.get('state'), (str, dict, list)):
+        raise ValueError('state must be string, object or array')
+    req = SystemOneRequest(**{**value, 'model': 'neohorse-jev'})
+    def description(v):
+        return v is None or isinstance(v, (str, dict, list))
+    for q in req.questions.values():
+        if not description(q.instructions):
+            raise ValueError('instructions must be string, object, array or null')
+        if q.type == 'score':
+            if not 2 <= len(q.criteria) <= 10:
+                raise ValueError('System One Score requires 2..10 levels')
+            values = q.criteria
+        elif q.type == 'noul':
+            if q.criteria and not set(q.criteria) <= {'true', 'false'}:
+                raise ValueError('Noul criteria keys must be true or false')
+            values = (q.criteria or {}).values()
+        else:
+            values = q.criteria.values()
+        if any(not description(v) for v in values):
+            raise ValueError('criteria descriptions must be string, object, array or null')
+    return req.model_dump()
+
+def with_confidence(result):
+    """Add the same local confidence statistic without removing native fields."""
+    answers = {}
+    for name, answer in result['answers'].items():
+        a = dict(answer)
+        if a['type'] in ('choice', 'score'):
+            p = list(a['probabilities'].values())
+            confidence = choice_confidence(p) if a['type'] == 'choice' else score_confidence(p)
+            a['confidence'] = min(1.0, max(0.0, confidence))
+        answers[name] = a
+    return {**result, 'answers': answers}
+
+def format_response(result, tokenizer):
+    answers = with_confidence(result)['answers']
+    for name, a in answers.items():
+        if a['type'] == 'noul':
+            answers[name] = {'type': 'noul', 'noul': a['noul']}
+    output_tokens = len(tokenizer(json.dumps(answers, ensure_ascii=False, separators=(',', ':')),
+                                  add_special_tokens=False).input_ids)
+    usage = {'input_tokens': result['input_tokens'], 'output_tokens': output_tokens}
+    if 'image_tokens' in result:
+        usage['image_tokens'] = result['image_tokens']
+    return {'model': MODEL_ID, 'answers': answers, 'usage': usage}

package/src/neohorse_decision/vision.py

@@ -0,0 +1,75 @@
+"""Local image decisions using the unified NeoHorse multimodal backbone.
+
+Local PIL image inference; HTTP transport is implemented in image_input.py/server.py.
+"""
+import json
+from pathlib import Path
+import torch
+from neohorse_decision import DecisionEngine
+from neohorse_decision.engine import BusyError
+from neohorse_decision._vendor.schema import SystemOneRequest, to_record
+from neohorse_decision.systemone import MODEL_ID, MODEL_NAMES, with_confidence
+
+
+class VisionDecisionEngine:
+    """One local image and one Choice/Noul/Score question per visual request."""
+
+    def __init__(self, model_dir, device='cuda', *, text_engine=None):
+        from transformers.models.qwen2_vl.image_processing_pil_qwen2_vl import Qwen2VLImageProcessorPil
+        self.root=Path(model_dir)
+        self.text=text_engine if text_engine is not None else DecisionEngine(self.root,device=device)
+        self.device=device
+        if not hasattr(self.text.model,'multimodal'):
+            raise ValueError('VisionDecisionEngine requires the unified multimodal backbone')
+        self.wrapper=self.text.model.multimodal
+        config=self.wrapper.config
+        self.visual_tensor_count=len(self.wrapper.visual.state_dict())
+        self.processor=Qwen2VLImageProcessorPil.from_pretrained(self.root/'backbone',local_files_only=True,
+            size={'shortest_edge':65536,'longest_edge':1048576})
+        for token,expected in (('<|image_pad|>',config.image_token_id),('<|vision_start|>',config.vision_start_token_id),('<|vision_end|>',config.vision_end_token_id)):
+            if self.text.tokenizer.convert_tokens_to_ids(token)!=expected:
+                raise ValueError('Vision token ID mismatch: '+token)
+
+    def predict(self,request,image=None,*,through_wrapper=False):
+        if image is None and not through_wrapper:
+            return with_confidence(self.text.predict(request))
+        req=SystemOneRequest(**request)
+        if req.model not in MODEL_NAMES:raise ValueError('Unknown model; use '+MODEL_ID)
+        if len(req.questions)!=1:raise ValueError('Visual adapter currently supports one question per request')
+        if len(json.dumps(request,ensure_ascii=False).encode())>1024*1024:raise ValueError('Request exceeds 1 MiB')
+        if image is not None:
+            from PIL import Image
+            if not isinstance(image,Image.Image):raise TypeError('image must be a locally decoded PIL image')
+            if image.width*image.height>4194304:raise ValueError('Source image exceeds 4 megapixels')
+        if not self.text._lock.acquire(blocking=False):raise BusyError('Model busy; retry later')
+        try:
+            rec,metadata=to_record(req)
+            enc=self.text.model.encode(self.text.tokenizer,rec,strict=True,max_state=self.text.max_state,max_branch=self.text.max_branch)
+            prefix=[];extra={};image_tokens=0
+            if image is not None:
+                batch=self.processor(images=image.convert('RGB'),return_tensors='pt')
+                image_tokens=int(batch['image_grid_thw'][0].prod())//self.processor.merge_size**2
+                if not 0<image_tokens<=1024:raise ValueError('Processed image exceeds 1024 visual tokens')
+                prefix=[self.wrapper.config.vision_start_token_id]+[self.wrapper.config.image_token_id]*image_tokens+[self.wrapper.config.vision_end_token_id]
+                extra={k:batch[k].to(self.device) for k in ('pixel_values','image_grid_thw')}
+            if len(enc['ids'])+len(prefix)>12288:raise ValueError('Expanded multimodal input exceeds 12288 tokens')
+            ids=torch.tensor([[enc['ids'][0]]+prefix+enc['ids'][1:]],device=self.device)
+            # Do not carry positional state across independent images/requests.
+            self.wrapper.rope_deltas=None
+            with torch.inference_mode():
+                h=self.wrapper(input_ids=ids,attention_mask=torch.ones_like(ids),
+                    mm_token_type_ids=(ids==self.wrapper.config.image_token_id).long(),use_cache=False,**extra).last_hidden_state[0].float()
+                delta=len(prefix)
+                z=self.text.model.head(h[enc['decide_idx'][0]+delta],h[torch.tensor(enc['opt_idx'][0],device=self.device)+delta])
+                p=z.softmax(-1).cpu().tolist()
+            m=metadata[0]
+            if not all(torch.isfinite(torch.tensor(p))):raise RuntimeError('Non-finite output probabilities')
+            if m['type']=='choice':
+                answer=dict(type='choice',choice=m['keys'][max(range(len(p)),key=p.__getitem__)],probabilities=dict(zip(m['keys'],p)))
+            elif m['type']=='noul':
+                answer=dict(type='noul',noul=p[1],probabilities={'false':p[0],'true':p[1]})
+            else:
+                answer=dict(type='score',score=sum(i*v for i,v in enumerate(p)),legend=m['legend'],probabilities={str(i):v for i,v in enumerate(p)})
+            return with_confidence(dict(model=MODEL_ID,answers={m['id']:answer},input_tokens=len(enc['ids'])+delta,image_tokens=image_tokens))
+        finally:
+            self.text._lock.release()

pointer_head.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:467ae48977b5c4bf87dd1db29021199e0c3401c52c5b57a1eccde679c0bade26
+size 5245232

tokenizer/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

tokenizer/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523
+size 19989325

tokenizer/tokenizer_config.json

@@ -0,0 +1,32 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": true,
+  "local_files_only": true,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

vision/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2026 Alibaba Cloud
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

vision/README.md

@@ -0,0 +1,28 @@
+# Image Inference Examples
+
+See the [Deployment guide](https://huggingface.co/TokenRhythm/NeoHorse-Jev-4B/blob/main/DEPLOYMENT.md) for environment setup, image request formats, response fields, limits, and error handling.
+
+| File | Purpose |
+| --- | --- |
+| `example.py` | Run image-based decisions locally |
+| `example_request.json` | Sample question; replace it with your own single-question request |
+| `http_example.py` | Encode a local image and send it to the HTTP API |
+| `predictor.py` | Import wrapper for the installed `neohorse_decision.vision.VisionDecisionEngine` |
+| `base_vision_provenance.json` | Record the source of the vision weights |
+| `verification.json` | Record basic vision smoke tests and consistency checks |
+| `LICENSE` | License for the vision components |
+
+The language and vision weights are stored together in `backbone/`. The decision head is stored in `pointer_head.safetensors` at the model root. The vision weights come from Qwen3.5-4B and have not undergone additional vision training. The hashes of separate vision files in the provenance record are for traceability only; those files do not need to be loaded separately at runtime.
+
+After installing the matching runtime package, set `MODEL_DIR` to the complete downloaded model directory. You can then run these commands from any working directory:
+
+```bash
+CUDA_VISIBLE_DEVICES=0 python "$MODEL_DIR/vision/example.py" \
+  --model-dir "$MODEL_DIR" --image /path/to/image.png \
+  --request "$MODEL_DIR/vision/example_request.json"
+
+python "$MODEL_DIR/vision/http_example.py" --image /path/to/image.png \
+  --base-url http://127.0.0.1:8080 --endpoint systemone
+```
+
+Each image request supports one static image and one question, using Noul, Choice, or Score. The basic vision verification records are not a general visual capability evaluation.

vision/__init__.py

@@ -0,0 +1,2 @@
+"""Optional local-image adapter shipped alongside the model, not the HTTP runtime."""
+from .predictor import VisionDecisionEngine

vision/base_vision_provenance.json

@@ -0,0 +1,2795 @@
+{
+  "version": "vision-v1",
+  "base_model": "Qwen/Qwen3.5-4B",
+  "source_index_sha256": "cf3f798ee02ba45f9622aa8892a47369ab667d0afbf154ee7c2212de42e6302d",
+  "copied_metadata": {
+    "config.json": "ddc63e1c717afa86c865bb5e01313d89d72bb53b97ad4a8a03ba8510c0621670",
+    "preprocessor_config.json": "27225450ac9c6529872ee1924fcb0962ff5634834f817040f444118116f4e516",
+    "LICENSE": "50cbab8a892c5f2993b8c7351a99182507472def3b1374558308605d99b86b32"
+  },
+  "tensor_count": 297,
+  "parameter_count": 333514240,
+  "artifact_sha256": "86a8e4f2a5373a3d1f2151216b549791528ba6e0aa87da778e99c20392b74f98",
+  "tensors": [
+    {
+      "source_key": "model.visual.blocks.0.attn.proj.bias",
+      "stored_key": "blocks.0.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "35e988e89cb4c9d2de166a7243a57da738cede31bdc7ffea79db195ab87f9c71"
+    },
+    {
+      "source_key": "model.visual.blocks.0.attn.proj.weight",
+      "stored_key": "blocks.0.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "78cca0b25e1e134deaf8dfc8c64a3fd06bde4cac1a29ebbf53657aca8a60e78b"
+    },
+    {
+      "source_key": "model.visual.blocks.0.attn.qkv.bias",
+      "stored_key": "blocks.0.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "7daad53c4533bbc09381760e5066eeee505137254ff14b6e16c2b7f9936958a9"
+    },
+    {
+      "source_key": "model.visual.blocks.0.attn.qkv.weight",
+      "stored_key": "blocks.0.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "8590f79f0af5e1b8b2e69baa28795df9986bc4a6f338eb06a07ad51cf8a796e7"
+    },
+    {
+      "source_key": "model.visual.blocks.0.mlp.linear_fc1.bias",
+      "stored_key": "blocks.0.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "49b76a4ae4cf5e8c9ca03ab2125655e9697e48178752d00781e0e2880a4ca04b"
+    },
+    {
+      "source_key": "model.visual.blocks.0.mlp.linear_fc1.weight",
+      "stored_key": "blocks.0.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "9e81579c6093752a9b710c68bb995fc2f2ca51a67068a363621781f9132c20be"
+    },
+    {
+      "source_key": "model.visual.blocks.0.mlp.linear_fc2.bias",
+      "stored_key": "blocks.0.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "39011ac5434dc5db9edab8ab64e7cc2b6dde8f5ee4c43bbfa8e4a9c9b1c03a8c"
+    },
+    {
+      "source_key": "model.visual.blocks.0.mlp.linear_fc2.weight",
+      "stored_key": "blocks.0.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "05a82c58e7d798848736214ac4b4df9527e8e3f8d2f6502ac91b423c072a7436"
+    },
+    {
+      "source_key": "model.visual.blocks.0.norm1.bias",
+      "stored_key": "blocks.0.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "6c7c81c3d295c17e4496fc40009859df63af957b498c30cad9c214b91456bf57"
+    },
+    {
+      "source_key": "model.visual.blocks.0.norm1.weight",
+      "stored_key": "blocks.0.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "0928383981c62db2ce3ffd152cf9f64b84d1dd195546ef7d90bdfef39fbf1caa"
+    },
+    {
+      "source_key": "model.visual.blocks.0.norm2.bias",
+      "stored_key": "blocks.0.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "c18e1278761361763c4cc471083ba75df6bf6cd96e21f2ba4c48dc092546c9a7"
+    },
+    {
+      "source_key": "model.visual.blocks.0.norm2.weight",
+      "stored_key": "blocks.0.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "8a238107a8b61536f2e5b98d7846f8aaebac2919133175fcbf0369fb80174998"
+    },
+    {
+      "source_key": "model.visual.blocks.1.attn.proj.bias",
+      "stored_key": "blocks.1.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "8299dee2cfc5fde55fe5344ed362a850b1fba1f92a76ba0972e2ceb678cfb3a6"
+    },
+    {
+      "source_key": "model.visual.blocks.1.attn.proj.weight",
+      "stored_key": "blocks.1.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "5bb8eee8d943ab25d9bc89e807d5a78e7437be7f34492f4b41169cdba254cde1"
+    },
+    {
+      "source_key": "model.visual.blocks.1.attn.qkv.bias",
+      "stored_key": "blocks.1.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "9a266b396e7792f6869dfc1dd21ea863926aa62cceb79219e146dba35b5436cd"
+    },
+    {
+      "source_key": "model.visual.blocks.1.attn.qkv.weight",
+      "stored_key": "blocks.1.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "f0462fd9bd00f5690f047328af2268dd7640802dac22c46a1e669581358eb229"
+    },
+    {
+      "source_key": "model.visual.blocks.1.mlp.linear_fc1.bias",
+      "stored_key": "blocks.1.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "7cfa4b2269ec4d386a42dda3edff2da7aad8286b0bbfbaa213c83cd5c8b09c89"
+    },
+    {
+      "source_key": "model.visual.blocks.1.mlp.linear_fc1.weight",
+      "stored_key": "blocks.1.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "56c7abf157f9a58f5c05ce6bbbf6720ce96aee15818f18cb479412146fc1e5ba"
+    },
+    {
+      "source_key": "model.visual.blocks.1.mlp.linear_fc2.bias",
+      "stored_key": "blocks.1.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "65162e4fd05745ccd3466431d26ccd304459dfbcb5c97633025bf3c3a9077d86"
+    },
+    {
+      "source_key": "model.visual.blocks.1.mlp.linear_fc2.weight",
+      "stored_key": "blocks.1.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "354b3c467f3fdd8efe7a2326c7dc553cea0a6f2479e0f834741105649c6abe83"
+    },
+    {
+      "source_key": "model.visual.blocks.1.norm1.bias",
+      "stored_key": "blocks.1.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "2d04662565c9020bd762a273e9aced97a3cd47e4361f3ad4a4c25ff5db2423af"
+    },
+    {
+      "source_key": "model.visual.blocks.1.norm1.weight",
+      "stored_key": "blocks.1.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "cc7d59e07d52aa2bb7300a0d6e5046026ed7ac45989d6831a14fd9b1934eb537"
+    },
+    {
+      "source_key": "model.visual.blocks.1.norm2.bias",
+      "stored_key": "blocks.1.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "70ff4e7bcf3cec5152c5a60d648a426d1a88c94bb1b643109a50cacf8d0c0d28"
+    },
+    {
+      "source_key": "model.visual.blocks.1.norm2.weight",
+      "stored_key": "blocks.1.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "57141cf054485c6b5c69e9b29ea4bda9131d0dad6ad04d071a550d7453563138"
+    },
+    {
+      "source_key": "model.visual.blocks.10.attn.proj.bias",
+      "stored_key": "blocks.10.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "b6ba11d898bf0c80afcfa4e14841d624b80963a76c5591c936f7cd6f908ad446"
+    },
+    {
+      "source_key": "model.visual.blocks.10.attn.proj.weight",
+      "stored_key": "blocks.10.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "59da85b56f1d458e6f1c999943758a4d1bb822cfeb8f0f1bbd57f9a06fbfa793"
+    },
+    {
+      "source_key": "model.visual.blocks.10.attn.qkv.bias",
+      "stored_key": "blocks.10.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "0c02d5b3cc37a0a36359afb7dbdb76e1aa030428bd9975364e7d9f5cb5c6652d"
+    },
+    {
+      "source_key": "model.visual.blocks.10.attn.qkv.weight",
+      "stored_key": "blocks.10.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "25860be82650a047b70e6a6da621ffd88cd66a1311d1d96cedf762c9571098a5"
+    },
+    {
+      "source_key": "model.visual.blocks.10.mlp.linear_fc1.bias",
+      "stored_key": "blocks.10.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "139e854517d7dcc3aec00f14bd727bc896037cae47eecedbf18d010a44e7f9b6"
+    },
+    {
+      "source_key": "model.visual.blocks.10.mlp.linear_fc1.weight",
+      "stored_key": "blocks.10.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "8c572d053d961cc1b3c39b75ee9ee2bc8d32c2e2ef87f2d66ba4ecdcd9a024d3"
+    },
+    {
+      "source_key": "model.visual.blocks.10.mlp.linear_fc2.bias",
+      "stored_key": "blocks.10.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "7d50fdbd95fd91057efd6302c68549565b3cf98dfa3eba850dcafa9f45d5a3c5"
+    },
+    {
+      "source_key": "model.visual.blocks.10.mlp.linear_fc2.weight",
+      "stored_key": "blocks.10.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "83ca6ba8955133aae1a2e54a6f10b422fcae03da01454d3341657db77ce13ec5"
+    },
+    {
+      "source_key": "model.visual.blocks.10.norm1.bias",
+      "stored_key": "blocks.10.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "dc06e2e746f93ecf3e64cebf269143627a0ce9d7c3db17612bf74e91c7e2e4c9"
+    },
+    {
+      "source_key": "model.visual.blocks.10.norm1.weight",
+      "stored_key": "blocks.10.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "719ffb80ffc4cf2f8e16e1c1df492c1db24edc03a600a09857d12023df3dd16e"
+    },
+    {
+      "source_key": "model.visual.blocks.10.norm2.bias",
+      "stored_key": "blocks.10.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "b653a3ca18bd4cadba1c22baae51470529490e462b9698b683d73fa6aeae5112"
+    },
+    {
+      "source_key": "model.visual.blocks.10.norm2.weight",
+      "stored_key": "blocks.10.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "9e212b794d6d3691b69856b810e1530ea40260d75f64849a6337ce7dd18745ca"
+    },
+    {
+      "source_key": "model.visual.blocks.11.attn.proj.bias",
+      "stored_key": "blocks.11.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "535edf09375cf57feeda11cd58ae4fe5d3737895c989ca4228f4c524bded5410"
+    },
+    {
+      "source_key": "model.visual.blocks.11.attn.proj.weight",
+      "stored_key": "blocks.11.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "d36a71957345f59dc753c14ef8328ce053057bdc3c5c5dee12f974f686f566ed"
+    },
+    {
+      "source_key": "model.visual.blocks.11.attn.qkv.bias",
+      "stored_key": "blocks.11.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "4e1e5bf8a6f07c53127831d4d69c012fa69a3c93832943d2c95ec1d5bebbb118"
+    },
+    {
+      "source_key": "model.visual.blocks.11.attn.qkv.weight",
+      "stored_key": "blocks.11.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "889e8de2ee3a15e6719594a2c60bafc2147bd01fee3c33192846bb6bb1e67243"
+    },
+    {
+      "source_key": "model.visual.blocks.11.mlp.linear_fc1.bias",
+      "stored_key": "blocks.11.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "e8aca63174bd64230ef4b8c378cb7f9d9f4a7bc5ee99092b8455df6f925c12fb"
+    },
+    {
+      "source_key": "model.visual.blocks.11.mlp.linear_fc1.weight",
+      "stored_key": "blocks.11.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "945c321d140d345fff1ab00f658435ce41d451c7b34d83424e15ede0b81cadb9"
+    },
+    {
+      "source_key": "model.visual.blocks.11.mlp.linear_fc2.bias",
+      "stored_key": "blocks.11.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "5d45fd33d1c64f67ede91784d3efd0fa257990a730a2158e4edb193a102469fc"
+    },
+    {
+      "source_key": "model.visual.blocks.11.mlp.linear_fc2.weight",
+      "stored_key": "blocks.11.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "2930219fe9770dc81932cb4535dbeb638dee12144eeef3db71cdaedf1fa65085"
+    },
+    {
+      "source_key": "model.visual.blocks.11.norm1.bias",
+      "stored_key": "blocks.11.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "d4581447c32da3116b30c41c449810cc4cc84a7970bca02e615f90b2d4115217"
+    },
+    {
+      "source_key": "model.visual.blocks.11.norm1.weight",
+      "stored_key": "blocks.11.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "1d2e5d2dce7cbeab8fc314d314c6647380c96f8ace9935f5cfff4b4345acc987"
+    },
+    {
+      "source_key": "model.visual.blocks.11.norm2.bias",
+      "stored_key": "blocks.11.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "3418649580c5163edebfd05f860e39d0a35cf6038d469b10b55a360e612b5ae1"
+    },
+    {
+      "source_key": "model.visual.blocks.11.norm2.weight",
+      "stored_key": "blocks.11.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "689cb527a1a646850a339bb03e4772f8c22da758610ff4f9ec5d8ef26e20fd8f"
+    },
+    {
+      "source_key": "model.visual.blocks.12.attn.proj.bias",
+      "stored_key": "blocks.12.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "c7b571852801e9560968307fb74b42716175ad7074317a11c69a653e3cfcb90c"
+    },
+    {
+      "source_key": "model.visual.blocks.12.attn.proj.weight",
+      "stored_key": "blocks.12.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "66794d4cb4658fe29d02cfe07c712ee078f09d5b38a2269a8c173994ae9a2e4b"
+    },
+    {
+      "source_key": "model.visual.blocks.12.attn.qkv.bias",
+      "stored_key": "blocks.12.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "59e614d618c65fdc30ebc76b1d7fd0070aae08e8ddf166718b78f6f98c46199f"
+    },
+    {
+      "source_key": "model.visual.blocks.12.attn.qkv.weight",
+      "stored_key": "blocks.12.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "d28bf809c5c142f96f00352170bd4211147570c6d7b50a3cb1c7a17a38f142df"
+    },
+    {
+      "source_key": "model.visual.blocks.12.mlp.linear_fc1.bias",
+      "stored_key": "blocks.12.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "03adf4517fd824e58ef9f3bf031b3731a02935648acbe834794da280308f79d0"
+    },
+    {
+      "source_key": "model.visual.blocks.12.mlp.linear_fc1.weight",
+      "stored_key": "blocks.12.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "66971a356754963afe4f1ff1d946641cf4d6a31b1a40acf9578f435623686ad7"
+    },
+    {
+      "source_key": "model.visual.blocks.12.mlp.linear_fc2.bias",
+      "stored_key": "blocks.12.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "f58521a738b075b6dd324eba35b58060962f28e1bf790c54f3c1c23b4de7890e"
+    },
+    {
+      "source_key": "model.visual.blocks.12.mlp.linear_fc2.weight",
+      "stored_key": "blocks.12.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "fcd763f6668a92b1678250b69e36e2c79a93ea333c6ccc79bccb8048db8f7f7f"
+    },
+    {
+      "source_key": "model.visual.blocks.12.norm1.bias",
+      "stored_key": "blocks.12.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "55b488bec64e4932ed12e5e750846c7c74e1e0cb315a61b35994bcc840753e40"
+    },
+    {
+      "source_key": "model.visual.blocks.12.norm1.weight",
+      "stored_key": "blocks.12.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "89d00ffecd4869aa34c0c652125a7ecf28f807f1d72ab54dde402596b3b70ba5"
+    },
+    {
+      "source_key": "model.visual.blocks.12.norm2.bias",
+      "stored_key": "blocks.12.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "21a52d9ed6c14cc7b5be5623936bc8ce9c1772f1fa59b86b6c16ab5d30378d4d"
+    },
+    {
+      "source_key": "model.visual.blocks.12.norm2.weight",
+      "stored_key": "blocks.12.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "e15ca91e098bc70d79fa65f39252d0184753746f907b6ad5dffc68cadbbcae19"
+    },
+    {
+      "source_key": "model.visual.blocks.13.attn.proj.bias",
+      "stored_key": "blocks.13.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "cc45962b5cb888d7fe1cb1d61a478cfa127bbeedcce239ae5684061c56ffaa96"
+    },
+    {
+      "source_key": "model.visual.blocks.13.attn.proj.weight",
+      "stored_key": "blocks.13.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "d469689ed99301bf9aea5981c116a2ac5da38dba5d14ab5adb1fdf9f85e4ce4e"
+    },
+    {
+      "source_key": "model.visual.blocks.13.attn.qkv.bias",
+      "stored_key": "blocks.13.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "03bf87b59e1100efd55bfef2caa438efd931a25661644837fe2bf2b977ed2656"
+    },
+    {
+      "source_key": "model.visual.blocks.13.attn.qkv.weight",
+      "stored_key": "blocks.13.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "9df7a0e82856323db50e5d4650225858b84fff6c17b06428cc18beee3251a492"
+    },
+    {
+      "source_key": "model.visual.blocks.13.mlp.linear_fc1.bias",
+      "stored_key": "blocks.13.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "75bdffb1019e93d21f224c813e7f6090565807a41f0d419720e27f6c88093861"
+    },
+    {
+      "source_key": "model.visual.blocks.13.mlp.linear_fc1.weight",
+      "stored_key": "blocks.13.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "46ffd7228f1061198aea0cc1f2907e8a8f01975ffd84afb949d5808b442ce7e5"
+    },
+    {
+      "source_key": "model.visual.blocks.13.mlp.linear_fc2.bias",
+      "stored_key": "blocks.13.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "e837415b90547e07aa7c78db82d5c0af184cbd0eb21bb9241520e7c57f8cafd3"
+    },
+    {
+      "source_key": "model.visual.blocks.13.mlp.linear_fc2.weight",
+      "stored_key": "blocks.13.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "d505906c14baf1c8e0e3e33cd8a7495afb8df9b1d187ceddc4d11327545a4cda"
+    },
+    {
+      "source_key": "model.visual.blocks.13.norm1.bias",
+      "stored_key": "blocks.13.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "8f13ea991cb9be703f530c556e439753d2ced2596d8532fcb85811a22502ca72"
+    },
+    {
+      "source_key": "model.visual.blocks.13.norm1.weight",
+      "stored_key": "blocks.13.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "14fcaa9177e529715de6d0fd8840bc3643126619555eb217eceadf989e515639"
+    },
+    {
+      "source_key": "model.visual.blocks.13.norm2.bias",
+      "stored_key": "blocks.13.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "708302ab8178654bcf1bfcea3f2d3df37ec073b1a531dd15757e3a0807354f8a"
+    },
+    {
+      "source_key": "model.visual.blocks.13.norm2.weight",
+      "stored_key": "blocks.13.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "9d7eb6708d8060db0f0614d46d04c7000cd1839c2e807806795b68c37153b45d"
+    },
+    {
+      "source_key": "model.visual.blocks.14.attn.proj.bias",
+      "stored_key": "blocks.14.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "78b522a35237c3bdb0a1f5c203a2774138f837d092cc523071dd4688bab5c0e1"
+    },
+    {
+      "source_key": "model.visual.blocks.14.attn.proj.weight",
+      "stored_key": "blocks.14.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "495db4516a2878d1778c1539a3279e93d0122ff0e84444346070484b82b946f5"
+    },
+    {
+      "source_key": "model.visual.blocks.14.attn.qkv.bias",
+      "stored_key": "blocks.14.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "770fde88a2e7cdad9cafc19e437414a127410e29174d494819dd8d588997dfdc"
+    },
+    {
+      "source_key": "model.visual.blocks.14.attn.qkv.weight",
+      "stored_key": "blocks.14.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "a654e0f77975ef02d00b5e2c13ccac4f4fc51be0dc947c9149123225a228f85b"
+    },
+    {
+      "source_key": "model.visual.blocks.14.mlp.linear_fc1.bias",
+      "stored_key": "blocks.14.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "96b5cedcc12924a57ebb92595fd14ef701ad1301d68485de48e2ed16df7cff76"
+    },
+    {
+      "source_key": "model.visual.blocks.14.mlp.linear_fc1.weight",
+      "stored_key": "blocks.14.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "c14522925ec3cd97ede4b23b6700e07d136f3f5294b06aa2da94a50bed2dbf6f"
+    },
+    {
+      "source_key": "model.visual.blocks.14.mlp.linear_fc2.bias",
+      "stored_key": "blocks.14.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "234c7760d5a8eb130e2aaf1992e86904fd6bc76dfbfda85556429d4fedb3db95"
+    },
+    {
+      "source_key": "model.visual.blocks.14.mlp.linear_fc2.weight",
+      "stored_key": "blocks.14.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "80983d2f8222003773fc0257bd81b781d653409b67ed956fe9a7c78db20c265e"
+    },
+    {
+      "source_key": "model.visual.blocks.14.norm1.bias",
+      "stored_key": "blocks.14.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "ae1634ad682282777b579d0c6f082c013a572155ff91cf943548cfc8ab57dbd4"
+    },
+    {
+      "source_key": "model.visual.blocks.14.norm1.weight",
+      "stored_key": "blocks.14.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "90ea83a611ffde5262720f085b6ef68c729a31d1874e93e8a266545a52c6574b"
+    },
+    {
+      "source_key": "model.visual.blocks.14.norm2.bias",
+      "stored_key": "blocks.14.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "bb8469b7c041783c4c3aacd0a5322e6083eddc8b760914b7955c0a1590d83f67"
+    },
+    {
+      "source_key": "model.visual.blocks.14.norm2.weight",
+      "stored_key": "blocks.14.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "63577cbee7225af4d2ba9a597d1b09a2f33f98f7a91c55d73c51de69dc40b228"
+    },
+    {
+      "source_key": "model.visual.blocks.15.attn.proj.bias",
+      "stored_key": "blocks.15.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "d2fe021fbbeec2eae2c2e45cd019870330b50989595c58300a34a2fcdd73d87c"
+    },
+    {
+      "source_key": "model.visual.blocks.15.attn.proj.weight",
+      "stored_key": "blocks.15.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "4e475e6517c9d7054524601379b74d2c244f6a8bb8a9c9d82e5919a48e83adc1"
+    },
+    {
+      "source_key": "model.visual.blocks.15.attn.qkv.bias",
+      "stored_key": "blocks.15.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "c199d529f8f5609a1d321170146d751fd5f265f9f06403d21cdd0646f2a5735b"
+    },
+    {
+      "source_key": "model.visual.blocks.15.attn.qkv.weight",
+      "stored_key": "blocks.15.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "2f0906f59c8d625695f4479fc600a6e93635599db07e7a4c36e74ffd7329c912"
+    },
+    {
+      "source_key": "model.visual.blocks.15.mlp.linear_fc1.bias",
+      "stored_key": "blocks.15.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "31c6ef245fc212fcbf57cfaae362f4ebabf4c5b1774e4b37351f45cc9c04309f"
+    },
+    {
+      "source_key": "model.visual.blocks.15.mlp.linear_fc1.weight",
+      "stored_key": "blocks.15.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "c7d6c4fde5be6180fa11a3c0830c664bda4eb1b9d30494808f90dfce1243684a"
+    },
+    {
+      "source_key": "model.visual.blocks.15.mlp.linear_fc2.bias",
+      "stored_key": "blocks.15.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "e4873d62c1369b812cbbe58452ae2545384b9255d8e329b59072b806b583c610"
+    },
+    {
+      "source_key": "model.visual.blocks.15.mlp.linear_fc2.weight",
+      "stored_key": "blocks.15.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "2ffec5982ec440803ebfad48d45c7154b292b9fa44bf0c9a74943d47b0910352"
+    },
+    {
+      "source_key": "model.visual.blocks.15.norm1.bias",
+      "stored_key": "blocks.15.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "26afcefb186185a391fbd4f20ac7167806fe2b41fb5dd2cfac924d85dcaa29a2"
+    },
+    {
+      "source_key": "model.visual.blocks.15.norm1.weight",
+      "stored_key": "blocks.15.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "9662b2e7b85e7fdd812feeda218a216999b49e302a4f56a519cee2702d53ebe5"
+    },
+    {
+      "source_key": "model.visual.blocks.15.norm2.bias",
+      "stored_key": "blocks.15.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "c1d98bba835d65324080f1bf236913083d509b0e1eeb7c454660c9290788327a"
+    },
+    {
+      "source_key": "model.visual.blocks.15.norm2.weight",
+      "stored_key": "blocks.15.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "42fd0802541e2a9b342e82ce403b093b916ed7f157f2a0e74fd2e8a2c64caa2e"
+    },
+    {
+      "source_key": "model.visual.blocks.16.attn.proj.bias",
+      "stored_key": "blocks.16.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "67e303d1d3775f4c90619f2291cbfce8dd2db63cdac3f561d98b68170513da8e"
+    },
+    {
+      "source_key": "model.visual.blocks.16.attn.proj.weight",
+      "stored_key": "blocks.16.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "e035562d9b97d71cf1d3abf455ee350d3b9c716ca199f24b564c176ad30361d1"
+    },
+    {
+      "source_key": "model.visual.blocks.16.attn.qkv.bias",
+      "stored_key": "blocks.16.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "36cd7d7ce88a5574cabc98c53015513820f0f679bfe1b107ae4d19f523913269"
+    },
+    {
+      "source_key": "model.visual.blocks.16.attn.qkv.weight",
+      "stored_key": "blocks.16.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "bdfed80d1ec1e6c6c6e428142250cd822e43de24002a869ad3386a256c65eff8"
+    },
+    {
+      "source_key": "model.visual.blocks.16.mlp.linear_fc1.bias",
+      "stored_key": "blocks.16.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "2679f3c0ab9440bab727659fe71545102d3d00ccf8ccce022af496c4a100439d"
+    },
+    {
+      "source_key": "model.visual.blocks.16.mlp.linear_fc1.weight",
+      "stored_key": "blocks.16.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "cf9ae34a3e694231e5f53c49bc82d5b5165d1cef9659f58c7904504bfe379af3"
+    },
+    {
+      "source_key": "model.visual.blocks.16.mlp.linear_fc2.bias",
+      "stored_key": "blocks.16.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "9438d005f317c4914318b9ab88255f5702b29a809b3cf55e01e548a99eb29ea7"
+    },
+    {
+      "source_key": "model.visual.blocks.16.mlp.linear_fc2.weight",
+      "stored_key": "blocks.16.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "7ec0e6ab70a768ae3ce247feafe6d3e9784a0c798ed0a08739f921124c5df224"
+    },
+    {
+      "source_key": "model.visual.blocks.16.norm1.bias",
+      "stored_key": "blocks.16.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "922b5a50f265446fc1a9ab52f54220ad7143cfc97404b0311aab7c39409d7b04"
+    },
+    {
+      "source_key": "model.visual.blocks.16.norm1.weight",
+      "stored_key": "blocks.16.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "f1c8a5a85fd74b44e8a6aede6b86cdfe6147f65f5cb0fd63dd76380d0cd8aedf"
+    },
+    {
+      "source_key": "model.visual.blocks.16.norm2.bias",
+      "stored_key": "blocks.16.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "e881a4df26fb9f6972013b9d2ad61772532eec2d90d1959d93c0f6696db0dff8"
+    },
+    {
+      "source_key": "model.visual.blocks.16.norm2.weight",
+      "stored_key": "blocks.16.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "7a0044429ab55246a1001b025a619354c94e0cd6df5f521f16e632170ac9cbee"
+    },
+    {
+      "source_key": "model.visual.blocks.17.attn.proj.bias",
+      "stored_key": "blocks.17.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "ecc37835e3f012ba28a738cfabee4aa548c4e8c833227a19eadf3881f543778a"
+    },
+    {
+      "source_key": "model.visual.blocks.17.attn.proj.weight",
+      "stored_key": "blocks.17.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "1c4fb2f1482de43b399d8108a607207f7a258711664b083effceb7608c08fcdd"
+    },
+    {
+      "source_key": "model.visual.blocks.17.attn.qkv.bias",
+      "stored_key": "blocks.17.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "3eec02c4679836ad3458f8ab0e8e9a5f3c0abc446a406c82845303703c977875"
+    },
+    {
+      "source_key": "model.visual.blocks.17.attn.qkv.weight",
+      "stored_key": "blocks.17.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "a76df2bc1117e32356129da359319caeccf348f48ef418198d209ab6251ac4e6"
+    },
+    {
+      "source_key": "model.visual.blocks.17.mlp.linear_fc1.bias",
+      "stored_key": "blocks.17.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "f4e4905f09f3fba726cc3729e0afb674553f856d3e48a00e2627ae449905f002"
+    },
+    {
+      "source_key": "model.visual.blocks.17.mlp.linear_fc1.weight",
+      "stored_key": "blocks.17.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "a653be31b0da49a30dd573359ce5bd3d6ae1001258db8916935e8778d44988b7"
+    },
+    {
+      "source_key": "model.visual.blocks.17.mlp.linear_fc2.bias",
+      "stored_key": "blocks.17.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "53f3fb23e9187cb3acf7fa2fce202178bfbba27e441a797a0e273715cd5d02fe"
+    },
+    {
+      "source_key": "model.visual.blocks.17.mlp.linear_fc2.weight",
+      "stored_key": "blocks.17.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "35a0bef5571381a0ddddbace0bff928e6e85b692050a9fa41ee875184d95cd67"
+    },
+    {
+      "source_key": "model.visual.blocks.17.norm1.bias",
+      "stored_key": "blocks.17.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "a70dbee2047483ef903515abd41def95857c933997574836d69b96434d6b2002"
+    },
+    {
+      "source_key": "model.visual.blocks.17.norm1.weight",
+      "stored_key": "blocks.17.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "6a249e8a02afd9e50b6a8a82df8980bea1b7f930bf8e081213d56939b9f82347"
+    },
+    {
+      "source_key": "model.visual.blocks.17.norm2.bias",
+      "stored_key": "blocks.17.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "67c6caeb0198d4c00066e3d08a81d30757f8c85521ade900e424291e5b34c94c"
+    },
+    {
+      "source_key": "model.visual.blocks.17.norm2.weight",
+      "stored_key": "blocks.17.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "d07112b08b9563293a057bf47f60f22343569ca52ad56d04d101cad8d81aad7b"
+    },
+    {
+      "source_key": "model.visual.blocks.18.attn.proj.bias",
+      "stored_key": "blocks.18.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "d189414425632c6b37d3c3f8bdff10a16519caca1b9cee8ea5d3d504bc6e8953"
+    },
+    {
+      "source_key": "model.visual.blocks.18.attn.proj.weight",
+      "stored_key": "blocks.18.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "70aa98a9e2db36a5e16d485f5a808bbf562bf7d55c2e6b6316cbaf543cc3ff07"
+    },
+    {
+      "source_key": "model.visual.blocks.18.attn.qkv.bias",
+      "stored_key": "blocks.18.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "e0db722d012df5d9af6e8bbf6f79260dd0746e1fcc4667c021ef4a7a0c49a552"
+    },
+    {
+      "source_key": "model.visual.blocks.18.attn.qkv.weight",
+      "stored_key": "blocks.18.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "518bf9d6418e421d00c5d3908f4c44186131afc074a813c43ddd6030a529d6c4"
+    },
+    {
+      "source_key": "model.visual.blocks.18.mlp.linear_fc1.bias",
+      "stored_key": "blocks.18.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "9d56fe5066357e6869c6f7d95855bb6f519bd0f5d1ed26b113392a6b1b0a5e30"
+    },
+    {
+      "source_key": "model.visual.blocks.18.mlp.linear_fc1.weight",
+      "stored_key": "blocks.18.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "23b3edd3baf70fa1db318d60b8d97ac163472ce5a29efd6a5121680c1368a652"
+    },
+    {
+      "source_key": "model.visual.blocks.18.mlp.linear_fc2.bias",
+      "stored_key": "blocks.18.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "cf4371fe8d5dac5e6907d6b18d2d23d842cdefdfcf3a6af4dec684b39db1f7aa"
+    },
+    {
+      "source_key": "model.visual.blocks.18.mlp.linear_fc2.weight",
+      "stored_key": "blocks.18.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "12f8fb5d6d417e24eff8df41b03d154e1f86ed68158f1d62ca81347a6f61a0f7"
+    },
+    {
+      "source_key": "model.visual.blocks.18.norm1.bias",
+      "stored_key": "blocks.18.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "2dd09a0b0b575c6fb2539920e3b1a3d30748d7808e24415e5d248b83c274ba79"
+    },
+    {
+      "source_key": "model.visual.blocks.18.norm1.weight",
+      "stored_key": "blocks.18.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "11440904da22b07846a01a67f36111d4df1cdafbbe83b5a263fe80510b2b8be0"
+    },
+    {
+      "source_key": "model.visual.blocks.18.norm2.bias",
+      "stored_key": "blocks.18.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "3a5ea949a98848992b5622056ea1255ed379bb0319583a6d7bebc645caa4bce4"
+    },
+    {
+      "source_key": "model.visual.blocks.18.norm2.weight",
+      "stored_key": "blocks.18.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "a552debe1a2ddfb61489569868979b5c6241d92296e0bec833d59994de494f00"
+    },
+    {
+      "source_key": "model.visual.blocks.19.attn.proj.bias",
+      "stored_key": "blocks.19.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "f2be32c53bdcb95dc24995aee285447268badc536ba0beb9f9533560e750f6f1"
+    },
+    {
+      "source_key": "model.visual.blocks.19.attn.proj.weight",
+      "stored_key": "blocks.19.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "a879c4ab75e252fce71be40052e2541ce36f6259808cbda1db243494500af3af"
+    },
+    {
+      "source_key": "model.visual.blocks.19.attn.qkv.bias",
+      "stored_key": "blocks.19.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "63eab9f5da59cfc96453e09f1ddb8bb5961d6a5166a07f21965b371083e75d3a"
+    },
+    {
+      "source_key": "model.visual.blocks.19.attn.qkv.weight",
+      "stored_key": "blocks.19.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "cf8284358d735cd78b61075796b001de9b5ae65dc0d6c353000e5bcbabf61566"
+    },
+    {
+      "source_key": "model.visual.blocks.19.mlp.linear_fc1.bias",
+      "stored_key": "blocks.19.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "c4231bf87f87eab9d97b8a843f9df38daedf450f5e438c21809f941fb3e63754"
+    },
+    {
+      "source_key": "model.visual.blocks.19.mlp.linear_fc1.weight",
+      "stored_key": "blocks.19.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "37758c33670d2393fe70dad407d77ebb7f6ee36d9f18a152dfdc58eec2bf1d9d"
+    },
+    {
+      "source_key": "model.visual.blocks.19.mlp.linear_fc2.bias",
+      "stored_key": "blocks.19.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "9d6d57bd7e8efa67b604dca0d8138a096e297569f752b839dec81cd45ca1528a"
+    },
+    {
+      "source_key": "model.visual.blocks.19.mlp.linear_fc2.weight",
+      "stored_key": "blocks.19.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "efa8961ef4c18b8c1af6471ea4fb0fed1649735f8798b371cd5ffb9f31844755"
+    },
+    {
+      "source_key": "model.visual.blocks.19.norm1.bias",
+      "stored_key": "blocks.19.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "14bf205dcbeff508fba49b85da9e3b110a228582ba925c27a32b2dc1f9852934"
+    },
+    {
+      "source_key": "model.visual.blocks.19.norm1.weight",
+      "stored_key": "blocks.19.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "bb1acdb28398581fd4d2872985498bad6f70658f24c8b82ddcf2f292dd4ee6e5"
+    },
+    {
+      "source_key": "model.visual.blocks.19.norm2.bias",
+      "stored_key": "blocks.19.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "6f4d0d1aa976eca8422fef218f36e233585f7a71d007921c7e0b7a7fa7cb20b7"
+    },
+    {
+      "source_key": "model.visual.blocks.19.norm2.weight",
+      "stored_key": "blocks.19.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "6c6b4fff1eec770266f86d6302fc538aaa3e99a055a64e5ac25a890a9d105cf1"
+    },
+    {
+      "source_key": "model.visual.blocks.2.attn.proj.bias",
+      "stored_key": "blocks.2.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "d15d8952876c443d2831242bbca7e2085b66fdb7cc9f028f86c4ce0c54ed10eb"
+    },
+    {
+      "source_key": "model.visual.blocks.2.attn.proj.weight",
+      "stored_key": "blocks.2.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "095bfe7575b6611583f259f734b85c29d3943ba3b03fb664f9101bd4454f8b29"
+    },
+    {
+      "source_key": "model.visual.blocks.2.attn.qkv.bias",
+      "stored_key": "blocks.2.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "36da5adcd26a03140fce35ef1fe043fd05d1e8f08be447e70f2695f637f581d5"
+    },
+    {
+      "source_key": "model.visual.blocks.2.attn.qkv.weight",
+      "stored_key": "blocks.2.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "c03817aa9ba2c70252eecc07da07e86a533392b91a507190c2eb2eb3603005bc"
+    },
+    {
+      "source_key": "model.visual.blocks.2.mlp.linear_fc1.bias",
+      "stored_key": "blocks.2.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "565baff198a031a862174d097fddf2762ffb5d3e380546e96b9a4b19233b0e0d"
+    },
+    {
+      "source_key": "model.visual.blocks.2.mlp.linear_fc1.weight",
+      "stored_key": "blocks.2.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "d537438f5898a64f7ac3175cd934cdc946e53deaf37c4b36ad085102c4bcf705"
+    },
+    {
+      "source_key": "model.visual.blocks.2.mlp.linear_fc2.bias",
+      "stored_key": "blocks.2.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "f0c2fab04bffd3bc60617339b7d0e4750b395b7af93aba14efa04c4f4376ebb8"
+    },
+    {
+      "source_key": "model.visual.blocks.2.mlp.linear_fc2.weight",
+      "stored_key": "blocks.2.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "4e1ce0b6b4d4e730b1fb1729e9ddd3f81fb1061b010aa08992be59bbf01991b7"
+    },
+    {
+      "source_key": "model.visual.blocks.2.norm1.bias",
+      "stored_key": "blocks.2.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "d53bb37eb27be8e8367972ee20877e0ff53c9c26d13a9796f756b35a80314f65"
+    },
+    {
+      "source_key": "model.visual.blocks.2.norm1.weight",
+      "stored_key": "blocks.2.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "53eb7e4d525bc90817a878bec7e9ab2d17ba965fad21194cc0e8825542de4dd1"
+    },
+    {
+      "source_key": "model.visual.blocks.2.norm2.bias",
+      "stored_key": "blocks.2.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "de42b54380f8d4bc114c8d329c2fb86caf33d3920665fbaab7f8b041e856e499"
+    },
+    {
+      "source_key": "model.visual.blocks.2.norm2.weight",
+      "stored_key": "blocks.2.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "72301bded64b8f8586d447c2a747a7237e4be7caed8e23a902278c3d8b23b0c9"
+    },
+    {
+      "source_key": "model.visual.blocks.20.attn.proj.bias",
+      "stored_key": "blocks.20.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "5e364ea41ac34ef692a35f2deb56f452cdbe752fd60affe6c2d212efa3c7156a"
+    },
+    {
+      "source_key": "model.visual.blocks.20.attn.proj.weight",
+      "stored_key": "blocks.20.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "8261f057e2b0acbd86da2618f384e4849164272b96de798760403240a883f0bf"
+    },
+    {
+      "source_key": "model.visual.blocks.20.attn.qkv.bias",
+      "stored_key": "blocks.20.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "c85dfcc71c85ca886ffbd410b375bdce1ec02eadd0320c5cb90e9af91babefcb"
+    },
+    {
+      "source_key": "model.visual.blocks.20.attn.qkv.weight",
+      "stored_key": "blocks.20.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "78934f651d247405b38476dc01be1ea2692ed605d0f18af1a726c9defed93fde"
+    },
+    {
+      "source_key": "model.visual.blocks.20.mlp.linear_fc1.bias",
+      "stored_key": "blocks.20.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "cd52afabacae8dd7cba4133d47a3c1efc317cbb3baacf3c2b70736ea1b8930ed"
+    },
+    {
+      "source_key": "model.visual.blocks.20.mlp.linear_fc1.weight",
+      "stored_key": "blocks.20.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "2793660b0fb055c039adcb4e885613a625336e17b8fea48c21607c0cb35ebdc6"
+    },
+    {
+      "source_key": "model.visual.blocks.20.mlp.linear_fc2.bias",
+      "stored_key": "blocks.20.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "90f48412166b57b1f83e92b9da46c72898fd8b00daed46cf5124609aa0dcc518"
+    },
+    {
+      "source_key": "model.visual.blocks.20.mlp.linear_fc2.weight",
+      "stored_key": "blocks.20.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "7bb8e17178071bd63796cdb123b125b2306c304da1540eed4be466b3702e37c0"
+    },
+    {
+      "source_key": "model.visual.blocks.20.norm1.bias",
+      "stored_key": "blocks.20.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "6f32885d3347215cf127e6de8ad993f2d075537af31b22038e2e4b16b8457aff"
+    },
+    {
+      "source_key": "model.visual.blocks.20.norm1.weight",
+      "stored_key": "blocks.20.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "dc8bb31922829c4336dc609c6bae2cbf895e4d926f264d5169fcb1078b2e73e1"
+    },
+    {
+      "source_key": "model.visual.blocks.20.norm2.bias",
+      "stored_key": "blocks.20.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "e7313b6bcd24a7a6a6709898f96d68cd77f5ed28f16f4d2facdb4b47355d66b6"
+    },
+    {
+      "source_key": "model.visual.blocks.20.norm2.weight",
+      "stored_key": "blocks.20.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "cd576cf8d2cb289107a5f36d200dcd6181e9b2aff110eb67fa8501e714c061fe"
+    },
+    {
+      "source_key": "model.visual.blocks.21.attn.proj.bias",
+      "stored_key": "blocks.21.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "1caea704b06a98ccf5b07e66800475e28a29a6a30545bb232d5e02bbceee5074"
+    },
+    {
+      "source_key": "model.visual.blocks.21.attn.proj.weight",
+      "stored_key": "blocks.21.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "e42ccabdb9a2a400a22742d1fbd81e4413827972188944e975f9eda2f75f2d3f"
+    },
+    {
+      "source_key": "model.visual.blocks.21.attn.qkv.bias",
+      "stored_key": "blocks.21.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "289f99f76968b3c9256921fe61947b9ad1a57e04b98362847036dcd1970b6221"
+    },
+    {
+      "source_key": "model.visual.blocks.21.attn.qkv.weight",
+      "stored_key": "blocks.21.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "4f1e6dfb1828b2cba6e232a4eb65d87205e929952ddc4e76b7596de082156202"
+    },
+    {
+      "source_key": "model.visual.blocks.21.mlp.linear_fc1.bias",
+      "stored_key": "blocks.21.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "acd9bbcd7f65285b6e72632f67714857f8c7b793114dd50d02a5222308be0e60"
+    },
+    {
+      "source_key": "model.visual.blocks.21.mlp.linear_fc1.weight",
+      "stored_key": "blocks.21.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "aa45c52155c439153ac1c1d44d122ff6f8a9eeb7b410fdc7514755fd70cd2e3b"
+    },
+    {
+      "source_key": "model.visual.blocks.21.mlp.linear_fc2.bias",
+      "stored_key": "blocks.21.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "bdf54d1be829f1a6e72a2969505a4ee41972377259c2de877adfb71a090a5f55"
+    },
+    {
+      "source_key": "model.visual.blocks.21.mlp.linear_fc2.weight",
+      "stored_key": "blocks.21.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "e091320ca94b4b06c33af4bff2f71df5a9c414ff542d5737c38629fc997df5c5"
+    },
+    {
+      "source_key": "model.visual.blocks.21.norm1.bias",
+      "stored_key": "blocks.21.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "66ff6f25051c4b7ade96f574c50ff7cc394524a3a77471d3472c7dbf579b5632"
+    },
+    {
+      "source_key": "model.visual.blocks.21.norm1.weight",
+      "stored_key": "blocks.21.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "721cea6b2e9d821ea3687de6a41f9d51da0105aaa37adc9f0707a2144fbbee21"
+    },
+    {
+      "source_key": "model.visual.blocks.21.norm2.bias",
+      "stored_key": "blocks.21.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "074ae062e8572f4468f98d3e87d3e6e8108726b3ddf47f8046e501791aad58ce"
+    },
+    {
+      "source_key": "model.visual.blocks.21.norm2.weight",
+      "stored_key": "blocks.21.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "85efd3a842a203524b24bcfa2c4e8dbe6a66c4c15e701db048cb5172700c10e4"
+    },
+    {
+      "source_key": "model.visual.blocks.22.attn.proj.bias",
+      "stored_key": "blocks.22.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "5aef38229be345796c155c7fe6571ceadf55ff05e797a888a1c55edd04a6255b"
+    },
+    {
+      "source_key": "model.visual.blocks.22.attn.proj.weight",
+      "stored_key": "blocks.22.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "dec478193ac6059b2b38fb0c3c6d9967d690164c7baf082dae1076d7f9cd1eb5"
+    },
+    {
+      "source_key": "model.visual.blocks.22.attn.qkv.bias",
+      "stored_key": "blocks.22.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "90a8fe246319b8b330ca3fad67b2ff43da0eed2421b5872ac8ade749114da75d"
+    },
+    {
+      "source_key": "model.visual.blocks.22.attn.qkv.weight",
+      "stored_key": "blocks.22.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "56a5c9eaa7061ee04b50ef49593c432e1cf3266e631c64bd239fec464128fd20"
+    },
+    {
+      "source_key": "model.visual.blocks.22.mlp.linear_fc1.bias",
+      "stored_key": "blocks.22.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "f6bb1804338a46e001f04de162efa193d2fe06078fedd0cf89b7097e3642dacc"
+    },
+    {
+      "source_key": "model.visual.blocks.22.mlp.linear_fc1.weight",
+      "stored_key": "blocks.22.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "8878d6c70d1223efb67eb560531575084607b373f9cf3093dec62cd80cfea01b"
+    },
+    {
+      "source_key": "model.visual.blocks.22.mlp.linear_fc2.bias",
+      "stored_key": "blocks.22.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "cb0343ab54df7a316159c2ea6822e1a2219f78541d3f56006a53c2eb8d8a8760"
+    },
+    {
+      "source_key": "model.visual.blocks.22.mlp.linear_fc2.weight",
+      "stored_key": "blocks.22.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "4d23ba22952fbc540a74d41269121283dd63a26d6299233275845e52af58b179"
+    },
+    {
+      "source_key": "model.visual.blocks.22.norm1.bias",
+      "stored_key": "blocks.22.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "7b658a00cf46afe84ca7e42b32282a40a31954eedf54b425c62b4b372c23aa1d"
+    },
+    {
+      "source_key": "model.visual.blocks.22.norm1.weight",
+      "stored_key": "blocks.22.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "613a10552c93f56ff5bac07a7bf3cb471f0946c63637f6e8fd8b08087d8d799b"
+    },
+    {
+      "source_key": "model.visual.blocks.22.norm2.bias",
+      "stored_key": "blocks.22.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "70fb9cdc8f888e9949cb3945439182cd90dbc92e06320a1faacac82791f57ded"
+    },
+    {
+      "source_key": "model.visual.blocks.22.norm2.weight",
+      "stored_key": "blocks.22.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "5d1d84f6f59a9db6fbb3bd3c1ed3e897d5de96ac11ddb41fc6d3e34a9056786e"
+    },
+    {
+      "source_key": "model.visual.blocks.23.attn.proj.bias",
+      "stored_key": "blocks.23.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "355fbbe43395049f2206ced956fe07361dfad2e8a6a80cf5c741c58081a0b2d1"
+    },
+    {
+      "source_key": "model.visual.blocks.23.attn.proj.weight",
+      "stored_key": "blocks.23.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "90770d5378a963bc3b69f61f04d617e6fc0086034fae3e285086353ee8c134e4"
+    },
+    {
+      "source_key": "model.visual.blocks.23.attn.qkv.bias",
+      "stored_key": "blocks.23.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "c76e9e6223e2d4b98a8921eaf31a9a8f6473dbd7bb1a29bef26336999a3dddbd"
+    },
+    {
+      "source_key": "model.visual.blocks.23.attn.qkv.weight",
+      "stored_key": "blocks.23.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "babd730e9ca3ee7a8d8a4e03aece2c0f003034fbc834983d7f93cfc9da49b56e"
+    },
+    {
+      "source_key": "model.visual.blocks.23.mlp.linear_fc1.bias",
+      "stored_key": "blocks.23.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "9d5a25961f99eca0f561435b25614206fba993bd814f2366503980b79bc87fbd"
+    },
+    {
+      "source_key": "model.visual.blocks.23.mlp.linear_fc1.weight",
+      "stored_key": "blocks.23.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "3f37bebcf28909d9e5739dd9c018824e0415989b30dfac31615cee2082ca9c64"
+    },
+    {
+      "source_key": "model.visual.blocks.23.mlp.linear_fc2.bias",
+      "stored_key": "blocks.23.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "797dd716b3b55db78566e29159ee8da7d2ca74a84f4e784cbf5d4b120db5f668"
+    },
+    {
+      "source_key": "model.visual.blocks.23.mlp.linear_fc2.weight",
+      "stored_key": "blocks.23.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "c099129a931c240c719768b43d59566b0c02b94e07cb90c9dd676701180e8729"
+    },
+    {
+      "source_key": "model.visual.blocks.23.norm1.bias",
+      "stored_key": "blocks.23.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "15dc7c26cd78d8e259bd592071c9923287fa643ed77988b18f6437c8de9fc774"
+    },
+    {
+      "source_key": "model.visual.blocks.23.norm1.weight",
+      "stored_key": "blocks.23.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "dab60e5b1828f0f302fb85e166b1a3b335d10b6a164f398dad8651dd3df9633c"
+    },
+    {
+      "source_key": "model.visual.blocks.23.norm2.bias",
+      "stored_key": "blocks.23.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "8ce06c37f923af0456619d405017ecc7f0468a828cd33de69faa5476c1800b35"
+    },
+    {
+      "source_key": "model.visual.blocks.23.norm2.weight",
+      "stored_key": "blocks.23.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "0b47bcbb28867fdf2cf58da7255b5a766c3676b9a3a73483a738b8c08b072580"
+    },
+    {
+      "source_key": "model.visual.blocks.3.attn.proj.bias",
+      "stored_key": "blocks.3.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "2906df962e6f718f2889e0733692498d828d7a47c921daac0e41f830947266a2"
+    },
+    {
+      "source_key": "model.visual.blocks.3.attn.proj.weight",
+      "stored_key": "blocks.3.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "536b81ffcc6ab4f3f5c873c098385af8da3bb5a8d49c27d204136aa19b2b3802"
+    },
+    {
+      "source_key": "model.visual.blocks.3.attn.qkv.bias",
+      "stored_key": "blocks.3.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "b092c8f5454519288f5bf47b5e10a44a9e2a6079b51e0e10e1a7ecd2654e5150"
+    },
+    {
+      "source_key": "model.visual.blocks.3.attn.qkv.weight",
+      "stored_key": "blocks.3.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "7e76a81b955c26dfbcdd872abbd120078896123ba925da5e37adb67b9b1a77e0"
+    },
+    {
+      "source_key": "model.visual.blocks.3.mlp.linear_fc1.bias",
+      "stored_key": "blocks.3.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "00077141432334861eeb1edb44fabe79d881937364039f0a657462c8d0af5aad"
+    },
+    {
+      "source_key": "model.visual.blocks.3.mlp.linear_fc1.weight",
+      "stored_key": "blocks.3.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "869faedbf272b0187c5d014f80c315117e5b8aaa2762da869b80a881095a640b"
+    },
+    {
+      "source_key": "model.visual.blocks.3.mlp.linear_fc2.bias",
+      "stored_key": "blocks.3.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "76ff805b1edefae7b3bea63fe7efffb0a29fb2c5f92e815b42a01ea7395281d7"
+    },
+    {
+      "source_key": "model.visual.blocks.3.mlp.linear_fc2.weight",
+      "stored_key": "blocks.3.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "317615bd5284e068e2bd750b9ca2ceea093508b308358185b47736c50a20869f"
+    },
+    {
+      "source_key": "model.visual.blocks.3.norm1.bias",
+      "stored_key": "blocks.3.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "d7b9392b156e16b2e61c9e1899e5ad227af1a0d9a59f06da3dd4bcd32f7af5ae"
+    },
+    {
+      "source_key": "model.visual.blocks.3.norm1.weight",
+      "stored_key": "blocks.3.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "30181cc25c28fe2b05ae78d48b511ac7e37da9e0687bc78c1fb413a33118a688"
+    },
+    {
+      "source_key": "model.visual.blocks.3.norm2.bias",
+      "stored_key": "blocks.3.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "2f27bb2f660f96da6f925290fa4356fdfaca3f8dc1c0f59f3baeb53ab1e98c4e"
+    },
+    {
+      "source_key": "model.visual.blocks.3.norm2.weight",
+      "stored_key": "blocks.3.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "347d99b48dc2daddf26f0b142cd0e147b1e7001715f1f8174fa4763f949d2a78"
+    },
+    {
+      "source_key": "model.visual.blocks.4.attn.proj.bias",
+      "stored_key": "blocks.4.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "4bea9d5edb4899dc3123838b610f43ffe3ab847781c09ee75ac96c5f5543a21e"
+    },
+    {
+      "source_key": "model.visual.blocks.4.attn.proj.weight",
+      "stored_key": "blocks.4.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "f949f8b84f16ab4c8d35549ba06865c6f9aac90e7ba4871e4fdfe9267c9e851b"
+    },
+    {
+      "source_key": "model.visual.blocks.4.attn.qkv.bias",
+      "stored_key": "blocks.4.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "56255d2c8fbfbc0e8c730e4ffa92edcb520b7414a519d838e8fc6886c57efa4b"
+    },
+    {
+      "source_key": "model.visual.blocks.4.attn.qkv.weight",
+      "stored_key": "blocks.4.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "78e94955c572d74473a1bf8dc18804171e3925ab785d8716ede4dbc9b1387d23"
+    },
+    {
+      "source_key": "model.visual.blocks.4.mlp.linear_fc1.bias",
+      "stored_key": "blocks.4.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "d861c45d68ff93f6d5c5292c0f55914318d1e18c3c538f74f08fd31cc27126e0"
+    },
+    {
+      "source_key": "model.visual.blocks.4.mlp.linear_fc1.weight",
+      "stored_key": "blocks.4.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "75a6adefb4c89cca46e1b1749b21e0a06d60b66c2107667df93758b96cc529b6"
+    },
+    {
+      "source_key": "model.visual.blocks.4.mlp.linear_fc2.bias",
+      "stored_key": "blocks.4.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "a0f49ebdb81855883f202562f011256cdd27f11e45fe5d6ae6ed7ade365a70b3"
+    },
+    {
+      "source_key": "model.visual.blocks.4.mlp.linear_fc2.weight",
+      "stored_key": "blocks.4.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "084f11d67c1473dd6d46965b3339a7eca94bc7e21ac7663f2b15e7ea573d56dd"
+    },
+    {
+      "source_key": "model.visual.blocks.4.norm1.bias",
+      "stored_key": "blocks.4.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "e3f21e806c215a36471ce272360df57bc533fa797372005f9f31a1046da52fa5"
+    },
+    {
+      "source_key": "model.visual.blocks.4.norm1.weight",
+      "stored_key": "blocks.4.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "912fa73a0040ccdd66e5cd3700c36cb879278d0285235a884739af4bdacf9873"
+    },
+    {
+      "source_key": "model.visual.blocks.4.norm2.bias",
+      "stored_key": "blocks.4.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "ecf54355a1afce4cb71c6a5e0dd1e89f3dc453dc6120ab7d1b43843f986af168"
+    },
+    {
+      "source_key": "model.visual.blocks.4.norm2.weight",
+      "stored_key": "blocks.4.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "53538fafbccf36456e0c38893a9ab5ed86ddc2eb467612525b02e05c025f3dc4"
+    },
+    {
+      "source_key": "model.visual.blocks.5.attn.proj.bias",
+      "stored_key": "blocks.5.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "50b4b834515e66d84b93a23baab3f88a82f26644a3f4e94251575ee88323aacb"
+    },
+    {
+      "source_key": "model.visual.blocks.5.attn.proj.weight",
+      "stored_key": "blocks.5.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "8b380198c4c4816879ee8207196d0618379f245d192cb3c7c7d3f3fd81470ac6"
+    },
+    {
+      "source_key": "model.visual.blocks.5.attn.qkv.bias",
+      "stored_key": "blocks.5.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "1c3e40dbb84ff086f14d0a2b318962300d64c7122735f50119e1931aea92c13b"
+    },
+    {
+      "source_key": "model.visual.blocks.5.attn.qkv.weight",
+      "stored_key": "blocks.5.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "f715a2bee024faff4d13049b53ca1d45ce8988054342756c7a69cfbc75726738"
+    },
+    {
+      "source_key": "model.visual.blocks.5.mlp.linear_fc1.bias",
+      "stored_key": "blocks.5.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "84af37bb9a0c3bb0cc5c04d15aedb31343d2ab1c7b122c3e8063d82b88512937"
+    },
+    {
+      "source_key": "model.visual.blocks.5.mlp.linear_fc1.weight",
+      "stored_key": "blocks.5.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "1b182ba753b9519751073b22f9c7e3eb85b24d7824e265ee73a2660df2c857e5"
+    },
+    {
+      "source_key": "model.visual.blocks.5.mlp.linear_fc2.bias",
+      "stored_key": "blocks.5.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "a0a50dda36aa78bd562cb741cf9f55550de9b73f7567eceb00a7bc0a7585c5eb"
+    },
+    {
+      "source_key": "model.visual.blocks.5.mlp.linear_fc2.weight",
+      "stored_key": "blocks.5.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "f0e0b9cc801daa75f4ac8969b0fae4b8ef54dadf1c7d0a20a829f9df805535f2"
+    },
+    {
+      "source_key": "model.visual.blocks.5.norm1.bias",
+      "stored_key": "blocks.5.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "6b667b92dea407df9c8a9fdbad542d35e7ac12cbb8d67a167e3534ff65ad32ad"
+    },
+    {
+      "source_key": "model.visual.blocks.5.norm1.weight",
+      "stored_key": "blocks.5.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "27a9d0f5055cc243eb1e3dfe3af5dd60391535dc36c1a7ffbfa72108dc59eb45"
+    },
+    {
+      "source_key": "model.visual.blocks.5.norm2.bias",
+      "stored_key": "blocks.5.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "61452e3ca053ba408bdbd70c510e8d99316bdc0f9e6daa269905e604e427eb27"
+    },
+    {
+      "source_key": "model.visual.blocks.5.norm2.weight",
+      "stored_key": "blocks.5.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "c2503f4b1f8d9759cd85e58d733e8908f54eb06a1333588ebcbf0d1da80856ac"
+    },
+    {
+      "source_key": "model.visual.blocks.6.attn.proj.bias",
+      "stored_key": "blocks.6.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "ee534892c5c917664794ea432c5cc5a59b468e87b5dfa9c14d457524918628ea"
+    },
+    {
+      "source_key": "model.visual.blocks.6.attn.proj.weight",
+      "stored_key": "blocks.6.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "da2dcfca947f1777341ce986767b895cff1bbb554722c9548fd73a0de3b13751"
+    },
+    {
+      "source_key": "model.visual.blocks.6.attn.qkv.bias",
+      "stored_key": "blocks.6.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "d251bd07c049f8f044dbe0246ad431a5d1e722d54e769eaf077779283c5d6339"
+    },
+    {
+      "source_key": "model.visual.blocks.6.attn.qkv.weight",
+      "stored_key": "blocks.6.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "bc7bc88703a0654465e2f24f01fcf0455d3c65aee5babfe02c2e2861c3396884"
+    },
+    {
+      "source_key": "model.visual.blocks.6.mlp.linear_fc1.bias",
+      "stored_key": "blocks.6.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "c91fcf391e724944303017c76130899b7f071f126812c5c5e5c89038f1bcdba9"
+    },
+    {
+      "source_key": "model.visual.blocks.6.mlp.linear_fc1.weight",
+      "stored_key": "blocks.6.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "aec0b03fb7e670015eb4d4652add7f4594322a71b5e75bcd81a44e539a3d0a45"
+    },
+    {
+      "source_key": "model.visual.blocks.6.mlp.linear_fc2.bias",
+      "stored_key": "blocks.6.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "0273437e510930b85a59540e217609e597bf7ad4dff1cb89c162d9b9c9dd5516"
+    },
+    {
+      "source_key": "model.visual.blocks.6.mlp.linear_fc2.weight",
+      "stored_key": "blocks.6.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "6ced53d3e857dcaf11230b09be55d5b7d130283fa652e5dfce6f009f469dab0e"
+    },
+    {
+      "source_key": "model.visual.blocks.6.norm1.bias",
+      "stored_key": "blocks.6.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "02b18baf1c0c0440276913acf7eba813ae28aaee3aef255ef30846d0877a0210"
+    },
+    {
+      "source_key": "model.visual.blocks.6.norm1.weight",
+      "stored_key": "blocks.6.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "6eb5a08f270365b7c434c500a7c1d77f5cfbfef4d09ea5626a120c97768517f4"
+    },
+    {
+      "source_key": "model.visual.blocks.6.norm2.bias",
+      "stored_key": "blocks.6.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "8c580f7575a955a4e20bf864e34486cfdaab448f97d72a9286fcf6b2da003bf6"
+    },
+    {
+      "source_key": "model.visual.blocks.6.norm2.weight",
+      "stored_key": "blocks.6.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "0e2fe864bd29a8be9f75e9d59e6177394b0b6b9af5ca69eeb26a91810725b659"
+    },
+    {
+      "source_key": "model.visual.blocks.7.attn.proj.bias",
+      "stored_key": "blocks.7.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "9cca012b5133c9c3264ecaeace4a5243c960d5f610ebc86e7405fa2696af663a"
+    },
+    {
+      "source_key": "model.visual.blocks.7.attn.proj.weight",
+      "stored_key": "blocks.7.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "e3faa107db4a78c2304caa047ad635e0a7a48670ebbdb634c50a23d5548a1520"
+    },
+    {
+      "source_key": "model.visual.blocks.7.attn.qkv.bias",
+      "stored_key": "blocks.7.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "13b56b2d0f927bbe0ebc514db50690394f75cd1820f4ae04c300261adf05ad8f"
+    },
+    {
+      "source_key": "model.visual.blocks.7.attn.qkv.weight",
+      "stored_key": "blocks.7.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "245e6747107d57e1552a7f4a95abf7fe40ffa0d08db990e37820d26a694abcda"
+    },
+    {
+      "source_key": "model.visual.blocks.7.mlp.linear_fc1.bias",
+      "stored_key": "blocks.7.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "0d29ece3b7b813b562889a2b649ccc749fa0088c2edee4f8d48269980900a2ef"
+    },
+    {
+      "source_key": "model.visual.blocks.7.mlp.linear_fc1.weight",
+      "stored_key": "blocks.7.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "e91ad20c42b9ba505462aee2bd595d9e2804db16d3af595afbec8bc447852476"
+    },
+    {
+      "source_key": "model.visual.blocks.7.mlp.linear_fc2.bias",
+      "stored_key": "blocks.7.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "be0f1974bfd8220b5705eb03cc73656d5af80bd72d9ad314cad66010a94e2793"
+    },
+    {
+      "source_key": "model.visual.blocks.7.mlp.linear_fc2.weight",
+      "stored_key": "blocks.7.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "613b645f34961897d789e856040bd72ef911b5f90d7c4f220edc63ae368aece4"
+    },
+    {
+      "source_key": "model.visual.blocks.7.norm1.bias",
+      "stored_key": "blocks.7.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "c3a5a56ebbf9fd793097dd662f87ea7b6ae6a509560f0f65fa5016ee87af87e6"
+    },
+    {
+      "source_key": "model.visual.blocks.7.norm1.weight",
+      "stored_key": "blocks.7.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "ae79ef0adfb422a71ec8b2e73f339c3f75d6f6654e6194937ec137e4fc3ff83a"
+    },
+    {
+      "source_key": "model.visual.blocks.7.norm2.bias",
+      "stored_key": "blocks.7.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "9d417bdf6bed94bee950892290a86ca970195dce24f7a4fdbb1e2a058d6c2cee"
+    },
+    {
+      "source_key": "model.visual.blocks.7.norm2.weight",
+      "stored_key": "blocks.7.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "0c98f7e075bb930791c41f0db436f91729787e2cfe983a041ef2d48b23416d5f"
+    },
+    {
+      "source_key": "model.visual.blocks.8.attn.proj.bias",
+      "stored_key": "blocks.8.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "d4059369dc9a761c283eedb8515347e7995b5f3fe457cf6dc5ec1c09514139c6"
+    },
+    {
+      "source_key": "model.visual.blocks.8.attn.proj.weight",
+      "stored_key": "blocks.8.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "cc15c3f0bb6e495e59f82810463454c5c4f109321ad2e5f24b9a87636e229884"
+    },
+    {
+      "source_key": "model.visual.blocks.8.attn.qkv.bias",
+      "stored_key": "blocks.8.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "9caf035624521bc5399d4cc93cea2fa8746335a025a90c7215380905c618b93b"
+    },
+    {
+      "source_key": "model.visual.blocks.8.attn.qkv.weight",
+      "stored_key": "blocks.8.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "bd0a00ef6d1c8ce4693d4da8fe740375cc7c1c29d7585649e33d902c6bd9e02b"
+    },
+    {
+      "source_key": "model.visual.blocks.8.mlp.linear_fc1.bias",
+      "stored_key": "blocks.8.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "f1f82eb38f18794de48b6fe82c872c958dae900e634584e4af881b907dc4b529"
+    },
+    {
+      "source_key": "model.visual.blocks.8.mlp.linear_fc1.weight",
+      "stored_key": "blocks.8.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "cb00bbeafc6d0cec9f0010a70d55e2b20ba551f2e054e6209b5be55be824c4d4"
+    },
+    {
+      "source_key": "model.visual.blocks.8.mlp.linear_fc2.bias",
+      "stored_key": "blocks.8.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "c3700a25f91f42bb058ba12a683a1d530ce29c844faecc97486042d8b16b70e1"
+    },
+    {
+      "source_key": "model.visual.blocks.8.mlp.linear_fc2.weight",
+      "stored_key": "blocks.8.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "1715953cd868e2a74c282d4c0e88939fc2b12550640173ba8705edfd8563448d"
+    },
+    {
+      "source_key": "model.visual.blocks.8.norm1.bias",
+      "stored_key": "blocks.8.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "7552227d1ddd71d5780e9ded54c96fec9d723e446126ccea760c65f31f7186d7"
+    },
+    {
+      "source_key": "model.visual.blocks.8.norm1.weight",
+      "stored_key": "blocks.8.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "3703a0f713f50d3de6ee20c70c1c13cc1b89904dbb8bcad8037bfc6f7df09e10"
+    },
+    {
+      "source_key": "model.visual.blocks.8.norm2.bias",
+      "stored_key": "blocks.8.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "9da81776e3a3b6b98ec668d6331b1b174f9c4c554461c62efffd4d3d8f5e3230"
+    },
+    {
+      "source_key": "model.visual.blocks.8.norm2.weight",
+      "stored_key": "blocks.8.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "60f27bf3028d886868ce407e5bd1b953f4bb57fefb035ca32b957562b3951618"
+    },
+    {
+      "source_key": "model.visual.blocks.9.attn.proj.bias",
+      "stored_key": "blocks.9.attn.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "08ab989846e1050de1f0807e768c52825493fc24a08095a246f129d9293b14e7"
+    },
+    {
+      "source_key": "model.visual.blocks.9.attn.proj.weight",
+      "stored_key": "blocks.9.attn.proj.weight",
+      "shape": [
+        1024,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "0d8b8516ecb8f6a8d81cce1f55325a5cde5d682c158d37e7a889733eb4c577fd"
+    },
+    {
+      "source_key": "model.visual.blocks.9.attn.qkv.bias",
+      "stored_key": "blocks.9.attn.qkv.bias",
+      "shape": [
+        3072
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "fb12ad88fdf52e20f8d46bcede21322cbc1c6912d0047216a4c3bbb71687926c"
+    },
+    {
+      "source_key": "model.visual.blocks.9.attn.qkv.weight",
+      "stored_key": "blocks.9.attn.qkv.weight",
+      "shape": [
+        3072,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "b457c7d033e417307a5b8bd4057cebc86b4b30f30b9704298341ef353ac5daa3"
+    },
+    {
+      "source_key": "model.visual.blocks.9.mlp.linear_fc1.bias",
+      "stored_key": "blocks.9.mlp.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "6cc5ab8dd007fe94dd56ba1dbb20ddb7f8d0cb14e83e62f6cc5cf489e761f696"
+    },
+    {
+      "source_key": "model.visual.blocks.9.mlp.linear_fc1.weight",
+      "stored_key": "blocks.9.mlp.linear_fc1.weight",
+      "shape": [
+        4096,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "9e9c3bbe0dada2b449f317cfb391f158e16b3f86bd750ee205bb1a91b25b5505"
+    },
+    {
+      "source_key": "model.visual.blocks.9.mlp.linear_fc2.bias",
+      "stored_key": "blocks.9.mlp.linear_fc2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "ca01bc12788e04ae72878494cb06a6a43f5d31b7ac977b121628cb6be6b5ebf8"
+    },
+    {
+      "source_key": "model.visual.blocks.9.mlp.linear_fc2.weight",
+      "stored_key": "blocks.9.mlp.linear_fc2.weight",
+      "shape": [
+        1024,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "9110b45920dccb3a4876414c9d84c489e526372dbc2414332e233e1f117d541d"
+    },
+    {
+      "source_key": "model.visual.blocks.9.norm1.bias",
+      "stored_key": "blocks.9.norm1.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "490553b048cde4248683d5bcfa8bd3f757c43992db6913e81db1b79d84b05e8e"
+    },
+    {
+      "source_key": "model.visual.blocks.9.norm1.weight",
+      "stored_key": "blocks.9.norm1.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "ed0e82ef5b3ec09ac4da1fa8e53d6b16aff2de7e119b6f3773062927753a8133"
+    },
+    {
+      "source_key": "model.visual.blocks.9.norm2.bias",
+      "stored_key": "blocks.9.norm2.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "e96a3880747aa9f42c73878ee71a927e631304f7508258a85c228c3dc6500cec"
+    },
+    {
+      "source_key": "model.visual.blocks.9.norm2.weight",
+      "stored_key": "blocks.9.norm2.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "8ed2c50e90f01fbfa26ff316da57af434a5b020473b1dcd920de42a803137a13"
+    },
+    {
+      "source_key": "model.visual.merger.linear_fc1.bias",
+      "stored_key": "merger.linear_fc1.bias",
+      "shape": [
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "d6658172d558ad239793f6a39558d1c967e7b0dd3e82f519ace2f65bd6c743ab"
+    },
+    {
+      "source_key": "model.visual.merger.linear_fc1.weight",
+      "stored_key": "merger.linear_fc1.weight",
+      "shape": [
+        4096,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "f1888b11250b29ae3511f67f0414d778f43437f22000b5b90c2b9bf6c28e5952"
+    },
+    {
+      "source_key": "model.visual.merger.linear_fc2.bias",
+      "stored_key": "merger.linear_fc2.bias",
+      "shape": [
+        2560
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "705f2e266ac98910d9172d9b335a7f7352df78cbdd6f8e637d15c54beadc9874"
+    },
+    {
+      "source_key": "model.visual.merger.linear_fc2.weight",
+      "stored_key": "merger.linear_fc2.weight",
+      "shape": [
+        2560,
+        4096
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "2fa153b8fb28097ec5d28f9dd6ea84ec9ac8a4c5b5968d867d99e238b7b02799"
+    },
+    {
+      "source_key": "model.visual.merger.norm.bias",
+      "stored_key": "merger.norm.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "ff652f828b74639b8547e8421ccb7f25c5ea3107d748e43dac6cef20702a2b34"
+    },
+    {
+      "source_key": "model.visual.merger.norm.weight",
+      "stored_key": "merger.norm.weight",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "16ebb133b30ae6750f69ca0d43032c82b0da0a39441679151970b5d254f7f33e"
+    },
+    {
+      "source_key": "model.visual.patch_embed.proj.bias",
+      "stored_key": "patch_embed.proj.bias",
+      "shape": [
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "a506a26a627cd872519f311d8de60e3b83c2898cdc8a2f5b9facef64fc160ca1"
+    },
+    {
+      "source_key": "model.visual.patch_embed.proj.weight",
+      "stored_key": "patch_embed.proj.weight",
+      "shape": [
+        1024,
+        3,
+        2,
+        16,
+        16
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "5f99071446aded5f294c4d746edc457a47379982e064c39620543fed13ac175b"
+    },
+    {
+      "source_key": "model.visual.pos_embed.weight",
+      "stored_key": "pos_embed.weight",
+      "shape": [
+        2304,
+        1024
+      ],
+      "dtype": "torch.bfloat16",
+      "sha256": "9262980077f73fc984d234482f7b9c88f1091bddcdfa9eb2d34f6794ea69cebf"
+    }
+  ],
+  "text_runtime": "0.1.3",
+  "visual_finetuning": false,
+  "base_commit": "not recorded in local source; identities pinned by hashes",
+  "note": "Original base vision encoder+merger. Existing text backbone and pointer head stay unchanged. Not a standalone generative multimodal checkpoint."
+}

vision/example.py

@@ -0,0 +1,20 @@
+"""Run from any cwd: python /model/vision/example.py --model-dir /model --image image.png --request question.json"""
+import argparse
+import json
+from pathlib import Path
+from PIL import Image
+from predictor import VisionDecisionEngine
+
+def main():
+    p=argparse.ArgumentParser()
+    p.add_argument('--model-dir',required=True)
+    p.add_argument('--image',required=True)
+    p.add_argument('--request',required=True)
+    p.add_argument('--device',default='cuda')
+    a=p.parse_args()
+    engine=VisionDecisionEngine(a.model_dir,a.device)
+    with Image.open(a.image) as image:
+        result=engine.predict(json.loads(Path(a.request).read_text()),image)
+    print(json.dumps(result,ensure_ascii=False,indent=2))
+
+if __name__=='__main__':main()

vision/example_request.json

@@ -0,0 +1,11 @@
+{
+  "model": "NeoHorse-Jev-4B",
+  "state": "Look at the supplied image.",
+  "questions": {
+    "color": {
+      "type": "choice",
+      "instructions": "What is the main color of the image?",
+      "criteria": {"red": "Mostly red.", "blue": "Mostly blue.", "green": "Mostly green."}
+    }
+  }
+}

vision/http_example.py

@@ -0,0 +1,32 @@
+"""Send one local image to either decision HTTP route using Python's standard library."""
+import argparse
+import base64
+import json
+import os
+from pathlib import Path
+from urllib.request import Request, urlopen
+
+def main():
+    p=argparse.ArgumentParser()
+    p.add_argument('--image',required=True)
+    p.add_argument('--request')
+    p.add_argument('--base-url',default='http://127.0.0.1:8080')
+    p.add_argument('--endpoint',choices=['decision','systemone'],default='decision')
+    args=p.parse_args()
+    path=Path(args.image)
+    mime={'.png':'image/png','.jpg':'image/jpeg','.jpeg':'image/jpeg','.webp':'image/webp'}.get(path.suffix.lower())
+    if mime is None:p.error('Use PNG/JPEG/WebP')
+    if path.stat().st_size>4*1024*1024:p.error('Image file exceeds 4 MiB')
+    req=json.loads(Path(args.request).read_text()) if args.request else dict(
+        model='NeoHorse-Jev-4B',state='Look at the supplied image.',questions={
+            'color':dict(type='choice',instructions='What is the dominant color?',criteria={'red':'red','blue':'blue','green':'green'})})
+    req.setdefault('model','NeoHorse-Jev-4B')
+    req['image']='data:'+mime+';base64,'+base64.b64encode(path.read_bytes()).decode()
+    headers={'Content-Type':'application/json'}
+    if os.environ.get('NEOHORSE_API_KEY'):headers['Authorization']='Bearer '+os.environ['NEOHORSE_API_KEY']
+    call=Request(args.base_url.rstrip('/')+'/v1/'+args.endpoint,
+                 data=json.dumps(req,ensure_ascii=False).encode(),headers=headers,method='POST')
+    with urlopen(call,timeout=120) as response:
+        print(json.dumps(json.load(response),ensure_ascii=False,indent=2))
+
+if __name__=='__main__':main()

vision/predictor.py

@@ -0,0 +1,2 @@
+"""Compatibility import for the bundled example; weights now live in backbone/."""
+from neohorse_decision.vision import VisionDecisionEngine

vision/verification.json

@@ -0,0 +1,130 @@
+{
+  "status": "PASS",
+  "runtime": "0.2.0",
+  "vision_version": "unified-v2",
+  "visual_tensors": 297,
+  "vision_probability_max_abs_vs_split_release": 0.0,
+  "text_reference_requests": 16,
+  "text_wrapper_questions": 18,
+  "text_probability_max_abs": 0.0,
+  "wrapper_same_single_question_probability_max_abs": 0.0,
+  "split_question_vs_original_batched_probability_difference": 0.0041863322257995605,
+  "comparison_note": "Wrapper compared with text using identical single-question requests; original multi-question batches checked separately without splitting. BF16 split-vs-batch difference recorded, not treated as vision drift.",
+  "image_smoke_correct": 5,
+  "image_smoke_total": 5,
+  "rows": [
+    {
+      "task": "color",
+      "truth": "red",
+      "result": {
+        "model": "NeoHorse-Jev-4B",
+        "answers": {
+          "color": {
+            "type": "choice",
+            "choice": "red",
+            "probabilities": {
+              "red": 0.9999173879623413,
+              "blue": 4.41447009507101e-05,
+              "green": 3.853620364679955e-05
+            },
+            "confidence": 0.999876081943512
+          }
+        },
+        "input_tokens": 137,
+        "image_tokens": 96
+      },
+      "correct": true
+    },
+    {
+      "task": "color",
+      "truth": "blue",
+      "result": {
+        "model": "NeoHorse-Jev-4B",
+        "answers": {
+          "color": {
+            "type": "choice",
+            "choice": "blue",
+            "probabilities": {
+              "red": 2.3972206690814346e-05,
+              "blue": 0.9998923540115356,
+              "green": 8.369787246920168e-05
+            },
+            "confidence": 0.9998385310173035
+          }
+        },
+        "input_tokens": 137,
+        "image_tokens": 96
+      },
+      "correct": true
+    },
+    {
+      "task": "color",
+      "truth": "green",
+      "result": {
+        "model": "NeoHorse-Jev-4B",
+        "answers": {
+          "color": {
+            "type": "choice",
+            "choice": "green",
+            "probabilities": {
+              "red": 1.2651908946281765e-05,
+              "blue": 2.656863580341451e-05,
+              "green": 0.9999607801437378
+            },
+            "confidence": 0.9999411702156067
+          }
+        },
+        "input_tokens": 137,
+        "image_tokens": 96
+      },
+      "correct": true
+    },
+    {
+      "task": "position",
+      "truth": "left",
+      "result": {
+        "model": "NeoHorse-Jev-4B",
+        "answers": {
+          "side": {
+            "type": "choice",
+            "choice": "left",
+            "probabilities": {
+              "left": 0.9987630844116211,
+              "right": 0.0012369066243991256
+            },
+            "confidence": 0.9975261688232422
+          }
+        },
+        "input_tokens": 140,
+        "image_tokens": 96
+      },
+      "correct": true
+    },
+    {
+      "task": "position",
+      "truth": "right",
+      "result": {
+        "model": "NeoHorse-Jev-4B",
+        "answers": {
+          "side": {
+            "type": "choice",
+            "choice": "right",
+            "probabilities": {
+              "left": 0.00043360565905459225,
+              "right": 0.999566376209259
+            },
+            "confidence": 0.9991327524185181
+          }
+        },
+        "input_tokens": 140,
+        "image_tokens": 96
+      },
+      "correct": true
+    }
+  ],
+  "choice_noul_score_executed": true,
+  "stale_image_state_check": true,
+  "visual_finetuning": false,
+  "peak_cuda_allocated_bytes": 9825804800,
+  "scope": "Local-image connectivity and text parity, not multimodal benchmark validation"
+}