Wald-Q4B v2.1 (checkpoint 056A0-c31): release weights (056A0-c31 language model + Qwen3.5-4B chat vision tower and MTP head), card, serving.json, evaluation/v2.1

README.md

@@ -27,7 +27,7 @@ tags:
 - gguf
 - llama.cpp
 model-index:
-- name: Wald-Q4B v2 (04701-c22, revision v2.0)
+- name: Wald-Q4B v2.1 (056A0-c31, revision v2.1)
   results:
   - task:
       type: text-classification
@@ -39,38 +39,42 @@ model-index:
     metrics:
     - type: balanced_skill_index
       name: Public index, Auto 0.7 (author-run with the official kit; not the organizer's Full score)
-      value: 54.18
+      value: 60.05
       verified: false
 ---
 
-# Wald-Q4B v2
+# Wald-Q4B v2.1
 
 An open 4B decision model: send a state and typed options, get a calibrated probability for every option. Served through a Jev-compatible `POST /v1/systemone` API on your own GPU.
 
-## Why Wald-4B v2
+## Why Wald-4B v2.1
 
 - **Probabilities, not text:** a calibrated probability for every option, nothing to parse.
-- **Fast:** median **42 ms** one pass, **148 ms** Auto 0.7 (serial, one RTX PRO 6000).
-- **Thinks only when unsure:** about **one third** of requests trigger a short native thought.
-- **JevBench public: 210/231**, against 200/231 for Jev 1.13.
-- **Decision Index 0.3: 54.18** on the complete public suite (our run).
-- **Reads images** zero-shot: **+2.8 points** over its base on CV-Bench, BLINK, RealWorldQA.
-- **Small and open:** 4B, Apache-2.0; GGUF from 2.7 GB in [org2ai/Wald-4B-GGUF](https://huggingface.co/org2ai/Wald-4B-GGUF).
+- **Decision Index 0.3: 60.05** on the complete public suite (our run; v2.0: 54.18).
+- **Thinks only when unsure:** Auto 0.7 thinks natively on about half of the requests.
+- **Fast:** median **158 ms** (serial, one request in flight, warm, processing time, one RTX PRO 6000, 200 DI 0.3 requests).
+- **Web actions:** Mind2Web element selection **92.3 %** (test splits, 6 candidates; v2.0: 84.6 %).
+- **Reads images** zero-shot through Qwen3.5-4B's unchanged vision tower.
+- **Small and open:** 4B, Apache-2.0.
 
 ## Scores
 
-| Benchmark | Wald-Q4B v2 · one pass | **Wald-Q4B v2 · Auto 0.7** | Jev 1.13 (hosted) |
+| Benchmark | Wald-Q4B v2.1 · one pass | **Wald-Q4B v2.1 · Auto 0.7** | Jev 1.13 (hosted) |
 |---|---:|---:|---:|
-| Decision Index 0.3, complete public suite | not run | **54.18** (our full run) | 57.96 (board public column) |
-| JevBench public set (231) | 204/231 | **210/231** | 200/231 |
-| JevBench-XL TEST (8,177) | 65.20 % | **65.78 %** | 67.85 % |
+| Decision Index 0.3, complete public suite | not run | **60.05** (our full run; raw 70.12, breadth 58.63) | 57.96 (board public column) |
+| JevBench public set (231) | — | **210\*** | 200/231 |
+| JevBench-XL TEST (8,177) | 65.17 % | **67.75 %** | 67.85 % |
+| Mind2Web element selection (test task / website / domain) | **94.9 / 91.5 / 91.7 %** | not run | — |
+| JevAdvBench flip rate, lower is better | — | **7.1 %** | 6.1 % |
 
-Self-run, not leaderboard results; how each number was produced: [evaluation/v2/](https://huggingface.co/org2ai/Wald-4B/tree/main/evaluation/v2).
+\* Measured on Wald-4B v2.0.
+
+Self-run, not leaderboard results; how each number was produced: [evaluation/v2.1/](https://huggingface.co/org2ai/Wald-4B/tree/main/evaluation/v2.1). DI 0.3 run files: [org2ai/Wald-Q4B-decision-index-results](https://huggingface.co/datasets/org2ai/Wald-Q4B-decision-index-results/tree/main/runs/wald-q4b-v21-056A0-auto07).
 
 ## Quick start
 
 ```sh
-hf download org2ai/Wald-4B --revision v2.0 --local-dir ./Wald-Q4B-v2 && cd Wald-Q4B-v2
+hf download org2ai/Wald-4B --revision v2.1 --local-dir ./Wald-Q4B-v2.1 && cd Wald-Q4B-v2.1
 ./run.sh "$PWD"          # POST /v1/systemone on :8000 (one NVIDIA GPU, uv)
 
 curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{
@@ -94,8 +98,8 @@ API fields: [docs/api.md](https://huggingface.co/org2ai/Wald-4B/blob/main/docs/a
 
 ## Details
 
-- **Training.** Full-parameter decision training from Qwen3.5-4B (chat); checkpoint `04701-c22`.
-- **Data.** About 15% of training tokens come from the train splits of public benchmarks that the Decision Index also draws on, reformatted as decisions. The rest is broad decision data: general decision tasks — rules and policies, classification, entailment and fact checking, tables and documents, web and tool actions, judging (about 40%); math and step-by-step reasoning (about 20%); and hard decisions written and/or labelled by large frontier models, including robustness cases with perturbed or adversarial states (about 25%). Public datasets are used as train splits only. Contamination scan against the full Decision Index 0.3 public suite: 0 strict hits.
+- **Training.** One full-parameter pass of decision training from Qwen3.5-4B (chat), with a light KL anchor to the base model; the released weights are 0.85 × the trained model + 0.15 × the base (checkpoint `056A0-c31`). Thoughts are the base model's own; no thought text was trained.
+- **Data.** About 23% of training tokens come from the train splits of public benchmarks that the Decision Index also draws on, reformatted as decisions. The rest is broad decision data: general decision tasks — rules and policies, classification, entailment and fact checking, tables and documents, judging; web and GUI actions; task-specific decision sets; math and step-by-step reasoning; and hard decisions written and/or labelled by large frontier models, including robustness cases with perturbed states. Public datasets are used as train splits only. Contamination scan against the full Decision Index 0.3 public suite: 0 strict hits.
 - **Limits.** Scores are self-run; confidence is not correctness, so set thresholds on your own data.
-  Answers after a thought are sampled and can vary slightly; one pass is deterministic.
-- **Licence.** Apache-2.0 for weights and code ([NOTICE](https://huggingface.co/org2ai/Wald-4B/blob/main/NOTICE)). Earlier versions: tags [`v1.2`](https://huggingface.co/org2ai/Wald-4B/tree/v1.2), [`v1.1`](https://huggingface.co/org2ai/Wald-4B/tree/v1.1), [`v1.0`](https://huggingface.co/org2ai/Wald-4B/tree/v1.0). Cite: [CITATION.cff](https://huggingface.co/org2ai/Wald-4B/blob/main/CITATION.cff).
+  Answers after a thought are sampled and can vary slightly; one pass is deterministic. Thoughts are longer than v2.0's.
+- **Licence.** Apache-2.0 for weights and code ([NOTICE](https://huggingface.co/org2ai/Wald-4B/blob/main/NOTICE)). Earlier versions: tags [`v2.0`](https://huggingface.co/org2ai/Wald-4B/tree/v2.0), [`v1.2`](https://huggingface.co/org2ai/Wald-4B/tree/v1.2), [`v1.1`](https://huggingface.co/org2ai/Wald-4B/tree/v1.1), [`v1.0`](https://huggingface.co/org2ai/Wald-4B/tree/v1.0). GGUF of v2.0: [org2ai/Wald-4B-GGUF](https://huggingface.co/org2ai/Wald-4B-GGUF). Cite: [CITATION.cff](https://huggingface.co/org2ai/Wald-4B/blob/main/CITATION.cff).

SHA256SUMS

@@ -1,11 +1,11 @@
 a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715  chat_template.jinja
 ddc63e1c717afa86c865bb5e01313d89d72bb53b97ad4a8a03ba8510c0621670  config.json
 62153eb6c69f2e1f426beaa8002b7186437e949c7588167085df14e10e9c0a73  generation_config.json
-e33ce57511cb924d2bd5559a143c37cf12f1df1e6de911f0a7d4fdbbc0b85eb8  graft-manifest.json
-d4ae76c16d404c665f84e10079f460221f4c1434c23e0b923bf9a6deb91c1385  graft-report.json
+4de700a433f11dd89ae29bf617722a96d4469d9e3b3dbff3ddd0faf450ffe442  graft-manifest.json
+9e6e9ab73a5b73006b815206cb5f8a5d8e4c64e8c07f1bdd4acef1f96c5b9072  graft-report.json
 a9d356d7bdf1ef4949e3e748e95b8e10ad9d4e2e838eddc38a0a7b6b94d1db8d  merges.txt
-10cb7cd22f524f1b33271f8432bcb09409bd879c11d999ea18ca7dd5f8828f5a  model-00001-of-00003.safetensors
-ed5e7bd4b2e67eee390857268a7660986aca32068c78e6f483e866c27f90b79b  model-00002-of-00003.safetensors
+1bf33f0d7f7722ae8f87bca21a9d5ebca8328962e6b170abd09d354d2c978a57  model-00001-of-00003.safetensors
+7cccb8e4ea4c9858c8c78754bee789f5ccc8b437836b307c7919a15165097aef  model-00002-of-00003.safetensors
 9a8340e453e63f70185e56fde60aeae26085fe3c39e20ac1662b4325f01a78eb  model-00003-of-00003.safetensors
 6db63a968a513eb0693feee5ccbc440cbbee43c6b644c0bd33e880ca7d0b54b3  model.safetensors.index.json
 27225450ac9c6529872ee1924fcb0962ff5634834f817040f444118116f4e516  preprocessor_config.json

evaluation/v2.1/README.md

@@ -0,0 +1,9 @@
+# Wald-Q4B v2.1 evaluation (checkpoint 056A0-c31)
+
+- Decision Index 0.3, complete public suite: 60.05 (balanced raw 70.12, breadth 58.63), Auto 0.7, 140,620 requests, 0 errors; official kit 62d2f51; run files: org2ai/Wald-Q4B-decision-index-results `runs/wald-q4b-v21-056A0-auto07`. Same engine settings as v2.0 (`evaluation/v2/`): frozen native reader 764f3a68, temperature table a0f72cd2, think budget 512.
+- JevBench-XL TEST (8,177): 65.17 % one pass, 67.75 % Auto 0.7.
+- Mind2Web element selection, test splits, 6 candidates (target + 5 negatives from the step's candidate list, last 5 actions as history), one pass: 94.91 / 91.49 / 91.69 % (task / website / domain), 92.32 % all.
+- JevAdvBench delivered-9 flip rate, Auto 0.7: 7.14 %.
+- JevBench public set: the card shows v2.0's 210/231.
+- Latency: median 158 ms (serial, one request in flight, warm, processing time, one RTX PRO 6000, 200 DI 0.3 requests).
+- Release weights: the 056A0-c31 language model (byte-identical) + Qwen3.5-4B chat vision tower and MTP head, as in v2.0 (`graft-report.json`); `serving-parity.json`: the release weights vs the text-only checkpoint.

evaluation/v2.1/latency.json

@@ -0,0 +1,4 @@
+{
+ "median_ms": 158,
+ "what": "serial, one request in flight, warm, processing time, one RTX PRO 6000, 200 DI 0.3 requests, Auto 0.7"
+}

evaluation/v2.1/serving-parity.json

@@ -0,0 +1,16 @@
+{
+ "text": {
+  "what": "release weights (738-key graft) vs the text-only checkpoint 056A0-c31, newdomainTEST 3,125 items, one pass, same systemone front end (frozen engine flags, temperature a0f72cd2), one RTX PRO 6000",
+  "n": 3125,
+  "argmax_agree": 3110,
+  "max_abs_dprob": 0.0313,
+  "mean_max_abs_dprob": 0.00394
+ },
+ "image": {
+  "what": "one OpenAI chat request with a 448x448 PNG (red square on white) to vLLM 0.30.0 serving the release weights with v2.0's vision sidecar args (--limit-mm-per-prompt image 16, mm-processor size 65,536-1,048,576 px), thinking off, temperature 0",
+  "question": "What colour is the large square? Answer with one word.",
+  "answer": "red",
+  "prompt_tokens": 222
+ },
+ "verdict": "pass: text differences are at vLLM's run-to-run noise level measured for v2.0 (max 0.030, mean 0.0027-0.0048); the image route answers correctly"
+}

evaluation/v2.1/summary.json

@@ -0,0 +1,34 @@
+{
+ "model": "Wald-Q4B v2.1",
+ "revision": "v2.1",
+ "checkpoint": "056A0-c31",
+ "policy": "Auto 0.7 (512 native thought tokens), frozen temperature table a0f72cd2",
+ "decision_index_0.3_public": {
+  "index": 60.05,
+  "balanced_raw": 70.12,
+  "breadth_skill": 58.63,
+  "requests": 140620,
+  "errors": 0,
+  "kit": "62d2f51de34a2de64906345b6bc3e98e27ff55c7",
+  "run": "https://huggingface.co/datasets/org2ai/Wald-Q4B-decision-index-results/tree/main/runs/wald-q4b-v21-056A0-auto07",
+  "note": "author-run with the official kit; not the organizer's Full score"
+ },
+ "jevbench_xl_test_8177": {
+  "one_pass": 65.17,
+  "auto_0.7": 67.75
+ },
+ "mind2web_element_test_6_candidates": {
+  "one_pass": {
+   "test_task": 94.91,
+   "test_website": 91.49,
+   "test_domain": 91.69,
+   "all": 92.32
+  },
+  "protocol": "Mind2Web test splits, the target plus 5 negatives drawn from the step's candidates, last 5 actions as history"
+ },
+ "jevadvbench_delivered9_flip_rate_auto_0.7": 7.14,
+ "jevbench_public_231": {
+  "value": "210/231",
+  "measured_on": "Wald-4B v2.0"
+ }
+}

graft-manifest.json

@@ -1,7 +1,7 @@
 {
 "model.language_model.embed_tokens.weight": {
 "source": "text",
-"sha256": "0ac6e47d29c761b54795e338bce5919a39e6ed7ddfacc58f065457d220e09e54"
+"sha256": "3eaa465b81e597ffd3d62727d2684333ed91882dd11a33b940181cf63f35d85d"
 },
 "model.language_model.layers.0.input_layernorm.weight": {
 "source": "text",
@@ -13,7 +13,7 @@
 },
 "model.language_model.layers.0.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "317babb029e8f6186bbf8d55062a2fcb5eda07901d926b7c479015b5e4b11952"
+"sha256": "8a56dc420427854d51646b58148116df8be307bd10c903868720bbc5baa1d8da"
 },
 "model.language_model.layers.0.linear_attn.dt_bias": {
 "source": "text",
@@ -21,19 +21,19 @@
 },
 "model.language_model.layers.0.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "4f24f69f9e9b9c18d79585b5a610bfbfec344df1aec8a04f4b196864f1b29251"
+"sha256": "e7dad268d9af817d9e8652ec86287efcd2f0b3163777caeabc0faedbbfcd4f71"
 },
 "model.language_model.layers.0.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "2baf42c3b5803f319db712848dba1a5ec44545b4c05dd9121561ce353a679a02"
+"sha256": "87f29e18f6e26c00518e37b2097dc86d943c6561404ad5aff335560e727c6fbd"
 },
 "model.language_model.layers.0.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "976c427c9e3da8af2c96e254809691908cd88f775703e9380ceb778c80a9d810"
+"sha256": "aef3518b461c7a2673980d5fa038ad92ab6990f419e0ebbb39ac062d19d29244"
 },
 "model.language_model.layers.0.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "edbfbff55bc800586a6d2830dfb2176180fb33fe3ebc75626f365e306d84a8b1"
+"sha256": "5aab7f19e0805f80f0aede34e75c63c49277912cdc04d9c9c81cfd1ee2e1993c"
 },
 "model.language_model.layers.0.linear_attn.norm.weight": {
 "source": "text",
@@ -41,27 +41,27 @@
 },
 "model.language_model.layers.0.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "c28f2ed084964553deee9137a7ca05d0782c2bf9fd0aa5bdbef015faaa5433f4"
+"sha256": "2f9de8c47a14e38974d470c8ea0022fe2030e180a4df64edf5f6d4f1c406fac3"
 },
 "model.language_model.layers.0.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "d7ec3eeadb2ac93f94d6aaa2ff3696daae8604194a209b1a7486d1aa8d1b6fb5"
+"sha256": "3f5652e0d1299761029b54b7d5f183cd917c6944440c41e1acf31879a0f0c5a3"
 },
 "model.language_model.layers.0.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "479b86a502bf51444dfd301e51b4e5d270e7606928cf61011f475d3c14b2029b"
+"sha256": "33f5790c2107781cfa370e97216ae1444c7b0c88a79e187d58933d410dd56c96"
 },
 "model.language_model.layers.0.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "f5d67368c2f25b82b84fb0e9799a1e9f92d4e13fd3cb7aa0404e5245f5d3417d"
+"sha256": "6d23354fc54a5c26dee8612865ee6406b9204f00d3de3408725a84763045e355"
 },
 "model.language_model.layers.0.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "3d616984beb54a99d0e0b7738e5e1e7dbc5162a35b26a4302fd01867239c4e5d"
+"sha256": "4a540ab80c4cb3d174c2ed4e109364158529c0ea4bf6ef4a1c9e50fc9e7ad4c9"
 },
 "model.language_model.layers.1.input_layernorm.weight": {
 "source": "text",
-"sha256": "8509e7c56a23120daad9392c93b3f1a439ae25ec3fd743f84fa7f2fca8484d5d"
+"sha256": "9e3564f81f62f18a3f3a22b651a2b4d3374d6db6d5a9360bcece538a6dc3fdc2"
 },
 "model.language_model.layers.1.linear_attn.A_log": {
 "source": "text",
@@ -69,7 +69,7 @@
 },
 "model.language_model.layers.1.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "04b0b1460eab895a47982186fec1c898929eb18ec5714bd0c59db15a420ccafb"
+"sha256": "b6eae26a8e80cef425f1496a5fe791eaa421cfa0dfd88488e08c976d94df2c3b"
 },
 "model.language_model.layers.1.linear_attn.dt_bias": {
 "source": "text",
@@ -77,19 +77,19 @@
 },
 "model.language_model.layers.1.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "fee8051fa7045f957bd8b620ae0c091cc0caca641a5c2b53ae459efd3fd9bd88"
+"sha256": "7ba6b0161432b54672be6c98b0d2038c9682fd96232b81183f81c17991747376"
 },
 "model.language_model.layers.1.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "e722c3061dc13fff4dce53ba0eeaa5c122e1917fe6bf9238e35b0f44584aeebf"
+"sha256": "cd0eac1788ceda5050be06f6aefc92b0003e31dd9e1a4417763ea3cb24bb43f2"
 },
 "model.language_model.layers.1.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "a59a345aea18182064e6d06b0da156a527f674d4a23666596023435f05b2b4a2"
+"sha256": "541ff9ac061bc9d9f5cb0376f780c3d1a0dfec7075a3d943b1da3ea98011bc52"
 },
 "model.language_model.layers.1.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "3f138de897b0f195242d744ad86c2a0d71b30365197499b83b10e413cf57eaf0"
+"sha256": "818ea3ddff93cdafb7c1cdafc7074bc35b64eb8b2ffb493f17341b3ac6c94eef"
 },
 "model.language_model.layers.1.linear_attn.norm.weight": {
 "source": "text",
@@ -97,27 +97,27 @@
 },
 "model.language_model.layers.1.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "566991242437523db752d1fa24c6e99ca400ecf5e18a734a9059eaa40263bf60"
+"sha256": "339abebc0b003fb197e7099f40fbf0c77146063ca9d4630591ab6c1baa03a947"
 },
 "model.language_model.layers.1.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "6aaf77bf67fbcb4ff981f3bd1900529a23062290b438e102854fb5998906a75a"
+"sha256": "9840e1740f1982594dc2dd741a68ccbbca323af1469d8c478d5d5383eecaa4bf"
 },
 "model.language_model.layers.1.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "40b9b95bacdcd1c1ca9c1b4b72ff0ed1f76c901b3a6811ba614dc9e580becb96"
+"sha256": "70e250f9f29f6e7e7e506f5dfed0583b40957d203e30c0abb4bde03fa7176a55"
 },
 "model.language_model.layers.1.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "b222e11fda44a09f11bee4e3f4a5fe32efab285971b9ebd67a020a8289140216"
+"sha256": "50ef27fe41dc640ed819ea5561c973a75e085524a9c9aa03fc3c5dbd47bd2511"
 },
 "model.language_model.layers.1.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "187266809dad17232ef794f6551a995aafa57d9e17ffb341cdbbd84654d58a90"
+"sha256": "73d370b53820a42546d9ce81f82213d0fe0aa0d5d9dbe63ee34b80be8a851806"
 },
 "model.language_model.layers.10.input_layernorm.weight": {
 "source": "text",
-"sha256": "74cf3309d48e441be746d5895b17cb308cacd3bef057062f7b62903103b00c4a"
+"sha256": "d140315287a1fbdcaaeb6c20211a8b88c7ca30cf5996dea1622f7bb1255652b1"
 },
 "model.language_model.layers.10.linear_attn.A_log": {
 "source": "text",
@@ -125,7 +125,7 @@
 },
 "model.language_model.layers.10.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "1233f2ef223739a8fd54355e51d4e2ccb82c75a52c3f8cac408109354f342ef2"
+"sha256": "16a501c3c9ddd5d6d0e40c228d1b053c741887295fbe90e1a0bece4214bda13f"
 },
 "model.language_model.layers.10.linear_attn.dt_bias": {
 "source": "text",
@@ -133,19 +133,19 @@
 },
 "model.language_model.layers.10.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "92bf49ef0cabb4c0d4b282a6260d78c144a127176d478c810b67d1270befb6ca"
+"sha256": "8d947fc0eceb444ce2f4f85e7f72495d32361b11ee3841cdc9847b8aee963576"
 },
 "model.language_model.layers.10.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "3eb8239536520d41eda2fb25aa36653de2e8fa496271e409ff5600da6dc914ab"
+"sha256": "1bfda582c4b5c84d6703be82cc0f6aefbeffeffcd35b4218341b2bce41bc38a6"
 },
 "model.language_model.layers.10.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "716603d09a0a87e979f16c5fae4fb73733a3fd079fa15560000da011ec496a70"
+"sha256": "8d80efd742d9282c895299df089da532fac55e03f8fee9fbe662991c65b4dfe8"
 },
 "model.language_model.layers.10.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "569a875ad605d1b16f0e21f8d1983cb955af4d1b12ec79fcd440a209683c9a2f"
+"sha256": "cea4d92892c9ffad3913c4ca9c0067bbe79e1b944463a3ed4339452f222e691e"
 },
 "model.language_model.layers.10.linear_attn.norm.weight": {
 "source": "text",
@@ -153,43 +153,43 @@
 },
 "model.language_model.layers.10.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "9382bdc6a03eed275e13587ad22d4e5524360353c9bba4bfaca5aa538f12ae7f"
+"sha256": "07ca5954b4198d9d82fc300343abb60f79f3b656c00ad5a0e228ba718cc5b8a4"
 },
 "model.language_model.layers.10.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "498864cb61c33799e77429bc43d64de412521b064f0546da3f8b14155aa35694"
+"sha256": "1ad2da8932e397f62d0e76733d46fe92459fa175a67c2b84cfbb3d11bbb74e2a"
 },
 "model.language_model.layers.10.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "b5734ee34186dccb03a41613ae0cc843518a43f8967ed3a05c0858780d39e266"
+"sha256": "0638edc4347d888a2dad88d1e037cdfadf26fe1a82e9d31ea3a96192fc688c63"
 },
 "model.language_model.layers.10.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "4b5355858a34b4d4ef72517ab35feb8b37e7288df5788a7ea7c1eb3f7c65f2e9"
+"sha256": "e8145bba5639919dc794486098d863d33c52156a9d78463cefd554abcedf5982"
 },
 "model.language_model.layers.10.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "2136501412547dbb9291aad7fcd7e146c386815649d061a88d5734f8fd04c7bd"
+"sha256": "97744a8af08103a2538084a0a3ee072ef4f5522fa5de8ed89acdd5d3f6ee74d3"
 },
 "model.language_model.layers.11.input_layernorm.weight": {
 "source": "text",
-"sha256": "bc9b0efdd1985ddd134bae4b935551edfcb03db31aaa4d805e6478d277a1e527"
+"sha256": "2612cd3d7f856058fe82ca2915a84b1c5daeb4db74e767d5d0506d8c30d40890"
 },
 "model.language_model.layers.11.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "d1097199aceaaa712d288c9b1cf955abf689ab5d6bfc199068568fa5dea30391"
+"sha256": "9cead6224f1f2bc33b5c3c16deca2a6ec1957ecb8ce434f7bffef16b3e5864da"
 },
 "model.language_model.layers.11.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "29e4c3a0ec449baf364981128d45a7b3fd48a0d95c4ffb589027f9dc549204f3"
+"sha256": "d3f6572ccaac875888bc5075d383420d6ce79f670ad4233becdb14f4abb75d8a"
 },
 "model.language_model.layers.11.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "1e8d44b0b218284d2d5a653ec507da5926cdde23da9bc8f038ab4775725d1a20"
+"sha256": "ca1ebf87fe402be92417be9bb2713a7c71bb6b4ab8f2441d56877423aaa734ed"
 },
 "model.language_model.layers.11.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "b0203b4bd17f6dafbb85d63a39faf523e7fb747c73e63776b76ba764b6ead00a"
+"sha256": "1c75983b55a2c9531eb3166ae171e0052eb1ecb8a9ef2039f46ce509f419352d"
 },
 "model.language_model.layers.11.self_attn.k_norm.weight": {
 "source": "text",
@@ -197,11 +197,11 @@
 },
 "model.language_model.layers.11.self_attn.k_proj.weight": {
 "source": "text",
-"sha256": "68e7c6a708b4d982bd7c33fb1f2e70061667809810bda9895b59012673ac7eef"
+"sha256": "4c0ff51dc953131b0a891e14bfcc1118143ebb7e3ac742ba1bc4e25c11f08fb7"
 },
 "model.language_model.layers.11.self_attn.o_proj.weight": {
 "source": "text",
-"sha256": "b8e5fc66fb5fbbf8816616aa9ceeacc23e8c16b171e37a1643dc38a0ebeae5b6"
+"sha256": "17dadbd15623a281248d795eef44c7f27fa2d4a9646ba2b867148cce5f61e043"
 },
 "model.language_model.layers.11.self_attn.q_norm.weight": {
 "source": "text",
@@ -209,15 +209,15 @@
 },
 "model.language_model.layers.11.self_attn.q_proj.weight": {
 "source": "text",
-"sha256": "c48c12319baf17c89c6d28d4c8336cddea6d9e193d6b1f3a81ff8a1744bdf765"
+"sha256": "f1f75e95b86328f46b13af2de7caabf1e3afe4d0b1e5a9c60589d3f4d1f00ee2"
 },
 "model.language_model.layers.11.self_attn.v_proj.weight": {
 "source": "text",
-"sha256": "9c39460c75e1e8c51ad18b3feeef0eb10b008d0ad9e6793d095fb5494bab2b67"
+"sha256": "eeeb062bb7a2cc86e3ffc0c80bf8ae51fee40df6345e3b898e494ffdfb5b3732"
 },
 "model.language_model.layers.12.input_layernorm.weight": {
 "source": "text",
-"sha256": "42f2aba02968b3d5fc10e633219a31f0cae89ddacda19456709292b1fcf6ae0b"
+"sha256": "2f633a68d42101c88bf6df8dd0f39cf05a1f77590fc03d7409e1c137153c0496"
 },
 "model.language_model.layers.12.linear_attn.A_log": {
 "source": "text",
@@ -225,7 +225,7 @@
 },
 "model.language_model.layers.12.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "2bbfbf2e3806c1f8cf8163d97cd9af9bc0c56137e2332dec5709b0129b18a6d3"
+"sha256": "5060630d7574bcb5d6a338aa0a9fa4e7b64b3e5b9d9d5b7e8775d57976fa0e4f"
 },
 "model.language_model.layers.12.linear_attn.dt_bias": {
 "source": "text",
@@ -233,19 +233,19 @@
 },
 "model.language_model.layers.12.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "92633557c6022a164af06000878f639093fb8ea233924a7a794e97781f2dc0b7"
+"sha256": "f76f3b8b74e5a50de36623f56e7fa092e41e288572cf6803d2afc87e4d86b0b5"
 },
 "model.language_model.layers.12.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "499406dfc9093ac3c2efce438cab9cfc7d1126729752f68440656bef625a9b36"
+"sha256": "fc42935ebd648100608de575b349ff367dc9a4b0a32621e6b86d5c341c5be114"
 },
 "model.language_model.layers.12.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "e953e71e01d4c9c2119edb116c44a541676619a029861e1a1b3805bc0c0ee694"
+"sha256": "e58a7161277aab322894d0303bdaae3f423a25158d118eed470e1e150124b36d"
 },
 "model.language_model.layers.12.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "9aff32eee74863dd150a491f33d9575e56b1f9cc1e0173a77ad52d123923af03"
+"sha256": "558e455006d8239357988b8fcebf85feb5c0aeaf9a3979c66e890870dcc697a8"
 },
 "model.language_model.layers.12.linear_attn.norm.weight": {
 "source": "text",
@@ -253,27 +253,27 @@
 },
 "model.language_model.layers.12.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "09a3994608cde166b8cd271316a02ea0a78c213cfbb6fb020fd3244b055d9549"
+"sha256": "ddae4014cb956ce3acf1fe3a06ad791df1f9067aca491e6e1795528c2c21b75c"
 },
 "model.language_model.layers.12.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "47375094d885f3e28290b11e367073ede4b4e32bcf7acab07c37bce77bd7392f"
+"sha256": "fef317995ab51c1ef6e2b9d3e32c339252f5b9beb7b1f005cbb453388ce2731e"
 },
 "model.language_model.layers.12.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "6d630f26c0a585a152e5e228184e0d052d0753a1c6bae8a50f94698bd62b14ea"
+"sha256": "e81885b0f6a93c724b4ef309c0d303205526cbf2f649176a2711a3f5cc04e336"
 },
 "model.language_model.layers.12.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "114874b85f0c008e952767a11e7a9cb9147b2a7c9f542d4715a61edb8c85d251"
+"sha256": "29d7434ca20eda67060d8f842f741913d6d500693c7c0e6a24c474ab0787b214"
 },
 "model.language_model.layers.12.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "f5cc239bb2eb15398f31c1953b9e6a81758f730e666a2d456b20830fdf130528"
+"sha256": "5b3a853136223ab7acc8000d3a5622741e1186a62ddea88c1f596dee3f7fc294"
 },
 "model.language_model.layers.13.input_layernorm.weight": {
 "source": "text",
-"sha256": "9758851b487c0ba1c1c2472d9afaa9da8737b36a62a7479a7e2e0ca4ce30bb2d"
+"sha256": "a0f68b8b087c1f8cd828e1e663210c3f893d162fa352b28111d25f3f754acbd1"
 },
 "model.language_model.layers.13.linear_attn.A_log": {
 "source": "text",
@@ -281,7 +281,7 @@
 },
 "model.language_model.layers.13.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "10b391db41818c8afb136a1f007ba4fa661ab7a26ab57f6f224199f6a5757abf"
+"sha256": "b0f1f037128254bae0bb898dea09b5b57524ca776efac80e8de6d302b1cae888"
 },
 "model.language_model.layers.13.linear_attn.dt_bias": {
 "source": "text",
@@ -289,19 +289,19 @@
 },
 "model.language_model.layers.13.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "42127963ea89163566c784d04fe64d4236ddc44d1a4d042f30d78526ca30bf29"
+"sha256": "9e8c3d63e20702debec1ed550187832c650ba880c3c145306b652631dbb2d375"
 },
 "model.language_model.layers.13.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "482f8147a9c1dc79d0ab6063c62c2990f1435d8d3245f2705741a0165da7da51"
+"sha256": "8e63d9f0c206969e64e88089733c5101cc6cfb0905e9a28cc28407fe6eca68ef"
 },
 "model.language_model.layers.13.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "7cdad916ae949833b4a60242626ef29071d8571e4a7c98c636c5d9df4e6667c1"
+"sha256": "499f3125ae2f00e42d4f698c7469d148a6be06fe8c4eaf2acb0cde2b9238e309"
 },
 "model.language_model.layers.13.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "8c9c7ea5e8f33b86d06d02445b4c7f56645473f3107c690a74bfee78b6dfce28"
+"sha256": "d6abad278d41d57e94aa2f95fffaa9b296c4b76ef3e421dc170b9dd903cb6a40"
 },
 "model.language_model.layers.13.linear_attn.norm.weight": {
 "source": "text",
@@ -309,27 +309,27 @@
 },
 "model.language_model.layers.13.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "6e7f1b5ccd9a2cd947861785946993751d6e593a838d6b4450a5c7e1410eadd9"
+"sha256": "0aa6fb3d6fbba8190b0664c4798b252e866c1e57a1f1b3c72ff303123e22dbd9"
 },
 "model.language_model.layers.13.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "ed2e4ed0979bc4e6907e8e0ad4281bf92ab3ac2bb9941e2c2b777a000ec52d40"
+"sha256": "7e933623285c8abfefd3d5ace7d9fca63125e4205fc4812f93cca761be64b51b"
 },
 "model.language_model.layers.13.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "ddcb7b956e5730a10bdbed9a6a00cac0dd099c55345393c7dd17e04b2fc5b285"
+"sha256": "06cf2d9b922c9fe8bc951cdd714ccae8660df8f9b05dc9625425833eb0746985"
 },
 "model.language_model.layers.13.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "3282255c37c28f2e77c4d45ccfba5739dcdfb6eb174e6372080b9c6d509c1a82"
+"sha256": "72be3bd9d48c7b8b4be1a6065e4a6268ca5754954ab1e82001add9f946318810"
 },
 "model.language_model.layers.13.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "165d8c10fda7c1a24f445b37d5f1967730012a8dbea4f3117fc8865a2134edf8"
+"sha256": "a8c69da77a95ec06302eae8d3e70689bdafc4f51e5303a69c11cbc2b70433e4c"
 },
 "model.language_model.layers.14.input_layernorm.weight": {
 "source": "text",
-"sha256": "f9529d505643f64f8216ec6b9590d36a4ede1a20fa25d4a59bf0b1989b16b2aa"
+"sha256": "dcd047b2973c2ac6bbff13740e31c739e916a5d3a934157c09d8001fdf4a92b5"
 },
 "model.language_model.layers.14.linear_attn.A_log": {
 "source": "text",
@@ -337,7 +337,7 @@
 },
 "model.language_model.layers.14.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "202e07ad84f218949993d87945954a8687c1b189ffc5d1ae278c74f4fab63d65"
+"sha256": "5b023efc5918ff4e5ce3851848769229659ae0e73ab1ae5110ab4506376127ab"
 },
 "model.language_model.layers.14.linear_attn.dt_bias": {
 "source": "text",
@@ -345,19 +345,19 @@
 },
 "model.language_model.layers.14.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "c5e4212efacc2e63cfb60838b255fdf9eddf7c1fdf76d11a67ec5326ca21e730"
+"sha256": "6aed906adb0de94fb58f9fb76d86f59cebab0992e1482bf7ab292548a55e1d08"
 },
 "model.language_model.layers.14.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "0e27f103515e113483f8b317592e7d249c55bde825d10485375f70bb03cc3f1e"
+"sha256": "d52bf0221c2b3e3538a3a587542254c75de8c6125a4a64a8f5992648e52f7b97"
 },
 "model.language_model.layers.14.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "998601dfc5a92473eae9e1baa35bc175bb3d71151afef3fe5af4b458529813c7"
+"sha256": "58eae90b4d4e29b4873f40c5f18b058ef2c026973f0064edbda364246fa2fbe1"
 },
 "model.language_model.layers.14.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "e5674fd09d43b65879ea55793949482ce6ebfb00fa51534769d12a4f82ab0937"
+"sha256": "23f75e86178d4e63beb0d5f3c47100941891a7a379c16c95c5bb181716776026"
 },
 "model.language_model.layers.14.linear_attn.norm.weight": {
 "source": "text",
@@ -365,43 +365,43 @@
 },
 "model.language_model.layers.14.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "ae1b8278c315f4a1aab98f9d7b8c9724189aa60b53897ce7287d6bd9f0c05ed5"
+"sha256": "bf9cf083d7aeb94a588d17cc9b16bbff7d28b7b0232e8ece753b11fc917b0838"
 },
 "model.language_model.layers.14.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "556602d079332d74bfd24fd0a4aaa38e69e3e0a182b3b1d535286ad4185b1b14"
+"sha256": "b55289555a2d8d12971b3cc928c31cc2b2ae7937ef996e7eb2673889a029c22d"
 },
 "model.language_model.layers.14.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "3159a404d3cdaeb04df6be3e6f7c6ff25072dc3286ce6befb02b64db1ca89d6d"
+"sha256": "ec127e58421e5ea1d076a5d8cf96ec6d88a8f2e48a0c26ee6a684428f5a853c3"
 },
 "model.language_model.layers.14.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "557a4c0547eb013cc26ec41b8dcbe28601b034cc7db486922a13141b125d10b3"
+"sha256": "402cd38392dd9b61a84b5ee6a7a4c29dd119f52f5fad94f3953aa0dc86725619"
 },
 "model.language_model.layers.14.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "59a97dd78b03fe4e63f6b058b815eab830dc904e72935b49e0228f212e1d579c"
+"sha256": "2c8cf2fb873b339679fd8b76a01a31d26fadfe434757d03a8354be3565abe071"
 },
 "model.language_model.layers.15.input_layernorm.weight": {
 "source": "text",
-"sha256": "d3599eaf19ab166f53bbe3ec5f4605d391797f361174c8d162dbc8a15c3ab657"
+"sha256": "0508dd7c0e4546350e7069b46e7d8d58d7aac4ce2af72125ce93f47e47ab57a0"
 },
 "model.language_model.layers.15.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "c064062aeb5b23ec6ca5cf46787f53c1c12602b19758b57112a07e3e12e10037"
+"sha256": "b1cc68112ed2283dbd9939e671df62cc81081b2828b2d37818be5bc2e1c7cc78"
 },
 "model.language_model.layers.15.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "6eea039c8d846121d697308b858798d9e4aefd757e4b3e72dcba552edc14288e"
+"sha256": "085c89b26870b2caf35a8461dcc1c95ab8ef3320a9306622ed559eb3f22459f1"
 },
 "model.language_model.layers.15.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "5863f05c920cacc1e43040002050e716645b5df6bdcc5ef18f26e625a2c5be5f"
+"sha256": "b6a6a40652b89e2a8104dc748a9f17e2494ef2b4f172644a2bc3252595ee86de"
 },
 "model.language_model.layers.15.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "d32cf3d5377cc9a32bfb987057ebf23cdb45172eed9b6bf0d027d9dde06012d3"
+"sha256": "22b16448210e2dc0673e8102096a35d809d9136dccfe051c4a7bd7256657f64c"
 },
 "model.language_model.layers.15.self_attn.k_norm.weight": {
 "source": "text",
@@ -409,11 +409,11 @@
 },
 "model.language_model.layers.15.self_attn.k_proj.weight": {
 "source": "text",
-"sha256": "26e90e78e3b576fbda7a5083c51e93698315ff2ee919fbbdd23fb9b10e0e82ea"
+"sha256": "95110755ccb0ce89f2679ad2197730fc31ea4825130a63dfab30bd083df44dc6"
 },
 "model.language_model.layers.15.self_attn.o_proj.weight": {
 "source": "text",
-"sha256": "2c790e75670a31486eb181922d5c0f063c1b0c2c70923d84a9890ba92ba7efb6"
+"sha256": "ca496f30464a0fc00a5c4358cc511d48f16768dbe6ffd6d3564e7117a1f144c9"
 },
 "model.language_model.layers.15.self_attn.q_norm.weight": {
 "source": "text",
@@ -421,15 +421,15 @@
 },
 "model.language_model.layers.15.self_attn.q_proj.weight": {
 "source": "text",
-"sha256": "45a4a6a5d67ebd7264c1dc455d8b7a4c2bfdf5f46a41a40e5c63c056254d2c11"
+"sha256": "1a37cea7f6122d77a2eab813ac191e11b92892ed6ae838ddfb206a3917d1e139"
 },
 "model.language_model.layers.15.self_attn.v_proj.weight": {
 "source": "text",
-"sha256": "a3ba6b5b307a9c38f1ad5b202377016b870a997f949e213b22b8de4a97267c23"
+"sha256": "c0d2b4aba9ba3fdc9bd035a062b72d5de522e9694987071539bb38c7ff5f3818"
 },
 "model.language_model.layers.16.input_layernorm.weight": {
 "source": "text",
-"sha256": "2f53cc47dc68f6ee9fce020875c676bb8769ab117c5e01583840b8a2a6ed0ea8"
+"sha256": "4f80813942550ea6cae1a77eb28c8686b2c3a7006780d211a97a74c39ec9689d"
 },
 "model.language_model.layers.16.linear_attn.A_log": {
 "source": "text",
@@ -437,7 +437,7 @@
 },
 "model.language_model.layers.16.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "41b7f0ce148ae671867a54b05013a5549e7326e9d8bf3b17cdc2497ba5b2ae7d"
+"sha256": "33ac5c1bd8e515a849c4ede6521c55c8de2ec335a76608c5ab8f9e9a6de69b4b"
 },
 "model.language_model.layers.16.linear_attn.dt_bias": {
 "source": "text",
@@ -445,19 +445,19 @@
 },
 "model.language_model.layers.16.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "1a5b28870bb371f4fc70521699e3a6af97b6591a0964b3812f17f365e606e824"
+"sha256": "62cc20879a4bcdcabe1d2db6d9ba716be1d086dbdecf773d9a68f9d3ff49b655"
 },
 "model.language_model.layers.16.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "f16671d9f54848195ea069f61794df9128a781f2549a5664925a7d35f94ff25a"
+"sha256": "000523d077aa31226aacd8167d8599d2bf4d61a5822884409b8a6f0abed4e64d"
 },
 "model.language_model.layers.16.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "47c654649a885ff486aa97196533feca85adba84c75557163150da6e3f1f8b63"
+"sha256": "b934500a77a1409b18e97bd76e7d49b64acae7a8d4c8301af5aa6c3fd0a37e73"
 },
 "model.language_model.layers.16.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "10cd584375d42e8dfcea8f3625c3d0ce99edd74d999bd349d4e5510127037966"
+"sha256": "b0632522a7d690d769cb3cad99cfcb13a4fe88d6ebd6b44fccecc6525f1e1633"
 },
 "model.language_model.layers.16.linear_attn.norm.weight": {
 "source": "text",
@@ -465,27 +465,27 @@
 },
 "model.language_model.layers.16.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "c397251158f3595a1ed3f2bbf657a9f9cd25e10d75ae988dcfb3ea22da400cc9"
+"sha256": "3d0c0a976378d7c6249aaeef51df12677e2f740abe29924411c83eed58d51196"
 },
 "model.language_model.layers.16.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "b1a9441828a42abd971f7e6c9b30a7f5a96a282f1c51e4a807db2114b71c3295"
+"sha256": "2cd7973bc03fdf107f1c25408936037efcb5df1596eba3fe1d35d0fe414f6760"
 },
 "model.language_model.layers.16.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "2671c14190538bba0c21a9a7ea2d7d1f1a3011a9ede0e09b634eefce4d0818a8"
+"sha256": "dd1ae125fc5404a7e2eb0101eb419727c8d1d9b0c5121e5f74c286e31baf9165"
 },
 "model.language_model.layers.16.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "c60372819b3f47cc00b882b07be63b65139f1fcdd409d1afe2ebe26986c042ed"
+"sha256": "b891392f389e0994d851b72846604ae810ee7e7d8eeb15b86145dd91c1d1cd89"
 },
 "model.language_model.layers.16.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "aa3e26d6569e12535b625440e5aa67db004b8d06b0a76f72c74271c4a2df0030"
+"sha256": "18393150a618eea3ff8822cf5c1bd99005620b8816cba4a9d61baa6f842774d0"
 },
 "model.language_model.layers.17.input_layernorm.weight": {
 "source": "text",
-"sha256": "5d108a661a3596a1d45126f2e4deb7559508f58357e4c0f66511353094a5703c"
+"sha256": "fbc2ec34e15c682054ce277b37172ee56b52e699a17f0d49687f217896a823f6"
 },
 "model.language_model.layers.17.linear_attn.A_log": {
 "source": "text",
@@ -493,7 +493,7 @@
 },
 "model.language_model.layers.17.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "93ffca59db397adf24ba5fae5f6e626b28e257be5e511e11ef76b4f0bec48dcc"
+"sha256": "795dc0d06621948bb8ca3e5b837b59e342120e6526427ddc0bc056070e621020"
 },
 "model.language_model.layers.17.linear_attn.dt_bias": {
 "source": "text",
@@ -501,19 +501,19 @@
 },
 "model.language_model.layers.17.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "893f1c3beb965bc8510dedd577c7fd202f93b807dfae1183d9311477ed656f35"
+"sha256": "8a55dc9ffecc57257cafcbf5d8f1a5ec01b81c9772982fe01703b0c0bbfff88d"
 },
 "model.language_model.layers.17.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "8be3c794710c339684314ed1648709a96992ea492d8bb160cbaf9aafdec5c5b4"
+"sha256": "e4b940a8a0744e0c9dddb3ed89e1fa29254c63c5669ec5346f9878c7697cd69f"
 },
 "model.language_model.layers.17.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "e7465c4e7a145240f96b71bb0adcda0df7d1cd7f98c0564b788d0f1081f67e5e"
+"sha256": "f979e8be1367c36410b3e08d80744c94188bd91eddc1a2cfb8b1013e786143ea"
 },
 "model.language_model.layers.17.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "380f839f2b7c415de31b340d69aef5d3c7c6a0571ecb7c75c3046915e7bcea35"
+"sha256": "a3abfa28e91216de3c9973fc5c0876d94d158bff6869417bab6f125bd3abc984"
 },
 "model.language_model.layers.17.linear_attn.norm.weight": {
 "source": "text",
@@ -521,27 +521,27 @@
 },
 "model.language_model.layers.17.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "3687c8445f3699fc013fed78f1772509f0e38299dfa446f9df9e19f703de9207"
+"sha256": "e2de0f2ba91fb1144a4aea54776500af3adf057eb04e9dffaebc95b49eb80894"
 },
 "model.language_model.layers.17.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "d2a9660b5eca6252b77014415ac5f6a9ad3a074fcbb3455f611a857ae8384d4c"
+"sha256": "570e6b7ac435d65e767949f41b58d28e9c9e762d747267bce4bb29f6e4107e99"
 },
 "model.language_model.layers.17.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "db11d710d50eb2267254faeee1c16e71039a6454523c163757cdf7f25a0608ec"
+"sha256": "207639573ae36b9744ad172d6a9e7d82b581a74416959cb9e5d23cced4b1b9c3"
 },
 "model.language_model.layers.17.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "1a73eef863550c86610d3a50ba954440b36024074f36f1d5242e6f311b8e15f5"
+"sha256": "9d8df9ba87f38a5d4dd51aa6b75fb85797601d008bf3b311005c8b4bc41b449d"
 },
 "model.language_model.layers.17.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "beb85b5fbc82c9f0d6e8e94c867b4b3abba7009e07875e64444bd61a95ba1783"
+"sha256": "3876275389dcb05bd78de07be3ee90745de2fecc2da5c8ae1ac15007a678f75e"
 },
 "model.language_model.layers.18.input_layernorm.weight": {
 "source": "text",
-"sha256": "645dbe92a08352dd036a06b11480ec0b667315af261dafe1e10690cabfc374ff"
+"sha256": "bca553b79a0d442ae8075e71458aaf17b3cd75d1dcdcc2e59ac74b16b197b2af"
 },
 "model.language_model.layers.18.linear_attn.A_log": {
 "source": "text",
@@ -549,7 +549,7 @@
 },
 "model.language_model.layers.18.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "70e906eed4609eecf6653eb8f21f42c68fb43ded06930a3ca292199cb2366bdb"
+"sha256": "cd4facf5b62640b4b744ee246e0001a00c4aab67a97554fed84613b1abf95152"
 },
 "model.language_model.layers.18.linear_attn.dt_bias": {
 "source": "text",
@@ -557,19 +557,19 @@
 },
 "model.language_model.layers.18.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "838fc5c714b955f2f87450d6034d5d64d6b3a76fb0766a2a565272bed0807982"
+"sha256": "8dcc7babcb27f4e3fe7563db85654e963d39a9ec9c26711d3f3dd3905ff4bfdb"
 },
 "model.language_model.layers.18.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "a6347850c9e76ec56691b172c4c9ad6fe2898ee0ffb3b865cc409037b41ab0ab"
+"sha256": "bffdde9cf952d5046b22aeaf83d54a2787b66ac85fe76a38180ca5230fb32461"
 },
 "model.language_model.layers.18.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "2faa8c71fbbcf4eac25996418f7b9a93689754c23950c469a4bc00553cd9b52e"
+"sha256": "de5eb58047a232d94babf3e988e30ce252544ee696d78b991b1159a9c88c1238"
 },
 "model.language_model.layers.18.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "4903b906f95f38f34e9bb4d133c2011dd427a7ff8711adccc902ecbf328d0e5b"
+"sha256": "5ff163e50d09182ea93c6ed46e2c1a7030cb67f72d3b2f6876ccffe7192a3344"
 },
 "model.language_model.layers.18.linear_attn.norm.weight": {
 "source": "text",
@@ -577,43 +577,43 @@
 },
 "model.language_model.layers.18.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "54083642cf6a4f3ccd0a71c2a87606c31037f1e42515a8f6bea0efdbbf9b08bd"
+"sha256": "7d4c82f6f4724bf8daa2e0335a18624874d021761ee4efa4d5d7bc817528709e"
 },
 "model.language_model.layers.18.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "5e925c2f547ac12afc0b03998aed9d652fcd5ca3dac593f7ce170cb21af929d4"
+"sha256": "f7cc3f0d9132d0053c872c58124587346b4d7b7168db836a6522a8b7a64d0b26"
 },
 "model.language_model.layers.18.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "7b8545258d3d9e8f5e026c4abbbae5aa6dc967dac048f2bc7d8d7dd7eb1e8320"
+"sha256": "1579ba954d1000099cd683a20962e7634a76627e4c328a995592b00309346732"
 },
 "model.language_model.layers.18.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "7e76e76e77aebba70ed477087777ff5693a1b88143925a9247f1f18c64eedecb"
+"sha256": "68e129e3d1eb103fb3c4a08209a03ffc2166e1c89b1a8f3ca5f5583071da3574"
 },
 "model.language_model.layers.18.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "a79adf8aff689be707c93f012671bc487a3d71327783fb1cb6edd34bb414219f"
+"sha256": "9a433b219a42b19cce7b57f949080cf68448982298e14e4ba06884a9428fee2e"
 },
 "model.language_model.layers.19.input_layernorm.weight": {
 "source": "text",
-"sha256": "a27ba0dc2d37dfeb21bc4a4f9125ce8509abd64051c7d2d46ce30c2753345ae8"
+"sha256": "bca82bf6279e6fe0a730aa318abb287f4087435cd53220e40a456933e781cec0"
 },
 "model.language_model.layers.19.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "b38b43d5c03dd9867cedd75d32ca9b8d3e4f81fb8cf3af5ae3de5abbc4ab2f24"
+"sha256": "f58146167cf799f5c3463e0e2c46146c51bfa43a2cf7706e7e28cb419c7a2801"
 },
 "model.language_model.layers.19.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "ff5e560dae134c1bae2f192d0e16e24fa62774bd9ab2357dec962a73a0de4d8b"
+"sha256": "ee73255f26dc04b7fbf367025e47093e303dd3ff2d2367e438180271831b934a"
 },
 "model.language_model.layers.19.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "03aa84c1b18cd18bc8376af55d14596588645d0c024efbae9caa0a8edcfa2bd2"
+"sha256": "6f629d12399d73977cd4700ae8915f0a484a70ab725cb1708891feb51c6dc2b1"
 },
 "model.language_model.layers.19.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "aac3c83a8a612ed58f34118e1fc96cbe11c72cf3669185d26e891a6826698f1f"
+"sha256": "d020104a8b412ebbc3cbd088523847c050ae443c72f75130827c21af60a666d1"
 },
 "model.language_model.layers.19.self_attn.k_norm.weight": {
 "source": "text",
@@ -621,11 +621,11 @@
 },
 "model.language_model.layers.19.self_attn.k_proj.weight": {
 "source": "text",
-"sha256": "0f35b24e7999c00143bbe44d8003dac4d32aa7cd3bfab48c4a94e5af0e55b541"
+"sha256": "4c2747817e44f76d0ab920eed067ba5a32eea4c9bccae97b6e585255c7837031"
 },
 "model.language_model.layers.19.self_attn.o_proj.weight": {
 "source": "text",
-"sha256": "4cf2373a4635777366bde4e87d46b86724e0d5a4f467c2102aff3540e6e4dda8"
+"sha256": "0ad525e5c1486c29e2782d513a24ed4a7ba630fa7c76cfcf7c5ffcde3e201c94"
 },
 "model.language_model.layers.19.self_attn.q_norm.weight": {
 "source": "text",
@@ -633,15 +633,15 @@
 },
 "model.language_model.layers.19.self_attn.q_proj.weight": {
 "source": "text",
-"sha256": "0930f392782273b021debe25dee12a32dba036ad0aac09953f1c9d40f6feb2b6"
+"sha256": "25d538123e77f8f7da8daa695894339deeae30683524aeb4a4f4a7e571c08dbb"
 },
 "model.language_model.layers.19.self_attn.v_proj.weight": {
 "source": "text",
-"sha256": "de9f60ebc3fc329971c2cbe27120bd967deaadabb0432b2fab9cdbee24a40d07"
+"sha256": "3b0dcfeb6b4c0b1a9d7feeb82e245e1633e34b50cd35c4abf493a669991bdc3f"
 },
 "model.language_model.layers.2.input_layernorm.weight": {
 "source": "text",
-"sha256": "d66d26a76246806cc1d30e704dc9b78c82393a9afbfaf35990e0df45bc800fef"
+"sha256": "e83838e6de02a666e076e9ffa2373417a78f1ce2aa0486c92581dee5ef321cea"
 },
 "model.language_model.layers.2.linear_attn.A_log": {
 "source": "text",
@@ -649,7 +649,7 @@
 },
 "model.language_model.layers.2.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "d0fa8d69130cc6f8ac24b04ba69aa2115dc219002510df6fec5a510b8e9f3f02"
+"sha256": "b4e56ade024e012ae926cff848affb2df6753066779a103d8f80ccb8783eac52"
 },
 "model.language_model.layers.2.linear_attn.dt_bias": {
 "source": "text",
@@ -657,19 +657,19 @@
 },
 "model.language_model.layers.2.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "6cbfd18b14092ee429d634bd8d9c3aa7f571f4209ec3722c6873fb85814598eb"
+"sha256": "a91a5827414b764d9359bda980b1f32b2ce76e30f64b9ddd82e555ed7f276bf0"
 },
 "model.language_model.layers.2.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "cf2c86622cc886bf8701e025fcdf4a51f471c8a8836a205d6b246b949f24ef2b"
+"sha256": "a9e00149117815e84875793bf25641941b652d2afe3c31aaa64f9baa6751a86f"
 },
 "model.language_model.layers.2.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "72f3f132ba889dfdee0c0ddbe55fb5dabdc091ef9c3b533a7a48edf11b276a37"
+"sha256": "551dadec3a014ab4b1171f2d7364a1c953faaab8091253650e7552842464157e"
 },
 "model.language_model.layers.2.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "2b8146f80b82fc2812570d997c34c4a669ae93de71bd64ba97a7c1f5d169d60b"
+"sha256": "2611126c75191656ce5cadeeaca0d34c271fa1d8efeb2abb1bd118fbf0ed38eb"
 },
 "model.language_model.layers.2.linear_attn.norm.weight": {
 "source": "text",
@@ -677,23 +677,23 @@
 },
 "model.language_model.layers.2.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "f3f4164b7eba7b053cf9960c87043d114d6e9842a7e6a461e992cc2310780d72"
+"sha256": "58883278f32f77ccd361d923f2fcb4251df627f4f53e786d5e1a40e11a48e167"
 },
 "model.language_model.layers.2.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "764402f87640cfb63753f2bcdd3ed85abd494b60d504a50e266ba7570f444add"
+"sha256": "fa07b31c687af4bd413afaed2a0c7ffd663d1ede0a68d37a77c4ba199c21ef14"
 },
 "model.language_model.layers.2.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "f16a685132cccfca1ecbc08b4c6edc0fb31935ef492918b0afbb82bfb07eceac"
+"sha256": "6d5664eb22e465c44ede0285431523ef37bd80f03ab24e594033512d7c321469"
 },
 "model.language_model.layers.2.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "ee7dc289049cc784068ac7476a215e7d339e9946fdec5027fc8549aa6c1a44bf"
+"sha256": "9c6258afe7490c2a8f4042966160ed6940d2a51cc0b3d379b08bfdcff3be43c0"
 },
 "model.language_model.layers.2.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "2bef3f0fe6bc91135e6195b66d34142c2ad892cc09388be9076e3e03207067f4"
+"sha256": "4c0c0988b8523ec6d92ab9448ee3d044ac48ec414c20ee5b9b43b682d974eb94"
 },
 "model.language_model.layers.20.input_layernorm.weight": {
 "source": "text",
@@ -705,7 +705,7 @@
 },
 "model.language_model.layers.20.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "50d886ffba4dce87b232eca7698aa429d56c25f7ce38288315950a1979078abf"
+"sha256": "7f78c0cce89d3b844e2b2868f2cd7d362baa09d42b4c7bbc3b9051d2c2fd3d2f"
 },
 "model.language_model.layers.20.linear_attn.dt_bias": {
 "source": "text",
@@ -713,19 +713,19 @@
 },
 "model.language_model.layers.20.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "ee4be731cf5e266f294e33eef8a4a90c033531e93b1ae7a4b95c3a825b746496"
+"sha256": "ce4d7fe5a8885e2ac488c24f46e35cf3083a065c0d38bf6e4f1edbb84b7e7002"
 },
 "model.language_model.layers.20.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "bcf8f6f72cb488074ba4993a3669a532664a10f0678e63c273f04d9a1ad42e31"
+"sha256": "89fa7669e5e04eaa5c0c7eaa8599df27876d3bbc45c8b8e72a659b7e7cb0bb7c"
 },
 "model.language_model.layers.20.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "024d23cbd1802adccb72612d5e142facdcaaf1939f84d1bb3cdba2410778f62c"
+"sha256": "825bb9bcd722e322fccb46ef006951c45866fa08e1fbd7dd72cbf632be5ce1ba"
 },
 "model.language_model.layers.20.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "ac0c8fa600697ebbbd5a2368b637cae84668fd7c79db0a1be082965c9ce873c0"
+"sha256": "85d85547597c7228e5a2a37dad4a651e4ecf07f96bc7b7fe75a3c246d70a4ea3"
 },
 "model.language_model.layers.20.linear_attn.norm.weight": {
 "source": "text",
@@ -733,23 +733,23 @@
 },
 "model.language_model.layers.20.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "da6f5cb6e0b8e81d306b04ec8c220c64f960f45c078bde8b7ca6487ac606b4ef"
+"sha256": "4b331d1c4f80590c1d0f0e85e331703fdeb5a8383e6c7860a841d0226a807cd6"
 },
 "model.language_model.layers.20.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "1eba84f49ba7513ece86d6dca869e0089888c4d4f3a2ecdf408838834919bbd7"
+"sha256": "ef1bcd52a932eb82e49dee7ea8e1c1a82ff50e6f44c09ea2e75484eae733cacf"
 },
 "model.language_model.layers.20.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "2bc92bcc022b6ab46f8573b775cb9010e385f7030eca03e703052dbfeb69e1af"
+"sha256": "c1f6be8dedc7086614e9f396ec2cabcb93746253edad7452ef7e57d936da106a"
 },
 "model.language_model.layers.20.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "6add50c32834039619bdab8c73f3acf177edc033ad0d228e20ef1d55cf123b99"
+"sha256": "1c5e02a3c209f92ce6cbfc6fd7dc911fd97447b6462a0b5f0a4beb2ba210cea5"
 },
 "model.language_model.layers.20.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "f7fadcc396fe9dbb1bc8d86f37c643d837f3c5f915aad4f9198c7817dd1ad253"
+"sha256": "6b9ba29169f4c4ea05bb5b8fc4fcb4ca892fa8842b3ca8d298795bbb1b1ae268"
 },
 "model.language_model.layers.21.input_layernorm.weight": {
 "source": "text",
@@ -761,7 +761,7 @@
 },
 "model.language_model.layers.21.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "3f5026e237d21956e2580e6b5497b5824d32cdc918f594ce7d9f1abf58803778"
+"sha256": "9d094bedc3f0a7703af86c9120a219275639e6c8e5fc1922a5b8725036027dad"
 },
 "model.language_model.layers.21.linear_attn.dt_bias": {
 "source": "text",
@@ -769,19 +769,19 @@
 },
 "model.language_model.layers.21.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "9a580dfd9778c7a8276a0363f48dadcd3aaa5cbc5b4db218eb69f1619c540952"
+"sha256": "762541dcb0ece6730711ffe9ec3319c5487f3ddf210208ec28ca6018e30f9e05"
 },
 "model.language_model.layers.21.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "90b3f39c64f764c0ce513af93c1f3e57d2e5640b85f2ad78870ad30f58c087e2"
+"sha256": "3def736653746db71c2e9500eccdc381a05e3b150a512b3b059d459b517feef5"
 },
 "model.language_model.layers.21.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "02729c850931783525bfe9a9bec320fc28c757c699f41dc72c8cf8eb0e20ab0c"
+"sha256": "dc214124454e7f3f510b09edaf505ba4ab2660b883a9a809feda0b18e40e072b"
 },
 "model.language_model.layers.21.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "d315e0974f691733228dd38260b6f825185a561e52c96a54a8013de89fa3049d"
+"sha256": "23617af8d70c8851d5bf53e4ea75d8d636c82e47f67eec3faae45d633dfb5b24"
 },
 "model.language_model.layers.21.linear_attn.norm.weight": {
 "source": "text",
@@ -789,27 +789,27 @@
 },
 "model.language_model.layers.21.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "09efb6d3282d196b0a7950787ea32a34acc5598cbfaf7da062a4cbd50d6f3148"
+"sha256": "3189bef92f50ddf440ba56536cf097c2859dfc8da4ad4a07cd58e786d08ac954"
 },
 "model.language_model.layers.21.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "a13cc07463faa00e5a0e177b720650b1cfe5e9a786fd15772b71dce6839fd082"
+"sha256": "038e7d4692d4a1e365da6bd35ec2257e4900104a398501462667763385425619"
 },
 "model.language_model.layers.21.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "5cbe09883ab5d9261aac8f4940eb55aec9a138fc5d42a41811edc7c4695da090"
+"sha256": "fbba3d8ceaeb026ed2994364ad2b9405256bcb344e992d45b49d81fc41dc92bd"
 },
 "model.language_model.layers.21.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "b501841547d744b50f2aa51e2cd6a274c9933a4e8b9a3f2bd24a25ed30c7248a"
+"sha256": "aefec615e7cc687fe039ee8cdd42017db47fb46ad7fca37cefa4ad2fe172c783"
 },
 "model.language_model.layers.21.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "fcdd01744aa9137bdef8f43ab054944d3025ed5924b9f97aa8a33a75b703fe8a"
+"sha256": "7b5c7adc2e83d3b097144ba2ff82ea95fc2b9d3cf05fbf8c9c574794c8af664c"
 },
 "model.language_model.layers.22.input_layernorm.weight": {
 "source": "text",
-"sha256": "87db398ecf4753281b82e4cd67af399308795bf77a16f95e661b31552d3d9730"
+"sha256": "5080354778641038a5986661923b038fd70594dc57e179c14ce15d8bff0374fa"
 },
 "model.language_model.layers.22.linear_attn.A_log": {
 "source": "text",
@@ -817,7 +817,7 @@
 },
 "model.language_model.layers.22.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "c73cd9531f1c96b359f0131774baeedd260f28f84e4d9744bc9e933a70b5784e"
+"sha256": "180d37f0c47c4169841ab873c890bcea989bf867c146fd73fa53766eca2bfba4"
 },
 "model.language_model.layers.22.linear_attn.dt_bias": {
 "source": "text",
@@ -825,19 +825,19 @@
 },
 "model.language_model.layers.22.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "a0661a460f1df0fe1eb7a235d6e34f50bd473a0a2abae6fd991fc8351bd67654"
+"sha256": "d69f57395b6aa5af82ab7b14896b295c2aa5fe570f7e61354c3a3bbc10368b95"
 },
 "model.language_model.layers.22.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "db809befc50a24d66626599de89197ec6c37eb94e3ee40b2a733ed22ae858bae"
+"sha256": "ee27734f7d6279679c06c90fbf2bd2d71a857fc0fc7f63bb6a4317e28a4ec415"
 },
 "model.language_model.layers.22.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "45b08671fc5538d7410ba6714d4fdd7a6e9c17756737de4ce1a286d1ca789706"
+"sha256": "e6265387d4b25ba10164e2bd4439d4a7bd0fa686928913b4ba6a4fcd027df0b6"
 },
 "model.language_model.layers.22.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "e01f90c13c9d2fcfa6f3cb2494df93742d8f33281e24ec8268f6356ac6a7994a"
+"sha256": "a3c7e90e7b8a029c68f56badb69f28e3568ca8a7092303cd6b80fa86da6c668a"
 },
 "model.language_model.layers.22.linear_attn.norm.weight": {
 "source": "text",
@@ -845,19 +845,19 @@
 },
 "model.language_model.layers.22.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "2faadaf4ff889c4bd4ba41e70c48895db2ca6294d3eb5542690cac59f1c22cd8"
+"sha256": "aeb21e5a5a6168a0932e97337d6e0b69b3185363c28088c80b43bae7cb38cd53"
 },
 "model.language_model.layers.22.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "cf52336ebc260117f1a0ba38abad197da488c1026f7c0aeb1f928895f04a117b"
+"sha256": "2aa8cf6ac61e86412bfab8950bc229bfe55e535863fc567ee99b6f57e8047c24"
 },
 "model.language_model.layers.22.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "a720edb52ba8f55f0d7d3adc18b7acce3af559be1716bda9d2da00e48cbec134"
+"sha256": "90318292c50481104b1cbe13392d4a6435af714494550f13fa158c350f614977"
 },
 "model.language_model.layers.22.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "822256b5835499029d4fe4bd22805c1212c22566c6bfa9cb931922dd9dd92c81"
+"sha256": "24082616c42b1869f78988472576ef6f4cb80c90f935de0477d5289f40603b88"
 },
 "model.language_model.layers.22.post_attention_layernorm.weight": {
 "source": "text",
@@ -865,23 +865,23 @@
 },
 "model.language_model.layers.23.input_layernorm.weight": {
 "source": "text",
-"sha256": "cda17cc223c7a6254fa6ce52ada48bd0a888dd6dad726e151b100c60070493d2"
+"sha256": "7c359493935dc89caa5edf0f8ee1574c017651de9be1a630d2e5bc6cc6c04c98"
 },
 "model.language_model.layers.23.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "20b0dcef37780ade69b2ffbd8b9f10ca9ce21e7f6cccec76f70e15e82ed31b44"
+"sha256": "106a6a572235e9f97fceb09ec459b8ea015b9d20af2d25ddbd261ea9ec39efba"
 },
 "model.language_model.layers.23.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "c30efca9545d9b9aa734c4d98069551e65c4fc08802bd89a91d3799068aa5d68"
+"sha256": "0fa07598e29c3c42e923b9b004bedf1c348c50cc9bf9367e46483e94046b363f"
 },
 "model.language_model.layers.23.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "e5d70252767f148e9140d0dfa26353cfc534845857c602bd90d1e5376e912867"
+"sha256": "95b37dc474ef0431238ddd0a783d2004a255439f8d239d4ad7bf6aabc99ad08e"
 },
 "model.language_model.layers.23.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "ed43871eba0a630cf32002fbe468277c42c383e0eddaae920a794d624b2d8a52"
+"sha256": "c8532a2413dbe1d7e8a2b533f5c1989e0d1c70029cba595335c5d325f482457b"
 },
 "model.language_model.layers.23.self_attn.k_norm.weight": {
 "source": "text",
@@ -889,11 +889,11 @@
 },
 "model.language_model.layers.23.self_attn.k_proj.weight": {
 "source": "text",
-"sha256": "2c1bfa85a3cc2001f53acb5bb380f0fec77c162554bbdbeb1a780a9e44ff44ab"
+"sha256": "3292d4308ce4bcc09b1cd19e8b0f12f3ba7662722ccb735b3ab96579fc7b694e"
 },
 "model.language_model.layers.23.self_attn.o_proj.weight": {
 "source": "text",
-"sha256": "0d7b4e0646cfc28c7bcc08849f227a11a67a265428edaafdfd3131e815a75807"
+"sha256": "c91ea59244d73fd8430f460ad64093272712dc278854d843c267308c7e66c277"
 },
 "model.language_model.layers.23.self_attn.q_norm.weight": {
 "source": "text",
@@ -901,15 +901,15 @@
 },
 "model.language_model.layers.23.self_attn.q_proj.weight": {
 "source": "text",
-"sha256": "1ea19f245f55242ace642555a68f7bf504c3c99a7667288a2adb914d51e5294b"
+"sha256": "4f497c755da96dfb96671ae0d11e81f1b340c8c0906204972ea152eb8a86b850"
 },
 "model.language_model.layers.23.self_attn.v_proj.weight": {
 "source": "text",
-"sha256": "084a67b05df29620b94d4614890da1caf8873b6c61eecdae6155b4d87d469e23"
+"sha256": "c36505bc2ba385aa46f62cb3aa6e2b2e8ed052c4525b51fab3700638dc35288c"
 },
 "model.language_model.layers.24.input_layernorm.weight": {
 "source": "text",
-"sha256": "482b1942360aed51ce068ac41c3d882c5867ba05bc59453de9c66d28cdf369b1"
+"sha256": "90558671d6f18df5f835e0015058ad30bea99d66f4e7dcbaa7913a6f9ac993cf"
 },
 "model.language_model.layers.24.linear_attn.A_log": {
 "source": "text",
@@ -917,7 +917,7 @@
 },
 "model.language_model.layers.24.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "3087a5327fc32559827165ed91ec13945ae834bd469638188172ed68ac69c297"
+"sha256": "333ac5d07922b3a01e542d04ec8b8a25e6cb9b45320423505d822547fe647447"
 },
 "model.language_model.layers.24.linear_attn.dt_bias": {
 "source": "text",
@@ -925,19 +925,19 @@
 },
 "model.language_model.layers.24.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "0cccf5220ade5a470401014e7ce3438177afda86c029161bf2b4b2ccc9b181d9"
+"sha256": "87af82f319358be416b629495c7b08a6a5c0207a65f1a1f120183991e0c3f472"
 },
 "model.language_model.layers.24.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "a48625e79469e8afe731a0f3355f7568335c83c8a6459467e34eb9343a892e25"
+"sha256": "8d64245fe487df9ece021fbc113a5e843a7bf79cf363f3b9cd30fed839b6a3a0"
 },
 "model.language_model.layers.24.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "13693f4cb2b25dc5b013152b2e956dd5ad6f2000180478bc5d07aa490277b7ec"
+"sha256": "21135b71591bc165eb20051e0e5581c7441fcd160168e9e2eb6d285884f7a206"
 },
 "model.language_model.layers.24.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "4b9d5b67a4335d83551a57a1fba22f763a7be6604cc0f80d683d1f0d6897f050"
+"sha256": "b65e9f081fdb6108fbc8d8547e4d6055cfd3dfc4bf1b2185e342e9a44ddd4887"
 },
 "model.language_model.layers.24.linear_attn.norm.weight": {
 "source": "text",
@@ -945,23 +945,23 @@
 },
 "model.language_model.layers.24.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "99e9ff67c92e62545118f950563fd48805d731344fc173fe9ed7422f2f8ebc8a"
+"sha256": "5df8e876e2eab157bce91ab573c671f9c4b6c047884e11479a2f7a451e830645"
 },
 "model.language_model.layers.24.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "fabe2cf5d51952a551a4807a2ea5e8b0aa91857f6de149299888a61d0139cd37"
+"sha256": "233dba80b853e622347f15b55b3897e6521491da21c29ba0586981c4f394eb40"
 },
 "model.language_model.layers.24.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "cc693665d118b6939db131337a2ed5504f29d1604fe62b215823eac15208c602"
+"sha256": "76a982aa0be4419d721e65334b944b31e29169c76651d13296fd48c825d94e9e"
 },
 "model.language_model.layers.24.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "db08ee6f4d8110b91c6b705092c3f2f297a245d804472f46065204788e02a162"
+"sha256": "882224c97d9b3f645707127a81ad9aad1f4ebea281fb1abb8bf2a906aed83ec8"
 },
 "model.language_model.layers.24.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "1c39fe09826c1c3e11b18997d5fdfda10b045f33a23b362b6cbbbaab5c7befdd"
+"sha256": "0f2d3502c250aa6d327a390c200d98ca7130d9ab3acaf3853c2a69870d385792"
 },
 "model.language_model.layers.25.input_layernorm.weight": {
 "source": "text",
@@ -973,7 +973,7 @@
 },
 "model.language_model.layers.25.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "dc81031ead5679a98902404fc0795bb937ffbacddf04d86dbd601fde972a1b5c"
+"sha256": "1573da0422e4be735637ac30469dcb9eee3d1296ca1f2c75734aabe8c45012dc"
 },
 "model.language_model.layers.25.linear_attn.dt_bias": {
 "source": "text",
@@ -981,19 +981,19 @@
 },
 "model.language_model.layers.25.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "03b532463148fe161cd9e784978421f3eab5c0fad121c5a22b0d3a5d77f12db3"
+"sha256": "76895f64ce66efa2a8d12d6d75ca8d33ca17fb5daf4e7edc8d1926c015363013"
 },
 "model.language_model.layers.25.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "623a4bde70c79c4f5ef6c37978c66741619772ae2d64e18476e5b047fc4aaa5a"
+"sha256": "af544e5f5951b3a55df4487087fde1433e717a075bfac29c6b4bec90481d45a5"
 },
 "model.language_model.layers.25.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "e07a8d1adf0429dec0df553ccadc780d2267550562dbdee51c9ac185a03d49c1"
+"sha256": "f6d4ea58c66a2f2e1412e16df714a84e30cc1eb1fd55f105bd7417a687ee6d16"
 },
 "model.language_model.layers.25.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "83f63774bdc3ad862a507a71b99183d1598f39a347188f2e61921d7e32ba6b5d"
+"sha256": "784696fb1327f153a5e858fed11a7971350c02fab5935fca8c02d0c92a59b72f"
 },
 "model.language_model.layers.25.linear_attn.norm.weight": {
 "source": "text",
@@ -1001,19 +1001,19 @@
 },
 "model.language_model.layers.25.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "8160470d81e802774a9f1b810d60811e9441e0881564be7cbbaa2d2f8f907ef3"
+"sha256": "5431c9c169f7b930d04ea3f28cb92a90a5d3e7912b4aebf751ccf8a0652927af"
 },
 "model.language_model.layers.25.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "d29c12d758d39340920a1ca557807b4a511035a137858db7a0ef793d8479f686"
+"sha256": "c0199bb0175591ecdf006c39b40309d551fb28707c4ba1cc8d15828c3601d161"
 },
 "model.language_model.layers.25.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "31f427184431ea838a87de04a47017474fa74e445fa0f9b613de789b03d95bbe"
+"sha256": "cc3a2737e5c47d06710d1c9bfa69af48f76606ba712b96dadecb9b599af7ce59"
 },
 "model.language_model.layers.25.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "c3162214f845844c5434a2041cadb37253e56ce733370e9c341fbe5405140d70"
+"sha256": "bdd4b52e6a80c96f9c670f2ae54d273003686fd8950a2933f2b94e5477ac4f98"
 },
 "model.language_model.layers.25.post_attention_layernorm.weight": {
 "source": "text",
@@ -1029,7 +1029,7 @@
 },
 "model.language_model.layers.26.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "9d55fe82208a1b604d4569e8538b7f7d278d19947e4909a958ae3a4c92fe2b73"
+"sha256": "1d5fdf84fc410794c071e69a17c31553c821f0fe9f87a720aef4716df47107ec"
 },
 "model.language_model.layers.26.linear_attn.dt_bias": {
 "source": "text",
@@ -1037,19 +1037,19 @@
 },
 "model.language_model.layers.26.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "7a59da467cb2fb1ab78a0231112ef2c91cc86dbce67e6ed005702b5580276f9d"
+"sha256": "85b37931972e6c13da2dc740fd7f3cbad5cc41fd7043404a675ca68a4cc7a436"
 },
 "model.language_model.layers.26.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "3a1d29a8402dc5c62cffc2949e99c38a8388239621ee14813dfc37e85f50bda7"
+"sha256": "aeee1843622fa095cd0a200dd5632f23201de61f8cd11bc9f4fb40fa1524ec80"
 },
 "model.language_model.layers.26.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "f8015697aa84002cfc68d5c870b14d4f430a0a97b6c9f41cd8061578199cf4a3"
+"sha256": "3a9a002ed363be7c5b4ff4c43b252d11e050eea2811d5405142fe3e894c808c1"
 },
 "model.language_model.layers.26.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "51c58aea0ce69509b2de8049516a3b56f6025e5b65aa4631541774129f1d537f"
+"sha256": "cd95afb62210131ecbad2cbe21fd3c8cde2c97e5488db06d3c52467a9e3b7e1d"
 },
 "model.language_model.layers.26.linear_attn.norm.weight": {
 "source": "text",
@@ -1057,19 +1057,19 @@
 },
 "model.language_model.layers.26.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "e3af743b5f2230a3565627e93cc8bd19c5ab3db22d11e711537340d397f9679a"
+"sha256": "c1a7986204b0defd643131fa021615ce6242c37440c9554d6dbb72b76b418a71"
 },
 "model.language_model.layers.26.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "22936dfcbb0375fd6ed15b0a92faef465ba52562534253ed9fd4e458994eca9e"
+"sha256": "d1809a1c9256b874f0ba0c1a93cdd25513083ad811298bbd0568067c25075474"
 },
 "model.language_model.layers.26.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "89128b1ac8d776fee7e147d433b51aaea10befe99bdf0b16b07b4c3c4cde6536"
+"sha256": "ceb0b4fbf5bae9d40b701cfb27afc33eb79db7cb7785c09bff453a3ba68cca7b"
 },
 "model.language_model.layers.26.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "e2c70bb4ef3ccb0bed9a94a4ff80e4f28558ea017ac654bf6eb6ff3a59108bde"
+"sha256": "821a5bb8b264f9e9a3e8c2265bd7c4216d5159e05cfdcf0fb3cf156de8bb809c"
 },
 "model.language_model.layers.26.post_attention_layernorm.weight": {
 "source": "text",
@@ -1077,19 +1077,19 @@
 },
 "model.language_model.layers.27.input_layernorm.weight": {
 "source": "text",
-"sha256": "ab05f495b3ef154d0ebe4a0b125840176ef5e06dc43e520f37aaf7f9615bfed9"
+"sha256": "8446f76326799c9e462629795b189245d4b7c23cf3a42a7d64ab30a6db6a146c"
 },
 "model.language_model.layers.27.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "f4747326d2da4007fea0d2943eadfd63beb6784768717e5ae73a6b5e696a8d37"
+"sha256": "2f6af88c3b195309df3e9e2e1b39e4c2a7fd2a3716d266fc65c7c8ae0628944b"
 },
 "model.language_model.layers.27.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "6cbf3102f6e8bee20f544c5fe96fde1b086fc634784b9f5d9c7fc904fe85be64"
+"sha256": "983a9c701078dfb1ac11d7a4c47cecca140987c52ee97e3f5d31202fad90d19e"
 },
 "model.language_model.layers.27.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "2b553b2bc952ca557844438a4479490df6bfef452a48305995bec4f85b9d747d"
+"sha256": "1bed40774d127b4f11883b611dd313575661025f2a8d5f6030ec43c0bbff0831"
 },
 "model.language_model.layers.27.post_attention_layernorm.weight": {
 "source": "text",
@@ -1101,11 +1101,11 @@
 },
 "model.language_model.layers.27.self_attn.k_proj.weight": {
 "source": "text",
-"sha256": "cfc7c55bd04c795dd36b3968f019de29dcdd0beb0c324b0e2e171425bb31a9ba"
+"sha256": "4556495e4fd73978f52099fdb557b1371cab05b13dd7b5dc2e2ee58ad446089f"
 },
 "model.language_model.layers.27.self_attn.o_proj.weight": {
 "source": "text",
-"sha256": "a3d7fc02d48379bdb44f7a0d644a4a7cfd4e67d610508e30fba8a67cce223c92"
+"sha256": "d23f5d9ae8d20f3ec2a320fcbfde377c44596e496c3a7c8aa53254b33aeed990"
 },
 "model.language_model.layers.27.self_attn.q_norm.weight": {
 "source": "text",
@@ -1113,11 +1113,11 @@
 },
 "model.language_model.layers.27.self_attn.q_proj.weight": {
 "source": "text",
-"sha256": "8e39bc6b60d93734140ef0d0b9dec261d8af2593eb8723ab22bf58efea7c4b2f"
+"sha256": "2ce6c0044dc77d128b9b8ba8d8522f33f32b57d0b6cea3cc43b27a54960fd23a"
 },
 "model.language_model.layers.27.self_attn.v_proj.weight": {
 "source": "text",
-"sha256": "322db4920a06eebbcacc84e4df0d595d784cef4b490c5e862387ea5b200de1e3"
+"sha256": "4cb0d50a950990bfdf2c87ef452bbdd7f38bfd4dbad13eb09e285b9bda20bd3d"
 },
 "model.language_model.layers.28.input_layernorm.weight": {
 "source": "text",
@@ -1129,7 +1129,7 @@
 },
 "model.language_model.layers.28.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "93438613b34e554ca66a45cca519a27d91b5c1976bc7dbb50bcabe0c03655d6e"
+"sha256": "1e93738aadb42b6c4b2c4eb82c151a47a89b99ab596b4a90ae9673dc422519b9"
 },
 "model.language_model.layers.28.linear_attn.dt_bias": {
 "source": "text",
@@ -1137,19 +1137,19 @@
 },
 "model.language_model.layers.28.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "898c1fe3e6706b78002e40626089721e9510c5e62365f5a02c598aa9ca244164"
+"sha256": "aa837021242db8815a4ba8fb45c54fdc93237f2f3efa654fd34d073bf73b48ef"
 },
 "model.language_model.layers.28.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "9c8b2b6c6141cc600739c704bcc8a5771cebf21207dc7b6e588566a1b6c64312"
+"sha256": "8f32ee1d8f03773ba171da1dca6f68b494cf7296d41fab3a1b04908a724b1221"
 },
 "model.language_model.layers.28.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "681859aa5142d2205f7b2bb9987e9fc368e1ee76027dbbd341a43ef76c7a0792"
+"sha256": "ec74c5f10865a87121509003527588f7ee9b48da8c921b58b5a6d1e1ac11eb1b"
 },
 "model.language_model.layers.28.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "261ecbf9a1086ec3df4a6c783a6e1d42851066ae2dfb1905aa23f8a4ae0fc962"
+"sha256": "ab9adf2195a3392bd50cf66d9f376a6bf32eb4ccfc4d2a22d6a36f3b2c65e4bf"
 },
 "model.language_model.layers.28.linear_attn.norm.weight": {
 "source": "text",
@@ -1157,19 +1157,19 @@
 },
 "model.language_model.layers.28.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "b06aca28a8b2ad66229a84db18aa205a5f4928edf3653cb13765291f00fecce1"
+"sha256": "c3f281594c30f453234dec1682c8fe8be7c65d8c9338132c16ce5335a6bcdbe5"
 },
 "model.language_model.layers.28.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "e8e7db55a1839b73fd28671243696597a5d63bea810aa730f651cfbb571b4b9a"
+"sha256": "6c01a91140a94905d8e0225a36baada51d940f88ea4b58983a8a0c9311efdd54"
 },
 "model.language_model.layers.28.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "73dd0e46c7576b3a971f57067afb020303a64d25025b681b5e70ec4fdaa6a480"
+"sha256": "d037afb7efe41f6617719ac0e4a802d8f9631b63e58b59fd917e1701ed55582f"
 },
 "model.language_model.layers.28.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "8784aa7ff7fd732d3d8f459729a8b283b4425e329a316382f56c77936733d0b6"
+"sha256": "c5d2e21a8b6fae151b48bb37891326ddbc2573cc3c7dce87940c92f0f83d07df"
 },
 "model.language_model.layers.28.post_attention_layernorm.weight": {
 "source": "text",
@@ -1185,7 +1185,7 @@
 },
 "model.language_model.layers.29.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "36517508552b49faf695726e8bbfdbe9980433a7cc5cc06872c2e69b67e0c9c2"
+"sha256": "77fc6e0863d13c3eab2ffd359e8456e571c9065bd7cf1ef4d240e7f1af7bdffb"
 },
 "model.language_model.layers.29.linear_attn.dt_bias": {
 "source": "text",
@@ -1193,19 +1193,19 @@
 },
 "model.language_model.layers.29.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "e3959d3f569a1d7fa03174db7b144c5eb16c9aa48e73ba3b5af914b3eacc839c"
+"sha256": "cc51f6e8f4f17787d3f416dea10bfb2e11cf8845be8f733f1f035faef584f480"
 },
 "model.language_model.layers.29.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "2495f8834330b309c6976a5aed0abe524a763b962236a3d36fe8dbece3113e1e"
+"sha256": "2ba96a810da9eb40351f8359883063018016821f8d2e14bd3024219d1594004d"
 },
 "model.language_model.layers.29.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "f5c68f51b984a98144de1ff3d17b5d976dd265aad681e80f334185efd427d8c8"
+"sha256": "274f557e83256423c5e431d5664ae5b0974046e0849547d1c6171218b3ad0e15"
 },
 "model.language_model.layers.29.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "a9a7bedd129d0b1f547efaa61ba8210af862151ecd1e8e2cc72f63a86304cd9d"
+"sha256": "7edf44a9ed2492e0aa78e7bfbb49d2ac350f87572d4978bfb55bb49e8b71c135"
 },
 "model.language_model.layers.29.linear_attn.norm.weight": {
 "source": "text",
@@ -1213,19 +1213,19 @@
 },
 "model.language_model.layers.29.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "984b00bf411847ccbc0dd834d15f3616dadc9090065fb4f9089b9919a0885153"
+"sha256": "c533089b0f9ee985472148e5ee93bad85bb7b83ad6bd23a2f6bce43ec05ff7e2"
 },
 "model.language_model.layers.29.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "2ef2bcda32dab8f811f9e142f6f0e945a01a7798419171e162b8418e461f709d"
+"sha256": "7a792f124bbc41b446cf3cd42588fc899cc1943ddd8629c2e9a2e0475408f8ac"
 },
 "model.language_model.layers.29.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "274f129f780c3624ecef6f16d73763a2680c5659c38f4e3bde8bf910d687555b"
+"sha256": "3d002f57322df87dd027c9697fed72fa40bf7ab11df5a6ea36dd55e4b7b7d8c8"
 },
 "model.language_model.layers.29.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "1bb9e54681388dea0da9354aa3062d8bf45b82b6637ad9219b5ffc915775bf0f"
+"sha256": "2d6263d684b2069e70bcfe63ac4847c46bbb48f86ff30d64e231653bf7332cbd"
 },
 "model.language_model.layers.29.post_attention_layernorm.weight": {
 "source": "text",
@@ -1237,19 +1237,19 @@
 },
 "model.language_model.layers.3.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "013861cd093dde34a3e66e41b8c3351ddc4edafa1d7894f56f7386522e3654a9"
+"sha256": "6f194643bffc639c2cc3bdf24b25189beed6fb60b6b320f0e39913afad110013"
 },
 "model.language_model.layers.3.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "ec49b14f96292894983d1f0d50fac189a716380b161e5876774ce9fda0ecac64"
+"sha256": "8b122f3d435ed3b0f37421c5fb76ad0de36568677d9ec3ada1c45b41ebebb2ca"
 },
 "model.language_model.layers.3.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "2fda72665f0cea631e1615506ac8b1a58186d3452fdfcdb4f0e72aa83ce9898f"
+"sha256": "5fd7c36df9142b3f02b7f8ffe0e6f9294a2a06d993cfbda7793379c129920e5b"
 },
 "model.language_model.layers.3.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "f8b688667f5028d5d6955315d0283f705e4d425e2f1d561de0cec1818f363c3d"
+"sha256": "4c6f8a24e61b58e3311397328cb2a261984107b39dbfb41d1ccddc620c3f0f6c"
 },
 "model.language_model.layers.3.self_attn.k_norm.weight": {
 "source": "text",
@@ -1257,11 +1257,11 @@
 },
 "model.language_model.layers.3.self_attn.k_proj.weight": {
 "source": "text",
-"sha256": "847b84d4328157ad23c5813c88b6344cf8ffdc4d33f0a7f293a49fb7faecde0c"
+"sha256": "fecdaaf28bb9a160dea21aa716efedec98980c6930edac482af923defb0140f7"
 },
 "model.language_model.layers.3.self_attn.o_proj.weight": {
 "source": "text",
-"sha256": "708fca638f815306b5d190ad69210d3b0ee738166d99c99708306249d80bc4e1"
+"sha256": "a480799c5157b60604e669e3f7b087545dac69e3508acfb9fac6f17064f53fe3"
 },
 "model.language_model.layers.3.self_attn.q_norm.weight": {
 "source": "text",
@@ -1269,11 +1269,11 @@
 },
 "model.language_model.layers.3.self_attn.q_proj.weight": {
 "source": "text",
-"sha256": "199e9da341ce7ae5f39ba5620f0564771220c5ccb1d2a5882f5569acac61770a"
+"sha256": "92b971de27c7c6f2e5cee2a96d9de1ebb8646a2a6f295c95c1d9d7a7d9597814"
 },
 "model.language_model.layers.3.self_attn.v_proj.weight": {
 "source": "text",
-"sha256": "6f35d5cf2e800f9714a308d08cac3d8699739a385a4d15058e7664833e299109"
+"sha256": "f5b97af81b565f2e9eed157d2b43e90f3838283d4c717ffa96d1189c2a85b5c5"
 },
 "model.language_model.layers.30.input_layernorm.weight": {
 "source": "text",
@@ -1285,7 +1285,7 @@
 },
 "model.language_model.layers.30.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "245c583ac7e39ba616d1cedddb64a0a21f9bf3f4ea2fca0cda8ed995e9e56111"
+"sha256": "ca7b83060c274aa29e8bbb9a57ee4daac909dae1b8f2bd12be33588bc48b707c"
 },
 "model.language_model.layers.30.linear_attn.dt_bias": {
 "source": "text",
@@ -1293,19 +1293,19 @@
 },
 "model.language_model.layers.30.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "1eef5d1bebbca665b9e403236d6ccbcbce11a5afadb6784b93cc7c3c24135834"
+"sha256": "f936d5e27de873d6542ee64d53fbf2fe8ab7e5caba5de78ec3f8a94faf8daa29"
 },
 "model.language_model.layers.30.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "6973aeeb0a4b6af5256597a4067485d11b6c6471c3491668b54add24c4d18af4"
+"sha256": "bf092ab056a03636b8cdc1626e89247d99e1abe5b876e64b9bff1bf11ceecd32"
 },
 "model.language_model.layers.30.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "d0bccbc394801418734e713be672da1bb9ec9db22e9a17bf090a1e3184d267a1"
+"sha256": "d9d58307dd4f6a57558830174d85e06ceaddebaa861ccd0faaf8eba89bfac2b2"
 },
 "model.language_model.layers.30.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "2e7da58a9064b920c0742ef18e08fb42a18ef0527b9eb560b06a589aa19c9f1e"
+"sha256": "9229d89351aa024953e89bbf4a4e7c1b18e96ba7242f7cf8854b6387d97785a5"
 },
 "model.language_model.layers.30.linear_attn.norm.weight": {
 "source": "text",
@@ -1313,23 +1313,23 @@
 },
 "model.language_model.layers.30.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "6ead00b21b7ba2d5bef8223fc9200eb66980a5c4eb24f25bbad537bb4d96a4ad"
+"sha256": "76032e1608d3aa223096120802740ef43217950fb090f62e18d588ceed895317"
 },
 "model.language_model.layers.30.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "cbcc67699cea2acee33da084f4787d3da5264191eb00bdefadd4bf51681c9185"
+"sha256": "1c10fa308223f93292c37a1651c07adecb104c54a0221cd929a79f20f43e3ddf"
 },
 "model.language_model.layers.30.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "807432df3cc4497d4d0831ab53bd5dcfdf8813c8d86c6c3566715241500e681e"
+"sha256": "3c2848da8db33428e33c23eabf4ab7c1dacd631148266cfb3f1eec3072abf0bc"
 },
 "model.language_model.layers.30.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "54589187c249c36e2fac43754669a98484e354614cf9fc2a6bb227c6beb38d36"
+"sha256": "fa98d08484948fa82ad3953593e625aa5c7815934e25d01c9d946743401c89b6"
 },
 "model.language_model.layers.30.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "74c98e017336ac95a3dda516ff5d05c7f3e8f9e48f6e59b369c3a34502d7777b"
+"sha256": "55e3e86571f65dffe58daf4db056644b03d5fd012c891afa1bce99c8e79b4fc3"
 },
 "model.language_model.layers.31.input_layernorm.weight": {
 "source": "text",
@@ -1337,15 +1337,15 @@
 },
 "model.language_model.layers.31.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "6ae0fb2818e0d0d123ff27f2a1ad6b05d5ee553fe2bf064e6184d45eb319569c"
+"sha256": "429fbfe586406c9debb30b7f503270bbe8d45ad23ee80ebec19258fa28dfaf71"
 },
 "model.language_model.layers.31.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "ffb9aa38e758634f61fe0e9df32080784b1852373b678e0394950207bfad1ca2"
+"sha256": "fe38e0de81b68adee8ba3ba0985f3e25943947f2a049d0e4c6aaa47342a11dc3"
 },
 "model.language_model.layers.31.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "96a9f6e9c3b17e91715e557e50abc9f15dae176a4ec8af986a65d44b0ea39271"
+"sha256": "8abdaa1e0bddc99a8bd0aedb2c7687fad22e4471a5c1a1a343053b65bec2c02b"
 },
 "model.language_model.layers.31.post_attention_layernorm.weight": {
 "source": "text",
@@ -1357,11 +1357,11 @@
 },
 "model.language_model.layers.31.self_attn.k_proj.weight": {
 "source": "text",
-"sha256": "f5a972a9076ee167910af896699c9a4868eb2c451985369d4150264c2a0991e7"
+"sha256": "3be508ae8dbfe3858b14d9e256cd2685396be94a99f8b517475b4a2e005812f5"
 },
 "model.language_model.layers.31.self_attn.o_proj.weight": {
 "source": "text",
-"sha256": "3cda933ef2d8b4d60709e01419289f8e8a66ddab193c2be774ae33f683b73790"
+"sha256": "ec4b0fb1434a1731735ab2f2bc212b2ce9b2939170cb16589846cabbc5043678"
 },
 "model.language_model.layers.31.self_attn.q_norm.weight": {
 "source": "text",
@@ -1369,15 +1369,15 @@
 },
 "model.language_model.layers.31.self_attn.q_proj.weight": {
 "source": "text",
-"sha256": "a6d56ec4db1e7efb79d58fc22d897152e2a840821f0e3f8aa06e9bb666172f7e"
+"sha256": "842e43cdd5662287fb490d8706554c1a464026998f308c097ec731100ff1a3af"
 },
 "model.language_model.layers.31.self_attn.v_proj.weight": {
 "source": "text",
-"sha256": "573b3457558c28797b8d0d555bca36c8c23ca56c8a5028f6d4bc5be968fce60f"
+"sha256": "c8db353dcaa263dc930969513d8c2730696f18c17228cfaecbde1403f6c457b2"
 },
 "model.language_model.layers.4.input_layernorm.weight": {
 "source": "text",
-"sha256": "ae8f790c718550bc2f10811a0c970a9c15336caa3f511bde3656ad73d1f7e144"
+"sha256": "56567cee7f432dc3a0c521906ced52f09fb854c863946011ceb2c322b1df21bb"
 },
 "model.language_model.layers.4.linear_attn.A_log": {
 "source": "text",
@@ -1385,7 +1385,7 @@
 },
 "model.language_model.layers.4.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "a2688589423fd8947d4527e9ad6808adf6b14244491fe1eea765ee1d1c67acd3"
+"sha256": "fdb21c97a1ec2666365ebe608714ce449fecaab8dad7fe3ea8f553bc123471e0"
 },
 "model.language_model.layers.4.linear_attn.dt_bias": {
 "source": "text",
@@ -1393,19 +1393,19 @@
 },
 "model.language_model.layers.4.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "23f71b0d999127e2d23c1ae04940e61b762a056d9e7da30670ef8b1f1f30b09d"
+"sha256": "b545249e70ad98c36cd8ccbd2518ef6fbc97d72260a4ac8a43ef118d56af27f0"
 },
 "model.language_model.layers.4.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "d472c75ba9da992bdad341485416f34f4d38127109ae441c1bc20031210170a2"
+"sha256": "7f5018e652772cb8cf4ee1cdea467c77f21c741f443b82b40762181bbf8307c1"
 },
 "model.language_model.layers.4.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "007f81dfd45ecbc3b7f2d770828dce7d9f979332a04ce664c2fd724d0a0058e1"
+"sha256": "302b6a11be4a3194a12d71dffde01dd0f18940757ce2b69258a1891a8ce59f25"
 },
 "model.language_model.layers.4.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "a7ac3bc9e80b21c11e71582fd645e97b8bbffea337acc0112772b69b4ed25d4b"
+"sha256": "f5a9ff4fa3567f7fac11159025382b05270230699f53a35247c6ad475439ca92"
 },
 "model.language_model.layers.4.linear_attn.norm.weight": {
 "source": "text",
@@ -1413,27 +1413,27 @@
 },
 "model.language_model.layers.4.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "44cbec0a7bad6ef4ba6b1643842a29a2db1bca4b0f0158d6224113bc740da05b"
+"sha256": "553b9ff4670a087b08e8f237ec2bd25ca7c63e40d6032f7dc42c19cd2fce4f34"
 },
 "model.language_model.layers.4.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "0e35450a425156f60250ac1a337fd565c060e8304c84cabafbfc4c43e7ad4b98"
+"sha256": "a1be0a4df3b24347a8f016063b70f844ea5518c121450d4958edfd54d2225c29"
 },
 "model.language_model.layers.4.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "81a8ce17b70d878724656394585685962daf7db7840d77e68d6896b227202295"
+"sha256": "39b967203a20730d5928486686c0b028a0025594a5da339cb87a5a7c68e1779e"
 },
 "model.language_model.layers.4.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "74f076807d6f0cbe23f50465cfcee21b4c13fe6e709860f0cf61e5cf7bfc1b4e"
+"sha256": "8fe65893921432011e2407fcad2a613c19d14ef9477fa13f5f425c6478c7d13b"
 },
 "model.language_model.layers.4.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "6790668e8a9ed97bf9f531c62be2986fbb4d36d5417141f76276676fc6ff27d2"
+"sha256": "5e1c68bf736419d7b73e56e3f2e816eb4adb83390d906c19b8604d0f2cc119f0"
 },
 "model.language_model.layers.5.input_layernorm.weight": {
 "source": "text",
-"sha256": "c020308e20af65108ff17002333b29992bedff3faf8846bcd29beca97e455916"
+"sha256": "2c8d7399529037a4531f617c6beb46cd072d5a65a45d07f87490a4e952e80316"
 },
 "model.language_model.layers.5.linear_attn.A_log": {
 "source": "text",
@@ -1441,7 +1441,7 @@
 },
 "model.language_model.layers.5.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "73fdddc962515851a9bab4fe6b32331e37f1d3b2881dfccc1a45c96be176fde0"
+"sha256": "e04c64326327413dd454c524df5adb8074e201f0e8fc432484ee991a76f93f89"
 },
 "model.language_model.layers.5.linear_attn.dt_bias": {
 "source": "text",
@@ -1449,19 +1449,19 @@
 },
 "model.language_model.layers.5.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "acca63b06e083e2a4197604e2c171f052db90ede0d78ec58502caf42f2a7ed9d"
+"sha256": "c37b00280d5abf5067f4dc62bc583146a827f4e5cdf2c750aab164957e56e39f"
 },
 "model.language_model.layers.5.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "574b434d24ae8cb25fd582389a34a27bd8b85f87abcb1d2439bb92604fc082af"
+"sha256": "ef5d332a48ba26f6ed9b764672e3942930426c67133378d9ab0287b5e258ce60"
 },
 "model.language_model.layers.5.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "6d40b4d5e5d25d2fa1ee5ad6a0af4e111c2da0156b0e9189f1ec89c1b0631e88"
+"sha256": "c14e09992b20de7db97e2a2ac8c93a5bc665052740c344f165ae3b94dfe71384"
 },
 "model.language_model.layers.5.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "f5f8fe2278d015ed94090e0601b0a44fe06a1684a98b99fe3969ce52676cd85f"
+"sha256": "ef0a762bd550014d8177c6cef6bf952db3b21e652db7f862c5f994f1c9700f7f"
 },
 "model.language_model.layers.5.linear_attn.norm.weight": {
 "source": "text",
@@ -1469,23 +1469,23 @@
 },
 "model.language_model.layers.5.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "f72039942056ffa1a9e26e5befc0ad1abf39e75b2802cb689397f17fd3be7d18"
+"sha256": "bb93203e51d564b4b86c23e4270a9ee7e6c411c652cb82472f641172e6918d98"
 },
 "model.language_model.layers.5.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "c4fb2aaf96e2a33149fb5e1ae42862fa2b1dfdc791cfabd53cdc190e515fbeae"
+"sha256": "58fb9df4415f4e92f186274392632ef012d06591ef1ff4d2d6f15ca06003aa8b"
 },
 "model.language_model.layers.5.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "d205ef9079a7cf50113d260353fd49b985e4d50d4d8204ddb8d0d8aee783398e"
+"sha256": "77410f03ab297a2fe32c038630f1a50bc008978cc1d8937b0ed78a17d97ae8f9"
 },
 "model.language_model.layers.5.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "c080f8e23817ca249fa24e8bb313b4d33d39082d97b1318e16e598d696afb2f1"
+"sha256": "2dd4c85f91c9198322e6004f3e55849bfd1709898b8a4d5c3c218059b9234474"
 },
 "model.language_model.layers.5.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "feefb655f531dff7c8fab0df48e9d219967f40da4fcacbaba024a3a10f2af7a6"
+"sha256": "b06428f0ab0e7e3f189d9f0631c50f1b083dd4bec39ce72fdb7b2094245f8464"
 },
 "model.language_model.layers.6.input_layernorm.weight": {
 "source": "text",
@@ -1497,7 +1497,7 @@
 },
 "model.language_model.layers.6.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "409a29905946b18aa339a854330510981264c881c9eebe10d9342e13c59b39c1"
+"sha256": "d3449ca7269a4cf1bde8394d2019278365dd265822ad213b059d40aaab51202e"
 },
 "model.language_model.layers.6.linear_attn.dt_bias": {
 "source": "text",
@@ -1505,19 +1505,19 @@
 },
 "model.language_model.layers.6.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "226edc1c01258ab51593a58609a4a099c86892d5d4f0c9ce0dca4eb2a83d5c50"
+"sha256": "5bf18fb37b9110f728cf4eb5489556fa71dfdf6141138055dc50c9e9788225d5"
 },
 "model.language_model.layers.6.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "856cf9464440f4f82c214d4ef5f36a9c4300cfa226610fb999bcecb316fa21e5"
+"sha256": "351208fd98892679873fcc1fd05369df908c846b4bc98d790c6731c9fc8bf81b"
 },
 "model.language_model.layers.6.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "c10aefe6ccf095e348ef40add13da2b6dd3242d61606d2016334c70dc40f299d"
+"sha256": "6899210c7b97360edfd86c441cddd48ac3ca3c86012a467e4467d75acbe3d1b2"
 },
 "model.language_model.layers.6.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "4901de25e46aee3e1ee40152e63d8942571ad001067f4d3268a8ed61e6d127cc"
+"sha256": "69f02533565b0741d1a69fd0294cf2654345ea226152ba646fa02d8aa50540c5"
 },
 "model.language_model.layers.6.linear_attn.norm.weight": {
 "source": "text",
@@ -1525,43 +1525,43 @@
 },
 "model.language_model.layers.6.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "dcda8f69dc418e5a4796520c3d96e22ef5715bdca0198ffb0ac64d9dba1200aa"
+"sha256": "89e2077fa5c05d47858ee8b095c607f74be07976800850025ffd0dff86d0f790"
 },
 "model.language_model.layers.6.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "cea7cd7dc47d28377cb0d4bf2b73203d68169e56382f8461f5aeb7264aa33427"
+"sha256": "37014653ee8427c8c0db33b4b2bce709204cd9e16207b8bf472d2940b83247e5"
 },
 "model.language_model.layers.6.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "9da8f746467281096e44dce10c69275a38205882193ae6f52897b2e6ed982f09"
+"sha256": "7a4ee018d28f57693cce53be250f45dd22996f08f5cbba373102f9f8089fda78"
 },
 "model.language_model.layers.6.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "9e699b942c26a9f4280eac5764229a4c9ebd74203c7eddf9edc270b64e328cf5"
+"sha256": "47e6e5cfbcdc40c65b60789697b398b1c7e223c0844f7179a8d8069f22f8ce1d"
 },
 "model.language_model.layers.6.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "75e70bf75988cadbc7d2baf23c21c2b9ee5f4d67f9d05595b3e23d3460633bb5"
+"sha256": "ac69e68bf0c5aca2ae23211c969b136228f3907c3dfed36856f75a0dcc236dce"
 },
 "model.language_model.layers.7.input_layernorm.weight": {
 "source": "text",
-"sha256": "1136119586d864937e624d66866176b283de73966a0d537efa527565850a1dbb"
+"sha256": "7cd7c8b5d817aa8e2831cdac2a36e2145bf51b24943b1dcd87787b9a0ab6bd92"
 },
 "model.language_model.layers.7.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "24e56f217d8657be4feb24e2b6c59b3d16c3da60664027389ef9ac9e3637278e"
+"sha256": "58971de1706e5d377c8dbc020cf26977d777c2d789d457c7466ac234f9c54f8f"
 },
 "model.language_model.layers.7.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "0153695ed35209fd12cf43c0266041061cd0200dfba176715925ceb35e2f36b0"
+"sha256": "8e8798ce363c6c6658900a35e027f72c5ba1cfcdc1d5b76eee50a8234d435359"
 },
 "model.language_model.layers.7.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "793db86ef0b159629d53aae28667d41f72993f736c62436fecd8068b34ab1ef1"
+"sha256": "368b52ba8a417f1d9f7b1d4b3671e7c3f6592b6bf9ebc5da7b7b7cf1bc90b87a"
 },
 "model.language_model.layers.7.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "a9836f15418ae378cff8b4f0e15b3369bc91d6c78d71fd27c820d056272a6bfd"
+"sha256": "ffb68f4c5722002cca02b41ca658b47a485711a8b5414c3e50446ed31e52d52a"
 },
 "model.language_model.layers.7.self_attn.k_norm.weight": {
 "source": "text",
@@ -1569,11 +1569,11 @@
 },
 "model.language_model.layers.7.self_attn.k_proj.weight": {
 "source": "text",
-"sha256": "e450b30377d3e141660a6e5eec6d0c05320de04cee36f5a4f4d1c7a1d8c5286f"
+"sha256": "c4c6bab029b5fc5231ebf1d0a718d3387714cb7cd752528151109857dfa3db15"
 },
 "model.language_model.layers.7.self_attn.o_proj.weight": {
 "source": "text",
-"sha256": "1201181e6a3d0d776ff68a35a24560971315687e960f5789c1c10a24d857b778"
+"sha256": "c2421d8b029d0ee451e1d63517da691b6512d65a29b4196910de6b451d71c8db"
 },
 "model.language_model.layers.7.self_attn.q_norm.weight": {
 "source": "text",
@@ -1581,15 +1581,15 @@
 },
 "model.language_model.layers.7.self_attn.q_proj.weight": {
 "source": "text",
-"sha256": "ca32717c980a9a172499f61e14a1e7c0b2f777b35b098baa09cdb3768040740a"
+"sha256": "f8226bafe4479699146bd8cc46fad29e147364e65ea6e4bce75f34e0ddac53dd"
 },
 "model.language_model.layers.7.self_attn.v_proj.weight": {
 "source": "text",
-"sha256": "ecaba8b8da40fcd1473aac78780c8628240e436feeacea592b0163135ec45166"
+"sha256": "1aca0c06024186307c211ba78cf1dacd79f71e7647e6b0f914177d53e7a6f1b2"
 },
 "model.language_model.layers.8.input_layernorm.weight": {
 "source": "text",
-"sha256": "9e29a931396d972ba4b45a261061c5e65ea3633a425e03fcfb6e43c0b0fa61c5"
+"sha256": "ec8153248b3c7f13da978939b2536240bb8d59520d593b565cd854ace6f1ff69"
 },
 "model.language_model.layers.8.linear_attn.A_log": {
 "source": "text",
@@ -1597,7 +1597,7 @@
 },
 "model.language_model.layers.8.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "418f01ca302bb64a4aa2d739bc265a1ebc2d633b0523e186cd4b17fec2fdf557"
+"sha256": "bd7f703e8f5d03a2c8320282913be96ce20bef4454122731fdd172fcc0dfb4d2"
 },
 "model.language_model.layers.8.linear_attn.dt_bias": {
 "source": "text",
@@ -1605,19 +1605,19 @@
 },
 "model.language_model.layers.8.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "fa93da4331c0a7ead087dc89346ceaeaa978a361f9c9d95a33612fd1d2ffe535"
+"sha256": "8869958026a2c822e219c3611ca96fdf477987d4d215a8cb8b79e842de2904c7"
 },
 "model.language_model.layers.8.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "4ee7cc0bf912189b80a5ce3eca687393736c2dbda7b5b124dc02ade424e7ab2f"
+"sha256": "404990d16c698cb2c9d0498e181ac90387132acfa340a3b49adfbbefc7458de3"
 },
 "model.language_model.layers.8.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "3853984ea8b5c28382545df07e215c1f2a939372afe355c27714ac7dc43719d2"
+"sha256": "631cb857f34bbfa6af86ff2e99986c342a5d652a57b97efbf68436abb2b711e0"
 },
 "model.language_model.layers.8.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "c01ea040dee77f3b0da7e36817d610073cd139eee3a485ded326c8f0b7b7e748"
+"sha256": "c40d7ef2af3f53e5d261dc64057fb841e1d993ccec45c5c15da0217167588ebc"
 },
 "model.language_model.layers.8.linear_attn.norm.weight": {
 "source": "text",
@@ -1625,27 +1625,27 @@
 },
 "model.language_model.layers.8.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "2795ec2c277d1f1629de8d56c4fed22e1d75cea31846d7ad46e80b9fa6d35174"
+"sha256": "652faaad61680ea584986c098bccadace37d092d530d50c2677410b6c1bea3be"
 },
 "model.language_model.layers.8.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "728202e1f3329a7891bcfc200ea5163c1bc5838cd1ffe4af9c46442ad858fdc3"
+"sha256": "31eed6e0007cab91df33854ca91477074a604154269d00497f39c75165e355af"
 },
 "model.language_model.layers.8.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "81f88444c5ec476e70c7e909002bcd9376719765c0c584e5f2f73633ad17a111"
+"sha256": "ee82e69a6632915040c7e6b63dd1db45be9eb16f56253347cc3009cf5a837dae"
 },
 "model.language_model.layers.8.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "62ab13624da44e93e11d0d1b060e56eeada74165aee410cce5facfd1d4c75a90"
+"sha256": "1cbe18f32ba3e1a645b71d287c01f2d2dc52cd95986572c7d93f343a331c9a3c"
 },
 "model.language_model.layers.8.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "58f37f4ff891bd42962a8614946c98d531a8b7158d2cc6726e86ae0e6976fec5"
+"sha256": "e3459692af9e9476469780a57633c4a03ae43afb7638dd9064814149864bfb80"
 },
 "model.language_model.layers.9.input_layernorm.weight": {
 "source": "text",
-"sha256": "b9a9782d65ba826e066b522635a37df5df21a543fd82c972b8f174699c36c183"
+"sha256": "25ef5139de69c3abc677886ca39209ebc598da1a0e2c65f02e589cb47e3c9a2a"
 },
 "model.language_model.layers.9.linear_attn.A_log": {
 "source": "text",
@@ -1653,7 +1653,7 @@
 },
 "model.language_model.layers.9.linear_attn.conv1d.weight": {
 "source": "text",
-"sha256": "0c3b5c05b56b7b1405210ac762bd03a4701cdf6bf5c0900717d49049261be1dd"
+"sha256": "27af3282d010201e4b7495e6ffb0088108c2dddadd775b17c60c2681b0232246"
 },
 "model.language_model.layers.9.linear_attn.dt_bias": {
 "source": "text",
@@ -1661,19 +1661,19 @@
 },
 "model.language_model.layers.9.linear_attn.in_proj_a.weight": {
 "source": "text",
-"sha256": "ec37377e5669288c0564aeb59491ef2fe523ca176ff5615f941e6448da4245ae"
+"sha256": "e6a4597eee5e9208d564b7362accf8f33ef724c829d47cb9f497bea4177633fc"
 },
 "model.language_model.layers.9.linear_attn.in_proj_b.weight": {
 "source": "text",
-"sha256": "ab627f05f28e0d6797d6b52fee02a2f44153faa75e9f38738cfef31b93be659f"
+"sha256": "819462c1865b3f4d9c93a3f26886b3601d508c64c9a57ffa59b9e939cbfe6edc"
 },
 "model.language_model.layers.9.linear_attn.in_proj_qkv.weight": {
 "source": "text",
-"sha256": "58ee3b7bc9a5c5f6efdefd5a130122d5223d610663eea338bb795155562a5050"
+"sha256": "dd21231d1e3ed6384615e2c55695395d497257b07b72ca14396fc9546193221c"
 },
 "model.language_model.layers.9.linear_attn.in_proj_z.weight": {
 "source": "text",
-"sha256": "cbaf2959100c2d10f4ede0037fd3bba8bba66433d0d4fc0cad30c0013342bdef"
+"sha256": "865d23078b6c59cdbb1c8f28f4bbf89627a36b0dc81e9ed0814820af05000b01"
 },
 "model.language_model.layers.9.linear_attn.norm.weight": {
 "source": "text",
@@ -1681,23 +1681,23 @@
 },
 "model.language_model.layers.9.linear_attn.out_proj.weight": {
 "source": "text",
-"sha256": "9a018a3ecd49b4e967f3a415502322b80a09135f1b9b57885611864e606ceeb8"
+"sha256": "76347554df25752a22c006b598cbea5d0919661279836b27c37f676bb3ee4ec3"
 },
 "model.language_model.layers.9.mlp.down_proj.weight": {
 "source": "text",
-"sha256": "9a1b50564e248ccc92cea76dacdbbc7f4ab701ea0d436312379653945252254a"
+"sha256": "fd02f421dd50efb3d2463ce6f5873edbc60c765823cb9d4eb7c86ed55116ae66"
 },
 "model.language_model.layers.9.mlp.gate_proj.weight": {
 "source": "text",
-"sha256": "1b56bc8310c68d495898b4186233dfa399f24516bec0ac6dde0a7815154c7e13"
+"sha256": "f6d72175b5948ed175047ef5a2d4e67d29dfef33f6c46c30006655819c4ba314"
 },
 "model.language_model.layers.9.mlp.up_proj.weight": {
 "source": "text",
-"sha256": "7fdd49f96facc99bca17c09f498641d95d530af9950e7a6bf905b4dc019404dd"
+"sha256": "ff522d343fb1dc2920aa39407f83a561c15b4bba72034ae3e322e33ea4da2220"
 },
 "model.language_model.layers.9.post_attention_layernorm.weight": {
 "source": "text",
-"sha256": "6d8168c6bd2f760c7c9d1c1a9a9871721ca36a697162fd649f4c2c852a184a22"
+"sha256": "a192f7daafb72f401f1c2d830f8d6e0b4d7e40b7d5f0bfc2951ed2b6d72f43b5"
 },
 "model.language_model.norm.weight": {
 "source": "text",

graft-report.json

@@ -1,5 +1,5 @@
 {
- "text_dir": "/root/autodl-fs/models/04701-c22",
+ "text_dir": "/root/autodl-fs/models/056A0-c31",
  "kit": "/root/autodl-fs/mm-graft-0929/04400-c18-chatvisual",
  "text_arch": [
   "Qwen3_5ForCausalLM"
@@ -14,13 +14,13 @@
  "visual_missing": [],
  "lm_keys_missing": [],
  "lm_keys_extra": [],
- "out": "/dev/shm/vzv/wald4b-v2",
+ "out": "/root/autodl-fs/release/wald4b-v21-build/wald4b-v21",
  "reread_mismatch": [],
  "total_bytes": 9319730176,
  "ok": true,
  "n_mtp": 15,
  "n_keys": 738,
- "lm_digest": "e0b434445e759b4d14195369b513e24035814b91e1d4d3939341e289f7d5f6a0",
+ "lm_digest": "3c874422dd4476866068439eb6c833cb13f8ff83bc62150ab8b5ad30d60c0838",
  "visual_digest": "f6fc999f76c2207af5ee1d3a63af9d278e021d07b981282f93328b82d5640c51",
- "text_model_safetensors_sha256": "6ae382f5a0ed9f4c53cd0953cb9606b1627a6816350340a59114860e6cf6f29a"
+ "text_model_safetensors_sha256": "5a8553b9452e8a655774ef449bad20f79d02783ed258b65068a230db0e07da4c"
 }

llms.txt

@@ -1,15 +1,15 @@
-# Wald-Q4B v2
+# Wald-Q4B v2.1
 
-> Open-weight 4B decision model. Given a state and typed questions (choice, yes/no, score), it returns a calibrated probability for every option through a Jev-compatible `POST /v1/systemone` API. v2 (revision `v2.0`, checkpoint `04701-c22`) starts from Qwen3.5-4B (chat) and thinks natively when unsure (Auto 0.7). Apache-2.0 weights and serving code; self-hosted with vLLM on one NVIDIA GPU.
+> Open-weight 4B decision model. Given a state and typed questions (choice, yes/no, score), it returns a calibrated probability for every option through a Jev-compatible `POST /v1/systemone` API. v2.1 (revision `v2.1`, checkpoint `056A0-c31`) starts from Qwen3.5-4B (chat) and thinks natively when unsure (Auto 0.7). Apache-2.0 weights and serving code; self-hosted with vLLM on one NVIDIA GPU.
 
 ## Docs
 
 - [Model card](https://huggingface.co/org2ai/Wald-4B): scores per policy, comparisons, limits
 - [API reference](https://huggingface.co/org2ai/Wald-4B/blob/main/docs/api.md)
 - [RUNBOOK.md](https://huggingface.co/org2ai/Wald-4B/blob/main/RUNBOOK.md): serving and exact evaluation settings
-- [model-info.json](https://huggingface.co/org2ai/Wald-4B/blob/main/model-info.json) and [evaluation/v2/summary.json](https://huggingface.co/org2ai/Wald-4B/blob/main/evaluation/v2/summary.json): machine-readable facts and scores
+- [model-info.json](https://huggingface.co/org2ai/Wald-4B/blob/main/model-info.json) and [evaluation/v2.1/summary.json](https://huggingface.co/org2ai/Wald-4B/blob/main/evaluation/v2.1/summary.json): machine-readable facts and scores
 
 ## Optional
 
 - [PROVENANCE.md](https://huggingface.co/org2ai/Wald-4B/blob/main/PROVENANCE.md) · [CONTAMINATION.md](https://huggingface.co/org2ai/Wald-4B/blob/main/CONTAMINATION.md) · [CITATION.cff](https://huggingface.co/org2ai/Wald-4B/blob/main/CITATION.cff)
-- Earlier releases: tags `v1.2`, `v1.1`, `v1.0`
+- Earlier releases: tags `v2.0`, `v1.2`, `v1.1`, `v1.0`

model-00001-of-00003.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:10cb7cd22f524f1b33271f8432bcb09409bd879c11d999ea18ca7dd5f8828f5a
+oid sha256:1bf33f0d7f7722ae8f87bca21a9d5ebca8328962e6b170abd09d354d2c978a57
 size 4757977272

model-00002-of-00003.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ed5e7bd4b2e67eee390857268a7660986aca32068c78e6f483e866c27f90b79b
+oid sha256:7cccb8e4ea4c9858c8c78754bee789f5ccc8b437836b307c7919a15165097aef
 size 3653581040

model-files/serving.json

@@ -1,8 +1,8 @@
 {
  "engine": "native-v2-vision",
- "checkpoint": "04701-c22",
- "variant": "vision: 04701-c22 language model (426 tensors, byte-identical) + Qwen/Qwen3.5-4B@851bf6e8 chat vision tower (297) + chat MTP head (15) = 738 keys; Qwen3_5ForConditionalGeneration",
- "model_sha256": "6ae382f5a0ed9f4c53cd0953cb9606b1627a6816350340a59114860e6cf6f29a",
+ "checkpoint": "056A0-c31",
+ "variant": "vision: 056A0-c31 language model (426 tensors, byte-identical) + Qwen/Qwen3.5-4B@851bf6e8 chat vision tower (297) + chat MTP head (15) = 738 keys; Qwen3_5ForConditionalGeneration",
+ "model_sha256": "5a8553b9452e8a655774ef449bad20f79d02783ed258b65068a230db0e07da4c",
  "effort": "auto",
  "prompt_format": "repeat_state_plain",
  "max_model_len": 131072,

model-info.json

@@ -1,9 +1,9 @@
 {
- "name": "Wald-Q4B v2",
+ "name": "Wald-Q4B v2.1",
  "repo": "org2ai/Wald-4B",
- "revision": "v2.0",
- "checkpoint": "04701-c22",
- "parent_checkpoint": "04400-c18",
+ "revision": "v2.1",
+ "checkpoint": "056A0-c31",
+ "parent_checkpoint": null,
  "base_model": "Qwen/Qwen3.5-4B",
  "base_revision": "851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a",
  "license": "apache-2.0",
@@ -27,11 +27,11 @@
    "bytes": 3353259
   },
   "model-00001-of-00003.safetensors": {
-   "sha256": "10cb7cd22f524f1b33271f8432bcb09409bd879c11d999ea18ca7dd5f8828f5a",
+   "sha256": "1bf33f0d7f7722ae8f87bca21a9d5ebca8328962e6b170abd09d354d2c978a57",
    "bytes": 4757977272
   },
   "model-00002-of-00003.safetensors": {
-   "sha256": "ed5e7bd4b2e67eee390857268a7660986aca32068c78e6f483e866c27f90b79b",
+   "sha256": "7cccb8e4ea4c9858c8c78754bee789f5ccc8b437836b307c7919a15165097aef",
    "bytes": 3653581040
   },
   "model-00003-of-00003.safetensors": {
@@ -63,11 +63,11 @@
    "bytes": 6722759
   },
   "graft-manifest.json": {
-   "sha256": "e33ce57511cb924d2bd5559a143c37cf12f1df1e6de911f0a7d4fdbbc0b85eb8",
+   "sha256": "4de700a433f11dd89ae29bf617722a96d4469d9e3b3dbff3ddd0faf450ffe442",
    "bytes": 113687
   },
   "graft-report.json": {
-   "sha256": "d4ae76c16d404c665f84e10079f460221f4c1434c23e0b923bf9a6deb91c1385",
+   "sha256": "9e6e9ab73a5b73006b815206cb5f8a5d8e4c64e8c07f1bdd4acef1f96c5b9072",
    "bytes": 765
   },
   "SHA256SUMS": {
@@ -116,11 +116,7 @@
    }
   }
  },
- "training": {
-  "stage1": "c16 v8, one pass, full parameter (04000-c16): 385,447 records / 106.6M text tokens (137.65M forward tokens in the chat template)",
-  "stage2": "c18 continuation (04400-c18): 125 steps, 11,974,079 forward tokens, 36,509 rows / 39,699 questions, lr 2e-6, state-KL 0.05 to the original chat model",
-  "stage3": "c22 continuation (04701-c22 = step 300 of a 638-step run): 29,440,183 forward tokens / 174,726 questions seen of a 61.6M-token, 82,807-row mix, lr 2e-6, thought LM on \u2264 512 tokens, state-KL 0.05 to 04400-c18"
- },
+ "training": "one full-parameter pass of decision training on corpus c31 from Qwen/Qwen3.5-4B chat (state-KL 0.05 to the base); released weights = 0.85 x trained (05600-c31) + 0.15 x base (WiSE)",
  "evaluation": {
   "policies_measured": [
    "onepass",

serving.json

@@ -1,8 +1,8 @@
 {
  "engine": "native-v2-vision",
- "checkpoint": "04701-c22",
- "variant": "vision: 04701-c22 language model (426 tensors, byte-identical) + Qwen/Qwen3.5-4B@851bf6e8 chat vision tower (297) + chat MTP head (15) = 738 keys; Qwen3_5ForConditionalGeneration",
- "model_sha256": "6ae382f5a0ed9f4c53cd0953cb9606b1627a6816350340a59114860e6cf6f29a",
+ "checkpoint": "056A0-c31",
+ "variant": "vision: 056A0-c31 language model (426 tensors, byte-identical) + Qwen/Qwen3.5-4B@851bf6e8 chat vision tower (297) + chat MTP head (15) = 738 keys; Qwen3_5ForConditionalGeneration",
+ "model_sha256": "5a8553b9452e8a655774ef449bad20f79d02783ed258b65068a230db0e07da4c",
  "effort": "auto",
  "prompt_format": "repeat_state_plain",
  "max_model_len": 131072,