Refresh latest capability comparison with Kev-9B and detailed matrices

.gitattributes

@@ -53,3 +53,4 @@ assets/decision-expanded-v3_core.png filter=lfs diff=lfs merge=lfs -text
 assets/decision-expanded-v4.png filter=lfs diff=lfs merge=lfs -text
 assets/decision-expanded-v5.png filter=lfs diff=lfs merge=lfs -text
 assets/decision-question-scaling.png filter=lfs diff=lfs merge=lfs -text
+assets/decision-expanded-v5-600px.png filter=lfs diff=lfs merge=lfs -text

EVALUATION.md

@@ -1,109 +1,124 @@
 # Evaluation
 
-Four complete panels, shared requests and recorded native decisions. Failures and rejections remain in the requested denominator. The four-panel mean uses 25% each for general decisions, compositional tasks, natural reading, and reading/inference. Within-panel source/family weights are retained. Intervals use the frozen paired source-cluster procedure; they are reported without introducing another release gate.
+This page compares the latest published Decision releases with fixed external references. All displayed cells are copied from verified completed measurements; no predictions, confidence intervals, task weights or release decisions were recomputed for this documentation update.
 
-| Model | Version | General decisions | Compositional tasks | Natural reading | Reading and inference | Mean |
+## Quality protocol
+
+The headline averages four panels equally: 880 general decisions, 880 compositional tasks, 480 natural-reading questions and 480 reading/inference questions. Original within-panel family/source weights are retained; this is not an unweighted average of all 27 tasks. Two native-interface supplements are outside the 2,720-decision mean. Confidence intervals use the original whole-component paired bootstrap, including shared language and counterfactual groups.
+
+The current model is shown first, the other current Decision model second, external open references next, and Jev last as a closed-service frontier reference. This display order is not a ranking. Bold uses unrounded values and the strict external-open-reference rule described in the card; exact ties are not bold.
+
+| Model | Version | General decisions | Compositional tasks | Natural reading | Reading and inference | Mean accuracy ↑ |
 |---|---|---:|---:|---:|---:|---:|
-| Jev | 1.13.0 | **79.10** | **66.38** | **94.53** | **89.79** | **82.45** |
+| Sol | 2B · v1.3 | 73.75 | 46.08 | 76.56 | 84.17 | 70.14 |
+| Nox | 4B · v1.3 | **83.00** | **51.79** | 79.06 | **86.25** | **75.03** |
+| Kev | 9B | 76.36 | 45.54 | 86.72 | 83.75 | 73.09 |
 | Decider | 2B | 64.01 | 46.58 | 92.03 | 84.38 | 71.75 |
 | Qwen3.5 | 4B · untuned | 69.89 | 43.33 | 87.97 | 79.79 | 70.25 |
-| Sol | v1.3 · this release | 73.75 | 46.08 | 76.56 | 84.17 | 70.14 |
-| Sol | v1.2 · previous | 73.64 | 46.92 | 75.31 | 83.33 | 69.80 |
 | Qwen3.5 | 2B · untuned | 57.12 | 39.00 | 73.75 | 72.29 | 60.54 |
 | Laya | Upstream default | 57.01 | 37.75 | 51.25 | 63.75 | 52.44 |
+| Jev | 1.13.0 · frontier | 79.10 | 66.38 | 94.53 | 89.79 | 82.45 |
 
 Accuracy (%). Mean weights each panel equally; each panel retains its frozen family/source weights.
-Bold marks the exact highest score in a column, including exact ties. Rounded visual ties can differ.
+Bold marks Sol or Nox strictly above every external open reference in that metric; Jev and the other Decision model are excluded from the threshold. Ties are not bold.
 
 ### General decisions
 
-| Task | Jev · 1.13.0 | Decider · 2B | Qwen3.5 · 4B · untuned | Sol · v1.3 · this release | Sol · v1.2 · previous | Qwen3.5 · 2B · untuned | Laya · Upstream default |
-|---|---:|---:|---:|---:|---:|---:|---:|
-| ag news | 85.16 | 86.72 | 84.38 | 83.59 | 83.59 | 80.47 | **91.41** |
-| boolean constraints | **100.00** | 71.88 | 62.50 | 50.00 | 50.00 | 37.50 | 43.75 |
-| dbpedia 14 | 96.43 | **98.21** | 97.32 | 95.54 | 95.54 | 92.86 | 83.93 |
-| natural intents | **100.00** | **100.00** | **100.00** | **100.00** | **100.00** | 96.88 | 85.94 |
-| option carrier | 30.21 | 8.33 | 72.92 | **98.96** | **98.96** | 9.38 | 95.83 |
-| ordinal rubric | **100.00** | 84.38 | 90.62 | 65.62 | 62.50 | 81.25 | 18.75 |
-| relational composition | **56.25** | 54.17 | 51.04 | 37.50 | 37.50 | 48.96 | 25.00 |
-| scoped evidence | **89.58** | 47.92 | 51.04 | 79.17 | 79.17 | 36.46 | 37.50 |
-| state tracking | **33.33** | 29.17 | 28.12 | 27.08 | 29.17 | 25.00 | 23.96 |
-| unknown rejection | **100.00** | 59.38 | 60.94 | **100.00** | **100.00** | 62.50 | 64.06 |
+| Task | Sol · 2B · v1.3 | Nox · 4B · v1.3 | Kev · 9B | Decider · 2B | Qwen3.5 · 4B · untuned | Qwen3.5 · 2B · untuned | Laya · Upstream default | Jev · 1.13.0 · frontier |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| News classification | 83.59 | 85.16 | 88.28 | 86.72 | 84.38 | 80.47 | 91.41 | 85.16 |
+| Boolean constraints | 50.00 | **93.75** | 92.19 | 71.88 | 62.50 | 37.50 | 43.75 | 100.00 |
+| Entity classification | 95.54 | 96.43 | 98.21 | 98.21 | 97.32 | 92.86 | 83.93 | 96.43 |
+| Intent routing | 100.00 | 100.00 | 100.00 | 100.00 | 100.00 | 96.88 | 85.94 | 100.00 |
+| Evidence placement | **98.96** | **100.00** | 50.00 | 8.33 | 72.92 | 9.38 | 95.83 | 30.21 |
+| Ordered rubric | 65.62 | 89.06 | 96.88 | 84.38 | 90.62 | 81.25 | 18.75 | 100.00 |
+| Relation composition | 37.50 | 52.08 | 52.08 | 54.17 | 51.04 | 48.96 | 25.00 | 56.25 |
+| Scoped evidence | **79.17** | **78.12** | 63.54 | 47.92 | 51.04 | 36.46 | 37.50 | 89.58 |
+| State tracking | 27.08 | 35.42 | 36.46 | 29.17 | 28.12 | 25.00 | 23.96 | 33.33 |
+| In / out of menu | **100.00** | **100.00** | 85.94 | 59.38 | 60.94 | 62.50 | 64.06 | 100.00 |
 
 ### Compositional tasks
 
-| Task | Jev · 1.13.0 | Decider · 2B | Qwen3.5 · 4B · untuned | Sol · v1.3 · this release | Sol · v1.2 · previous | Qwen3.5 · 2B · untuned | Laya · Upstream default |
-|---|---:|---:|---:|---:|---:|---:|---:|
-| canonical record identity | **76.25** | 56.25 | 50.00 | 50.00 | 50.00 | 46.25 | 47.50 |
-| capacitated assignment | **68.75** | 51.25 | 50.00 | 47.50 | 47.50 | 50.00 | 63.75 |
-| constraint assignment | **55.00** | 23.75 | 23.75 | 26.25 | 26.25 | 21.25 | 22.50 |
-| massive en | **91.67** | 90.00 | **91.67** | 90.83 | 90.83 | 70.83 | 74.17 |
-| massive zh | **88.33** | **88.33** | 86.67 | 87.50 | **88.33** | 74.17 | 73.33 |
-| multiset reconciliation | **58.75** | 23.75 | 27.50 | 28.75 | 30.00 | 27.50 | 26.25 |
-| ordinal service loss | **42.50** | 22.50 | 21.25 | 25.00 | 28.75 | 20.00 | 20.00 |
-| paraconsistent rule closure | **66.25** | 26.25 | 25.00 | 25.00 | 25.00 | 27.50 | 18.75 |
-| temporal exclusion | 41.25 | **42.50** | 31.25 | 41.25 | **42.50** | 28.75 | 12.50 |
-| transaction recovery | **75.00** | 41.25 | 26.25 | 38.75 | 40.00 | 23.75 | 18.75 |
+| Task | Sol · 2B · v1.3 | Nox · 4B · v1.3 | Kev · 9B | Decider · 2B | Qwen3.5 · 4B · untuned | Qwen3.5 · 2B · untuned | Laya · Upstream default | Jev · 1.13.0 · frontier |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| Record identity | 50.00 | 53.75 | 46.25 | 56.25 | 50.00 | 46.25 | 47.50 | 76.25 |
+| Capacity assignment | 47.50 | 46.25 | 50.00 | 51.25 | 50.00 | 50.00 | 63.75 | 68.75 |
+| Constraint assignment | **26.25** | **41.25** | 25.00 | 23.75 | 23.75 | 21.25 | 22.50 | 55.00 |
+| Intent routing · EN | 90.83 | 91.67 | 86.67 | 90.00 | 91.67 | 70.83 | 74.17 | 91.67 |
+| Intent routing · ZH | 87.50 | 87.50 | 85.00 | 88.33 | 86.67 | 74.17 | 73.33 | 88.33 |
+| Multiset reconciliation | 28.75 | 28.75 | 35.00 | 23.75 | 27.50 | 27.50 | 26.25 | 58.75 |
+| Ordered service loss | **25.00** | **30.00** | 21.25 | 22.50 | 21.25 | 20.00 | 20.00 | 42.50 |
+| Conflicting rule closure | 25.00 | **33.75** | 28.75 | 26.25 | 25.00 | 27.50 | 18.75 | 66.25 |
+| Temporal exclusion | 41.25 | 42.50 | 32.50 | 42.50 | 31.25 | 28.75 | 12.50 | 41.25 |
+| Transaction recovery | 38.75 | **62.50** | 45.00 | 41.25 | 26.25 | 23.75 | 18.75 | 75.00 |
 
 ### Natural reading
 
-| Task | Jev · 1.13.0 | Decider · 2B | Qwen3.5 · 4B · untuned | Sol · v1.3 · this release | Sol · v1.2 · previous | Qwen3.5 · 2B · untuned | Laya · Upstream default |
-|---|---:|---:|---:|---:|---:|---:|---:|
-| boolq | **92.50** | 91.88 | 83.75 | 83.12 | 81.25 | 65.00 | 69.38 |
-| belebele en | **96.88** | 93.12 | 93.12 | 70.00 | 70.00 | 83.75 | 38.12 |
-| belebele zh | **96.25** | 91.25 | 91.25 | 70.00 | 68.75 | 81.25 | 28.12 |
+| Task | Sol · 2B · v1.3 | Nox · 4B · v1.3 | Kev · 9B | Decider · 2B | Qwen3.5 · 4B · untuned | Qwen3.5 · 2B · untuned | Laya · Upstream default | Jev · 1.13.0 · frontier |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| Yes / no reading | 83.12 | 86.25 | 90.62 | 91.88 | 83.75 | 65.00 | 69.38 | 92.50 |
+| Reading · EN | 70.00 | 73.12 | 83.75 | 93.12 | 93.12 | 83.75 | 38.12 | 96.88 |
+| Reading · ZH | 70.00 | 70.62 | 81.88 | 91.25 | 91.25 | 81.25 | 28.12 | 96.25 |
 
 ### Reading and inference
 
-| Task | Jev · 1.13.0 | Decider · 2B | Qwen3.5 · 4B · untuned | Sol · v1.3 · this release | Sol · v1.2 · previous | Qwen3.5 · 2B · untuned | Laya · Upstream default |
-|---|---:|---:|---:|---:|---:|---:|---:|
-| cosmos qa | **86.67** | 66.67 | 60.83 | 69.17 | 66.67 | 56.67 | 30.00 |
-| squad2 answerability | **90.83** | 84.17 | 81.67 | 83.33 | 82.50 | 82.50 | 57.50 |
-| snli | 82.50 | **90.83** | 80.00 | 88.33 | 87.50 | 64.17 | 72.50 |
-| qasc | **99.17** | 95.83 | 96.67 | 95.83 | 96.67 | 85.83 | 95.00 |
+| Task | Sol · 2B · v1.3 | Nox · 4B · v1.3 | Kev · 9B | Decider · 2B | Qwen3.5 · 4B · untuned | Qwen3.5 · 2B · untuned | Laya · Upstream default | Jev · 1.13.0 · frontier |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| Contextual reasoning | **69.17** | **70.83** | 67.50 | 66.67 | 60.83 | 56.67 | 30.00 | 86.67 |
+| Answerability | 83.33 | **88.33** | 80.83 | 84.17 | 81.67 | 82.50 | 57.50 | 90.83 |
+| Textual entailment | 88.33 | 89.17 | 90.00 | 90.83 | 80.00 | 64.17 | 72.50 | 82.50 |
+| Scientific inference | 95.83 | 96.67 | 96.67 | 95.83 | 96.67 | 85.83 | 95.00 | 99.17 |
 
 ### Probability quality
 
 | Model | Version | Weighted Brier ↓ | Valid probability rows |
 |---|---|---:|---:|
-| Jev | 1.13.0 | **0.2287** | 2,720 / 2,720 |
+| Sol | 2B · v1.3 | 0.4258 | 2,720 / 2,720 |
+| Nox | 4B · v1.3 | **0.3323** | 2,720 / 2,720 |
+| Kev | 9B | 0.3715 | 2,720 / 2,720 |
 | Decider | 2B | 0.3535 | 2,720 / 2,720 |
 | Qwen3.5 | 4B · untuned | 0.3996 | 2,720 / 2,720 |
-| Sol | v1.3 · this release | 0.4258 | 2,720 / 2,720 |
-| Sol | v1.2 · previous | 0.4281 | 2,720 / 2,720 |
 | Qwen3.5 | 2B · untuned | 0.5130 | 2,720 / 2,720 |
 | Laya | Upstream default | 0.5903 | 2,720 / 2,720 |
+| Jev | 1.13.0 · frontier | 0.2287 | 2,720 / 2,720 |
 
 Missing Brier means probability coverage was incomplete; no supported-only average is substituted.
 The two native contract supplements are reported separately and are outside this quality mean.
 
-![Detailed General decisions](assets/decision-expanded-old_core.png)
+## Current probability quality
+
+Sol v1.3 raw T=1 weighted Brier is 0.443265; the shipped CAL-fitted Brier is 0.425840. Calibration uses the fixed development CAL split after weights are selected. It does not improve category accuracy by itself, and confidence is not a factuality guarantee.
+
+## Latency
 
-![Detailed Compositional tasks](assets/decision-expanded-v3_core.png)
+Only this release is displayed in [the latency tables and curve](QUESTION-SCALING.md). Tokenization and inference are included; loading and network are excluded. Three fresh processes supply 30 measured requests per point on an otherwise idle AMD gfx942 GPU. These measurements do not imply service concurrency or a cross-model throughput ranking.
 
-![Detailed Natural reading](assets/decision-expanded-v4.png)
+## Comparator identities and scope
 
-![Detailed Reading and inference](assets/decision-expanded-v5.png)
+[Nox v1.3](https://huggingface.co/llm-semantic-router/Decision-1.0-Nox/tree/ad089ad3a5dc9a7a21e6d96db546bb53e2212654) and [Sol v1.3](https://huggingface.co/llm-semantic-router/Decision-1.0-Sol/tree/2412d9470d3aa125b346262dad80f6161847aa09) use their actually published bundles and completed four-panel evaluations.
 
-## This update versus v1.2
+[Kev-9B](https://huggingface.co/jaredpalmer/kev-9b/tree/6281032426a9ca3a08374d137bf9dfd8afd78e8a) uses its unchanged native adapter at source revision `e0bcf50153f1bda4ca6a8be5e12cbd5f9ebbce1c` under the recorded matched-FLA runtime. Its four quality panels have complete 2,720-row coverage; its separate native panels each succeed on 217 of 220 requests, so complete quality coverage must not be read as universal native-interface support.
 
-The four-panel mean changes from **69.8012% to 70.1397%** (+0.3385 points; paired 95% interval −0.3326 to +1.0233 points). This point estimate meets the unchanged update rule; the interval crosses zero and does not establish a statistically clear improvement.
+Untuned Qwen references use the frozen chat/LM-head adapter. Laya retains its upstream default language router and native truncation; Decider retains its native adapter. Jev is the recorded 1.13.0 service snapshot; its server-side truncation is not observable. These previously observed evaluation suites are not all pristine holdouts. Supplemental finite-probe scores are outside this mean and do not alter the release rule.
 
-| Panel | Accuracy change (points) |
-|---|---:|
-| General decisions | +0.1042 |
-| Compositional tasks | −0.8333 |
-| Natural reading | +1.2500 |
-| Reading and inference | +0.8333 |
+The architecture, weights, tokenizer, normalization profile, temperature and original qualification are unchanged by this documentation refresh. Historical paired-update evidence remains archived as machine-readable provenance; no old release is presented as a current comparison row.
 
-Weighted Brier decreases from 0.428080 to 0.425840; lower is better. It worsens on general decisions and compositional tasks, and improves on both natural-text panels. Sol remains below Decider on the reported four-panel mean. These results do not imply that every capability improved.
+[Exact displayed statistics](metrics/expanded-quality.json) · [Comparator provenance](metrics/comparator-coverage.json) · [Figure evidence](metrics/figure-evidence.json)
 
-The development-selected checkpoint continues the v1.2 training lineage for 375 updates on 24,000 examples, with the same model architecture and cross-entropy objective. Selection used development results; no test result chose the checkpoint or temperature. The alternative balanced-data arm and its development results remain part of the research record.
+## Semantic consistency
 
-Fresh matched Choice Q1–Q32 latency has a geometric-mean candidate/predecessor ratio of **0.995974** (0.40% lower). This small measured difference is descriptive; it does not establish a general speedup. See [all three native types and p95](QUESTION-SCALING.md).
+| Model | Consistent / eligible groups | Consistency ↑ |
+|---|---:|---:|
+| Sol · 2B · v1.3 | 167/192 | **86.98** |
+| Nox · 4B · v1.3 | 174/192 | **90.62** |
+| Kev · 9B | 143/192 | 74.48 |
+| Decider · 2B | 126/192 | 65.62 |
+| Qwen3.5 · 4B · untuned | 98/192 | 51.04 |
+| Qwen3.5 · 2B · untuned | 94/192 | 48.96 |
+| Laya · Upstream default | 115/192 | 59.90 |
+| Jev · 1.13.0 · frontier | 158/192 | 82.29 |
 
-## Scope
+Mixed equivalent variants of language, keys and presentation; **consistency is not accuracy**. A consistently wrong answer still counts. This observed-core supplement weights 192 eligible semantic groups equally and is outside the four-panel mean. [Definitions and exact counts](metrics/semantic-consistency.json).
 
-These panels include previously observed regression sets. They are not all pristine holdouts. The untuned Qwen models use their frozen chat/LM-head adapters. Laya uses its unmodified upstream default language router (no explicit language override); Decider retains its native adapter. Documented native truncation is retained. Jev is a closed service snapshot, not a reproducible weight release. Full native-interface supplements are outside the quality mean.
 
-[Exact aggregate statistics](metrics/expanded-quality.json) · [Probability and adapter coverage](metrics/comparator-coverage.json) · [Measured latency](QUESTION-SCALING.md) · [Material provenance](metrics/materials-provenance.json)
+Eligible groups require more than one variant, uniquely identifiable option semantics, and a shared semantic gold. All outputs must be valid and agree semantically. The 240 singleton news/entity groups are excluded. This is a mixed-variation measure, not an isolated option-order experiment.

MATERIALS.json

@@ -0,0 +1,219 @@
+{
+  "status": "docs-only-review-candidate-not-uploaded",
+  "generator_sha256": "2de88a794052e4cad6d18e0c0fb3efa7aaa2a7d20e321a851906dec04a0ac81d",
+  "source_evidence_sha256": "8ef8f73315fc185dca5c887728c44ed517a97623cbe25710c2dc83480204d3a8",
+  "files": [
+    {
+      "file": "EVALUATION.md",
+      "bytes": 9573,
+      "sha256": "eec0f6e285ae5e48b86e83e0588586a5198f96fc7d3844f96c8801e7ceffaab5"
+    },
+    {
+      "file": "QUESTION-SCALING.md",
+      "bytes": 1549,
+      "sha256": "9eb8e8537bf1a045d510e57c8fdaf81b58192f5f760eb54778602c60eb441d1e"
+    },
+    {
+      "file": "README.md",
+      "bytes": 4999,
+      "sha256": "166c19d0cdb09b51329f28e0f8e6ed46c30ce02978510ae48635c5141eb9fd5b"
+    },
+    {
+      "file": "assets/architecture.png",
+      "bytes": 511058,
+      "sha256": "8fcfa10b3941ee4aec1f5fba20a25e382a7d5de83074071c9ef68e627996de40"
+    },
+    {
+      "file": "assets/architecture.svg",
+      "bytes": 14718,
+      "sha256": "215d9af6f94cc24847fc1d23a0b2a287b36afdc1e10ea4f5c82895b2fe103002"
+    },
+    {
+      "file": "assets/decision-expanded-old_core-600px.png",
+      "bytes": 167493,
+      "sha256": "6d36e595bea7465903ee787d217ef07ad648472d336655e88e51263325d0a7e2"
+    },
+    {
+      "file": "assets/decision-expanded-old_core.pdf",
+      "bytes": 44384,
+      "sha256": "cbd80145027dd79b37ed799a64b4c173d44973b358bc05414da80355ba7072a2"
+    },
+    {
+      "file": "assets/decision-expanded-old_core.png",
+      "bytes": 312133,
+      "sha256": "2744aafba8afb133b1c92d7b1813d3e769da8babe9439e3d966f94ac28963bbd"
+    },
+    {
+      "file": "assets/decision-expanded-old_core.svg",
+      "bytes": 79821,
+      "sha256": "0648a4f0ed3720dcc9dc0f19dd43c2f46e08b2dce1cf7a057dba957fa475f288"
+    },
+    {
+      "file": "assets/decision-expanded-overview-600px.png",
+      "bytes": 94850,
+      "sha256": "b8e8361345b814083eef97dc08b9a4fd173bcef83faa72c5f66eac5d0cc1c10b"
+    },
+    {
+      "file": "assets/decision-expanded-overview.pdf",
+      "bytes": 41563,
+      "sha256": "4f008072cb5b8268a31e058d0839d68e2416c7f63795ff40cd931906cf74552e"
+    },
+    {
+      "file": "assets/decision-expanded-overview.png",
+      "bytes": 187355,
+      "sha256": "2f12699c29ecde8146482e63925b9e80e3bab4047f79fbec17a87e50a1e64ec1"
+    },
+    {
+      "file": "assets/decision-expanded-overview.svg",
+      "bytes": 48619,
+      "sha256": "6565620295efd3180f1c2079ea7fec332f17b977fcc9781101e9a82001eff743"
+    },
+    {
+      "file": "assets/decision-expanded-ranking-600px.png",
+      "bytes": 83184,
+      "sha256": "ddbb71c528b3741f1258367e5436b35148461c628aaf4d03d209ae6c4eccdee1"
+    },
+    {
+      "file": "assets/decision-expanded-ranking.pdf",
+      "bytes": 37133,
+      "sha256": "bac1d9bf032cd06d4d81170fd10e033e56c1a793e958e9599f8bc570700bf400"
+    },
+    {
+      "file": "assets/decision-expanded-ranking.png",
+      "bytes": 186098,
+      "sha256": "a78d29ea6a73eab02cc8897140614d92b0d7bb4e1295c43c9a2ece60d03ec155"
+    },
+    {
+      "file": "assets/decision-expanded-ranking.svg",
+      "bytes": 38118,
+      "sha256": "8de46cff93bdf4b899e74b6b7baddafae6956b6e32b9713702472c8aace0a164"
+    },
+    {
+      "file": "assets/decision-expanded-v3_core-600px.png",
+      "bytes": 168267,
+      "sha256": "df7099ba80f0a000f7c64cfe6b8aca7ead06d5311e6967a49ae8de32317a2077"
+    },
+    {
+      "file": "assets/decision-expanded-v3_core.pdf",
+      "bytes": 44464,
+      "sha256": "2ffe3a8fdad5dc957f524dbb03d437aaa50a098920358322575f84d65e95e09f"
+    },
+    {
+      "file": "assets/decision-expanded-v3_core.png",
+      "bytes": 300452,
+      "sha256": "a621a7ffee11dc4909e89a200b7c1207a75bff57a9a237b66e8ed8f2fbac6983"
+    },
+    {
+      "file": "assets/decision-expanded-v3_core.svg",
+      "bytes": 79901,
+      "sha256": "75c74babb4214337caff4b3a5ec36bb1de5824f486312bde33ab3264d2c50f48"
+    },
+    {
+      "file": "assets/decision-expanded-v4-600px.png",
+      "bytes": 76498,
+      "sha256": "613e8f9c4bc1a6e398190414ae6bf0a70e784cd36e278beec10becbf6c5e71cc"
+    },
+    {
+      "file": "assets/decision-expanded-v4.pdf",
+      "bytes": 37680,
+      "sha256": "b56a46b9f26ebd18280b73f5bdb381cbc207312f477a82c002a662409e86e32f"
+    },
+    {
+      "file": "assets/decision-expanded-v4.png",
+      "bytes": 152677,
+      "sha256": "287486dbbdd9e8f89ae5ea358d20170ef2496ccf6ac79b3b0b6169990ff14da8"
+    },
+    {
+      "file": "assets/decision-expanded-v4.svg",
+      "bytes": 42942,
+      "sha256": "8abbe97a67a7a89291dea776f93990a42b0f2fafe47330bbafcfe6c14c4cc404"
+    },
+    {
+      "file": "assets/decision-expanded-v5-600px.png",
+      "bytes": 102382,
+      "sha256": "8fb899a4acca41fd9cc4cb6885e5e7f6b77a5c962af78f4ad00e99929fb8ce54"
+    },
+    {
+      "file": "assets/decision-expanded-v5.pdf",
+      "bytes": 40179,
+      "sha256": "b08d52344bd383766b19e967d4ab9ab0783a50ea423de93140dce26c7c91354a"
+    },
+    {
+      "file": "assets/decision-expanded-v5.png",
+      "bytes": 187722,
+      "sha256": "bf0d8f84d12b6f1e46ae02a70e0ed42adf45a818d8b4b594b1461ddcb635a13f"
+    },
+    {
+      "file": "assets/decision-expanded-v5.svg",
+      "bytes": 48613,
+      "sha256": "a5fba4d5176a8edb6668051678b5926cd0aaa40eabf192d9cefea03e6a64338f"
+    },
+    {
+      "file": "assets/decision-family-header.png",
+      "bytes": 2962868,
+      "sha256": "213511289ce8df038d938ac470e803c427ed57f0f85cc397dd4d79964b866541"
+    },
+    {
+      "file": "assets/decision-question-scaling-600px.png",
+      "bytes": 40784,
+      "sha256": "3de11b7e5d1b49ca874b50ef6ae8ced1dcdc3a83dae4889170a0134f0ca3f8cc"
+    },
+    {
+      "file": "assets/decision-question-scaling.pdf",
+      "bytes": 31275,
+      "sha256": "57f5e52d5722ba9c5643bbb18bab9ad2aa19cc58f2723780cfa2452c523f2fb0"
+    },
+    {
+      "file": "assets/decision-question-scaling.png",
+      "bytes": 94954,
+      "sha256": "0236ed14e75d90cc7691ef6c9d892191070d5c380083fd2602b222bad33b34cd"
+    },
+    {
+      "file": "assets/decision-question-scaling.svg",
+      "bytes": 25462,
+      "sha256": "9c5a9d5bdec8dd4f80c2af20f828709e45dbc32c825f6f93fae8f980c79199d6"
+    },
+    {
+      "file": "assets/readout.png",
+      "bytes": 285481,
+      "sha256": "fb6b3d1cf149a1a3ccb584eb1c213df73c07bcd328732b4238928e2a2a4af085"
+    },
+    {
+      "file": "metrics/comparator-coverage.json",
+      "bytes": 21708,
+      "sha256": "7ec368a3aefedff14af32d94a320c5f57d9c0bea2779272e2d171a4afc1d0fa9"
+    },
+    {
+      "file": "metrics/expanded-quality.json",
+      "bytes": 64295,
+      "sha256": "7b8936b43bce9d19d9010ab61225a6460160b3aa76c37f5d6df5ba650b433c6c"
+    },
+    {
+      "file": "metrics/figure-evidence.json",
+      "bytes": 9339,
+      "sha256": "1fde66bcafc9e602b030e7492a01d72fca510cc5fbd690f4fdef34696db38f23"
+    },
+    {
+      "file": "metrics/materials-provenance.json",
+      "bytes": 1333,
+      "sha256": "3f052b894ab0780825ef698556483a5900d1e024cfcfaa1e89919261d812adb4"
+    },
+    {
+      "file": "metrics/question-scaling.json",
+      "bytes": 9036,
+      "sha256": "c74fb9c207f804a53c8f7cb57fd13eb87b6313e334c7ec2b5d7e05df9b4c9d5b"
+    },
+    {
+      "file": "metrics/semantic-consistency.json",
+      "bytes": 20784,
+      "sha256": "5a389236a22e1068be862b2daf88a5caf5a384a9260f0cafbaca9a37b18d8d26"
+    },
+    {
+      "file": "model-card-example.json",
+      "bytes": 3773,
+      "sha256": "f9086df8ed8256e6ce9b054f29d008497dea5216e9fabfd2c99cec10a703f2bb"
+    }
+  ],
+  "post_render_finalizer_sha256": "4818899220640baf74e6f87915dfe4ca5f77aedeca40e63c6f95ff9db1cf3d3e",
+  "linear_latency_finalizer_sha256": "750bfff4297d24237896ef8080758045bc2f12c5fb7ce1a6a790fe871371c161"
+}

QUESTION-SCALING.md

@@ -1,64 +1,42 @@
 # Request latency
 
-Fresh matched measurements of this candidate and its own published predecessor.
+Current published Sol v1.3, measured on one otherwise idle AMD gfx942 GPU.
 
-The same physical AMD gfx942 GPU runs three independently loaded blocks per model in a counterbalanced order. Each of 18 fixed Choice/Noul/Score cases has three warmups and ten measurements per block. The tables pool all 30 measurements per point. Loading and network time are excluded; tokenization and inference are included. No cross-request prefix cache is enabled. These are sequential requests, not concurrent-service throughput.
+Three independently loaded blocks provide 30 measured requests per point after three warmups per block. End-to-end Python latency includes tokenization and inference, excluding model loading and network. No cross-request prefix cache is enabled. These are sequential requests, not concurrent-service throughput.
 
-![Choice question scaling](assets/decision-question-scaling.png)
+![Question scaling](assets/decision-question-scaling.png)
 
 ## Choice
 
-| Model | Q | p50 ms ↓ | p95 ms ↓ | Input tokens |
-|---|---:|---:|---:|---:|
-| Sol · v1.3 · this release | 1 | 24.456 | **24.624** | 309 |
-| Sol · v1.2 · previous | 1 | **24.421** | 24.863 | 309 |
-| Sol · v1.3 · this release | 2 | **25.003** | **25.263** | 618 |
-| Sol · v1.2 · previous | 2 | 25.182 | 25.749 | 618 |
-| Sol · v1.3 · this release | 4 | **25.974** | 26.297 | 1236 |
-| Sol · v1.2 · previous | 4 | 26.082 | **26.283** | 1236 |
-| Sol · v1.3 · this release | 8 | **37.481** | **37.572** | 2472 |
-| Sol · v1.2 · previous | 8 | 37.781 | 37.906 | 2472 |
-| Sol · v1.3 · this release | 16 | **74.391** | 75.485 | 4944 |
-| Sol · v1.2 · previous | 16 | 74.721 | **75.293** | 4944 |
-| Sol · v1.3 · this release | 32 | **148.705** | **149.854** | 9888 |
-| Sol · v1.2 · previous | 32 | 149.006 | 149.994 | 9888 |
+| Questions | p50 ms ↓ | p95 ms ↓ | Input tokens |
+|---:|---:|---:|---:|
+| 1 | 24.456 | 24.624 | 309 |
+| 2 | 25.003 | 25.263 | 618 |
+| 4 | 25.974 | 26.297 | 1236 |
+| 8 | 37.481 | 37.572 | 2472 |
+| 16 | 74.391 | 75.485 | 4944 |
+| 32 | 148.705 | 149.854 | 9888 |
 
 ## Noul
 
-| Model | Q | p50 ms ↓ | p95 ms ↓ | Input tokens |
-|---|---:|---:|---:|---:|
-| Sol · v1.3 · this release | 1 | 24.321 | 24.801 | 262 |
-| Sol · v1.2 · previous | 1 | **24.316** | **24.648** | 262 |
-| Sol · v1.3 · this release | 2 | 25.055 | **25.375** | 524 |
-| Sol · v1.2 · previous | 2 | **24.912** | 25.707 | 524 |
-| Sol · v1.3 · this release | 4 | 25.623 | 26.369 | 1048 |
-| Sol · v1.2 · previous | 4 | **25.245** | **25.803** | 1048 |
-| Sol · v1.3 · this release | 8 | **33.521** | **33.974** | 2096 |
-| Sol · v1.2 · previous | 8 | 34.066 | 34.235 | 2096 |
-| Sol · v1.3 · this release | 16 | **67.472** | **67.723** | 4192 |
-| Sol · v1.2 · previous | 16 | 67.599 | 67.965 | 4192 |
-| Sol · v1.3 · this release | 32 | **133.348** | **134.522** | 8384 |
-| Sol · v1.2 · previous | 32 | 134.354 | 135.091 | 8384 |
+| Questions | p50 ms ↓ | p95 ms ↓ | Input tokens |
+|---:|---:|---:|---:|
+| 1 | 24.321 | 24.801 | 262 |
+| 2 | 25.055 | 25.375 | 524 |
+| 4 | 25.623 | 26.369 | 1048 |
+| 8 | 33.521 | 33.974 | 2096 |
+| 16 | 67.472 | 67.723 | 4192 |
+| 32 | 133.348 | 134.522 | 8384 |
 
 ## Score
 
-| Model | Q | p50 ms ↓ | p95 ms ↓ | Input tokens |
-|---|---:|---:|---:|---:|
-| Sol · v1.3 · this release | 1 | **24.529** | **24.781** | 307 |
-| Sol · v1.2 · previous | 1 | 24.689 | 25.177 | 307 |
-| Sol · v1.3 · this release | 2 | **25.017** | **25.207** | 614 |
-| Sol · v1.2 · previous | 2 | 25.265 | 25.687 | 614 |
-| Sol · v1.3 · this release | 4 | **25.725** | **26.390** | 1228 |
-| Sol · v1.2 · previous | 4 | 25.912 | 26.874 | 1228 |
-| Sol · v1.3 · this release | 8 | **37.491** | 37.811 | 2456 |
-| Sol · v1.2 · previous | 8 | 37.592 | **37.689** | 2456 |
-| Sol · v1.3 · this release | 16 | **74.585** | **74.987** | 4912 |
-| Sol · v1.2 · previous | 16 | 74.744 | 75.224 | 4912 |
-| Sol · v1.3 · this release | 32 | **148.353** | **149.268** | 9824 |
-| Sol · v1.2 · previous | 32 | 148.473 | 149.621 | 9824 |
+| Questions | p50 ms ↓ | p95 ms ↓ | Input tokens |
+|---:|---:|---:|---:|
+| 1 | 24.529 | 24.781 | 307 |
+| 2 | 25.017 | 25.207 | 614 |
+| 4 | 25.725 | 26.390 | 1228 |
+| 8 | 37.491 | 37.811 | 2456 |
+| 16 | 74.585 | 74.987 | 4912 |
+| 32 | 148.353 | 149.268 | 9824 |
 
-Choice six-point geometric-mean latency ratio (candidate / predecessor): **0.9960**.
-
-The candidate has 0.40% lower measured latency by this summary. This is a descriptive result for these six request sizes, not a claim of universal speedup.
-
-The runtime/profile and temperature belong to these exact measured bundles. Older release timing is not reused.
+The bundled runtime, normalization profile and temperature belong to this exact measured release. The source experiment used a counterbalanced matched design; this display contains only the current release. No cross-model latency conclusion is inferred.

README.md

@@ -34,24 +34,50 @@ tags:
 
 ## Measured capability
 
-**70.14% four-panel mean**, +0.34 points versus v1.2. The paired 95% interval is −0.33 to +1.02 points; this small observed gain is uncertain.
+**70.14% four-panel mean** across 2,720 decisions. Results include strengths and remaining gaps across all 27 tasks.
 
 | Model | Mean accuracy ↑ | Decisions | Composition | Reading | Inference |
 |---|---:|---:|---:|---:|---:|
-| Jev · 1.13.0 | **82.45** | **79.10** | **66.38** | **94.53** | **89.79** |
+| Sol · 2B · v1.3 | 70.14 | 73.75 | 46.08 | 76.56 | 84.17 |
+| Nox · 4B · v1.3 | **75.03** | **83.00** | **51.79** | 79.06 | **86.25** |
+| Kev · 9B | 73.09 | 76.36 | 45.54 | 86.72 | 83.75 |
 | Decider · 2B | 71.75 | 64.01 | 46.58 | 92.03 | 84.38 |
 | Qwen3.5 · 4B · untuned | 70.25 | 69.89 | 43.33 | 87.97 | 79.79 |
-| Sol · v1.3 · this release | 70.14 | 73.75 | 46.08 | 76.56 | 84.17 |
-| Sol · v1.2 · previous | 69.80 | 73.64 | 46.92 | 75.31 | 83.33 |
 | Qwen3.5 · 2B · untuned | 60.54 | 57.12 | 39.00 | 73.75 | 72.29 |
 | Laya · Upstream default | 52.44 | 57.01 | 37.75 | 51.25 | 63.75 |
+| Jev · 1.13.0 · frontier | 82.45 | 79.10 | 66.38 | 94.53 | 89.79 |
 
-Accuracy (%), **2,720 decisions**. Each panel contributes one quarter; its original family/source weights are retained. Bold marks each column's exact highest score. [Methods, uncertainty and all task results](EVALUATION.md).
+Accuracy (%). Each panel contributes one quarter with its frozen family/source weights. **Bold** marks Sol or Nox strictly above every external open reference for that metric; the other Decision model and Jev are excluded from the threshold. Jev is a closed-service frontier reference. [Methods and uncertainty](EVALUATION.md).
 
-![Expanded decision ranking](assets/decision-expanded-ranking.png)
+![Decision comparison](assets/decision-expanded-ranking.png)
 
 ![Four-panel capability overview](assets/decision-expanded-overview.png)
 
+## Detailed capabilities
+
+![General decisions](assets/decision-expanded-old_core.png)
+
+![Compositional tasks](assets/decision-expanded-v3_core.png)
+
+![Natural reading](assets/decision-expanded-v4.png)
+
+![Reading and inference](assets/decision-expanded-v5.png)
+
+## Semantic consistency
+
+| Model | Consistent / eligible groups | Consistency ↑ |
+|---|---:|---:|
+| Sol · 2B · v1.3 | 167/192 | **86.98** |
+| Nox · 4B · v1.3 | 174/192 | **90.62** |
+| Kev · 9B | 143/192 | 74.48 |
+| Decider · 2B | 126/192 | 65.62 |
+| Qwen3.5 · 4B · untuned | 98/192 | 51.04 |
+| Qwen3.5 · 2B · untuned | 94/192 | 48.96 |
+| Laya · Upstream default | 115/192 | 59.90 |
+| Jev · 1.13.0 · frontier | 158/192 | 82.29 |
+
+Mixed equivalent variants of language, keys and presentation; **consistency is not accuracy**. A consistently wrong answer still counts. This observed-core supplement weights 192 eligible semantic groups equally and is outside the four-panel mean. [Definitions and exact counts](metrics/semantic-consistency.json).
+
 ## More questions, measured
 
 ![Current release request latency](assets/decision-question-scaling.png)

metrics/comparator-coverage.json

@@ -436,5 +436,23 @@
       },
       "runtime_fingerprint_sha256": "b5af3d997c95d0a6d2088a64f341726fc0daee5eebc730f851b2b65bf0de8d1b"
     }
+  },
+  "kev-9b": {
+    "repo_id": "jaredpalmer/kev-9b",
+    "revision": "6281032426a9ca3a08374d137bf9dfd8afd78e8a",
+    "source_revision": "e0bcf50153f1bda4ca6a8be5e12cbd5f9ebbce1c",
+    "weight_fingerprint": "2b29b6a7e43e94b467a0d516e5d17008cba0f5c6e36c2793c57140604accc4a9",
+    "four_panel_rows": 2720,
+    "native_supplement_success_counts": [
+      [
+        217,
+        220
+      ],
+      [
+        217,
+        220
+      ]
+    ],
+    "adapter": "upstream native adapter; matched FLA0.5.2 / Transformers5.17"
   }
 }

metrics/expanded-quality.json

@@ -1,78 +1,78 @@
 {
   "models": {
-    "Decider \u00b7 2B": {
+    "Nox-retention-selected": {
       "old_core": {
-        "point": 0.6401413690476191,
-        "exact_fraction": "17207/26880",
+        "point": 0.8300223214285715,
+        "exact_fraction": "7437/8960",
         "ci95": [
-          0.5992559523809524,
-          0.6806184895833334
+          0.7974693080357143,
+          0.862016369047619
         ]
       },
       "v3_core": {
-        "point": 0.4658333333333333,
-        "exact_fraction": "559/1200",
+        "point": 0.5179166666666667,
+        "exact_fraction": "1243/2400",
         "ci95": [
-          0.4362500000000001,
-          0.49499999999999994
+          0.4920833333333333,
+          0.545
         ]
       },
       "v4": {
-        "point": 0.9203125,
-        "exact_fraction": "589/640",
+        "point": 0.790625,
+        "exact_fraction": "253/320",
         "ci95": [
-          0.8903095518867924,
-          0.9479299363057325
+          0.7453125,
+          0.8331463068181818
         ]
       },
       "v5": {
-        "point": 0.84375,
-        "exact_fraction": "27/32",
+        "point": 0.8625,
+        "exact_fraction": "69/80",
         "ci95": [
-          0.8130660699474849,
-          0.8734622752789641
+          0.8321482316646345,
+          0.8915620903779563
         ]
       },
       "v3_joint": {
-        "point": 0.5529873511904762,
-        "exact_fraction": "148643/268800",
+        "point": 0.6739694940476191,
+        "exact_fraction": "181163/268800",
         "ci95": [
-          0.5282024739583334,
-          0.5777246093749999
+          0.652767857142857,
+          0.6950187872023811
         ]
       },
       "expanded_joint": {
-        "point": 0.7175093005952381,
-        "exact_fraction": "385733/537600",
+        "point": 0.7502659970238095,
+        "exact_fraction": "403343/537600",
         "ci95": [
-          0.7012097728950374,
-          0.7336543123969403
+          0.7330469621683191,
+          0.7670605596068275
         ]
       },
       "families": {
         "old_core": {
           "ag_news": {
-            "correct": 111,
+            "correct": 109,
             "requested": 128,
             "accuracy_ci95": [
-              0.8046875,
-              0.921875
+              0.7890625,
+              0.90625
             ]
           },
           "boolean_constraints": {
-            "correct": 46,
+            "correct": 60,
             "requested": 64,
             "accuracy_ci95": [
-              0.578125,
-              0.84375
+              0.859375,
+              1.0
             ]
           },
           "dbpedia_14": {
-            "correct": 110,
+            "correct": 108,
             "requested": 112,
             "accuracy_ci95": [
-              0.9553571428571429,
-              1.0
+              0.9285714285714286,
+              0.9910714285714286
             ]
           },
           "natural_intents": {
@@ -84,203 +84,203 @@
             ]
           },
           "option_carrier": {
-            "correct": 8,
+            "correct": 96,
             "requested": 96,
             "accuracy_ci95": [
-              0.041666666666666664,
-              0.13541666666666666
+              1.0,
+              1.0
             ]
           },
           "ordinal_rubric": {
-            "correct": 54,
+            "correct": 57,
             "requested": 64,
             "accuracy_ci95": [
-              0.71875,
-              0.9375
+              0.78125,
+              0.984375
             ]
           },
           "relational_composition": {
-            "correct": 52,
+            "correct": 50,
             "requested": 96,
             "accuracy_ci95": [
-              0.3854166666666667,
+              0.34375,
               0.6979166666666666
             ]
           },
           "scoped_evidence": {
-            "correct": 46,
+            "correct": 75,
             "requested": 96,
             "accuracy_ci95": [
-              0.2916666666666667,
-              0.6666666666666666
+              0.6145833333333334,
+              0.9270833333333334
             ]
           },
           "state_tracking": {
-            "correct": 28,
+            "correct": 34,
             "requested": 96,
             "accuracy_ci95": [
-              0.13541666666666666,
-              0.4583333333333333
+              0.19791666666666666,
+              0.53125
             ]
           },
           "unknown_rejection": {
-            "correct": 38,
+            "correct": 64,
             "requested": 64,
             "accuracy_ci95": [
-              0.375,
-              0.796875
+              1.0,
+              1.0
             ]
           }
         },
         "v3_core": {
           "canonical_record_identity": {
-            "correct": 45,
+            "correct": 43,
             "requested": 80,
             "accuracy_ci95": [
-              0.45,
-              0.675
+              0.5,
+              0.575
             ]
           },
           "capacitated_assignment": {
-            "correct": 41,
+            "correct": 37,
             "requested": 80,
             "accuracy_ci95": [
-              0.4375,
-              0.5875
+              0.4125,
+              0.5125
             ]
           },
           "constraint_assignment": {
-            "correct": 19,
+            "correct": 33,
             "requested": 80,
             "accuracy_ci95": [
-              0.1625,
-              0.3
+              0.3375,
+              0.475
             ]
           },
           "massive_en": {
-            "correct": 108,
+            "correct": 110,
             "requested": 120,
             "accuracy_ci95": [
-              0.8416666666666667,
-              0.9500000000000001
+              0.8666666666666667,
+              0.9583333333333334
             ]
           },
           "massive_zh": {
-            "correct": 106,
+            "correct": 105,
             "requested": 120,
             "accuracy_ci95": [
-              0.825,
-              0.9333333333333333
+              0.8166666666666667,
+              0.925
             ]
           },
           "multiset_reconciliation": {
-            "correct": 19,
+            "correct": 23,
             "requested": 80,
             "accuracy_ci95": [
-              0.1875,
-              0.2875
+              0.2125,
+              0.375
             ]
           },
           "ordinal_service_loss": {
-            "correct": 18,
+            "correct": 24,
             "requested": 80,
             "accuracy_ci95": [
-              0.175,
-              0.275
+              0.225,
+              0.375
             ]
           },
           "paraconsistent_rule_closure": {
-            "correct": 21,
+            "correct": 27,
             "requested": 80,
             "accuracy_ci95": [
-              0.1875,
-              0.3375
+              0.275,
+              0.425
             ]
           },
           "temporal_exclusion": {
             "correct": 34,
             "requested": 80,
             "accuracy_ci95": [
-              0.2625,
-              0.6125
+              0.3,
+              0.55
             ]
           },
           "transaction_recovery": {
-            "correct": 33,
+            "correct": 50,
             "requested": 80,
             "accuracy_ci95": [
-              0.3125,
-              0.5125
+              0.4875,
+              0.7625
             ]
           }
         },
         "v4": {
           "boolq": {
-            "correct": 147,
+            "correct": 138,
             "requested": 160,
             "accuracy_ci95": [
-              0.875,
-              0.95625
+              0.80625,
+              0.9125
             ]
           },
           "belebele_en": {
-            "correct": 149,
+            "correct": 117,
             "requested": 160,
             "accuracy_ci95": [
-              0.8910256410256411,
-              0.9683544303797469
+              0.6585365853658537,
+              0.802547770700637
             ]
           },
           "belebele_zh": {
-            "correct": 146,
+            "correct": 113,
             "requested": 160,
             "accuracy_ci95": [
-              0.8622754491017964,
-              0.9559748427672956
+              0.632258064516129,
+              0.779874213836478
             ]
           }
         },
         "v5": {
           "cosmos_qa": {
-            "correct": 80,
+            "correct": 85,
             "requested": 120,
             "accuracy_ci95": [
-              0.5833333333333334,
-              0.75
+              0.625,
+              0.7833333333333333
             ]
           },
           "squad2_answerability": {
-            "correct": 101,
+            "correct": 106,
             "requested": 120,
             "accuracy_ci95": [
-              0.7857142857142857,
-              0.896551724137931
+              0.8214285714285714,
+              0.9407026836158189
             ]
           },
           "snli": {
-            "correct": 109,
+            "correct": 107,
             "requested": 120,
             "accuracy_ci95": [
-              0.85,
-              0.9583333333333334
+              0.8333333333333334,
+              0.9416666666666667
             ]
           },
           "qasc": {
-            "correct": 115,
+            "correct": 116,
             "requested": 120,
             "accuracy_ci95": [
-              0.9112903225806451,
+              0.9327731092436975,
               0.991869918699187
             ]
           }
         }
       },
       "proper_scores": {
-        "old_core": 0.44246049270704674,
-        "v3_core": 0.5974556966463511,
-        "v4": 0.13737783476250795,
-        "v5": 0.23664932218586204,
-        "expanded_weighted_brier": 0.35348583657544197
+        "old_core": 0.22558726136256252,
+        "v3_core": 0.6082926735018094,
+        "v4": 0.3013823815596702,
+        "v5": 0.19389530242905464,
+        "expanded_weighted_brier": 0.3322894047132742
       },
       "probability_coverage": {
         "old_core": {
@@ -302,99 +302,99 @@
       },
       "brier_ci95": {
         "old_core": [
-          0.40579693831982216,
-          0.4787313843802038
+          0.1846602761421124,
+          0.2688592667224405
         ],
         "v3_core": [
-          0.5767822346729764,
-          0.6189924929703139
+          0.5803874232294962,
+          0.6368345885571338
         ],
         "v4": [
-          0.10073496035335808,
-          0.17666521563140353
+          0.24180651680296794,
+          0.3645626677453216
         ],
         "v5": [
-          0.2031026857962821,
-          0.27234790035750356
+          0.16179407555902875,
+          0.22847239437277253
         ],
         "expanded_weighted_brier": [
-          0.3373790190368837,
-          0.37038546676401063
+          0.3111201217342611,
+          0.35396540259832066
         ]
       }
     },
-    "Jev \u00b7 1.13.0": {
+    "Sol-composition-selected": {
       "old_core": {
-        "point": 0.7909598214285715,
-        "exact_fraction": "7087/8960",
+        "point": 0.7374627976190476,
+        "exact_fraction": "19823/26880",
         "ci95": [
-          0.7616062127976191,
-          0.819271763392857
+          0.6992931547619048,
+          0.7742950148809524
         ]
       },
       "v3_core": {
-        "point": 0.66375,
-        "exact_fraction": "531/800",
+        "point": 0.4608333333333333,
+        "exact_fraction": "553/1200",
         "ci95": [
-          0.6316666666666667,
-          0.6945833333333333
+          0.43875000000000003,
+          0.48291666666666666
         ]
       },
       "v4": {
-        "point": 0.9453125,
-        "exact_fraction": "121/128",
+        "point": 0.765625,
+        "exact_fraction": "49/64",
         "ci95": [
-          0.9197115384615384,
-          0.9685135514495712
+          0.7210721817484662,
+          0.8086081288343558
         ]
       },
       "v5": {
-        "point": 0.8979166666666667,
-        "exact_fraction": "431/480",
+        "point": 0.8416666666666667,
+        "exact_fraction": "101/120",
         "ci95": [
-          0.8713251366120218,
-          0.9229854264081012
+          0.8111792524213076,
+          0.8710143196177259
         ]
       },
       "v3_joint": {
-        "point": 0.7273549107142857,
-        "exact_fraction": "65171/89600",
+        "point": 0.5991480654761905,
+        "exact_fraction": "161051/268800",
         "ci95": [
-          0.7058627232142857,
-          0.7482447916666666
+          0.5773286830357143,
+          0.6207081473214285
         ]
       },
       "expanded_joint": {
-        "point": 0.8244847470238095,
-        "exact_fraction": "443243/537600",
+        "point": 0.7013969494047619,
+        "exact_fraction": "377071/537600",
         "ci95": [
-          0.8104711862785249,
-          0.8378880015726611
+          0.6845734141379205,
+          0.7180071456587547
         ]
       },
       "families": {
         "old_core": {
           "ag_news": {
-            "correct": 109,
+            "correct": 107,
             "requested": 128,
             "accuracy_ci95": [
-              0.7890625,
-              0.90625
+              0.7734375,
+              0.8984375
             ]
           },
           "boolean_constraints": {
-            "correct": 64,
+            "correct": 32,
             "requested": 64,
             "accuracy_ci95": [
-              1.0,
-              1.0
+              0.328125,
+              0.671875
             ]
           },
           "dbpedia_14": {
-            "correct": 108,
+            "correct": 107,
             "requested": 112,
             "accuracy_ci95": [
-              0.9285714285714286,
+              0.9107142857142857,
               0.9910714285714286
             ]
           },
@@ -407,43 +407,43 @@
             ]
           },
           "option_carrier": {
-            "correct": 29,
+            "correct": 95,
             "requested": 96,
             "accuracy_ci95": [
-              0.2604166666666667,
-              0.34375
+              0.96875,
+              1.0
             ]
           },
           "ordinal_rubric": {
-            "correct": 64,
+            "correct": 42,
             "requested": 64,
             "accuracy_ci95": [
-              1.0,
-              1.0
+              0.484375,
+              0.8125
             ]
           },
           "relational_composition": {
-            "correct": 54,
+            "correct": 36,
             "requested": 96,
             "accuracy_ci95": [
-              0.3645833333333333,
-              0.75
+              0.19791666666666666,
+              0.5625
             ]
           },
           "scoped_evidence": {
-            "correct": 86,
+            "correct": 76,
             "requested": 96,
             "accuracy_ci95": [
-              0.78125,
-              0.9895833333333334
+              0.6354166666666666,
+              0.9375
             ]
           },
           "state_tracking": {
-            "correct": 32,
+            "correct": 26,
             "requested": 96,
             "accuracy_ci95": [
-              0.16640625000000023,
-              0.5106770833333295
+              0.125,
+              0.4270833333333333
             ]
           },
           "unknown_rejection": {
@@ -457,153 +457,153 @@
         },
         "v3_core": {
           "canonical_record_identity": {
-            "correct": 61,
+            "correct": 40,
             "requested": 80,
             "accuracy_ci95": [
-              0.6375,
-              0.875
+              0.5,
+              0.5
             ]
           },
           "capacitated_assignment": {
-            "correct": 55,
+            "correct": 38,
             "requested": 80,
             "accuracy_ci95": [
-              0.575,
-              0.7875
+              0.4375,
+              0.5
             ]
           },
           "constraint_assignment": {
-            "correct": 44,
+            "correct": 21,
             "requested": 80,
             "accuracy_ci95": [
-              0.4,
-              0.675
+              0.1875,
+              0.3125
             ]
           },
           "massive_en": {
-            "correct": 110,
+            "correct": 109,
             "requested": 120,
             "accuracy_ci95": [
-              0.8666666666666667,
-              0.9583333333333334
+              0.8500000000000001,
+              0.9583333333333333
             ]
           },
           "massive_zh": {
-            "correct": 106,
+            "correct": 105,
             "requested": 120,
             "accuracy_ci95": [
-              0.825,
-              0.9416666666666667
+              0.8166666666666667,
+              0.9333333333333332
             ]
           },
           "multiset_reconciliation": {
-            "correct": 47,
+            "correct": 23,
             "requested": 80,
             "accuracy_ci95": [
-              0.5125,
-              0.65
+              0.25,
+              0.325
             ]
           },
           "ordinal_service_loss": {
-            "correct": 34,
+            "correct": 20,
             "requested": 80,
             "accuracy_ci95": [
-              0.325,
-              0.525
+              0.1875,
+              0.3125
             ]
           },
           "paraconsistent_rule_closure": {
-            "correct": 53,
+            "correct": 20,
             "requested": 80,
             "accuracy_ci95": [
-              0.5875,
-              0.725
+              0.25,
+              0.25
             ]
           },
           "temporal_exclusion": {
             "correct": 33,
             "requested": 80,
             "accuracy_ci95": [
-              0.3125,
-              0.5125
+              0.275,
+              0.5625
             ]
           },
           "transaction_recovery": {
-            "correct": 60,
+            "correct": 31,
             "requested": 80,
             "accuracy_ci95": [
-              0.625,
-              0.8625
+              0.3,
+              0.475
             ]
           }
         },
         "v4": {
           "boolq": {
-            "correct": 148,
+            "correct": 133,
             "requested": 160,
             "accuracy_ci95": [
-              0.88125,
-              0.9625
+              0.775,
+              0.8875
             ]
           },
           "belebele_en": {
-            "correct": 155,
+            "correct": 112,
             "requested": 160,
             "accuracy_ci95": [
-              0.9390243902439024,
-              0.9937106918238994
+              0.6280452353942144,
+              0.7692307692307693
             ]
           },
           "belebele_zh": {
-            "correct": 154,
+            "correct": 112,
             "requested": 160,
             "accuracy_ci95": [
-              0.930379746835443,
-              0.9877300613496932
+              0.6289261100440628,
+              0.7682926829268293
             ]
           }
         },
         "v5": {
           "cosmos_qa": {
-            "correct": 104,
+            "correct": 83,
             "requested": 120,
             "accuracy_ci95": [
-              0.8,
-              0.925
+              0.6083333333333333,
+              0.775
             ]
           },
           "squad2_answerability": {
-            "correct": 109,
+            "correct": 100,
             "requested": 120,
             "accuracy_ci95": [
-              0.8620689655172413,
-              0.9508196721311475
+              0.7786885245901639,
+              0.8859649122807017
             ]
           },
           "snli": {
-            "correct": 99,
+            "correct": 106,
             "requested": 120,
             "accuracy_ci95": [
-              0.75,
-              0.8916666666666667
+              0.825,
+              0.9416666666666667
             ]
           },
           "qasc": {
-            "correct": 119,
+            "correct": 115,
             "requested": 120,
             "accuracy_ci95": [
-              0.9745762711864406,
-              1.0
+              0.912,
+              0.9918032786885246
             ]
           }
         }
       },
       "proper_scores": {
-        "old_core": 0.24806216931976088,
-        "v3_core": 0.4403896043771044,
-        "v4": 0.07270531250000001,
-        "v5": 0.15378333333333333,
-        "expanded_weighted_brier": 0.22873510488254967
+        "old_core": 0.42193820645312435,
+        "v3_core": 0.6913074500248156,
+        "v4": 0.33935305266273397,
+        "v5": 0.250760098789168,
+        "expanded_weighted_brier": 0.42583970198246046
       },
       "probability_coverage": {
         "old_core": {
@@ -625,308 +625,308 @@
       },
       "brier_ci95": {
         "old_core": [
-          0.21912044641352113,
-          0.27809132807795134
+          0.365622760488921,
+          0.48051917201614724
         ],
         "v3_core": [
-          0.4130360873769258,
-          0.4688325815363228
+          0.6695390244240492,
+          0.7144598051610135
         ],
         "v4": [
-          0.04802243933725408,
-          0.100621887515886
+          0.28386912550452903,
+          0.3966449902668765
         ],
         "v5": [
-          0.12116203971412033,
-          0.18851406142759564
+          0.21482273267248006,
+          0.28856469231607446
         ],
         "expanded_weighted_brier": [
-          0.21435641903529326,
-          0.2433957477840872
+          0.40383487736748497,
+          0.44863844813421033
         ]
       }
     },
-    "Laya \u00b7 EN/ML": {
+    "kev-9b": {
       "old_core": {
-        "point": 0.5701264880952381,
-        "exact_fraction": "3065/5376",
+        "point": 0.7635788690476191,
+        "exact_fraction": "4105/5376",
         "ci95": [
-          0.5321056547619049,
-          0.6094494047619048
+          0.729052269345238,
+          0.798104538690476
         ]
       },
       "v3_core": {
-        "point": 0.3775,
-        "exact_fraction": "151/400",
+        "point": 0.4554166666666667,
+        "exact_fraction": "1093/2400",
         "ci95": [
-          0.35458333333333336,
-          0.39958333333333335
+          0.4304166666666666,
+          0.48
         ]
       },
       "v4": {
-        "point": 0.5125,
-        "exact_fraction": "41/80",
+        "point": 0.8671875,
+        "exact_fraction": "111/128",
         "ci95": [
-          0.4661671277501309,
-          0.557454268292683
+          0.8295129243827161,
+          0.9025080271804062
         ]
       },
       "v5": {
-        "point": 0.6375,
-        "exact_fraction": "51/80",
+        "point": 0.8375,
+        "exact_fraction": "67/80",
         "ci95": [
-          0.6032948178983685,
-          0.6716809713450388
+          0.8058128867311815,
+          0.8681624928267716
         ]
       },
       "v3_joint": {
-        "point": 0.47381324404761904,
-        "exact_fraction": "127361/268800",
+        "point": 0.6094977678571428,
+        "exact_fraction": "54611/89600",
         "ci95": [
-          0.4519083891369048,
-          0.4964252232142857
+          0.58812109375,
+          0.6303536086309525
         ]
       },
       "expanded_joint": {
-        "point": 0.5244066220238095,
-        "exact_fraction": "281921/537600",
+        "point": 0.7309207589285714,
+        "exact_fraction": "130981/179200",
         "ci95": [
-          0.5065901244344904,
-          0.5426481205211884
+          0.7149211829146931,
+          0.7468337863817799
         ]
       },
       "families": {
         "old_core": {
           "ag_news": {
-            "correct": 117,
+            "correct": 113,
             "requested": 128,
             "accuracy_ci95": [
-              0.859375,
-              0.9609375
+              0.828125,
+              0.9375
             ]
           },
           "boolean_constraints": {
-            "correct": 28,
+            "correct": 59,
             "requested": 64,
             "accuracy_ci95": [
-              0.28125,
-              0.609375
+              0.84375,
+              0.984375
             ]
           },
           "dbpedia_14": {
-            "correct": 94,
+            "correct": 110,
             "requested": 112,
             "accuracy_ci95": [
-              0.7678571428571429,
-              0.9017857142857143
+              0.9553571428571429,
+              1.0
             ]
           },
           "natural_intents": {
-            "correct": 55,
+            "correct": 64,
             "requested": 64,
             "accuracy_ci95": [
-              0.75,
-              0.9375
+              1.0,
+              1.0
             ]
           },
           "option_carrier": {
-            "correct": 92,
+            "correct": 48,
             "requested": 96,
             "accuracy_ci95": [
-              0.8854166666666666,
-              1.0
+              0.5,
+              0.5
             ]
           },
           "ordinal_rubric": {
-            "correct": 12,
+            "correct": 62,
             "requested": 64,
             "accuracy_ci95": [
-              0.0625,
-              0.34375
+              0.921875,
+              1.0
             ]
           },
           "relational_composition": {
-            "correct": 24,
+            "correct": 50,
             "requested": 96,
             "accuracy_ci95": [
-              0.17708333333333334,
-              0.3333333333333333
+              0.3229166666666667,
+              0.7083333333333334
             ]
           },
           "scoped_evidence": {
-            "correct": 36,
+            "correct": 61,
             "requested": 96,
             "accuracy_ci95": [
-              0.23958333333333334,
-              0.5208333333333334
+              0.4583333333333333,
+              0.8125
             ]
           },
           "state_tracking": {
-            "correct": 23,
+            "correct": 35,
             "requested": 96,
             "accuracy_ci95": [
-              0.11458333333333333,
-              0.3854166666666667
+              0.19791666666666666,
+              0.53125
             ]
           },
           "unknown_rejection": {
-            "correct": 41,
+            "correct": 55,
             "requested": 64,
             "accuracy_ci95": [
-              0.4375,
-              0.84375
+              0.75,
+              0.953125
             ]
           }
         },
         "v3_core": {
           "canonical_record_identity": {
-            "correct": 38,
+            "correct": 37,
             "requested": 80,
             "accuracy_ci95": [
-              0.425,
-              0.525
+              0.375,
+              0.55
             ]
           },
           "capacitated_assignment": {
-            "correct": 51,
+            "correct": 40,
             "requested": 80,
             "accuracy_ci95": [
-              0.55,
-              0.725
+              0.5,
+              0.5
             ]
           },
           "constraint_assignment": {
-            "correct": 18,
+            "correct": 20,
             "requested": 80,
             "accuracy_ci95": [
               0.15,
-              0.3125
+              0.35
             ]
           },
           "massive_en": {
-            "correct": 89,
+            "correct": 104,
             "requested": 120,
             "accuracy_ci95": [
-              0.6583333333333333,
-              0.8166666666666667
+              0.8,
+              0.925
             ]
           },
           "massive_zh": {
-            "correct": 88,
+            "correct": 102,
             "requested": 120,
             "accuracy_ci95": [
-              0.6583333333333332,
-              0.8083333333333333
+              0.7833333333333333,
+              0.9083333333333334
             ]
           },
           "multiset_reconciliation": {
-            "correct": 21,
+            "correct": 28,
             "requested": 80,
             "accuracy_ci95": [
               0.25,
-              0.2875
+              0.45
             ]
           },
           "ordinal_service_loss": {
-            "correct": 16,
+            "correct": 17,
             "requested": 80,
             "accuracy_ci95": [
-              0.15,
+              0.175,
               0.25
             ]
           },
           "paraconsistent_rule_closure": {
-            "correct": 15,
+            "correct": 23,
             "requested": 80,
             "accuracy_ci95": [
-              0.1375,
-              0.2375
+              0.225,
+              0.3625
             ]
           },
           "temporal_exclusion": {
-            "correct": 10,
+            "correct": 26,
             "requested": 80,
             "accuracy_ci95": [
-              0.05,
-              0.2125
+              0.2375,
+              0.4
             ]
           },
           "transaction_recovery": {
-            "correct": 15,
+            "correct": 36,
             "requested": 80,
             "accuracy_ci95": [
-              0.1,
-              0.2625
+              0.3625,
+              0.5375
             ]
           }
         },
         "v4": {
           "boolq": {
-            "correct": 111,
+            "correct": 145,
             "requested": 160,
             "accuracy_ci95": [
-              0.61875,
-              0.7625
+              0.85625,
+              0.95
             ]
           },
           "belebele_en": {
-            "correct": 61,
+            "correct": 134,
             "requested": 160,
             "accuracy_ci95": [
-              0.3062370621019109,
-              0.4585987261146497
+              0.7751442307692308,
+              0.8954248366013072
             ]
           },
           "belebele_zh": {
-            "correct": 45,
+            "correct": 131,
             "requested": 160,
             "accuracy_ci95": [
-              0.2138364779874214,
-              0.3503184713375796
+              0.7564102564102564,
+              0.8773006134969326
             ]
           }
         },
         "v5": {
           "cosmos_qa": {
-            "correct": 36,
+            "correct": 81,
             "requested": 120,
             "accuracy_ci95": [
-              0.21666666666666667,
-              0.38333333333333336
+              0.5916666666666667,
+              0.7583333333333333
             ]
           },
           "squad2_answerability": {
-            "correct": 69,
+            "correct": 97,
             "requested": 120,
             "accuracy_ci95": [
-              0.5084745762711864,
-              0.6416666666666667
+              0.7410061813186815,
+              0.8769262295081967
             ]
           },
           "snli": {
-            "correct": 87,
+            "correct": 108,
             "requested": 120,
             "accuracy_ci95": [
-              0.6416666666666667,
-              0.8
+              0.8416666666666667,
+              0.95
             ]
           },
           "qasc": {
-            "correct": 114,
+            "correct": 116,
             "requested": 120,
             "accuracy_ci95": [
-              0.9083333333333333,
-              0.983739837398374
+              0.9333333333333333,
+              0.991869918699187
             ]
           }
         }
       },
       "proper_scores": {
-        "old_core": 0.5709760997718798,
-        "v3_core": 0.700479457442883,
-        "v4": 0.6149664146731222,
-        "v5": 0.4747850267787647,
-        "expanded_weighted_brier": 0.5903017496666624
+        "old_core": 0.3579333453812052,
+        "v3_core": 0.6866233781387137,
+        "v4": 0.19851043272472813,
+        "v5": 0.24294908882289837,
+        "expanded_weighted_brier": 0.37150406126688634
       },
       "probability_coverage": {
         "old_core": {
@@ -948,308 +948,308 @@
       },
       "brier_ci95": {
         "old_core": [
-          0.5307603405543191,
-          0.611972788595099
+          0.3081633137767088,
+          0.40859251388765916
         ],
         "v3_core": [
-          0.677830213791377,
-          0.7242813502662896
+          0.656917870943704,
+          0.7174003704260583
         ],
         "v4": [
-          0.5601239092953858,
-          0.6718388215324546
+          0.1515888351649323,
+          0.24959797650555063
         ],
         "v5": [
-          0.4441678879307976,
-          0.5061559610657542
+          0.20024454898292451,
+          0.2894231406637825
         ],
         "expanded_weighted_brier": [
-          0.5709748439741565,
-          0.6102902589236013
+          0.34973259999393813,
+          0.39395437787789234
         ]
       }
     },
-    "Qwen3.5 \u00b7 2B, untuned": {
+    "Decider \u00b7 2B": {
       "old_core": {
-        "point": 0.5712425595238095,
-        "exact_fraction": "3071/5376",
+        "point": 0.6401413690476191,
+        "exact_fraction": "17207/26880",
         "ci95": [
-          0.535453869047619,
-          0.607627418154762
+          0.5992559523809524,
+          0.6806184895833334
         ]
       },
       "v3_core": {
-        "point": 0.39,
-        "exact_fraction": "39/100",
+        "point": 0.4658333333333333,
+        "exact_fraction": "559/1200",
         "ci95": [
-          0.3733333333333334,
-          0.40625
+          0.4362500000000001,
+          0.49499999999999994
         ]
       },
       "v4": {
-        "point": 0.7375,
-        "exact_fraction": "59/80",
+        "point": 0.9203125,
+        "exact_fraction": "589/640",
         "ci95": [
-          0.6928571428571428,
-          0.782292158018868
+          0.8903095518867924,
+          0.9479299363057325
         ]
       },
       "v5": {
-        "point": 0.7229166666666667,
-        "exact_fraction": "347/480",
+        "point": 0.84375,
+        "exact_fraction": "27/32",
         "ci95": [
-          0.6848579953997695,
-          0.760121271215623
+          0.8130660699474849,
+          0.8734622752789641
         ]
       },
       "v3_joint": {
-        "point": 0.4806212797619048,
-        "exact_fraction": "129191/268800",
+        "point": 0.5529873511904762,
+        "exact_fraction": "148643/268800",
         "ci95": [
-          0.46085509672619046,
-          0.5003615141369048
+          0.5282024739583334,
+          0.5777246093749999
         ]
       },
       "expanded_joint": {
-        "point": 0.605414806547619,
-        "exact_fraction": "325471/537600",
+        "point": 0.7175093005952381,
+        "exact_fraction": "385733/537600",
         "ci95": [
-          0.5881936025503637,
-          0.6231347666599296
+          0.7012097728950374,
+          0.7336543123969403
         ]
       },
       "families": {
         "old_core": {
           "ag_news": {
-            "correct": 103,
+            "correct": 111,
             "requested": 128,
             "accuracy_ci95": [
-              0.734375,
-              0.8671875
+              0.8046875,
+              0.921875
             ]
           },
           "boolean_constraints": {
-            "correct": 24,
+            "correct": 46,
             "requested": 64,
             "accuracy_ci95": [
-              0.21875,
-              0.53125
+              0.578125,
+              0.84375
             ]
           },
           "dbpedia_14": {
-            "correct": 104,
+            "correct": 110,
             "requested": 112,
             "accuracy_ci95": [
-              0.875,
-              0.9732142857142857
+              0.9553571428571429,
+              1.0
             ]
           },
           "natural_intents": {
-            "correct": 62,
+            "correct": 64,
             "requested": 64,
             "accuracy_ci95": [
-              0.921875,
+              1.0,
               1.0
             ]
           },
           "option_carrier": {
-            "correct": 9,
+            "correct": 8,
             "requested": 96,
             "accuracy_ci95": [
               0.041666666666666664,
-              0.15625
+              0.13541666666666666
             ]
           },
           "ordinal_rubric": {
-            "correct": 52,
+            "correct": 54,
             "requested": 64,
             "accuracy_ci95": [
-              0.671875,
+              0.71875,
               0.9375
             ]
           },
           "relational_composition": {
-            "correct": 47,
+            "correct": 52,
             "requested": 96,
             "accuracy_ci95": [
-              0.375,
-              0.6145833333333334
+              0.3854166666666667,
+              0.6979166666666666
             ]
           },
           "scoped_evidence": {
-            "correct": 35,
+            "correct": 46,
             "requested": 96,
             "accuracy_ci95": [
-              0.23958333333333334,
-              0.5
+              0.2916666666666667,
+              0.6666666666666666
             ]
           },
           "state_tracking": {
-            "correct": 24,
+            "correct": 28,
             "requested": 96,
             "accuracy_ci95": [
-              0.15625,
-              0.3541666666666667
+              0.13541666666666666,
+              0.4583333333333333
             ]
           },
           "unknown_rejection": {
-            "correct": 40,
+            "correct": 38,
             "requested": 64,
             "accuracy_ci95": [
-              0.4375,
-              0.8125
+              0.375,
+              0.796875
             ]
           }
         },
         "v3_core": {
           "canonical_record_identity": {
-            "correct": 37,
+            "correct": 45,
             "requested": 80,
             "accuracy_ci95": [
-              0.425,
-              0.5
+              0.45,
+              0.675
             ]
           },
           "capacitated_assignment": {
-            "correct": 40,
+            "correct": 41,
             "requested": 80,
             "accuracy_ci95": [
-              0.5,
-              0.5
+              0.4375,
+              0.5875
             ]
           },
           "constraint_assignment": {
-            "correct": 17,
+            "correct": 19,
             "requested": 80,
             "accuracy_ci95": [
-              0.15,
-              0.275
+              0.1625,
+              0.3
             ]
           },
           "massive_en": {
-            "correct": 85,
+            "correct": 108,
             "requested": 120,
             "accuracy_ci95": [
-              0.625,
-              0.7833333333333334
+              0.8416666666666667,
+              0.9500000000000001
             ]
           },
           "massive_zh": {
-            "correct": 89,
+            "correct": 106,
             "requested": 120,
             "accuracy_ci95": [
-              0.6666666666666666,
-              0.8166666666666667
+              0.825,
+              0.9333333333333333
             ]
           },
           "multiset_reconciliation": {
-            "correct": 22,
+            "correct": 19,
             "requested": 80,
             "accuracy_ci95": [
-              0.25,
-              0.3125
+              0.1875,
+              0.2875
             ]
           },
           "ordinal_service_loss": {
-            "correct": 16,
+            "correct": 18,
             "requested": 80,
             "accuracy_ci95": [
-              0.2,
-              0.2
+              0.175,
+              0.275
             ]
           },
           "paraconsistent_rule_closure": {
-            "correct": 22,
+            "correct": 21,
             "requested": 80,
             "accuracy_ci95": [
-              0.25,
-              0.3125
+              0.1875,
+              0.3375
             ]
           },
           "temporal_exclusion": {
-            "correct": 23,
+            "correct": 34,
             "requested": 80,
             "accuracy_ci95": [
-              0.2375,
-              0.3375
+              0.2625,
+              0.6125
             ]
           },
           "transaction_recovery": {
-            "correct": 19,
+            "correct": 33,
             "requested": 80,
             "accuracy_ci95": [
-              0.2125,
-              0.25
+              0.3125,
+              0.5125
             ]
           }
         },
         "v4": {
           "boolq": {
-            "correct": 104,
+            "correct": 147,
             "requested": 160,
             "accuracy_ci95": [
-              0.575,
-              0.725
+              0.875,
+              0.95625
             ]
           },
           "belebele_en": {
-            "correct": 134,
+            "correct": 149,
             "requested": 160,
             "accuracy_ci95": [
-              0.7791411042944786,
-              0.8924050632911392
+              0.8910256410256411,
+              0.9683544303797469
             ]
           },
           "belebele_zh": {
-            "correct": 130,
+            "correct": 146,
             "requested": 160,
             "accuracy_ci95": [
-              0.7484276729559748,
-              0.8742138364779874
+              0.8622754491017964,
+              0.9559748427672956
             ]
           }
         },
         "v5": {
           "cosmos_qa": {
-            "correct": 68,
+            "correct": 80,
             "requested": 120,
             "accuracy_ci95": [
-              0.48333333333333334,
-              0.65
+              0.5833333333333334,
+              0.75
             ]
           },
           "squad2_answerability": {
-            "correct": 99,
+            "correct": 101,
             "requested": 120,
             "accuracy_ci95": [
-              0.7631578947368421,
-              0.8833333333333333
+              0.7857142857142857,
+              0.896551724137931
             ]
           },
           "snli": {
-            "correct": 77,
+            "correct": 109,
             "requested": 120,
             "accuracy_ci95": [
-              0.55,
-              0.725
+              0.85,
+              0.9583333333333334
             ]
           },
           "qasc": {
-            "correct": 103,
+            "correct": 115,
             "requested": 120,
             "accuracy_ci95": [
-              0.7950819672131147,
-              0.9173553719008265
+              0.9112903225806451,
+              0.991869918699187
             ]
           }
         }
       },
       "proper_scores": {
-        "old_core": 0.5723777597574953,
-        "v3_core": 0.7477724847216414,
-        "v4": 0.31678065323351606,
-        "v5": 0.4151003191775817,
-        "expanded_weighted_brier": 0.5130078042225586
+        "old_core": 0.44246049270704674,
+        "v3_core": 0.5974556966463511,
+        "v4": 0.13737783476250795,
+        "v5": 0.23664932218586204,
+        "expanded_weighted_brier": 0.35348583657544197
       },
       "probability_coverage": {
         "old_core": {
@@ -1271,165 +1271,488 @@
       },
       "brier_ci95": {
         "old_core": [
-          0.5306199339702083,
-          0.613544919176592
+          0.40579693831982216,
+          0.4787313843802038
         ],
         "v3_core": [
-          0.7296525923390248,
-          0.7668117066670981
+          0.5767822346729764,
+          0.6189924929703139
         ],
         "v4": [
-          0.27446950339258713,
-          0.3607698891917115
+          0.10073496035335808,
+          0.17666521563140353
         ],
         "v5": [
-          0.37186021348012016,
-          0.45924139988994434
+          0.2031026857962821,
+          0.27234790035750356
         ],
         "expanded_weighted_brier": [
-          0.4934107964402564,
-          0.5319870117266131
+          0.3373790190368837,
+          0.37038546676401063
         ]
       }
     },
-    "Qwen3.5 \u00b7 4B, untuned": {
+    "Laya \u00b7 EN/ML": {
       "old_core": {
-        "point": 0.6988839285714286,
-        "exact_fraction": "3131/4480",
+        "point": 0.5701264880952381,
+        "exact_fraction": "3065/5376",
         "ci95": [
-          0.6632803199404762,
-          0.7337797619047619
+          0.5321056547619049,
+          0.6094494047619048
         ]
       },
       "v3_core": {
-        "point": 0.43333333333333335,
-        "exact_fraction": "13/30",
+        "point": 0.3775,
+        "exact_fraction": "151/400",
         "ci95": [
-          0.4179166666666667,
-          0.4479166666666667
+          0.35458333333333336,
+          0.39958333333333335
         ]
       },
       "v4": {
-        "point": 0.8796875,
-        "exact_fraction": "563/640",
+        "point": 0.5125,
+        "exact_fraction": "41/80",
         "ci95": [
-          0.8461463032298809,
-          0.9125000000000001
+          0.4661671277501309,
+          0.557454268292683
         ]
       },
       "v5": {
-        "point": 0.7979166666666667,
-        "exact_fraction": "383/480",
+        "point": 0.6375,
+        "exact_fraction": "51/80",
         "ci95": [
-          0.7641944237372369,
-          0.830579286387442
+          0.6032948178983685,
+          0.6716809713450388
         ]
       },
       "v3_joint": {
-        "point": 0.566108630952381,
-        "exact_fraction": "15217/26880",
+        "point": 0.47381324404761904,
+        "exact_fraction": "127361/268800",
         "ci95": [
-          0.5467666480654761,
-          0.5849108072916667
+          0.4519083891369048,
+          0.4964252232142857
         ]
       },
       "expanded_joint": {
-        "point": 0.7024553571428571,
-        "exact_fraction": "3147/4480",
+        "point": 0.5244066220238095,
+        "exact_fraction": "281921/537600",
         "ci95": [
-          0.6873480253417569,
-          0.7176086887612844
+          0.5065901244344904,
+          0.5426481205211884
         ]
       },
       "families": {
         "old_core": {
           "ag_news": {
-            "correct": 108,
+            "correct": 117,
             "requested": 128,
             "accuracy_ci95": [
-              0.78125,
-              0.90625
+              0.859375,
+              0.9609375
             ]
           },
           "boolean_constraints": {
-            "correct": 40,
+            "correct": 28,
             "requested": 64,
             "accuracy_ci95": [
-              0.484375,
-              0.765625
+              0.28125,
+              0.609375
             ]
           },
           "dbpedia_14": {
-            "correct": 109,
+            "correct": 94,
             "requested": 112,
             "accuracy_ci95": [
-              0.9375,
+              0.7678571428571429,
+              0.9017857142857143
+            ]
+          },
+          "natural_intents": {
+            "correct": 55,
+            "requested": 64,
+            "accuracy_ci95": [
+              0.75,
+              0.9375
+            ]
+          },
+          "option_carrier": {
+            "correct": 92,
+            "requested": 96,
+            "accuracy_ci95": [
+              0.8854166666666666,
               1.0
             ]
           },
+          "ordinal_rubric": {
+            "correct": 12,
+            "requested": 64,
+            "accuracy_ci95": [
+              0.0625,
+              0.34375
+            ]
+          },
+          "relational_composition": {
+            "correct": 24,
+            "requested": 96,
+            "accuracy_ci95": [
+              0.17708333333333334,
+              0.3333333333333333
+            ]
+          },
+          "scoped_evidence": {
+            "correct": 36,
+            "requested": 96,
+            "accuracy_ci95": [
+              0.23958333333333334,
+              0.5208333333333334
+            ]
+          },
+          "state_tracking": {
+            "correct": 23,
+            "requested": 96,
+            "accuracy_ci95": [
+              0.11458333333333333,
+              0.3854166666666667
+            ]
+          },
+          "unknown_rejection": {
+            "correct": 41,
+            "requested": 64,
+            "accuracy_ci95": [
+              0.4375,
+              0.84375
+            ]
+          }
+        },
+        "v3_core": {
+          "canonical_record_identity": {
+            "correct": 38,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.425,
+              0.525
+            ]
+          },
+          "capacitated_assignment": {
+            "correct": 51,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.55,
+              0.725
+            ]
+          },
+          "constraint_assignment": {
+            "correct": 18,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.15,
+              0.3125
+            ]
+          },
+          "massive_en": {
+            "correct": 89,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.6583333333333333,
+              0.8166666666666667
+            ]
+          },
+          "massive_zh": {
+            "correct": 88,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.6583333333333332,
+              0.8083333333333333
+            ]
+          },
+          "multiset_reconciliation": {
+            "correct": 21,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.25,
+              0.2875
+            ]
+          },
+          "ordinal_service_loss": {
+            "correct": 16,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.15,
+              0.25
+            ]
+          },
+          "paraconsistent_rule_closure": {
+            "correct": 15,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.1375,
+              0.2375
+            ]
+          },
+          "temporal_exclusion": {
+            "correct": 10,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.05,
+              0.2125
+            ]
+          },
+          "transaction_recovery": {
+            "correct": 15,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.1,
+              0.2625
+            ]
+          }
+        },
+        "v4": {
+          "boolq": {
+            "correct": 111,
+            "requested": 160,
+            "accuracy_ci95": [
+              0.61875,
+              0.7625
+            ]
+          },
+          "belebele_en": {
+            "correct": 61,
+            "requested": 160,
+            "accuracy_ci95": [
+              0.3062370621019109,
+              0.4585987261146497
+            ]
+          },
+          "belebele_zh": {
+            "correct": 45,
+            "requested": 160,
+            "accuracy_ci95": [
+              0.2138364779874214,
+              0.3503184713375796
+            ]
+          }
+        },
+        "v5": {
+          "cosmos_qa": {
+            "correct": 36,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.21666666666666667,
+              0.38333333333333336
+            ]
+          },
+          "squad2_answerability": {
+            "correct": 69,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.5084745762711864,
+              0.6416666666666667
+            ]
+          },
+          "snli": {
+            "correct": 87,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.6416666666666667,
+              0.8
+            ]
+          },
+          "qasc": {
+            "correct": 114,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.9083333333333333,
+              0.983739837398374
+            ]
+          }
+        }
+      },
+      "proper_scores": {
+        "old_core": 0.5709760997718798,
+        "v3_core": 0.700479457442883,
+        "v4": 0.6149664146731222,
+        "v5": 0.4747850267787647,
+        "expanded_weighted_brier": 0.5903017496666624
+      },
+      "probability_coverage": {
+        "old_core": {
+          "requested": 880,
+          "valid_probability_rows": 880
+        },
+        "v3_core": {
+          "requested": 880,
+          "valid_probability_rows": 880
+        },
+        "v4": {
+          "requested": 480,
+          "valid_probability_rows": 480
+        },
+        "v5": {
+          "requested": 480,
+          "valid_probability_rows": 480
+        }
+      },
+      "brier_ci95": {
+        "old_core": [
+          0.5307603405543191,
+          0.611972788595099
+        ],
+        "v3_core": [
+          0.677830213791377,
+          0.7242813502662896
+        ],
+        "v4": [
+          0.5601239092953858,
+          0.6718388215324546
+        ],
+        "v5": [
+          0.4441678879307976,
+          0.5061559610657542
+        ],
+        "expanded_weighted_brier": [
+          0.5709748439741565,
+          0.6102902589236013
+        ]
+      }
+    },
+    "Qwen3.5 \u00b7 2B, untuned": {
+      "old_core": {
+        "point": 0.5712425595238095,
+        "exact_fraction": "3071/5376",
+        "ci95": [
+          0.535453869047619,
+          0.607627418154762
+        ]
+      },
+      "v3_core": {
+        "point": 0.39,
+        "exact_fraction": "39/100",
+        "ci95": [
+          0.3733333333333334,
+          0.40625
+        ]
+      },
+      "v4": {
+        "point": 0.7375,
+        "exact_fraction": "59/80",
+        "ci95": [
+          0.6928571428571428,
+          0.782292158018868
+        ]
+      },
+      "v5": {
+        "point": 0.7229166666666667,
+        "exact_fraction": "347/480",
+        "ci95": [
+          0.6848579953997695,
+          0.760121271215623
+        ]
+      },
+      "v3_joint": {
+        "point": 0.4806212797619048,
+        "exact_fraction": "129191/268800",
+        "ci95": [
+          0.46085509672619046,
+          0.5003615141369048
+        ]
+      },
+      "expanded_joint": {
+        "point": 0.605414806547619,
+        "exact_fraction": "325471/537600",
+        "ci95": [
+          0.5881936025503637,
+          0.6231347666599296
+        ]
+      },
+      "families": {
+        "old_core": {
+          "ag_news": {
+            "correct": 103,
+            "requested": 128,
+            "accuracy_ci95": [
+              0.734375,
+              0.8671875
+            ]
+          },
+          "boolean_constraints": {
+            "correct": 24,
+            "requested": 64,
+            "accuracy_ci95": [
+              0.21875,
+              0.53125
+            ]
+          },
+          "dbpedia_14": {
+            "correct": 104,
+            "requested": 112,
+            "accuracy_ci95": [
+              0.875,
+              0.9732142857142857
+            ]
+          },
           "natural_intents": {
-            "correct": 64,
+            "correct": 62,
             "requested": 64,
             "accuracy_ci95": [
-              1.0,
+              0.921875,
               1.0
             ]
           },
           "option_carrier": {
-            "correct": 70,
+            "correct": 9,
             "requested": 96,
             "accuracy_ci95": [
-              0.6354166666666666,
-              0.8229166666666666
+              0.041666666666666664,
+              0.15625
             ]
           },
           "ordinal_rubric": {
-            "correct": 58,
+            "correct": 52,
             "requested": 64,
             "accuracy_ci95": [
-              0.8125,
-              0.984375
+              0.671875,
+              0.9375
             ]
           },
           "relational_composition": {
-            "correct": 49,
+            "correct": 47,
             "requested": 96,
             "accuracy_ci95": [
-              0.3854166666666667,
-              0.6354166666666666
+              0.375,
+              0.6145833333333334
             ]
           },
           "scoped_evidence": {
-            "correct": 49,
+            "correct": 35,
             "requested": 96,
             "accuracy_ci95": [
-              0.3645833333333333,
-              0.65625
+              0.23958333333333334,
+              0.5
             ]
           },
           "state_tracking": {
-            "correct": 27,
+            "correct": 24,
             "requested": 96,
             "accuracy_ci95": [
-              0.16666666666666666,
-              0.3958333333333333
+              0.15625,
+              0.3541666666666667
             ]
           },
           "unknown_rejection": {
-            "correct": 39,
+            "correct": 40,
             "requested": 64,
             "accuracy_ci95": [
-              0.421875,
-              0.7816406249999943
+              0.4375,
+              0.8125
             ]
           }
         },
         "v3_core": {
           "canonical_record_identity": {
-            "correct": 40,
+            "correct": 37,
             "requested": 80,
             "accuracy_ci95": [
-              0.5,
+              0.425,
               0.5
             ]
           },
@@ -1442,137 +1765,137 @@
             ]
           },
           "constraint_assignment": {
-            "correct": 19,
+            "correct": 17,
             "requested": 80,
             "accuracy_ci95": [
-              0.1625,
-              0.3125
+              0.15,
+              0.275
             ]
           },
           "massive_en": {
-            "correct": 110,
+            "correct": 85,
             "requested": 120,
             "accuracy_ci95": [
-              0.8666666666666667,
-              0.9583333333333334
+              0.625,
+              0.7833333333333334
             ]
           },
           "massive_zh": {
-            "correct": 104,
+            "correct": 89,
             "requested": 120,
             "accuracy_ci95": [
-              0.8,
-              0.925
+              0.6666666666666666,
+              0.8166666666666667
             ]
           },
           "multiset_reconciliation": {
             "correct": 22,
             "requested": 80,
             "accuracy_ci95": [
-              0.225,
-              0.325
+              0.25,
+              0.3125
             ]
           },
           "ordinal_service_loss": {
-            "correct": 17,
+            "correct": 16,
             "requested": 80,
             "accuracy_ci95": [
               0.2,
-              0.2375
+              0.2
             ]
           },
           "paraconsistent_rule_closure": {
-            "correct": 20,
+            "correct": 22,
             "requested": 80,
             "accuracy_ci95": [
-              0.2125,
-              0.2875
+              0.25,
+              0.3125
             ]
           },
           "temporal_exclusion": {
-            "correct": 25,
+            "correct": 23,
             "requested": 80,
             "accuracy_ci95": [
-              0.25,
-              0.375
+              0.2375,
+              0.3375
             ]
           },
           "transaction_recovery": {
-            "correct": 21,
+            "correct": 19,
             "requested": 80,
             "accuracy_ci95": [
-              0.25,
-              0.2875
+              0.2125,
+              0.25
             ]
           }
         },
         "v4": {
           "boolq": {
-            "correct": 134,
+            "correct": 104,
             "requested": 160,
             "accuracy_ci95": [
-              0.78125,
-              0.89375
+              0.575,
+              0.725
             ]
           },
           "belebele_en": {
-            "correct": 149,
+            "correct": 134,
             "requested": 160,
             "accuracy_ci95": [
-              0.8910256410256411,
-              0.967948717948718
+              0.7791411042944786,
+              0.8924050632911392
             ]
           },
           "belebele_zh": {
-            "correct": 146,
+            "correct": 130,
             "requested": 160,
             "accuracy_ci95": [
-              0.8633475672877846,
-              0.95625
+              0.7484276729559748,
+              0.8742138364779874
             ]
           }
         },
         "v5": {
           "cosmos_qa": {
-            "correct": 73,
+            "correct": 68,
             "requested": 120,
             "accuracy_ci95": [
-              0.5166666666666667,
-              0.6916666666666667
+              0.48333333333333334,
+              0.65
             ]
           },
           "squad2_answerability": {
-            "correct": 98,
+            "correct": 99,
             "requested": 120,
             "accuracy_ci95": [
-              0.7540983606557377,
-              0.8790322580645161
+              0.7631578947368421,
+              0.8833333333333333
             ]
           },
           "snli": {
-            "correct": 96,
+            "correct": 77,
             "requested": 120,
             "accuracy_ci95": [
-              0.725,
-              0.8666666666666667
+              0.55,
+              0.725
             ]
           },
           "qasc": {
-            "correct": 116,
+            "correct": 103,
             "requested": 120,
             "accuracy_ci95": [
-              0.9333333333333333,
-              0.991869918699187
+              0.7950819672131147,
+              0.9173553719008265
             ]
           }
         }
       },
       "proper_scores": {
-        "old_core": 0.427127994572904,
-        "v3_core": 0.6845537392617043,
-        "v4": 0.19639894537679964,
-        "v5": 0.2904012063572707,
-        "expanded_weighted_brier": 0.39962047139216966
+        "old_core": 0.5723777597574953,
+        "v3_core": 0.7477724847216414,
+        "v4": 0.31678065323351606,
+        "v5": 0.4151003191775817,
+        "expanded_weighted_brier": 0.5130078042225586
       },
       "probability_coverage": {
         "old_core": {
@@ -1594,100 +1917,100 @@
       },
       "brier_ci95": {
         "old_core": [
-          0.377528048361006,
-          0.4790590681446513
+          0.5306199339702083,
+          0.613544919176592
         ],
         "v3_core": [
-          0.6674510320346829,
-          0.7036301143486854
+          0.7296525923390248,
+          0.7668117066670981
         ],
         "v4": [
-          0.1623582012987519,
-          0.23277732548629693
+          0.27446950339258713,
+          0.3607698891917115
         ],
         "v5": [
-          0.24711830906655044,
-          0.33490954796052225
+          0.37186021348012016,
+          0.45924139988994434
         ],
         "expanded_weighted_brier": [
-          0.38062758349672393,
-          0.4193732417566481
+          0.4934107964402564,
+          0.5319870117266131
         ]
       }
     },
-    "Sol-composition-selected": {
+    "Qwen3.5 \u00b7 4B, untuned": {
       "old_core": {
-        "point": 0.7374627976190476,
-        "exact_fraction": "19823/26880",
+        "point": 0.6988839285714286,
+        "exact_fraction": "3131/4480",
         "ci95": [
-          0.6992931547619048,
-          0.7742950148809524
+          0.6632803199404762,
+          0.7337797619047619
         ]
       },
       "v3_core": {
-        "point": 0.4608333333333333,
-        "exact_fraction": "553/1200",
+        "point": 0.43333333333333335,
+        "exact_fraction": "13/30",
         "ci95": [
-          0.43875000000000003,
-          0.48291666666666666
+          0.4179166666666667,
+          0.4479166666666667
         ]
       },
       "v4": {
-        "point": 0.765625,
-        "exact_fraction": "49/64",
+        "point": 0.8796875,
+        "exact_fraction": "563/640",
         "ci95": [
-          0.7210721817484662,
-          0.8086081288343558
+          0.8461463032298809,
+          0.9125000000000001
         ]
       },
       "v5": {
-        "point": 0.8416666666666667,
-        "exact_fraction": "101/120",
+        "point": 0.7979166666666667,
+        "exact_fraction": "383/480",
         "ci95": [
-          0.8111792524213076,
-          0.8710143196177259
+          0.7641944237372369,
+          0.830579286387442
         ]
       },
       "v3_joint": {
-        "point": 0.5991480654761905,
-        "exact_fraction": "161051/268800",
+        "point": 0.566108630952381,
+        "exact_fraction": "15217/26880",
         "ci95": [
-          0.5773286830357143,
-          0.6207081473214285
+          0.5467666480654761,
+          0.5849108072916667
         ]
       },
       "expanded_joint": {
-        "point": 0.7013969494047619,
-        "exact_fraction": "377071/537600",
+        "point": 0.7024553571428571,
+        "exact_fraction": "3147/4480",
         "ci95": [
-          0.6845734141379205,
-          0.7180071456587547
+          0.6873480253417569,
+          0.7176086887612844
         ]
       },
       "families": {
         "old_core": {
           "ag_news": {
-            "correct": 107,
+            "correct": 108,
             "requested": 128,
             "accuracy_ci95": [
-              0.7734375,
-              0.8984375
+              0.78125,
+              0.90625
             ]
           },
           "boolean_constraints": {
-            "correct": 32,
+            "correct": 40,
             "requested": 64,
             "accuracy_ci95": [
-              0.328125,
-              0.671875
+              0.484375,
+              0.765625
             ]
           },
           "dbpedia_14": {
-            "correct": 107,
+            "correct": 109,
             "requested": 112,
             "accuracy_ci95": [
-              0.9107142857142857,
-              0.9910714285714286
+              0.9375,
+              1.0
             ]
           },
           "natural_intents": {
@@ -1699,51 +2022,51 @@
             ]
           },
           "option_carrier": {
-            "correct": 95,
+            "correct": 70,
             "requested": 96,
             "accuracy_ci95": [
-              0.96875,
-              1.0
+              0.6354166666666666,
+              0.8229166666666666
             ]
           },
           "ordinal_rubric": {
-            "correct": 42,
+            "correct": 58,
             "requested": 64,
             "accuracy_ci95": [
-              0.484375,
-              0.8125
+              0.8125,
+              0.984375
             ]
           },
           "relational_composition": {
-            "correct": 36,
+            "correct": 49,
             "requested": 96,
             "accuracy_ci95": [
-              0.19791666666666666,
-              0.5625
+              0.3854166666666667,
+              0.6354166666666666
             ]
           },
           "scoped_evidence": {
-            "correct": 76,
+            "correct": 49,
             "requested": 96,
             "accuracy_ci95": [
-              0.6354166666666666,
-              0.9375
+              0.3645833333333333,
+              0.65625
             ]
           },
           "state_tracking": {
-            "correct": 26,
+            "correct": 27,
             "requested": 96,
             "accuracy_ci95": [
-              0.125,
-              0.4270833333333333
+              0.16666666666666666,
+              0.3958333333333333
             ]
           },
           "unknown_rejection": {
-            "correct": 64,
+            "correct": 39,
             "requested": 64,
             "accuracy_ci95": [
-              1.0,
-              1.0
+              0.421875,
+              0.7816406249999943
             ]
           }
         },
@@ -1757,145 +2080,145 @@
             ]
           },
           "capacitated_assignment": {
-            "correct": 38,
+            "correct": 40,
             "requested": 80,
             "accuracy_ci95": [
-              0.4375,
+              0.5,
               0.5
             ]
           },
           "constraint_assignment": {
-            "correct": 21,
+            "correct": 19,
             "requested": 80,
             "accuracy_ci95": [
-              0.1875,
+              0.1625,
               0.3125
             ]
           },
           "massive_en": {
-            "correct": 109,
+            "correct": 110,
             "requested": 120,
             "accuracy_ci95": [
-              0.8500000000000001,
-              0.9583333333333333
+              0.8666666666666667,
+              0.9583333333333334
             ]
           },
           "massive_zh": {
-            "correct": 105,
+            "correct": 104,
             "requested": 120,
             "accuracy_ci95": [
-              0.8166666666666667,
-              0.9333333333333332
+              0.8,
+              0.925
             ]
           },
           "multiset_reconciliation": {
-            "correct": 23,
+            "correct": 22,
             "requested": 80,
             "accuracy_ci95": [
-              0.25,
+              0.225,
               0.325
             ]
           },
           "ordinal_service_loss": {
-            "correct": 20,
+            "correct": 17,
             "requested": 80,
             "accuracy_ci95": [
-              0.1875,
-              0.3125
+              0.2,
+              0.2375
             ]
           },
           "paraconsistent_rule_closure": {
             "correct": 20,
             "requested": 80,
             "accuracy_ci95": [
-              0.25,
-              0.25
+              0.2125,
+              0.2875
             ]
           },
           "temporal_exclusion": {
-            "correct": 33,
+            "correct": 25,
             "requested": 80,
             "accuracy_ci95": [
-              0.275,
-              0.5625
+              0.25,
+              0.375
             ]
           },
           "transaction_recovery": {
-            "correct": 31,
+            "correct": 21,
             "requested": 80,
             "accuracy_ci95": [
-              0.3,
-              0.475
+              0.25,
+              0.2875
             ]
           }
         },
         "v4": {
           "boolq": {
-            "correct": 133,
+            "correct": 134,
             "requested": 160,
             "accuracy_ci95": [
-              0.775,
-              0.8875
+              0.78125,
+              0.89375
             ]
           },
           "belebele_en": {
-            "correct": 112,
+            "correct": 149,
             "requested": 160,
             "accuracy_ci95": [
-              0.6280452353942144,
-              0.7692307692307693
+              0.8910256410256411,
+              0.967948717948718
             ]
           },
           "belebele_zh": {
-            "correct": 112,
+            "correct": 146,
             "requested": 160,
             "accuracy_ci95": [
-              0.6289261100440628,
-              0.7682926829268293
+              0.8633475672877846,
+              0.95625
             ]
           }
         },
         "v5": {
           "cosmos_qa": {
-            "correct": 83,
+            "correct": 73,
             "requested": 120,
             "accuracy_ci95": [
-              0.6083333333333333,
-              0.775
+              0.5166666666666667,
+              0.6916666666666667
             ]
           },
           "squad2_answerability": {
-            "correct": 100,
+            "correct": 98,
             "requested": 120,
             "accuracy_ci95": [
-              0.7786885245901639,
-              0.8859649122807017
+              0.7540983606557377,
+              0.8790322580645161
             ]
           },
           "snli": {
-            "correct": 106,
+            "correct": 96,
             "requested": 120,
             "accuracy_ci95": [
-              0.825,
-              0.9416666666666667
+              0.725,
+              0.8666666666666667
             ]
           },
           "qasc": {
-            "correct": 115,
+            "correct": 116,
             "requested": 120,
             "accuracy_ci95": [
-              0.912,
-              0.9918032786885246
+              0.9333333333333333,
+              0.991869918699187
             ]
           }
         }
       },
       "proper_scores": {
-        "old_core": 0.42193820645312435,
-        "v3_core": 0.6913074500248156,
-        "v4": 0.33935305266273397,
-        "v5": 0.250760098789168,
-        "expanded_weighted_brier": 0.42583970198246046
+        "old_core": 0.427127994572904,
+        "v3_core": 0.6845537392617043,
+        "v4": 0.19639894537679964,
+        "v5": 0.2904012063572707,
+        "expanded_weighted_brier": 0.39962047139216966
       },
       "probability_coverage": {
         "old_core": {
@@ -1917,99 +2240,99 @@
       },
       "brier_ci95": {
         "old_core": [
-          0.365622760488921,
-          0.48051917201614724
+          0.377528048361006,
+          0.4790590681446513
         ],
         "v3_core": [
-          0.6695390244240492,
-          0.7144598051610135
+          0.6674510320346829,
+          0.7036301143486854
         ],
         "v4": [
-          0.28386912550452903,
-          0.3966449902668765
+          0.1623582012987519,
+          0.23277732548629693
         ],
         "v5": [
-          0.21482273267248006,
-          0.28856469231607446
+          0.24711830906655044,
+          0.33490954796052225
         ],
         "expanded_weighted_brier": [
-          0.40383487736748497,
-          0.44863844813421033
+          0.38062758349672393,
+          0.4193732417566481
         ]
       }
     },
-    "Sol-v1.2-released": {
+    "Jev \u00b7 1.13.0": {
       "old_core": {
-        "point": 0.7364211309523809,
-        "exact_fraction": "3959/5376",
+        "point": 0.7909598214285715,
+        "exact_fraction": "7087/8960",
         "ci95": [
-          0.6978794642857142,
-          0.7738848586309524
+          0.7616062127976191,
+          0.819271763392857
         ]
       },
       "v3_core": {
-        "point": 0.4691666666666667,
-        "exact_fraction": "563/1200",
+        "point": 0.66375,
+        "exact_fraction": "531/800",
         "ci95": [
-          0.4466666666666666,
-          0.4929166666666667
+          0.6316666666666667,
+          0.6945833333333333
         ]
       },
       "v4": {
-        "point": 0.753125,
-        "exact_fraction": "241/320",
+        "point": 0.9453125,
+        "exact_fraction": "121/128",
         "ci95": [
-          0.7076214657449122,
-          0.796224597392638
+          0.9197115384615384,
+          0.9685135514495712
         ]
       },
       "v5": {
-        "point": 0.8333333333333334,
-        "exact_fraction": "5/6",
+        "point": 0.8979166666666667,
+        "exact_fraction": "431/480",
         "ci95": [
-          0.8020406719533801,
-          0.8636711401441725
+          0.8713251366120218,
+          0.9229854264081012
         ]
       },
       "v3_joint": {
-        "point": 0.6027938988095238,
-        "exact_fraction": "162031/268800",
+        "point": 0.7273549107142857,
+        "exact_fraction": "65171/89600",
         "ci95": [
-          0.5805984933035714,
-          0.6250373883928573
+          0.7058627232142857,
+          0.7482447916666666
         ]
       },
       "expanded_joint": {
-        "point": 0.6980115327380952,
-        "exact_fraction": "375251/537600",
+        "point": 0.8244847470238095,
+        "exact_fraction": "443243/537600",
         "ci95": [
-          0.6806066766116663,
-          0.7149157919111048
+          0.8104711862785249,
+          0.8378880015726611
         ]
       },
       "families": {
         "old_core": {
           "ag_news": {
-            "correct": 107,
+            "correct": 109,
             "requested": 128,
             "accuracy_ci95": [
-              0.7734375,
-              0.8984375
+              0.7890625,
+              0.90625
             ]
           },
           "boolean_constraints": {
-            "correct": 32,
+            "correct": 64,
             "requested": 64,
             "accuracy_ci95": [
-              0.328125,
-              0.671875
+              1.0,
+              1.0
             ]
           },
           "dbpedia_14": {
-            "correct": 107,
+            "correct": 108,
             "requested": 112,
             "accuracy_ci95": [
-              0.9107142857142857,
+              0.9285714285714286,
               0.9910714285714286
             ]
           },
@@ -2022,43 +2345,43 @@
             ]
           },
           "option_carrier": {
-            "correct": 95,
+            "correct": 29,
             "requested": 96,
             "accuracy_ci95": [
-              0.96875,
-              1.0
+              0.2604166666666667,
+              0.34375
             ]
           },
           "ordinal_rubric": {
-            "correct": 40,
+            "correct": 64,
             "requested": 64,
             "accuracy_ci95": [
-              0.453125,
-              0.78125
+              1.0,
+              1.0
             ]
           },
           "relational_composition": {
-            "correct": 36,
+            "correct": 54,
             "requested": 96,
             "accuracy_ci95": [
-              0.19791666666666666,
-              0.5625
+              0.3645833333333333,
+              0.75
             ]
           },
           "scoped_evidence": {
-            "correct": 76,
+            "correct": 86,
             "requested": 96,
             "accuracy_ci95": [
-              0.6354166666666666,
-              0.9375
+              0.78125,
+              0.9895833333333334
             ]
           },
           "state_tracking": {
-            "correct": 28,
+            "correct": 32,
             "requested": 96,
             "accuracy_ci95": [
-              0.13541666666666666,
-              0.4583333333333333
+              0.16640625000000023,
+              0.5106770833333295
             ]
           },
           "unknown_rejection": {
@@ -2072,35 +2395,35 @@
         },
         "v3_core": {
           "canonical_record_identity": {
-            "correct": 40,
+            "correct": 61,
             "requested": 80,
             "accuracy_ci95": [
-              0.5,
-              0.5
+              0.6375,
+              0.875
             ]
           },
           "capacitated_assignment": {
-            "correct": 38,
+            "correct": 55,
             "requested": 80,
             "accuracy_ci95": [
-              0.4375,
-              0.5
+              0.575,
+              0.7875
             ]
           },
           "constraint_assignment": {
-            "correct": 21,
+            "correct": 44,
             "requested": 80,
             "accuracy_ci95": [
-              0.1875,
-              0.3375
+              0.4,
+              0.675
             ]
           },
           "massive_en": {
-            "correct": 109,
+            "correct": 110,
             "requested": 120,
             "accuracy_ci95": [
-              0.8500000000000001,
-              0.9583333333333333
+              0.8666666666666667,
+              0.9583333333333334
             ]
           },
           "massive_zh": {
@@ -2108,117 +2431,117 @@
             "requested": 120,
             "accuracy_ci95": [
               0.825,
-              0.9333333333333333
+              0.9416666666666667
             ]
           },
           "multiset_reconciliation": {
-            "correct": 24,
+            "correct": 47,
             "requested": 80,
             "accuracy_ci95": [
-              0.25,
-              0.35
+              0.5125,
+              0.65
             ]
           },
           "ordinal_service_loss": {
-            "correct": 23,
+            "correct": 34,
             "requested": 80,
             "accuracy_ci95": [
-              0.2375,
-              0.3375
+              0.325,
+              0.525
             ]
           },
           "paraconsistent_rule_closure": {
-            "correct": 20,
+            "correct": 53,
             "requested": 80,
             "accuracy_ci95": [
-              0.25,
-              0.25
+              0.5875,
+              0.725
             ]
           },
           "temporal_exclusion": {
-            "correct": 34,
+            "correct": 33,
             "requested": 80,
             "accuracy_ci95": [
-              0.275,
-              0.5875
+              0.3125,
+              0.5125
             ]
           },
           "transaction_recovery": {
-            "correct": 32,
+            "correct": 60,
             "requested": 80,
             "accuracy_ci95": [
-              0.3125,
-              0.5
+              0.625,
+              0.8625
             ]
           }
         },
         "v4": {
           "boolq": {
-            "correct": 130,
+            "correct": 148,
             "requested": 160,
             "accuracy_ci95": [
-              0.75,
-              0.86875
+              0.88125,
+              0.9625
             ]
           },
           "belebele_en": {
-            "correct": 112,
+            "correct": 155,
             "requested": 160,
             "accuracy_ci95": [
-              0.6280452353942144,
-              0.76875
+              0.9390243902439024,
+              0.9937106918238994
             ]
           },
           "belebele_zh": {
-            "correct": 110,
+            "correct": 154,
             "requested": 160,
             "accuracy_ci95": [
-              0.6125,
-              0.759493670886076
+              0.930379746835443,
+              0.9877300613496932
             ]
           }
         },
         "v5": {
           "cosmos_qa": {
-            "correct": 80,
+            "correct": 104,
             "requested": 120,
             "accuracy_ci95": [
-              0.5833333333333334,
-              0.75
+              0.8,
+              0.925
             ]
           },
           "squad2_answerability": {
-            "correct": 99,
+            "correct": 109,
             "requested": 120,
             "accuracy_ci95": [
-              0.7666532258064517,
-              0.8813559322033898
+              0.8620689655172413,
+              0.9508196721311475
             ]
           },
           "snli": {
-            "correct": 105,
+            "correct": 99,
             "requested": 120,
             "accuracy_ci95": [
-              0.8166666666666667,
-              0.9333333333333333
+              0.75,
+              0.8916666666666667
             ]
           },
           "qasc": {
-            "correct": 116,
+            "correct": 119,
             "requested": 120,
             "accuracy_ci95": [
-              0.9243697478991597,
+              0.9745762711864406,
               1.0
             ]
           }
         }
       },
       "proper_scores": {
-        "old_core": 0.411074134119444,
-        "v3_core": 0.6843092162356459,
-        "v4": 0.35107866262032555,
-        "v5": 0.2658598466375241,
-        "expanded_weighted_brier": 0.42808046490323487
+        "old_core": 0.24806216931976088,
+        "v3_core": 0.4403896043771044,
+        "v4": 0.07270531250000001,
+        "v5": 0.15378333333333333,
+        "expanded_weighted_brier": 0.22873510488254967
       },
       "probability_coverage": {
         "old_core": {
@@ -2240,119 +2563,28 @@
       },
       "brier_ci95": {
         "old_core": [
-          0.35500481047576726,
-          0.4694446340818619
+          0.21912044641352113,
+          0.27809132807795134
         ],
         "v3_core": [
-          0.6631373371753402,
-          0.7066243522110535
+          0.4130360873769258,
+          0.4688325815363228
         ],
         "v4": [
-          0.2947080991408087,
-          0.41015228438675966
+          0.04802243933725408,
+          0.100621887515886
         ],
         "v5": [
-          0.22948466896481354,
-          0.3040059590019518
+          0.12116203971412033,
+          0.18851406142759564
         ],
         "expanded_weighted_brier": [
-          0.40571771019565184,
-          0.451117185111431
+          0.21435641903529326,
+          0.2433957477840872
         ]
       }
     }
   },
-  "paired_comparisons": {
-    "Sol-composition-selected minus Sol-v1.2-released": {
-      "panels": {
-        "old_core": {
-          "delta": 0.0010416666666666667,
-          "exact_fraction": "1/960",
-          "paired_ci95": [
-            -0.006249999999999978,
-            0.009375000000000022
-          ]
-        },
-        "v3_core": {
-          "delta": -0.008333333333333333,
-          "exact_fraction": "-1/120",
-          "paired_ci95": [
-            -0.020416666666666583,
-            0.0029166666666667895
-          ]
-        },
-        "v4": {
-          "delta": 0.0125,
-          "exact_fraction": "1/80",
-          "paired_ci95": [
-            -0.004738055042751673,
-            0.03141072850318471
-          ]
-        },
-        "v5": {
-          "delta": 0.008333333333333333,
-          "exact_fraction": "1/120",
-          "paired_ci95": [
-            -0.006285021423823853,
-            0.023276051119177216
-          ]
-        },
-        "v3_joint": {
-          "delta": -0.0036458333333333334,
-          "exact_fraction": "-7/1920",
-          "paired_ci95": [
-            -0.010729166666666567,
-            0.0034374999999999822
-          ]
-        },
-        "expanded_joint": {
-          "delta": 0.0033854166666666668,
-          "exact_fraction": "13/3840",
-          "paired_ci95": [
-            -0.0033255459389013777,
-            0.010233014577181404
-          ]
-        }
-      },
-      "brier": {
-        "old_core": {
-          "delta": 0.010864072333680364,
-          "paired_ci95": [
-            0.0029110032097614397,
-            0.01951073578118077
-          ]
-        },
-        "v3_core": {
-          "delta": 0.00699823378916975,
-          "paired_ci95": [
-            0.0033694272617522253,
-            0.010844522902194289
-          ]
-        },
-        "v4": {
-          "delta": -0.011725609957591587,
-          "paired_ci95": [
-            -0.020557947390196268,
-            -0.003641365566461738
-          ]
-        },
-        "v5": {
-          "delta": -0.015099747848356104,
-          "paired_ci95": [
-            -0.02269500622256997,
-            -0.007432007685509654
-          ]
-        },
-        "expanded_weighted_brier": {
-          "delta": -0.0022407629207744084,
-          "paired_ci95": [
-            -0.0059394173100002804,
-            0.001420806882718375
-          ]
-        }
-      }
-    }
-  },
   "method": {
     "seed": 202609225801,
     "repeats": 10000,

metrics/figure-evidence.json

@@ -1,33 +1,44 @@
 {
   "format": "decision-expanded-figure-evidence-v1",
   "kind": "measured",
-  "manifest_sha256": "a881795f5154df5510c3263272adb6befe849eba033f76ecab60f4057078269c",
-  "statistics_sha256": "fe2be9746c4302b829b5aa373c027910cfa84881b80c2068af7c69db76c4659c",
-  "public_projection_sha256": "3824d5e5f01d3ff3cc32cc32b85df727c57dd6bc859c024f60f3b8a4b8441917",
+  "manifest_sha256": "182dfbad245c160bddd7ad6627d076e8a3dd9c7117bac9c6da2c2e8240163fbc",
+  "statistics_sha256": "7b8936b43bce9d19d9010ab61225a6460160b3aa76c37f5d6df5ba650b433c6c",
+  "public_projection_sha256": "69477b6a3f6693d8af367f2ecbcc14db3a485871c0db58ee9bb07d752ed4f971",
   "base_style_sha256": "93fa22aa96a6dd5f06fe9a5670d84691f2092f84a91bac600c9777a29a36123a",
   "branding_sha256": "d83cf7878c3839f3085feb8c02334217ad2fef0a615b805e1726b292552c46e8",
   "source_sha256": {
-    "render.py": "f6ecc60bfa616eb41c422bc30aaebd9f30a123318991bbedb56a9f9920dfa3c0",
-    "contract.py": "55256a32b7759d993b5aff10a39d185b3ad67191495f840f9ef882a2001ff325"
+    "render.py": "ce19f3a2e3c9ca38b79373ebcec9ad4fcbde05d6acb6089cfb3817e1057985c2",
+    "contract.py": "dbddb4094f8c412d52aa4e195c59a669e7b9c084f6c5b9105f1114e6d12ef445"
   },
   "display_order": [
-    "Jev \u00b7 1.13.0",
+    "Sol-composition-selected",
+    "Nox-retention-selected",
+    "kev-9b",
     "Decider \u00b7 2B",
     "Qwen3.5 \u00b7 4B, untuned",
-    "Sol-composition-selected",
-    "Sol-v1.2-released",
     "Qwen3.5 \u00b7 2B, untuned",
-    "Laya \u00b7 EN/ML"
+    "Laya \u00b7 EN/ML",
+    "Jev \u00b7 1.13.0"
   ],
   "display_labels": {
+    "Nox-retention-selected": {
+      "label": "Nox",
+      "detail": "4B \u00b7 v1.3",
+      "color": "#965837"
+    },
+    "Sol-composition-selected": {
+      "label": "Sol",
+      "detail": "2B \u00b7 v1.3",
+      "color": "#B07B51"
+    },
+    "kev-9b": {
+      "label": "Kev",
+      "detail": "9B"
+    },
     "Decider \u00b7 2B": {
       "label": "Decider",
       "detail": "2B"
     },
-    "Jev \u00b7 1.13.0": {
-      "label": "Jev",
-      "detail": "1.13.0"
-    },
     "Laya \u00b7 EN/ML": {
       "label": "Laya",
       "detail": "Upstream default"
@@ -40,212 +51,34 @@
       "label": "Qwen3.5",
       "detail": "4B \u00b7 untuned"
     },
-    "Sol-composition-selected": {
-      "label": "Sol",
-      "detail": "v1.3 \u00b7 this release"
-    },
-    "Sol-v1.2-released": {
-      "label": "Sol",
-      "detail": "v1.2 \u00b7 previous"
+    "Jev \u00b7 1.13.0": {
+      "label": "Jev",
+      "detail": "1.13.0 \u00b7 frontier",
+      "color": "#879095"
     }
   },
   "model_versions": {
-    "Jev \u00b7 1.13.0": {
-      "old_core": [
-        "jev-1.13.0"
-      ],
-      "v3_core": [
-        "jev-1.13.0"
-      ],
-      "v4": [
-        "jev-1.13.0"
-      ],
-      "v5": [
-        "jev-1.13.0"
-      ]
-    },
-    "Laya \u00b7 EN/ML": {
-      "old_core": {
-        "repo": "convaiinnovations/laya",
-        "revision": "1c5edc17a7acd8701df6fc341c0d179f1c62c982",
-        "router_source_revision": "42626c348753fbb17572a813127df2278a1ec527",
-        "adapter_sha256": "a5484e30d9fad04ca578265e5f18df2a10731f5055bef070311102c3b35643fc",
-        "mode": "routed",
-        "weight_fingerprints": {
-          "english": "5bf5d0f9462d860e4c09000fce3131a070578d3574777ae7981077f11cd68185",
-          "multilingual": "a472bfcc4aa52a25dda0b2864b3e7c7c1267ddd1752b04a1e70bf6ba7e4b514c"
-        },
-        "routing": "Unmodified upstream Router.route(state,questions), no explicit lang, no automatic workflow detection, default english"
-      },
-      "v3_core": {
-        "repo": "convaiinnovations/laya",
-        "revision": "1c5edc17a7acd8701df6fc341c0d179f1c62c982",
-        "router_source_revision": "42626c348753fbb17572a813127df2278a1ec527",
-        "adapter_sha256": "a5484e30d9fad04ca578265e5f18df2a10731f5055bef070311102c3b35643fc",
-        "mode": "routed",
-        "weight_fingerprints": {
-          "english": "5bf5d0f9462d860e4c09000fce3131a070578d3574777ae7981077f11cd68185",
-          "multilingual": "a472bfcc4aa52a25dda0b2864b3e7c7c1267ddd1752b04a1e70bf6ba7e4b514c"
-        },
-        "routing": "Unmodified upstream Router.route(state,questions), no explicit lang, no automatic workflow detection, default english"
-      },
-      "v4": {
-        "repo": "convaiinnovations/laya",
-        "revision": "1c5edc17a7acd8701df6fc341c0d179f1c62c982",
-        "router_source_revision": "42626c348753fbb17572a813127df2278a1ec527",
-        "adapter_sha256": "a5484e30d9fad04ca578265e5f18df2a10731f5055bef070311102c3b35643fc",
-        "mode": "routed",
-        "weight_fingerprints": {
-          "english": "5bf5d0f9462d860e4c09000fce3131a070578d3574777ae7981077f11cd68185",
-          "multilingual": "a472bfcc4aa52a25dda0b2864b3e7c7c1267ddd1752b04a1e70bf6ba7e4b514c"
-        },
-        "routing": "Unmodified upstream Router.route(state,questions), no explicit lang, no automatic workflow detection, default english"
-      },
-      "v5": {
-        "repo": "convaiinnovations/laya",
-        "revision": "1c5edc17a7acd8701df6fc341c0d179f1c62c982",
-        "router_source_revision": "42626c348753fbb17572a813127df2278a1ec527",
-        "adapter_sha256": "a5484e30d9fad04ca578265e5f18df2a10731f5055bef070311102c3b35643fc",
-        "mode": "routed",
-        "weight_fingerprints": {
-          "english": "5bf5d0f9462d860e4c09000fce3131a070578d3574777ae7981077f11cd68185",
-          "multilingual": "a472bfcc4aa52a25dda0b2864b3e7c7c1267ddd1752b04a1e70bf6ba7e4b514c"
-        },
-        "routing": "Unmodified upstream Router.route(state,questions), no explicit lang, no automatic workflow detection, default english"
-      }
-    },
-    "Decider \u00b7 2B": {
-      "old_core": {
-        "model_id": "Mapika/decider-2b",
-        "original_native_adapter_sha256": "983a2a0d582a1c5e7e58fe3fe4e7c1cd254dd0fe52f65449c68549108d0af387",
-        "wrapper_sha256": "95b9fcc132d1ae2f0818ea8bb4720f4b5342450e741fefb4315827225652e7f6",
-        "model_fingerprint": "35f7335c92cf1766c11f52fb4bcdb9c3894d3edad7408a00ae7b59c104b2d318",
-        "temperature": 1.3,
-        "layout": "state_first",
-        "independent": true,
-        "revision": "b37f7e1ba3fbc9238004cf531fabbee2619973fd"
-      },
-      "v3_core": {
-        "model_id": "Mapika/decider-2b",
-        "original_native_adapter_sha256": "983a2a0d582a1c5e7e58fe3fe4e7c1cd254dd0fe52f65449c68549108d0af387",
-        "wrapper_sha256": "95b9fcc132d1ae2f0818ea8bb4720f4b5342450e741fefb4315827225652e7f6",
-        "model_fingerprint": "35f7335c92cf1766c11f52fb4bcdb9c3894d3edad7408a00ae7b59c104b2d318",
-        "temperature": 1.3,
-        "layout": "state_first",
-        "independent": true,
-        "revision": "b37f7e1ba3fbc9238004cf531fabbee2619973fd"
-      },
-      "v4": {
-        "model_id": "Mapika/decider-2b",
-        "original_native_adapter_sha256": "983a2a0d582a1c5e7e58fe3fe4e7c1cd254dd0fe52f65449c68549108d0af387",
-        "wrapper_sha256": "95b9fcc132d1ae2f0818ea8bb4720f4b5342450e741fefb4315827225652e7f6",
-        "model_fingerprint": "35f7335c92cf1766c11f52fb4bcdb9c3894d3edad7408a00ae7b59c104b2d318",
-        "temperature": 1.3,
-        "layout": "state_first",
-        "independent": true,
-        "revision": "b37f7e1ba3fbc9238004cf531fabbee2619973fd"
-      },
-      "v5": {
-        "model_id": "Mapika/decider-2b",
-        "original_native_adapter_sha256": "983a2a0d582a1c5e7e58fe3fe4e7c1cd254dd0fe52f65449c68549108d0af387",
-        "wrapper_sha256": "95b9fcc132d1ae2f0818ea8bb4720f4b5342450e741fefb4315827225652e7f6",
-        "model_fingerprint": "35f7335c92cf1766c11f52fb4bcdb9c3894d3edad7408a00ae7b59c104b2d318",
-        "temperature": 1.3,
-        "layout": "state_first",
-        "independent": true,
-        "revision": "b37f7e1ba3fbc9238004cf531fabbee2619973fd"
-      }
-    },
-    "Qwen3.5 \u00b7 2B, untuned": {
-      "old_core": {
-        "base_repo": "Qwen/Qwen3.5-2B",
-        "base_revision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
-        "adapter_sha256": "e08336ecd148e9240792dc3d915536fc52998364f248f9f2016b8b83e289b989",
-        "prompt_version": "decision-letter-v1",
-        "temperature": 1.0,
-        "readout": "Frozen pretrained next-token LM logits restricted to A..Z; conditional softmax over listed candidates",
-        "new_trainable_head": false,
-        "task_finetuning": false
-      },
-      "v3_core": {
-        "base_repo": "Qwen/Qwen3.5-2B",
-        "base_revision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
-        "adapter_sha256": "e08336ecd148e9240792dc3d915536fc52998364f248f9f2016b8b83e289b989",
-        "prompt_version": "decision-letter-v1",
-        "temperature": 1.0,
-        "readout": "Frozen pretrained next-token LM logits restricted to A..Z; conditional softmax over listed candidates",
-        "new_trainable_head": false,
-        "task_finetuning": false
-      },
-      "v4": {
-        "base_repo": "Qwen/Qwen3.5-2B",
-        "base_revision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
-        "adapter_sha256": "e08336ecd148e9240792dc3d915536fc52998364f248f9f2016b8b83e289b989",
-        "prompt_version": "decision-letter-v1",
-        "temperature": 1.0,
-        "readout": "Frozen pretrained next-token LM logits restricted to A..Z; conditional softmax over listed candidates",
-        "new_trainable_head": false,
-        "task_finetuning": false
-      },
-      "v5": {
-        "base_repo": "Qwen/Qwen3.5-2B",
-        "base_revision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
-        "adapter_sha256": "e08336ecd148e9240792dc3d915536fc52998364f248f9f2016b8b83e289b989",
-        "prompt_version": "decision-letter-v1",
-        "temperature": 1.0,
-        "readout": "Frozen pretrained next-token LM logits restricted to A..Z; conditional softmax over listed candidates",
-        "new_trainable_head": false,
-        "task_finetuning": false
-      }
-    },
-    "Qwen3.5 \u00b7 4B, untuned": {
-      "old_core": {
-        "base_repo": "Qwen/Qwen3.5-4B",
-        "base_revision": "851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a",
-        "adapter_sha256": "e08336ecd148e9240792dc3d915536fc52998364f248f9f2016b8b83e289b989",
-        "prompt_version": "decision-letter-v1",
-        "temperature": 1.0,
-        "readout": "Frozen pretrained next-token LM logits restricted to A..Z; conditional softmax over listed candidates",
-        "new_trainable_head": false,
-        "task_finetuning": false
-      },
-      "v3_core": {
-        "base_repo": "Qwen/Qwen3.5-4B",
-        "base_revision": "851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a",
-        "adapter_sha256": "e08336ecd148e9240792dc3d915536fc52998364f248f9f2016b8b83e289b989",
-        "prompt_version": "decision-letter-v1",
-        "temperature": 1.0,
-        "readout": "Frozen pretrained next-token LM logits restricted to A..Z; conditional softmax over listed candidates",
-        "new_trainable_head": false,
-        "task_finetuning": false
-      },
-      "v4": {
-        "base_repo": "Qwen/Qwen3.5-4B",
-        "base_revision": "851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a",
-        "adapter_sha256": "e08336ecd148e9240792dc3d915536fc52998364f248f9f2016b8b83e289b989",
-        "prompt_version": "decision-letter-v1",
-        "temperature": 1.0,
-        "readout": "Frozen pretrained next-token LM logits restricted to A..Z; conditional softmax over listed candidates",
-        "new_trainable_head": false,
-        "task_finetuning": false
-      },
-      "v5": {
-        "base_repo": "Qwen/Qwen3.5-4B",
-        "base_revision": "851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a",
-        "adapter_sha256": "e08336ecd148e9240792dc3d915536fc52998364f248f9f2016b8b83e289b989",
-        "prompt_version": "decision-letter-v1",
-        "temperature": 1.0,
-        "readout": "Frozen pretrained next-token LM logits restricted to A..Z; conditional softmax over listed candidates",
-        "new_trainable_head": false,
-        "task_finetuning": false
-      }
+    "Nox-retention-selected": {
+      "repo_id": "llm-semantic-router/Decision-1.0-Nox",
+      "revision": "ad089ad3a5dc9a7a21e6d96db546bb53e2212654",
+      "release_tag": "v1.3",
+      "bundle_manifest_sha256": "92d7f5be5e1ef21ee682f574b35cc01de4dd8ab16b8aa6edf0dfbdfcfcfabba3",
+      "publication_receipt_sha256": "fe15d9e49608c6d2d89301272a993e7d17cffe0e5b0966c32e9bb2740f3ad1e6",
+      "statistics_sha256": "0eb70b3dbc8a5ab8d78279beadd3ab9796623dfe64218dae06ca5edb2795f46d"
     },
     "Sol-composition-selected": {
-      "bundle_manifest_sha256": "6f0970bfbe5594feecffc14b4824a92080ef09318e12567bd22264c3844e2fc4"
-    },
-    "Sol-v1.2-released": {
       "repo_id": "llm-semantic-router/Decision-1.0-Sol",
-      "revision": "7c7b9e97a2885929719886238370130e9dcec48b"
+      "revision": "2412d9470d3aa125b346262dad80f6161847aa09",
+      "release_tag": "v1.3",
+      "bundle_manifest_sha256": "6f0970bfbe5594feecffc14b4824a92080ef09318e12567bd22264c3844e2fc4",
+      "publication_receipt_sha256": "7316ce19ffc5bbdd440316f7899dc33a8eecb9c36ebdd0cd9f9f8b49f48a446f",
+      "statistics_sha256": "fe2be9746c4302b829b5aa373c027910cfa84881b80c2068af7c69db76c4659c"
+    },
+    "kev-9b": {
+      "repo_id": "jaredpalmer/kev-9b",
+      "revision": "6281032426a9ca3a08374d137bf9dfd8afd78e8a",
+      "source_revision": "e0bcf50153f1bda4ca6a8be5e12cbd5f9ebbce1c",
+      "weight_fingerprint": "2b29b6a7e43e94b467a0d516e5d17008cba0f5c6e36c2793c57140604accc4a9"
     }
   },
   "weights": {
@@ -293,7 +126,9 @@
   },
   "families": 27,
   "quality_decisions_per_model": 2720,
-  "exact_bold_ties": true,
+  "strict_external_reference_bolding": true,
+  "ties_not_bold": true,
+  "frontier_not_bold": true,
   "no_raw_examples_or_predictions_read": true,
   "background": "white",
   "footnotes_in_image": false,
@@ -301,128 +136,143 @@
   "files": [
     {
       "file": "TABLES.md",
-      "bytes": 4677,
-      "sha256": "ad857dc95fa059b7ef0d3ca8c90d729d0b46beeb4e25bf6f4cd0429a5ee9fac1"
+      "bytes": 5093,
+      "sha256": "2f078e7339457e04a8832c068a9cf9b67aaff39b147794006a17ddd1a72bf41a"
     },
     {
       "file": "decision-expanded-old_core-600px.png",
-      "bytes": 139324,
-      "sha256": "522f361db1d2e747950a5d5b82f0f640f1ccc60af45509b919de031c0268eaa0"
+      "bytes": 167493,
+      "sha256": "6d36e595bea7465903ee787d217ef07ad648472d336655e88e51263325d0a7e2"
     },
     {
       "file": "decision-expanded-old_core.pdf",
-      "bytes": 44058,
-      "sha256": "deb028656a8fc299e75921e7df7c241e71c21bcea70774d517e62bbfc7c1011c"
+      "bytes": 44384,
+      "sha256": "cbd80145027dd79b37ed799a64b4c173d44973b358bc05414da80355ba7072a2"
     },
     {
       "file": "decision-expanded-old_core.png",
-      "bytes": 261230,
-      "sha256": "8079e58974a7511525be2c9c374db482a249f9528ed1f5beca3ab856b17ad14e"
+      "bytes": 312133,
+      "sha256": "2744aafba8afb133b1c92d7b1813d3e769da8babe9439e3d966f94ac28963bbd"
     },
     {
       "file": "decision-expanded-old_core.svg",
-      "bytes": 72440,
-      "sha256": "8370213f60887f29e72c92e15f9307fc356949792666aba3efefdf03df709ee4"
+      "bytes": 79821,
+      "sha256": "0648a4f0ed3720dcc9dc0f19dd43c2f46e08b2dce1cf7a057dba957fa475f288"
     },
     {
       "file": "decision-expanded-overview-600px.png",
-      "bytes": 83372,
-      "sha256": "b7862ca139d91ca29076bf93839b4dc5a5ba694803e62cea2af2bd650d7f5e5b"
+      "bytes": 94850,
+      "sha256": "b8e8361345b814083eef97dc08b9a4fd173bcef83faa72c5f66eac5d0cc1c10b"
     },
     {
       "file": "decision-expanded-overview.pdf",
-      "bytes": 40744,
-      "sha256": "46e92ce18d346fe98d023d1fca0981800e484bfc232bc18df5786f53f7af76a6"
+      "bytes": 41563,
+      "sha256": "4f008072cb5b8268a31e058d0839d68e2416c7f63795ff40cd931906cf74552e"
     },
     {
       "file": "decision-expanded-overview.png",
-      "bytes": 167386,
-      "sha256": "59651d7aa614d710a6cdbd633ccc0a14eff2cebd7b2dcb25c6a9d92f805f7880"
+      "bytes": 187355,
+      "sha256": "2f12699c29ecde8146482e63925b9e80e3bab4047f79fbec17a87e50a1e64ec1"
     },
     {
       "file": "decision-expanded-overview.svg",
-      "bytes": 44985,
-      "sha256": "0f08f7de09ae2f66633ed438ff170b87a8f5ddb3f6a7b2c4e1a475d9472ddb60"
+      "bytes": 48619,
+      "sha256": "6565620295efd3180f1c2079ea7fec332f17b977fcc9781101e9a82001eff743"
     },
     {
       "file": "decision-expanded-ranking-600px.png",
-      "bytes": 75763,
-      "sha256": "1e2318b9c98e1ea02799b2b1db3ca8e6d2022cf11b8c66340a87f24adca3aeed"
+      "bytes": 83184,
+      "sha256": "ddbb71c528b3741f1258367e5436b35148461c628aaf4d03d209ae6c4eccdee1"
     },
     {
       "file": "decision-expanded-ranking.pdf",
-      "bytes": 36962,
-      "sha256": "055c507042a965421f594c083e38c42f564ca7a0a14073519409eb068c9c90f3"
+      "bytes": 37133,
+      "sha256": "bac1d9bf032cd06d4d81170fd10e033e56c1a793e958e9599f8bc570700bf400"
     },
     {
       "file": "decision-expanded-ranking.png",
-      "bytes": 173524,
-      "sha256": "fa7628d846e0db48c079b3d45559a56e46239db6e7cbb13e98610d22e64c09e0"
+      "bytes": 186098,
+      "sha256": "a78d29ea6a73eab02cc8897140614d92b0d7bb4e1295c43c9a2ece60d03ec155"
     },
     {
       "file": "decision-expanded-ranking.svg",
-      "bytes": 36198,
-      "sha256": "422dd9ec53bd5a92d039e6c9dabd7f6b75c32cc1518dc5b2ef8c5b4fbbe83b4f"
+      "bytes": 38118,
+      "sha256": "8de46cff93bdf4b899e74b6b7baddafae6956b6e32b9713702472c8aace0a164"
     },
     {
       "file": "decision-expanded-v3_core-600px.png",
-      "bytes": 143151,
-      "sha256": "0c87d8b3a9dc9efde75f3e552c2fa59bb00b1edf3566df0e09cb42d06c8e72b1"
+      "bytes": 168267,
+      "sha256": "df7099ba80f0a000f7c64cfe6b8aca7ead06d5311e6967a49ae8de32317a2077"
     },
     {
       "file": "decision-expanded-v3_core.pdf",
-      "bytes": 44010,
-      "sha256": "c4919ffd03d1e6b0b228ca96019db772b2e025783bd01b5289b4ec7e9c48ba4f"
+      "bytes": 44464,
+      "sha256": "2ffe3a8fdad5dc957f524dbb03d437aaa50a098920358322575f84d65e95e09f"
     },
     {
       "file": "decision-expanded-v3_core.png",
-      "bytes": 264012,
-      "sha256": "5ecac72bca73c3611d0c1aef365c03ee7194ca7c0a76fa9cbbaa918aa8c9922c"
+      "bytes": 300452,
+      "sha256": "a621a7ffee11dc4909e89a200b7c1207a75bff57a9a237b66e8ed8f2fbac6983"
     },
     {
       "file": "decision-expanded-v3_core.svg",
-      "bytes": 72471,
-      "sha256": "2b07e3b7bcf1b90b2c0522e57e5967c1a23c0c11f77df62451e012110c7f0a46"
+      "bytes": 79901,
+      "sha256": "75c74babb4214337caff4b3a5ec36bb1de5824f486312bde33ab3264d2c50f48"
     },
     {
       "file": "decision-expanded-v4-600px.png",
-      "bytes": 66385,
-      "sha256": "d4795f77521996b32aacefe6aba401f05aa1ea69d78cbc4bf76f92419de4254b"
+      "bytes": 76498,
+      "sha256": "613e8f9c4bc1a6e398190414ae6bf0a70e784cd36e278beec10becbf6c5e71cc"
     },
     {
       "file": "decision-expanded-v4.pdf",
-      "bytes": 38120,
-      "sha256": "be35f997f91283f7bb251e8fd3e6a6ca778be2bfa87a8f564010221982a5c46f"
+      "bytes": 37680,
+      "sha256": "b56a46b9f26ebd18280b73f5bdb381cbc207312f477a82c002a662409e86e32f"
     },
     {
       "file": "decision-expanded-v4.png",
-      "bytes": 135398,
-      "sha256": "2dcd4bb2c76d8eac0801348104a48c85e9c0559d077b73c44a68a5c1c9c9c789"
+      "bytes": 152677,
+      "sha256": "287486dbbdd9e8f89ae5ea358d20170ef2496ccf6ac79b3b0b6169990ff14da8"
     },
     {
       "file": "decision-expanded-v4.svg",
-      "bytes": 39921,
-      "sha256": "50aab8a9120b3196c8fce2f4010cbc5c78f1843ca9e869572da053d85e1374e4"
+      "bytes": 42942,
+      "sha256": "8abbe97a67a7a89291dea776f93990a42b0f2fafe47330bbafcfe6c14c4cc404"
     },
     {
       "file": "decision-expanded-v5-600px.png",
-      "bytes": 78963,
-      "sha256": "61cc194f229afa38cf89e31df823e22819b37ace2dbd7e604dd7ee98a5bd4b9b"
+      "bytes": 102382,
+      "sha256": "8fb899a4acca41fd9cc4cb6885e5e7f6b77a5c962af78f4ad00e99929fb8ce54"
     },
     {
       "file": "decision-expanded-v5.pdf",
-      "bytes": 40262,
-      "sha256": "5ed38900e63a5a56c6a8198c56eb920c55724cfc9abcab8efd91f9788fa30507"
+      "bytes": 40179,
+      "sha256": "b08d52344bd383766b19e967d4ab9ab0783a50ea423de93140dce26c7c91354a"
     },
     {
       "file": "decision-expanded-v5.png",
-      "bytes": 159959,
-      "sha256": "a0df5d94e03faf873a1c9275459f9d9cc8354b3ce6e32c8f8a31c6f8c934d209"
+      "bytes": 187722,
+      "sha256": "bf0d8f84d12b6f1e46ae02a70e0ed42adf45a818d8b4b594b1461ddcb635a13f"
     },
     {
       "file": "decision-expanded-v5.svg",
-      "bytes": 44945,
-      "sha256": "3a634ae3bb2bb3b33f7c89c8ff0c1ad99d7937021654b4b1a99187cadc812d82"
+      "bytes": 48613,
+      "sha256": "a5fba4d5176a8edb6668051678b5926cd0aaa40eabf192d9cefea03e6a64338f"
     }
-  ]
+  ],
+  "post_render_finalizer_sha256": "4818899220640baf74e6f87915dfe4ca5f77aedeca40e63c6f95ff9db1cf3d3e",
+  "preview_reencoding": {
+    "asset": "decision-expanded-v5-600px.png",
+    "reason": "Compressed pixel stream accidentally matched generic private-host text pattern; no textual host metadata",
+    "before_sha256": "afd80c6dc022fe4020885ec6eed87838854cda45d87ae99a0794d86be9ac8da9",
+    "after_sha256": "8fb899a4acca41fd9cc4cb6885e5e7f6b77a5c962af78f4ad00e99929fb8ce54",
+    "decoded_pixels_sha256": "dd1453b3d518a513f067473100b45aa76b91737035c026311d28f76c78fa4d13",
+    "mode": "RGBA",
+    "size": [
+      600,
+      379
+    ],
+    "pixels_exact": true,
+    "scanner_unchanged": true
+  }
 }

metrics/materials-provenance.json

@@ -1,11 +1,27 @@
 {
-  "candidate_bundle_manifest_sha256": "6f0970bfbe5594feecffc14b4824a92080ef09318e12567bd22264c3844e2fc4",
-  "qualification_sha256": "7abb465b93ad2ffbbedab5440c380e02f4d65a058c704a4b2e03277a15197131",
-  "evaluation_receipt_sha256": "5cf55e670d407aaafd727d29a465889fda252df7e3e98f330f0a2b8c88d87fe7",
-  "statistics_sha256": "fe2be9746c4302b829b5aa373c027910cfa84881b80c2068af7c69db76c4659c",
-  "public_projection_sha256": "3824d5e5f01d3ff3cc32cc32b85df727c57dd6bc859c024f60f3b8a4b8441917",
-  "efficiency_receipt_sha256": "ef47c5fb912c91c7e1e50b1a2f9df21c53710f4195e7cd0d9823268207cb95b7",
-  "release_tag": "v1.3",
-  "source_sha256": "5706c0b36cb6c31c157ba1d043ee13873f7383278adb015e3e99694f1636a8a8",
-  "publication_performed": false
+  "docs_only": true,
+  "public_release_bindings": {
+    "Nox": {
+      "repo_id": "llm-semantic-router/Decision-1.0-Nox",
+      "revision": "ad089ad3a5dc9a7a21e6d96db546bb53e2212654",
+      "release_tag": "v1.3",
+      "bundle_manifest_sha256": "92d7f5be5e1ef21ee682f574b35cc01de4dd8ab16b8aa6edf0dfbdfcfcfabba3",
+      "publication_receipt_sha256": "fe15d9e49608c6d2d89301272a993e7d17cffe0e5b0966c32e9bb2740f3ad1e6",
+      "statistics_sha256": "0eb70b3dbc8a5ab8d78279beadd3ab9796623dfe64218dae06ca5edb2795f46d"
+    },
+    "Sol": {
+      "repo_id": "llm-semantic-router/Decision-1.0-Sol",
+      "revision": "2412d9470d3aa125b346262dad80f6161847aa09",
+      "release_tag": "v1.3",
+      "bundle_manifest_sha256": "6f0970bfbe5594feecffc14b4824a92080ef09318e12567bd22264c3844e2fc4",
+      "publication_receipt_sha256": "7316ce19ffc5bbdd440316f7899dc33a8eecb9c36ebdd0cd9f9f8b49f48a446f",
+      "statistics_sha256": "fe2be9746c4302b829b5aa373c027910cfa84881b80c2068af7c69db76c4659c"
+    }
+  },
+  "aggregate_source_sha256": "af8274ce0ba2c1dd1fe35a36dc77d347872443ce144e95b709a5e050680d3fda",
+  "source_evidence_sha256": "8ef8f73315fc185dca5c887728c44ed517a97623cbe25710c2dc83480204d3a8",
+  "original_weights_and_qualification_unchanged": true,
+  "displayed_latest_only": true,
+  "no_new_model_calls": true,
+  "no_APUS_no_V6_mean": true
 }

metrics/question-scaling.json

@@ -217,224 +217,6 @@
         "max_reserved_bytes": 4181721088,
         "input_tokens": 9824
       }
-    },
-    "Sol-v1.2-released": {
-      "choice-q1": {
-        "samples": 30,
-        "p50_ms": 24.421186499999997,
-        "p95_ms": 24.86269545,
-        "requests_per_second_from_median": 40.948051397912224,
-        "questions_per_second_from_median": 40.948051397912224,
-        "measured_requests_per_second": 40.9162057165133,
-        "measured_questions_per_second": 40.9162057165133,
-        "max_allocated_bytes": 3879227392,
-        "max_reserved_bytes": 4177526784,
-        "input_tokens": 309
-      },
-      "choice-q2": {
-        "samples": 30,
-        "p50_ms": 25.181895,
-        "p95_ms": 25.74931005,
-        "requests_per_second_from_median": 39.711070195471784,
-        "questions_per_second_from_median": 79.42214039094357,
-        "measured_requests_per_second": 39.77550614175778,
-        "measured_questions_per_second": 79.55101228351556,
-        "max_allocated_bytes": 3903851520,
-        "max_reserved_bytes": 4179623936,
-        "input_tokens": 618
-      },
-      "choice-q4": {
-        "samples": 30,
-        "p50_ms": 26.0816265,
-        "p95_ms": 26.28338645,
-        "requests_per_second_from_median": 38.341167104743256,
-        "questions_per_second_from_median": 153.36466841897303,
-        "measured_requests_per_second": 38.33365718458782,
-        "measured_questions_per_second": 153.33462873835128,
-        "max_allocated_bytes": 3957335040,
-        "max_reserved_bytes": 4221566976,
-        "input_tokens": 1236
-      },
-      "choice-q8": {
-        "samples": 30,
-        "p50_ms": 37.780621,
-        "p95_ms": 37.90625805,
-        "requests_per_second_from_median": 26.468596162037677,
-        "questions_per_second_from_median": 211.74876929630142,
-        "measured_requests_per_second": 26.492096747886897,
-        "measured_questions_per_second": 211.93677398309518,
-        "max_allocated_bytes": 4058682368,
-        "max_reserved_bytes": 4232052736,
-        "input_tokens": 2472
-      },
-      "choice-q16": {
-        "samples": 30,
-        "p50_ms": 74.72107700000001,
-        "p95_ms": 75.2925489,
-        "requests_per_second_from_median": 13.383104742989717,
-        "questions_per_second_from_median": 214.12967588783548,
-        "measured_requests_per_second": 13.37243576189426,
-        "measured_questions_per_second": 213.95897219030817,
-        "max_allocated_bytes": 4058682880,
-        "max_reserved_bytes": 4221566976,
-        "input_tokens": 4944
-      },
-      "choice-q32": {
-        "samples": 30,
-        "p50_ms": 149.0055205,
-        "p95_ms": 149.9944233,
-        "requests_per_second_from_median": 6.711160745215477,
-        "questions_per_second_from_median": 214.75714384689527,
-        "measured_requests_per_second": 6.711137576884814,
-        "measured_questions_per_second": 214.75640246031404,
-        "max_allocated_bytes": 4058682880,
-        "max_reserved_bytes": 4232052736,
-        "input_tokens": 9888
-      },
-      "noul-q1": {
-        "samples": 30,
-        "p50_ms": 24.315884,
-        "p95_ms": 24.6480419,
-        "requests_per_second_from_median": 41.12538125284691,
-        "questions_per_second_from_median": 41.12538125284691,
-        "measured_requests_per_second": 41.132195715334305,
-        "measured_questions_per_second": 41.132195715334305,
-        "max_allocated_bytes": 3878991872,
-        "max_reserved_bytes": 4177526784,
-        "input_tokens": 262
-      },
-      "noul-q2": {
-        "samples": 30,
-        "p50_ms": 24.912046,
-        "p95_ms": 25.70692795,
-        "requests_per_second_from_median": 40.14122324597506,
-        "questions_per_second_from_median": 80.28244649195013,
-        "measured_requests_per_second": 39.82163554935704,
-        "measured_questions_per_second": 79.64327109871408,
-        "max_allocated_bytes": 3900553216,
-        "max_reserved_bytes": 4448059392,
-        "input_tokens": 524
-      },
-      "noul-q4": {
-        "samples": 30,
-        "p50_ms": 25.244834500000003,
-        "p95_ms": 25.80336805,
-        "requests_per_second_from_median": 39.612064004618446,
-        "questions_per_second_from_median": 158.44825601847378,
-        "measured_requests_per_second": 39.38051173334143,
-        "measured_questions_per_second": 157.52204693336571,
-        "max_allocated_bytes": 3946953728,
-        "max_reserved_bytes": 4248829952,
-        "input_tokens": 1048
-      },
-      "noul-q8": {
-        "samples": 30,
-        "p50_ms": 34.065572,
-        "p95_ms": 34.2347665,
-        "requests_per_second_from_median": 29.35515070758242,
-        "questions_per_second_from_median": 234.84120566065937,
-        "measured_requests_per_second": 29.364579195405305,
-        "measured_questions_per_second": 234.91663356324244,
-        "max_allocated_bytes": 4040541184,
-        "max_reserved_bytes": 4492099584,
-        "input_tokens": 2096
-      },
-      "noul-q16": {
-        "samples": 30,
-        "p50_ms": 67.5993325,
-        "p95_ms": 67.96526985,
-        "requests_per_second_from_median": 14.793045478666523,
-        "questions_per_second_from_median": 236.68872765866436,
-        "measured_requests_per_second": 14.866682769530627,
-        "measured_questions_per_second": 237.86692431249003,
-        "max_allocated_bytes": 4042114560,
-        "max_reserved_bytes": 4232052736,
-        "input_tokens": 4192
-      },
-      "noul-q32": {
-        "samples": 30,
-        "p50_ms": 134.3543275,
-        "p95_ms": 135.0910169,
-        "requests_per_second_from_median": 7.443005510931532,
-        "questions_per_second_from_median": 238.17617634980903,
-        "measured_requests_per_second": 7.468755477211829,
-        "measured_questions_per_second": 239.00017527077853,
-        "max_allocated_bytes": 4043687424,
-        "max_reserved_bytes": 4181721088,
-        "input_tokens": 8384
-      },
-      "score-q1": {
-        "samples": 30,
-        "p50_ms": 24.6888355,
-        "p95_ms": 25.1770665,
-        "requests_per_second_from_median": 40.504137993871765,
-        "questions_per_second_from_median": 40.504137993871765,
-        "measured_requests_per_second": 40.57560652802106,
-        "measured_questions_per_second": 40.57560652802106,
-        "max_allocated_bytes": 3879227392,
-        "max_reserved_bytes": 4221566976,
-        "input_tokens": 307
-      },
-      "score-q2": {
-        "samples": 30,
-        "p50_ms": 25.2654285,
-        "p95_ms": 25.68663195,
-        "requests_per_second_from_median": 39.579775977280576,
-        "questions_per_second_from_median": 79.15955195456115,
-        "measured_requests_per_second": 39.62438277676804,
-        "measured_questions_per_second": 79.24876555353607,
-        "max_allocated_bytes": 3903851520,
-        "max_reserved_bytes": 4221566976,
-        "input_tokens": 614
-      },
-      "score-q4": {
-        "samples": 30,
-        "p50_ms": 25.9118355,
-        "p95_ms": 26.8743311,
-        "requests_per_second_from_median": 38.592403073877186,
-        "questions_per_second_from_median": 154.36961229550874,
-        "measured_requests_per_second": 38.34077367571147,
-        "measured_questions_per_second": 153.36309470284587,
-        "max_allocated_bytes": 3955237888,
-        "max_reserved_bytes": 4445962240,
-        "input_tokens": 1228
-      },
-      "score-q8": {
-        "samples": 30,
-        "p50_ms": 37.5915095,
-        "p95_ms": 37.6891344,
-        "requests_per_second_from_median": 26.601751653521653,
-        "questions_per_second_from_median": 212.81401322817322,
-        "measured_requests_per_second": 26.632498969661853,
-        "measured_questions_per_second": 213.05999175729482,
-        "max_allocated_bytes": 4058682368,
-        "max_reserved_bytes": 4599054336,
-        "input_tokens": 2456
-      },
-      "score-q16": {
-        "samples": 30,
-        "p50_ms": 74.743778,
-        "p95_ms": 75.22428245,
-        "requests_per_second_from_median": 13.379040058692242,
-        "questions_per_second_from_median": 214.06464093907587,
-        "measured_requests_per_second": 13.39541536220642,
-        "measured_questions_per_second": 214.3266457953027,
-        "max_allocated_bytes": 4058682880,
-        "max_reserved_bytes": 4599054336,
-        "input_tokens": 4912
-      },
-      "score-q32": {
-        "samples": 30,
-        "p50_ms": 148.472903,
-        "p95_ms": 149.6212565,
-        "requests_per_second_from_median": 6.735235721766685,
-        "questions_per_second_from_median": 215.52754309653392,
-        "measured_requests_per_second": 6.725016656263462,
-        "measured_questions_per_second": 215.2005330004308,
-        "max_allocated_bytes": 4058682880,
-        "max_reserved_bytes": 4221566976,
-        "input_tokens": 9824
-      }
     }
   },
   "choice_q_counts": [
@@ -445,7 +227,7 @@
     16,
     32
   ],
-  "candidate_p50_ms": [
+  "current_p50_ms": [
     24.456031,
     25.0034785,
     25.974363500000003,
@@ -453,15 +235,6 @@
     74.391125,
     148.7048505
   ],
-  "reference_p50_ms": [
-    24.421186499999997,
-    25.181895,
-    26.0816265,
-    37.780621,
-    74.72107700000001,
-    149.0055205
-  ],
-  "geomean_latency_ratio": 0.9959738244334195,
-  "raw_observations": 1404,
-  "claim_boundary": "Sequential whole requests with fixed state/question bytes; not concurrent-serving throughput."
+  "source_summary_sha256": "7fcdfa1b22bc3e531e1d7dc296e5ba541be78d67d4fab1d9102f1b946a36c5bb",
+  "display_scope": "Current release only; full historical timing evidence retained in its immutable evaluation receipt."
 }

metrics/semantic-consistency.json

@@ -0,0 +1,702 @@
+{
+  "metric": "semantic_prediction_consistency_group_fraction",
+  "display_metric": "Same-semantics group consistency",
+  "panel": "Observed original core (880 rows)",
+  "in_quality_average": false,
+  "unit": "semantic groups, not individual questions",
+  "rows": [
+    {
+      "model_key": "Nox-retention-selected",
+      "display_name": "Nox v1.3",
+      "consistent_groups": 174,
+      "eligible_groups": 192,
+      "excluded_groups": 240,
+      "fraction": 0.90625,
+      "all_groups": 432,
+      "families": {
+        "ag_news": {
+          "consistent_groups": null,
+          "eligible_groups": 0,
+          "excluded_groups": 128,
+          "fraction": null,
+          "all_groups": 128
+        },
+        "boolean_constraints": {
+          "consistent_groups": 30,
+          "eligible_groups": 32,
+          "excluded_groups": 0,
+          "fraction": 0.9375,
+          "all_groups": 32
+        },
+        "dbpedia_14": {
+          "consistent_groups": null,
+          "eligible_groups": 0,
+          "excluded_groups": 112,
+          "fraction": null,
+          "all_groups": 112
+        },
+        "natural_intents": {
+          "consistent_groups": 16,
+          "eligible_groups": 16,
+          "excluded_groups": 0,
+          "fraction": 1.0,
+          "all_groups": 16
+        },
+        "option_carrier": {
+          "consistent_groups": 24,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 1.0,
+          "all_groups": 24
+        },
+        "ordinal_rubric": {
+          "consistent_groups": 31,
+          "eligible_groups": 32,
+          "excluded_groups": 0,
+          "fraction": 0.96875,
+          "all_groups": 32
+        },
+        "relational_composition": {
+          "consistent_groups": 15,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.625,
+          "all_groups": 24
+        },
+        "scoped_evidence": {
+          "consistent_groups": 23,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.9583333333333334,
+          "all_groups": 24
+        },
+        "state_tracking": {
+          "consistent_groups": 19,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.7916666666666666,
+          "all_groups": 24
+        },
+        "unknown_rejection": {
+          "consistent_groups": 16,
+          "eligible_groups": 16,
+          "excluded_groups": 0,
+          "fraction": 1.0,
+          "all_groups": 16
+        }
+      }
+    },
+    {
+      "model_key": "Sol-composition-selected",
+      "display_name": "Sol v1.3",
+      "consistent_groups": 167,
+      "eligible_groups": 192,
+      "excluded_groups": 240,
+      "fraction": 0.8697916666666666,
+      "all_groups": 432,
+      "families": {
+        "ag_news": {
+          "consistent_groups": null,
+          "eligible_groups": 0,
+          "excluded_groups": 128,
+          "fraction": null,
+          "all_groups": 128
+        },
+        "boolean_constraints": {
+          "consistent_groups": 30,
+          "eligible_groups": 32,
+          "excluded_groups": 0,
+          "fraction": 0.9375,
+          "all_groups": 32
+        },
+        "dbpedia_14": {
+          "consistent_groups": null,
+          "eligible_groups": 0,
+          "excluded_groups": 112,
+          "fraction": null,
+          "all_groups": 112
+        },
+        "natural_intents": {
+          "consistent_groups": 16,
+          "eligible_groups": 16,
+          "excluded_groups": 0,
+          "fraction": 1.0,
+          "all_groups": 16
+        },
+        "option_carrier": {
+          "consistent_groups": 23,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.9583333333333334,
+          "all_groups": 24
+        },
+        "ordinal_rubric": {
+          "consistent_groups": 29,
+          "eligible_groups": 32,
+          "excluded_groups": 0,
+          "fraction": 0.90625,
+          "all_groups": 32
+        },
+        "relational_composition": {
+          "consistent_groups": 17,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.7083333333333334,
+          "all_groups": 24
+        },
+        "scoped_evidence": {
+          "consistent_groups": 21,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.875,
+          "all_groups": 24
+        },
+        "state_tracking": {
+          "consistent_groups": 15,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.625,
+          "all_groups": 24
+        },
+        "unknown_rejection": {
+          "consistent_groups": 16,
+          "eligible_groups": 16,
+          "excluded_groups": 0,
+          "fraction": 1.0,
+          "all_groups": 16
+        }
+      }
+    },
+    {
+      "model_key": "kev-9b",
+      "display_name": "Kev 9B",
+      "consistent_groups": 143,
+      "eligible_groups": 192,
+      "excluded_groups": 240,
+      "fraction": 0.7447916666666666,
+      "all_groups": 432,
+      "families": {
+        "ag_news": {
+          "consistent_groups": null,
+          "eligible_groups": 0,
+          "excluded_groups": 128,
+          "fraction": null,
+          "all_groups": 128
+        },
+        "boolean_constraints": {
+          "consistent_groups": 29,
+          "eligible_groups": 32,
+          "excluded_groups": 0,
+          "fraction": 0.90625,
+          "all_groups": 32
+        },
+        "dbpedia_14": {
+          "consistent_groups": null,
+          "eligible_groups": 0,
+          "excluded_groups": 112,
+          "fraction": null,
+          "all_groups": 112
+        },
+        "natural_intents": {
+          "consistent_groups": 16,
+          "eligible_groups": 16,
+          "excluded_groups": 0,
+          "fraction": 1.0,
+          "all_groups": 16
+        },
+        "option_carrier": {
+          "consistent_groups": 0,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.0,
+          "all_groups": 24
+        },
+        "ordinal_rubric": {
+          "consistent_groups": 30,
+          "eligible_groups": 32,
+          "excluded_groups": 0,
+          "fraction": 0.9375,
+          "all_groups": 32
+        },
+        "relational_composition": {
+          "consistent_groups": 21,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.875,
+          "all_groups": 24
+        },
+        "scoped_evidence": {
+          "consistent_groups": 20,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.8333333333333334,
+          "all_groups": 24
+        },
+        "state_tracking": {
+          "consistent_groups": 16,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.6666666666666666,
+          "all_groups": 24
+        },
+        "unknown_rejection": {
+          "consistent_groups": 11,
+          "eligible_groups": 16,
+          "excluded_groups": 0,
+          "fraction": 0.6875,
+          "all_groups": 16
+        }
+      }
+    },
+    {
+      "model_key": "Decider · 2B",
+      "display_name": "Decider 2B",
+      "consistent_groups": 126,
+      "eligible_groups": 192,
+      "excluded_groups": 240,
+      "fraction": 0.65625,
+      "all_groups": 432,
+      "families": {
+        "ag_news": {
+          "consistent_groups": null,
+          "eligible_groups": 0,
+          "excluded_groups": 128,
+          "fraction": null,
+          "all_groups": 128
+        },
+        "boolean_constraints": {
+          "consistent_groups": 24,
+          "eligible_groups": 32,
+          "excluded_groups": 0,
+          "fraction": 0.75,
+          "all_groups": 32
+        },
+        "dbpedia_14": {
+          "consistent_groups": null,
+          "eligible_groups": 0,
+          "excluded_groups": 112,
+          "fraction": null,
+          "all_groups": 112
+        },
+        "natural_intents": {
+          "consistent_groups": 16,
+          "eligible_groups": 16,
+          "excluded_groups": 0,
+          "fraction": 1.0,
+          "all_groups": 16
+        },
+        "option_carrier": {
+          "consistent_groups": 0,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.0,
+          "all_groups": 24
+        },
+        "ordinal_rubric": {
+          "consistent_groups": 28,
+          "eligible_groups": 32,
+          "excluded_groups": 0,
+          "fraction": 0.875,
+          "all_groups": 32
+        },
+        "relational_composition": {
+          "consistent_groups": 14,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.5833333333333334,
+          "all_groups": 24
+        },
+        "scoped_evidence": {
+          "consistent_groups": 22,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.9166666666666666,
+          "all_groups": 24
+        },
+        "state_tracking": {
+          "consistent_groups": 14,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.5833333333333334,
+          "all_groups": 24
+        },
+        "unknown_rejection": {
+          "consistent_groups": 8,
+          "eligible_groups": 16,
+          "excluded_groups": 0,
+          "fraction": 0.5,
+          "all_groups": 16
+        }
+      }
+    },
+    {
+      "model_key": "Laya · EN/ML",
+      "display_name": "Laya · upstream default",
+      "consistent_groups": 115,
+      "eligible_groups": 192,
+      "excluded_groups": 240,
+      "fraction": 0.5989583333333334,
+      "all_groups": 432,
+      "families": {
+        "ag_news": {
+          "consistent_groups": null,
+          "eligible_groups": 0,
+          "excluded_groups": 128,
+          "fraction": null,
+          "all_groups": 128
+        },
+        "boolean_constraints": {
+          "consistent_groups": 28,
+          "eligible_groups": 32,
+          "excluded_groups": 0,
+          "fraction": 0.875,
+          "all_groups": 32
+        },
+        "dbpedia_14": {
+          "consistent_groups": null,
+          "eligible_groups": 0,
+          "excluded_groups": 112,
+          "fraction": null,
+          "all_groups": 112
+        },
+        "natural_intents": {
+          "consistent_groups": 9,
+          "eligible_groups": 16,
+          "excluded_groups": 0,
+          "fraction": 0.5625,
+          "all_groups": 16
+        },
+        "option_carrier": {
+          "consistent_groups": 22,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.9166666666666666,
+          "all_groups": 24
+        },
+        "ordinal_rubric": {
+          "consistent_groups": 25,
+          "eligible_groups": 32,
+          "excluded_groups": 0,
+          "fraction": 0.78125,
+          "all_groups": 32
+        },
+        "relational_composition": {
+          "consistent_groups": 2,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.08333333333333333,
+          "all_groups": 24
+        },
+        "scoped_evidence": {
+          "consistent_groups": 9,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.375,
+          "all_groups": 24
+        },
+        "state_tracking": {
+          "consistent_groups": 12,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.5,
+          "all_groups": 24
+        },
+        "unknown_rejection": {
+          "consistent_groups": 8,
+          "eligible_groups": 16,
+          "excluded_groups": 0,
+          "fraction": 0.5,
+          "all_groups": 16
+        }
+      }
+    },
+    {
+      "model_key": "Qwen3.5 · 2B, untuned",
+      "display_name": "Qwen3.5 2B · untuned",
+      "consistent_groups": 94,
+      "eligible_groups": 192,
+      "excluded_groups": 240,
+      "fraction": 0.4895833333333333,
+      "all_groups": 432,
+      "families": {
+        "ag_news": {
+          "consistent_groups": null,
+          "eligible_groups": 0,
+          "excluded_groups": 128,
+          "fraction": null,
+          "all_groups": 128
+        },
+        "boolean_constraints": {
+          "consistent_groups": 32,
+          "eligible_groups": 32,
+          "excluded_groups": 0,
+          "fraction": 1.0,
+          "all_groups": 32
+        },
+        "dbpedia_14": {
+          "consistent_groups": null,
+          "eligible_groups": 0,
+          "excluded_groups": 112,
+          "fraction": null,
+          "all_groups": 112
+        },
+        "natural_intents": {
+          "consistent_groups": 14,
+          "eligible_groups": 16,
+          "excluded_groups": 0,
+          "fraction": 0.875,
+          "all_groups": 16
+        },
+        "option_carrier": {
+          "consistent_groups": 0,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.0,
+          "all_groups": 24
+        },
+        "ordinal_rubric": {
+          "consistent_groups": 30,
+          "eligible_groups": 32,
+          "excluded_groups": 0,
+          "fraction": 0.9375,
+          "all_groups": 32
+        },
+        "relational_composition": {
+          "consistent_groups": 3,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.125,
+          "all_groups": 24
+        },
+        "scoped_evidence": {
+          "consistent_groups": 6,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.25,
+          "all_groups": 24
+        },
+        "state_tracking": {
+          "consistent_groups": 0,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.0,
+          "all_groups": 24
+        },
+        "unknown_rejection": {
+          "consistent_groups": 9,
+          "eligible_groups": 16,
+          "excluded_groups": 0,
+          "fraction": 0.5625,
+          "all_groups": 16
+        }
+      }
+    },
+    {
+      "model_key": "Qwen3.5 · 4B, untuned",
+      "display_name": "Qwen3.5 4B · untuned",
+      "consistent_groups": 98,
+      "eligible_groups": 192,
+      "excluded_groups": 240,
+      "fraction": 0.5104166666666666,
+      "all_groups": 432,
+      "families": {
+        "ag_news": {
+          "consistent_groups": null,
+          "eligible_groups": 0,
+          "excluded_groups": 128,
+          "fraction": null,
+          "all_groups": 128
+        },
+        "boolean_constraints": {
+          "consistent_groups": 22,
+          "eligible_groups": 32,
+          "excluded_groups": 0,
+          "fraction": 0.6875,
+          "all_groups": 32
+        },
+        "dbpedia_14": {
+          "consistent_groups": null,
+          "eligible_groups": 0,
+          "excluded_groups": 112,
+          "fraction": null,
+          "all_groups": 112
+        },
+        "natural_intents": {
+          "consistent_groups": 16,
+          "eligible_groups": 16,
+          "excluded_groups": 0,
+          "fraction": 1.0,
+          "all_groups": 16
+        },
+        "option_carrier": {
+          "consistent_groups": 10,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.4166666666666667,
+          "all_groups": 24
+        },
+        "ordinal_rubric": {
+          "consistent_groups": 30,
+          "eligible_groups": 32,
+          "excluded_groups": 0,
+          "fraction": 0.9375,
+          "all_groups": 32
+        },
+        "relational_composition": {
+          "consistent_groups": 4,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.16666666666666666,
+          "all_groups": 24
+        },
+        "scoped_evidence": {
+          "consistent_groups": 6,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.25,
+          "all_groups": 24
+        },
+        "state_tracking": {
+          "consistent_groups": 1,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.041666666666666664,
+          "all_groups": 24
+        },
+        "unknown_rejection": {
+          "consistent_groups": 9,
+          "eligible_groups": 16,
+          "excluded_groups": 0,
+          "fraction": 0.5625,
+          "all_groups": 16
+        }
+      }
+    },
+    {
+      "model_key": "Jev · 1.13.0",
+      "display_name": "Jev 1.13.0",
+      "consistent_groups": 158,
+      "eligible_groups": 192,
+      "excluded_groups": 240,
+      "fraction": 0.8229166666666666,
+      "all_groups": 432,
+      "families": {
+        "ag_news": {
+          "consistent_groups": null,
+          "eligible_groups": 0,
+          "excluded_groups": 128,
+          "fraction": null,
+          "all_groups": 128
+        },
+        "boolean_constraints": {
+          "consistent_groups": 32,
+          "eligible_groups": 32,
+          "excluded_groups": 0,
+          "fraction": 1.0,
+          "all_groups": 32
+        },
+        "dbpedia_14": {
+          "consistent_groups": null,
+          "eligible_groups": 0,
+          "excluded_groups": 112,
+          "fraction": null,
+          "all_groups": 112
+        },
+        "natural_intents": {
+          "consistent_groups": 16,
+          "eligible_groups": 16,
+          "excluded_groups": 0,
+          "fraction": 1.0,
+          "all_groups": 16
+        },
+        "option_carrier": {
+          "consistent_groups": 0,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.0,
+          "all_groups": 24
+        },
+        "ordinal_rubric": {
+          "consistent_groups": 32,
+          "eligible_groups": 32,
+          "excluded_groups": 0,
+          "fraction": 1.0,
+          "all_groups": 32
+        },
+        "relational_composition": {
+          "consistent_groups": 21,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.875,
+          "all_groups": 24
+        },
+        "scoped_evidence": {
+          "consistent_groups": 21,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.875,
+          "all_groups": 24
+        },
+        "state_tracking": {
+          "consistent_groups": 20,
+          "eligible_groups": 24,
+          "excluded_groups": 0,
+          "fraction": 0.8333333333333334,
+          "all_groups": 24
+        },
+        "unknown_rejection": {
+          "consistent_groups": 16,
+          "eligible_groups": 16,
+          "excluded_groups": 0,
+          "fraction": 1.0,
+          "all_groups": 16
+        }
+      }
+    }
+  ],
+  "eligible_rule": "Group has more than one row; every row has unique identifiable option semantics and consistency_expected=true; all rows share one semantic gold. Success requires every output valid and one identical semantic prediction across the group.",
+  "coverage": "192 eligible groups across 8 families (640 rows); 240 singleton groups from AG News and DBpedia excluded. Groups are weighted equally, not families.",
+  "limits": [
+    "Mixed variations of language, opaque keys, option order and evidence-carrier position/role; not an isolated order intervention.",
+    "Consistent wrong answers count as consistent. This is not accuracy, semantic correctness, or a new quality/release gate.",
+    "Bilingual Boolean/ordinal groups contain language variation but no Choice option permutation.",
+    "The same group count can hide different errors; retain task accuracy, probability calibration and input-support disclosures.",
+    "These are already-observed regression data. V6 no-op, combined key/order and genuine reversal are separate diagnostics."
+  ],
+  "count_proof": "Each numerator recovered from its stored aggregate ratio, verified by exact Python integer division equal to the stored float; family integer sums equal overall. No new scoring or inference.",
+  "sources": [
+    {
+      "artifact": "aggregate-details.json",
+      "sha256": "428275514d77ca32ac5ae81826aa284136b70ff5eeb919b8dd66aa8d6aca5563"
+    },
+    {
+      "artifact": "statistics.json",
+      "sha256": "af8274ce0ba2c1dd1fe35a36dc77d347872443ce144e95b709a5e050680d3fda"
+    },
+    {
+      "artifact": "identities.json",
+      "sha256": "89bd6fde6da572da6f631d9b45216377fc5716009ff43b1ef2bd7fa394de029b"
+    },
+    {
+      "artifact": "score.py",
+      "sha256": "36a4f1ea811fcb40f299ed311636322482b28047034b5e6e75b0a1c70864a9d7"
+    },
+    {
+      "artifact": "build_suite.py",
+      "sha256": "12def3caa03277120b5fb27be7bb6afa4a86ac39adc4cffbe755a7cfb735297f"
+    },
+    {
+      "artifact": "PUBLICATION-COMPLETE.json",
+      "sha256": "46a25490dbacdbe07d1587c617b2115c4fde5f7a15233d442fa7f3238864fea9"
+    },
+    {
+      "artifact": "PUBLICATION-COMPLETE.json",
+      "sha256": "9e9b777ad714b2f31cc36b25be9aa6e457b8ec51cdc9795d0bafd25789b63b34"
+    },
+    {
+      "artifact": "PUBLISHED-NOX-v1.3-ADDENDUM.json",
+      "sha256": "6682e9881cc6ab707f775323c2807f940a94838507b63212bc81f032c0fd9ba7"
+    }
+  ],
+  "source_sha256": "d78e0762e7bc0b12c774510acbaa7d135c4103abdd2305901fa3d7cbc6238e02"
+}

release-manifest.json

@@ -2,8 +2,8 @@
   "format": "decision-public-release-v1",
   "status": "assembled-not-published",
   "bundle_manifest_sha256": "6f0970bfbe5594feecffc14b4824a92080ef09318e12567bd22264c3844e2fc4",
-  "readiness_sha256": "5c60910455e94a5aba72f2348bda5beaa3e1d7da140c2dacece306f1a18ce4da",
-  "model_card_sha256": "def12f1aaa0f47c7a723e1d745e639b2bc777d5d095b275c338fbf06b2f1cbb7",
+  "readiness_sha256": "c4cbf477f046aca93a68c59394d765867eeba53ec7e8d0b083fe104dd3c49381",
+  "model_card_sha256": "166c19d0cdb09b51329f28e0f8e6ed46c30ce02978510ae48635c5141eb9fd5b",
   "repo_id": "llm-semantic-router/Decision-1.0-Sol",
   "assembly_script_sha256": "7831ef5952b8256e585d4efa91cac4ce87738f4eb93c84b1cf1e7250def1f430",
   "original_bundle_manifest_preserved": true,
@@ -21,14 +21,19 @@
     },
     {
       "file": "EVALUATION.md",
-      "bytes": 7469,
-      "sha256": "881f29f6c275f7ebbb822a48d707a8aec57ade9ca11c3c64339e4235c5d631e6"
+      "bytes": 9573,
+      "sha256": "eec0f6e285ae5e48b86e83e0588586a5198f96fc7d3844f96c8801e7ceffaab5"
     },
     {
       "file": "LICENSE",
       "bytes": 11544,
       "sha256": "bbedc3fda3305820b977265f01b8619d87570a6739de3a5582c3464840f1e57a"
     },
+    {
+      "file": "MATERIALS.json",
+      "bytes": 7549,
+      "sha256": "92b1b270e25276b0b72c4d477c09cec1c6043d1f28187e319843a53fff9bc791"
+    },
     {
       "file": "NORMALIZATION_RUNTIME.md",
       "bytes": 754,
@@ -36,8 +41,8 @@
     },
     {
       "file": "QUESTION-SCALING.md",
-      "bytes": 3514,
-      "sha256": "97cb877e7ef34584c01d3b236f6ac91885a783ef8e757135f2db90d4e7a34ea9"
+      "bytes": 1549,
+      "sha256": "9eb8e8537bf1a045d510e57c8fdaf81b58192f5f760eb54778602c60eb441d1e"
     },
     {
       "file": "QWEN-LICENSE",
@@ -46,8 +51,8 @@
     },
     {
       "file": "README.md",
-      "bytes": 3864,
-      "sha256": "def12f1aaa0f47c7a723e1d745e639b2bc777d5d095b275c338fbf06b2f1cbb7"
+      "bytes": 4999,
+      "sha256": "166c19d0cdb09b51329f28e0f8e6ed46c30ce02978510ae48635c5141eb9fd5b"
     },
     {
       "file": "RUNTIME.md",
@@ -81,123 +86,123 @@
     },
     {
       "file": "assets/decision-expanded-old_core-600px.png",
-      "bytes": 139324,
-      "sha256": "522f361db1d2e747950a5d5b82f0f640f1ccc60af45509b919de031c0268eaa0"
+      "bytes": 167493,
+      "sha256": "6d36e595bea7465903ee787d217ef07ad648472d336655e88e51263325d0a7e2"
     },
     {
       "file": "assets/decision-expanded-old_core.pdf",
-      "bytes": 44058,
-      "sha256": "deb028656a8fc299e75921e7df7c241e71c21bcea70774d517e62bbfc7c1011c"
+      "bytes": 44384,
+      "sha256": "cbd80145027dd79b37ed799a64b4c173d44973b358bc05414da80355ba7072a2"
     },
     {
       "file": "assets/decision-expanded-old_core.png",
-      "bytes": 261230,
-      "sha256": "8079e58974a7511525be2c9c374db482a249f9528ed1f5beca3ab856b17ad14e"
+      "bytes": 312133,
+      "sha256": "2744aafba8afb133b1c92d7b1813d3e769da8babe9439e3d966f94ac28963bbd"
     },
     {
       "file": "assets/decision-expanded-old_core.svg",
-      "bytes": 72440,
-      "sha256": "8370213f60887f29e72c92e15f9307fc356949792666aba3efefdf03df709ee4"
+      "bytes": 79821,
+      "sha256": "0648a4f0ed3720dcc9dc0f19dd43c2f46e08b2dce1cf7a057dba957fa475f288"
     },
     {
       "file": "assets/decision-expanded-overview-600px.png",
-      "bytes": 83372,
-      "sha256": "b7862ca139d91ca29076bf93839b4dc5a5ba694803e62cea2af2bd650d7f5e5b"
+      "bytes": 94850,
+      "sha256": "b8e8361345b814083eef97dc08b9a4fd173bcef83faa72c5f66eac5d0cc1c10b"
     },
     {
       "file": "assets/decision-expanded-overview.pdf",
-      "bytes": 40744,
-      "sha256": "46e92ce18d346fe98d023d1fca0981800e484bfc232bc18df5786f53f7af76a6"
+      "bytes": 41563,
+      "sha256": "4f008072cb5b8268a31e058d0839d68e2416c7f63795ff40cd931906cf74552e"
     },
     {
       "file": "assets/decision-expanded-overview.png",
-      "bytes": 167386,
-      "sha256": "59651d7aa614d710a6cdbd633ccc0a14eff2cebd7b2dcb25c6a9d92f805f7880"
+      "bytes": 187355,
+      "sha256": "2f12699c29ecde8146482e63925b9e80e3bab4047f79fbec17a87e50a1e64ec1"
     },
     {
       "file": "assets/decision-expanded-overview.svg",
-      "bytes": 44985,
-      "sha256": "0f08f7de09ae2f66633ed438ff170b87a8f5ddb3f6a7b2c4e1a475d9472ddb60"
+      "bytes": 48619,
+      "sha256": "6565620295efd3180f1c2079ea7fec332f17b977fcc9781101e9a82001eff743"
     },
     {
       "file": "assets/decision-expanded-ranking-600px.png",
-      "bytes": 75763,
-      "sha256": "1e2318b9c98e1ea02799b2b1db3ca8e6d2022cf11b8c66340a87f24adca3aeed"
+      "bytes": 83184,
+      "sha256": "ddbb71c528b3741f1258367e5436b35148461c628aaf4d03d209ae6c4eccdee1"
     },
     {
       "file": "assets/decision-expanded-ranking.pdf",
-      "bytes": 36962,
-      "sha256": "055c507042a965421f594c083e38c42f564ca7a0a14073519409eb068c9c90f3"
+      "bytes": 37133,
+      "sha256": "bac1d9bf032cd06d4d81170fd10e033e56c1a793e958e9599f8bc570700bf400"
     },
     {
       "file": "assets/decision-expanded-ranking.png",
-      "bytes": 173524,
-      "sha256": "fa7628d846e0db48c079b3d45559a56e46239db6e7cbb13e98610d22e64c09e0"
+      "bytes": 186098,
+      "sha256": "a78d29ea6a73eab02cc8897140614d92b0d7bb4e1295c43c9a2ece60d03ec155"
     },
     {
       "file": "assets/decision-expanded-ranking.svg",
-      "bytes": 36198,
-      "sha256": "422dd9ec53bd5a92d039e6c9dabd7f6b75c32cc1518dc5b2ef8c5b4fbbe83b4f"
+      "bytes": 38118,
+      "sha256": "8de46cff93bdf4b899e74b6b7baddafae6956b6e32b9713702472c8aace0a164"
     },
     {
       "file": "assets/decision-expanded-v3_core-600px.png",
-      "bytes": 143151,
-      "sha256": "0c87d8b3a9dc9efde75f3e552c2fa59bb00b1edf3566df0e09cb42d06c8e72b1"
+      "bytes": 168267,
+      "sha256": "df7099ba80f0a000f7c64cfe6b8aca7ead06d5311e6967a49ae8de32317a2077"
     },
     {
       "file": "assets/decision-expanded-v3_core.pdf",
-      "bytes": 44010,
-      "sha256": "c4919ffd03d1e6b0b228ca96019db772b2e025783bd01b5289b4ec7e9c48ba4f"
+      "bytes": 44464,
+      "sha256": "2ffe3a8fdad5dc957f524dbb03d437aaa50a098920358322575f84d65e95e09f"
     },
     {
       "file": "assets/decision-expanded-v3_core.png",
-      "bytes": 264012,
-      "sha256": "5ecac72bca73c3611d0c1aef365c03ee7194ca7c0a76fa9cbbaa918aa8c9922c"
+      "bytes": 300452,
+      "sha256": "a621a7ffee11dc4909e89a200b7c1207a75bff57a9a237b66e8ed8f2fbac6983"
     },
     {
       "file": "assets/decision-expanded-v3_core.svg",
-      "bytes": 72471,
-      "sha256": "2b07e3b7bcf1b90b2c0522e57e5967c1a23c0c11f77df62451e012110c7f0a46"
+      "bytes": 79901,
+      "sha256": "75c74babb4214337caff4b3a5ec36bb1de5824f486312bde33ab3264d2c50f48"
     },
     {
       "file": "assets/decision-expanded-v4-600px.png",
-      "bytes": 66385,
-      "sha256": "d4795f77521996b32aacefe6aba401f05aa1ea69d78cbc4bf76f92419de4254b"
+      "bytes": 76498,
+      "sha256": "613e8f9c4bc1a6e398190414ae6bf0a70e784cd36e278beec10becbf6c5e71cc"
     },
     {
       "file": "assets/decision-expanded-v4.pdf",
-      "bytes": 38120,
-      "sha256": "be35f997f91283f7bb251e8fd3e6a6ca778be2bfa87a8f564010221982a5c46f"
+      "bytes": 37680,
+      "sha256": "b56a46b9f26ebd18280b73f5bdb381cbc207312f477a82c002a662409e86e32f"
     },
     {
       "file": "assets/decision-expanded-v4.png",
-      "bytes": 135398,
-      "sha256": "2dcd4bb2c76d8eac0801348104a48c85e9c0559d077b73c44a68a5c1c9c9c789"
+      "bytes": 152677,
+      "sha256": "287486dbbdd9e8f89ae5ea358d20170ef2496ccf6ac79b3b0b6169990ff14da8"
     },
     {
       "file": "assets/decision-expanded-v4.svg",
-      "bytes": 39921,
-      "sha256": "50aab8a9120b3196c8fce2f4010cbc5c78f1843ca9e869572da053d85e1374e4"
+      "bytes": 42942,
+      "sha256": "8abbe97a67a7a89291dea776f93990a42b0f2fafe47330bbafcfe6c14c4cc404"
     },
     {
       "file": "assets/decision-expanded-v5-600px.png",
-      "bytes": 78963,
-      "sha256": "61cc194f229afa38cf89e31df823e22819b37ace2dbd7e604dd7ee98a5bd4b9b"
+      "bytes": 102382,
+      "sha256": "8fb899a4acca41fd9cc4cb6885e5e7f6b77a5c962af78f4ad00e99929fb8ce54"
     },
     {
       "file": "assets/decision-expanded-v5.pdf",
-      "bytes": 40262,
-      "sha256": "5ed38900e63a5a56c6a8198c56eb920c55724cfc9abcab8efd91f9788fa30507"
+      "bytes": 40179,
+      "sha256": "b08d52344bd383766b19e967d4ab9ab0783a50ea423de93140dce26c7c91354a"
     },
     {
       "file": "assets/decision-expanded-v5.png",
-      "bytes": 159959,
-      "sha256": "a0df5d94e03faf873a1c9275459f9d9cc8354b3ce6e32c8f8a31c6f8c934d209"
+      "bytes": 187722,
+      "sha256": "bf0d8f84d12b6f1e46ae02a70e0ed42adf45a818d8b4b594b1461ddcb635a13f"
     },
     {
       "file": "assets/decision-expanded-v5.svg",
-      "bytes": 44945,
-      "sha256": "3a634ae3bb2bb3b33f7c89c8ff0c1ad99d7937021654b4b1a99187cadc812d82"
+      "bytes": 48613,
+      "sha256": "a5fba4d5176a8edb6668051678b5926cd0aaa40eabf192d9cefea03e6a64338f"
     },
     {
       "file": "assets/decision-family-header.png",
@@ -206,23 +211,23 @@
     },
     {
       "file": "assets/decision-question-scaling-600px.png",
-      "bytes": 44217,
-      "sha256": "dc8c73aae76fff9892349f26ce5ba7174e528380f989def8aff16ec33c60fd7f"
+      "bytes": 40784,
+      "sha256": "3de11b7e5d1b49ca874b50ef6ae8ced1dcdc3a83dae4889170a0134f0ca3f8cc"
     },
     {
       "file": "assets/decision-question-scaling.pdf",
-      "bytes": 31855,
-      "sha256": "f9a67798673f28f812bdf5cdbed4d011bfd13450593f89da2bec463fd70fe1ad"
+      "bytes": 31275,
+      "sha256": "57f5e52d5722ba9c5643bbb18bab9ad2aa19cc58f2723780cfa2452c523f2fb0"
     },
     {
       "file": "assets/decision-question-scaling.png",
-      "bytes": 104528,
-      "sha256": "990539a03785119aecdcde3dec413292ca504c18fd501b94d2cae2d724a200b8"
+      "bytes": 94954,
+      "sha256": "0236ed14e75d90cc7691ef6c9d892191070d5c380083fd2602b222bad33b34cd"
     },
     {
       "file": "assets/decision-question-scaling.svg",
-      "bytes": 27436,
-      "sha256": "50d3cb046410589384d1d13a7bd38d4435bc0eb8835cfb98ae147807907f36dc"
+      "bytes": 25462,
+      "sha256": "9c5a9d5bdec8dd4f80c2af20f828709e45dbc32c825f6f93fae8f980c79199d6"
     },
     {
       "file": "assets/readout.png",
@@ -286,28 +291,33 @@
     },
     {
       "file": "metrics/comparator-coverage.json",
-      "bytes": 21192,
-      "sha256": "7612486e073438ad11f91122af39e9ffc89e774157f7f14aebe58184ab9b4632"
+      "bytes": 21708,
+      "sha256": "7ec368a3aefedff14af32d94a320c5f57d9c0bea2779272e2d171a4afc1d0fa9"
     },
     {
       "file": "metrics/expanded-quality.json",
-      "bytes": 58933,
-      "sha256": "fe2be9746c4302b829b5aa373c027910cfa84881b80c2068af7c69db76c4659c"
+      "bytes": 64295,
+      "sha256": "7b8936b43bce9d19d9010ab61225a6460160b3aa76c37f5d6df5ba650b433c6c"
     },
     {
       "file": "metrics/figure-evidence.json",
-      "bytes": 16658,
-      "sha256": "45a10a4bf8fac2118fb32103d32cc39d80bd0278c6dbf14b63a14fa7a0b5493f"
+      "bytes": 9339,
+      "sha256": "1fde66bcafc9e602b030e7492a01d72fca510cc5fbd690f4fdef34696db38f23"
     },
     {
       "file": "metrics/materials-provenance.json",
-      "bytes": 736,
-      "sha256": "3792811d69ddcc5e3dfc9ba27faa3950bbc88c269a24f0656ff18f44b3da3cf7"
+      "bytes": 1333,
+      "sha256": "3f052b894ab0780825ef698556483a5900d1e024cfcfaa1e89919261d812adb4"
     },
     {
       "file": "metrics/question-scaling.json",
-      "bytes": 17715,
-      "sha256": "7fcdfa1b22bc3e531e1d7dc296e5ba541be78d67d4fab1d9102f1b946a36c5bb"
+      "bytes": 9036,
+      "sha256": "c74fb9c207f804a53c8f7cb57fd13eb87b6313e334c7ec2b5d7e05df9b4c9d5b"
+    },
+    {
+      "file": "metrics/semantic-consistency.json",
+      "bytes": 20784,
+      "sha256": "5a389236a22e1068be862b2daf88a5caf5a384a9260f0cafbaca9a37b18d8d26"
     },
     {
       "file": "model-card-example.json",
@@ -454,7 +464,9 @@
     "src/decision/model.py": "copy",
     "temperature.json": "copy",
     "tokenizer.json": "copy",
-    "tokenizer_config.json": "copy"
+    "tokenizer_config.json": "copy",
+    "MATERIALS.json": "copy",
+    "metrics/semantic-consistency.json": "copy"
   },
   "scope": "Inference weights, original numerical code, calibrated runtime metadata, wrapper, model card, license, attribution and approved assets only."
 }