Release measured v1.3 update

EVALUATION.md

@@ -1,14 +1,17 @@
 # Evaluation
 
-Four complete panels, shared requests and recorded native decisions. Failures and rejections remain in the requested denominator. The four-panel mean uses 25% each for general decisions, compositional tasks, natural reading, and reading/inference. Within-panel source/family weights are retained. Intervals use the frozen paired source-cluster procedure; they are reported without introducing another release gate.
+Four complete panels contribute one quarter each: general decisions, composition, natural reading, and reading/inference. Original within-panel weights and the frozen paired source-cluster intervals are retained. The mean covers 2,720 decisions; 440 additional native-interface cases are reported separately.
+
+The update passes the predeclared accuracy route against the actual published Nox v1.2. The expanded mean rises from 74.1985% to 75.0266% (+0.828125 percentage points; paired 95% interval +0.2726 to +1.4246). General decisions regress by 0.2083 points; composition, reading and inference improve by 0.7083, 0.9375 and 1.8750 points, respectively. These intervals describe the fixed evaluation sample, not uncertainty across training seeds.
 
 | Model | Version | General decisions | Compositional tasks | Natural reading | Reading and inference | Mean |
 |---|---|---:|---:|---:|---:|---:|
 | Jev | 1.13.0 | 79.10 | **66.38** | **94.53** | **89.79** | **82.45** |
-| Nox | v1.2 · this release | **83.21** | 51.08 | 78.12 | 84.38 | 74.20 |
-| Nox | v1.1 · previous | 82.85 | 51.25 | 78.44 | 79.17 | 72.93 |
+| Nox | v1.3 · this release | 83.00 | 51.79 | 79.06 | 86.25 | 75.03 |
+| Nox | v1.2 · previous | **83.21** | 51.08 | 78.12 | 84.38 | 74.20 |
 | Decider | 2B | 64.01 | 46.58 | 92.03 | 84.38 | 71.75 |
 | Qwen3.5 | 4B · untuned | 69.89 | 43.33 | 87.97 | 79.79 | 70.25 |
+| Sol | v1.3 · published | 73.75 | 46.08 | 76.56 | 84.17 | 70.14 |
 | Qwen3.5 | 2B · untuned | 57.12 | 39.00 | 73.75 | 72.29 | 60.54 |
 | Laya | Upstream default | 57.01 | 37.75 | 51.25 | 63.75 | 52.44 |
 
@@ -17,60 +20,61 @@ Bold marks the exact highest score in a column, including exact ties. Rounded vi
 
 ### General decisions
 
-| Task | Jev · 1.13.0 | Nox · v1.2 · this release | Nox · v1.1 · previous | Decider · 2B | Qwen3.5 · 4B · untuned | Qwen3.5 · 2B · untuned | Laya · Upstream default |
-|---|---:|---:|---:|---:|---:|---:|---:|
-| ag news | 85.16 | 85.16 | 83.59 | 86.72 | 84.38 | 80.47 | **91.41** |
-| boolean constraints | **100.00** | 93.75 | 93.75 | 71.88 | 62.50 | 37.50 | 43.75 |
-| dbpedia 14 | 96.43 | 96.43 | 96.43 | **98.21** | 97.32 | 92.86 | 83.93 |
-| natural intents | **100.00** | **100.00** | **100.00** | **100.00** | **100.00** | 96.88 | 85.94 |
-| option carrier | 30.21 | **100.00** | **100.00** | 8.33 | 72.92 | 9.38 | 95.83 |
-| ordinal rubric | **100.00** | 89.06 | 89.06 | 84.38 | 90.62 | 81.25 | 18.75 |
-| relational composition | **56.25** | 54.17 | 55.21 | 54.17 | 51.04 | 48.96 | 25.00 |
-| scoped evidence | **89.58** | 78.12 | 75.00 | 47.92 | 51.04 | 36.46 | 37.50 |
-| state tracking | 33.33 | **35.42** | **35.42** | 29.17 | 28.12 | 25.00 | 23.96 |
-| unknown rejection | **100.00** | **100.00** | **100.00** | 59.38 | 60.94 | 62.50 | 64.06 |
+| Task | Jev · 1.13.0 | Nox · v1.3 · this release | Nox · v1.2 · previous | Decider · 2B | Qwen3.5 · 4B · untuned | Sol · v1.3 · published | Qwen3.5 · 2B · untuned | Laya · Upstream default |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| ag news | 85.16 | 85.16 | 85.16 | 86.72 | 84.38 | 83.59 | 80.47 | **91.41** |
+| boolean constraints | **100.00** | 93.75 | 93.75 | 71.88 | 62.50 | 50.00 | 37.50 | 43.75 |
+| dbpedia 14 | 96.43 | 96.43 | 96.43 | **98.21** | 97.32 | 95.54 | 92.86 | 83.93 |
+| natural intents | **100.00** | **100.00** | **100.00** | **100.00** | **100.00** | **100.00** | 96.88 | 85.94 |
+| option carrier | 30.21 | **100.00** | **100.00** | 8.33 | 72.92 | 98.96 | 9.38 | 95.83 |
+| ordinal rubric | **100.00** | 89.06 | 89.06 | 84.38 | 90.62 | 65.62 | 81.25 | 18.75 |
+| relational composition | **56.25** | 52.08 | 54.17 | 54.17 | 51.04 | 37.50 | 48.96 | 25.00 |
+| scoped evidence | **89.58** | 78.12 | 78.12 | 47.92 | 51.04 | 79.17 | 36.46 | 37.50 |
+| state tracking | 33.33 | **35.42** | **35.42** | 29.17 | 28.12 | 27.08 | 25.00 | 23.96 |
+| unknown rejection | **100.00** | **100.00** | **100.00** | 59.38 | 60.94 | **100.00** | 62.50 | 64.06 |
 
 ### Compositional tasks
 
-| Task | Jev · 1.13.0 | Nox · v1.2 · this release | Nox · v1.1 · previous | Decider · 2B | Qwen3.5 · 4B · untuned | Qwen3.5 · 2B · untuned | Laya · Upstream default |
-|---|---:|---:|---:|---:|---:|---:|---:|
-| canonical record identity | **76.25** | 50.00 | 50.00 | 56.25 | 50.00 | 46.25 | 47.50 |
-| capacitated assignment | **68.75** | 41.25 | 43.75 | 51.25 | 50.00 | 50.00 | 63.75 |
-| constraint assignment | **55.00** | 41.25 | 42.50 | 23.75 | 23.75 | 21.25 | 22.50 |
-| massive en | **91.67** | 90.83 | **91.67** | 90.00 | **91.67** | 70.83 | 74.17 |
-| massive zh | **88.33** | 87.50 | **88.33** | **88.33** | 86.67 | 74.17 | 73.33 |
-| multiset reconciliation | **58.75** | 28.75 | 31.25 | 23.75 | 27.50 | 27.50 | 26.25 |
-| ordinal service loss | **42.50** | 31.25 | 28.75 | 22.50 | 21.25 | 20.00 | 20.00 |
-| paraconsistent rule closure | **66.25** | 33.75 | 35.00 | 26.25 | 25.00 | 27.50 | 18.75 |
-| temporal exclusion | 41.25 | **45.00** | 41.25 | 42.50 | 31.25 | 28.75 | 12.50 |
-| transaction recovery | **75.00** | 61.25 | 60.00 | 41.25 | 26.25 | 23.75 | 18.75 |
+| Task | Jev · 1.13.0 | Nox · v1.3 · this release | Nox · v1.2 · previous | Decider · 2B | Qwen3.5 · 4B · untuned | Sol · v1.3 · published | Qwen3.5 · 2B · untuned | Laya · Upstream default |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| canonical record identity | **76.25** | 53.75 | 50.00 | 56.25 | 50.00 | 50.00 | 46.25 | 47.50 |
+| capacitated assignment | **68.75** | 46.25 | 41.25 | 51.25 | 50.00 | 47.50 | 50.00 | 63.75 |
+| constraint assignment | **55.00** | 41.25 | 41.25 | 23.75 | 23.75 | 26.25 | 21.25 | 22.50 |
+| massive en | **91.67** | **91.67** | 90.83 | 90.00 | **91.67** | 90.83 | 70.83 | 74.17 |
+| massive zh | **88.33** | 87.50 | 87.50 | **88.33** | 86.67 | 87.50 | 74.17 | 73.33 |
+| multiset reconciliation | **58.75** | 28.75 | 28.75 | 23.75 | 27.50 | 28.75 | 27.50 | 26.25 |
+| ordinal service loss | **42.50** | 30.00 | 31.25 | 22.50 | 21.25 | 25.00 | 20.00 | 20.00 |
+| paraconsistent rule closure | **66.25** | 33.75 | 33.75 | 26.25 | 25.00 | 25.00 | 27.50 | 18.75 |
+| temporal exclusion | 41.25 | 42.50 | **45.00** | 42.50 | 31.25 | 41.25 | 28.75 | 12.50 |
+| transaction recovery | **75.00** | 62.50 | 61.25 | 41.25 | 26.25 | 38.75 | 23.75 | 18.75 |
 
 ### Natural reading
 
-| Task | Jev · 1.13.0 | Nox · v1.2 · this release | Nox · v1.1 · previous | Decider · 2B | Qwen3.5 · 4B · untuned | Qwen3.5 · 2B · untuned | Laya · Upstream default |
-|---|---:|---:|---:|---:|---:|---:|---:|
-| boolq | **92.50** | 85.62 | 87.50 | 91.88 | 83.75 | 65.00 | 69.38 |
-| belebele en | **96.88** | 71.88 | 71.25 | 93.12 | 93.12 | 83.75 | 38.12 |
-| belebele zh | **96.25** | 69.38 | 67.50 | 91.25 | 91.25 | 81.25 | 28.12 |
+| Task | Jev · 1.13.0 | Nox · v1.3 · this release | Nox · v1.2 · previous | Decider · 2B | Qwen3.5 · 4B · untuned | Sol · v1.3 · published | Qwen3.5 · 2B · untuned | Laya · Upstream default |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| boolq | **92.50** | 86.25 | 85.62 | 91.88 | 83.75 | 83.12 | 65.00 | 69.38 |
+| belebele en | **96.88** | 73.12 | 71.88 | 93.12 | 93.12 | 70.00 | 83.75 | 38.12 |
+| belebele zh | **96.25** | 70.62 | 69.38 | 91.25 | 91.25 | 70.00 | 81.25 | 28.12 |
 
 ### Reading and inference
 
-| Task | Jev · 1.13.0 | Nox · v1.2 · this release | Nox · v1.1 · previous | Decider · 2B | Qwen3.5 · 4B · untuned | Qwen3.5 · 2B · untuned | Laya · Upstream default |
-|---|---:|---:|---:|---:|---:|---:|---:|
-| cosmos qa | **86.67** | 68.33 | 57.50 | 66.67 | 60.83 | 56.67 | 30.00 |
-| squad2 answerability | **90.83** | 85.00 | 75.83 | 84.17 | 81.67 | 82.50 | 57.50 |
-| snli | 82.50 | 86.67 | 85.83 | **90.83** | 80.00 | 64.17 | 72.50 |
-| qasc | **99.17** | 97.50 | 97.50 | 95.83 | 96.67 | 85.83 | 95.00 |
+| Task | Jev · 1.13.0 | Nox · v1.3 · this release | Nox · v1.2 · previous | Decider · 2B | Qwen3.5 · 4B · untuned | Sol · v1.3 · published | Qwen3.5 · 2B · untuned | Laya · Upstream default |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| cosmos qa | **86.67** | 70.83 | 68.33 | 66.67 | 60.83 | 69.17 | 56.67 | 30.00 |
+| squad2 answerability | **90.83** | 88.33 | 85.00 | 84.17 | 81.67 | 83.33 | 82.50 | 57.50 |
+| snli | 82.50 | 89.17 | 86.67 | **90.83** | 80.00 | 88.33 | 64.17 | 72.50 |
+| qasc | **99.17** | 96.67 | 97.50 | 95.83 | 96.67 | 95.83 | 85.83 | 95.00 |
 
 ### Probability quality
 
 | Model | Version | Weighted Brier ↓ | Valid probability rows |
 |---|---|---:|---:|
 | Jev | 1.13.0 | **0.2287** | 2,720 / 2,720 |
-| Nox | v1.2 · this release | 0.3416 | 2,720 / 2,720 |
-| Nox | v1.1 · previous | 0.3630 | 2,720 / 2,720 |
+| Nox | v1.3 · this release | 0.3323 | 2,720 / 2,720 |
+| Nox | v1.2 · previous | 0.3416 | 2,720 / 2,720 |
 | Decider | 2B | 0.3535 | 2,720 / 2,720 |
 | Qwen3.5 | 4B · untuned | 0.3996 | 2,720 / 2,720 |
+| Sol | v1.3 · published | 0.4258 | 2,720 / 2,720 |
 | Qwen3.5 | 2B · untuned | 0.5130 | 2,720 / 2,720 |
 | Laya | Upstream default | 0.5903 | 2,720 / 2,720 |
 
@@ -85,8 +89,27 @@ The two native contract supplements are reported separately and are outside this
 
 ![Detailed Reading and inference](assets/decision-expanded-v5.png)
 
-## Scope
+## Probabilities and latency
+
+| Probability measure | Nox v1.2 | Nox v1.3 |
+|---|---:|---:|
+| Raw T=1 four-panel Brier ↓ | 0.342555 | 0.349587 |
+| Shipped, CAL-fitted Brier ↓ | 0.341623 | 0.332289 |
+
+Raw probabilities become less accurate by Brier; the independently fitted global temperature improves the shipped probabilities. Nox v1.3 uses T=1.323135, fitted only on the fixed CAL split after SELECT had locked the weights. The predecessor used T=1.015583. The shipped Brier difference is −0.009334 (paired 95% interval −0.012842 to −0.005978). This is not evidence of uniformly improved raw confidence.
+
+Six fresh timing processes ran candidate and predecessor in the frozen counterbalanced order on one otherwise idle AMD gfx942 GPU. Choice median latency is 32.732 ms at Q=1 and 282.028 ms at Q=32. The six-size geometric latency ratio is 0.99565: approximately unchanged, not a universal speedup. An earlier attempt was interrupted by unrelated background CPU work and excluded in full; none of its partial measurements were reused. [All three types, p95 and methodology](QUESTION-SCALING.md).
+
+## Training and interpretation
+
+This release continues the common Nox v1.2 training checkpoint with the pointer objective and an auxiliary supervised original-chat readout (weight 0.25). The auxiliary path uses the frozen original full vocabulary projection, then selects the legal candidate-letter logits for categorical cross-entropy. It is supervised interface training, not teacher distillation. The vocabulary matrix is neither a trainable parameter nor part of the released inference model.
+
+SELECT chose step 282. The selected continuation inherits 18,048 unique main examples and 16,586,803 main-input tokens; 6,037 auxiliary views reuse those examples rather than adding new examples. Both trial arms completed the full budget, but the later unselected steps are not part of this release. This single-seed selected-product improvement does not establish a general advantage of the auxiliary objective; the matched-budget mechanism comparison is separate.
+
+## Comparator provenance and limitations
+
+The Sol row is the actually published Sol v1.3 at revision `2412d9470d3aa125b346262dad80f6161847aa09`, reused from its completed evaluation on the same four panels. It was appended only to presentation; Nox qualification remains the original candidate-versus-Nox-v1.2 comparison. No new cross-model paired interval is inferred.
 
-These panels include previously observed regression sets. They are not all pristine holdouts. The untuned Qwen models use their frozen chat/LM-head adapters. Laya uses its unmodified upstream default language router (no explicit language override); Decider retains its native adapter. Documented native truncation is retained. Jev is a closed service snapshot, not a reproducible weight release. Full native-interface supplements are outside the quality mean.
+These panels include previously observed regression sets; they are not all pristine holdouts. Untuned Qwen models use the frozen chat/LM-head adapter. Laya uses its unchanged upstream default language router; Decider retains its native adapter. Native truncation is preserved and documented. Jev is a closed-service snapshot, not reproducible released weights. This report does not include the separate V6 finite-probe scores in its mean.
 
-[Exact aggregate statistics](metrics/expanded-quality.json) · [Probability and adapter coverage](metrics/comparator-coverage.json) · [Measured latency](QUESTION-SCALING.md) · [Material provenance](metrics/materials-provenance.json)
+[Exact display statistics](metrics/expanded-quality.json) · [Raw T=1 scores](metrics/uncalibrated-quality.json) · [Comparator coverage](metrics/comparator-coverage.json) · [Material provenance](metrics/materials-provenance.json)

QUESTION-SCALING.md

@@ -10,55 +10,55 @@ The same physical AMD gfx942 GPU runs three independently loaded blocks per mode
 
 | Model | Q | p50 ms ↓ | p95 ms ↓ | Input tokens |
 |---|---:|---:|---:|---:|
-| Nox · v1.2 · this release | 1 | 32.108 | 33.047 | 309 |
-| Nox · v1.1 · previous | 1 | **28.976** | **30.240** | 309 |
-| Nox · v1.2 · this release | 2 | 33.025 | 34.047 | 618 |
-| Nox · v1.1 · previous | 2 | **29.337** | **29.953** | 618 |
-| Nox · v1.2 · this release | 4 | **41.706** | **42.005** | 1236 |
-| Nox · v1.1 · previous | 4 | 42.041 | 42.413 | 1236 |
-| Nox · v1.2 · this release | 8 | **69.483** | **70.719** | 2472 |
-| Nox · v1.1 · previous | 8 | 70.106 | 70.850 | 2472 |
-| Nox · v1.2 · this release | 16 | **141.299** | **141.921** | 4944 |
-| Nox · v1.1 · previous | 16 | 141.369 | 142.338 | 4944 |
-| Nox · v1.2 · this release | 32 | **282.326** | **284.856** | 9888 |
-| Nox · v1.1 · previous | 32 | 283.681 | 286.013 | 9888 |
+| Nox · v1.3 · this release | 1 | **32.732** | **33.474** | 309 |
+| Nox · v1.2 · previous | 1 | 32.977 | 34.752 | 309 |
+| Nox · v1.3 · this release | 2 | **33.319** | **33.560** | 618 |
+| Nox · v1.2 · previous | 2 | 33.589 | 33.905 | 618 |
+| Nox · v1.3 · this release | 4 | **41.779** | **42.215** | 1236 |
+| Nox · v1.2 · previous | 4 | 42.226 | 42.328 | 1236 |
+| Nox · v1.3 · this release | 8 | **69.642** | 70.728 | 2472 |
+| Nox · v1.2 · previous | 8 | 70.052 | **70.707** | 2472 |
+| Nox · v1.3 · this release | 16 | 141.020 | **141.853** | 4944 |
+| Nox · v1.2 · previous | 16 | **140.361** | 142.610 | 4944 |
+| Nox · v1.3 · this release | 32 | 282.028 | 283.684 | 9888 |
+| Nox · v1.2 · previous | 32 | **281.687** | **283.445** | 9888 |
 
 ## Noul
 
 | Model | Q | p50 ms ↓ | p95 ms ↓ | Input tokens |
 |---|---:|---:|---:|---:|
-| Nox · v1.2 · this release | 1 | 32.282 | 32.977 | 262 |
-| Nox · v1.1 · previous | 1 | **28.993** | **29.371** | 262 |
-| Nox · v1.2 · this release | 2 | 32.573 | 33.978 | 524 |
-| Nox · v1.1 · previous | 2 | **29.196** | **30.459** | 524 |
-| Nox · v1.2 · this release | 4 | **38.967** | **39.277** | 1048 |
-| Nox · v1.1 · previous | 4 | 39.373 | 39.573 | 1048 |
-| Nox · v1.2 · this release | 8 | 64.497 | **65.068** | 2096 |
-| Nox · v1.1 · previous | 8 | **64.382** | 65.297 | 2096 |
-| Nox · v1.2 · this release | 16 | **129.421** | **130.165** | 4192 |
-| Nox · v1.1 · previous | 16 | 129.926 | 130.926 | 4192 |
-| Nox · v1.2 · this release | 32 | 260.115 | **261.017** | 8384 |
-| Nox · v1.1 · previous | 32 | **259.700** | 262.418 | 8384 |
+| Nox · v1.3 · this release | 1 | **32.685** | **33.001** | 262 |
+| Nox · v1.2 · previous | 1 | 33.134 | 33.994 | 262 |
+| Nox · v1.3 · this release | 2 | **32.823** | **33.581** | 524 |
+| Nox · v1.2 · previous | 2 | 33.543 | 33.825 | 524 |
+| Nox · v1.3 · this release | 4 | **38.836** | 39.629 | 1048 |
+| Nox · v1.2 · previous | 4 | 39.400 | **39.513** | 1048 |
+| Nox · v1.3 · this release | 8 | **64.279** | **65.077** | 2096 |
+| Nox · v1.2 · previous | 8 | 64.420 | 65.109 | 2096 |
+| Nox · v1.3 · this release | 16 | **129.265** | 130.600 | 4192 |
+| Nox · v1.2 · previous | 16 | 129.363 | **129.752** | 4192 |
+| Nox · v1.3 · this release | 32 | 259.156 | 261.597 | 8384 |
+| Nox · v1.2 · previous | 32 | **259.016** | **260.040** | 8384 |
 
 ## Score
 
 | Model | Q | p50 ms ↓ | p95 ms ↓ | Input tokens |
 |---|---:|---:|---:|---:|
-| Nox · v1.2 · this release | 1 | 32.601 | 33.154 | 307 |
-| Nox · v1.1 · previous | 1 | **28.816** | **29.119** | 307 |
-| Nox · v1.2 · this release | 2 | 33.221 | 33.625 | 614 |
-| Nox · v1.1 · previous | 2 | **29.339** | **29.777** | 614 |
-| Nox · v1.2 · this release | 4 | **41.776** | 42.379 | 1228 |
-| Nox · v1.1 · previous | 4 | 41.965 | **42.220** | 1228 |
-| Nox · v1.2 · this release | 8 | **69.535** | **70.140** | 2456 |
-| Nox · v1.1 · previous | 8 | 69.583 | 70.672 | 2456 |
-| Nox · v1.2 · this release | 16 | **140.447** | **141.969** | 4912 |
-| Nox · v1.1 · previous | 16 | 141.124 | 142.791 | 4912 |
-| Nox · v1.2 · this release | 32 | **281.576** | **284.902** | 9824 |
-| Nox · v1.1 · previous | 32 | 283.684 | 286.097 | 9824 |
+| Nox · v1.3 · this release | 1 | 33.319 | 34.269 | 307 |
+| Nox · v1.2 · previous | 1 | **33.022** | **33.438** | 307 |
+| Nox · v1.3 · this release | 2 | **33.356** | **33.767** | 614 |
+| Nox · v1.2 · previous | 2 | 33.781 | 34.337 | 614 |
+| Nox · v1.3 · this release | 4 | **41.859** | **42.210** | 1228 |
+| Nox · v1.2 · previous | 4 | 41.968 | 42.276 | 1228 |
+| Nox · v1.3 · this release | 8 | 69.734 | 70.124 | 2456 |
+| Nox · v1.2 · previous | 8 | **69.559** | **70.087** | 2456 |
+| Nox · v1.3 · this release | 16 | 141.065 | 141.961 | 4912 |
+| Nox · v1.2 · previous | 16 | **140.220** | **141.315** | 4912 |
+| Nox · v1.3 · this release | 32 | 282.330 | 285.689 | 9824 |
+| Nox · v1.2 · previous | 32 | **280.643** | **284.870** | 9824 |
 
-Choice six-point geometric-mean latency ratio (candidate / predecessor): **1.0337**.
+Choice six-point geometric-mean latency ratio (candidate / predecessor): **0.9956**.
 
-The candidate has 3.37% higher measured latency by this summary. This is a descriptive result for these six request sizes, not a claim of universal speedup.
+The candidate has 0.44% lower measured latency by this summary. This is a descriptive result for these six request sizes, not a claim of universal speedup.
 
 The runtime/profile and temperature belong to these exact measured bundles. Older release timing is not reused.

README.md

@@ -34,15 +34,16 @@ tags:
 
 ## Measured capability
 
-**74.20% four-panel mean**, +1.27 points versus its published predecessor.
+**75.03% four-panel mean**, up from 74.20% in v1.2 (**+0.83 points**, paired 95% interval +0.27 to +1.42). Three panels improve; the general-decision panel falls by 0.21 points. Request latency is approximately unchanged.
 
 | Model | Mean accuracy ↑ | Decisions | Composition | Reading | Inference |
 |---|---:|---:|---:|---:|---:|
 | Jev · 1.13.0 | **82.45** | 79.10 | **66.38** | **94.53** | **89.79** |
-| Nox · v1.2 · this release | 74.20 | **83.21** | 51.08 | 78.12 | 84.38 |
-| Nox · v1.1 · previous | 72.93 | 82.85 | 51.25 | 78.44 | 79.17 |
+| Nox · v1.3 · this release | 75.03 | 83.00 | 51.79 | 79.06 | 86.25 |
+| Nox · v1.2 · previous | 74.20 | **83.21** | 51.08 | 78.12 | 84.38 |
 | Decider · 2B | 71.75 | 64.01 | 46.58 | 92.03 | 84.38 |
 | Qwen3.5 · 4B · untuned | 70.25 | 69.89 | 43.33 | 87.97 | 79.79 |
+| Sol · v1.3 · published | 70.14 | 73.75 | 46.08 | 76.56 | 84.17 |
 | Qwen3.5 · 2B · untuned | 60.54 | 57.12 | 39.00 | 73.75 | 72.29 |
 | Laya · Upstream default | 52.44 | 57.01 | 37.75 | 51.25 | 63.75 |
 
@@ -60,7 +61,7 @@ Same inputs and physical AMD gfx942 GPU; 30 measured requests per point across t
 
 ## Try it
 
-Download `hf download llm-semantic-router/Decision-1.0-Nox --revision v1.2 --local-dir decision-model`, then follow [ROCm setup](RUNTIME.md). In that container, with the model mounted at `/model`:
+Download `hf download llm-semantic-router/Decision-1.0-Nox --revision v1.3 --local-dir decision-model`, then follow [ROCm setup](RUNTIME.md). In that container, with the model mounted at `/model`:
 
 ```python
 from decision import DecisionModel

RUNTIME_BINDING.json

@@ -1,5 +1,5 @@
 {
-  "source_bundle_manifest_sha256": "6d402d6d55e734851cb0d6417d6f3b416b25ae807c3a0d6dcf12e40acb6ac546",
+  "source_bundle_manifest_sha256": "d437bc0149bcb8c9891fbb33c5abc4c2336c3a89ab6cfa0981da5a7f1d19f1f4",
   "profile_sha256": "be32858d15233e0a3fbee0e4257fb02be0b3439deee4eb9c3f61151df7b73850",
   "profile_validation_receipt_sha256": "7a66474325b5575fdd15a33e5e0340c19e809bdcc22adf4cbb45baa0e1b32693",
   "unchanged_files": [
@@ -11,17 +11,17 @@
     {
       "file": "backbone/model-00001-of-00003.safetensors",
       "bytes": 3991295368,
-      "sha256": "5f9c4bc396605b551b5983a1699de8e96755fafdb6afdb228a5cc4d9b438a5da"
+      "sha256": "5ebccc395ef9fa4d61c79894ecefcdaa2a319c1c154221e1ad73663b6f82aacc"
     },
     {
       "file": "backbone/model-00002-of-00003.safetensors",
       "bytes": 3979828128,
-      "sha256": "656cc757ba03f87cedc0e4c88237db1ae8215686f9b6477334a34dc0a32136db"
+      "sha256": "990e2e79fc1ab009df9846ef9fddb31f4b988589bbda84f4728dbaa4196d7fcd"
     },
     {
       "file": "backbone/model-00003-of-00003.safetensors",
       "bytes": 440425856,
-      "sha256": "0a30a07b6efbaa37d016008848c1fa6dadcb46ae451a0e5e4b927ac9b5771804"
+      "sha256": "c47859a3192bdae5003e4732fb911c8dddcf6e8caa167b4971e3a9801b828d64"
     },
     {
       "file": "backbone/model.safetensors.index.json",
@@ -51,12 +51,12 @@
     {
       "file": "decision_head.safetensors",
       "bytes": 10529624,
-      "sha256": "8b8e342445035e503b4963e16c5f7a4fef95c08368e27eace312145fc3660772"
+      "sha256": "9cb6f639714e31bcb76b58eaf94af0b72ac3db9d091ebcda45d9b575efd489de"
     },
     {
       "file": "temperature.json",
-      "bytes": 6095,
-      "sha256": "1d652e75f49b5c232d1e742b189c1c826ad52ced1a931a124e29e65e7d7fb569"
+      "bytes": 6083,
+      "sha256": "69d80e5b215e2c1e6872f146fdb7ea5aa95c9fd678ef96208960d50e4e7b635d"
     },
     {
       "file": "tokenizer.json",

SOURCE_BUNDLE_MANIFEST.json

@@ -10,17 +10,17 @@
     {
       "file": "backbone/model-00001-of-00003.safetensors",
       "bytes": 3991295368,
-      "sha256": "5f9c4bc396605b551b5983a1699de8e96755fafdb6afdb228a5cc4d9b438a5da"
+      "sha256": "5ebccc395ef9fa4d61c79894ecefcdaa2a319c1c154221e1ad73663b6f82aacc"
     },
     {
       "file": "backbone/model-00002-of-00003.safetensors",
       "bytes": 3979828128,
-      "sha256": "656cc757ba03f87cedc0e4c88237db1ae8215686f9b6477334a34dc0a32136db"
+      "sha256": "990e2e79fc1ab009df9846ef9fddb31f4b988589bbda84f4728dbaa4196d7fcd"
     },
     {
       "file": "backbone/model-00003-of-00003.safetensors",
       "bytes": 440425856,
-      "sha256": "0a30a07b6efbaa37d016008848c1fa6dadcb46ae451a0e5e4b927ac9b5771804"
+      "sha256": "c47859a3192bdae5003e4732fb911c8dddcf6e8caa167b4971e3a9801b828d64"
     },
     {
       "file": "backbone/model.safetensors.index.json",
@@ -55,7 +55,7 @@
     {
       "file": "decision_head.safetensors",
       "bytes": 10529624,
-      "sha256": "8b8e342445035e503b4963e16c5f7a4fef95c08368e27eace312145fc3660772"
+      "sha256": "9cb6f639714e31bcb76b58eaf94af0b72ac3db9d091ebcda45d9b575efd489de"
     },
     {
       "file": "runtime.json",
@@ -64,8 +64,8 @@
     },
     {
       "file": "temperature.json",
-      "bytes": 6095,
-      "sha256": "1d652e75f49b5c232d1e742b189c1c826ad52ced1a931a124e29e65e7d7fb569"
+      "bytes": 6083,
+      "sha256": "69d80e5b215e2c1e6872f146fdb7ea5aa95c9fd678ef96208960d50e4e7b635d"
     },
     {
       "file": "tokenizer.json",
@@ -117,37 +117,37 @@
     {
       "file": "backbone/model-00001-of-00005.safetensors",
       "bytes": 3992269864,
-      "sha256": "8adb3dbf9db30b3ecd1f29ce6a853bc6d7e3f465857471cfe579c570537250f8"
+      "sha256": "d65565b8988a20b07988db09ed2321111e57b4fe656112fe4636715f8418c40b"
     },
     {
       "file": "backbone/model-00002-of-00005.safetensors",
       "bytes": 3990302320,
-      "sha256": "8aa80bed34a466a7208ad177d5e5742773e4140c55cba69911fcc805633afa5e"
+      "sha256": "99197c2df234da167190a18b34efe6275622687500346ad3d42d9fad6b25d52d"
     },
     {
       "file": "backbone/model-00003-of-00005.safetensors",
       "bytes": 3937871784,
-      "sha256": "9af681e845c99e0d933dc3fa643271ee9f5bb382f98454ceb41eb1faf0979f87"
+      "sha256": "0d0af48a06e40b0afdcb7aa02cf6d8d6df885ccc9e2593e3db72cd3a92e798a7"
     },
     {
       "file": "backbone/model-00004-of-00005.safetensors",
       "bytes": 3926578128,
-      "sha256": "34de5c39c2338e1c7110beaf2834c856f3830411002893f7946045c0a083cc92"
+      "sha256": "b960486d5d833e016c9e2a32d42214db14a8e48afef49d61a7f514f8c636a264"
     },
     {
       "file": "backbone/model-00005-of-00005.safetensors",
       "bytes": 976029360,
-      "sha256": "ce20083dd8deb786b14fe1126a12cdbf678a8a3c25367a6676c0ca55d9ae7980"
+      "sha256": "994dac35c408a89d89768febad1ed52a255da62f7924f86bd76371381e9cc751"
     },
     {
       "file": "decision_config.json",
-      "bytes": 1110,
-      "sha256": "a12bd49b459771f22d22f29c59fe3cb6779c52a5d39b40cdbe5e247c5181b3fc"
+      "bytes": 1070,
+      "sha256": "3324b9318a873fc27a8610e19d6d7ba82b0890a49011abbc8b097bdf429e6747"
     },
     {
       "file": "decision_head.safetensors",
       "bytes": 10529624,
-      "sha256": "8b8e342445035e503b4963e16c5f7a4fef95c08368e27eace312145fc3660772"
+      "sha256": "9cb6f639714e31bcb76b58eaf94af0b72ac3db9d091ebcda45d9b575efd489de"
     },
     {
       "file": "tokenizer.json",
@@ -163,8 +163,8 @@
   "source_model_code_sha256": "d3e28489c09f3bd7130e2d43d92e0b5c4a08e25b09b21303904defb0ff1c3646",
   "source_api_code_sha256": "147b2fec32cbbbbb1b92cf2a19bb887f9d945e4974e141ca7ea93b8c542f5b21",
   "dev_sha256": "45b4cd46acc4be53b95c7ed8f333b3533972296a4d0557c7b8c48c9cab6ced61",
-  "production_predictions_sha256": "cf0b1a17008bddc2e3e18289866993770c41d4a5af5f6f4219508bb34fcabefa",
-  "temperature_sha256": "1d652e75f49b5c232d1e742b189c1c826ad52ced1a931a124e29e65e7d7fb569",
+  "production_predictions_sha256": "ba311f5a6bc73182651bdf5c9744f0ed95921110501ddc93dcfd20a1d06bceff",
+  "temperature_sha256": "69d80e5b215e2c1e6872f146fdb7ea5aa95c9fd678ef96208960d50e4e7b635d",
   "runtime_sha256": "c5d3521358b2817f4e56ea8150c5c612139b4e2b2c5bb09f512d9a7c0b5298b9",
   "production_batch_size": 8,
   "input_length_limit": 16384,

backbone/model-00001-of-00003.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5f9c4bc396605b551b5983a1699de8e96755fafdb6afdb228a5cc4d9b438a5da
+oid sha256:5ebccc395ef9fa4d61c79894ecefcdaa2a319c1c154221e1ad73663b6f82aacc
 size 3991295368

backbone/model-00002-of-00003.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:656cc757ba03f87cedc0e4c88237db1ae8215686f9b6477334a34dc0a32136db
+oid sha256:990e2e79fc1ab009df9846ef9fddb31f4b988589bbda84f4728dbaa4196d7fcd
 size 3979828128

backbone/model-00003-of-00003.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0a30a07b6efbaa37d016008848c1fa6dadcb46ae451a0e5e4b927ac9b5771804
+oid sha256:c47859a3192bdae5003e4732fb911c8dddcf6e8caa167b4971e3a9801b828d64
 size 440425856

bundle-manifest.json

@@ -10,12 +10,12 @@
     {
       "file": "RUNTIME_BINDING.json",
       "bytes": 2621,
-      "sha256": "65a7772ed57e811faefe6b3b3868e174c5e4a3d7f3aa0138a3838464bb2612fe"
+      "sha256": "fb76d9fc9147e15a678eb91dfc40d7039845a4eaebdecce041a5c9e76c3d3e0f"
     },
     {
       "file": "SOURCE_BUNDLE_MANIFEST.json",
       "bytes": 5637,
-      "sha256": "6d402d6d55e734851cb0d6417d6f3b416b25ae807c3a0d6dcf12e40acb6ac546"
+      "sha256": "d437bc0149bcb8c9891fbb33c5abc4c2336c3a89ab6cfa0981da5a7f1d19f1f4"
     },
     {
       "file": "backbone/config.json",
@@ -25,17 +25,17 @@
     {
       "file": "backbone/model-00001-of-00003.safetensors",
       "bytes": 3991295368,
-      "sha256": "5f9c4bc396605b551b5983a1699de8e96755fafdb6afdb228a5cc4d9b438a5da"
+      "sha256": "5ebccc395ef9fa4d61c79894ecefcdaa2a319c1c154221e1ad73663b6f82aacc"
     },
     {
       "file": "backbone/model-00002-of-00003.safetensors",
       "bytes": 3979828128,
-      "sha256": "656cc757ba03f87cedc0e4c88237db1ae8215686f9b6477334a34dc0a32136db"
+      "sha256": "990e2e79fc1ab009df9846ef9fddb31f4b988589bbda84f4728dbaa4196d7fcd"
     },
     {
       "file": "backbone/model-00003-of-00003.safetensors",
       "bytes": 440425856,
-      "sha256": "0a30a07b6efbaa37d016008848c1fa6dadcb46ae451a0e5e4b927ac9b5771804"
+      "sha256": "c47859a3192bdae5003e4732fb911c8dddcf6e8caa167b4971e3a9801b828d64"
     },
     {
       "file": "backbone/model.safetensors.index.json",
@@ -80,7 +80,7 @@
     {
       "file": "decision_head.safetensors",
       "bytes": 10529624,
-      "sha256": "8b8e342445035e503b4963e16c5f7a4fef95c08368e27eace312145fc3660772"
+      "sha256": "9cb6f639714e31bcb76b58eaf94af0b72ac3db9d091ebcda45d9b575efd489de"
     },
     {
       "file": "pyproject.toml",
@@ -119,8 +119,8 @@
     },
     {
       "file": "temperature.json",
-      "bytes": 6095,
-      "sha256": "1d652e75f49b5c232d1e742b189c1c826ad52ced1a931a124e29e65e7d7fb569"
+      "bytes": 6083,
+      "sha256": "69d80e5b215e2c1e6872f146fdb7ea5aa95c9fd678ef96208960d50e4e7b635d"
     },
     {
       "file": "tokenizer.json",
@@ -172,37 +172,37 @@
     {
       "file": "backbone/model-00001-of-00005.safetensors",
       "bytes": 3992269864,
-      "sha256": "8adb3dbf9db30b3ecd1f29ce6a853bc6d7e3f465857471cfe579c570537250f8"
+      "sha256": "d65565b8988a20b07988db09ed2321111e57b4fe656112fe4636715f8418c40b"
     },
     {
       "file": "backbone/model-00002-of-00005.safetensors",
       "bytes": 3990302320,
-      "sha256": "8aa80bed34a466a7208ad177d5e5742773e4140c55cba69911fcc805633afa5e"
+      "sha256": "99197c2df234da167190a18b34efe6275622687500346ad3d42d9fad6b25d52d"
     },
     {
       "file": "backbone/model-00003-of-00005.safetensors",
       "bytes": 3937871784,
-      "sha256": "9af681e845c99e0d933dc3fa643271ee9f5bb382f98454ceb41eb1faf0979f87"
+      "sha256": "0d0af48a06e40b0afdcb7aa02cf6d8d6df885ccc9e2593e3db72cd3a92e798a7"
     },
     {
       "file": "backbone/model-00004-of-00005.safetensors",
       "bytes": 3926578128,
-      "sha256": "34de5c39c2338e1c7110beaf2834c856f3830411002893f7946045c0a083cc92"
+      "sha256": "b960486d5d833e016c9e2a32d42214db14a8e48afef49d61a7f514f8c636a264"
     },
     {
       "file": "backbone/model-00005-of-00005.safetensors",
       "bytes": 976029360,
-      "sha256": "ce20083dd8deb786b14fe1126a12cdbf678a8a3c25367a6676c0ca55d9ae7980"
+      "sha256": "994dac35c408a89d89768febad1ed52a255da62f7924f86bd76371381e9cc751"
     },
     {
       "file": "decision_config.json",
-      "bytes": 1110,
-      "sha256": "a12bd49b459771f22d22f29c59fe3cb6779c52a5d39b40cdbe5e247c5181b3fc"
+      "bytes": 1070,
+      "sha256": "3324b9318a873fc27a8610e19d6d7ba82b0890a49011abbc8b097bdf429e6747"
     },
     {
       "file": "decision_head.safetensors",
       "bytes": 10529624,
-      "sha256": "8b8e342445035e503b4963e16c5f7a4fef95c08368e27eace312145fc3660772"
+      "sha256": "9cb6f639714e31bcb76b58eaf94af0b72ac3db9d091ebcda45d9b575efd489de"
     },
     {
       "file": "tokenizer.json",
@@ -218,14 +218,14 @@
   "source_model_code_sha256": "d3e28489c09f3bd7130e2d43d92e0b5c4a08e25b09b21303904defb0ff1c3646",
   "source_api_code_sha256": "1b068eccdffd3c3b67bfa52f8f526e6b482d92551c927668ad28a794767f8a40",
   "dev_sha256": "45b4cd46acc4be53b95c7ed8f333b3533972296a4d0557c7b8c48c9cab6ced61",
-  "production_predictions_sha256": "cf0b1a17008bddc2e3e18289866993770c41d4a5af5f6f4219508bb34fcabefa",
-  "temperature_sha256": "1d652e75f49b5c232d1e742b189c1c826ad52ced1a931a124e29e65e7d7fb569",
+  "production_predictions_sha256": "ba311f5a6bc73182651bdf5c9744f0ed95921110501ddc93dcfd20a1d06bceff",
+  "temperature_sha256": "69d80e5b215e2c1e6872f146fdb7ea5aa95c9fd678ef96208960d50e4e7b635d",
   "runtime_sha256": "78a2c21f8138beeffb3e6b1c11e03cb48972a78ca5ef3d1f58c50651031c34b6",
   "production_batch_size": 8,
   "input_length_limit": 16384,
   "original_checkpoint_name": "winner",
   "no_publication_performed": true,
-  "source_bundle_manifest_sha256": "6d402d6d55e734851cb0d6417d6f3b416b25ae807c3a0d6dcf12e40acb6ac546",
+  "source_bundle_manifest_sha256": "d437bc0149bcb8c9891fbb33c5abc4c2336c3a89ab6cfa0981da5a7f1d19f1f4",
   "normalization_profile_sha256": "be32858d15233e0a3fbee0e4257fb02be0b3439deee4eb9c3f61151df7b73850",
   "public_wrapper_included": true
 }

decision_head.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8b8e342445035e503b4963e16c5f7a4fef95c08368e27eace312145fc3660772
+oid sha256:9cb6f639714e31bcb76b58eaf94af0b72ac3db9d091ebcda45d9b575efd489de
 size 10529624

metrics/comparator-coverage.json

@@ -436,5 +436,52 @@
       },
       "runtime_fingerprint_sha256": "b5af3d997c95d0a6d2088a64f341726fc0daee5eebc730f851b2b65bf0de8d1b"
     }
+  },
+  "Sol-composition-selected": {
+    "source": "Previously completed same-panel public release evaluation; aggregate display only.",
+    "repo_id": "llm-semantic-router/Decision-1.0-Sol",
+    "model_revision": "2412d9470d3aa125b346262dad80f6161847aa09",
+    "bundle_manifest_sha256": "6f0970bfbe5594feecffc14b4824a92080ef09318e12567bd22264c3844e2fc4",
+    "evaluation_receipt_sha256": "5cf55e670d407aaafd727d29a465889fda252df7e3e98f330f0a2b8c88d87fe7",
+    "statistics_sha256": "fe2be9746c4302b829b5aa373c027910cfa84881b80c2068af7c69db76c4659c",
+    "panel_registry_sha256": "1aefdf58d56e832e765194d0fa53a8647acea7d2ad2fc25ac95380bf85949fd2",
+    "predictions_sha256": {
+      "old_core": {
+        "raw": "75cebaae4bbc1403ff38caf9a4abeb61e4268796fdca8319481b539554b380d0",
+        "normalized": "5fc0b3e49a5de576f7621839130855c178b12d942dc357aaf6b4dd9735e0a6a8",
+        "raw_T1": "a4b53f6798ffbda38a1f90d502975d38c3fb4b0050181c2a5d1e2fcf05c7751e",
+        "complete": "a8dcc60410176c68792431b14c4bd5bc12fe95a8c2ac6458567dbd50a47aac96"
+      },
+      "v3_core": {
+        "raw": "2709c606bdce40d5209ca281378d61f6d4607a642e4ad410d8f77acefda6de81",
+        "normalized": "b2c99fbd739b849a66a55929a658de4517a7ffaa4496255566e6972ba28d4f15",
+        "raw_T1": "773480ad99f50a47ac6d60bf1d5ba8b0897793fb772ea103b542b7083c3dad0c",
+        "complete": "3b9ca11254c52e193682a019f99d0c95bc7417797618782d5a42f310a4995cc2"
+      },
+      "v4": {
+        "raw": "b58f33eacbd211c60f1e4f06a46dc36b0108d849c05170cab00ba66121fc5762",
+        "normalized": "3c28302750e02d4cead0aacd66a323734e773f726c4c41db658dd6461251b28b",
+        "raw_T1": "c09d07fbcf11c04b7b03b3b7024c78891cba2d821f7708ca87583433231d4a07",
+        "complete": "d3604b83892fb7b7203704b83c60d73dcb0fb12b13812acfd0441df1b4356d7c"
+      },
+      "v5": {
+        "raw": "3071a4f20ec67fe1d7384d7bd306d66b0bedfa235fcb802aee2f884e234d5e5b",
+        "normalized": "6ddb0496012b596ea122afa4db4c4158e57dfb934e9ae29eb006756b652e31e1",
+        "raw_T1": "acd11aaab8598361858d0a7717c5cc50f8e748b7f265c2a9611949b10a553b9b",
+        "complete": "b3e2b0e381e6a393a45c594434a0643a57e7424dc426b739c4bcf24f8eca8f5a"
+      },
+      "old_native": {
+        "raw": "2249d3dfb5c1de38abecbc663ba3dadfc3f99565012cf1fb686cc087d77a0151",
+        "normalized": "76ca55c504438160bdfc0f5952946da91943c5c51dd030c3d8ae3f41d50009cd",
+        "raw_T1": "acc85f0a77a9fc8aa8f46645591149047399eda09b7eb4630f25e1d3d60ae918",
+        "complete": "d77f1bdda20f21f70876713843e64fca15b2695ac7883c4fa1ef0ab76a8799a5"
+      },
+      "v3_native": {
+        "raw": "dc9ca7279980fd92178199b2ca4f44eccd8bf4a3736d4841495ee93836c1fafc",
+        "normalized": "86d970bb0073b0188fc5b9100c18f9117850e748f5887b23d0cbe79489033cf0",
+        "raw_T1": "de0672e4fc1d7492ea571a592d8866c486643d87ef515f5e0dfc0039e50453f8",
+        "complete": "421f709c868c78558d299df84efbe1abc519d38d0035cbb62e53de2d9a2ae6c6"
+      }
+    }
   }
 }

metrics/expanded-quality.json

@@ -969,53 +969,53 @@
         ]
       }
     },
-    "Nox-natural-selected": {
+    "Nox-retention-selected": {
       "old_core": {
-        "point": 0.8321056547619048,
-        "exact_fraction": "22367/26880",
+        "point": 0.8300223214285715,
+        "exact_fraction": "7437/8960",
         "ci95": [
-          0.799775855654762,
-          0.8639146205357143
+          0.7974693080357143,
+          0.862016369047619
         ]
       },
       "v3_core": {
-        "point": 0.5108333333333334,
-        "exact_fraction": "613/1200",
+        "point": 0.5179166666666667,
+        "exact_fraction": "1243/2400",
         "ci95": [
-          0.48540624999999993,
-          0.5366666666666667
+          0.4920833333333333,
+          0.545
         ]
       },
       "v4": {
-        "point": 0.78125,
-        "exact_fraction": "25/32",
+        "point": 0.790625,
+        "exact_fraction": "253/320",
         "ci95": [
-          0.7348237223129993,
-          0.825
+          0.7453125,
+          0.8331463068181818
         ]
       },
       "v5": {
-        "point": 0.84375,
-        "exact_fraction": "27/32",
+        "point": 0.8625,
+        "exact_fraction": "69/80",
         "ci95": [
-          0.8125286977325981,
-          0.8745048035497992
+          0.8321482316646345,
+          0.8915620903779563
         ]
       },
       "v3_joint": {
-        "point": 0.671469494047619,
-        "exact_fraction": "180491/268800",
+        "point": 0.6739694940476191,
+        "exact_fraction": "181163/268800",
         "ci95": [
-          0.6504053199404762,
-          0.6917820870535715
+          0.652767857142857,
+          0.6950187872023811
         ]
       },
       "expanded_joint": {
-        "point": 0.7419847470238096,
-        "exact_fraction": "398891/537600",
+        "point": 0.7502659970238095,
+        "exact_fraction": "403343/537600",
         "ci95": [
-          0.7246839089844096,
-          0.7585761485522727
+          0.7330469621683191,
+          0.7670605596068275
         ]
       },
       "families": {
@@ -1069,11 +1069,11 @@
             ]
           },
           "relational_composition": {
-            "correct": 52,
+            "correct": 50,
             "requested": 96,
             "accuracy_ci95": [
-              0.375,
-              0.7083333333333334
+              0.34375,
+              0.6979166666666666
             ]
           },
           "scoped_evidence": {
@@ -1103,19 +1103,19 @@
         },
         "v3_core": {
           "canonical_record_identity": {
-            "correct": 40,
+            "correct": 43,
             "requested": 80,
             "accuracy_ci95": [
               0.5,
-              0.5
+              0.575
             ]
           },
           "capacitated_assignment": {
-            "correct": 33,
+            "correct": 37,
             "requested": 80,
             "accuracy_ci95": [
-              0.35,
-              0.475
+              0.4125,
+              0.5125
             ]
           },
           "constraint_assignment": {
@@ -1127,11 +1127,11 @@
             ]
           },
           "massive_en": {
-            "correct": 109,
+            "correct": 110,
             "requested": 120,
             "accuracy_ci95": [
-              0.8581250000000001,
-              0.9583333333333333
+              0.8666666666666667,
+              0.9583333333333334
             ]
           },
           "massive_zh": {
@@ -1151,11 +1151,11 @@
             ]
           },
           "ordinal_service_loss": {
-            "correct": 25,
+            "correct": 24,
             "requested": 80,
             "accuracy_ci95": [
-              0.2375,
-              0.3625
+              0.225,
+              0.375
             ]
           },
           "paraconsistent_rule_closure": {
@@ -1167,89 +1167,89 @@
             ]
           },
           "temporal_exclusion": {
-            "correct": 36,
+            "correct": 34,
             "requested": 80,
             "accuracy_ci95": [
               0.3,
-              0.5875
+              0.55
             ]
           },
           "transaction_recovery": {
-            "correct": 49,
+            "correct": 50,
             "requested": 80,
             "accuracy_ci95": [
-              0.5125,
-              0.7125
+              0.4875,
+              0.7625
             ]
           }
         },
         "v4": {
           "boolq": {
-            "correct": 137,
+            "correct": 138,
             "requested": 160,
             "accuracy_ci95": [
-              0.8,
-              0.90625
+              0.80625,
+              0.9125
             ]
           },
           "belebele_en": {
-            "correct": 115,
+            "correct": 117,
             "requested": 160,
             "accuracy_ci95": [
-              0.6441717791411042,
-              0.7922077922077922
+              0.6585365853658537,
+              0.802547770700637
             ]
           },
           "belebele_zh": {
-            "correct": 111,
+            "correct": 113,
             "requested": 160,
             "accuracy_ci95": [
-              0.61875,
-              0.7664670658682635
+              0.632258064516129,
+              0.779874213836478
             ]
           }
         },
         "v5": {
           "cosmos_qa": {
-            "correct": 82,
+            "correct": 85,
             "requested": 120,
             "accuracy_ci95": [
-              0.6,
-              0.7666666666666667
+              0.625,
+              0.7833333333333333
             ]
           },
           "squad2_answerability": {
-            "correct": 102,
+            "correct": 106,
             "requested": 120,
             "accuracy_ci95": [
-              0.7829998478393184,
-              0.9137931034482759
+              0.8214285714285714,
+              0.9407026836158189
             ]
           },
           "snli": {
-            "correct": 104,
+            "correct": 107,
             "requested": 120,
             "accuracy_ci95": [
-              0.8,
-              0.925
+              0.8333333333333334,
+              0.9416666666666667
             ]
           },
           "qasc": {
-            "correct": 117,
+            "correct": 116,
             "requested": 120,
             "accuracy_ci95": [
-              0.9435483870967742,
-              1.0
+              0.9327731092436975,
+              0.991869918699187
             ]
           }
         }
       },
       "proper_scores": {
-        "old_core": 0.22574338686556997,
-        "v3_core": 0.6216883113559425,
-        "v4": 0.3087204159143027,
-        "v5": 0.21034073450197993,
-        "expanded_weighted_brier": 0.3416232121594488
+        "old_core": 0.22558726136256252,
+        "v3_core": 0.6082926735018094,
+        "v4": 0.3013823815596702,
+        "v5": 0.19389530242905464,
+        "expanded_weighted_brier": 0.3322894047132742
       },
       "probability_coverage": {
         "old_core": {
@@ -1271,84 +1271,84 @@
       },
       "brier_ci95": {
         "old_core": [
-          0.18384172890538428,
-          0.26968518800214547
+          0.1846602761421124,
+          0.2688592667224405
         ],
         "v3_core": [
-          0.594235087513965,
-          0.6497064829839175
+          0.5803874232294962,
+          0.6368345885571338
         ],
         "v4": [
-          0.24847257992866453,
-          0.37369416170641057
+          0.24180651680296794,
+          0.3645626677453216
         ],
         "v5": [
-          0.1763606748543631,
-          0.24678438691822258
+          0.16179407555902875,
+          0.22847239437277253
         ],
         "expanded_weighted_brier": [
-          0.3200581388236899,
-          0.3637875894103542
+          0.3111201217342611,
+          0.35396540259832066
         ]
       }
     },
-    "Nox-v1.1-released": {
+    "Nox-v1.2-released": {
       "old_core": {
-        "point": 0.8284598214285714,
-        "exact_fraction": "7423/8960",
+        "point": 0.8321056547619048,
+        "exact_fraction": "22367/26880",
         "ci95": [
-          0.7947907366071428,
-          0.861123511904762
+          0.799775855654762,
+          0.8639146205357143
         ]
       },
       "v3_core": {
-        "point": 0.5125,
-        "exact_fraction": "41/80",
+        "point": 0.5108333333333334,
+        "exact_fraction": "613/1200",
         "ci95": [
-          0.48708333333333337,
-          0.5383333333333333
+          0.48540624999999993,
+          0.5366666666666667
         ]
       },
       "v4": {
-        "point": 0.784375,
-        "exact_fraction": "251/320",
+        "point": 0.78125,
+        "exact_fraction": "25/32",
         "ci95": [
-          0.7390625,
-          0.8279707260614273
+          0.7348237223129993,
+          0.825
         ]
       },
       "v5": {
-        "point": 0.7916666666666666,
-        "exact_fraction": "19/24",
+        "point": 0.84375,
+        "exact_fraction": "27/32",
         "ci95": [
-          0.7571779119014277,
-          0.8248930921052631
+          0.8125286977325981,
+          0.8745048035497992
         ]
       },
       "v3_joint": {
-        "point": 0.6704799107142857,
-        "exact_fraction": "2403/3584",
+        "point": 0.671469494047619,
+        "exact_fraction": "180491/268800",
         "ci95": [
-          0.6491663876488095,
-          0.6914181547619047
+          0.6504053199404762,
+          0.6917820870535715
         ]
       },
       "expanded_joint": {
-        "point": 0.7292503720238095,
-        "exact_fraction": "78409/107520",
+        "point": 0.7419847470238096,
+        "exact_fraction": "398891/537600",
         "ci95": [
-          0.7118337656360736,
-          0.746332556571124
+          0.7246839089844096,
+          0.7585761485522727
         ]
       },
       "families": {
         "old_core": {
           "ag_news": {
-            "correct": 107,
+            "correct": 109,
             "requested": 128,
             "accuracy_ci95": [
-              0.7734375,
-              0.8984375
+              0.7890625,
+              0.90625
             ]
           },
           "boolean_constraints": {
@@ -1392,19 +1392,19 @@
             ]
           },
           "relational_composition": {
-            "correct": 53,
+            "correct": 52,
             "requested": 96,
             "accuracy_ci95": [
               0.375,
-              0.7291666666666666
+              0.7083333333333334
             ]
           },
           "scoped_evidence": {
-            "correct": 72,
+            "correct": 75,
             "requested": 96,
             "accuracy_ci95": [
-              0.5833333333333334,
-              0.9166666666666666
+              0.6145833333333334,
+              0.9270833333333334
             ]
           },
           "state_tracking": {
@@ -1434,127 +1434,127 @@
             ]
           },
           "capacitated_assignment": {
-            "correct": 35,
+            "correct": 33,
             "requested": 80,
             "accuracy_ci95": [
-              0.3875,
+              0.35,
               0.475
             ]
           },
           "constraint_assignment": {
-            "correct": 34,
+            "correct": 33,
             "requested": 80,
             "accuracy_ci95": [
-              0.3625,
-              0.4875
+              0.3375,
+              0.475
             ]
           },
           "massive_en": {
-            "correct": 110,
+            "correct": 109,
             "requested": 120,
             "accuracy_ci95": [
-              0.8666666666666667,
-              0.9583333333333334
+              0.8581250000000001,
+              0.9583333333333333
             ]
           },
           "massive_zh": {
-            "correct": 106,
+            "correct": 105,
             "requested": 120,
             "accuracy_ci95": [
-              0.825,
-              0.9333333333333333
+              0.8166666666666667,
+              0.925
             ]
           },
           "multiset_reconciliation": {
-            "correct": 25,
+            "correct": 23,
             "requested": 80,
             "accuracy_ci95": [
-              0.2375,
-              0.425
+              0.2125,
+              0.375
             ]
           },
           "ordinal_service_loss": {
-            "correct": 23,
+            "correct": 25,
             "requested": 80,
             "accuracy_ci95": [
-              0.225,
-              0.35
+              0.2375,
+              0.3625
             ]
           },
           "paraconsistent_rule_closure": {
-            "correct": 28,
+            "correct": 27,
             "requested": 80,
             "accuracy_ci95": [
-              0.2875,
-              0.4375
+              0.275,
+              0.425
             ]
           },
           "temporal_exclusion": {
-            "correct": 33,
+            "correct": 36,
             "requested": 80,
             "accuracy_ci95": [
-              0.2625,
-              0.5625
+              0.3,
+              0.5875
             ]
           },
           "transaction_recovery": {
-            "correct": 48,
+            "correct": 49,
             "requested": 80,
             "accuracy_ci95": [
-              0.5,
-              0.7
+              0.5125,
+              0.7125
             ]
           }
         },
         "v4": {
           "boolq": {
-            "correct": 140,
+            "correct": 137,
             "requested": 160,
             "accuracy_ci95": [
-              0.81875,
-              0.925
+              0.8,
+              0.90625
             ]
           },
           "belebele_en": {
-            "correct": 114,
+            "correct": 115,
             "requested": 160,
             "accuracy_ci95": [
-              0.6387057909055577,
-              0.7848217752582121
+              0.6441717791411042,
+              0.7922077922077922
             ]
           },
           "belebele_zh": {
-            "correct": 108,
+            "correct": 111,
             "requested": 160,
             "accuracy_ci95": [
-              0.5999709302325581,
-              0.7484662576687117
+              0.61875,
+              0.7664670658682635
             ]
           }
         },
         "v5": {
           "cosmos_qa": {
-            "correct": 69,
+            "correct": 82,
             "requested": 120,
             "accuracy_ci95": [
-              0.48333333333333334,
-              0.6583333333333333
+              0.6,
+              0.7666666666666667
             ]
           },
           "squad2_answerability": {
-            "correct": 91,
+            "correct": 102,
             "requested": 120,
             "accuracy_ci95": [
-              0.6833333333333333,
-              0.8305084745762712
+              0.7829998478393184,
+              0.9137931034482759
             ]
           },
           "snli": {
-            "correct": 103,
+            "correct": 104,
             "requested": 120,
             "accuracy_ci95": [
-              0.7916666666666666,
-              0.9166666666666666
+              0.8,
+              0.925
             ]
           },
           "qasc": {
@@ -1568,11 +1568,11 @@
         }
       },
       "proper_scores": {
-        "old_core": 0.2331125387762136,
-        "v3_core": 0.6105027919948053,
-        "v4": 0.3216080833637166,
-        "v5": 0.2866349864374189,
-        "expanded_weighted_brier": 0.3629646001430386
+        "old_core": 0.22574338686556997,
+        "v3_core": 0.6216883113559425,
+        "v4": 0.3087204159143027,
+        "v5": 0.21034073450197993,
+        "expanded_weighted_brier": 0.3416232121594488
       },
       "probability_coverage": {
         "old_core": {
@@ -1594,24 +1594,24 @@
       },
       "brier_ci95": {
         "old_core": [
-          0.19013891719456238,
-          0.2779526850258884
+          0.18384172890538428,
+          0.26968518800214547
         ],
         "v3_core": [
-          0.5846362480182131,
-          0.637147854744837
+          0.594235087513965,
+          0.6497064829839175
         ],
         "v4": [
-          0.2589502553113817,
-          0.38874975412458873
+          0.24847257992866453,
+          0.37369416170641057
         ],
         "v5": [
-          0.24724384093998833,
-          0.3276631365833609
+          0.1763606748543631,
+          0.24678438691822258
         ],
         "expanded_weighted_brier": [
-          0.340384554547154,
-          0.3860851198040383
+          0.3200581388236899,
+          0.3637875894103542
         ]
       }
     },
@@ -2260,94 +2260,417 @@
           0.4193732417566481
         ]
       }
+    },
+    "Sol-composition-selected": {
+      "old_core": {
+        "point": 0.7374627976190476,
+        "exact_fraction": "19823/26880",
+        "ci95": [
+          0.6992931547619048,
+          0.7742950148809524
+        ]
+      },
+      "v3_core": {
+        "point": 0.4608333333333333,
+        "exact_fraction": "553/1200",
+        "ci95": [
+          0.43875000000000003,
+          0.48291666666666666
+        ]
+      },
+      "v4": {
+        "point": 0.765625,
+        "exact_fraction": "49/64",
+        "ci95": [
+          0.7210721817484662,
+          0.8086081288343558
+        ]
+      },
+      "v5": {
+        "point": 0.8416666666666667,
+        "exact_fraction": "101/120",
+        "ci95": [
+          0.8111792524213076,
+          0.8710143196177259
+        ]
+      },
+      "v3_joint": {
+        "point": 0.5991480654761905,
+        "exact_fraction": "161051/268800",
+        "ci95": [
+          0.5773286830357143,
+          0.6207081473214285
+        ]
+      },
+      "expanded_joint": {
+        "point": 0.7013969494047619,
+        "exact_fraction": "377071/537600",
+        "ci95": [
+          0.6845734141379205,
+          0.7180071456587547
+        ]
+      },
+      "families": {
+        "old_core": {
+          "ag_news": {
+            "correct": 107,
+            "requested": 128,
+            "accuracy_ci95": [
+              0.7734375,
+              0.8984375
+            ]
+          },
+          "boolean_constraints": {
+            "correct": 32,
+            "requested": 64,
+            "accuracy_ci95": [
+              0.328125,
+              0.671875
+            ]
+          },
+          "dbpedia_14": {
+            "correct": 107,
+            "requested": 112,
+            "accuracy_ci95": [
+              0.9107142857142857,
+              0.9910714285714286
+            ]
+          },
+          "natural_intents": {
+            "correct": 64,
+            "requested": 64,
+            "accuracy_ci95": [
+              1.0,
+              1.0
+            ]
+          },
+          "option_carrier": {
+            "correct": 95,
+            "requested": 96,
+            "accuracy_ci95": [
+              0.96875,
+              1.0
+            ]
+          },
+          "ordinal_rubric": {
+            "correct": 42,
+            "requested": 64,
+            "accuracy_ci95": [
+              0.484375,
+              0.8125
+            ]
+          },
+          "relational_composition": {
+            "correct": 36,
+            "requested": 96,
+            "accuracy_ci95": [
+              0.19791666666666666,
+              0.5625
+            ]
+          },
+          "scoped_evidence": {
+            "correct": 76,
+            "requested": 96,
+            "accuracy_ci95": [
+              0.6354166666666666,
+              0.9375
+            ]
+          },
+          "state_tracking": {
+            "correct": 26,
+            "requested": 96,
+            "accuracy_ci95": [
+              0.125,
+              0.4270833333333333
+            ]
+          },
+          "unknown_rejection": {
+            "correct": 64,
+            "requested": 64,
+            "accuracy_ci95": [
+              1.0,
+              1.0
+            ]
+          }
+        },
+        "v3_core": {
+          "canonical_record_identity": {
+            "correct": 40,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.5,
+              0.5
+            ]
+          },
+          "capacitated_assignment": {
+            "correct": 38,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.4375,
+              0.5
+            ]
+          },
+          "constraint_assignment": {
+            "correct": 21,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.1875,
+              0.3125
+            ]
+          },
+          "massive_en": {
+            "correct": 109,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.8500000000000001,
+              0.9583333333333333
+            ]
+          },
+          "massive_zh": {
+            "correct": 105,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.8166666666666667,
+              0.9333333333333332
+            ]
+          },
+          "multiset_reconciliation": {
+            "correct": 23,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.25,
+              0.325
+            ]
+          },
+          "ordinal_service_loss": {
+            "correct": 20,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.1875,
+              0.3125
+            ]
+          },
+          "paraconsistent_rule_closure": {
+            "correct": 20,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.25,
+              0.25
+            ]
+          },
+          "temporal_exclusion": {
+            "correct": 33,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.275,
+              0.5625
+            ]
+          },
+          "transaction_recovery": {
+            "correct": 31,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.3,
+              0.475
+            ]
+          }
+        },
+        "v4": {
+          "boolq": {
+            "correct": 133,
+            "requested": 160,
+            "accuracy_ci95": [
+              0.775,
+              0.8875
+            ]
+          },
+          "belebele_en": {
+            "correct": 112,
+            "requested": 160,
+            "accuracy_ci95": [
+              0.6280452353942144,
+              0.7692307692307693
+            ]
+          },
+          "belebele_zh": {
+            "correct": 112,
+            "requested": 160,
+            "accuracy_ci95": [
+              0.6289261100440628,
+              0.7682926829268293
+            ]
+          }
+        },
+        "v5": {
+          "cosmos_qa": {
+            "correct": 83,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.6083333333333333,
+              0.775
+            ]
+          },
+          "squad2_answerability": {
+            "correct": 100,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.7786885245901639,
+              0.8859649122807017
+            ]
+          },
+          "snli": {
+            "correct": 106,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.825,
+              0.9416666666666667
+            ]
+          },
+          "qasc": {
+            "correct": 115,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.912,
+              0.9918032786885246
+            ]
+          }
+        }
+      },
+      "proper_scores": {
+        "old_core": 0.42193820645312435,
+        "v3_core": 0.6913074500248156,
+        "v4": 0.33935305266273397,
+        "v5": 0.250760098789168,
+        "expanded_weighted_brier": 0.42583970198246046
+      },
+      "probability_coverage": {
+        "old_core": {
+          "requested": 880,
+          "valid_probability_rows": 880
+        },
+        "v3_core": {
+          "requested": 880,
+          "valid_probability_rows": 880
+        },
+        "v4": {
+          "requested": 480,
+          "valid_probability_rows": 480
+        },
+        "v5": {
+          "requested": 480,
+          "valid_probability_rows": 480
+        }
+      },
+      "brier_ci95": {
+        "old_core": [
+          0.365622760488921,
+          0.48051917201614724
+        ],
+        "v3_core": [
+          0.6695390244240492,
+          0.7144598051610135
+        ],
+        "v4": [
+          0.28386912550452903,
+          0.3966449902668765
+        ],
+        "v5": [
+          0.21482273267248006,
+          0.28856469231607446
+        ],
+        "expanded_weighted_brier": [
+          0.40383487736748497,
+          0.44863844813421033
+        ]
+      }
     }
   },
   "paired_comparisons": {
-    "Nox-natural-selected minus Nox-v1.1-released": {
+    "Nox-retention-selected minus Nox-v1.2-released": {
       "panels": {
         "old_core": {
-          "delta": 0.0036458333333333334,
-          "exact_fraction": "7/1920",
+          "delta": -0.0020833333333333333,
+          "exact_fraction": "-1/480",
           "paired_ci95": [
-            -0.002343750000000089,
-            0.011979166666666652
+            -0.011458333333333348,
+            0.006249999999999978
           ]
         },
         "v3_core": {
-          "delta": -0.0016666666666666668,
-          "exact_fraction": "-1/600",
+          "delta": 0.007083333333333333,
+          "exact_fraction": "17/2400",
           "paired_ci95": [
-            -0.012916666666666465,
-            0.009583333333333277
+            -0.0029166666666666785,
+            0.01791666666666658
           ]
         },
         "v4": {
-          "delta": -0.003125,
-          "exact_fraction": "-1/320",
+          "delta": 0.009375,
+          "exact_fraction": "3/320",
           "paired_ci95": [
-            -0.02179487179487172,
-            0.014091990520971005
+            0.0016025641025640969,
+            0.018711665372670876
           ]
         },
         "v5": {
-          "delta": 0.052083333333333336,
-          "exact_fraction": "5/96",
+          "delta": 0.01875,
+          "exact_fraction": "3/160",
           "paired_ci95": [
-            0.0255676834130781,
-            0.0798375706214689
+            0.0038977764142121454,
+            0.03615855263157889
           ]
         },
         "v3_joint": {
-          "delta": 0.0009895833333333334,
-          "exact_fraction": "19/19200",
+          "delta": 0.0025,
+          "exact_fraction": "1/400",
           "paired_ci95": [
-            -0.005546874999999954,
-            0.00789062500000004
+            -0.004375000000000018,
+            0.00927083333333334
           ]
         },
         "expanded_joint": {
-          "delta": 0.012734375,
-          "exact_fraction": "163/12800",
+          "delta": 0.00828125,
+          "exact_fraction": "53/6400",
           "paired_ci95": [
-            0.004166573619049133,
-            0.021864398132660924
+            0.002726124295854282,
+            0.014246129815964542
           ]
         }
       },
       "brier": {
         "old_core": {
-          "delta": -0.007369151910643634,
+          "delta": -0.0001561255030074482,
           "paired_ci95": [
-            -0.013906686611963625,
-            -0.0019409726778719298
+            -0.008261470901879037,
+            0.0063028606664876944
           ]
         },
         "v3_core": {
-          "delta": 0.011185519361137208,
+          "delta": -0.013395637854133113,
           "paired_ci95": [
-            0.006392271109264711,
-            0.015911233958325322
+            -0.01805605540303321,
+            -0.008843713581992051
           ]
         },
         "v4": {
-          "delta": -0.012887667449413887,
+          "delta": -0.007338034354632539,
           "paired_ci95": [
-            -0.025660045932955716,
-            0.00020353457611990694
+            -0.014089448375965645,
+            -0.0009515318737706299
           ]
         },
         "v5": {
-          "delta": -0.07629425193543898,
+          "delta": -0.016445432072925287,
           "paired_ci95": [
-            -0.09835822974417568,
-            -0.05511732226877328
+            -0.024874198044980738,
+            -0.008411447606536929
           ]
         },
         "expanded_weighted_brier": {
-          "delta": -0.02134138798358981,
+          "delta": -0.009333807446174625,
           "paired_ci95": [
-            -0.027988975580296924,
-            -0.014743547095748813
+            -0.012841700455865458,
+            -0.005977590592203745
           ]
         }
       }

metrics/figure-evidence.json

@@ -1,9 +1,9 @@
 {
   "format": "decision-expanded-figure-evidence-v1",
   "kind": "measured",
-  "manifest_sha256": "837ef051c424ff947e307d2b862e1f771aa3ba0a56e7f20ff4f453c665951ac5",
-  "statistics_sha256": "d8f37dcb8fdb274a4d3c11bd4d059c4d52f018087e33ca6b340ca8d64b2b6cce",
-  "public_projection_sha256": "3f8e6d9be291dae6532d6e7fb16232c12c6017cdbf64c4055ddb0d41a1ee363f",
+  "manifest_sha256": "a28f81c3c97bff1298164bf3f8f2ac8ad2b7f83374ada99209b8d7f4e67ac45a",
+  "statistics_sha256": "b1ae34460e36a11126b6865245be3ca6f1a89f908f8a3197bd1d976f878e6fe1",
+  "public_projection_sha256": "0b04a4667e6daa3fa8619c9666323e0bb91b9bd7092a5115b73a612e7b9707f5",
   "base_style_sha256": "93fa22aa96a6dd5f06fe9a5670d84691f2092f84a91bac600c9777a29a36123a",
   "branding_sha256": "d83cf7878c3839f3085feb8c02334217ad2fef0a615b805e1726b292552c46e8",
   "source_sha256": {
@@ -12,10 +12,11 @@
   },
   "display_order": [
     "Jev \u00b7 1.13.0",
-    "Nox-natural-selected",
-    "Nox-v1.1-released",
+    "Nox-retention-selected",
+    "Nox-v1.2-released",
     "Decider \u00b7 2B",
     "Qwen3.5 \u00b7 4B, untuned",
+    "Sol-composition-selected",
     "Qwen3.5 \u00b7 2B, untuned",
     "Laya \u00b7 EN/ML"
   ],
@@ -40,13 +41,17 @@
       "label": "Qwen3.5",
       "detail": "4B \u00b7 untuned"
     },
-    "Nox-natural-selected": {
+    "Nox-retention-selected": {
       "label": "Nox",
-      "detail": "v1.2 \u00b7 this release"
+      "detail": "v1.3 \u00b7 this release"
     },
-    "Nox-v1.1-released": {
+    "Nox-v1.2-released": {
       "label": "Nox",
-      "detail": "v1.1 \u00b7 previous"
+      "detail": "v1.2 \u00b7 previous"
+    },
+    "Sol-composition-selected": {
+      "label": "Sol",
+      "detail": "v1.3 \u00b7 published"
     }
   },
   "model_versions": {
@@ -240,12 +245,17 @@
         "task_finetuning": false
       }
     },
-    "Nox-natural-selected": {
-      "bundle_manifest_sha256": "7762761b5da5d51d33e06d322c037f7cfdee54f910e6e83d3e8c7ce02ae42b2e"
+    "Nox-retention-selected": {
+      "bundle_manifest_sha256": "92d7f5be5e1ef21ee682f574b35cc01de4dd8ab16b8aa6edf0dfbdfcfcfabba3"
     },
-    "Nox-v1.1-released": {
+    "Nox-v1.2-released": {
       "repo_id": "llm-semantic-router/Decision-1.0-Nox",
-      "revision": "5bee3061a7abf675813963e4e05267cb1a28f92f"
+      "revision": "d330a0bb3f6f34e8ad4afaf4465b304093061714"
+    },
+    "Sol-composition-selected": {
+      "bundle_manifest_sha256": "6f0970bfbe5594feecffc14b4824a92080ef09318e12567bd22264c3844e2fc4",
+      "model_revision": "2412d9470d3aa125b346262dad80f6161847aa09",
+      "release_tag": "v1.3"
     }
   },
   "weights": {
@@ -301,128 +311,128 @@
   "files": [
     {
       "file": "TABLES.md",
-      "bytes": 4683,
-      "sha256": "ba6f316af904830c323818df2f3006ac7f2b6c7888b36630238d6a0444aa341a"
+      "bytes": 5154,
+      "sha256": "248cf0280bde4f6786940aa737cf89385dd45defa221ee97743854eebbd595bd"
     },
     {
       "file": "decision-expanded-old_core-600px.png",
-      "bytes": 140155,
-      "sha256": "519923f36d62df99c6769731416d70413c9f5ec3991ba43a75de0714767a12cc"
+      "bytes": 122449,
+      "sha256": "52adf93f0f433d914fa688ad71a1bfc4ec4ae92a1d4be66419e259e8e3b2fe82"
     },
     {
       "file": "decision-expanded-old_core.pdf",
-      "bytes": 44043,
-      "sha256": "29e7f84737b7bfe2d0a444db76ce9d0b867fb206cb4253192cd19594bba0f9c0"
+      "bytes": 45439,
+      "sha256": "ef7b06d43352721a0f8fa4d5b437c4fc9ffa5388555ee77e7bdbf2c39aef72b3"
     },
     {
       "file": "decision-expanded-old_core.png",
-      "bytes": 261350,
-      "sha256": "c59e464a07320a13a182ec841fef0a826eed777d03264260d3a3d6bcf99a2819"
+      "bytes": 282885,
+      "sha256": "5375b996a9b0059ce74f35c5960e8434214da37886679c3c0ba90fabb2e4facf"
     },
     {
       "file": "decision-expanded-old_core.svg",
-      "bytes": 72452,
-      "sha256": "593cada5637013c20daf90c8c6d8656554666c134e0123c5099d703f6a750be6"
+      "bytes": 79583,
+      "sha256": "df85cfeb67ffe0dde79e7370cc02829ecb93fe5b15c6b57656c3fc0930717bb5"
     },
     {
       "file": "decision-expanded-overview-600px.png",
-      "bytes": 83667,
-      "sha256": "f1e659e6ba773c872d6f6e64ca783acface0a29c7db57f0e2d06758bbcf1a397"
+      "bytes": 72204,
+      "sha256": "31f18af4aa3bb42886745427bbdfb9beb8edaac419976b51d665434d336d7452"
     },
     {
       "file": "decision-expanded-overview.pdf",
-      "bytes": 40642,
-      "sha256": "b0427a480be62113f7c14f60c882673eb0c822b4b23b2444411f939a1fab638d"
+      "bytes": 41610,
+      "sha256": "4ed812817f4ec4ef251dbf068976c9a37fa785ffb87249e90e3b6b4674dbc7d9"
     },
     {
       "file": "decision-expanded-overview.png",
-      "bytes": 167930,
-      "sha256": "fa9972beb8e9081bb5a8c59e487e69f31567470fd4af6b0d6d36bdb82d60cdf1"
+      "bytes": 181324,
+      "sha256": "d56231558e0c9f0ea94379564db78260452f3711c8db0bf7de54cbd7c5ef55ac"
     },
     {
       "file": "decision-expanded-overview.svg",
-      "bytes": 44991,
-      "sha256": "c8c4dff1759965471229ccd5abb782e4050099adc328c28a18fb451694683f5a"
+      "bytes": 48483,
+      "sha256": "064d93c2a6036191c430ec842a910dd0bfea87d36c891bc8417156607d51a5cc"
     },
     {
       "file": "decision-expanded-ranking-600px.png",
-      "bytes": 75990,
-      "sha256": "80d65d7fbe04b86fa6ca45c3659b18b968f767ff1102ca5b23de2da2fbd91a59"
+      "bytes": 83453,
+      "sha256": "701abec4f3b26543a0d15374b2e303adbd1bd2758e7190fc0ee99d00c48e73fc"
     },
     {
       "file": "decision-expanded-ranking.pdf",
-      "bytes": 37284,
-      "sha256": "38ef0ccf62b82ec3da5076241646f42c21f2387c864e186165a3a9c1b2b61e37"
+      "bytes": 37394,
+      "sha256": "00f8998cb9b6272e69a23b90b25208fff26a2006943c9d46422be0692b5d6dc3"
     },
     {
       "file": "decision-expanded-ranking.png",
-      "bytes": 171628,
-      "sha256": "038b995974df964403278b2d121bf9b888c52387aeed4e1726ddf5cbbb18f0b3"
+      "bytes": 188168,
+      "sha256": "1a0d8ed0f54e54de7bf68e0a076378088e21482b43aecba16ec5a4374f0c596a"
     },
     {
       "file": "decision-expanded-ranking.svg",
-      "bytes": 36214,
-      "sha256": "236c5add9d316a3ecbd328beb5c171c71d4b1501a84dee5a83581e808c5f6fa3"
+      "bytes": 37944,
+      "sha256": "a6aa770529822374f4990c02810f6e0e5e53620465545a31a5788646542832c2"
     },
     {
       "file": "decision-expanded-v3_core-600px.png",
-      "bytes": 143107,
-      "sha256": "ba4b96800e5f24d1029911d75fae19d7cf9dfd0f2ded82c05fef3b4997259e8a"
+      "bytes": 125061,
+      "sha256": "c0ca23862c8f590a1f1218572ae9972ded5603bf0ff73a510516dd0fa31dc2ba"
     },
     {
       "file": "decision-expanded-v3_core.pdf",
-      "bytes": 44142,
-      "sha256": "f253e527c21c26544ab23a024ad31e28cab1336b6fdc14d7b8e42b533e02a13d"
+      "bytes": 45469,
+      "sha256": "7c9764e8c08cacbe059af01f0d00978374641235e7b46d7a6ce90a6c9752f3cc"
     },
     {
       "file": "decision-expanded-v3_core.png",
-      "bytes": 269511,
-      "sha256": "8c9b3b703cacfca5eed2099665cfcd4fab210c2b0dc6e7540e8c6a9b84b2096f"
+      "bytes": 285114,
+      "sha256": "e15634ebdaa66d49b44371e18124d8b7842aa945e3cca5a5aac1aef9a7daefcf"
     },
     {
       "file": "decision-expanded-v3_core.svg",
-      "bytes": 72477,
-      "sha256": "27fc95d04dc6fd6dbaa1a353fe30588a0c1832650350509cbb5c88b6a60b7476"
+      "bytes": 79594,
+      "sha256": "3be72b1bed0f6888d75f22fef9520198b29c3b8b1adfb71f639f5a65b467decf"
     },
     {
       "file": "decision-expanded-v4-600px.png",
-      "bytes": 66445,
-      "sha256": "b22c8b55bc594fccfeda1b33132cfd94932fb9dd101749699e7c91899a3f1166"
+      "bytes": 58004,
+      "sha256": "3116e158a094448751f76d4a6791d2d8b42bb84b53674b2c8e4f856b4519811c"
     },
     {
       "file": "decision-expanded-v4.pdf",
-      "bytes": 38130,
-      "sha256": "005cac2146b983187002fab407d611f9b506f9cc53fec9ece370c6cf3a638eec"
+      "bytes": 38977,
+      "sha256": "e9a62671976b0e38afe6050f17209c30c3db2483298264e0eccba343aeacb5e0"
     },
     {
       "file": "decision-expanded-v4.png",
-      "bytes": 135301,
-      "sha256": "3b392a7f29d9cf0d87920658d2d57ddd2d5fc500f9658b916a518a9570d2fd08"
+      "bytes": 147606,
+      "sha256": "bc480fbd76003dd8e5745ab8168651a13d185e096916d0045209ebb696aa3348"
     },
     {
       "file": "decision-expanded-v4.svg",
-      "bytes": 39927,
-      "sha256": "1210b42e075012c39001ef33324ff2391681e86592212878d26086ebe3a496f2"
+      "bytes": 42801,
+      "sha256": "58b167837301366f0693138ba5aeb1d7b00a3ef578baa0bdf72cab8ddcd8961e"
     },
     {
       "file": "decision-expanded-v5-600px.png",
-      "bytes": 79064,
-      "sha256": "ef46e061eb5411360b846e4df64eafca18ce4561d5fb7eaf5dafd21681e7a193"
+      "bytes": 69262,
+      "sha256": "a27711e0e066373e6557a65a5d456e543f5e9bfdf90f69fc648a1acd68ae61c6"
     },
     {
       "file": "decision-expanded-v5.pdf",
-      "bytes": 40093,
-      "sha256": "9988a0ba3538f0ddf1c83430dd0763963dc4b6eba1e80969b4c81311389e2018"
+      "bytes": 41205,
+      "sha256": "a94e27aa4b82f32dd687a19bcec709ad2fc4efda19d4757b6d2198c2e371689a"
     },
     {
       "file": "decision-expanded-v5.png",
-      "bytes": 161808,
-      "sha256": "aef547fd323bf044b1f81edf89f07e1e0175f9a52b9742b14eda6890e7fa0f4d"
+      "bytes": 177367,
+      "sha256": "b4573db1b7b9108094a2f0faf68c3df8d8eb99a0735f79d3e742567518447f51"
     },
     {
       "file": "decision-expanded-v5.svg",
-      "bytes": 44951,
-      "sha256": "27bd11dbac7c6bdcc35dff7b9c7cc2c3ed984454121045694e3555dfc73f7b1a"
+      "bytes": 48443,
+      "sha256": "451ea2b5fb1a8d7ef242d853ac111b177307cc3d67d2e8bfdf8ddc73036afbef"
     }
   ]
 }

metrics/materials-provenance.json

@@ -1,11 +1,18 @@
 {
-  "candidate_bundle_manifest_sha256": "7762761b5da5d51d33e06d322c037f7cfdee54f910e6e83d3e8c7ce02ae42b2e",
-  "qualification_sha256": "49258b41a55935e73d01f86966f7d3167047dfe3550a1d5a4a4b1e32bc9cc0d3",
-  "evaluation_receipt_sha256": "f7d59d85873d2a612d7d4926e693246dba7bc01da22284e1a85a86f719a348af",
-  "statistics_sha256": "d8f37dcb8fdb274a4d3c11bd4d059c4d52f018087e33ca6b340ca8d64b2b6cce",
-  "public_projection_sha256": "3f8e6d9be291dae6532d6e7fb16232c12c6017cdbf64c4055ddb0d41a1ee363f",
-  "efficiency_receipt_sha256": "89829d047ebb4fd6d2b511bd74307b584feb9b0af8c9cffbe1b58bdf3e2bb2b3",
-  "release_tag": "v1.2",
-  "source_sha256": "5706c0b36cb6c31c157ba1d043ee13873f7383278adb015e3e99694f1636a8a8",
-  "publication_performed": false
+  "candidate_bundle_manifest_sha256": "92d7f5be5e1ef21ee682f574b35cc01de4dd8ab16b8aa6edf0dfbdfcfcfabba3",
+  "qualification_sha256": "4739782f57e625a273380462420701bf307986b9b85333080ba19aa485894f0d",
+  "evaluation_receipt_sha256": "ef512d08f4610b5ba4a6441a3f405fcf18afb62580fbc8213aeb2190028e5496",
+  "statistics_sha256": "0eb70b3dbc8a5ab8d78279beadd3ab9796623dfe64218dae06ca5edb2795f46d",
+  "public_projection_sha256": "554d74761d5d7ae41074766681b6034b53c7d4961b832d81a9095b992fa53769",
+  "efficiency_receipt_sha256": "3a2ea7e1a49cf2ff2e0cdb966b91b13539d2a15a7e129da0dcef47fefb1a3816",
+  "release_tag": "v1.3",
+  "source_sha256": "21799a7cf26e950f9bf1463dc40f66ad9df285ee88478a1647381073633aa48c",
+  "publication_performed": false,
+  "display_statistics_sha256": "b1ae34460e36a11126b6865245be3ca6f1a89f908f8a3197bd1d976f878e6fe1",
+  "display_projection_sha256": "0b04a4667e6daa3fa8619c9666323e0bb91b9bd7092a5115b73a612e7b9707f5",
+  "Sol_v13_publication_receipt_sha256": "7316ce19ffc5bbdd440316f7899dc33a8eecb9c36ebdd0cd9f9f8b49f48a446f",
+  "Sol_v13_evaluation_receipt_sha256": "5cf55e670d407aaafd727d29a465889fda252df7e3e98f330f0a2b8c88d87fe7",
+  "Sol_v13_model_revision": "2412d9470d3aa125b346262dad80f6161847aa09",
+  "qualification_and_original_statistics_unchanged": true,
+  "no_V6_in_primary_mean": true
 }

metrics/question-scaling.json

@@ -1,436 +1,436 @@
 {
   "all_types": {
-    "Nox-natural-selected": {
+    "Nox-retention-selected": {
       "choice-q1": {
         "samples": 30,
-        "p50_ms": 32.107608,
-        "p95_ms": 33.04698645,
-        "requests_per_second_from_median": 31.14526625589798,
-        "questions_per_second_from_median": 31.14526625589798,
-        "measured_requests_per_second": 30.8760998861839,
-        "measured_questions_per_second": 30.8760998861839,
+        "p50_ms": 32.731553500000004,
+        "p95_ms": 33.4735869,
+        "requests_per_second_from_median": 30.551559369157346,
+        "questions_per_second_from_median": 30.551559369157346,
+        "measured_requests_per_second": 30.5029057383203,
+        "measured_questions_per_second": 30.5029057383203,
         "max_allocated_bytes": 8649496576,
         "max_reserved_bytes": 8923381760,
         "input_tokens": 309
       },
       "choice-q2": {
         "samples": 30,
-        "p50_ms": 33.0248195,
-        "p95_ms": 34.04666545,
-        "requests_per_second_from_median": 30.280256338721244,
-        "questions_per_second_from_median": 60.56051267744249,
-        "measured_requests_per_second": 30.32447258444086,
-        "measured_questions_per_second": 60.64894516888172,
+        "p50_ms": 33.3187065,
+        "p95_ms": 33.56022575,
+        "requests_per_second_from_median": 30.01316992903071,
+        "questions_per_second_from_median": 60.02633985806142,
+        "measured_requests_per_second": 30.051085192016746,
+        "measured_questions_per_second": 60.10217038403349,
         "max_allocated_bytes": 8694344704,
         "max_reserved_bytes": 8938061824,
         "input_tokens": 618
       },
       "choice-q4": {
         "samples": 30,
-        "p50_ms": 41.706045,
-        "p95_ms": 42.00540085,
-        "requests_per_second_from_median": 23.977339495989128,
-        "questions_per_second_from_median": 95.90935798395651,
-        "measured_requests_per_second": 23.96644908033505,
-        "measured_questions_per_second": 95.8657963213402,
+        "p50_ms": 41.779055,
+        "p95_ms": 42.214628999999995,
+        "requests_per_second_from_median": 23.935438463124644,
+        "questions_per_second_from_median": 95.74175385249858,
+        "measured_requests_per_second": 23.884399734574895,
+        "measured_questions_per_second": 95.53759893829958,
         "max_allocated_bytes": 8783385600,
         "max_reserved_bytes": 9141485568,
         "input_tokens": 1236
       },
       "choice-q8": {
         "samples": 30,
-        "p50_ms": 69.48287,
-        "p95_ms": 70.71858545,
-        "requests_per_second_from_median": 14.392036483236803,
-        "questions_per_second_from_median": 115.13629186589442,
-        "measured_requests_per_second": 14.32914310038675,
-        "measured_questions_per_second": 114.633144803094,
+        "p50_ms": 69.641899,
+        "p95_ms": 70.72795845,
+        "requests_per_second_from_median": 14.359171911725154,
+        "questions_per_second_from_median": 114.87337529380123,
+        "measured_requests_per_second": 14.311513624064359,
+        "measured_questions_per_second": 114.49210899251487,
         "max_allocated_bytes": 8963302400,
         "max_reserved_bytes": 9376366592,
         "input_tokens": 2472
       },
       "choice-q16": {
         "samples": 30,
-        "p50_ms": 141.298589,
-        "p95_ms": 141.920981,
-        "requests_per_second_from_median": 7.0772115070448445,
-        "questions_per_second_from_median": 113.23538411271751,
-        "measured_requests_per_second": 7.084327224101568,
-        "measured_questions_per_second": 113.34923558562508,
+        "p50_ms": 141.0197655,
+        "p95_ms": 141.85338695,
+        "requests_per_second_from_median": 7.091204530474134,
+        "questions_per_second_from_median": 113.45927248758615,
+        "measured_requests_per_second": 7.1075954042236935,
+        "measured_questions_per_second": 113.7215264675791,
         "max_allocated_bytes": 8962254336,
         "max_reserved_bytes": 9244246016,
         "input_tokens": 4944
       },
       "choice-q32": {
         "samples": 30,
-        "p50_ms": 282.326369,
-        "p95_ms": 284.8558578,
-        "requests_per_second_from_median": 3.542000003549084,
-        "questions_per_second_from_median": 113.34400011357069,
-        "measured_requests_per_second": 3.5446644265419387,
-        "measured_questions_per_second": 113.42926164934204,
+        "p50_ms": 282.0275835,
+        "p95_ms": 283.68355145,
+        "requests_per_second_from_median": 3.545752467151852,
+        "questions_per_second_from_median": 113.46407894885927,
+        "measured_requests_per_second": 3.5466119610851,
+        "measured_questions_per_second": 113.4915827547232,
         "max_allocated_bytes": 8963302912,
         "max_reserved_bytes": 9141485568,
         "input_tokens": 9888
       },
       "noul-q1": {
         "samples": 30,
-        "p50_ms": 32.281844500000005,
-        "p95_ms": 32.977001,
-        "requests_per_second_from_median": 30.977164269532363,
-        "questions_per_second_from_median": 30.977164269532363,
-        "measured_requests_per_second": 31.030133981643576,
-        "measured_questions_per_second": 31.030133981643576,
+        "p50_ms": 32.685345,
+        "p95_ms": 33.000833899999996,
+        "requests_per_second_from_median": 30.594751256258732,
+        "questions_per_second_from_median": 30.594751256258732,
+        "measured_requests_per_second": 30.61657989325799,
+        "measured_questions_per_second": 30.61657989325799,
         "max_allocated_bytes": 8645022720,
         "max_reserved_bytes": 8917090304,
         "input_tokens": 262
       },
       "noul-q2": {
         "samples": 30,
-        "p50_ms": 32.572923,
-        "p95_ms": 33.97754645,
-        "requests_per_second_from_median": 30.700345805625116,
-        "questions_per_second_from_median": 61.40069161125023,
-        "measured_requests_per_second": 30.42684666972788,
-        "measured_questions_per_second": 60.85369333945576,
+        "p50_ms": 32.822731000000005,
+        "p95_ms": 33.580505,
+        "requests_per_second_from_median": 30.466690903934833,
+        "questions_per_second_from_median": 60.933381807869665,
+        "measured_requests_per_second": 30.306271635312687,
+        "measured_questions_per_second": 60.61254327062537,
         "max_allocated_bytes": 8686444544,
         "max_reserved_bytes": 9216983040,
         "input_tokens": 524
       },
       "noul-q4": {
         "samples": 30,
-        "p50_ms": 38.9671165,
-        "p95_ms": 39.276742999999996,
-        "requests_per_second_from_median": 25.662663543503403,
-        "questions_per_second_from_median": 102.65065417401361,
-        "measured_requests_per_second": 25.722971332785946,
-        "measured_questions_per_second": 102.89188533114378,
+        "p50_ms": 38.8360825,
+        "p95_ms": 39.62854105,
+        "requests_per_second_from_median": 25.749250069185013,
+        "questions_per_second_from_median": 102.99700027674005,
+        "measured_requests_per_second": 25.627138601267134,
+        "measured_questions_per_second": 102.50855440506854,
         "max_allocated_bytes": 8767323136,
         "max_reserved_bytes": 9191817216,
         "input_tokens": 1048
       },
       "noul-q8": {
         "samples": 30,
-        "p50_ms": 64.496848,
-        "p95_ms": 65.0679283,
-        "requests_per_second_from_median": 15.50463365279494,
-        "questions_per_second_from_median": 124.03706922235952,
-        "measured_requests_per_second": 15.492551650332626,
-        "measured_questions_per_second": 123.940413202661,
+        "p50_ms": 64.278516,
+        "p95_ms": 65.07748314999999,
+        "requests_per_second_from_median": 15.557297558020787,
+        "questions_per_second_from_median": 124.4583804641663,
+        "measured_requests_per_second": 15.523243079930335,
+        "measured_questions_per_second": 124.18594463944268,
         "max_allocated_bytes": 8932226048,
         "max_reserved_bytes": 9141485568,
         "input_tokens": 2096
       },
       "noul-q16": {
         "samples": 30,
-        "p50_ms": 129.420839,
-        "p95_ms": 130.16497075,
-        "requests_per_second_from_median": 7.726730932411897,
-        "questions_per_second_from_median": 123.62769491859035,
-        "measured_requests_per_second": 7.723849562904405,
-        "measured_questions_per_second": 123.58159300647048,
+        "p50_ms": 129.2654275,
+        "p95_ms": 130.59987905,
+        "requests_per_second_from_median": 7.7360205225794045,
+        "questions_per_second_from_median": 123.77632836127047,
+        "measured_requests_per_second": 7.725552951411574,
+        "measured_questions_per_second": 123.60884722258518,
         "max_allocated_bytes": 8931440128,
         "max_reserved_bytes": 9244246016,
         "input_tokens": 4192
       },
       "noul-q32": {
         "samples": 30,
-        "p50_ms": 260.114736,
-        "p95_ms": 261.01741085000003,
-        "requests_per_second_from_median": 3.8444573167127296,
-        "questions_per_second_from_median": 123.02263413480735,
-        "measured_requests_per_second": 3.8574251583945887,
-        "measured_questions_per_second": 123.43760506862684,
+        "p50_ms": 259.156278,
+        "p95_ms": 261.59688335,
+        "requests_per_second_from_median": 3.8586755748977075,
+        "questions_per_second_from_median": 123.47761839672664,
+        "measured_requests_per_second": 3.8613862674541757,
+        "measured_questions_per_second": 123.56436055853362,
         "max_allocated_bytes": 8932226560,
         "max_reserved_bytes": 9244246016,
         "input_tokens": 8384
       },
       "score-q1": {
         "samples": 30,
-        "p50_ms": 32.600662,
-        "p95_ms": 33.154171049999995,
-        "requests_per_second_from_median": 30.674223732021147,
-        "questions_per_second_from_median": 30.674223732021147,
-        "measured_requests_per_second": 30.70316656105357,
-        "measured_questions_per_second": 30.70316656105357,
+        "p50_ms": 33.319307,
+        "p95_ms": 34.269436,
+        "requests_per_second_from_median": 30.01262901416287,
+        "questions_per_second_from_median": 30.01262901416287,
+        "measured_requests_per_second": 29.995543652066246,
+        "measured_questions_per_second": 29.995543652066246,
         "max_allocated_bytes": 8648972288,
         "max_reserved_bytes": 9244246016,
         "input_tokens": 307
       },
       "score-q2": {
         "samples": 30,
-        "p50_ms": 33.2208605,
-        "p95_ms": 33.62534145,
-        "requests_per_second_from_median": 30.101568260099704,
-        "questions_per_second_from_median": 60.20313652019941,
-        "measured_requests_per_second": 30.219942037607215,
-        "measured_questions_per_second": 60.43988407521443,
+        "p50_ms": 33.3555535,
+        "p95_ms": 33.76658145,
+        "requests_per_second_from_median": 29.980015171986278,
+        "questions_per_second_from_median": 59.960030343972555,
+        "measured_requests_per_second": 30.07546038274141,
+        "measured_questions_per_second": 60.15092076548282,
         "max_allocated_bytes": 8693689344,
         "max_reserved_bytes": 9141485568,
         "input_tokens": 614
       },
       "score-q4": {
         "samples": 30,
-        "p50_ms": 41.775667999999996,
-        "p95_ms": 42.3791635,
-        "requests_per_second_from_median": 23.937379050408005,
-        "questions_per_second_from_median": 95.74951620163202,
-        "measured_requests_per_second": 23.877435233851767,
-        "measured_questions_per_second": 95.50974093540707,
+        "p50_ms": 41.8594245,
+        "p95_ms": 42.2104965,
+        "requests_per_second_from_median": 23.889482761522437,
+        "questions_per_second_from_median": 95.55793104608975,
+        "measured_requests_per_second": 23.829400385190663,
+        "measured_questions_per_second": 95.31760154076265,
         "max_allocated_bytes": 8782337024,
         "max_reserved_bytes": 9244246016,
         "input_tokens": 1228
       },
       "score-q8": {
         "samples": 30,
-        "p50_ms": 69.53526400000001,
-        "p95_ms": 70.1399854,
-        "requests_per_second_from_median": 14.381192253760622,
-        "questions_per_second_from_median": 115.04953803008497,
-        "measured_requests_per_second": 14.403771734792818,
-        "measured_questions_per_second": 115.23017387834254,
+        "p50_ms": 69.734315,
+        "p95_ms": 70.12355445,
+        "requests_per_second_from_median": 14.340142295797987,
+        "questions_per_second_from_median": 114.7211383663839,
+        "measured_requests_per_second": 14.393887061255466,
+        "measured_questions_per_second": 115.15109649004373,
         "max_allocated_bytes": 8963302400,
         "max_reserved_bytes": 9376366592,
         "input_tokens": 2456
       },
       "score-q16": {
         "samples": 30,
-        "p50_ms": 140.446957,
-        "p95_ms": 141.9686554,
-        "requests_per_second_from_median": 7.120125785281343,
-        "questions_per_second_from_median": 113.92201256450149,
-        "measured_requests_per_second": 7.106122379909049,
-        "measured_questions_per_second": 113.69795807854479,
+        "p50_ms": 141.064813,
+        "p95_ms": 141.96145435,
+        "requests_per_second_from_median": 7.088940032125517,
+        "questions_per_second_from_median": 113.42304051400828,
+        "measured_requests_per_second": 7.0981910052955834,
+        "measured_questions_per_second": 113.57105608472934,
         "max_allocated_bytes": 8963302912,
         "max_reserved_bytes": 9244246016,
         "input_tokens": 4912
       },
       "score-q32": {
         "samples": 30,
-        "p50_ms": 281.5755855,
-        "p95_ms": 284.90178425,
-        "requests_per_second_from_median": 3.5514442710801006,
-        "questions_per_second_from_median": 113.64621667456322,
-        "measured_requests_per_second": 3.5406794630737664,
-        "measured_questions_per_second": 113.30174281836052,
+        "p50_ms": 282.3302425,
+        "p95_ms": 285.68878665,
+        "requests_per_second_from_median": 3.541951408198858,
+        "questions_per_second_from_median": 113.34244506236345,
+        "measured_requests_per_second": 3.5352975301699447,
+        "measured_questions_per_second": 113.12952096543823,
         "max_allocated_bytes": 8963302912,
         "max_reserved_bytes": 9244246016,
         "input_tokens": 9824
       }
     },
-    "Nox-v1.1-released": {
+    "Nox-v1.2-released": {
       "choice-q1": {
         "samples": 30,
-        "p50_ms": 28.9757945,
-        "p95_ms": 30.240221699999996,
-        "requests_per_second_from_median": 34.511564471510866,
-        "questions_per_second_from_median": 34.511564471510866,
-        "measured_requests_per_second": 34.32893449270674,
-        "measured_questions_per_second": 34.32893449270674,
+        "p50_ms": 32.9773255,
+        "p95_ms": 34.75197285,
+        "requests_per_second_from_median": 30.32386601515032,
+        "questions_per_second_from_median": 30.32386601515032,
+        "measured_requests_per_second": 30.077481547502668,
+        "measured_questions_per_second": 30.077481547502668,
         "max_allocated_bytes": 8648972288,
-        "max_reserved_bytes": 8929673216,
+        "max_reserved_bytes": 8923381760,
         "input_tokens": 309
       },
       "choice-q2": {
         "samples": 30,
-        "p50_ms": 29.3374375,
-        "p95_ms": 29.953046999999998,
-        "requests_per_second_from_median": 34.08613993638674,
-        "questions_per_second_from_median": 68.17227987277349,
-        "measured_requests_per_second": 33.9753724393917,
-        "measured_questions_per_second": 67.9507448787834,
+        "p50_ms": 33.5885535,
+        "p95_ms": 33.9047395,
+        "requests_per_second_from_median": 29.772047194589664,
+        "questions_per_second_from_median": 59.54409438917933,
+        "measured_requests_per_second": 29.744622251120244,
+        "measured_questions_per_second": 59.48924450224049,
         "max_allocated_bytes": 8694475776,
         "max_reserved_bytes": 8713666560,
         "input_tokens": 618
       },
       "choice-q4": {
         "samples": 30,
-        "p50_ms": 42.0405665,
-        "p95_ms": 42.4125123,
-        "requests_per_second_from_median": 23.78654911798108,
-        "questions_per_second_from_median": 95.14619647192433,
-        "measured_requests_per_second": 23.793745836748805,
-        "measured_questions_per_second": 95.17498334699522,
+        "p50_ms": 42.225716000000006,
+        "p95_ms": 42.3277483,
+        "requests_per_second_from_median": 23.682250882376984,
+        "questions_per_second_from_median": 94.72900352950793,
+        "measured_requests_per_second": 23.691972066242528,
+        "measured_questions_per_second": 94.76788826497011,
         "max_allocated_bytes": 8783385600,
         "max_reserved_bytes": 9256828928,
         "input_tokens": 1236
       },
       "choice-q8": {
         "samples": 30,
-        "p50_ms": 70.1055465,
-        "p95_ms": 70.8502012,
-        "requests_per_second_from_median": 14.264206613095869,
-        "questions_per_second_from_median": 114.11365290476695,
-        "measured_requests_per_second": 14.24194480835554,
-        "measured_questions_per_second": 113.93555846684433,
+        "p50_ms": 70.05188100000001,
+        "p95_ms": 70.7074044,
+        "requests_per_second_from_median": 14.275134168060381,
+        "questions_per_second_from_median": 114.20107334448305,
+        "measured_requests_per_second": 14.280790527554691,
+        "measured_questions_per_second": 114.24632422043753,
         "max_allocated_bytes": 8963302400,
         "max_reserved_bytes": 9126805504,
         "input_tokens": 2472
       },
       "choice-q16": {
         "samples": 30,
-        "p50_ms": 141.3689395,
-        "p95_ms": 142.33760285,
-        "requests_per_second_from_median": 7.073689620484137,
-        "questions_per_second_from_median": 113.17903392774619,
-        "measured_requests_per_second": 7.082735221822023,
-        "measured_questions_per_second": 113.32376354915237,
+        "p50_ms": 140.3605955,
+        "p95_ms": 142.61031055,
+        "requests_per_second_from_median": 7.124506678229362,
+        "questions_per_second_from_median": 113.99210685166979,
+        "measured_requests_per_second": 7.106102620497567,
+        "measured_questions_per_second": 113.69764192796107,
         "max_allocated_bytes": 8963302912,
         "max_reserved_bytes": 9256828928,
         "input_tokens": 4944
       },
       "choice-q32": {
         "samples": 30,
-        "p50_ms": 283.68131300000005,
-        "p95_ms": 286.0126798,
-        "requests_per_second_from_median": 3.5250823870798986,
-        "questions_per_second_from_median": 112.80263638655676,
-        "measured_requests_per_second": 3.524747236128377,
-        "measured_questions_per_second": 112.79191155610806,
+        "p50_ms": 281.68696150000005,
+        "p95_ms": 283.44484195,
+        "requests_per_second_from_median": 3.5500400681484856,
+        "questions_per_second_from_median": 113.60128218075154,
+        "measured_requests_per_second": 3.5488099888848748,
+        "measured_questions_per_second": 113.56191964431599,
         "max_allocated_bytes": 8963302912,
         "max_reserved_bytes": 9256828928,
         "input_tokens": 9888
       },
       "noul-q1": {
         "samples": 30,
-        "p50_ms": 28.9925315,
-        "p95_ms": 29.37066045,
-        "requests_per_second_from_median": 34.49164140771909,
-        "questions_per_second_from_median": 34.49164140771909,
-        "measured_requests_per_second": 34.58104068093915,
-        "measured_questions_per_second": 34.58104068093915,
+        "p50_ms": 33.1341925,
+        "p95_ms": 33.99351695,
+        "requests_per_second_from_median": 30.180303926223647,
+        "questions_per_second_from_median": 30.180303926223647,
+        "measured_requests_per_second": 30.049516314170884,
+        "measured_questions_per_second": 30.049516314170884,
         "max_allocated_bytes": 8645022720,
         "max_reserved_bytes": 8919187456,
         "input_tokens": 262
       },
       "noul-q2": {
         "samples": 30,
-        "p50_ms": 29.196136,
-        "p95_ms": 30.45851935,
-        "requests_per_second_from_median": 34.25110774932683,
-        "questions_per_second_from_median": 68.50221549865365,
-        "measured_requests_per_second": 34.08654368923435,
-        "measured_questions_per_second": 68.1730873784687,
+        "p50_ms": 33.543437499999996,
+        "p95_ms": 33.8247975,
+        "requests_per_second_from_median": 29.81209066602074,
+        "questions_per_second_from_median": 59.62418133204148,
+        "measured_requests_per_second": 29.8875148708319,
+        "measured_questions_per_second": 59.7750297416638,
         "max_allocated_bytes": 8686018560,
-        "max_reserved_bytes": 9196011520,
+        "max_reserved_bytes": 8948547584,
         "input_tokens": 524
       },
       "noul-q4": {
         "samples": 30,
-        "p50_ms": 39.3726845,
-        "p95_ms": 39.57324895,
-        "requests_per_second_from_median": 25.398318979240546,
-        "questions_per_second_from_median": 101.59327591696218,
-        "measured_requests_per_second": 25.417992661917022,
-        "measured_questions_per_second": 101.67197064766809,
+        "p50_ms": 39.399696,
+        "p95_ms": 39.5125585,
+        "requests_per_second_from_median": 25.380906492273443,
+        "questions_per_second_from_median": 101.52362596909377,
+        "measured_requests_per_second": 25.37583637392737,
+        "measured_questions_per_second": 101.50334549570948,
         "max_allocated_bytes": 8767454208,
         "max_reserved_bytes": 9170845696,
         "input_tokens": 1048
       },
       "noul-q8": {
         "samples": 30,
-        "p50_ms": 64.3817725,
-        "p95_ms": 65.2974674,
-        "requests_per_second_from_median": 15.532346519350645,
-        "questions_per_second_from_median": 124.25877215480516,
-        "measured_requests_per_second": 15.53924018787304,
-        "measured_questions_per_second": 124.31392150298431,
+        "p50_ms": 64.420319,
+        "p95_ms": 65.1090079,
+        "requests_per_second_from_median": 15.523052594632446,
+        "questions_per_second_from_median": 124.18442075705957,
+        "measured_requests_per_second": 15.531637426422071,
+        "measured_questions_per_second": 124.25309941137657,
         "max_allocated_bytes": 8932226048,
-        "max_reserved_bytes": 9191817216,
+        "max_reserved_bytes": 9212788736,
         "input_tokens": 2096
       },
       "noul-q16": {
         "samples": 30,
-        "p50_ms": 129.92605450000002,
-        "p95_ms": 130.92555945,
-        "requests_per_second_from_median": 7.696685655916688,
-        "questions_per_second_from_median": 123.146970494667,
-        "measured_requests_per_second": 7.715408426351679,
-        "measured_questions_per_second": 123.44653482162687,
+        "p50_ms": 129.3631795,
+        "p95_ms": 129.7524684,
+        "requests_per_second_from_median": 7.730174875610567,
+        "questions_per_second_from_median": 123.68279800976907,
+        "measured_requests_per_second": 7.734049202633538,
+        "measured_questions_per_second": 123.74478724213661,
         "max_allocated_bytes": 8931440128,
         "max_reserved_bytes": 9256828928,
         "input_tokens": 4192
       },
       "noul-q32": {
         "samples": 30,
-        "p50_ms": 259.700112,
-        "p95_ms": 262.4176393,
-        "requests_per_second_from_median": 3.8505951818765487,
-        "questions_per_second_from_median": 123.21904582004956,
-        "measured_requests_per_second": 3.841294712746642,
-        "measured_questions_per_second": 122.92143080789255,
+        "p50_ms": 259.01647449999996,
+        "p95_ms": 260.04006150000004,
+        "requests_per_second_from_median": 3.86075828547346,
+        "questions_per_second_from_median": 123.54426513515072,
+        "measured_requests_per_second": 3.863530404032038,
+        "measured_questions_per_second": 123.63297292902521,
         "max_allocated_bytes": 8931440128,
         "max_reserved_bytes": 9158262784,
         "input_tokens": 8384
       },
       "score-q1": {
         "samples": 30,
-        "p50_ms": 28.816428000000002,
-        "p95_ms": 29.11869395,
-        "requests_per_second_from_median": 34.70242737927129,
-        "questions_per_second_from_median": 34.70242737927129,
-        "measured_requests_per_second": 34.6747983353027,
-        "measured_questions_per_second": 34.6747983353027,
+        "p50_ms": 33.022303,
+        "p95_ms": 33.4376394,
+        "requests_per_second_from_median": 30.28256387811595,
+        "questions_per_second_from_median": 30.28256387811595,
+        "measured_requests_per_second": 30.287766675936002,
+        "measured_questions_per_second": 30.287766675936002,
         "max_allocated_bytes": 8649496576,
         "max_reserved_bytes": 9256828928,
         "input_tokens": 307
       },
       "score-q2": {
         "samples": 30,
-        "p50_ms": 29.3393325,
-        "p95_ms": 29.776925,
-        "requests_per_second_from_median": 34.08393834454141,
-        "questions_per_second_from_median": 68.16787668908282,
-        "measured_requests_per_second": 34.03598008577042,
-        "measured_questions_per_second": 68.07196017154084,
+        "p50_ms": 33.780904,
+        "p95_ms": 34.33654235,
+        "requests_per_second_from_median": 29.602523366455795,
+        "questions_per_second_from_median": 59.20504673291159,
+        "measured_requests_per_second": 29.78998673028873,
+        "measured_questions_per_second": 59.57997346057746,
         "max_allocated_bytes": 8694344704,
-        "max_reserved_bytes": 9198108672,
+        "max_reserved_bytes": 9219080192,
         "input_tokens": 614
       },
       "score-q4": {
         "samples": 30,
-        "p50_ms": 41.9650575,
-        "p95_ms": 42.2195708,
-        "requests_per_second_from_median": 23.82934897682435,
-        "questions_per_second_from_median": 95.3173959072974,
-        "measured_requests_per_second": 23.82153615943987,
-        "measured_questions_per_second": 95.28614463775948,
+        "p50_ms": 41.9683725,
+        "p95_ms": 42.27619345,
+        "requests_per_second_from_median": 23.827466742962216,
+        "questions_per_second_from_median": 95.30986697184886,
+        "measured_requests_per_second": 23.79795751110445,
+        "measured_questions_per_second": 95.1918300444178,
         "max_allocated_bytes": 8782861312,
-        "max_reserved_bytes": 9198108672,
+        "max_reserved_bytes": 9191817216,
         "input_tokens": 1228
       },
       "score-q8": {
         "samples": 30,
-        "p50_ms": 69.58278200000001,
-        "p95_ms": 70.67227025,
-        "requests_per_second_from_median": 14.371371354482491,
-        "questions_per_second_from_median": 114.97097083585993,
-        "measured_requests_per_second": 14.322159646664494,
-        "measured_questions_per_second": 114.57727717331595,
+        "p50_ms": 69.5594625,
+        "p95_ms": 70.08738735,
+        "requests_per_second_from_median": 14.376189292722037,
+        "questions_per_second_from_median": 115.0095143417763,
+        "measured_requests_per_second": 14.362227747761676,
+        "measured_questions_per_second": 114.8978219820934,
         "max_allocated_bytes": 8963302400,
-        "max_reserved_bytes": 9355395072,
+        "max_reserved_bytes": 9376366592,
         "input_tokens": 2456
       },
       "score-q16": {
         "samples": 30,
-        "p50_ms": 141.1238315,
-        "p95_ms": 142.7912914,
-        "requests_per_second_from_median": 7.085975411601548,
-        "questions_per_second_from_median": 113.37560658562477,
-        "measured_requests_per_second": 7.091980381610694,
-        "measured_questions_per_second": 113.4716861057711,
+        "p50_ms": 140.220064,
+        "p95_ms": 141.31451855,
+        "requests_per_second_from_median": 7.1316470088046735,
+        "questions_per_second_from_median": 114.10635214087478,
+        "measured_requests_per_second": 7.131560615003727,
+        "measured_questions_per_second": 114.10496984005964,
         "max_allocated_bytes": 8963302912,
-        "max_reserved_bytes": 9355395072,
+        "max_reserved_bytes": 9376366592,
         "input_tokens": 4912
       },
       "score-q32": {
         "samples": 30,
-        "p50_ms": 283.68416249999996,
-        "p95_ms": 286.0974013,
-        "requests_per_second_from_median": 3.5250469789620356,
-        "questions_per_second_from_median": 112.80150332678514,
-        "measured_requests_per_second": 3.5312236579984573,
-        "measured_questions_per_second": 112.99915705595063,
+        "p50_ms": 280.6431195,
+        "p95_ms": 284.87025265,
+        "requests_per_second_from_median": 3.5632443146356914,
+        "questions_per_second_from_median": 114.02381806834212,
+        "measured_requests_per_second": 3.5497310134469697,
+        "measured_questions_per_second": 113.59139243030303,
         "max_allocated_bytes": 8963302912,
         "max_reserved_bytes": 9256828928,
         "input_tokens": 9824
@@ -446,22 +446,22 @@
     32
   ],
   "candidate_p50_ms": [
-    32.107608,
-    33.0248195,
-    41.706045,
-    69.48287,
-    141.298589,
-    282.326369
+    32.731553500000004,
+    33.3187065,
+    41.779055,
+    69.641899,
+    141.0197655,
+    282.0275835
   ],
   "reference_p50_ms": [
-    28.9757945,
-    29.3374375,
-    42.0405665,
-    70.1055465,
-    141.3689395,
-    283.68131300000005
+    32.9773255,
+    33.5885535,
+    42.225716000000006,
+    70.05188100000001,
+    140.3605955,
+    281.68696150000005
   ],
-  "geomean_latency_ratio": 1.0336937058472013,
+  "geomean_latency_ratio": 0.9956499431886017,
   "raw_observations": 1404,
   "claim_boundary": "Sequential whole requests with fixed state/question bytes; not concurrent-serving throughput."
 }

metrics/uncalibrated-quality.json

@@ -0,0 +1,795 @@
+{
+  "models": {
+    "Nox-retention-selected": {
+      "old_core": {
+        "point": 0.8300223214285715,
+        "exact_fraction": "7437/8960",
+        "ci95": [
+          0.7974693080357143,
+          0.862016369047619
+        ]
+      },
+      "v3_core": {
+        "point": 0.5179166666666667,
+        "exact_fraction": "1243/2400",
+        "ci95": [
+          0.4920833333333333,
+          0.545
+        ]
+      },
+      "v4": {
+        "point": 0.790625,
+        "exact_fraction": "253/320",
+        "ci95": [
+          0.7453125,
+          0.8331463068181818
+        ]
+      },
+      "v5": {
+        "point": 0.8625,
+        "exact_fraction": "69/80",
+        "ci95": [
+          0.8321482316646345,
+          0.8915620903779563
+        ]
+      },
+      "v3_joint": {
+        "point": 0.6739694940476191,
+        "exact_fraction": "181163/268800",
+        "ci95": [
+          0.652767857142857,
+          0.6950187872023811
+        ]
+      },
+      "expanded_joint": {
+        "point": 0.7502659970238095,
+        "exact_fraction": "403343/537600",
+        "ci95": [
+          0.7330469621683191,
+          0.7670605596068275
+        ]
+      },
+      "families": {
+        "old_core": {
+          "ag_news": {
+            "correct": 109,
+            "requested": 128,
+            "accuracy_ci95": [
+              0.7890625,
+              0.90625
+            ]
+          },
+          "boolean_constraints": {
+            "correct": 60,
+            "requested": 64,
+            "accuracy_ci95": [
+              0.859375,
+              1.0
+            ]
+          },
+          "dbpedia_14": {
+            "correct": 108,
+            "requested": 112,
+            "accuracy_ci95": [
+              0.9285714285714286,
+              0.9910714285714286
+            ]
+          },
+          "natural_intents": {
+            "correct": 64,
+            "requested": 64,
+            "accuracy_ci95": [
+              1.0,
+              1.0
+            ]
+          },
+          "option_carrier": {
+            "correct": 96,
+            "requested": 96,
+            "accuracy_ci95": [
+              1.0,
+              1.0
+            ]
+          },
+          "ordinal_rubric": {
+            "correct": 57,
+            "requested": 64,
+            "accuracy_ci95": [
+              0.78125,
+              0.984375
+            ]
+          },
+          "relational_composition": {
+            "correct": 50,
+            "requested": 96,
+            "accuracy_ci95": [
+              0.34375,
+              0.6979166666666666
+            ]
+          },
+          "scoped_evidence": {
+            "correct": 75,
+            "requested": 96,
+            "accuracy_ci95": [
+              0.6145833333333334,
+              0.9270833333333334
+            ]
+          },
+          "state_tracking": {
+            "correct": 34,
+            "requested": 96,
+            "accuracy_ci95": [
+              0.19791666666666666,
+              0.53125
+            ]
+          },
+          "unknown_rejection": {
+            "correct": 64,
+            "requested": 64,
+            "accuracy_ci95": [
+              1.0,
+              1.0
+            ]
+          }
+        },
+        "v3_core": {
+          "canonical_record_identity": {
+            "correct": 43,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.5,
+              0.575
+            ]
+          },
+          "capacitated_assignment": {
+            "correct": 37,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.4125,
+              0.5125
+            ]
+          },
+          "constraint_assignment": {
+            "correct": 33,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.3375,
+              0.475
+            ]
+          },
+          "massive_en": {
+            "correct": 110,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.8666666666666667,
+              0.9583333333333334
+            ]
+          },
+          "massive_zh": {
+            "correct": 105,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.8166666666666667,
+              0.925
+            ]
+          },
+          "multiset_reconciliation": {
+            "correct": 23,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.2125,
+              0.375
+            ]
+          },
+          "ordinal_service_loss": {
+            "correct": 24,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.225,
+              0.375
+            ]
+          },
+          "paraconsistent_rule_closure": {
+            "correct": 27,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.275,
+              0.425
+            ]
+          },
+          "temporal_exclusion": {
+            "correct": 34,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.3,
+              0.55
+            ]
+          },
+          "transaction_recovery": {
+            "correct": 50,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.4875,
+              0.7625
+            ]
+          }
+        },
+        "v4": {
+          "boolq": {
+            "correct": 138,
+            "requested": 160,
+            "accuracy_ci95": [
+              0.80625,
+              0.9125
+            ]
+          },
+          "belebele_en": {
+            "correct": 117,
+            "requested": 160,
+            "accuracy_ci95": [
+              0.6585365853658537,
+              0.802547770700637
+            ]
+          },
+          "belebele_zh": {
+            "correct": 113,
+            "requested": 160,
+            "accuracy_ci95": [
+              0.632258064516129,
+              0.779874213836478
+            ]
+          }
+        },
+        "v5": {
+          "cosmos_qa": {
+            "correct": 85,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.625,
+              0.7833333333333333
+            ]
+          },
+          "squad2_answerability": {
+            "correct": 106,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.8214285714285714,
+              0.9407026836158189
+            ]
+          },
+          "snli": {
+            "correct": 107,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.8333333333333334,
+              0.9416666666666667
+            ]
+          },
+          "qasc": {
+            "correct": 116,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.9327731092436975,
+              0.991869918699187
+            ]
+          }
+        }
+      },
+      "proper_scores": {
+        "old_core": 0.236949143663811,
+        "v3_core": 0.6420493423368422,
+        "v4": 0.3198838954699546,
+        "v5": 0.1994643937036534,
+        "expanded_weighted_brier": 0.34958669379356533
+      },
+      "probability_coverage": {
+        "old_core": {
+          "requested": 880,
+          "valid_probability_rows": 880
+        },
+        "v3_core": {
+          "requested": 880,
+          "valid_probability_rows": 880
+        },
+        "v4": {
+          "requested": 480,
+          "valid_probability_rows": 480
+        },
+        "v5": {
+          "requested": 480,
+          "valid_probability_rows": 480
+        }
+      },
+      "brier_ci95": {
+        "old_core": [
+          0.1923714470208656,
+          0.2842244724489109
+        ],
+        "v3_core": [
+          0.609522057990806,
+          0.6754018539103575
+        ],
+        "v4": [
+          0.2556471642199712,
+          0.3883116318676653
+        ],
+        "v5": [
+          0.16307492742416962,
+          0.23853947114140675
+        ],
+        "expanded_weighted_brier": [
+          0.32625676202196663,
+          0.3735703756627187
+        ]
+      }
+    },
+    "Nox-v1.2-released": {
+      "old_core": {
+        "point": 0.8321056547619048,
+        "exact_fraction": "22367/26880",
+        "ci95": [
+          0.799775855654762,
+          0.8639146205357143
+        ]
+      },
+      "v3_core": {
+        "point": 0.5108333333333334,
+        "exact_fraction": "613/1200",
+        "ci95": [
+          0.48540624999999993,
+          0.5366666666666667
+        ]
+      },
+      "v4": {
+        "point": 0.78125,
+        "exact_fraction": "25/32",
+        "ci95": [
+          0.7348237223129993,
+          0.825
+        ]
+      },
+      "v5": {
+        "point": 0.84375,
+        "exact_fraction": "27/32",
+        "ci95": [
+          0.8125286977325981,
+          0.8745048035497992
+        ]
+      },
+      "v3_joint": {
+        "point": 0.671469494047619,
+        "exact_fraction": "180491/268800",
+        "ci95": [
+          0.6504053199404762,
+          0.6917820870535715
+        ]
+      },
+      "expanded_joint": {
+        "point": 0.7419847470238096,
+        "exact_fraction": "398891/537600",
+        "ci95": [
+          0.7246839089844096,
+          0.7585761485522727
+        ]
+      },
+      "families": {
+        "old_core": {
+          "ag_news": {
+            "correct": 109,
+            "requested": 128,
+            "accuracy_ci95": [
+              0.7890625,
+              0.90625
+            ]
+          },
+          "boolean_constraints": {
+            "correct": 60,
+            "requested": 64,
+            "accuracy_ci95": [
+              0.859375,
+              1.0
+            ]
+          },
+          "dbpedia_14": {
+            "correct": 108,
+            "requested": 112,
+            "accuracy_ci95": [
+              0.9285714285714286,
+              0.9910714285714286
+            ]
+          },
+          "natural_intents": {
+            "correct": 64,
+            "requested": 64,
+            "accuracy_ci95": [
+              1.0,
+              1.0
+            ]
+          },
+          "option_carrier": {
+            "correct": 96,
+            "requested": 96,
+            "accuracy_ci95": [
+              1.0,
+              1.0
+            ]
+          },
+          "ordinal_rubric": {
+            "correct": 57,
+            "requested": 64,
+            "accuracy_ci95": [
+              0.78125,
+              0.984375
+            ]
+          },
+          "relational_composition": {
+            "correct": 52,
+            "requested": 96,
+            "accuracy_ci95": [
+              0.375,
+              0.7083333333333334
+            ]
+          },
+          "scoped_evidence": {
+            "correct": 75,
+            "requested": 96,
+            "accuracy_ci95": [
+              0.6145833333333334,
+              0.9270833333333334
+            ]
+          },
+          "state_tracking": {
+            "correct": 34,
+            "requested": 96,
+            "accuracy_ci95": [
+              0.19791666666666666,
+              0.53125
+            ]
+          },
+          "unknown_rejection": {
+            "correct": 64,
+            "requested": 64,
+            "accuracy_ci95": [
+              1.0,
+              1.0
+            ]
+          }
+        },
+        "v3_core": {
+          "canonical_record_identity": {
+            "correct": 40,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.5,
+              0.5
+            ]
+          },
+          "capacitated_assignment": {
+            "correct": 33,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.35,
+              0.475
+            ]
+          },
+          "constraint_assignment": {
+            "correct": 33,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.3375,
+              0.475
+            ]
+          },
+          "massive_en": {
+            "correct": 109,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.8581250000000001,
+              0.9583333333333333
+            ]
+          },
+          "massive_zh": {
+            "correct": 105,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.8166666666666667,
+              0.925
+            ]
+          },
+          "multiset_reconciliation": {
+            "correct": 23,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.2125,
+              0.375
+            ]
+          },
+          "ordinal_service_loss": {
+            "correct": 25,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.2375,
+              0.3625
+            ]
+          },
+          "paraconsistent_rule_closure": {
+            "correct": 27,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.275,
+              0.425
+            ]
+          },
+          "temporal_exclusion": {
+            "correct": 36,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.3,
+              0.5875
+            ]
+          },
+          "transaction_recovery": {
+            "correct": 49,
+            "requested": 80,
+            "accuracy_ci95": [
+              0.5125,
+              0.7125
+            ]
+          }
+        },
+        "v4": {
+          "boolq": {
+            "correct": 137,
+            "requested": 160,
+            "accuracy_ci95": [
+              0.8,
+              0.90625
+            ]
+          },
+          "belebele_en": {
+            "correct": 115,
+            "requested": 160,
+            "accuracy_ci95": [
+              0.6441717791411042,
+              0.7922077922077922
+            ]
+          },
+          "belebele_zh": {
+            "correct": 111,
+            "requested": 160,
+            "accuracy_ci95": [
+              0.61875,
+              0.7664670658682635
+            ]
+          }
+        },
+        "v5": {
+          "cosmos_qa": {
+            "correct": 82,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.6,
+              0.7666666666666667
+            ]
+          },
+          "squad2_answerability": {
+            "correct": 102,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.7829998478393184,
+              0.9137931034482759
+            ]
+          },
+          "snli": {
+            "correct": 104,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.8,
+              0.925
+            ]
+          },
+          "qasc": {
+            "correct": 117,
+            "requested": 120,
+            "accuracy_ci95": [
+              0.9435483870967742,
+              1.0
+            ]
+          }
+        }
+      },
+      "proper_scores": {
+        "old_core": 0.22631387940194642,
+        "v3_core": 0.6235274727371851,
+        "v4": 0.3097296984868416,
+        "v5": 0.21064944276547515,
+        "expanded_weighted_brier": 0.3425551233478621
+      },
+      "probability_coverage": {
+        "old_core": {
+          "requested": 880,
+          "valid_probability_rows": 880
+        },
+        "v3_core": {
+          "requested": 880,
+          "valid_probability_rows": 880
+        },
+        "v4": {
+          "requested": 480,
+          "valid_probability_rows": 480
+        },
+        "v5": {
+          "requested": 480,
+          "valid_probability_rows": 480
+        }
+      },
+      "brier_ci95": {
+        "old_core": [
+          0.18408643417046128,
+          0.2705460910410972
+        ],
+        "v3_core": [
+          0.5958006946200504,
+          0.6518586943819807
+        ],
+        "v4": [
+          0.24910637969525015,
+          0.3748812866575687
+        ],
+        "v5": [
+          0.1763941044924108,
+          0.24745947579321434
+        ],
+        "expanded_weighted_brier": [
+          0.3208655591911214,
+          0.3648390087307264
+        ]
+      }
+    }
+  },
+  "paired_comparisons": {
+    "Nox-retention-selected minus Nox-v1.2-released": {
+      "panels": {
+        "old_core": {
+          "delta": -0.0020833333333333333,
+          "exact_fraction": "-1/480",
+          "paired_ci95": [
+            -0.011458333333333348,
+            0.006249999999999978
+          ]
+        },
+        "v3_core": {
+          "delta": 0.007083333333333333,
+          "exact_fraction": "17/2400",
+          "paired_ci95": [
+            -0.0029166666666666785,
+            0.01791666666666658
+          ]
+        },
+        "v4": {
+          "delta": 0.009375,
+          "exact_fraction": "3/320",
+          "paired_ci95": [
+            0.0016025641025640969,
+            0.018711665372670876
+          ]
+        },
+        "v5": {
+          "delta": 0.01875,
+          "exact_fraction": "3/160",
+          "paired_ci95": [
+            0.0038977764142121454,
+            0.03615855263157889
+          ]
+        },
+        "v3_joint": {
+          "delta": 0.0025,
+          "exact_fraction": "1/400",
+          "paired_ci95": [
+            -0.004375000000000018,
+            0.00927083333333334
+          ]
+        },
+        "expanded_joint": {
+          "delta": 0.00828125,
+          "exact_fraction": "53/6400",
+          "paired_ci95": [
+            0.002726124295854282,
+            0.014246129815964542
+          ]
+        }
+      },
+      "brier": {
+        "old_core": {
+          "delta": 0.010635264261864591,
+          "paired_ci95": [
+            0.0016195091593130292,
+            0.018874193567589192
+          ]
+        },
+        "v3_core": {
+          "delta": 0.01852186959965707,
+          "paired_ci95": [
+            0.010744459978380852,
+            0.026241609712213
+          ]
+        },
+        "v4": {
+          "delta": 0.010154196983112984,
+          "paired_ci95": [
+            0.0018178765816714087,
+            0.018759051594017338
+          ]
+        },
+        "v5": {
+          "delta": -0.011185049061821745,
+          "paired_ci95": [
+            -0.02095325842236825,
+            -0.0011938002219903945
+          ]
+        },
+        "expanded_weighted_brier": {
+          "delta": 0.007031570445703239,
+          "paired_ci95": [
+            0.0026019668472251974,
+            0.01136522520052578
+          ]
+        }
+      }
+    }
+  },
+  "method": {
+    "seed": 202609225801,
+    "repeats": 10000,
+    "weights": {
+      "old_core": "1/4",
+      "v3_core": "1/4",
+      "v4": "1/4",
+      "v5": "1/4"
+    },
+    "within_panel_weights": {
+      "old_core": {
+        "ag_news": "1/10",
+        "boolean_constraints": "1/10",
+        "dbpedia_14": "1/10",
+        "natural_intents": "1/10",
+        "option_carrier": "1/10",
+        "ordinal_rubric": "1/10",
+        "relational_composition": "1/10",
+        "scoped_evidence": "1/10",
+        "state_tracking": "1/10",
+        "unknown_rejection": "1/10"
+      },
+      "v3_core": {
+        "canonical_record_identity": "1/10",
+        "capacitated_assignment": "1/10",
+        "constraint_assignment": "1/10",
+        "massive_en": "1/10",
+        "massive_zh": "1/10",
+        "multiset_reconciliation": "1/10",
+        "ordinal_service_loss": "1/10",
+        "paraconsistent_rule_closure": "1/10",
+        "temporal_exclusion": "1/10",
+        "transaction_recovery": "1/10"
+      },
+      "v4": {
+        "boolq": "1/2",
+        "belebele_en": "1/4",
+        "belebele_zh": "1/4"
+      },
+      "v5": {
+        "cosmos_qa": "1/4",
+        "squad2_answerability": "1/4",
+        "snli": "1/4",
+        "qasc": "1/4"
+      }
+    },
+    "v3_joint_unchanged": "(old_core+v3_core)/2",
+    "exact_accuracy": "Reduced Fraction from integer family correct/total counts; no rounded eligibility.",
+    "native_decision": "Use recorded prediction; never replace it with FP64-softmax argmax.",
+    "brier": "Recorded valid probability vector normalized once as specified by shared score.assess; sum squared category errors, no division by K. Complete coverage required for a panel proper-score point.",
+    "clusters": "Original V3 within-family groups and coupled MASSIVE translations; original V4 shared passages/translations; original V5 complete source/near components. Identical panel seeds reused across models and accuracy/Brier. Panels sampled independently.",
+    "interval_role": "Report only; no new CI/family gate.",
+    "native_supplements": "Both 220-answer native panels remain outside the quality average, fully reported."
+  }
+}

model-card-example.json

@@ -31,25 +31,25 @@
       "destination": {
         "type": "choice",
         "probabilities": {
-          "billing": 0.9995198700709675,
-          "technical": 0.00048012992903245
+          "billing": 0.9990422083691034,
+          "technical": 0.0009577916308966501
         },
-        "confidence": 0.9990397401419351,
+        "confidence": 0.9980844167382068,
         "choice": "billing"
       },
       "refund_requested": {
         "type": "noul",
-        "noul": 0.9997257333959462
+        "noul": 0.9994888159055739
       },
       "urgency": {
         "type": "score",
         "probabilities": {
-          "0": 0.01145875993979324,
-          "1": 0.9885278778333887,
-          "2": 1.3362226818050175e-05
+          "0": 0.012138526904188826,
+          "1": 0.9878299250528677,
+          "2": 3.1548042943515704e-05
         },
-        "confidence": 0.9827918167500831,
-        "score": 0.9885546022870247,
+        "confidence": 0.9817448875793016,
+        "score": 0.9878930211387548,
         "legend": {
           "0": "Routine information request with no payment problem or outage",
           "1": "A payment or billing problem, with no product outage",
@@ -65,7 +65,7 @@
   "direct_engine_exact_response": true,
   "overflow_rejected": true,
   "overflow_message": "check: 40083 tokens exceeds max_length=16384; no truncation allowed",
-  "bundle_manifest_sha256": "7762761b5da5d51d33e06d322c037f7cfdee54f910e6e83d3e8c7ce02ae42b2e",
+  "bundle_manifest_sha256": "92d7f5be5e1ef21ee682f574b35cc01de4dd8ab16b8aa6edf0dfbdfcfcfabba3",
   "runtime": {
     "actual": {
       "torch": "2.12.0+git6bbd260",

release-manifest.json

@@ -1,9 +1,9 @@
 {
   "format": "decision-public-release-v1",
   "status": "assembled-not-published",
-  "bundle_manifest_sha256": "7762761b5da5d51d33e06d322c037f7cfdee54f910e6e83d3e8c7ce02ae42b2e",
-  "readiness_sha256": "82bc316e1f85083b32315e894f9bf3cf2f731f78376b33bbdf206492fd41c7f1",
-  "model_card_sha256": "9a9f62761cffdef286d9916169b27f40d540e38085834882c4923f29c6c6ca2d",
+  "bundle_manifest_sha256": "92d7f5be5e1ef21ee682f574b35cc01de4dd8ab16b8aa6edf0dfbdfcfcfabba3",
+  "readiness_sha256": "0e280d31ad830a88eacf016c527f7016cc5f36f4f213056c92c79318f4207cd9",
+  "model_card_sha256": "bf18b6761ae31b7e6eadf7f55634a497aeea708ed60974c1a76597b6481bae0e",
   "repo_id": "llm-semantic-router/Decision-1.0-Nox",
   "assembly_script_sha256": "7831ef5952b8256e585d4efa91cac4ce87738f4eb93c84b1cf1e7250def1f430",
   "original_bundle_manifest_preserved": true,
@@ -21,8 +21,8 @@
     },
     {
       "file": "EVALUATION.md",
-      "bytes": 6091,
-      "sha256": "2402026626d5f2ef76f68eb9480baf98da0cf454b1410f195d2a7bda05525513"
+      "bytes": 9311,
+      "sha256": "3231494555997d640639e9350e95140754bac7a9cb04bca07120216b7dcd6608"
     },
     {
       "file": "LICENSE",
@@ -36,8 +36,8 @@
     },
     {
       "file": "QUESTION-SCALING.md",
-      "bytes": 3527,
-      "sha256": "9f8c8bf81b64da7d3309f4d9f79cdbcef60b315f8ed4fa77cc98d1d912d0d29a"
+      "bytes": 3526,
+      "sha256": "64b2df97b5e06c25f3fdffccfb32f34a7ef453bcfec5cd544acce7acc08217d7"
     },
     {
       "file": "QWEN-LICENSE",
@@ -46,8 +46,8 @@
     },
     {
       "file": "README.md",
-      "bytes": 3796,
-      "sha256": "9a9f62761cffdef286d9916169b27f40d540e38085834882c4923f29c6c6ca2d"
+      "bytes": 4012,
+      "sha256": "bf18b6761ae31b7e6eadf7f55634a497aeea708ed60974c1a76597b6481bae0e"
     },
     {
       "file": "RUNTIME.md",
@@ -57,12 +57,12 @@
     {
       "file": "RUNTIME_BINDING.json",
       "bytes": 2621,
-      "sha256": "65a7772ed57e811faefe6b3b3868e174c5e4a3d7f3aa0138a3838464bb2612fe"
+      "sha256": "fb76d9fc9147e15a678eb91dfc40d7039845a4eaebdecce041a5c9e76c3d3e0f"
     },
     {
       "file": "SOURCE_BUNDLE_MANIFEST.json",
       "bytes": 5637,
-      "sha256": "6d402d6d55e734851cb0d6417d6f3b416b25ae807c3a0d6dcf12e40acb6ac546"
+      "sha256": "d437bc0149bcb8c9891fbb33c5abc4c2336c3a89ab6cfa0981da5a7f1d19f1f4"
     },
     {
       "file": "USAGE.md",
@@ -81,123 +81,123 @@
     },
     {
       "file": "assets/decision-expanded-old_core-600px.png",
-      "bytes": 140155,
-      "sha256": "519923f36d62df99c6769731416d70413c9f5ec3991ba43a75de0714767a12cc"
+      "bytes": 122449,
+      "sha256": "52adf93f0f433d914fa688ad71a1bfc4ec4ae92a1d4be66419e259e8e3b2fe82"
     },
     {
       "file": "assets/decision-expanded-old_core.pdf",
-      "bytes": 44043,
-      "sha256": "29e7f84737b7bfe2d0a444db76ce9d0b867fb206cb4253192cd19594bba0f9c0"
+      "bytes": 45439,
+      "sha256": "ef7b06d43352721a0f8fa4d5b437c4fc9ffa5388555ee77e7bdbf2c39aef72b3"
     },
     {
       "file": "assets/decision-expanded-old_core.png",
-      "bytes": 261350,
-      "sha256": "c59e464a07320a13a182ec841fef0a826eed777d03264260d3a3d6bcf99a2819"
+      "bytes": 282885,
+      "sha256": "5375b996a9b0059ce74f35c5960e8434214da37886679c3c0ba90fabb2e4facf"
     },
     {
       "file": "assets/decision-expanded-old_core.svg",
-      "bytes": 72452,
-      "sha256": "593cada5637013c20daf90c8c6d8656554666c134e0123c5099d703f6a750be6"
+      "bytes": 79583,
+      "sha256": "df85cfeb67ffe0dde79e7370cc02829ecb93fe5b15c6b57656c3fc0930717bb5"
     },
     {
       "file": "assets/decision-expanded-overview-600px.png",
-      "bytes": 83667,
-      "sha256": "f1e659e6ba773c872d6f6e64ca783acface0a29c7db57f0e2d06758bbcf1a397"
+      "bytes": 72204,
+      "sha256": "31f18af4aa3bb42886745427bbdfb9beb8edaac419976b51d665434d336d7452"
     },
     {
       "file": "assets/decision-expanded-overview.pdf",
-      "bytes": 40642,
-      "sha256": "b0427a480be62113f7c14f60c882673eb0c822b4b23b2444411f939a1fab638d"
+      "bytes": 41610,
+      "sha256": "4ed812817f4ec4ef251dbf068976c9a37fa785ffb87249e90e3b6b4674dbc7d9"
     },
     {
       "file": "assets/decision-expanded-overview.png",
-      "bytes": 167930,
-      "sha256": "fa9972beb8e9081bb5a8c59e487e69f31567470fd4af6b0d6d36bdb82d60cdf1"
+      "bytes": 181324,
+      "sha256": "d56231558e0c9f0ea94379564db78260452f3711c8db0bf7de54cbd7c5ef55ac"
     },
     {
       "file": "assets/decision-expanded-overview.svg",
-      "bytes": 44991,
-      "sha256": "c8c4dff1759965471229ccd5abb782e4050099adc328c28a18fb451694683f5a"
+      "bytes": 48483,
+      "sha256": "064d93c2a6036191c430ec842a910dd0bfea87d36c891bc8417156607d51a5cc"
     },
     {
       "file": "assets/decision-expanded-ranking-600px.png",
-      "bytes": 75990,
-      "sha256": "80d65d7fbe04b86fa6ca45c3659b18b968f767ff1102ca5b23de2da2fbd91a59"
+      "bytes": 83453,
+      "sha256": "701abec4f3b26543a0d15374b2e303adbd1bd2758e7190fc0ee99d00c48e73fc"
     },
     {
       "file": "assets/decision-expanded-ranking.pdf",
-      "bytes": 37284,
-      "sha256": "38ef0ccf62b82ec3da5076241646f42c21f2387c864e186165a3a9c1b2b61e37"
+      "bytes": 37394,
+      "sha256": "00f8998cb9b6272e69a23b90b25208fff26a2006943c9d46422be0692b5d6dc3"
     },
     {
       "file": "assets/decision-expanded-ranking.png",
-      "bytes": 171628,
-      "sha256": "038b995974df964403278b2d121bf9b888c52387aeed4e1726ddf5cbbb18f0b3"
+      "bytes": 188168,
+      "sha256": "1a0d8ed0f54e54de7bf68e0a076378088e21482b43aecba16ec5a4374f0c596a"
     },
     {
       "file": "assets/decision-expanded-ranking.svg",
-      "bytes": 36214,
-      "sha256": "236c5add9d316a3ecbd328beb5c171c71d4b1501a84dee5a83581e808c5f6fa3"
+      "bytes": 37944,
+      "sha256": "a6aa770529822374f4990c02810f6e0e5e53620465545a31a5788646542832c2"
     },
     {
       "file": "assets/decision-expanded-v3_core-600px.png",
-      "bytes": 143107,
-      "sha256": "ba4b96800e5f24d1029911d75fae19d7cf9dfd0f2ded82c05fef3b4997259e8a"
+      "bytes": 125061,
+      "sha256": "c0ca23862c8f590a1f1218572ae9972ded5603bf0ff73a510516dd0fa31dc2ba"
     },
     {
       "file": "assets/decision-expanded-v3_core.pdf",
-      "bytes": 44142,
-      "sha256": "f253e527c21c26544ab23a024ad31e28cab1336b6fdc14d7b8e42b533e02a13d"
+      "bytes": 45469,
+      "sha256": "7c9764e8c08cacbe059af01f0d00978374641235e7b46d7a6ce90a6c9752f3cc"
     },
     {
       "file": "assets/decision-expanded-v3_core.png",
-      "bytes": 269511,
-      "sha256": "8c9b3b703cacfca5eed2099665cfcd4fab210c2b0dc6e7540e8c6a9b84b2096f"
+      "bytes": 285114,
+      "sha256": "e15634ebdaa66d49b44371e18124d8b7842aa945e3cca5a5aac1aef9a7daefcf"
     },
     {
       "file": "assets/decision-expanded-v3_core.svg",
-      "bytes": 72477,
-      "sha256": "27fc95d04dc6fd6dbaa1a353fe30588a0c1832650350509cbb5c88b6a60b7476"
+      "bytes": 79594,
+      "sha256": "3be72b1bed0f6888d75f22fef9520198b29c3b8b1adfb71f639f5a65b467decf"
     },
     {
       "file": "assets/decision-expanded-v4-600px.png",
-      "bytes": 66445,
-      "sha256": "b22c8b55bc594fccfeda1b33132cfd94932fb9dd101749699e7c91899a3f1166"
+      "bytes": 58004,
+      "sha256": "3116e158a094448751f76d4a6791d2d8b42bb84b53674b2c8e4f856b4519811c"
     },
     {
       "file": "assets/decision-expanded-v4.pdf",
-      "bytes": 38130,
-      "sha256": "005cac2146b983187002fab407d611f9b506f9cc53fec9ece370c6cf3a638eec"
+      "bytes": 38977,
+      "sha256": "e9a62671976b0e38afe6050f17209c30c3db2483298264e0eccba343aeacb5e0"
     },
     {
       "file": "assets/decision-expanded-v4.png",
-      "bytes": 135301,
-      "sha256": "3b392a7f29d9cf0d87920658d2d57ddd2d5fc500f9658b916a518a9570d2fd08"
+      "bytes": 147606,
+      "sha256": "bc480fbd76003dd8e5745ab8168651a13d185e096916d0045209ebb696aa3348"
     },
     {
       "file": "assets/decision-expanded-v4.svg",
-      "bytes": 39927,
-      "sha256": "1210b42e075012c39001ef33324ff2391681e86592212878d26086ebe3a496f2"
+      "bytes": 42801,
+      "sha256": "58b167837301366f0693138ba5aeb1d7b00a3ef578baa0bdf72cab8ddcd8961e"
     },
     {
       "file": "assets/decision-expanded-v5-600px.png",
-      "bytes": 79064,
-      "sha256": "ef46e061eb5411360b846e4df64eafca18ce4561d5fb7eaf5dafd21681e7a193"
+      "bytes": 69262,
+      "sha256": "a27711e0e066373e6557a65a5d456e543f5e9bfdf90f69fc648a1acd68ae61c6"
     },
     {
       "file": "assets/decision-expanded-v5.pdf",
-      "bytes": 40093,
-      "sha256": "9988a0ba3538f0ddf1c83430dd0763963dc4b6eba1e80969b4c81311389e2018"
+      "bytes": 41205,
+      "sha256": "a94e27aa4b82f32dd687a19bcec709ad2fc4efda19d4757b6d2198c2e371689a"
     },
     {
       "file": "assets/decision-expanded-v5.png",
-      "bytes": 161808,
-      "sha256": "aef547fd323bf044b1f81edf89f07e1e0175f9a52b9742b14eda6890e7fa0f4d"
+      "bytes": 177367,
+      "sha256": "b4573db1b7b9108094a2f0faf68c3df8d8eb99a0735f79d3e742567518447f51"
     },
     {
       "file": "assets/decision-expanded-v5.svg",
-      "bytes": 44951,
-      "sha256": "27bd11dbac7c6bdcc35dff7b9c7cc2c3ed984454121045694e3555dfc73f7b1a"
+      "bytes": 48443,
+      "sha256": "451ea2b5fb1a8d7ef242d853ac111b177307cc3d67d2e8bfdf8ddc73036afbef"
     },
     {
       "file": "assets/decision-family-header.png",
@@ -206,23 +206,23 @@
     },
     {
       "file": "assets/decision-question-scaling-600px.png",
-      "bytes": 44739,
-      "sha256": "6d39aae1592415df0413bba6f35a6dfa41af8c32caffa4804717bc3001cca774"
+      "bytes": 44578,
+      "sha256": "ccf4e7a1904bc115220fa17f2572dd9bc23ae587510428941d723392367de398"
     },
     {
       "file": "assets/decision-question-scaling.pdf",
-      "bytes": 32062,
-      "sha256": "887da4cd137ed378dae40c44f79953e27b07e771cfbd5fa0fe788b1bae665517"
+      "bytes": 32058,
+      "sha256": "476e4e6aaccc22628ed1807eb0687706a461e876b1b87119f3a3f1b69d1508e1"
     },
     {
       "file": "assets/decision-question-scaling.png",
-      "bytes": 105103,
-      "sha256": "9f032e63eb2e5e30b2c42f5db70d68ef2039222afdb7495f8f3a3dfb0286747e"
+      "bytes": 105527,
+      "sha256": "666312a18f1cc334ea7840d93237c19894e22291379c82a36c606cb5ec5612f7"
     },
     {
       "file": "assets/decision-question-scaling.svg",
-      "bytes": 26992,
-      "sha256": "cd038389b040662e57b2129dee8da8aef9d0c4c7918ce82c923fe11a7439bcac"
+      "bytes": 26990,
+      "sha256": "c3d7428f652badd93b2fb349f39eecc6ccd5b8f7156eb98c8764a38ca3d1a70b"
     },
     {
       "file": "assets/readout.png",
@@ -237,17 +237,17 @@
     {
       "file": "backbone/model-00001-of-00003.safetensors",
       "bytes": 3991295368,
-      "sha256": "5f9c4bc396605b551b5983a1699de8e96755fafdb6afdb228a5cc4d9b438a5da"
+      "sha256": "5ebccc395ef9fa4d61c79894ecefcdaa2a319c1c154221e1ad73663b6f82aacc"
     },
     {
       "file": "backbone/model-00002-of-00003.safetensors",
       "bytes": 3979828128,
-      "sha256": "656cc757ba03f87cedc0e4c88237db1ae8215686f9b6477334a34dc0a32136db"
+      "sha256": "990e2e79fc1ab009df9846ef9fddb31f4b988589bbda84f4728dbaa4196d7fcd"
     },
     {
       "file": "backbone/model-00003-of-00003.safetensors",
       "bytes": 440425856,
-      "sha256": "0a30a07b6efbaa37d016008848c1fa6dadcb46ae451a0e5e4b927ac9b5771804"
+      "sha256": "c47859a3192bdae5003e4732fb911c8dddcf6e8caa167b4971e3a9801b828d64"
     },
     {
       "file": "backbone/model.safetensors.index.json",
@@ -257,7 +257,7 @@
     {
       "file": "bundle-manifest.json",
       "bytes": 7635,
-      "sha256": "7762761b5da5d51d33e06d322c037f7cfdee54f910e6e83d3e8c7ce02ae42b2e"
+      "sha256": "92d7f5be5e1ef21ee682f574b35cc01de4dd8ab16b8aa6edf0dfbdfcfcfabba3"
     },
     {
       "file": "chat_template.jinja",
@@ -297,37 +297,42 @@
     {
       "file": "decision_head.safetensors",
       "bytes": 10529624,
-      "sha256": "8b8e342445035e503b4963e16c5f7a4fef95c08368e27eace312145fc3660772"
+      "sha256": "9cb6f639714e31bcb76b58eaf94af0b72ac3db9d091ebcda45d9b575efd489de"
     },
     {
       "file": "metrics/comparator-coverage.json",
-      "bytes": 21192,
-      "sha256": "7612486e073438ad11f91122af39e9ffc89e774157f7f14aebe58184ab9b4632"
+      "bytes": 24112,
+      "sha256": "b1ac1497681a3b9112319893e235d8bdfcf19964d1f17f9af73b58427f08d140"
     },
     {
       "file": "metrics/expanded-quality.json",
-      "bytes": 58851,
-      "sha256": "d8f37dcb8fdb274a4d3c11bd4d059c4d52f018087e33ca6b340ca8d64b2b6cce"
+      "bytes": 66608,
+      "sha256": "b1ae34460e36a11126b6865245be3ca6f1a89f908f8a3197bd1d976f878e6fe1"
     },
     {
       "file": "metrics/figure-evidence.json",
-      "bytes": 16646,
-      "sha256": "00703ecd5f2ac1ecfd4fcdd8a42712f78c943ab4bb8dab7d18689d44aecce87d"
+      "bytes": 17024,
+      "sha256": "a208c6ea774ced7fc572a6aa83b216c2bb45b0a19c84c7806d998834e75bc8ce"
     },
     {
       "file": "metrics/materials-provenance.json",
-      "bytes": 736,
-      "sha256": "a8277d5a8cc7f06d90ff97f55219eb3ce9de8ed285c82416020b83da031fb7fa"
+      "bytes": 1313,
+      "sha256": "3143ecd4a556d9e963c809c1520fed03c6921b61be960a55401dc27a9a383905"
     },
     {
       "file": "metrics/question-scaling.json",
-      "bytes": 17711,
-      "sha256": "25fc1e00f836156e1c58d933ace5bfc24774ebae33eebd64f632f8febd92d21c"
+      "bytes": 17752,
+      "sha256": "6f50e96db05a593bb24ce1c7816f52648aba80716323b9c214347eaa1295aa5c"
+    },
+    {
+      "file": "metrics/uncalibrated-quality.json",
+      "bytes": 19925,
+      "sha256": "2bc4aca25ef6468bbf3d872e0d72d50476c921717b66ad8b2469f70b051ae735"
     },
     {
       "file": "model-card-example.json",
-      "bytes": 3772,
-      "sha256": "847cc6ebe4def51d7172345cc42706b864c0078fd227c23f05973faa7fba8e73"
+      "bytes": 3775,
+      "sha256": "9d17a7495a0986d4850a966e282cd3d9e0a31130bdf08c0a9e60461795ce5502"
     },
     {
       "file": "pyproject.toml",
@@ -381,8 +386,8 @@
     },
     {
       "file": "temperature.json",
-      "bytes": 6095,
-      "sha256": "1d652e75f49b5c232d1e742b189c1c826ad52ced1a931a124e29e65e7d7fb569"
+      "bytes": 6083,
+      "sha256": "69d80e5b215e2c1e6872f146fdb7ea5aa95c9fd678ef96208960d50e4e7b635d"
     },
     {
       "file": "tokenizer.json",
@@ -459,6 +464,7 @@
     "metrics/figure-evidence.json": "copy",
     "metrics/materials-provenance.json": "copy",
     "metrics/question-scaling.json": "copy",
+    "metrics/uncalibrated-quality.json": "copy",
     "model-card-example.json": "copy",
     "pyproject.toml": "copy",
     "runtime-build-provenance.json": "copy",

temperature.json

@@ -1,60 +1,60 @@
 {
-  "arm": "natural_treatment",
-  "selected_step": 375,
-  "temperature": 1.0155830584882959,
+  "arm": "lambda025",
+  "selected_step": 282,
+  "temperature": 1.3231350559653137,
   "temperatures": {
-    "choice": 1.0155830584882959,
-    "noul": 1.0155830584882959,
-    "score": 1.0155830584882959
+    "choice": 1.3231350559653137,
+    "noul": 1.3231350559653137,
+    "score": 1.3231350559653137
   },
   "fit": {
-    "temperature": 1.0155830584882959,
-    "inverse_temperature": 0.9846560472252349,
+    "temperature": 1.3231350559653137,
+    "inverse_temperature": 0.7557807462598249,
     "status": "relative-bracket-converged",
     "boundary_hit": false,
     "iterations": 38,
     "beta_final_bracket": [
-      0.9846560471889461,
-      0.9846560472615238
+      0.7557807462235361,
+      0.7557807462961137
     ],
-    "objective_at_T1": 0.4262873409965549,
-    "objective_at_selected_T": 0.42624728463092443,
-    "derivative_at_selected_beta": 7.788978184145204e-12,
+    "objective_at_T1": 0.44021343659702844,
+    "objective_at_selected_T": 0.4260766624652933,
+    "derivative_at_selected_beta": -1.8373920391834685e-12,
     "initial_boundary_derivatives": {
-      "beta_0.05": -3.0366540479066213,
-      "beta_20": 0.19310407449992384
+      "beta_0.05": -3.6546078204867998,
+      "beta_20": 0.255508441854847
     },
-    "derivative_at_beta1": 0.005185775194699754,
+    "derivative_at_beta1": 0.10257030038687498,
     "initial_boundary_objectives": [
-      1.0957737013940982,
-      3.8927751504820978
+      1.0580632285716105,
+      5.133197394579512
     ]
   },
   "raw": {
     "rows": 1600,
     "accuracy_fraction": [
-      1007,
-      1200
+      1511,
+      1800
     ],
-    "accuracy": 0.8391666666666666,
-    "brier": 0.22605177485769778,
-    "nll": 0.4262873409965549,
-    "score_rps": 0.033206021482489016,
+    "accuracy": 0.8394444444444444,
+    "brier": 0.2232446037975029,
+    "nll": 0.44021343659702844,
+    "score_rps": 0.034960309670060405,
     "by_family": {
       "old/stage4_dense_table": [
         92,
         100
       ],
       "old/stage4_scope": [
-        94,
+        95,
         100
       ],
       "old/stage4_arithmetic": [
-        80,
+        79,
         100
       ],
       "old/stage4_ordinal": [
-        90,
+        89,
         100
       ],
       "old/stage4_natural_nli": [
@@ -62,11 +62,11 @@
         200
       ],
       "old/stage4_automaton": [
-        54,
+        52,
         100
       ],
       "old/stage4_boolean": [
-        69,
+        71,
         100
       ],
       "old/stage4_relations": [
@@ -74,19 +74,19 @@
         100
       ],
       "old/stage4_registers": [
-        77,
+        74,
         100
       ],
       "natural/natural_squad2_answerability": [
-        186,
+        185,
         200
       ],
       "natural/natural_cosmos_qa": [
-        166,
+        171,
         200
       ],
       "natural/natural_snli": [
-        179,
+        178,
         200
       ]
     },
@@ -96,28 +96,28 @@
   "calibrated": {
     "rows": 1600,
     "accuracy_fraction": [
-      1007,
-      1200
+      1511,
+      1800
     ],
-    "accuracy": 0.8391666666666666,
-    "brier": 0.2261092775549145,
-    "nll": 0.42624728463092443,
-    "score_rps": 0.03323698212819523,
+    "accuracy": 0.8394444444444444,
+    "brier": 0.22190586234646303,
+    "nll": 0.4260766624652933,
+    "score_rps": 0.03535865911571265,
     "by_family": {
       "old/stage4_dense_table": [
         92,
         100
       ],
       "old/stage4_scope": [
-        94,
+        95,
         100
       ],
       "old/stage4_arithmetic": [
-        80,
+        79,
         100
       ],
       "old/stage4_ordinal": [
-        90,
+        89,
         100
       ],
       "old/stage4_natural_nli": [
@@ -125,11 +125,11 @@
         200
       ],
       "old/stage4_automaton": [
-        54,
+        52,
         100
       ],
       "old/stage4_boolean": [
-        69,
+        71,
         100
       ],
       "old/stage4_relations": [
@@ -137,47 +137,47 @@
         100
       ],
       "old/stage4_registers": [
-        77,
+        74,
         100
       ],
       "natural/natural_squad2_answerability": [
-        186,
+        185,
         200
       ],
       "natural/natural_cosmos_qa": [
-        166,
+        171,
         200
       ],
       "natural/natural_snli": [
-        179,
+        178,
         200
       ]
     },
-    "temperature": 1.0155830584882959,
+    "temperature": 1.3231350559653137,
     "probability_source": "binary64-logits-diagnostic"
   },
   "inputs": {
     "old": {
       "data_sha256": "5ba3bd7527da831d77cec8c6a2f260a71e10a57742e356510c719f8f30d61a70",
-      "predictions_sha256": "07786988e0631422be3ec70e769abcc3b03c63785890aa51a6a147b1db43c54a",
-      "summary_sha256": "7237145e9368e083602d9efba7fe1c412084908db7212c673dbb656dbf20c2d9"
+      "predictions_sha256": "7212da3cc1423e28f046003b68287ba46eb51b196a37f96ce1f826e489fb5bea",
+      "summary_sha256": "8181fa1f1b5c0be1b74730d441ee0c437779533e0ad400c7488c555896cf6453"
     },
     "natural": {
       "data_sha256": "cc75c902fe9b51c4654d6ceaeb590fe135bdef6377d45bb774e391a66bb1fa25",
-      "predictions_sha256": "84212958359e8a716203d394b10a4f1f6bb2aaa00082ef09ad5ee0cec442cba7",
-      "summary_sha256": "8a772b0ec26315adc15d4fc9d2365f9456c01906f7c496e34f55f84202b21bfc"
+      "predictions_sha256": "3553fb8bbae557ad6fcc2ad8962baf704525a0f9f54cda43e650019e05000a32",
+      "summary_sha256": "380fb6686d467c19f9579ceec31ada73c4c3b6369fc3e8eb9e7599727409f7b8"
     }
   },
-  "protocol_sha256": "441f06467d11f3374f9bda7a3abdfc23fb2274fb3db122dfae97a97715375e1b",
+  "protocol_sha256": "843b6c9f4db403c14e2e2e5ed1314c72f8d174b2fe66cb7a80a238d1091439c6",
   "weighted_math_sha256": "19c8dd94e9f078810be943bdcbf946b842a20ba77aac359a7faade5f62b19709",
-  "model_manifest_sha256": "a98e10438585d0891fabd8ba8447cc49385ac349334143d38dd28df397843424",
-  "winner_lock_sha256": "c369b72c875eafd3a9c78128a91a94f7e2c22762303419d882b1746309386d89",
-  "both_winners_lock_sha256": "a9e0dd8cbffae656b20cb8bebe664319462edc5f5be1d5b07bee7e5550b0d106",
+  "model_manifest_sha256": "9755d4c393ebfba77fa48bcb7b0949323c468ec68839fe6a52674187b85cc260",
+  "winner_lock_sha256": "b1e182dadfbbb46a26eedf423eec702c72f0ccbd36e6ccc15664dc7fd5a0779f",
+  "both_winners_lock_sha256": "a53765cf1ce7c541a11980538a858793ee967abb166b86dd32b07de1548f66bc",
   "heldout_used": false,
   "SELECT_used_for_temperature": false,
   "qualification_established": false,
   "dev_sha256": "45b4cd46acc4be53b95c7ed8f333b3533972296a4d0557c7b8c48c9cab6ced61",
-  "raw_predictions_sha256": "cf0b1a17008bddc2e3e18289866993770c41d4a5af5f6f4219508bb34fcabefa",
+  "raw_predictions_sha256": "ba311f5a6bc73182651bdf5c9744f0ed95921110501ddc93dcfd20a1d06bceff",
   "ranking_unchanged": true,
   "ranking_unchanged_scope": "Positive scalar preserves mathematical within-row logit order; native float32 outputs still require the separate full public reload proof.",
   "calibration_objective": "0.5mean9(old family mean NLL)+0.5mean3(natural family mean NLL)",
@@ -203,9 +203,9 @@
   "uniform_1600_fit": false,
   "temperature_refitted": false,
   "bridge": {
-    "schema": "natural-CAL-export-v1",
+    "schema": "Nox-retention-CAL-export-v1",
     "family": "Nox",
-    "arm": "natural_treatment",
+    "arm": "lambda025",
     "rows": 1600,
     "panel_order": [
       "old",
@@ -215,11 +215,11 @@
       "old": 1000,
       "natural": 600
     },
-    "original_temperature_sha256": "7ee6178ac029d830efea71305957eb4b8b788cce2e9e44d1717ffcb523ce732b",
-    "producer_complete_sha256": "3341c16d87acc08f36cf3ae18b3813413e01a05898e58db97ae6b332fad842d0",
-    "producer_freeze_sha256": "5ad8ecbd1151f303216d698ed90444c8f318265879896802c4a8e4c4e9b7d654",
-    "original_protocol_sha256": "441f06467d11f3374f9bda7a3abdfc23fb2274fb3db122dfae97a97715375e1b",
-    "ranking_sha256": "e888c82008dd82106ad160e187eac0c358c33b8d88face0f925d8702a204f11d",
+    "original_temperature_sha256": "4839d6c75d9710c8db5485802a2391a9d15bb7da5d3085d6b59b853dc91bcf51",
+    "producer_complete_sha256": "40f185c928724ceb10d1aecbe35ef36b633f5330ec9db5a19b2bfc7bd22af693",
+    "producer_freeze_sha256": "f012c928ec060983f5a6523ed5d24c7913af32978247ff4b00317ef1e97957f2",
+    "original_protocol_sha256": "843b6c9f4db403c14e2e2e5ed1314c72f8d174b2fe66cb7a80a238d1091439c6",
+    "ranking_sha256": "fe4910d9eef281729279ce9c9fda7eed77fbea431910d35f8876355cf3058e6c",
     "sequential_B8_boundary_preserved": true,
     "normalization_profile_sha256": "be32858d15233e0a3fbee0e4257fb02be0b3439deee4eb9c3f61151df7b73850"
   }