# Official typed-decisions TEST

The fixed final Lex checkpoint and the official `laya-typed-decisions` specialist were each evaluated once on the original 2,000 TEST decisions, from 400 state components. Both models support all 2,000 complete inputs within 1,024 tokens, their native windows and their untruncated official-default support. No TRAIN state component or full input overlaps this TEST support. The test contains four English workflows and 20 question schemas. This is not evidence of multilingual or unseen-schema generalization.

| Metric | Lex | Official specialist (shipped temperature) |
|---|---:|---:|
| Original-label hard accuracy | **78.15%** (1,563/2,000) | 76.60% (1,532/2,000) |
| Soft NLL | 0.961902 | **0.884390** |
| Mean soft Brier | 0.030858 | **0.018452** |
| Score RPS | 0.022994 | **0.015602** |
| Score expected-value MAE | 0.251408 | **0.242408** |

Lex's accuracy gain is 1.55 percentage points (31 decisions). The paired 95% interval is **[-0.20, +3.15125] percentage points**, which includes zero. The interval uses 2,000 state-component bootstrap samples, seed 20260921, stratified by each component's workflow set; complete components are retained. This supports a higher observed hard-accuracy point, not statistically significant or uniformly better performance. Probability metrics are worse for Lex in this comparison. No calibration was fitted.

| Type | Decisions | Lex accuracy | Official accuracy |
|---|---:|---:|---:|
| Choice | 600 | 74.00% | 73.33% |
| Noul | 600 | 84.67% | 85.67% |
| Score | 800 | 76.375% | 72.25% |

| Workflow | Decisions | Lex accuracy | Official accuracy |
|---|---:|---:|---:|
| Agent trace observability | 500 | 72.4% | 73.0% |
| Customer service | 500 | 79.6% | 76.4% |
| Invoice processing | 500 | 84.4% | 80.4% |
| Security incidents | 500 | 76.2% | 76.6% |

Hard accuracy uses the source's original semantic hard label. Choice and Score use the first maximum in supplied order; Noul uses `p_yes >= 0.5`. Soft NLL uses the explicitly normalized original soft label distribution and natural logarithms (probability floor 1e-12). Brier averages squared probability error over the valid candidates; ordinal Score RPS averages cumulative-distribution squared error over K−1 boundaries. Score MAE compares the predicted expected value against the original source score. The official raw-temperature accuracy is also 76.60%; raw soft NLL is 0.869598.

The frozen TEST parquet SHA256 is `4f294f218ea1da27f3efef936359389c62ea4d3973a41457732990f1d31b647c`. The actual saved-summary SHA256 is `1a21955b24be16cefa2963779e71edf4d6b5adcd14679fc438e6a87e3e604b88`. No TEST text, labels or predictions are redistributed. The annotations are synthetic/teacher-derived; accuracy does not certify operational safety or calibrated business utility.