Add SV0: Swedish one-pass form specialist (fp16 + int8 Core ML, eval receipts, MIT)

.gitattributes

@@ -1,35 +1,5 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
 *.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
+coreml/**/weight.bin filter=lfs diff=lfs merge=lfs -text
+coreml/sv0_forms_fp16_options40.mlpackage/Data/com.apple.CoreML/model.mlmodel filter=lfs diff=lfs merge=lfs -text
+coreml/sv0_forms_int4_options40.mlpackage/Data/com.apple.CoreML/model.mlmodel filter=lfs diff=lfs merge=lfs -text
+coreml/sv0_forms_int8_options40.mlpackage/Data/com.apple.CoreML/model.mlmodel filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Precisit AB
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,115 @@
+---
+license: mit
+language:
+  - sv
+library_name: coreml
+pipeline_tag: text-classification
+tags:
+  - jev
+  - system-one
+  - one-pass-scorer
+  - computer-use
+  - form-filling
+  - swedish
+  - coreml
+  - apple-silicon
+base_model: []
+---
+
+# One-Pass SV-Forms (SV0) — a Swedish one-pass form specialist (research checkpoint)
+
+**This is a research checkpoint, not a product.** It is a 706 048-parameter model that scores a
+supplied list of options in **one forward pass** — the "Jev"/System One shape — trained on
+**synthetic** Swedish forms by a synthetic-data recipe we are publishing alongside it. It
+generates no text, needs no tokenizer, and runs in ~1 ms per decision on an Apple laptop or
+phone (Core ML package: 1.4 MB fp16, 787 KB int8).
+
+| | |
+| --- | --- |
+| Task | given a UI element (role, label, state) plus the entities extracted from a document, pick one option: `fyll <entity>` / `kryssa` / `klicka` / `hoppa över` |
+| Architecture | byte-level embeddings + 2-layer Transformer encoder (width 128, 4 heads) + option-attention head; `tinyx` |
+| Parameters | 706 048 (2.83 MB fp16 checkpoint) |
+| Context / option budget | 224 bytes of context, 96 bytes per option, **up to 40 options** |
+| Training data | 900 synthetic Swedish form episodes (21 306 decisions), generated locally; **no real form, person or customer data** |
+| Licence | MIT (weights and code) |
+| Lineage | independent implementation of the same contract as [Cua's CUA-S1](https://huggingface.co/cua-ai/cua-s1-forms) and [jevlike](https://github.com/vinnylarouge/jevlike); trained from scratch, not a fine-tune of either |
+
+## Measured results (our harness, Apple M4)
+
+| Run | decisions | top-1 | majority-class baseline | ECE | silently skipped a required fill |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| Held-out synthetic test (form-signature disjoint) | 2 315 | **83.02 %** | 50.45 % | 0.017 | **76 (7.5 % of fills)** |
+| Hand-written out-of-distribution Swedish demo | 50 | **86.00 %** | 64.00 % | 0.092 | 3 (of 32 fills) |
+| Shuffled-context control (test) | 2 315 | 34.08 % | 50.45 % | 0.497 | 264 |
+
+Per-action accuracy on the test split: `check` 98.3 %, `click` 93.7 %, `skip` 88.0 %,
+`fill` 75.5 %. The shuffled-context control is the interesting one: rotating contexts between
+rows drops the model below the majority baseline, i.e. it really reads the element label and the
+document rather than exploiting option statistics.
+
+For comparison, the released English checkpoint of the same family scores **21.4 %** on exactly
+these Swedish rows — below the majority baseline, at its own shuffled-control floor. The
+language was the barrier, not the contract.
+
+Core ML export (same checkpoint, converted with coremltools 9.0):
+
+| Variant | package | top-1 | argmax parity vs PyTorch | median latency | p95 |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| fp16 (CPU + ANE) | 1.4 MB | 83.00 % | 0.99870 | 1.31 ms | 1.46 ms |
+| **int8 (CPU + ANE)** | **787 KB** | **83.09 %** | 0.99611 | 1.32 ms | 1.40 ms |
+| int4 (CPU + ANE) | 481 KB | 50.87 % | 0.48702 | 1.97 ms | 2.10 ms |
+| PyTorch reference | 2.83 MB | 83.02 % | — | — | — |
+
+We publish the int4 result because it is the most useful thing in this table: the same 4-bit
+palettisation costs 0.06 pp on a converged model and destroys this one, and it also fails ANE
+compilation. **Quantisation headroom is a property of the training run, not of the architecture.**
+Use int8. `int4-weights/` ships the collapsed variant for reproducibility only.
+
+## What it is NOT
+
+- **Not trained or tested on real Swedish forms.** The corpus is synthetic (`Label: value`
+  document entities, `.invalid` e-mail domains, fictional names, locally generated
+  personnummer-shaped strings with no link to any real person) and the only non-synthetic
+  evidence is a 50-decision set we wrote ourselves.
+- **Not a general-purpose assistant or an autonomous agent.** It does not generate text, cannot
+  invent a value it was not given, and does not decide execution order.
+- **Not finished.** Validation was still improving when the run stopped (48 % → 63 % → 79 % →
+  83 % top-1 over four epochs), so treat 83 % as a floor for this recipe, not a ceiling.
+- **Not safe to run unsupervised on real data.** 7.5 % of required fills are answered "skip" —
+  a required field that silently stays empty. Any real integration must verify outcomes outside
+  the model (fail-closed execution, dry run, one submit, human review before consequential
+  actions) — see the runtime contract in Cua's `planner.py` for a good pattern.
+
+## Intended use
+
+Research on bounded, high-volume Swedish interface workflows where the option set is supplied by
+deterministic code: filling forms from an extracted document, triaging/routing where the choices
+are known in advance, or as a **criteria-decision** layer where a calibrated distribution per
+question is more useful than generated prose. Also as a worked example of how to build and
+measure a language-local one-pass specialist.
+
+## Usage
+
+```python
+from pathlib import Path
+from huggingface_hub import hf_hub_download
+
+# the checkpoint format is <name>.safetensors + <name>.json (architecture + hashes)
+weights = Path(hf_hub_download("precisit/one-pass-sv-forms", "sv0-forms.safetensors"))
+hf_hub_download("precisit/one-pass-sv-forms", "sv0-forms.json", local_dir=weights.parent)
+# then either use the Core ML packages (no Python runtime needed) or the PyTorch code in
+# precisit/onepass — both are in this repository.
+```
+
+The context string and options must be built exactly as in training (byte ids = UTF-8 byte + 1,
+zero-padded; `UPPGIFT fyll i formuläret från dokumentet och skicka sedan in` / `FORM <title>` /
+`ELEMENT <role> "<label>" value="…"`). A mismatch there is the most likely cause of poor output —
+the recipe and the generator are in the toolkit repository.
+
+## Licence and attribution
+
+MIT. The architecture, training loop and evaluation metrics come from Cua's MIT-licensed
+`libs/cua-s1` (see `THIRD_PARTY_NOTICES.md`); the option-attention head design is credited there
+to `jevlike` (MIT). The Swedish catalogue, synthetic generator, training run, measurements and
+Core ML export are ours — **Precisit AB, 2026**. No TypeSafe AI code, weights or data was used;
+"Jev" is their product and this is an independent implementation of a similar interface.

THIRD_PARTY_NOTICES.md

@@ -0,0 +1,34 @@
+# Third-party notices
+
+This release is an independent implementation of a public model interface. It uses code and
+design from two MIT-licensed projects, and it is **not** affiliated with, derived from, or
+endorsed by TypeSafe AI (the makers of "Jev").
+
+## trycua/cua — libs/cua-s1 (MIT)
+
+- Source: https://github.com/trycua/cua/tree/main/libs/cua-s1
+- Revision used: `9bbfa7d` (the commit that links the Hugging Face artifacts)
+- What we use: the `tinyx` architecture (`cua_s1/model.py`), the checkpoint format
+  (`cua_s1/checkpoint.py`), the synthetic-data generator logic (`cua_s1/synth.py`), the
+  training loop (`training/train.py`) and the evaluation metrics (`evals/metrics.py`).
+  The files are vendored unmodified in the toolkit repository under `vendor/`, and the
+  generated corpus follows the same row/dataset format.
+- Copyright and licence text: see the upstream repository's `LICENSE` (MIT).
+
+Our model is **trained from scratch** on our own Swedish synthetic corpus: it shares the
+architecture, the trainer and the data format, not any weights or data with Cua's release.
+
+## vinnylarouge/jevlike (MIT), via Cua's attribution
+
+- Source: https://github.com/vinnylarouge/jevlike
+- The option-attention head design (`AttentionHead`) and the byte-level collation shape used by
+  the `tinyx` model originate in `jevlike`; Cua's `model.py` credits commit `94f5fd1`
+  (MIT, Copyright 2026 Minimal Labs) and we keep that attribution intact in the vendored code.
+
+## What was NOT used
+
+- No TypeSafe AI code, weights, data or API output. "Jev" and "System One" are TypeSafe AI's
+  terms; this release is an independent model that implements a similar input/output contract.
+- No real personal data. The training corpus is synthetic: fictional names, `.invalid`
+  e-mail domains, locally generated personnummer-shaped identifiers that do not correspond to
+  any individual, and invented organisations.

coreml/conversion.json

@@ -0,0 +1,48 @@
+{
+  "checkpoint_sha256": "b9030d7a7b7abfaf70dd1cc2a390568eaa3a55849daa0757b60d01dbab162356",
+  "created_utc": "2026-09-21T08:00:57Z",
+  "limits": {
+    "context_bytes": 224,
+    "max_options": 40,
+    "option_bytes": 96
+  },
+  "model": "sv0-forms (spike jev-sv0)",
+  "model_config": {
+    "context_tokens": 224,
+    "encoder": "tinyx",
+    "heads": 4,
+    "layers": 2,
+    "option_tokens": 96,
+    "rank": 128,
+    "width": 128
+  },
+  "parameters": 706048,
+  "tooling": {
+    "coremltools": "9.0",
+    "python": "3.12.14",
+    "torch": "2.7.0"
+  },
+  "trace_check": null,
+  "variants": {
+    "fp16": {
+      "minimum_target": "iOS17/macOS14",
+      "path": "sv0_forms_fp16_options40.mlpackage",
+      "precision": "float16"
+    },
+    "int4": {
+      "granularity": "per_grouped_channel",
+      "group_size": 32,
+      "minimum_target": "iOS18/macOS15",
+      "mode": "KMEANS",
+      "path": "sv0_forms_int4_options40.mlpackage",
+      "precision": "int4_weights_float16_compute"
+    },
+    "int8": {
+      "granularity": "per_tensor",
+      "minimum_target": "iOS17/macOS14",
+      "mode": "UNIFORM",
+      "path": "sv0_forms_int8_options40.mlpackage",
+      "precision": "int8_weights_float16_compute"
+    }
+  }
+}

coreml/sv0_forms_fp16_options40.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfe30d2473f7c9a0d804f7b3306f252fcad4a307fa9202d7bf218049be41401a
+size 58608

coreml/sv0_forms_fp16_options40.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf1b0db68a002fd003cf68278f5e8c049c9360b9b5b3754205692e20a63bb43c
+size 1448256

coreml/sv0_forms_fp16_options40.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "9B448D09-24D9-4E7E-B194-43D3D4AEE91F": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        },
+        "D3C625CB-29B6-456F-8FB3-B1111990E4E6": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        }
+    },
+    "rootModelIdentifier": "9B448D09-24D9-4E7E-B194-43D3D4AEE91F"
+}

coreml/sv0_forms_int4_options40.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e145df71d2ee1a4566126487afd80cd6409aeda278d42494640f49dc17016168
+size 58303

coreml/sv0_forms_int4_options40.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:53d889e570414557693f59309731d573aa6ddee95746dc9b9735a63afa2ad9ec
+size 433664

coreml/sv0_forms_int4_options40.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "495DA951-9B89-4CE4-8656-67B3D2E0038C": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        },
+        "7B688A87-1DA1-4178-9B4F-2F6A9FD6C274": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        }
+    },
+    "rootModelIdentifier": "495DA951-9B89-4CE4-8656-67B3D2E0038C"
+}

coreml/sv0_forms_int8_options40.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c63811d8b278b206c03cfc40c7d311a1e91997e9f710d524a1372477a28be74
+size 62987

coreml/sv0_forms_int8_options40.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36c31c5e396138a45eb191fec6f637d3831ffaca6005ebed12b305979be6b732
+size 742528

coreml/sv0_forms_int8_options40.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "BAE712EF-2551-49DE-BA14-3D8AB687D268": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "E219151B-08B6-4AFD-BFEA-72D3F71825A6": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "E219151B-08B6-4AFD-BFEA-72D3F71825A6"
+}

eval/RESULTS-SV0-COREML.md

@@ -0,0 +1,94 @@
+# RESULTS-SV0-COREML — the Swedish SV0 specialist exported to Core ML
+
+**Verdict: PARTIAL — fp16 and int8 export cleanly and run on the Neural Engine at the same
+accuracy as PyTorch; int4 is _not_ usable for this checkpoint (83 % → 51 % top-1) and does not
+compile for the ANE.** Recommended deployment artifact: **int8, 787 KB, ~1.3 ms per decision.**
+
+## What was exported
+
+`export_sv_coreml.py` in this directory converts `runs/sv-tinyx/model` (the SV0 checkpoint,
+706 048 parameters) with coremltools 9.0, mirroring the fixed-shape contract of the
+FluidInference CUA-S1 release: `context_ids (1, 224)` / `option_ids (1, 40, 96)` /
+`option_mask (1, 40)`, all int32 byte-ids (byte + 1, 0 = pad), output `logits (1, 40)`.
+`max_options` is **40** here rather than their 32 because our corpus tops out at 37 options —
+the ceiling is baked into the export, so it must be sized from the widest real input.
+
+Variants: fp16 (iOS17), int8 (UNIFORM, per-tensor, iOS17), int4 (KMEANS, per-grouped-channel,
+group 32, iOS18). `conversion.json` records hashes, config and tooling versions.
+
+## Export-time checks (all passed before any measurement)
+
+| Check | Result |
+| --- | --- |
+| Export forward vs vendored model, 256 real rows, max abs logit difference | **0.0** (bit-identical) |
+| Traced graph vs model, 64 real rows | max abs Δ 7.2 × 10⁻⁶, argmax agreement **1.0** |
+
+## Measurements (Apple M4, macOS 26.6.2; 2 312 scored rows after 3 warm-up rows; same grader as the spike)
+
+| Variant | package | test top-1 | demo top-1 | silent skips (of 1 008 fills) | argmax parity vs PyTorch | mismatches | median | p95 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| PyTorch (MPS), reference | 2 828 784 B | **83.02 %** | 86 % | 76 | — | — | — | — |
+| **fp16, CPU + ANE** | 1 507 481 B (1.44 MB) | 83.00 % | 85.11 % | 76 | 0.99870 | 3 / 2 312 | 1.307 ms | 1.460 ms |
+| fp16, CPU only | 1 507 481 B | 83.04 % | — | 76 | 0.99913 | 2 / 2 312 | 1.734 ms | 1.917 ms |
+| **int8, CPU + ANE** | **806 132 B (787 KB)** | **83.09 %** | 85.11 % | 76 | 0.99611 | 9 / 2 312 | 1.316 ms | 1.402 ms |
+| int4, CPU + ANE | 492 584 B (481 KB) | **50.87 %** | 27.66 % | 954 | 0.48702 | 1 186 / 2 312 | 1.968 ms | 2.099 ms |
+
+Fills per-action on fp16: `fill` 75.40 %, `skip` 88.26 %, `check` 96.55 %, `click` 93.67 % —
+within a few rows of the PyTorch run, i.e. the export is behaviour-preserving.
+
+## What the numbers say
+
+1. **The export works, and the ANE helps here too**: fp16 median 1.307 ms with the ANE vs
+   1.734 ms CPU-only, p95 1.460 ms vs 1.917 ms. Smaller absolute win than on the English model
+   (whose weights are better conditioned), but the same direction.
+2. **int8 is free and is the artifact to ship**: 787 KB (52 % of fp16), top-1 83.09 % — one
+   decision *better* than PyTorch on this split — 9 argmax deviations out of 2 312, identical
+   silent-skip count, and the same ANE behaviour.
+3. **int4 does not transfer to an undertrained model.** On Cua's English checkpoint the same
+   kind of palettisation cost 14 decisions in 24 367 (0.06 pp); here it costs 1 186 in 2 312
+   (51 pp) and raises silent skips from 76 to 954. Two mechanisms are plausible and both point
+   the same way: a model that has not converged stores information in fine weight differences
+   that a 4-bit palette cannot represent, and the per-grouped-channel kmeans graph also fails
+   to compile for the ANE (`ANECCompile() FAILED`, seen repeatedly during save and load), which
+   is why its latency is worst despite being the smallest artifact. **Quantisation headroom is
+   a property of the training run, not of the architecture** — that is the transferable finding.
+4. **fp16 parity is 99.87 %, not 100 %**, and that is expected rather than alarming: three
+   rows change their argmax under fp16 rounding on a model whose decision margins are thin
+   (83 % top-1). On the well-trained English model the same conversion showed zero deviations.
+   Parity must therefore be *measured* per checkpoint, not assumed — exactly what this harness
+   does.
+
+## Pitfalls found while exporting (all cost a failed run)
+
+- **`torch.jit.trace` under `torch.no_grad()` takes PyTorch's fused sparsity fast path**
+  (`torch._transformer_encoder_layer_fwd`), which coremltools cannot convert. Trace with grad
+  *enabled*.
+- **The coremltools torch frontend has no `__or__` for bool tensors** → use
+  `torch.logical_or` / `torch.logical_not`. (`&`, `~` alone, and in-place `mask[:, 0] = True`
+  all fail; probe the op support in a 5-line script before rewriting a model.)
+- **`clamp_min(1)` with a Python int trips `assert x.dtype == y.dtype`** in the frontend; pass a
+  same-dtype tensor constant.
+- **`per_grouped_channel` palettisation requires an iOS18 deployment target**, so the int4
+  variant needs its own fp16 base converted at iOS18 (the upstream release does the same).
+- **k-means palettisation needs `scikit-learn`**, which is not a coremltools dependency; the
+  error arrives mid-run, after the fp16 conversion has already succeeded — hence the
+  `REUSE_FP16=1` cache flag in the export script.
+- **Palettizer API in coremltools 9.0**: wrap the op config —
+  `OptimizationConfig(global_config=OpPalettizerConfig(...))` — and note the mode names are
+  upper-case (`UNIFORM`, `KMEANS`) while granularity is `per_tensor` / `per_grouped_channel`
+  only.
+- **Check that the traced graph is the model** before converting: `torch.jit.trace(..., check_trace=False)`
+  suppresses a noisy replay check, so the export script compares logits itself and aborts on
+  mismatch.
+
+## Reproduction
+
+```bash
+# (in the spike venv with coremltools 9.0, torch 2.7, safetensors, scikit-learn)
+python check_export_forward.py     # export forward is bit-identical to the model
+REUSE_FP16=1 python export_sv_coreml.py    # fp16 + int8 + int4 mlpackages, conversion.json
+python eval_sv_coreml.py           # accuracy, silent skips, parity, latency -> RESULTS-sv0-coreml.json
+```
+
+Artifacts (not in Git): `sv-coreml/*.mlpackage`, `RESULTS-sv0-coreml.json`,
+`export.log`, `eval-coreml.log`.

eval/RESULTS-SV0-pytorch.md

@@ -0,0 +1,94 @@
+# RESULTS-SV0 — Swedish Jev-shaped form specialist
+
+**Verdict: VALIDATED, with documented limits.**
+The Cua-S1 contract trains on our own Swedish synthetic data with the upstream MIT code,
+on one laptop, and reaches **83.02 %** top-1 on a held-out, form-disjoint synthetic test
+(2 315 decisions) and **86 %** on a hand-written out-of-distribution Swedish demo (50
+decisions) — against **21.4 %** / **20 %** for the released English `cua-s1-forms`
+checkpoint on the very same rows. The result is undertrained (validation was still
+improving at the last epoch), so 83 % is a floor, not a ceiling.
+
+## Protocol as run
+
+| Item | Setting |
+| --- | --- |
+| Architecture | `tinyx` (byte-level 2-layer Transformer encoder, width 128, 4 heads) + jevlike `AttentionHead`, context 224 / option 96 bytes — identical to upstream |
+| Code | `trycua/cua` → `libs/cua-s1` @ `9bbfa7d`, vendored unmodified (`vendor/`, MIT) |
+| Corpus | `sv_synth.py` + `concepts_sv.py` (ours): 900 episodes, seed 2026, splits by form field signature |
+| Rows | train 21 306 / validation 2 855 / test 2 315 decisions; 725 / 96 / 79 distinct form signatures, **zero overlap** between splits |
+| Training | AdamW lr 2e-3, cosine + 5 % warmup, wd 1e-2, 4 epochs, batch 64, seed 7, MPS, 1 927 s |
+| Checkpoint rule | best validation NLL (upstream's rule) — epoch 4 |
+| Grader | `eval_sv.py`, unchanged for both checkpoints |
+
+## Numbers
+
+| Run | decisions | top-1 | majority baseline | ECE | skipped a required fill | per action |
+| --- | ---: | ---: | ---: | ---: | ---: | --- |
+| Swedish specialist — synthetic test (form-disjoint) | 2 315 | **83.02 %** | 50.45 % | 0.01656 | 76 | `check` 98.28 % (n=58), `click` 93.67 % (n=79), `fill` 75.55 % (n=1010), `skip` 88.01 % (n=1168) |
+| Swedish specialist — handwritten OOD demo (50) | 50 | **86.00 %** | 64.00 % | 0.09176 | 3 | `check` 100 % (n=3), `click` 100 % (n=3), `fill` 81.25 % (n=32), `skip` 91.67 % (n=12) |
+| Swedish specialist — shuffled-context control (test) | 2 315 | 34.08 % | 50.45 % | 0.49735 | 264 | `check` 15.52 %, `click` 0 %, `fill` 1.29 %, `skip` 65.67 % |
+| Swedish specialist — shuffled-context control (demo) | 50 | 6.00 % | 64.00 % | 0.78579 | 6 | all actions ≤ 25 % |
+| English reference (`cua-s1-forms`) — synthetic test | 2 315 | 21.38 % | 50.45 % | 0.64072 | 371 | `check` 93.10 %, `click` 26.58 %, `fill` 5.54 %, `skip` 31.16 % |
+| English reference — handwritten OOD demo | 50 | 20.00 % | 64.00 % | 0.66098 | 16 | `check` 66.67 %, `click` 33.33 %, `fill` 15.62 %, `skip` 16.67 % |
+| English reference — shuffled-context control (test) | 2 315 | 17.50 % | 50.45 % | 0.67954 | 363 | `check` 60.35 %, `click` 6.33 %, `fill` 1.78 %, `skip` 29.71 % |
+
+Per-epoch validation: 48.23 % → 62.98 % → 78.63 % → **83.01 %** top-1 (train NLL
+2.06 → 0.45; val NLL 1.75 → 0.52). Still improving when the run stopped.
+
+## Reading the numbers
+
+1. **The contract transfers; the language boundary was the real barrier.** The English
+   checkpoint on Swedish rows sits at **21.4 %**, barely above its own shuffled-context
+   control (17.5 %) and *below* the majority-action baseline (50.5 %), with ECE 0.64 — it
+   is not reading the content, it is guessing confidently. On the hand-written demo it
+   scores 20 % against a 64 % baseline, i.e. worse than always answering "fill". The
+   Swedish specialist, same architecture, same grader, same rows: 83 % / 86 %.
+2. **The shuffled control is the floor that matters.** Our specialist drops from 83.02 %
+   to 34.08 % when contexts are rotated between rows (below the 50.45 % majority
+   baseline), which is the evidence that it uses the element label and the document, not
+   option statistics. The English reference barely moves (21.4 % → 17.5 %) — its score was
+   near the floor to begin with.
+3. **`fill` is the weak action, and the silent-skip failure mode persists.** 75.5 % of
+   fills are right; 76 of 1 010 fill decisions (7.5 %) are answered `hoppa över` instead.
+   In a real workflow that is the dangerous class: a required field that looks done and is
+   not. It is much smaller than the English reference's (371/1 010), but it is not zero,
+   and it is the reason a production integration needs a per-item check outside the model.
+4. **Calibration is good in-distribution and degrades out-of-distribution** (ECE 0.017 →
+   0.092), which is exactly the wrong direction for confidence routing if the threshold is
+   tuned on synthetic data.
+5. **It is undertrained.** Both the validation curve and the comparison with the reference
+   (trained on ~150 k rows vs our 21 k) say the remaining error is largely a data-scale
+   effect. The next honest step is more episodes, not a different architecture.
+
+## What this does not establish
+
+- Nothing about real Swedish forms. The demo set is hand-written by the same person who
+  wrote the catalogue, so it shares the author's vocabulary; a genuine test needs real
+  blanketter (Skolverket/Försäkringskassan/kommunala) or our own live forms.
+- Nothing about the runtime: this spike scores decisions, it does not drive a GUI.
+  Fill/click execution, retries and fail-closed behaviour were not exercised (upstream's
+  `planner.py` refuses to execute without a compatible driver contract).
+- Nothing about English transfer in the other direction: we never trained a Swedish model
+  on English rows, and the corpus changes *language and label vocabulary at once*, so the
+  ablation "language vs vocabulary" is still open.
+- Reproducibility is bounded by the environment: torch 2.14 on Apple Silicon MPS, seed 7,
+  `deterministic=False` (deterministic algorithms were too slow on MPS), so the exact
+  numbers are one draw, not a bit-reproducible receipt.
+
+## Follow-ups worth pre-registering
+
+1. **Scale test**: 4 000–10 000 episodes, 6 epochs, batch 32–64 on a machine with room —
+   does top-1 cross ~95 % and does the silent-skip rate fall below 2 % of fills?
+2. **Language ✕ vocabulary ablation**: Swedish documents + English labels, and the mirror,
+   to attribute the win.
+3. **Real-form OOD**: hand-build 3–5 authentic Swedish blanketter (never in the catalogue)
+   and score before touching the corpus again.
+4. **Abstention outside the model**: a spread/top-probability gate plus a "required field
+   still empty" check, measured on the silent-skip class specifically.
+
+## Reproduction
+
+See `README.md` in this directory. Runtime: ~5 min generation, 32 min training, seconds of
+scoring on one Apple Silicon laptop. Corpus and checkpoint are not in Git
+(`~/agent-data/jevlike-sv-spike/`); `data/sv/manifest.json` carries the SHA-256 of each split
+(test `92c0dfec…d272`).

eval/corpus-manifest.json

@@ -0,0 +1,44 @@
+{
+  "episodes": 900,
+  "format": "cua-s1-choice-jsonl",
+  "format_version": 1,
+  "generator": {
+    "config": {
+      "filled_field_probability": 0.6,
+      "hard_negative_probability": 0.35,
+      "max_distractor_entities": 3,
+      "max_extra_entities": 5,
+      "missing_entity_probability": 0.12,
+      "partial_form_probability": 0.3,
+      "stale_value_probability": 0.05
+    },
+    "format": "spike.sv_synth",
+    "task_line": "UPPGIFT fyll i formul\u00e4ret fr\u00e5n dokumentet och skicka sedan in"
+  },
+  "ratios": {
+    "test": 0.1,
+    "train": 0.8,
+    "validation": 0.1
+  },
+  "seed": 2026,
+  "splits": {
+    "test": {
+      "episodes": 79,
+      "file": "test.jsonl",
+      "rows": 2315,
+      "sha256": "92c0dfec00239b6e1d5cc6db4f098d12fc90bc5a8832428d9df0f6aea380d272"
+    },
+    "train": {
+      "episodes": 725,
+      "file": "train.jsonl",
+      "rows": 21306,
+      "sha256": "b3a7ef25e4edb47b18be2601d1060a3589c7ed8563397fab25e27088017ee3fa"
+    },
+    "validation": {
+      "episodes": 96,
+      "file": "validation.jsonl",
+      "rows": 2855,
+      "sha256": "09298b6319b8471379b8f9e87f983558f5fa194b38271eac195670a8eb2ef5f1"
+    }
+  }
+}

eval/results-coreml.json

@@ -0,0 +1,246 @@
+{
+  "checkpoint": "runs/sv-tinyx/model",
+  "limits": {
+    "context_bytes": 224,
+    "max_options": 40,
+    "option_bytes": 96
+  },
+  "torch_baseline": {
+    "per_action": {
+      "check": {
+        "acc": 0.9827586206896551,
+        "n": 58
+      },
+      "click": {
+        "acc": 0.9367088607594937,
+        "n": 79
+      },
+      "fill": {
+        "acc": 0.7554455445544555,
+        "n": 1010
+      },
+      "skip": {
+        "acc": 0.8801369863013698,
+        "n": 1168
+      }
+    },
+    "rows": 2315,
+    "silent_skip_rate_of_fills": 0.07524752475247524,
+    "silent_skips": 76,
+    "top1": 0.8302375809935205
+  },
+  "variants": {
+    "fp16": {
+      "handwritten_demo__ane": {
+        "compute_units": "ComputeUnit.CPU_AND_NE",
+        "latency_ms_median": 1.2992918491363525,
+        "latency_ms_p95": 1.4392081648111343,
+        "mismatches_vs_torch": 0,
+        "parity_with_torch": 1.0,
+        "per_action": {
+          "check": {
+            "acc": 1.0,
+            "n": 3
+          },
+          "click": {
+            "acc": 1.0,
+            "n": 3
+          },
+          "fill": {
+            "acc": 0.7931034482758621,
+            "n": 29
+          },
+          "skip": {
+            "acc": 0.9166666666666666,
+            "n": 12
+          }
+        },
+        "rows": 47,
+        "silent_skip_rate_of_fills": 0.10344827586206896,
+        "silent_skips": 3,
+        "top1": 0.851063829787234
+      },
+      "package_bytes": 1507481,
+      "synthetic_test__ane": {
+        "compute_units": "ComputeUnit.CPU_AND_NE",
+        "latency_ms_median": 1.3071454595774412,
+        "latency_ms_p95": 1.4601247385144234,
+        "mismatches_vs_torch": 3,
+        "parity_with_torch": 0.9987024221453287,
+        "per_action": {
+          "check": {
+            "acc": 0.9827586206896551,
+            "n": 58
+          },
+          "click": {
+            "acc": 0.9367088607594937,
+            "n": 79
+          },
+          "fill": {
+            "acc": 0.753968253968254,
+            "n": 1008
+          },
+          "skip": {
+            "acc": 0.8808911739502999,
+            "n": 1167
+          }
+        },
+        "rows": 2312,
+        "silent_skip_rate_of_fills": 0.07539682539682539,
+        "silent_skips": 76,
+        "top1": 0.8300173010380623
+      },
+      "synthetic_test__cpu": {
+        "compute_units": "ComputeUnit.CPU_ONLY",
+        "latency_ms_median": 1.733562909066677,
+        "latency_ms_p95": 1.9167079590260983,
+        "mismatches_vs_torch": 2,
+        "parity_with_torch": 0.9991349480968859,
+        "per_action": {
+          "check": {
+            "acc": 0.9827586206896551,
+            "n": 58
+          },
+          "click": {
+            "acc": 0.9367088607594937,
+            "n": 79
+          },
+          "fill": {
+            "acc": 0.7549603174603174,
+            "n": 1008
+          },
+          "skip": {
+            "acc": 0.8808911739502999,
+            "n": 1167
+          }
+        },
+        "rows": 2312,
+        "silent_skip_rate_of_fills": 0.07539682539682539,
+        "silent_skips": 76,
+        "top1": 0.8304498269896193
+      }
+    },
+    "int4": {
+      "handwritten_demo__ane": {
+        "compute_units": "ComputeUnit.CPU_AND_NE",
+        "latency_ms_median": 1.8591247498989105,
+        "latency_ms_p95": 1.993041019886732,
+        "mismatches_vs_torch": 32,
+        "parity_with_torch": 0.3191489361702128,
+        "per_action": {
+          "check": {
+            "acc": 0.0,
+            "n": 3
+          },
+          "click": {
+            "acc": 0.0,
+            "n": 3
+          },
+          "fill": {
+            "acc": 0.034482758620689655,
+            "n": 29
+          },
+          "skip": {
+            "acc": 1.0,
+            "n": 12
+          }
+        },
+        "rows": 47,
+        "silent_skip_rate_of_fills": 0.9655172413793104,
+        "silent_skips": 28,
+        "top1": 0.2765957446808511
+      },
+      "package_bytes": 492584,
+      "synthetic_test__ane": {
+        "compute_units": "ComputeUnit.CPU_AND_NE",
+        "latency_ms_median": 1.9683334976434708,
+        "latency_ms_p95": 2.099000383168459,
+        "mismatches_vs_torch": 1186,
+        "parity_with_torch": 0.4870242214532872,
+        "per_action": {
+          "check": {
+            "acc": 0.0,
+            "n": 58
+          },
+          "click": {
+            "acc": 0.0,
+            "n": 79
+          },
+          "fill": {
+            "acc": 0.018849206349206348,
+            "n": 1008
+          },
+          "skip": {
+            "acc": 0.9914310197086547,
+            "n": 1167
+          }
+        },
+        "rows": 2312,
+        "silent_skip_rate_of_fills": 0.9464285714285714,
+        "silent_skips": 954,
+        "top1": 0.5086505190311419
+      }
+    },
+    "int8": {
+      "handwritten_demo__ane": {
+        "compute_units": "ComputeUnit.CPU_AND_NE",
+        "latency_ms_median": 1.31650036200881,
+        "latency_ms_p95": 1.4507090672850609,
+        "mismatches_vs_torch": 0,
+        "parity_with_torch": 1.0,
+        "per_action": {
+          "check": {
+            "acc": 1.0,
+            "n": 3
+          },
+          "click": {
+            "acc": 1.0,
+            "n": 3
+          },
+          "fill": {
+            "acc": 0.7931034482758621,
+            "n": 29
+          },
+          "skip": {
+            "acc": 0.9166666666666666,
+            "n": 12
+          }
+        },
+        "rows": 47,
+        "silent_skip_rate_of_fills": 0.10344827586206896,
+        "silent_skips": 3,
+        "top1": 0.851063829787234
+      },
+      "package_bytes": 806132,
+      "synthetic_test__ane": {
+        "compute_units": "ComputeUnit.CPU_AND_NE",
+        "latency_ms_median": 1.315583474934101,
+        "latency_ms_p95": 1.401625107973814,
+        "mismatches_vs_torch": 9,
+        "parity_with_torch": 0.9961072664359861,
+        "per_action": {
+          "check": {
+            "acc": 0.9827586206896551,
+            "n": 58
+          },
+          "click": {
+            "acc": 0.9367088607594937,
+            "n": 79
+          },
+          "fill": {
+            "acc": 0.753968253968254,
+            "n": 1008
+          },
+          "skip": {
+            "acc": 0.882604970008569,
+            "n": 1167
+          }
+        },
+        "rows": 2312,
+        "silent_skip_rate_of_fills": 0.07539682539682539,
+        "silent_skips": 76,
+        "top1": 0.8308823529411765
+      }
+    }
+  }
+}

eval/results-pytorch.json

@@ -0,0 +1,282 @@
+{
+  "device": "mps",
+  "reference": {
+    "checkpoint": "/tmp/cua1ckpt/cua-s1-forms.safetensors",
+    "checkpoint_bytes": 2828784,
+    "config": {
+      "context_tokens": 224,
+      "encoder": "tinyx",
+      "heads": 4,
+      "hf_model": "Qwen/Qwen2.5-0.5B",
+      "layers": 2,
+      "option_tokens": 96,
+      "rank": 128,
+      "width": 128
+    },
+    "label": "cua-s1-forms-english",
+    "sv_handwritten_demo": {
+      "action_mix": {
+        "check": 3,
+        "click": 3,
+        "fill": 32,
+        "skip": 12
+      },
+      "ece": 0.660977,
+      "examples": 50,
+      "majority_baseline": 0.64,
+      "per_action": {
+        "check": {
+          "acc": 0.66667,
+          "n": 3
+        },
+        "click": {
+          "acc": 0.33333,
+          "n": 3
+        },
+        "fill": {
+          "acc": 0.15625,
+          "n": 32
+        },
+        "skip": {
+          "acc": 0.16667,
+          "n": 12
+        }
+      },
+      "skipped_when_fill_expected": 16,
+      "top1": 0.2
+    },
+    "sv_handwritten_demo__shuffled_context": {
+      "action_mix": {
+        "check": 3,
+        "click": 3,
+        "fill": 32,
+        "skip": 12
+      },
+      "ece": 0.760258,
+      "examples": 50,
+      "majority_baseline": 0.64,
+      "per_action": {
+        "check": {
+          "acc": 0.66667,
+          "n": 3
+        },
+        "click": {
+          "acc": 0.0,
+          "n": 3
+        },
+        "fill": {
+          "acc": 0.0,
+          "n": 32
+        },
+        "skip": {
+          "acc": 0.25,
+          "n": 12
+        }
+      },
+      "skipped_when_fill_expected": 13,
+      "top1": 0.1
+    },
+    "sv_synthetic_test": {
+      "action_mix": {
+        "check": 58,
+        "click": 79,
+        "fill": 1010,
+        "skip": 1168
+      },
+      "ece": 0.640725,
+      "examples": 2315,
+      "majority_baseline": 0.50454,
+      "per_action": {
+        "check": {
+          "acc": 0.93103,
+          "n": 58
+        },
+        "click": {
+          "acc": 0.26582,
+          "n": 79
+        },
+        "fill": {
+          "acc": 0.05545,
+          "n": 1010
+        },
+        "skip": {
+          "acc": 0.31164,
+          "n": 1168
+        }
+      },
+      "skipped_when_fill_expected": 371,
+      "top1": 0.21382
+    },
+    "sv_synthetic_test__shuffled_context": {
+      "action_mix": {
+        "check": 58,
+        "click": 79,
+        "fill": 1010,
+        "skip": 1168
+      },
+      "ece": 0.679541,
+      "examples": 2315,
+      "majority_baseline": 0.50454,
+      "per_action": {
+        "check": {
+          "acc": 0.60345,
+          "n": 58
+        },
+        "click": {
+          "acc": 0.06329,
+          "n": 79
+        },
+        "fill": {
+          "acc": 0.01782,
+          "n": 1010
+        },
+        "skip": {
+          "acc": 0.29709,
+          "n": 1168
+        }
+      },
+      "skipped_when_fill_expected": 363,
+      "top1": 0.17495
+    },
+    "trainable_params": 706048
+  },
+  "sv": {
+    "checkpoint": "runs/sv-tinyx/model/model.safetensors",
+    "checkpoint_bytes": 2828784,
+    "config": {
+      "context_tokens": 224,
+      "encoder": "tinyx",
+      "heads": 4,
+      "layers": 2,
+      "option_tokens": 96,
+      "rank": 128,
+      "width": 128
+    },
+    "label": "swedish-specialist",
+    "sv_handwritten_demo": {
+      "action_mix": {
+        "check": 3,
+        "click": 3,
+        "fill": 32,
+        "skip": 12
+      },
+      "ece": 0.09176,
+      "examples": 50,
+      "majority_baseline": 0.64,
+      "per_action": {
+        "check": {
+          "acc": 1.0,
+          "n": 3
+        },
+        "click": {
+          "acc": 1.0,
+          "n": 3
+        },
+        "fill": {
+          "acc": 0.8125,
+          "n": 32
+        },
+        "skip": {
+          "acc": 0.91667,
+          "n": 12
+        }
+      },
+      "skipped_when_fill_expected": 3,
+      "top1": 0.86
+    },
+    "sv_handwritten_demo__shuffled_context": {
+      "action_mix": {
+        "check": 3,
+        "click": 3,
+        "fill": 32,
+        "skip": 12
+      },
+      "ece": 0.785789,
+      "examples": 50,
+      "majority_baseline": 0.64,
+      "per_action": {
+        "check": {
+          "acc": 0.0,
+          "n": 3
+        },
+        "click": {
+          "acc": 0.0,
+          "n": 3
+        },
+        "fill": {
+          "acc": 0.0,
+          "n": 32
+        },
+        "skip": {
+          "acc": 0.25,
+          "n": 12
+        }
+      },
+      "skipped_when_fill_expected": 6,
+      "top1": 0.06
+    },
+    "sv_synthetic_test": {
+      "action_mix": {
+        "check": 58,
+        "click": 79,
+        "fill": 1010,
+        "skip": 1168
+      },
+      "ece": 0.016565,
+      "examples": 2315,
+      "majority_baseline": 0.50454,
+      "per_action": {
+        "check": {
+          "acc": 0.98276,
+          "n": 58
+        },
+        "click": {
+          "acc": 0.93671,
+          "n": 79
+        },
+        "fill": {
+          "acc": 0.75545,
+          "n": 1010
+        },
+        "skip": {
+          "acc": 0.88014,
+          "n": 1168
+        }
+      },
+      "skipped_when_fill_expected": 76,
+      "top1": 0.83024
+    },
+    "sv_synthetic_test__shuffled_context": {
+      "action_mix": {
+        "check": 58,
+        "click": 79,
+        "fill": 1010,
+        "skip": 1168
+      },
+      "ece": 0.497346,
+      "examples": 2315,
+      "majority_baseline": 0.50454,
+      "per_action": {
+        "check": {
+          "acc": 0.15517,
+          "n": 58
+        },
+        "click": {
+          "acc": 0.0,
+          "n": 79
+        },
+        "fill": {
+          "acc": 0.01287,
+          "n": 1010
+        },
+        "skip": {
+          "acc": 0.65668,
+          "n": 1168
+        }
+      },
+      "skipped_when_fill_expected": 264,
+      "top1": 0.34082
+    },
+    "trainable_params": 706048
+  }
+}

example.py

@@ -0,0 +1,109 @@
+"""Minimal inference example for precisit/sv0-forms.
+
+Two routes: the Core ML package (no PyTorch needed) and the PyTorch checkpoint through the
+vendored Cua-S1 loader. Both take the same byte-level inputs; the encoder is:
+
+    ids = utf-8 bytes, truncated to the limit, each byte + 1, zero-padded
+    context: 224 bytes | option: 96 bytes | up to 40 options (the export's ceiling)
+
+Run:  python example.py            (Core ML, needs coremltools)
+      python example.py --torch    (PyTorch, needs torch + safetensors + the vendored package)
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+
+HERE = Path(__file__).resolve().parent
+CONTEXT_BYTES, OPTION_BYTES, MAX_OPTIONS = 224, 96, 40
+
+TASK = "UPPGIFT fyll i formuläret från dokumentet och skicka sedan in"
+FORM = "Exempelkliniken - Ny patientregistrering"
+ELEMENT = ('Edit "E-post" value=""',)
+OPTIONS = [
+    "fyll Förnamn: Anna",
+    "fyll Efternamn: Lindqvist",
+    "fyll E-post: anna.lindqvist@exempel.invalid",
+    "fyll Telefon: 070-341 22 87",
+    "kryssa",
+    "klicka",
+    "hoppa över",
+]
+
+
+def context_string() -> str:
+    role, label, state = ELEMENT[0], "E-post", 'value=""'
+    return f"{TASK}\nFORM {FORM}\nELEMENT {role} \"{label}\" {state}"
+
+
+def byte_ids(text: str, length: int) -> list[int]:
+    return [byte + 1 for byte in text.encode("utf-8", errors="replace")[:length]]
+
+
+def pad(ids: list[int], length: int) -> list[int]:
+    return ids + [0] * (length - len(ids))
+
+
+def via_coreml() -> None:
+    import coremltools as ct
+    import numpy as np
+
+    package = HERE / "coreml" / "sv0_forms_int8_options40.mlpackage"
+    model = ct.models.MLModel(str(package), compute_units=ct.ComputeUnit.CPU_AND_NE)
+    context = np.array([pad(byte_ids(context_string(), CONTEXT_BYTES), CONTEXT_BYTES)], dtype=np.int32)
+    options = np.zeros((1, MAX_OPTIONS, OPTION_BYTES), dtype=np.int32)
+    for index, option in enumerate(OPTIONS):
+        encoded = pad(byte_ids(option, OPTION_BYTES), OPTION_BYTES)
+        options[0, index] = np.array(encoded, dtype=np.int32)
+    mask = np.zeros((1, MAX_OPTIONS), dtype=np.int32)
+    mask[0, : len(OPTIONS)] = 1
+    output = model.predict({"context_ids": context, "option_ids": options, "option_mask": mask})
+    scores = output["logits"][0][: len(OPTIONS)]  # raw logits; softmax them if you need probabilities
+    best = int(scores.argmax())
+    print(f"chosen: {OPTIONS[best]!r}  (logit {float(scores[best]):.3f})")
+    print("logits:", {option: round(float(score), 3) for option, score in zip(OPTIONS, scores)})
+
+
+def via_torch() -> None:
+    import torch
+    from huggingface_hub import hf_hub_download
+
+    try:
+        from cua_s1.model import ChoiceExample, load_checkpoint, select_device  # type: ignore
+    except ImportError:  # pragma: no cover
+        raise SystemExit(
+            "The PyTorch route needs the vendored Cua-S1 loader: clone the toolkit repository "
+            "(precisit/onepass) and put its `vendor/` directory on sys.path, or import this "
+            "repository's Core ML packages instead — they need no PyTorch."
+        )
+
+    weights = HERE / "sv0-forms.safetensors"
+    if not weights.exists():  # when the script is run outside the repository
+        weights = Path(hf_hub_download("precisit/one-pass-sv-forms", "sv0-forms.safetensors"))
+        hf_hub_download("precisit/one-pass-sv-forms", "sv0-forms.json", local_dir=weights.parent)
+    device = select_device("auto")
+    model, collator, _ = load_checkpoint(weights, device)
+    batch = collator([ChoiceExample(context=context_string(), options=tuple(OPTIONS), label=0)])
+    with torch.no_grad():
+        scores = model({key: value.to(device) for key, value in batch.items()})[0].softmax(-1)
+    best = int(scores.argmax())
+    print(f"chosen: {OPTIONS[best]!r}  (probability {float(scores[best]):.3f})")
+
+
+def metadata() -> None:
+    print(json.dumps(json.loads((HERE / "sv0-forms.json").read_text(encoding="utf-8"))["metadata"], indent=2))
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--torch", action="store_true", help="use the PyTorch checkpoint instead of Core ML")
+    parser.add_argument("--metadata", action="store_true", help="print the training metadata and exit")
+    arguments = parser.parse_args()
+    if arguments.metadata:
+        metadata()
+    elif arguments.torch:
+        via_torch()
+    else:
+        via_coreml()

sv0-forms.json

@@ -0,0 +1,162 @@
+{
+  "config": {
+    "context_tokens": 224,
+    "encoder": "tinyx",
+    "heads": 4,
+    "layers": 2,
+    "option_tokens": 96,
+    "rank": 128,
+    "width": 128
+  },
+  "format": "cua-s1",
+  "format_version": 1,
+  "metadata": {
+    "batch_size": 64,
+    "best_validation": {
+      "ece": 0.018766840075841174,
+      "examples": 2855,
+      "nll": 0.51966637646463,
+      "per_action": {
+        "check": {
+          "acc": 1.0,
+          "n": 87
+        },
+        "click": {
+          "acc": 0.9479166666666666,
+          "n": 96
+        },
+        "fill": {
+          "acc": 0.7756680731364276,
+          "n": 1422
+        },
+        "skip": {
+          "acc": 0.8712,
+          "n": 1250
+        }
+      },
+      "top1": 0.8301225900650024
+    },
+    "deterministic": false,
+    "epochs": 4,
+    "history": [
+      {
+        "epoch": 1,
+        "train_nll": 2.060510016020325,
+        "validation": {
+          "ece": 0.05491860159418138,
+          "examples": 2855,
+          "nll": 1.748936438351713,
+          "per_action": {
+            "check": {
+              "acc": 0.4367816091954023,
+              "n": 87
+            },
+            "click": {
+              "acc": 0.3958333333333333,
+              "n": 96
+            },
+            "fill": {
+              "acc": 0.06118143459915612,
+              "n": 1422
+            },
+            "skip": {
+              "acc": 0.9712,
+              "n": 1250
+            }
+          },
+          "top1": 0.4823117256164551
+        }
+      },
+      {
+        "epoch": 2,
+        "train_nll": 1.3498624871227647,
+        "validation": {
+          "ece": 0.029052662350295577,
+          "examples": 2855,
+          "nll": 1.1316383522153945,
+          "per_action": {
+            "check": {
+              "acc": 0.9770114942528736,
+              "n": 87
+            },
+            "click": {
+              "acc": 0.8854166666666666,
+              "n": 96
+            },
+            "fill": {
+              "acc": 0.4212376933895921,
+              "n": 1422
+            },
+            "skip": {
+              "acc": 0.8232,
+              "n": 1250
+            }
+          },
+          "top1": 0.6297723054885864
+        }
+      },
+      {
+        "epoch": 3,
+        "train_nll": 0.7656327178699136,
+        "validation": {
+          "ece": 0.02394736194401048,
+          "examples": 2855,
+          "nll": 0.6415376866970296,
+          "per_action": {
+            "check": {
+              "acc": 0.9770114942528736,
+              "n": 87
+            },
+            "click": {
+              "acc": 0.9791666666666666,
+              "n": 96
+            },
+            "fill": {
+              "acc": 0.7088607594936709,
+              "n": 1422
+            },
+            "skip": {
+              "acc": 0.8464,
+              "n": 1250
+            }
+          },
+          "top1": 0.7863397598266602
+        }
+      },
+      {
+        "epoch": 4,
+        "train_nll": 0.4496109453594249,
+        "validation": {
+          "ece": 0.018766840075841174,
+          "examples": 2855,
+          "nll": 0.51966637646463,
+          "per_action": {
+            "check": {
+              "acc": 1.0,
+              "n": 87
+            },
+            "click": {
+              "acc": 0.9479166666666666,
+              "n": 96
+            },
+            "fill": {
+              "acc": 0.7756680731364276,
+              "n": 1422
+            },
+            "skip": {
+              "acc": 0.8712,
+              "n": 1250
+            }
+          },
+          "top1": 0.8301225900650024
+        }
+      }
+    ],
+    "learning_rate": 0.002,
+    "seed": 7,
+    "task": "form-specialist-choice-classification",
+    "warmup": 0.05,
+    "workers": 0
+  },
+  "state_signature": "86e836863df8a0b0caa34ac5eab18bcd4fb44e420f83e271c96379fac0b0aec4"
+}

sv0-forms.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9030d7a7b7abfaf70dd1cc2a390568eaa3a55849daa0757b60d01dbab162356
+size 2828784