v2: corrected generator, 10 000-episode checkpoint, changelog, Core ML packages

README.md

@@ -14,79 +14,110 @@ tags:
   - coreml
   - apple-silicon
 base_model: []
+datasets:
+  - precisit/one-pass-sv-forms-synthetic
 ---
 
 # One-Pass SV-Forms (SV0) — a Swedish one-pass form specialist (research checkpoint)
 
 **This is a research checkpoint, not a product.** It is a 706 048-parameter model that scores a
-supplied list of options in **one forward pass** — the "Jev"/System One shape — trained on
-**synthetic** Swedish forms by a synthetic-data recipe we are publishing alongside it. It
-generates no text, needs no tokenizer, and runs in ~1 ms per decision on an Apple laptop or
-phone (Core ML package: 1.4 MB fp16, 787 KB int8).
+supplied list of options in **one forward pass** — the "Jev"/System One shape — trained from
+scratch on **synthetic** Swedish forms. It generates no text, needs no tokenizer, and answers in
+about a millisecond on an Apple device (Core ML: 1.4 MB fp16, 787 KB int8).
 
 | | |
 | --- | --- |
-| Task | given a UI element (role, label, state) plus the entities extracted from a document, pick one option: `fyll <entity>` / `kryssa` / `klicka` / `hoppa över` |
+| Task | given a UI element (role, label, state) plus the entities extracted from a document, pick one option: `fyll <entitet>` / `kryssa` / `klicka` / `hoppa över` |
 | Architecture | byte-level embeddings + 2-layer Transformer encoder (width 128, 4 heads) + option-attention head; `tinyx` |
-| Parameters | 706 048 (2.83 MB fp16 checkpoint) |
+| Parameters | 706 048 |
 | Context / option budget | 224 bytes of context, 96 bytes per option, **up to 40 options** |
-| Training data | 900 synthetic Swedish form episodes (21 306 decisions), generated locally; **no real form, person or customer data** |
+| Training data | 10 000 synthetic Swedish form episodes (234 921 decisions); **no real form, person or customer data** |
 | Licence | MIT (weights and code) |
-| Lineage | independent implementation of the same contract as [Cua's CUA-S1](https://huggingface.co/cua-ai/cua-s1-forms) and [jevlike](https://github.com/vinnylarouge/jevlike); trained from scratch, not a fine-tune of either |
+| Lineage | independent implementation of the same interface as [Cua's CUA-S1](https://huggingface.co/cua-ai/cua-s1-forms) and [jevlike](https://github.com/vinnylarouge/jevlike); trained from scratch, not a fine-tune of either |
 
 ## Measured results (our harness, Apple M4)
 
 | Run | decisions | top-1 | majority-class baseline | ECE | silently skipped a required fill |
 | --- | ---: | ---: | ---: | ---: | ---: |
-| Held-out synthetic test (form-signature disjoint) | 2 315 | **83.02 %** | 50.45 % | 0.017 | **76 (7.5 % of fills)** |
-| Hand-written out-of-distribution Swedish demo | 50 | **86.00 %** | 64.00 % | 0.092 | 3 (of 32 fills) |
-| Shuffled-context control (test) | 2 315 | 34.08 % | 50.45 % | 0.497 | 264 |
+| Held-out synthetic test (form-signature disjoint) | 2 310 | **99.29 %** | 47.36 % | 0.0014 | **1** |
+| Hand-written out-of-distribution demo | 50 | **100.00 %** | 64.00 % | — | 0 |
+| Shuffled-context control (same test rows) | 2 310 | 31.99 % | 47.36 % | — | — |
+
+The shuffled-context control is the number to look at first: rotating contexts between rows drops
+the model to 31.99 %, well below the majority baseline, so it is reading the element label and the
+document rather than exploiting option statistics. Without that control a 100 % demo set means
+nothing.
+
+For comparison, the released English checkpoint of the same family scores **21.4 %** on these
+Swedish rows — below the majority baseline, at its own shuffled-control floor. The language was
+the barrier, not the interface.
+
+### Data-size curve (same 4-epoch schedule, same seed, corrected generator)
+
+| Episodes | Train decisions | Test top-1 | Silent skips | ECE | Training time (M4) |
+| ---: | ---: | ---: | ---: | ---: | ---: |
+| 900 | 21 305 | 77.10 % | 127 | 0.0216 | 32 min |
+| 4 000 | 94 405 | 98.94 % | 6 | 0.0014 | 56 min |
+| **10 000** | **234 921** | **99.29 %** | **1** | 0.0014 | 163 min |
+
+This checkpoint is the 10 000-episode point. The curve is published because it is the useful
+part: at 900 episodes the same pipeline sits at 77 %, so a single small run would have been
+misread as a statement about the method rather than about the data.
+
+## Changelog
+
+- **v2 (2026-09-21)** — this release. Two fixes and a scale-up, all three visible above:
+  1. **Corpus correctness.** The Swedish catalogue's `personnummer` generator emitted nine digits
+     formatted `DDDDDDDD-DD` and `organisationsnummer` emitted `DDDDD-DDDDD`; both are now the
+     real Swedish shape `YYMMDD-XXXX`. As a side effect the models can no longer identify those
+     two concepts from the *shape of the value* alone — the bug had been an accidental
+     giveaway — so per-action accuracy on `fill` is the honest kind now.
+  2. **Scale.** 900 → 10 000 episodes, which moved top-1 from 77 % to 99.29 % and silent skips
+     from 127 to 1.
+  3. **Correction of our own first explanation.** An intermediate 900-episode run on the
+     corrected corpus scored 83.02 % (v1) versus 77.10 % (corrected), and we initially attributed
+     that six-point drop to the removed giveaway. The 4 000- and 10 000-episode points show the
+     dominant factor was data size, not corpus correctness. The measurement is recorded here
+     because we published the wrong explanation for part of a working day.
+- **v1 (2026-09-21)** — first release: 900 synthetic episodes, 83.02 % top-1, 7.5 % silent skips,
+  787 KB int8. Superseded; kept in the history of this repository rather than hidden.
 
-Per-action accuracy on the test split: `check` 98.3 %, `click` 93.7 %, `skip` 88.0 %,
-`fill` 75.5 %. The shuffled-context control is the interesting one: rotating contexts between
-rows drops the model below the majority baseline, i.e. it really reads the element label and the
-document rather than exploiting option statistics.
-
-For comparison, the released English checkpoint of the same family scores **21.4 %** on exactly
-these Swedish rows — below the majority baseline, at its own shuffled-control floor. The
-language was the barrier, not the contract.
+## What it is NOT
 
-Core ML export (same checkpoint, converted with coremltools 9.0):
+- **Not trained or tested on real Swedish forms.** The corpus is synthetic by construction:
+  fictional names, `.invalid` e-mail domains, generated personnummer-shaped identifiers that pass
+  their checksum but belong to nobody. The only non-synthetic evidence is a 50-decision set we
+  wrote ourselves. A small set built from two of our own shipped forms (Kanslist, Pratsam) is the
+  next measurement, not a claim in this card.
+- **Not an agent.** It does not generate text, cannot invent a value it was not given, and does
+  not decide execution order — the option list and the sequence come from ordinary code around it.
+- **Not safe unsupervised.** One required fill out of 29 839 was still answered "skip": a silent
+  failure. Any real integration must check outcomes outside the model (fail closed, dry run, one
+  submit, human review before consequential actions).
+- **Not tuned for throughput.** One decision per `predict` call is what the latency numbers
+  describe; batching is untested.
+
+## Core ML export (same checkpoint, coremltools 9.0, Apple M4)
 
 | Variant | package | top-1 | argmax parity vs PyTorch | median latency | p95 |
 | --- | ---: | ---: | ---: | ---: | ---: |
-| fp16 (CPU + ANE) | 1.4 MB | 83.00 % | 0.99870 | 1.31 ms | 1.46 ms |
-| **int8 (CPU + ANE)** | **787 KB** | **83.09 %** | 0.99611 | 1.32 ms | 1.40 ms |
-| int4 (CPU + ANE) | 481 KB | 50.87 % | 0.48702 | 1.97 ms | 2.10 ms |
-| PyTorch reference | 2.83 MB | 83.02 % | — | — | — |
-
-We publish the int4 result because it is the most useful thing in this table: the same 4-bit
-palettisation costs 0.06 pp on a converged model and destroys this one, and it also fails ANE
-compilation. **Quantisation headroom is a property of the training run, not of the architecture.**
-Use int8. `int4-weights/` ships the collapsed variant for reproducibility only.
-
-## What it is NOT
-
-- **Not trained or tested on real Swedish forms.** The corpus is synthetic (`Label: value`
-  document entities, `.invalid` e-mail domains, fictional names, locally generated
-  personnummer-shaped strings with no link to any real person) and the only non-synthetic
-  evidence is a 50-decision set we wrote ourselves.
-- **Not a general-purpose assistant or an autonomous agent.** It does not generate text, cannot
-  invent a value it was not given, and does not decide execution order.
-- **Not finished.** Validation was still improving when the run stopped (48 % → 63 % → 79 % →
-  83 % top-1 over four epochs), so treat 83 % as a floor for this recipe, not a ceiling.
-- **Not safe to run unsupervised on real data.** 7.5 % of required fills are answered "skip" —
-  a required field that silently stays empty. Any real integration must verify outcomes outside
-  the model (fail-closed execution, dry run, one submit, human review before consequential
-  actions) — see the runtime contract in Cua's `planner.py` for a good pattern.
-
-## Intended use
-
-Research on bounded, high-volume Swedish interface workflows where the option set is supplied by
-deterministic code: filling forms from an extracted document, triaging/routing where the choices
-are known in advance, or as a **criteria-decision** layer where a calibrated distribution per
-question is more useful than generated prose. Also as a worked example of how to build and
-measure a language-local one-pass specialist.
+| fp16, CPU + ANE | 1.44 MB | 99.29 % | 0.99950 (15 / 29 839) | 1.314 ms | 1.412 ms |
+| fp16, CPU only | 1.44 MB | 99.28 % | — | 1.727 ms | 1.929 ms |
+| **int8, CPU + ANE** | **787 KB** | **99.29 %** | 0.99956 (13 / 29 839) | 1.314 ms | 1.411 ms |
+| int4, CPU + ANE | 481 KB | **49.92 %** | 0.49675 (15 015 / 29 839) | 1.984 ms | 2.115 ms |
+
+**int8 is the variant to use; int4 does not work for this checkpoint** and the collapsed variant is
+published as evidence rather than omitted. Two honest notes about it:
+
+- On the *converged English* checkpoint of this family, the same 4-bit palettisation cost 14
+  decisions in 24 367. On both of ours it destroys the model (83 % → 51 % on v1, 99.29 % → 49.92 %
+  here) and the 4-bit graph does not compile for the Neural Engine (`ANECCompile() FAILED`).
+  Whatever the difference is, it is not training level: we first assumed quantisation headroom
+  tracked convergence, and the well-trained checkpoint falsifies that. We do not know the exact
+  quantisation recipe behind the English release, so we report the contradiction instead of
+  explaining it away.
+- fp16 parity is 99.95 %, not 100 %, on 29 839 rows. Measure parity per checkpoint; do not inherit
+  someone else's 1.000000.
 
 ## Usage
 
@@ -94,27 +125,32 @@ measure a language-local one-pass specialist.
 from pathlib import Path
 from huggingface_hub import hf_hub_download
 
-# the checkpoint format is <name>.safetensors + <name>.json (architecture + hashes)
 weights = Path(hf_hub_download("precisit/one-pass-sv-forms", "sv0-forms.safetensors"))
 hf_hub_download("precisit/one-pass-sv-forms", "sv0-forms.json", local_dir=weights.parent)
-# then either use the Core ML packages (no Python runtime needed) or the PyTorch code in
-# the toolkit repository: https://github.com/precisit/one-pass-specialists — the Core ML
-# packages below live in *this* repository and need no PyTorch.
+# then either use the Core ML packages (no Python runtime needed, see example.py) or the PyTorch
+# code and the full recipe in the toolkit repository:
+# https://github.com/precisit/one-pass-specialists
 ```
 
 The context string and options must be built exactly as in training (byte ids = UTF-8 byte + 1,
-zero-padded; `UPPGIFT fyll i formuläret från dokumentet och skicka sedan in` / `FORM <title>` /
-`ELEMENT <role> "<label>" value="…"`). A mismatch there is the most likely cause of poor output —
-the recipe and the generator are in the toolkit repository.
+zero padded; `UPPGIFT fyll i formuläret från dokumentet och skicka sedan in` / `FORM <titel>` /
+`ELEMENT <roll> "<etikett>" value="…"`). A mismatch there is the most likely cause of poor output —
+the generator, the catalogue and the scoring harness are in the toolkit repository.
 
 ## Building your own
 
-The toolkit that produced this checkpoint is public: [`precisit/one-pass-specialists`](https://github.com/precisit/one-pass-specialists). The interesting work is the **catalogue** — the concepts, the labels they appear under, and their value formats — because everything above the trainer is language- and vertical-neutral. The recipe, the corpus manifest and both result files for this checkpoint are in that repository under `examples/sv-forms/`.
+[`precisit/one-pass-specialists`](https://github.com/precisit/one-pass-specialists) is the public
+toolkit that produced this checkpoint: catalogue, synthetic generator with form-disjoint splits,
+trainer, evaluation (including the shuffled-context control and the silent-skip count), Core ML
+export with a bit-identity proof before anything is written, and the SV0 recipe with all receipts.
+The synthetic corpus itself is published as
+[`precisit/one-pass-sv-forms-synthetic`](https://huggingface.co/datasets/precisit/one-pass-sv-forms-synthetic).
 
 ## Licence and attribution
 
-MIT. The architecture, training loop and evaluation metrics come from Cua's MIT-licensed
-`libs/cua-s1` (see `THIRD_PARTY_NOTICES.md`); the option-attention head design is credited there
-to `jevlike` (MIT). The Swedish catalogue, synthetic generator, training run, measurements and
-Core ML export are ours — **Precisit AB, 2026**. No TypeSafe AI code, weights or data was used;
-"Jev" is their product and this is an independent implementation of a similar interface.
+MIT. The architecture, training loop and evaluation metrics are vendored unmodified from Cua's
+MIT-licensed [`libs/cua-s1`](https://github.com/trycua/cua/tree/main/libs/cua-s1) (which credits
+`jevlike` for the option-attention head design). The Swedish catalogue, synthetic generator,
+training runs, measurements and Core ML tooling are Precisit's. **No TypeSafe AI code, weights,
+data or API output was used** — "Jev" and "System One" are their names for a similar interface, and
+this is an independent implementation. See `THIRD_PARTY_NOTICES.md`.

coreml/conversion.json

@@ -1,12 +1,12 @@
 {
-  "checkpoint_sha256": "b9030d7a7b7abfaf70dd1cc2a390568eaa3a55849daa0757b60d01dbab162356",
-  "created_utc": "2026-09-21T08:00:57Z",
+  "checkpoint_sha256": "4f35bfac96dd23c058dfc098986c3f4cff39b836a177fb60e026b5f06c2ba782",
+  "created_utc": "2026-09-21T18:30:58Z",
   "limits": {
     "context_bytes": 224,
     "max_options": 40,
     "option_bytes": 96
   },
-  "model": "sv0-forms (spike jev-sv0)",
+  "model": "one_pass_forms (one-pass specialist)",
   "model_config": {
     "context_tokens": 224,
     "encoder": "tinyx",
@@ -22,11 +22,17 @@
     "python": "3.12.14",
     "torch": "2.7.0"
   },
-  "trace_check": null,
+  "trace_check": {
+    "logit_tolerance": 0.0001,
+    "trace_argmax_agreement": 1.0,
+    "trace_max_abs_logit_difference": 2.0742416381835938e-05,
+    "trace_max_abs_probability_difference": 1.0132789611816406e-06,
+    "verify_rows": 512
+  },
   "variants": {
     "fp16": {
       "minimum_target": "iOS17/macOS14",
-      "path": "sv0_forms_fp16_options40.mlpackage",
+      "path": "one_pass_forms_fp16_options40.mlpackage",
       "precision": "float16"
     },
     "int4": {
@@ -34,14 +40,14 @@
       "group_size": 32,
       "minimum_target": "iOS18/macOS15",
       "mode": "KMEANS",
-      "path": "sv0_forms_int4_options40.mlpackage",
+      "path": "one_pass_forms_int4_options40.mlpackage",
       "precision": "int4_weights_float16_compute"
     },
     "int8": {
       "granularity": "per_tensor",
       "minimum_target": "iOS17/macOS14",
       "mode": "UNIFORM",
-      "path": "sv0_forms_int8_options40.mlpackage",
+      "path": "one_pass_forms_int8_options40.mlpackage",
       "precision": "int8_weights_float16_compute"
     }
   }

coreml/sv0_forms_fp16_options40.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bfe30d2473f7c9a0d804f7b3306f252fcad4a307fa9202d7bf218049be41401a
-size 58608
+oid sha256:495202889ec32f7c0b0423d8b7c621ee9ccb502e00b3c3118d3f4a08a168d956
+size 58601

coreml/sv0_forms_fp16_options40.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cf1b0db68a002fd003cf68278f5e8c049c9360b9b5b3754205692e20a63bb43c
+oid sha256:aabd5c5ee30d9cb706cae1a4b49df4b8059cbad6f44e8be504d54eec547d8064
 size 1448256

coreml/sv0_forms_fp16_options40.mlpackage/Manifest.json

@@ -1,18 +1,18 @@
 {
     "fileFormatVersion": "1.0.0",
     "itemInfoEntries": {
-        "9B448D09-24D9-4E7E-B194-43D3D4AEE91F": {
-            "author": "com.apple.CoreML",
-            "description": "CoreML Model Specification",
-            "name": "model.mlmodel",
-            "path": "com.apple.CoreML/model.mlmodel"
-        },
-        "D3C625CB-29B6-456F-8FB3-B1111990E4E6": {
+        "A03F5FFD-6BD4-420E-A7D8-A61BBF858DEF": {
             "author": "com.apple.CoreML",
             "description": "CoreML Model Weights",
             "name": "weights",
             "path": "com.apple.CoreML/weights"
+        },
+        "A5AD9673-5F84-42BF-89B0-E543BEEC1402": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
         }
     },
-    "rootModelIdentifier": "9B448D09-24D9-4E7E-B194-43D3D4AEE91F"
+    "rootModelIdentifier": "A5AD9673-5F84-42BF-89B0-E543BEEC1402"
 }

coreml/sv0_forms_int4_options40.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e145df71d2ee1a4566126487afd80cd6409aeda278d42494640f49dc17016168
-size 58303
+oid sha256:7739d3302c68e6284a6d50542f01fb4a72b665f66c5f27f41afaf7859b2356a0
+size 58296

coreml/sv0_forms_int4_options40.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:53d889e570414557693f59309731d573aa6ddee95746dc9b9735a63afa2ad9ec
+oid sha256:d5530285655800fb5a12ea7698c8dd424c2c108de0985f9ba76a5f5ddbc969ed
 size 433664

coreml/sv0_forms_int4_options40.mlpackage/Manifest.json

@@ -1,18 +1,18 @@
 {
     "fileFormatVersion": "1.0.0",
     "itemInfoEntries": {
-        "495DA951-9B89-4CE4-8656-67B3D2E0038C": {
-            "author": "com.apple.CoreML",
-            "description": "CoreML Model Specification",
-            "name": "model.mlmodel",
-            "path": "com.apple.CoreML/model.mlmodel"
-        },
-        "7B688A87-1DA1-4178-9B4F-2F6A9FD6C274": {
+        "87C06B3D-7FF4-41E3-AF3D-5E6344006172": {
             "author": "com.apple.CoreML",
             "description": "CoreML Model Weights",
             "name": "weights",
             "path": "com.apple.CoreML/weights"
+        },
+        "A57B64A4-F072-4A85-9FC2-C7734A8FF115": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
         }
     },
-    "rootModelIdentifier": "495DA951-9B89-4CE4-8656-67B3D2E0038C"
+    "rootModelIdentifier": "A57B64A4-F072-4A85-9FC2-C7734A8FF115"
 }

coreml/sv0_forms_int8_options40.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4c63811d8b278b206c03cfc40c7d311a1e91997e9f710d524a1372477a28be74
-size 62987
+oid sha256:4090812ee4717e31f4cb6b40d74c68cf9c9a9f3b27530f62cda5e527fdf849e8
+size 62980

coreml/sv0_forms_int8_options40.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:36c31c5e396138a45eb191fec6f637d3831ffaca6005ebed12b305979be6b732
+oid sha256:ce77dcb8f8a241808fb0c112d923ba9e3524feb0334c3e610e59b8bcf3ea2981
 size 742528

coreml/sv0_forms_int8_options40.mlpackage/Manifest.json

@@ -1,18 +1,18 @@
 {
     "fileFormatVersion": "1.0.0",
     "itemInfoEntries": {
-        "BAE712EF-2551-49DE-BA14-3D8AB687D268": {
+        "B757B5C4-7ED5-408A-BF9D-AC936B5068B1": {
             "author": "com.apple.CoreML",
             "description": "CoreML Model Weights",
             "name": "weights",
             "path": "com.apple.CoreML/weights"
         },
-        "E219151B-08B6-4AFD-BFEA-72D3F71825A6": {
+        "E7FEE6B0-8F77-4EDD-81C4-21BCC5EB9583": {
             "author": "com.apple.CoreML",
             "description": "CoreML Model Specification",
             "name": "model.mlmodel",
             "path": "com.apple.CoreML/model.mlmodel"
         }
     },
-    "rootModelIdentifier": "E219151B-08B6-4AFD-BFEA-72D3F71825A6"
+    "rootModelIdentifier": "E7FEE6B0-8F77-4EDD-81C4-21BCC5EB9583"
 }

eval/RESULTS-SV0-COREML.md

@@ -1,94 +0,0 @@
-# RESULTS-SV0-COREML — the Swedish SV0 specialist exported to Core ML
-
-**Verdict: PARTIAL — fp16 and int8 export cleanly and run on the Neural Engine at the same
-accuracy as PyTorch; int4 is _not_ usable for this checkpoint (83 % → 51 % top-1) and does not
-compile for the ANE.** Recommended deployment artifact: **int8, 787 KB, ~1.3 ms per decision.**
-
-## What was exported
-
-`export_sv_coreml.py` in this directory converts `runs/sv-tinyx/model` (the SV0 checkpoint,
-706 048 parameters) with coremltools 9.0, mirroring the fixed-shape contract of the
-FluidInference CUA-S1 release: `context_ids (1, 224)` / `option_ids (1, 40, 96)` /
-`option_mask (1, 40)`, all int32 byte-ids (byte + 1, 0 = pad), output `logits (1, 40)`.
-`max_options` is **40** here rather than their 32 because our corpus tops out at 37 options —
-the ceiling is baked into the export, so it must be sized from the widest real input.
-
-Variants: fp16 (iOS17), int8 (UNIFORM, per-tensor, iOS17), int4 (KMEANS, per-grouped-channel,
-group 32, iOS18). `conversion.json` records hashes, config and tooling versions.
-
-## Export-time checks (all passed before any measurement)
-
-| Check | Result |
-| --- | --- |
-| Export forward vs vendored model, 256 real rows, max abs logit difference | **0.0** (bit-identical) |
-| Traced graph vs model, 64 real rows | max abs Δ 7.2 × 10⁻⁶, argmax agreement **1.0** |
-
-## Measurements (Apple M4, macOS 26.6.2; 2 312 scored rows after 3 warm-up rows; same grader as the spike)
-
-| Variant | package | test top-1 | demo top-1 | silent skips (of 1 008 fills) | argmax parity vs PyTorch | mismatches | median | p95 |
-| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
-| PyTorch (MPS), reference | 2 828 784 B | **83.02 %** | 86 % | 76 | — | — | — | — |
-| **fp16, CPU + ANE** | 1 507 481 B (1.44 MB) | 83.00 % | 85.11 % | 76 | 0.99870 | 3 / 2 312 | 1.307 ms | 1.460 ms |
-| fp16, CPU only | 1 507 481 B | 83.04 % | — | 76 | 0.99913 | 2 / 2 312 | 1.734 ms | 1.917 ms |
-| **int8, CPU + ANE** | **806 132 B (787 KB)** | **83.09 %** | 85.11 % | 76 | 0.99611 | 9 / 2 312 | 1.316 ms | 1.402 ms |
-| int4, CPU + ANE | 492 584 B (481 KB) | **50.87 %** | 27.66 % | 954 | 0.48702 | 1 186 / 2 312 | 1.968 ms | 2.099 ms |
-
-Fills per-action on fp16: `fill` 75.40 %, `skip` 88.26 %, `check` 96.55 %, `click` 93.67 % —
-within a few rows of the PyTorch run, i.e. the export is behaviour-preserving.
-
-## What the numbers say
-
-1. **The export works, and the ANE helps here too**: fp16 median 1.307 ms with the ANE vs
-   1.734 ms CPU-only, p95 1.460 ms vs 1.917 ms. Smaller absolute win than on the English model
-   (whose weights are better conditioned), but the same direction.
-2. **int8 is free and is the artifact to ship**: 787 KB (52 % of fp16), top-1 83.09 % — one
-   decision *better* than PyTorch on this split — 9 argmax deviations out of 2 312, identical
-   silent-skip count, and the same ANE behaviour.
-3. **int4 does not transfer to an undertrained model.** On Cua's English checkpoint the same
-   kind of palettisation cost 14 decisions in 24 367 (0.06 pp); here it costs 1 186 in 2 312
-   (51 pp) and raises silent skips from 76 to 954. Two mechanisms are plausible and both point
-   the same way: a model that has not converged stores information in fine weight differences
-   that a 4-bit palette cannot represent, and the per-grouped-channel kmeans graph also fails
-   to compile for the ANE (`ANECCompile() FAILED`, seen repeatedly during save and load), which
-   is why its latency is worst despite being the smallest artifact. **Quantisation headroom is
-   a property of the training run, not of the architecture** — that is the transferable finding.
-4. **fp16 parity is 99.87 %, not 100 %**, and that is expected rather than alarming: three
-   rows change their argmax under fp16 rounding on a model whose decision margins are thin
-   (83 % top-1). On the well-trained English model the same conversion showed zero deviations.
-   Parity must therefore be *measured* per checkpoint, not assumed — exactly what this harness
-   does.
-
-## Pitfalls found while exporting (all cost a failed run)
-
-- **`torch.jit.trace` under `torch.no_grad()` takes PyTorch's fused sparsity fast path**
-  (`torch._transformer_encoder_layer_fwd`), which coremltools cannot convert. Trace with grad
-  *enabled*.
-- **The coremltools torch frontend has no `__or__` for bool tensors** → use
-  `torch.logical_or` / `torch.logical_not`. (`&`, `~` alone, and in-place `mask[:, 0] = True`
-  all fail; probe the op support in a 5-line script before rewriting a model.)
-- **`clamp_min(1)` with a Python int trips `assert x.dtype == y.dtype`** in the frontend; pass a
-  same-dtype tensor constant.
-- **`per_grouped_channel` palettisation requires an iOS18 deployment target**, so the int4
-  variant needs its own fp16 base converted at iOS18 (the upstream release does the same).
-- **k-means palettisation needs `scikit-learn`**, which is not a coremltools dependency; the
-  error arrives mid-run, after the fp16 conversion has already succeeded — hence the
-  `REUSE_FP16=1` cache flag in the export script.
-- **Palettizer API in coremltools 9.0**: wrap the op config —
-  `OptimizationConfig(global_config=OpPalettizerConfig(...))` — and note the mode names are
-  upper-case (`UNIFORM`, `KMEANS`) while granularity is `per_tensor` / `per_grouped_channel`
-  only.
-- **Check that the traced graph is the model** before converting: `torch.jit.trace(..., check_trace=False)`
-  suppresses a noisy replay check, so the export script compares logits itself and aborts on
-  mismatch.
-
-## Reproduction
-
-```bash
-# (in the spike venv with coremltools 9.0, torch 2.7, safetensors, scikit-learn)
-python check_export_forward.py     # export forward is bit-identical to the model
-REUSE_FP16=1 python export_sv_coreml.py    # fp16 + int8 + int4 mlpackages, conversion.json
-python eval_sv_coreml.py           # accuracy, silent skips, parity, latency -> RESULTS-sv0-coreml.json
-```
-
-Artifacts (not in Git): `sv-coreml/*.mlpackage`, `RESULTS-sv0-coreml.json`,
-`export.log`, `eval-coreml.log`.

eval/RESULTS-SV0-pytorch.md

@@ -1,94 +0,0 @@
-# RESULTS-SV0 — Swedish Jev-shaped form specialist
-
-**Verdict: VALIDATED, with documented limits.**
-The Cua-S1 contract trains on our own Swedish synthetic data with the upstream MIT code,
-on one laptop, and reaches **83.02 %** top-1 on a held-out, form-disjoint synthetic test
-(2 315 decisions) and **86 %** on a hand-written out-of-distribution Swedish demo (50
-decisions) — against **21.4 %** / **20 %** for the released English `cua-s1-forms`
-checkpoint on the very same rows. The result is undertrained (validation was still
-improving at the last epoch), so 83 % is a floor, not a ceiling.
-
-## Protocol as run
-
-| Item | Setting |
-| --- | --- |
-| Architecture | `tinyx` (byte-level 2-layer Transformer encoder, width 128, 4 heads) + jevlike `AttentionHead`, context 224 / option 96 bytes — identical to upstream |
-| Code | `trycua/cua` → `libs/cua-s1` @ `9bbfa7d`, vendored unmodified (`vendor/`, MIT) |
-| Corpus | `sv_synth.py` + `concepts_sv.py` (ours): 900 episodes, seed 2026, splits by form field signature |
-| Rows | train 21 306 / validation 2 855 / test 2 315 decisions; 725 / 96 / 79 distinct form signatures, **zero overlap** between splits |
-| Training | AdamW lr 2e-3, cosine + 5 % warmup, wd 1e-2, 4 epochs, batch 64, seed 7, MPS, 1 927 s |
-| Checkpoint rule | best validation NLL (upstream's rule) — epoch 4 |
-| Grader | `eval_sv.py`, unchanged for both checkpoints |
-
-## Numbers
-
-| Run | decisions | top-1 | majority baseline | ECE | skipped a required fill | per action |
-| --- | ---: | ---: | ---: | ---: | ---: | --- |
-| Swedish specialist — synthetic test (form-disjoint) | 2 315 | **83.02 %** | 50.45 % | 0.01656 | 76 | `check` 98.28 % (n=58), `click` 93.67 % (n=79), `fill` 75.55 % (n=1010), `skip` 88.01 % (n=1168) |
-| Swedish specialist — handwritten OOD demo (50) | 50 | **86.00 %** | 64.00 % | 0.09176 | 3 | `check` 100 % (n=3), `click` 100 % (n=3), `fill` 81.25 % (n=32), `skip` 91.67 % (n=12) |
-| Swedish specialist — shuffled-context control (test) | 2 315 | 34.08 % | 50.45 % | 0.49735 | 264 | `check` 15.52 %, `click` 0 %, `fill` 1.29 %, `skip` 65.67 % |
-| Swedish specialist — shuffled-context control (demo) | 50 | 6.00 % | 64.00 % | 0.78579 | 6 | all actions ≤ 25 % |
-| English reference (`cua-s1-forms`) — synthetic test | 2 315 | 21.38 % | 50.45 % | 0.64072 | 371 | `check` 93.10 %, `click` 26.58 %, `fill` 5.54 %, `skip` 31.16 % |
-| English reference — handwritten OOD demo | 50 | 20.00 % | 64.00 % | 0.66098 | 16 | `check` 66.67 %, `click` 33.33 %, `fill` 15.62 %, `skip` 16.67 % |
-| English reference — shuffled-context control (test) | 2 315 | 17.50 % | 50.45 % | 0.67954 | 363 | `check` 60.35 %, `click` 6.33 %, `fill` 1.78 %, `skip` 29.71 % |
-
-Per-epoch validation: 48.23 % → 62.98 % → 78.63 % → **83.01 %** top-1 (train NLL
-2.06 → 0.45; val NLL 1.75 → 0.52). Still improving when the run stopped.
-
-## Reading the numbers
-
-1. **The contract transfers; the language boundary was the real barrier.** The English
-   checkpoint on Swedish rows sits at **21.4 %**, barely above its own shuffled-context
-   control (17.5 %) and *below* the majority-action baseline (50.5 %), with ECE 0.64 — it
-   is not reading the content, it is guessing confidently. On the hand-written demo it
-   scores 20 % against a 64 % baseline, i.e. worse than always answering "fill". The
-   Swedish specialist, same architecture, same grader, same rows: 83 % / 86 %.
-2. **The shuffled control is the floor that matters.** Our specialist drops from 83.02 %
-   to 34.08 % when contexts are rotated between rows (below the 50.45 % majority
-   baseline), which is the evidence that it uses the element label and the document, not
-   option statistics. The English reference barely moves (21.4 % → 17.5 %) — its score was
-   near the floor to begin with.
-3. **`fill` is the weak action, and the silent-skip failure mode persists.** 75.5 % of
-   fills are right; 76 of 1 010 fill decisions (7.5 %) are answered `hoppa över` instead.
-   In a real workflow that is the dangerous class: a required field that looks done and is
-   not. It is much smaller than the English reference's (371/1 010), but it is not zero,
-   and it is the reason a production integration needs a per-item check outside the model.
-4. **Calibration is good in-distribution and degrades out-of-distribution** (ECE 0.017 →
-   0.092), which is exactly the wrong direction for confidence routing if the threshold is
-   tuned on synthetic data.
-5. **It is undertrained.** Both the validation curve and the comparison with the reference
-   (trained on ~150 k rows vs our 21 k) say the remaining error is largely a data-scale
-   effect. The next honest step is more episodes, not a different architecture.
-
-## What this does not establish
-
-- Nothing about real Swedish forms. The demo set is hand-written by the same person who
-  wrote the catalogue, so it shares the author's vocabulary; a genuine test needs real
-  blanketter (Skolverket/Försäkringskassan/kommunala) or our own live forms.
-- Nothing about the runtime: this spike scores decisions, it does not drive a GUI.
-  Fill/click execution, retries and fail-closed behaviour were not exercised (upstream's
-  `planner.py` refuses to execute without a compatible driver contract).
-- Nothing about English transfer in the other direction: we never trained a Swedish model
-  on English rows, and the corpus changes *language and label vocabulary at once*, so the
-  ablation "language vs vocabulary" is still open.
-- Reproducibility is bounded by the environment: torch 2.14 on Apple Silicon MPS, seed 7,
-  `deterministic=False` (deterministic algorithms were too slow on MPS), so the exact
-  numbers are one draw, not a bit-reproducible receipt.
-
-## Follow-ups worth pre-registering
-
-1. **Scale test**: 4 000–10 000 episodes, 6 epochs, batch 32–64 on a machine with room —
-   does top-1 cross ~95 % and does the silent-skip rate fall below 2 % of fills?
-2. **Language ✕ vocabulary ablation**: Swedish documents + English labels, and the mirror,
-   to attribute the win.
-3. **Real-form OOD**: hand-build 3–5 authentic Swedish blanketter (never in the catalogue)
-   and score before touching the corpus again.
-4. **Abstention outside the model**: a spread/top-probability gate plus a "required field
-   still empty" check, measured on the silent-skip class specifically.
-
-## Reproduction
-
-See `README.md` in this directory. Runtime: ~5 min generation, 32 min training, seconds of
-scoring on one Apple Silicon laptop. Corpus and checkpoint are not in Git
-(`~/agent-data/jevlike-sv-spike/`); `data/sv/manifest.json` carries the SHA-256 of each split
-(test `92c0dfec…d272`).

eval/RESULTS-SV0-v2.md

@@ -0,0 +1,57 @@
+# RESULTS-SV0 v2 — the released Swedish one-pass specialist
+
+Raw artefacts: `results-pytorch.json`, `results-coreml.json`, `size-sweep.jsonl`,
+`corpus-manifest-10000-episodes.json`. Model card:
+[`precisit/one-pass-sv-forms`](https://huggingface.co/precisit/one-pass-sv-forms).
+
+## The checkpoint
+
+- 706 048 parameters (`tinyx`), 2.83 MB fp16, trained from scratch.
+- Corpus: 10 000 synthetic episodes, seed 2026, split by form signature → 234 921 train /
+  29 649 validation / 29 839 test decisions.
+- Schedule: 4 epochs, batch 64, learning rate 2e-3, seed 7; 163 minutes on an Apple M4 (24 GB).
+
+## Held-out results (the corpus's own test split, and the hand-written set)
+
+| Run | decisions | top-1 | majority-class baseline | ECE | silent skips |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| Synthetic test (form-disjoint) | 29 839 | 99.29 % | 47.36 % | 0.0014 | 1 |
+| Hand-written out-of-distribution demo | 50 | 100.00 % | 64.00 % | — | 0 |
+| Shuffled-context control | 29 839 | 31.99 % | 47.36 % | — | — |
+| The *earlier* checkpoint on Swedish rows: released English model of the same family | 2 310 | 21.4 % | 48.87 % | — | 371 of 1 044 fills |
+
+The shuffled control (31.99 %, below the 47.36 % majority baseline) is the load-bearing control:
+it shows the score comes from reading the element and the document rather than from the option
+statistics. Silent skips fell from 127 (900 episodes) to 1.
+
+## Data-size curve (fixed 4-epoch schedule, fixed seed, corrected generator)
+
+| Episodes | Train decisions | Test decisions | top-1 | silent skips | ECE | training time |
+| ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| 900 | 21 305 | 2 310 | 77.10 % | 127 | 0.0216 | 32 min |
+| 4 000 | 94 405 | 11 556 | 98.94 % | 6 | 0.0014 | 56 min |
+| 10 000 | 234 921 | 29 839 | 99.29 % | 1 | 0.0014 | 163 min |
+
+Each point is scored on its own corpus's held-out split, so the test set grows with the corpus.
+
+## What changed from v1, and what we got wrong
+
+1. **A format bug in the generator, fixed.** `gen_personnummer` produced nine digits formatted
+   `DDDDDDDD-DD` and `gen_orgnummer` appended the check digit, producing eleven characters. Both
+   now emit the real Swedish shape (`YYMMDD-XXXX`, 6+4, Luhn-valid). Tests assert the shape and the
+   checksum.
+2. **An accidental giveaway, removed.** The buggy shapes happened to be unique among the 50
+   concepts, so those two concepts could be identified from the *shape of the value* alone. After
+   the fix both share `DDDDDD-DDDD` with each other only. Per-action `fill` accuracy is therefore
+   the honest kind now (`shape_analysis` in the working notes; the shape census is ten lines).
+3. **A wrong explanation, corrected.** The first 900-episode run on the corrected corpus scored
+   77.10 % against 83.02 % for the buggy-corpus run, and we attributed the six-point drop to the
+   removed giveaway. The 4 000- and 10 000-episode points show the dominant factor was data size.
+   The claim was wrong for part of a working day and is recorded here rather than quietly dropped.
+
+## Limits
+
+- Synthetic corpus only: form-disjoint held-out splits, not distribution-disjoint ones. No real
+  Swedish form has been through the model; the 50-decision set is the only non-generated evidence.
+- One silent skip remains: out-of-model verification is still mandatory.
+- Latency figures are single-decision calls; batching untested.

eval/corpus-manifest-10000-episodes.json

@@ -0,0 +1,44 @@
+{
+  "episodes": 10000,
+  "format": "cua-s1-choice-jsonl",
+  "format_version": 1,
+  "generator": {
+    "config": {
+      "filled_field_probability": 0.6,
+      "hard_negative_probability": 0.35,
+      "max_distractor_entities": 3,
+      "max_extra_entities": 5,
+      "missing_entity_probability": 0.12,
+      "partial_form_probability": 0.3,
+      "stale_value_probability": 0.05
+    },
+    "format": "spike.sv_synth",
+    "task_line": "UPPGIFT fyll i formul\u00e4ret fr\u00e5n dokumentet och skicka sedan in"
+  },
+  "ratios": {
+    "test": 0.1,
+    "train": 0.8,
+    "validation": 0.1
+  },
+  "seed": 2026,
+  "splits": {
+    "test": {
+      "episodes": 1009,
+      "file": "test.jsonl",
+      "rows": 29839,
+      "sha256": "105b7c0509af8667c11653f77d3866dc528e846a3f439e284dccb62164d2dc3f"
+    },
+    "train": {
+      "episodes": 7984,
+      "file": "train.jsonl",
+      "rows": 234921,
+      "sha256": "071edb46d1a5f03a4b85e4ca5adfc2a089b31b334c85c3446e0f75d378dae731"
+    },
+    "validation": {
+      "episodes": 1007,
+      "file": "validation.jsonl",
+      "rows": 29649,
+      "sha256": "e58cf782a94deb6532675bc0de202819c75f8f347912dfddf9337dd4a0992baa"
+    }
+  }
+}

eval/demo-handwritten.jsonl

@@ -0,0 +1,50 @@
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT Edit \"Förnamn\" value=\"\"","label":0,"meta":{"action":"fill","form":"Exempelkliniken - Ny patientregistrering","gold_option":"fyll Förnamn: Anna","role":"Edit","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT Edit \"Efternamn\" value=\"\"","label":1,"meta":{"action":"fill","form":"Exempelkliniken - Ny patientregistrering","gold_option":"fyll Efternamn: Lindqvist","role":"Edit","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT Edit \"Personnummer\" value=\"\" hint=\"ÅÅMMDD-XXXX\"","label":2,"meta":{"action":"fill","form":"Exempelkliniken - Ny patientregistrering","gold_option":"fyll Personnummer: 560314-2381","role":"Edit","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT Edit \"Mobilnummer\" value=\"\"","label":3,"meta":{"action":"fill","form":"Exempelkliniken - Ny patientregistrering","gold_option":"fyll Telefon: 070-341 22 87","role":"Edit","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT Edit \"E-post\" value=\"\"","label":4,"meta":{"action":"fill","form":"Exempelkliniken - Ny patientregistrering","gold_option":"fyll E-post: anna.lindqvist@exempel.invalid","role":"Edit","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT Edit \"Adress\" value=\"\"","label":5,"meta":{"action":"fill","form":"Exempelkliniken - Ny patientregistrering","gold_option":"fyll Gatuadress: Kungsgatan 42","role":"Edit","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT Edit \"Postnummer\" value=\"\"","label":6,"meta":{"action":"fill","form":"Exempelkliniken - Ny patientregistrering","gold_option":"fyll Postnummer: 753 20","role":"Edit","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT Edit \"Ort\" value=\"\"","label":7,"meta":{"action":"fill","form":"Exempelkliniken - Ny patientregistrering","gold_option":"fyll Ort: Uppsala","role":"Edit","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT Edit \"Vårdcentral\" value=\"\"","label":8,"meta":{"action":"fill","form":"Exempelkliniken - Ny patientregistrering","gold_option":"fyll Vårdcentral: Exempelvårdcentralen","role":"Edit","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT Edit \"Kända allergier\" value=\"\"","label":9,"meta":{"action":"fill","form":"Exempelkliniken - Ny patientregistrering","gold_option":"fyll Allergier: Penicillin","role":"Edit","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT Edit \"Anhörig\" value=\"\"","label":10,"meta":{"action":"fill","form":"Exempelkliniken - Ny patientregistrering","gold_option":"fyll Anhörig: Erik Lindqvist","role":"Edit","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT Edit \"Nödkontakt telefon\" value=\"\"","label":11,"meta":{"action":"fill","form":"Exempelkliniken - Ny patientregistrering","gold_option":"fyll Anhörigs telefon: 070-882 19 03","role":"Edit","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT Edit \"Ort\" value=\"Uppsala\"","label":17,"meta":{"action":"skip","form":"Exempelkliniken - Ny patientregistrering","gold_option":"hoppa över","role":"Edit","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT Edit \"Läkare\" value=\"\"","label":17,"meta":{"action":"skip","form":"Exempelkliniken - Ny patientregistrering","gold_option":"hoppa över","role":"Edit","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT CheckBox \"Jag samtycker till behandling av personuppgifter\" unchecked","label":15,"meta":{"action":"check","form":"Exempelkliniken - Ny patientregistrering","gold_option":"kryssa","role":"CheckBox","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT CheckBox \"Skicka nyhetsbrev\" unchecked","label":17,"meta":{"action":"skip","form":"Exempelkliniken - Ny patientregistrering","gold_option":"hoppa över","role":"CheckBox","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT Button \"Skicka in\" value=\"\"","label":16,"meta":{"action":"click","form":"Exempelkliniken - Ny patientregistrering","gold_option":"klicka","role":"Button","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT Button \"Avbryt\" value=\"\"","label":17,"meta":{"action":"skip","form":"Exempelkliniken - Ny patientregistrering","gold_option":"hoppa över","role":"Button","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Exempelkliniken - Ny patientregistrering\nELEMENT Edit \"Adressfält\" value=\"https://exempel.invalid/formular\"","label":17,"meta":{"action":"skip","form":"Exempelkliniken - Ny patientregistrering","gold_option":"hoppa över","role":"Edit","source":"hand-authored"},"options":["fyll Förnamn: Anna","fyll Efternamn: Lindqvist","fyll Personnummer: 560314-2381","fyll Telefon: 070-341 22 87","fyll E-post: anna.lindqvist@exempel.invalid","fyll Gatuadress: Kungsgatan 42","fyll Postnummer: 753 20","fyll Ort: Uppsala","fyll Vårdcentral: Exempelvårdcentralen","fyll Allergier: Penicillin","fyll Anhörig: Erik Lindqvist","fyll Anhörigs telefon: 070-882 19 03","fyll Kursnamn: Anatomi och fysiologi","fyll Bankgiro: 552-7731","fyll Fakturadatum: 2026-09-14","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Norrsken Teknik AB - Reseräkning\nELEMENT Edit \"Namn\" value=\"\"","label":0,"meta":{"action":"fill","form":"Norrsken Teknik AB - Reseräkning","gold_option":"fyll Namn: Johan Bergström","role":"Edit","source":"hand-authored"},"options":["fyll Namn: Johan Bergström","fyll Anställningsnummer: AN-338201","fyll Avdelning: IT och digitalisering","fyll Bankgiro: 589-2240","fyll Plusgiro: 38 24 19-6","fyll Belopp: 12 450,00 kr","fyll Momsregistreringsnummer: SE556677889901","fyll Utfärdad: 2026-09-11","fyll Förfallodatum: 2026-10-15","fyll Referens: 8836001928","fyll Organisationsnummer: 556677-8899","fyll Kundnummer: M-441023","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Norrsken Teknik AB - Reseräkning\nELEMENT Edit \"Anställningsnummer\" value=\"\"","label":1,"meta":{"action":"fill","form":"Norrsken Teknik AB - Reseräkning","gold_option":"fyll Anställningsnummer: AN-338201","role":"Edit","source":"hand-authored"},"options":["fyll Namn: Johan Bergström","fyll Anställningsnummer: AN-338201","fyll Avdelning: IT och digitalisering","fyll Bankgiro: 589-2240","fyll Plusgiro: 38 24 19-6","fyll Belopp: 12 450,00 kr","fyll Momsregistreringsnummer: SE556677889901","fyll Utfärdad: 2026-09-11","fyll Förfallodatum: 2026-10-15","fyll Referens: 8836001928","fyll Organisationsnummer: 556677-8899","fyll Kundnummer: M-441023","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Norrsken Teknik AB - Reseräkning\nELEMENT Edit \"Avdelning\" value=\"\"","label":2,"meta":{"action":"fill","form":"Norrsken Teknik AB - Reseräkning","gold_option":"fyll Avdelning: IT och digitalisering","role":"Edit","source":"hand-authored"},"options":["fyll Namn: Johan Bergström","fyll Anställningsnummer: AN-338201","fyll Avdelning: IT och digitalisering","fyll Bankgiro: 589-2240","fyll Plusgiro: 38 24 19-6","fyll Belopp: 12 450,00 kr","fyll Momsregistreringsnummer: SE556677889901","fyll Utfärdad: 2026-09-11","fyll Förfallodatum: 2026-10-15","fyll Referens: 8836001928","fyll Organisationsnummer: 556677-8899","fyll Kundnummer: M-441023","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Norrsken Teknik AB - Reseräkning\nELEMENT Edit \"Bankgironummer\" value=\"\" hint=\"XXX-XXXX\"","label":3,"meta":{"action":"fill","form":"Norrsken Teknik AB - Reseräkning","gold_option":"fyll Bankgiro: 589-2240","role":"Edit","source":"hand-authored"},"options":["fyll Namn: Johan Bergström","fyll Anställningsnummer: AN-338201","fyll Avdelning: IT och digitalisering","fyll Bankgiro: 589-2240","fyll Plusgiro: 38 24 19-6","fyll Belopp: 12 450,00 kr","fyll Momsregistreringsnummer: SE556677889901","fyll Utfärdad: 2026-09-11","fyll Förfallodatum: 2026-10-15","fyll Referens: 8836001928","fyll Organisationsnummer: 556677-8899","fyll Kundnummer: M-441023","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Norrsken Teknik AB - Reseräkning\nELEMENT Edit \"Plusgironummer\" value=\"\"","label":4,"meta":{"action":"fill","form":"Norrsken Teknik AB - Reseräkning","gold_option":"fyll Plusgiro: 38 24 19-6","role":"Edit","source":"hand-authored"},"options":["fyll Namn: Johan Bergström","fyll Anställningsnummer: AN-338201","fyll Avdelning: IT och digitalisering","fyll Bankgiro: 589-2240","fyll Plusgiro: 38 24 19-6","fyll Belopp: 12 450,00 kr","fyll Momsregistreringsnummer: SE556677889901","fyll Utfärdad: 2026-09-11","fyll Förfallodatum: 2026-10-15","fyll Referens: 8836001928","fyll Organisationsnummer: 556677-8899","fyll Kundnummer: M-441023","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Norrsken Teknik AB - Reseräkning\nELEMENT Edit \"Summa\" value=\"\"","label":5,"meta":{"action":"fill","form":"Norrsken Teknik AB - Reseräkning","gold_option":"fyll Belopp: 12 450,00 kr","role":"Edit","source":"hand-authored"},"options":["fyll Namn: Johan Bergström","fyll Anställningsnummer: AN-338201","fyll Avdelning: IT och digitalisering","fyll Bankgiro: 589-2240","fyll Plusgiro: 38 24 19-6","fyll Belopp: 12 450,00 kr","fyll Momsregistreringsnummer: SE556677889901","fyll Utfärdad: 2026-09-11","fyll Förfallodatum: 2026-10-15","fyll Referens: 8836001928","fyll Organisationsnummer: 556677-8899","fyll Kundnummer: M-441023","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Norrsken Teknik AB - Reseräkning\nELEMENT Edit \"Momsnr\" value=\"\"","label":6,"meta":{"action":"fill","form":"Norrsken Teknik AB - Reseräkning","gold_option":"fyll Momsregistreringsnummer: SE556677889901","role":"Edit","source":"hand-authored"},"options":["fyll Namn: Johan Bergström","fyll Anställningsnummer: AN-338201","fyll Avdelning: IT och digitalisering","fyll Bankgiro: 589-2240","fyll Plusgiro: 38 24 19-6","fyll Belopp: 12 450,00 kr","fyll Momsregistreringsnummer: SE556677889901","fyll Utfärdad: 2026-09-11","fyll Förfallodatum: 2026-10-15","fyll Referens: 8836001928","fyll Organisationsnummer: 556677-8899","fyll Kundnummer: M-441023","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Norrsken Teknik AB - Reseräkning\nELEMENT Edit \"Fakturadatum\" value=\"\"","label":7,"meta":{"action":"fill","form":"Norrsken Teknik AB - Reseräkning","gold_option":"fyll Utfärdad: 2026-09-11","role":"Edit","source":"hand-authored"},"options":["fyll Namn: Johan Bergström","fyll Anställningsnummer: AN-338201","fyll Avdelning: IT och digitalisering","fyll Bankgiro: 589-2240","fyll Plusgiro: 38 24 19-6","fyll Belopp: 12 450,00 kr","fyll Momsregistreringsnummer: SE556677889901","fyll Utfärdad: 2026-09-11","fyll Förfallodatum: 2026-10-15","fyll Referens: 8836001928","fyll Organisationsnummer: 556677-8899","fyll Kundnummer: M-441023","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Norrsken Teknik AB - Reseräkning\nELEMENT Edit \"Betalas senast\" value=\"\"","label":8,"meta":{"action":"fill","form":"Norrsken Teknik AB - Reseräkning","gold_option":"fyll Förfallodatum: 2026-10-15","role":"Edit","source":"hand-authored"},"options":["fyll Namn: Johan Bergström","fyll Anställningsnummer: AN-338201","fyll Avdelning: IT och digitalisering","fyll Bankgiro: 589-2240","fyll Plusgiro: 38 24 19-6","fyll Belopp: 12 450,00 kr","fyll Momsregistreringsnummer: SE556677889901","fyll Utfärdad: 2026-09-11","fyll Förfallodatum: 2026-10-15","fyll Referens: 8836001928","fyll Organisationsnummer: 556677-8899","fyll Kundnummer: M-441023","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Norrsken Teknik AB - Reseräkning\nELEMENT Edit \"Referensnummer\" value=\"\"","label":9,"meta":{"action":"fill","form":"Norrsken Teknik AB - Reseräkning","gold_option":"fyll Referens: 8836001928","role":"Edit","source":"hand-authored"},"options":["fyll Namn: Johan Bergström","fyll Anställningsnummer: AN-338201","fyll Avdelning: IT och digitalisering","fyll Bankgiro: 589-2240","fyll Plusgiro: 38 24 19-6","fyll Belopp: 12 450,00 kr","fyll Momsregistreringsnummer: SE556677889901","fyll Utfärdad: 2026-09-11","fyll Förfallodatum: 2026-10-15","fyll Referens: 8836001928","fyll Organisationsnummer: 556677-8899","fyll Kundnummer: M-441023","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Norrsken Teknik AB - Reseräkning\nELEMENT Edit \"Org.nr\" value=\"\"","label":10,"meta":{"action":"fill","form":"Norrsken Teknik AB - Reseräkning","gold_option":"fyll Organisationsnummer: 556677-8899","role":"Edit","source":"hand-authored"},"options":["fyll Namn: Johan Bergström","fyll Anställningsnummer: AN-338201","fyll Avdelning: IT och digitalisering","fyll Bankgiro: 589-2240","fyll Plusgiro: 38 24 19-6","fyll Belopp: 12 450,00 kr","fyll Momsregistreringsnummer: SE556677889901","fyll Utfärdad: 2026-09-11","fyll Förfallodatum: 2026-10-15","fyll Referens: 8836001928","fyll Organisationsnummer: 556677-8899","fyll Kundnummer: M-441023","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Norrsken Teknik AB - Reseräkning\nELEMENT Edit \"Kundnummer\" value=\"\"","label":11,"meta":{"action":"fill","form":"Norrsken Teknik AB - Reseräkning","gold_option":"fyll Kundnummer: M-441023","role":"Edit","source":"hand-authored"},"options":["fyll Namn: Johan Bergström","fyll Anställningsnummer: AN-338201","fyll Avdelning: IT och digitalisering","fyll Bankgiro: 589-2240","fyll Plusgiro: 38 24 19-6","fyll Belopp: 12 450,00 kr","fyll Momsregistreringsnummer: SE556677889901","fyll Utfärdad: 2026-09-11","fyll Förfallodatum: 2026-10-15","fyll Referens: 8836001928","fyll Organisationsnummer: 556677-8899","fyll Kundnummer: M-441023","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Norrsken Teknik AB - Reseräkning\nELEMENT Edit \"Summa\" value=\"12 450,00 kr\"","label":14,"meta":{"action":"skip","form":"Norrsken Teknik AB - Reseräkning","gold_option":"hoppa över","role":"Edit","source":"hand-authored"},"options":["fyll Namn: Johan Bergström","fyll Anställningsnummer: AN-338201","fyll Avdelning: IT och digitalisering","fyll Bankgiro: 589-2240","fyll Plusgiro: 38 24 19-6","fyll Belopp: 12 450,00 kr","fyll Momsregistreringsnummer: SE556677889901","fyll Utfärdad: 2026-09-11","fyll Förfallodatum: 2026-10-15","fyll Referens: 8836001928","fyll Organisationsnummer: 556677-8899","fyll Kundnummer: M-441023","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Norrsken Teknik AB - Reseräkning\nELEMENT CheckBox \"Jag intygar att uppgifterna är korrekta\" unchecked","label":12,"meta":{"action":"check","form":"Norrsken Teknik AB - Reseräkning","gold_option":"kryssa","role":"CheckBox","source":"hand-authored"},"options":["fyll Namn: Johan Bergström","fyll Anställningsnummer: AN-338201","fyll Avdelning: IT och digitalisering","fyll Bankgiro: 589-2240","fyll Plusgiro: 38 24 19-6","fyll Belopp: 12 450,00 kr","fyll Momsregistreringsnummer: SE556677889901","fyll Utfärdad: 2026-09-11","fyll Förfallodatum: 2026-10-15","fyll Referens: 8836001928","fyll Organisationsnummer: 556677-8899","fyll Kundnummer: M-441023","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Norrsken Teknik AB - Reseräkning\nELEMENT Button \"Spara utkast\" value=\"\"","label":14,"meta":{"action":"skip","form":"Norrsken Teknik AB - Reseräkning","gold_option":"hoppa över","role":"Button","source":"hand-authored"},"options":["fyll Namn: Johan Bergström","fyll Anställningsnummer: AN-338201","fyll Avdelning: IT och digitalisering","fyll Bankgiro: 589-2240","fyll Plusgiro: 38 24 19-6","fyll Belopp: 12 450,00 kr","fyll Momsregistreringsnummer: SE556677889901","fyll Utfärdad: 2026-09-11","fyll Förfallodatum: 2026-10-15","fyll Referens: 8836001928","fyll Organisationsnummer: 556677-8899","fyll Kundnummer: M-441023","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Norrsken Teknik AB - Reseräkning\nELEMENT Button \"Skicka\" value=\"\"","label":13,"meta":{"action":"click","form":"Norrsken Teknik AB - Reseräkning","gold_option":"klicka","role":"Button","source":"hand-authored"},"options":["fyll Namn: Johan Bergström","fyll Anställningsnummer: AN-338201","fyll Avdelning: IT och digitalisering","fyll Bankgiro: 589-2240","fyll Plusgiro: 38 24 19-6","fyll Belopp: 12 450,00 kr","fyll Momsregistreringsnummer: SE556677889901","fyll Utfärdad: 2026-09-11","fyll Förfallodatum: 2026-10-15","fyll Referens: 8836001928","fyll Organisationsnummer: 556677-8899","fyll Kundnummer: M-441023","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Norrsken Teknik AB - Reseräkning\nELEMENT MenuItem \"Arkiv\" value=\"\"","label":14,"meta":{"action":"skip","form":"Norrsken Teknik AB - Reseräkning","gold_option":"hoppa över","role":"MenuItem","source":"hand-authored"},"options":["fyll Namn: Johan Bergström","fyll Anställningsnummer: AN-338201","fyll Avdelning: IT och digitalisering","fyll Bankgiro: 589-2240","fyll Plusgiro: 38 24 19-6","fyll Belopp: 12 450,00 kr","fyll Momsregistreringsnummer: SE556677889901","fyll Utfärdad: 2026-09-11","fyll Förfallodatum: 2026-10-15","fyll Referens: 8836001928","fyll Organisationsnummer: 556677-8899","fyll Kundnummer: M-441023","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Testförsäkring - Skadeanmälan\nELEMENT Edit \"Försäkringsnr\" value=\"\"","label":0,"meta":{"action":"fill","form":"Testförsäkring - Skadeanmälan","gold_option":"fyll Försäkringsnummer: F-2291-48310","role":"Edit","source":"hand-authored"},"options":["fyll Försäkringsnummer: F-2291-48310","fyll Skadenummer: S-771204","fyll Skadedatum: 2026-08-27","fyll Händelseort: Västerås","fyll Besök: Ort","fyll Registreringsnummer: JHL 402","fyll Fordonsmärke: Testmotors Prov","fyll Beskrivning: Skadan upptäcktes vid leverans.","fyll Handläggare: Malin Sjögren","fyll Ärendenummer: EX-2026-00281","fyll Bankgiro: 341-9087","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Testförsäkring - Skadeanmälan\nELEMENT Edit \"Skadenummer\" value=\"\"","label":1,"meta":{"action":"fill","form":"Testförsäkring - Skadeanmälan","gold_option":"fyll Skadenummer: S-771204","role":"Edit","source":"hand-authored"},"options":["fyll Försäkringsnummer: F-2291-48310","fyll Skadenummer: S-771204","fyll Skadedatum: 2026-08-27","fyll Händelseort: Västerås","fyll Besök: Ort","fyll Registreringsnummer: JHL 402","fyll Fordonsmärke: Testmotors Prov","fyll Beskrivning: Skadan upptäcktes vid leverans.","fyll Handläggare: Malin Sjögren","fyll Ärendenummer: EX-2026-00281","fyll Bankgiro: 341-9087","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Testförsäkring - Skadeanmälan\nELEMENT Edit \"Datum för skadan\" value=\"\"","label":2,"meta":{"action":"fill","form":"Testförsäkring - Skadeanmälan","gold_option":"fyll Skadedatum: 2026-08-27","role":"Edit","source":"hand-authored"},"options":["fyll Försäkringsnummer: F-2291-48310","fyll Skadenummer: S-771204","fyll Skadedatum: 2026-08-27","fyll Händelseort: Västerås","fyll Besök: Ort","fyll Registreringsnummer: JHL 402","fyll Fordonsmärke: Testmotors Prov","fyll Beskrivning: Skadan upptäcktes vid leverans.","fyll Handläggare: Malin Sjögren","fyll Ärendenummer: EX-2026-00281","fyll Bankgiro: 341-9087","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Testförsäkring - Skadeanmälan\nELEMENT Edit \"Plats\" value=\"\"","label":3,"meta":{"action":"fill","form":"Testförsäkring - Skadeanmälan","gold_option":"fyll Händelseort: Västerås","role":"Edit","source":"hand-authored"},"options":["fyll Försäkringsnummer: F-2291-48310","fyll Skadenummer: S-771204","fyll Skadedatum: 2026-08-27","fyll Händelseort: Västerås","fyll Besök: Ort","fyll Registreringsnummer: JHL 402","fyll Fordonsmärke: Testmotors Prov","fyll Beskrivning: Skadan upptäcktes vid leverans.","fyll Handläggare: Malin Sjögren","fyll Ärendenummer: EX-2026-00281","fyll Bankgiro: 341-9087","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Testförsäkring - Skadeanmälan\nELEMENT Edit \"Registreringsnummer\" value=\"\" hint=\"ABC 123\"","label":5,"meta":{"action":"fill","form":"Testförsäkring - Skadeanmälan","gold_option":"fyll Registreringsnummer: JHL 402","role":"Edit","source":"hand-authored"},"options":["fyll Försäkringsnummer: F-2291-48310","fyll Skadenummer: S-771204","fyll Skadedatum: 2026-08-27","fyll Händelseort: Västerås","fyll Besök: Ort","fyll Registreringsnummer: JHL 402","fyll Fordonsmärke: Testmotors Prov","fyll Beskrivning: Skadan upptäcktes vid leverans.","fyll Handläggare: Malin Sjögren","fyll Ärendenummer: EX-2026-00281","fyll Bankgiro: 341-9087","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Testförsäkring - Skadeanmälan\nELEMENT Edit \"Fordonsmärke\" value=\"\"","label":6,"meta":{"action":"fill","form":"Testförsäkring - Skadeanmälan","gold_option":"fyll Fordonsmärke: Testmotors Prov","role":"Edit","source":"hand-authored"},"options":["fyll Försäkringsnummer: F-2291-48310","fyll Skadenummer: S-771204","fyll Skadedatum: 2026-08-27","fyll Händelseort: Västerås","fyll Besök: Ort","fyll Registreringsnummer: JHL 402","fyll Fordonsmärke: Testmotors Prov","fyll Beskrivning: Skadan upptäcktes vid leverans.","fyll Handläggare: Malin Sjögren","fyll Ärendenummer: EX-2026-00281","fyll Bankgiro: 341-9087","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Testförsäkring - Skadeanmälan\nELEMENT Edit \"Vad inträffade?\" value=\"\"","label":7,"meta":{"action":"fill","form":"Testförsäkring - Skadeanmälan","gold_option":"fyll Beskrivning: Skadan upptäcktes vid leverans.","role":"Edit","source":"hand-authored"},"options":["fyll Försäkringsnummer: F-2291-48310","fyll Skadenummer: S-771204","fyll Skadedatum: 2026-08-27","fyll Händelseort: Västerås","fyll Besök: Ort","fyll Registreringsnummer: JHL 402","fyll Fordonsmärke: Testmotors Prov","fyll Beskrivning: Skadan upptäcktes vid leverans.","fyll Handläggare: Malin Sjögren","fyll Ärendenummer: EX-2026-00281","fyll Bankgiro: 341-9087","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Testförsäkring - Skadeanmälan\nELEMENT Edit \"Ärendenummer\" value=\"\"","label":9,"meta":{"action":"fill","form":"Testförsäkring - Skadeanmälan","gold_option":"fyll Ärendenummer: EX-2026-00281","role":"Edit","source":"hand-authored"},"options":["fyll Försäkringsnummer: F-2291-48310","fyll Skadenummer: S-771204","fyll Skadedatum: 2026-08-27","fyll Händelseort: Västerås","fyll Besök: Ort","fyll Registreringsnummer: JHL 402","fyll Fordonsmärke: Testmotors Prov","fyll Beskrivning: Skadan upptäcktes vid leverans.","fyll Handläggare: Malin Sjögren","fyll Ärendenummer: EX-2026-00281","fyll Bankgiro: 341-9087","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Testförsäkring - Skadeanmälan\nELEMENT Edit \"Skadenummer\" value=\"S-771204\"","label":13,"meta":{"action":"skip","form":"Testförsäkring - Skadeanmälan","gold_option":"hoppa över","role":"Edit","source":"hand-authored"},"options":["fyll Försäkringsnummer: F-2291-48310","fyll Skadenummer: S-771204","fyll Skadedatum: 2026-08-27","fyll Händelseort: Västerås","fyll Besök: Ort","fyll Registreringsnummer: JHL 402","fyll Fordonsmärke: Testmotors Prov","fyll Beskrivning: Skadan upptäcktes vid leverans.","fyll Handläggare: Malin Sjögren","fyll Ärendenummer: EX-2026-00281","fyll Bankgiro: 341-9087","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Testförsäkring - Skadeanmälan\nELEMENT CheckBox \"Jag har tagit del av informationen om behandling av personuppgifter\" unchecked","label":11,"meta":{"action":"check","form":"Testförsäkring - Skadeanmälan","gold_option":"kryssa","role":"CheckBox","source":"hand-authored"},"options":["fyll Försäkringsnummer: F-2291-48310","fyll Skadenummer: S-771204","fyll Skadedatum: 2026-08-27","fyll Händelseort: Västerås","fyll Besök: Ort","fyll Registreringsnummer: JHL 402","fyll Fordonsmärke: Testmotors Prov","fyll Beskrivning: Skadan upptäcktes vid leverans.","fyll Handläggare: Malin Sjögren","fyll Ärendenummer: EX-2026-00281","fyll Bankgiro: 341-9087","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Testförsäkring - Skadeanmälan\nELEMENT CheckBox \"Kontakta mig per e-post\" unchecked","label":13,"meta":{"action":"skip","form":"Testförsäkring - Skadeanmälan","gold_option":"hoppa över","role":"CheckBox","source":"hand-authored"},"options":["fyll Försäkringsnummer: F-2291-48310","fyll Skadenummer: S-771204","fyll Skadedatum: 2026-08-27","fyll Händelseort: Västerås","fyll Besök: Ort","fyll Registreringsnummer: JHL 402","fyll Fordonsmärke: Testmotors Prov","fyll Beskrivning: Skadan upptäcktes vid leverans.","fyll Handläggare: Malin Sjögren","fyll Ärendenummer: EX-2026-00281","fyll Bankgiro: 341-9087","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Testförsäkring - Skadeanmälan\nELEMENT Button \"Ladda upp bilaga\" value=\"\"","label":13,"meta":{"action":"skip","form":"Testförsäkring - Skadeanmälan","gold_option":"hoppa över","role":"Button","source":"hand-authored"},"options":["fyll Försäkringsnummer: F-2291-48310","fyll Skadenummer: S-771204","fyll Skadedatum: 2026-08-27","fyll Händelseort: Västerås","fyll Besök: Ort","fyll Registreringsnummer: JHL 402","fyll Fordonsmärke: Testmotors Prov","fyll Beskrivning: Skadan upptäcktes vid leverans.","fyll Handläggare: Malin Sjögren","fyll Ärendenummer: EX-2026-00281","fyll Bankgiro: 341-9087","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Testförsäkring - Skadeanmälan\nELEMENT Button \"Skicka ansökan\" value=\"\"","label":12,"meta":{"action":"click","form":"Testförsäkring - Skadeanmälan","gold_option":"klicka","role":"Button","source":"hand-authored"},"options":["fyll Försäkringsnummer: F-2291-48310","fyll Skadenummer: S-771204","fyll Skadedatum: 2026-08-27","fyll Händelseort: Västerås","fyll Besök: Ort","fyll Registreringsnummer: JHL 402","fyll Fordonsmärke: Testmotors Prov","fyll Beskrivning: Skadan upptäcktes vid leverans.","fyll Handläggare: Malin Sjögren","fyll Ärendenummer: EX-2026-00281","fyll Bankgiro: 341-9087","kryssa","klicka","hoppa över"]}
+{"context":"UPPGIFT fyll i formuläret från dokumentet och skicka sedan in\nFORM Testförsäkring - Skadeanmälan\nELEMENT Button \"Stäng\" value=\"\"","label":13,"meta":{"action":"skip","form":"Testförsäkring - Skadeanmälan","gold_option":"hoppa över","role":"Button","source":"hand-authored"},"options":["fyll Försäkringsnummer: F-2291-48310","fyll Skadenummer: S-771204","fyll Skadedatum: 2026-08-27","fyll Händelseort: Västerås","fyll Besök: Ort","fyll Registreringsnummer: JHL 402","fyll Fordonsmärke: Testmotors Prov","fyll Beskrivning: Skadan upptäcktes vid leverans.","fyll Handläggare: Malin Sjögren","fyll Ärendenummer: EX-2026-00281","fyll Bankgiro: 341-9087","kryssa","klicka","hoppa över"]}

eval/results-coreml.json

@@ -1,40 +1,41 @@
 {
-  "checkpoint": "runs/sv-tinyx/model",
+  "checkpoint": "runs/sv-tinyx-10000/model",
   "limits": {
     "context_bytes": 224,
     "max_options": 40,
     "option_bytes": 96
   },
   "torch_baseline": {
+    "examples": 29839,
     "per_action": {
       "check": {
-        "acc": 0.9827586206896551,
-        "n": 58
+        "acc": 1.0,
+        "n": 855
       },
       "click": {
-        "acc": 0.9367088607594937,
-        "n": 79
+        "acc": 1.0,
+        "n": 1009
       },
       "fill": {
-        "acc": 0.7554455445544555,
-        "n": 1010
+        "acc": 0.9900317827217567,
+        "n": 13844
       },
       "skip": {
-        "acc": 0.8801369863013698,
-        "n": 1168
+        "acc": 0.9947632863916213,
+        "n": 14131
       }
     },
-    "rows": 2315,
-    "silent_skip_rate_of_fills": 0.07524752475247524,
-    "silent_skips": 76,
-    "top1": 0.8302375809935205
+    "silent_skip_rate_of_fills": 7.22334585379948e-05,
+    "skipped_when_fill_expected": 1,
+    "top1": 0.9928952042628775
   },
   "variants": {
     "fp16": {
       "handwritten_demo__ane": {
         "compute_units": "ComputeUnit.CPU_AND_NE",
-        "latency_ms_median": 1.2992918491363525,
-        "latency_ms_p95": 1.4392081648111343,
+        "examples": 47,
+        "latency_ms_median": 1.3022092171013355,
+        "latency_ms_p95": 1.6720420680940151,
         "mismatches_vs_torch": 0,
         "parity_with_torch": 1.0,
         "per_action": {
@@ -47,84 +48,84 @@
             "n": 3
           },
           "fill": {
-            "acc": 0.7931034482758621,
+            "acc": 1.0,
             "n": 29
           },
           "skip": {
-            "acc": 0.9166666666666666,
+            "acc": 1.0,
             "n": 12
           }
         },
-        "rows": 47,
-        "silent_skip_rate_of_fills": 0.10344827586206896,
-        "silent_skips": 3,
-        "top1": 0.851063829787234
+        "silent_skip_rate_of_fills": 0.0,
+        "skipped_when_fill_expected": 0,
+        "top1": 1.0
       },
-      "package_bytes": 1507481,
+      "package_bytes": 1507474,
       "synthetic_test__ane": {
         "compute_units": "ComputeUnit.CPU_AND_NE",
-        "latency_ms_median": 1.3071454595774412,
-        "latency_ms_p95": 1.4601247385144234,
-        "mismatches_vs_torch": 3,
-        "parity_with_torch": 0.9987024221453287,
+        "examples": 29836,
+        "latency_ms_median": 1.313916640356183,
+        "latency_ms_p95": 1.412208192050457,
+        "mismatches_vs_torch": 15,
+        "parity_with_torch": 0.9994972516423113,
         "per_action": {
           "check": {
-            "acc": 0.9827586206896551,
-            "n": 58
+            "acc": 1.0,
+            "n": 855
           },
           "click": {
-            "acc": 0.9367088607594937,
-            "n": 79
+            "acc": 1.0,
+            "n": 1009
           },
           "fill": {
-            "acc": 0.753968253968254,
-            "n": 1008
+            "acc": 0.9899580985406733,
+            "n": 13842
           },
           "skip": {
-            "acc": 0.8808911739502999,
-            "n": 1167
+            "acc": 0.994762915782024,
+            "n": 14130
           }
         },
-        "rows": 2312,
-        "silent_skip_rate_of_fills": 0.07539682539682539,
-        "silent_skips": 76,
-        "top1": 0.8300173010380623
+        "silent_skip_rate_of_fills": 7.224389539083947e-05,
+        "skipped_when_fill_expected": 1,
+        "top1": 0.9928609733208205
       },
       "synthetic_test__cpu": {
         "compute_units": "ComputeUnit.CPU_ONLY",
-        "latency_ms_median": 1.733562909066677,
-        "latency_ms_p95": 1.9167079590260983,
-        "mismatches_vs_torch": 2,
-        "parity_with_torch": 0.9991349480968859,
+        "examples": 29836,
+        "latency_ms_median": 1.7267910297960043,
+        "latency_ms_p95": 1.9287499599158764,
+        "mismatches_vs_torch": 14,
+        "parity_with_torch": 0.9995307681994905,
         "per_action": {
           "check": {
-            "acc": 0.9827586206896551,
-            "n": 58
+            "acc": 1.0,
+            "n": 855
           },
           "click": {
-            "acc": 0.9367088607594937,
-            "n": 79
+            "acc": 1.0,
+            "n": 1009
           },
           "fill": {
-            "acc": 0.7549603174603174,
-            "n": 1008
+            "acc": 0.9897413668545008,
+            "n": 13842
           },
           "skip": {
-            "acc": 0.8808911739502999,
-            "n": 1167
+            "acc": 0.994762915782024,
+            "n": 14130
           }
         },
-        "rows": 2312,
-        "silent_skip_rate_of_fills": 0.07539682539682539,
-        "silent_skips": 76,
-        "top1": 0.8304498269896193
+        "silent_skip_rate_of_fills": 7.224389539083947e-05,
+        "skipped_when_fill_expected": 1,
+        "top1": 0.9927604236492827
       }
     },
     "int4": {
       "handwritten_demo__ane": {
         "compute_units": "ComputeUnit.CPU_AND_NE",
-        "latency_ms_median": 1.8591247498989105,
-        "latency_ms_p95": 1.993041019886732,
+        "examples": 47,
+        "latency_ms_median": 1.8585827201604843,
+        "latency_ms_p95": 2.018083818256855,
         "mismatches_vs_torch": 32,
         "parity_with_torch": 0.3191489361702128,
         "per_action": {
@@ -137,7 +138,7 @@
             "n": 3
           },
           "fill": {
-            "acc": 0.034482758620689655,
+            "acc": 0.10344827586206896,
             "n": 29
           },
           "skip": {
@@ -145,47 +146,47 @@
             "n": 12
           }
         },
-        "rows": 47,
-        "silent_skip_rate_of_fills": 0.9655172413793104,
-        "silent_skips": 28,
-        "top1": 0.2765957446808511
+        "silent_skip_rate_of_fills": 0.896551724137931,
+        "skipped_when_fill_expected": 26,
+        "top1": 0.3191489361702128
       },
-      "package_bytes": 492584,
+      "package_bytes": 492577,
       "synthetic_test__ane": {
         "compute_units": "ComputeUnit.CPU_AND_NE",
-        "latency_ms_median": 1.9683334976434708,
-        "latency_ms_p95": 2.099000383168459,
-        "mismatches_vs_torch": 1186,
-        "parity_with_torch": 0.4870242214532872,
+        "examples": 29836,
+        "latency_ms_median": 1.9839999731630087,
+        "latency_ms_p95": 2.11525009945035,
+        "mismatches_vs_torch": 15015,
+        "parity_with_torch": 0.4967488939536131,
         "per_action": {
           "check": {
-            "acc": 0.0,
-            "n": 58
+            "acc": 0.012865497076023392,
+            "n": 855
           },
           "click": {
             "acc": 0.0,
-            "n": 79
+            "n": 1009
           },
           "fill": {
-            "acc": 0.018849206349206348,
-            "n": 1008
+            "acc": 0.06408033521167461,
+            "n": 13842
           },
           "skip": {
-            "acc": 0.9914310197086547,
-            "n": 1167
+            "acc": 0.9905166312809625,
+            "n": 14130
           }
         },
-        "rows": 2312,
-        "silent_skip_rate_of_fills": 0.9464285714285714,
-        "silent_skips": 954,
-        "top1": 0.5086505190311419
+        "silent_skip_rate_of_fills": 0.9143909839618553,
+        "skipped_when_fill_expected": 12657,
+        "top1": 0.4991956026276981
       }
     },
     "int8": {
       "handwritten_demo__ane": {
         "compute_units": "ComputeUnit.CPU_AND_NE",
-        "latency_ms_median": 1.31650036200881,
-        "latency_ms_p95": 1.4507090672850609,
+        "examples": 47,
+        "latency_ms_median": 1.3158749788999557,
+        "latency_ms_p95": 1.4334581792354584,
         "mismatches_vs_torch": 0,
         "parity_with_torch": 1.0,
         "per_action": {
@@ -198,48 +199,47 @@
             "n": 3
           },
           "fill": {
-            "acc": 0.7931034482758621,
+            "acc": 1.0,
             "n": 29
           },
           "skip": {
-            "acc": 0.9166666666666666,
+            "acc": 1.0,
             "n": 12
           }
         },
-        "rows": 47,
-        "silent_skip_rate_of_fills": 0.10344827586206896,
-        "silent_skips": 3,
-        "top1": 0.851063829787234
+        "silent_skip_rate_of_fills": 0.0,
+        "skipped_when_fill_expected": 0,
+        "top1": 1.0
       },
-      "package_bytes": 806132,
+      "package_bytes": 806125,
       "synthetic_test__ane": {
         "compute_units": "ComputeUnit.CPU_AND_NE",
-        "latency_ms_median": 1.315583474934101,
-        "latency_ms_p95": 1.401625107973814,
-        "mismatches_vs_torch": 9,
-        "parity_with_torch": 0.9961072664359861,
+        "examples": 29836,
+        "latency_ms_median": 1.3136658817529678,
+        "latency_ms_p95": 1.4113341458141804,
+        "mismatches_vs_torch": 13,
+        "parity_with_torch": 0.9995642847566698,
         "per_action": {
           "check": {
-            "acc": 0.9827586206896551,
-            "n": 58
+            "acc": 1.0,
+            "n": 855
           },
           "click": {
-            "acc": 0.9367088607594937,
-            "n": 79
+            "acc": 1.0,
+            "n": 1009
           },
           "fill": {
-            "acc": 0.753968253968254,
-            "n": 1008
+            "acc": 0.990102586331455,
+            "n": 13842
           },
           "skip": {
-            "acc": 0.882604970008569,
-            "n": 1167
+            "acc": 0.994762915782024,
+            "n": 14130
           }
         },
-        "rows": 2312,
-        "silent_skip_rate_of_fills": 0.07539682539682539,
-        "silent_skips": 76,
-        "top1": 0.8308823529411765
+        "silent_skip_rate_of_fills": 7.224389539083947e-05,
+        "skipped_when_fill_expected": 1,
+        "top1": 0.992928006435179
       }
     }
   }

eval/results-pytorch.json

@@ -1,147 +1,11 @@
 {
-  "device": "mps",
-  "reference": {
-    "checkpoint": "/tmp/cua1ckpt/cua-s1-forms.safetensors",
-    "checkpoint_bytes": 2828784,
-    "config": {
-      "context_tokens": 224,
-      "encoder": "tinyx",
-      "heads": 4,
-      "hf_model": "Qwen/Qwen2.5-0.5B",
-      "layers": 2,
-      "option_tokens": 96,
-      "rank": 128,
-      "width": 128
-    },
-    "label": "cua-s1-forms-english",
-    "sv_handwritten_demo": {
-      "action_mix": {
-        "check": 3,
-        "click": 3,
-        "fill": 32,
-        "skip": 12
-      },
-      "ece": 0.660977,
-      "examples": 50,
-      "majority_baseline": 0.64,
-      "per_action": {
-        "check": {
-          "acc": 0.66667,
-          "n": 3
-        },
-        "click": {
-          "acc": 0.33333,
-          "n": 3
-        },
-        "fill": {
-          "acc": 0.15625,
-          "n": 32
-        },
-        "skip": {
-          "acc": 0.16667,
-          "n": 12
-        }
-      },
-      "skipped_when_fill_expected": 16,
-      "top1": 0.2
-    },
-    "sv_handwritten_demo__shuffled_context": {
-      "action_mix": {
-        "check": 3,
-        "click": 3,
-        "fill": 32,
-        "skip": 12
-      },
-      "ece": 0.760258,
-      "examples": 50,
-      "majority_baseline": 0.64,
-      "per_action": {
-        "check": {
-          "acc": 0.66667,
-          "n": 3
-        },
-        "click": {
-          "acc": 0.0,
-          "n": 3
-        },
-        "fill": {
-          "acc": 0.0,
-          "n": 32
-        },
-        "skip": {
-          "acc": 0.25,
-          "n": 12
-        }
-      },
-      "skipped_when_fill_expected": 13,
-      "top1": 0.1
-    },
-    "sv_synthetic_test": {
-      "action_mix": {
-        "check": 58,
-        "click": 79,
-        "fill": 1010,
-        "skip": 1168
-      },
-      "ece": 0.640725,
-      "examples": 2315,
-      "majority_baseline": 0.50454,
-      "per_action": {
-        "check": {
-          "acc": 0.93103,
-          "n": 58
-        },
-        "click": {
-          "acc": 0.26582,
-          "n": 79
-        },
-        "fill": {
-          "acc": 0.05545,
-          "n": 1010
-        },
-        "skip": {
-          "acc": 0.31164,
-          "n": 1168
-        }
-      },
-      "skipped_when_fill_expected": 371,
-      "top1": 0.21382
-    },
-    "sv_synthetic_test__shuffled_context": {
-      "action_mix": {
-        "check": 58,
-        "click": 79,
-        "fill": 1010,
-        "skip": 1168
-      },
-      "ece": 0.679541,
-      "examples": 2315,
-      "majority_baseline": 0.50454,
-      "per_action": {
-        "check": {
-          "acc": 0.60345,
-          "n": 58
-        },
-        "click": {
-          "acc": 0.06329,
-          "n": 79
-        },
-        "fill": {
-          "acc": 0.01782,
-          "n": 1010
-        },
-        "skip": {
-          "acc": 0.29709,
-          "n": 1168
-        }
-      },
-      "skipped_when_fill_expected": 363,
-      "top1": 0.17495
-    },
-    "trainable_params": 706048
+  "datasets": {
+    "handwritten_demo": "/Users/hermes-tool/builder-work/one-pass-specialists-67k19_tm/data/sv-10000/demo-handwritten.jsonl",
+    "synthetic_test": "/Users/hermes-tool/builder-work/one-pass-specialists-67k19_tm/data/sv-10000/test.jsonl"
   },
-  "sv": {
-    "checkpoint": "runs/sv-tinyx/model/model.safetensors",
+  "device": "mps",
+  "specialist": {
+    "checkpoint": "/Users/hermes-tool/builder-work/one-pass-specialists-67k19_tm/runs/sv-tinyx-10000/model/model.safetensors",
     "checkpoint_bytes": 2828784,
     "config": {
       "context_tokens": 224,
@@ -152,15 +16,14 @@
       "rank": 128,
       "width": 128
     },
-    "label": "swedish-specialist",
-    "sv_handwritten_demo": {
+    "handwritten_demo": {
       "action_mix": {
         "check": 3,
         "click": 3,
         "fill": 32,
         "skip": 12
       },
-      "ece": 0.09176,
+      "ece": 0.000869,
       "examples": 50,
       "majority_baseline": 0.64,
       "per_action": {
@@ -173,25 +36,25 @@
           "n": 3
         },
         "fill": {
-          "acc": 0.8125,
+          "acc": 1.0,
           "n": 32
         },
         "skip": {
-          "acc": 0.91667,
+          "acc": 1.0,
           "n": 12
         }
       },
-      "skipped_when_fill_expected": 3,
-      "top1": 0.86
+      "skipped_when_fill_expected": 0,
+      "top1": 1.0
     },
-    "sv_handwritten_demo__shuffled_context": {
+    "handwritten_demo__shuffled_context": {
       "action_mix": {
         "check": 3,
         "click": 3,
         "fill": 32,
         "skip": 12
       },
-      "ece": 0.785789,
+      "ece": 0.819131,
       "examples": 50,
       "majority_baseline": 0.64,
       "per_action": {
@@ -212,70 +75,71 @@
           "n": 12
         }
       },
-      "skipped_when_fill_expected": 6,
+      "skipped_when_fill_expected": 3,
       "top1": 0.06
     },
-    "sv_synthetic_test": {
+    "label": "specialist",
+    "synthetic_test": {
       "action_mix": {
-        "check": 58,
-        "click": 79,
-        "fill": 1010,
-        "skip": 1168
-      },
-      "ece": 0.016565,
-      "examples": 2315,
-      "majority_baseline": 0.50454,
+        "check": 855,
+        "click": 1009,
+        "fill": 13844,
+        "skip": 14131
+      },
+      "ece": 0.001373,
+      "examples": 29839,
+      "majority_baseline": 0.47357,
       "per_action": {
         "check": {
-          "acc": 0.98276,
-          "n": 58
+          "acc": 1.0,
+          "n": 855
         },
         "click": {
-          "acc": 0.93671,
-          "n": 79
+          "acc": 1.0,
+          "n": 1009
         },
         "fill": {
-          "acc": 0.75545,
-          "n": 1010
+          "acc": 0.99003,
+          "n": 13844
         },
         "skip": {
-          "acc": 0.88014,
-          "n": 1168
+          "acc": 0.99476,
+          "n": 14131
         }
       },
-      "skipped_when_fill_expected": 76,
-      "top1": 0.83024
+      "skipped_when_fill_expected": 1,
+      "top1": 0.9929
     },
-    "sv_synthetic_test__shuffled_context": {
+    "synthetic_test__shuffled_context": {
       "action_mix": {
-        "check": 58,
-        "click": 79,
-        "fill": 1010,
-        "skip": 1168
-      },
-      "ece": 0.497346,
-      "examples": 2315,
-      "majority_baseline": 0.50454,
+        "check": 855,
+        "click": 1009,
+        "fill": 13844,
+        "skip": 14131
+      },
+      "ece": 0.607719,
+      "examples": 29839,
+      "majority_baseline": 0.47357,
       "per_action": {
         "check": {
-          "acc": 0.15517,
-          "n": 58
+          "acc": 0.20936,
+          "n": 855
         },
         "click": {
           "acc": 0.0,
-          "n": 79
+          "n": 1009
         },
         "fill": {
-          "acc": 0.01287,
-          "n": 1010
+          "acc": 0.00079,
+          "n": 13844
         },
         "skip": {
-          "acc": 0.65668,
-          "n": 1168
+          "acc": 0.66209,
+          "n": 14131
         }
       },
-      "skipped_when_fill_expected": 264,
-      "top1": 0.34082
+      "skipped_when_fill_expected": 3774,
+      "top1": 0.31992
     },
     "trainable_params": 706048
   }

eval/size-sweep.jsonl

@@ -0,0 +1,3 @@
+{"checkpoint": "runs/sv-tinyx-v2/model", "demo_top1": 0.76, "episodes": 900, "epochs": 4, "note": "900-point: corrected generator, 4 epochs, seed 2026", "shuffled_top1": 0.33203, "test_ece": 0.021596, "test_majority": 0.48874, "test_silent_skips": 127, "test_top1": 0.771}
+{"checkpoint": "/Users/hermes-tool/builder-work/one-pass-specialists-67k19_tm/runs/sv-tinyx-4000/model", "demo_top1": 1.0, "episodes": 4000, "epochs": 4, "rows": {"train": 94405}, "seconds": 3376.7, "shuffled_top1": 0.31568, "test_ece": 0.001389, "test_majority": 0.47127, "test_silent_skips": 6, "test_top1": 0.98944}
+{"checkpoint": "/Users/hermes-tool/builder-work/one-pass-specialists-67k19_tm/runs/sv-tinyx-10000/model", "demo_top1": 1.0, "episodes": 10000, "epochs": 4, "rows": {"train": 234921}, "seconds": 9763.0, "shuffled_top1": 0.31992, "test_ece": 0.001373, "test_majority": 0.47357, "test_silent_skips": 1, "test_top1": 0.9929}

sv0-forms.json

@@ -13,142 +13,142 @@
   "metadata": {
     "batch_size": 64,
     "best_validation": {
-      "ece": 0.018766840075841174,
-      "examples": 2855,
-      "nll": 0.51966637646463,
+      "ece": 0.0011067466822396455,
+      "examples": 29649,
+      "nll": 0.014722759442557139,
       "per_action": {
         "check": {
           "acc": 1.0,
-          "n": 87
+          "n": 833
         },
         "click": {
-          "acc": 0.9479166666666666,
-          "n": 96
+          "acc": 1.0,
+          "n": 1007
         },
         "fill": {
-          "acc": 0.7756680731364276,
-          "n": 1422
+          "acc": 0.9877524709926945,
+          "n": 13962
         },
         "skip": {
-          "acc": 0.8712,
-          "n": 1250
+          "acc": 0.9952336246118293,
+          "n": 13847
         }
       },
-      "top1": 0.8301225900650024
+      "top1": 0.9920064806938171
     },
     "deterministic": false,
     "epochs": 4,
     "history": [
       {
         "epoch": 1,
-        "train_nll": 2.060510016020325,
+        "train_nll": 0.6810737938714119,
         "validation": {
-          "ece": 0.05491860159418138,
-          "examples": 2855,
-          "nll": 1.748936438351713,
+          "ece": 0.0038536921401828295,
+          "examples": 29649,
+          "nll": 0.10877726682664057,
           "per_action": {
             "check": {
-              "acc": 0.4367816091954023,
-              "n": 87
+              "acc": 0.9867947178871549,
+              "n": 833
             },
             "click": {
-              "acc": 0.3958333333333333,
-              "n": 96
+              "acc": 1.0,
+              "n": 1007
             },
             "fill": {
-              "acc": 0.06118143459915612,
-              "n": 1422
+              "acc": 0.9732846297092107,
+              "n": 13962
             },
             "skip": {
-              "acc": 0.9712,
-              "n": 1250
+              "acc": 0.9447533761825666,
+              "n": 13847
             }
           },
-          "top1": 0.4823117256164551
+          "top1": 0.9612466096878052
         }
       },
       {
         "epoch": 2,
-        "train_nll": 1.3498624871227647,
+        "train_nll": 0.08150138849369405,
         "validation": {
-          "ece": 0.029052662350295577,
-          "examples": 2855,
-          "nll": 1.1316383522153945,
+          "ece": 0.004962782920756581,
+          "examples": 29649,
+          "nll": 0.04922109633023761,
           "per_action": {
             "check": {
-              "acc": 0.9770114942528736,
-              "n": 87
+              "acc": 1.0,
+              "n": 833
             },
             "click": {
-              "acc": 0.8854166666666666,
-              "n": 96
+              "acc": 0.9940417080436942,
+              "n": 1007
             },
             "fill": {
-              "acc": 0.4212376933895921,
-              "n": 1422
+              "acc": 0.9852456668099127,
+              "n": 13962
             },
             "skip": {
-              "acc": 0.8232,
-              "n": 1250
+              "acc": 0.9790568354156135,
+              "n": 13847
             }
           },
-          "top1": 0.6297723054885864
+          "top1": 0.983068585395813
         }
       },
       {
         "epoch": 3,
-        "train_nll": 0.7656327178699136,
+        "train_nll": 0.02742721090312225,
         "validation": {
-          "ece": 0.02394736194401048,
-          "examples": 2855,
-          "nll": 0.6415376866970296,
+          "ece": 0.0023602779376687977,
+          "examples": 29649,
+          "nll": 0.01792668065080643,
           "per_action": {
             "check": {
-              "acc": 0.9770114942528736,
-              "n": 87
+              "acc": 1.0,
+              "n": 833
             },
             "click": {
-              "acc": 0.9791666666666666,
-              "n": 96
+              "acc": 1.0,
+              "n": 1007
             },
             "fill": {
-              "acc": 0.7088607594936709,
-              "n": 1422
+              "acc": 0.988110585875949,
+              "n": 13962
             },
             "skip": {
-              "acc": 0.8464,
-              "n": 1250
+              "acc": 0.9944392287138009,
+              "n": 13847
             }
           },
-          "top1": 0.7863397598266602
+          "top1": 0.9918041229248047
         }
       },
       {
         "epoch": 4,
-        "train_nll": 0.4496109453594249,
+        "train_nll": 0.01454558765468025,
         "validation": {
-          "ece": 0.018766840075841174,
-          "examples": 2855,
-          "nll": 0.51966637646463,
+          "ece": 0.0011067466822396455,
+          "examples": 29649,
+          "nll": 0.014722759442557139,
           "per_action": {
             "check": {
               "acc": 1.0,
-              "n": 87
+              "n": 833
             },
             "click": {
-              "acc": 0.9479166666666666,
-              "n": 96
+              "acc": 1.0,
+              "n": 1007
             },
             "fill": {
-              "acc": 0.7756680731364276,
-              "n": 1422
+              "acc": 0.9877524709926945,
+              "n": 13962
             },
             "skip": {
-              "acc": 0.8712,
-              "n": 1250
+              "acc": 0.9952336246118293,
+              "n": 13847
             }
           },
-          "top1": 0.8301225900650024
+          "top1": 0.9920064806938171
         }
       }
     ],
@@ -158,5 +158,5 @@
     "warmup": 0.05,
     "workers": 0
   },
-  "state_signature": "86e836863df8a0b0caa34ac5eab18bcd4fb44e420f83e271c96379fac0b0aec4"
+  "state_signature": "7c37ee78ca9c522d7bc0470eb8b4294f35bbf48246197677fe7745972a09824e"
 }

sv0-forms.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b9030d7a7b7abfaf70dd1cc2a390568eaa3a55849daa0757b60d01dbab162356
+oid sha256:4f35bfac96dd23c058dfc098986c3f4cff39b836a177fb60e026b5f06c2ba782
 size 2828784