README.md

---
license: mit
language:
  - sv
library_name: coreml
pipeline_tag: text-classification
tags:
  - jev
  - system-one
  - one-pass-scorer
  - computer-use
  - form-filling
  - swedish
  - coreml
  - apple-silicon
base_model: []
datasets:
  - precisit/one-pass-sv-forms-synthetic
---

# One-Pass SV-Forms (SV0): a Swedish one-pass form specialist (research checkpoint)

**This is a research checkpoint, not a product.** It is a 706 048-parameter model that scores a
supplied list of options in **one forward pass**, using a System One-style interface.
It was trained from scratch on **synthetic** Swedish forms. It generates no text, needs no tokenizer, and answers in
a measured median of 1.314 ms per prediction on an Apple M4 with `CPU_AND_NE`
(Core ML packages: about 1,472 KiB fp16, 787 KiB int8). This excludes input encoding,
model loading and warm-up; it is not an end-to-end form completion time.

| | |
| --- | --- |
| Task | given a UI element (role, label, state) plus the entities extracted from a document, pick one option: `fyll <entitet>` / `kryssa` / `klicka` / `hoppa över` |
| Architecture | byte-level embeddings + 2-layer Transformer encoder (width 128, 4 heads) + option-attention head; `tinyx` |
| Parameters | 706 048 |
| Context / option budget | 224 bytes of context, 96 bytes per option, **up to 40 options** |
| Training data | 10 000 synthetic Swedish form episodes (234 921 decisions); **generated rather than collected from people or customers** |
| Licence | MIT (weights and code) |
| Lineage | uses vendored [Cua-S1](https://github.com/trycua/cua/tree/main/libs/cua-s1) architecture and training code, with an option-attention head credited to [jevlike](https://github.com/vinnylarouge/jevlike); new weights trained from scratch, not fine-tuned from Cua weights |

## Measured results (our harness, Apple M4)

| Run | decisions | top-1 | majority-class baseline | ECE | silently skipped an expected fill |
| --- | ---: | ---: | ---: | ---: | ---: |
| Held-out synthetic test (form-signature disjoint) | 29 839 | **99.29 %** | 47.36 % | 0.0014 | **1** |
| Hand-written out-of-distribution demo | 50 | **100.00 %** | 64.00 % | n/a | 0 |
| Shuffled-context control (same test rows) | 29 839 | 31.99 % | 47.36 % | n/a | n/a |

The shuffled-context control rotates contexts between rows while retaining their options
and labels. Accuracy falls to 31.99%, showing that the model uses the context rather than
only option statistics. It does not identify which context clues the model has learned.
The synthetic test contains 13,844 expected fill decisions; one is answered skip. This
count excludes choosing a wrong fill value. Fill accuracy is 99.00%.

For comparison, the unchanged released English checkpoint scores **20.75%** on the same
29,839 rows, with 5,412 of 13,844 expected fills skipped. Labels, action strings and the
generated distribution all change; the result does not isolate language as the cause.
The 50 handwritten decisions comprise 32 fills, 12 skips, three checks and three clicks.
They are a small diagnostic set, not a broad test of Swedish forms.

### Data-size curve (same 4-epoch schedule, same seed, corrected generator)

| Episodes | Train decisions | Test top-1 | Silent skips | ECE | Training time (M4) |
| ---: | ---: | ---: | ---: | ---: | ---: |
| 900 | 21 305 | 77.10 % | 127 | 0.0216 | 32 min |
| 4 000 | 94 405 | 98.94 % | 6 | 0.0014 | 56 min |
| **10 000** | **234 921** | **99.29 %** | **1** | 0.0014 | 163 min |

This checkpoint is the 10,000-episode point. Each run has its own generated test set.
At a fixed four epochs, the larger training sets also receive more optimization steps.
These runs do not isolate the effect of data size or establish convergence.

## Changelog

- **v2 (2026-09-21)**: corrected synthetic identifier formats and increased the corpus
  to 10,000 episodes. The generator previously emitted malformed personnummer- and
  organisationsnummer-shaped values. Tests now check the intended formats and checksums.
  Such formatting differences could provide shortcuts, but the runs do not measure how
  much the model used them. Comparing the earlier 83.02% run with the corrected
  900-episode run at 77.10% does not establish a cause for the difference: the generated
  training and test examples changed. Larger subsequent runs also used more training steps.
- **Core ML clarification**: int4 was initially evaluated with `CPU_AND_NE` and described
  as losing too much weight precision. The same package reaches 99.27% on CPU, versus
  49.92% with CPU and Neural Engine allowed. Actual operation placement and the root
  cause remain unknown. Int8 and fp16 are not prediction-identical. See the public
  [execution investigation](https://github.com/precisit/one-pass-specialists/blob/9240818b2f9221ecce9e27af11eab9816dfc51f9/examples/sv-forms/COREML-EXECUTION.md) for the full M4 comparison, M1 Max follow-up,
  decompression control and remaining tests.
- **v1 (2026-09-21)**: earlier 900-episode release, 83.02% top-1 and a reported 7.5%
  silent-skip rate. Its artifacts remain in repository history. Use each revision's
  own result files and denominators when comparing releases.

## What it is NOT

- **Not validated for real form completion.** Training data are generated, not collected
  from people or customer submissions. Names are constructed and email addresses use
  `.invalid`. Generated identifiers are not checked against registries; coincidental
  matches with real identifiers cannot be ruled out. The external checks are small:
  50 handwritten decisions and 270 decisions in simplified fixtures based on Kanslist
  and Pratsam forms. Their labels and controls differ from the current sites. They use
  synthetic scenarios and hand-mapped expected actions, not real submissions or browser runs.
  The fixture results are **52.8% and 77.3%**, with 57 of 65 expected Kanslist fills
  answered skip. These are expected fills, not necessarily HTML-required fields.
  Pratsam's fixture also has a gold-label defect: its scenario uses `mig själv`, while
  the radio mapping checks for `ja`, labelling all 15 radio rows skip. Those results
  are provisional. Vocabulary, input lengths, roles and fixture conventions need
  investigation; the current evidence does not isolate one cause.
- **Not an agent.** It does not generate text, cannot invent a value it was not given, and does
  not decide execution order: the option list and the sequence come from ordinary code around it.
- **Not safe unsupervised.** One expected fill out of 13,844 was answered "skip" in the
  synthetic test: a silent
  failure. Any real integration must check outcomes outside the model (fail closed, dry run, one
  submit, human review before consequential actions).
- **Not tuned for throughput.** One decision per `predict` call is what the latency numbers
  describe; batching is untested.

## Core ML export (same checkpoint, coremltools 9.0, Apple M4)

| Variant | package | CPU top-1 | CPU parity vs PyTorch (mismatches) | CPU + NE allowed top-1 | CPU + NE allowed parity (mismatches) |
| --- | ---: | ---: | ---: | ---: | ---: |
| fp16 | 1,472 KiB | 99.28 % | 0.999531 (14) | 99.29 % | 0.999497 (15) |
| **int8** | **787 KiB** | **99.31 %** | 0.999598 (12) | 99.29 % | 0.999564 (13) |
| int4 | 481 KiB | **99.27 %** | 0.998928 (32) | **49.92 %** | 0.496732 (15 017) |

All 29,839 held-out decisions are included, with the same encoding and masked argmax.
`CPU_AND_NE` allows CPU and Neural Engine use; it does not report which operations ran
on which processor. Package sizes sum the files, not runtime memory. One KiB is 1,024 bytes.

The int8 recipe uses a uniform 256-entry palette per tensor with float16 computation.
Int4 uses grouped-channel K-means palettes, groups of 32, and a different deployment
target: iOS 18, versus iOS 17 for fp16 and int8. These are different export recipes,
not only different bit widths.

The same int4 package preserves 99.27% top-1 on CPU but falls to 49.92% when CPU and
Neural Engine are allowed. That is an execution-dependent discrepancy, not evidence
that these rounded weights cannot represent the task. Int8's rounded top-1 matches
fp16 under `CPU_AND_NE`, but it changes 13 choices versus PyTorch; it is not lossless
or prediction-identical to fp16.

A follow-up on M1 Max, macOS 26.5.1, used 512 deterministic samples spread across the
test set. Int4 scored 511/512 on CPU and 266/512 with `CPU_AND_NE`. Decompressing its
palettes to dense constants retained exactly those choices under each setting. This
removes the palette-expansion operations but keeps the rounded weights and iOS 18
target. It also runs conversion passes, so it is not proof of an otherwise identical graph.
`ANECCompile() FAILED` messages were observed, but the buffered output could not be
attributed to individual calls. Neither these messages nor the predictions establish
operation placement, fallback or a faulty operation.

See the [public investigation](https://github.com/precisit/one-pass-specialists/blob/9240818b2f9221ecce9e27af11eab9816dfc51f9/examples/sv-forms/COREML-EXECUTION.md) for pinned data, scripts, artifact hashes
and the remaining target-matched export and graph-inspection work. The chip and OS
both differ between the M4 and M1 Max tests; their effects are not isolated.

The original M4 latency run excluded three warm-up rows, leaving 29,836 predictions.
Int8 with `CPU_AND_NE` had median 1.314 ms and p95 1.411 ms; fp16 on CPU had median
1.727 ms. These time `MLModel.predict`, not preprocessing, model loading or a full form.

## Usage

```python
from pathlib import Path
from huggingface_hub import hf_hub_download

weights = Path(hf_hub_download("precisit/one-pass-sv-forms", "sv0-forms.safetensors"))
hf_hub_download("precisit/one-pass-sv-forms", "sv0-forms.json", local_dir=weights.parent)
# then either use the Core ML packages (no Python runtime needed, see example.py) or the PyTorch
# code and the full recipe in the toolkit repository:
# https://github.com/precisit/one-pass-specialists
```

The context string and options must be built exactly as in training (byte ids = UTF-8 byte + 1,
zero padded; `UPPGIFT fyll i formuläret från dokumentet och skicka sedan in` / `FORM <titel>` /
`ELEMENT <roll> "<etikett>" value="…"`). An encoding mismatch is one possible source of poor output; vocabulary, truncation,
fixture labels and runtime behavior also need checking. The generator, catalogue and
scoring harness are in the toolkit repository.

## Building your own

[`precisit/one-pass-specialists`](https://github.com/precisit/one-pass-specialists) is the public
toolkit that produced this checkpoint: catalogue, synthetic generator with form-disjoint splits,
trainer, evaluation (including the shuffled-context control and the silent-skip count), Core ML
export with a 512-example check of the export-friendly eager implementation against
its trace, followed by Core ML decision comparisons, and the SV0 recipe. The trace
check does not establish bit identity with the original unmodified forward pass.
The synthetic corpus itself is published as
[`precisit/one-pass-sv-forms-synthetic`](https://huggingface.co/datasets/precisit/one-pass-sv-forms-synthetic).

## Licence and attribution

MIT. The architecture, training loop and evaluation metrics are vendored unmodified from Cua's
MIT-licensed [`libs/cua-s1`](https://github.com/trycua/cua/tree/main/libs/cua-s1) (which credits
`jevlike` for the option-attention head design). The Swedish catalogue, synthetic generator,
training runs, measurements and Core ML tooling are Precisit's. **No TypeSafe AI code, weights,
data or API output was used**: "Jev" and "System One" are their names for a similar interface, and
this is an independent implementation. See `THIRD_PARTY_NOTICES.md`.