HerBERT Polish legal NER (general): weights + ONNX + card + examples

.gitattributes

@@ -1,35 +1,3 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
 *.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,210 @@
+---
+language:
+- pl
+license: cc-by-4.0
+base_model: allegro/herbert-base-cased
+base_model_relation: finetune
+pipeline_tag: token-classification
+library_name: transformers
+metrics:
+- f1
+tags:
+- token-classification
+- named-entity-recognition
+- ner
+- pii
+- pii-detection
+- anonymization
+- privacy
+- gdpr
+- polish
+- legal
+- legal-nlp
+- herbert
+- bert
+- onnx
+model-index:
+- name: herbert-polish-legal-ner
+  results:
+  - task:
+      type: token-classification
+      name: Named Entity Recognition (PII)
+    dataset:
+      type: internal-polish-legal-eval
+      name: Internal Polish legal documents (identity-level eval)
+    metrics:
+    - type: f1
+      value: 0.94
+      name: Token-level F1 (held-out test)
+widget:
+- text: "Pozwany Jan Kowalski, zam. ul. Słoneczna 5 w Krakowie, PESEL 02070803628."
+---
+
+# HerBERT Polish Legal NER — PII / Anonymization
+
+A Polish token-classification (NER) model for **detecting personally identifiable
+information (PII) in legal and administrative text**, fine-tuned from
+[`allegro/herbert-base-cased`](https://huggingface.co/allegro/herbert-base-cased).
+
+This is the **general-purpose** variant — the best overall accuracy on **clean,
+digital** Polish legal documents.
+
+> For **scanned / photographed (OCR'd)** documents, use the OCR-robust sibling
+> **[`lexedit/herbert-polish-legal-ner-ocr`](https://huggingface.co/lexedit/herbert-polish-legal-ner-ocr)**,
+> which cuts the person-name leak on scanned text by ~35–41 % (at a small precision
+> cost on clean text).
+
+> **Intended use is defensive:** flagging PII so it can be masked / anonymised
+> before a document is shared or processed. It is **not** a guarantee of complete
+> anonymisation — see *Limitations*.
+
+**▶ Try it in your browser:** [lexedit.ai/lexedit-privacy](https://lexedit.ai/lexedit-privacy)
+— an interactive, **fully client-side** Polish legal-document anonymisation demo
+(the text never leaves your browser).
+
+**📂 Example cases:** [`examples/STRONG_CASES.md`](examples/STRONG_CASES.md) (hard
+inputs it handles) · [`examples/KNOWN_LIMITATIONS.md`](examples/KNOWN_LIMITATIONS.md)
+(where it still fails).
+
+## Labels (29, BIO scheme)
+
+`PER` (person) · `ORG` (organisation) · `LOC` (private address/location) ·
+`LOC_PUB` (public place: city, country) · `DATE` · `MONEY` · `EMAIL` · `PHONE` ·
+`ID` (national id / case / document number) · `IBAN` · `DIAGNOSIS` ·
+`HEALTH_FACILITY` · `MEDICAL_ID` · `WATERMARK`, each as `B-…` / `I-…`, plus `O`.
+
+The model distinguishes **private** locations (`LOC`, masked) from **public**
+ones (`LOC_PUB`, usually kept), and treats `DATE` / `MONEY` as non-anonymised by
+default.
+
+## Usage (ONNX, no PyTorch required)
+
+```bash
+pip install onnxruntime transformers numpy
+python examples/inference_onnx.py
+```
+
+```python
+import numpy as np, onnxruntime as ort
+from transformers import AutoTokenizer
+
+tok = AutoTokenizer.from_pretrained("lexedit/herbert-polish-legal-ner")
+sess = ort.InferenceSession("onnx/model_quantized.onnx")
+enc = tok("Pozwany Jan Kowalski, PESEL 02070803628.",
+          return_offsets_mapping=True, return_tensors="np")
+feeds = {"input_ids": enc["input_ids"].astype(np.int64),
+         "attention_mask": enc["attention_mask"].astype(np.int64)}
+logits = sess.run(None, feeds)[0]          # (1, seq, 29)
+# argmax per token -> map ids via config.id2label -> group B-/I- with offset_mapping
+```
+
+## Usage (PyTorch / transformers pipeline)
+
+```python
+from transformers import pipeline
+ner = pipeline("token-classification",
+               model="lexedit/herbert-polish-legal-ner",
+               aggregation_strategy="first")
+ner("Pozwany Jan Kowalski, zam. ul. Słoneczna 5 w Krakowie.")
+```
+
+## Runs in the browser (the intended setup)
+
+This model is designed to run **entirely client-side**. The quantized ONNX
+(~125 MB, int8) loads in the browser via
+[onnxruntime-web](https://onnxruntime.ai/docs/tutorials/web/) or
+[transformers.js](https://huggingface.co/docs/transformers.js) (WASM), so **the
+document never leaves the user's device** — which is the whole point for sensitive
+legal / medical text. It also runs anywhere ONNX Runtime does (Python, Node,
+server, mobile). The demo at
+[lexedit.ai/lexedit-privacy](https://lexedit.ai/lexedit-privacy) is exactly this.
+
+### Speed (rough)
+
+Quantized ONNX, measured on a laptop CPU (Apple Silicon):
+
+| input | 1 core | all cores |
+|---|---|---|
+| short sentence (~30 tokens) | ~18 ms | ~13 ms |
+| full chunk (~500 tokens) | ~0.2 s | ~0.1 s |
+
+≈ **4–5 chunks/second single-threaded** natively. In the browser (WASM,
+single-threaded) it is slower but practical: short text stays interactive, a 1–2
+page document takes a few seconds, a large scanned document can take ~a minute or
+two; cross-origin isolation (COOP/COEP → multi-threaded WASM) speeds it up.
+
+## Recommended production setup
+
+This model is **recall-first** and is one layer of a pipeline, not the whole
+solution. For best anonymisation, pair it with:
+
+1. **A recall-first threshold** for `PER` (flip a token to PER when the summed PER
+   probability ≥ ~0.2, even if it is not the arg-max).
+2. **A deterministic document-safety post-pass** — snap spans to whole words, merge
+   hyphenated surnames, and propagate a detected surname to its other inflected /
+   OCR-variant mentions across the document.
+3. **Checksum-validated regex** for structured PII (PESEL, NIP, REGON, IBAN, …).
+4. **Human review** for high-stakes use.
+
+## Evaluation
+
+Identity-level **leak rate** = a person is "leaked" if *any* mention of them is
+missed. Internal set of 50 real Polish legal documents (179 persons), recall-first
+threshold 0.2.
+
+| Document type | metric | this model | OCR-robust variant |
+|---|---|---|---|
+| **Clean / digital** | leak (threshold) | **10.1%** | 11.7% |
+| **Clean / digital** | leak (+ post-pass) | **7.3%** | 7.3% |
+| Scanned / OCR'd | leak (threshold) | 31.8% | **20.7%** |
+| Scanned / OCR'd | leak (+ post-pass) | 25.7% | **15.1%** |
+
+Token-level test F1 ≈ **0.94**.
+
+**Takeaway:** this is the strongest variant on **clean digital** text (lowest leak,
+best precision). On **scanned / OCR'd** text it is weaker — there the
+[OCR-robust variant](https://huggingface.co/lexedit/herbert-polish-legal-ner-ocr)
+wins. If you process both, **route by document type**.
+
+## Training data
+
+Fine-tuned on Polish legal and administrative documents — a mix of document
+templates, programmatically-generated labelled examples (valid checksum-correct
+synthetic identifiers, rule-based Polish name declension), and real-world
+legal-document samples.
+
+No raw personal data is distributed with this model. Because this is a
+**token-classification** model (it outputs a label per input token and cannot
+generate text), the weights do not reproduce or expose any training document.
+
+## Limitations
+
+> **Concrete, reproducible failure cases:** see
+> [`examples/KNOWN_LIMITATIONS.md`](examples/KNOWN_LIMITATIONS.md) (synthetic inputs).
+> Heavily OCR-garbled names are the main weakness — for scanned documents prefer the
+> [OCR-robust variant](https://huggingface.co/lexedit/herbert-polish-legal-ner-ocr).
+
+- **Polish only.**
+- **Not a guarantee.** A residual leak rate remains (≈7–10 % identity-level on clean
+  text); always combine with the deterministic post-pass + checksum regex and human
+  review for high-stakes use.
+- **Scanned / OCR'd text** is the weak spot of this variant (heavily garbled names
+  can be missed) — route those to the OCR-robust variant.
+- Small evaluation set; numbers are indicative, not a benchmark.
+- Not legal advice; not a substitute for a privacy/compliance review.
+
+## License — CC BY 4.0 (attribution required, commercial use allowed)
+
+Released under **[Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/)**.
+You may use, modify and redistribute this model — **including in commercial
+products** — **provided you give appropriate credit**. Attribution is required in
+any use, commercial or not; no other restrictions are added.
+
+Suggested attribution:
+
+> Polish legal NER / anonymisation model by **lexedit** (https://lexedit.ai),
+> licensed CC BY 4.0, fine-tuned from HerBERT
+> ([`allegro/herbert-base-cased`](https://huggingface.co/allegro/herbert-base-cased)).
+
+This model is a derivative of HerBERT (Allegro), which is itself CC BY 4.0 — please
+retain attribution to the base model as well.

config.json

@@ -0,0 +1,88 @@
+{
+  "architectures": [
+    "BertForTokenClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "B-DATE",
+    "1": "B-DIAGNOSIS",
+    "2": "B-EMAIL",
+    "3": "B-HEALTH_FACILITY",
+    "4": "B-IBAN",
+    "5": "B-ID",
+    "6": "B-LOC",
+    "7": "B-LOC_PUB",
+    "8": "B-MEDICAL_ID",
+    "9": "B-MONEY",
+    "10": "B-ORG",
+    "11": "B-PER",
+    "12": "B-PHONE",
+    "13": "B-WATERMARK",
+    "14": "I-DATE",
+    "15": "I-DIAGNOSIS",
+    "16": "I-EMAIL",
+    "17": "I-HEALTH_FACILITY",
+    "18": "I-IBAN",
+    "19": "I-ID",
+    "20": "I-LOC",
+    "21": "I-LOC_PUB",
+    "22": "I-MEDICAL_ID",
+    "23": "I-MONEY",
+    "24": "I-ORG",
+    "25": "I-PER",
+    "26": "I-PHONE",
+    "27": "I-WATERMARK",
+    "28": "O"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "B-DATE": 0,
+    "B-DIAGNOSIS": 1,
+    "B-EMAIL": 2,
+    "B-HEALTH_FACILITY": 3,
+    "B-IBAN": 4,
+    "B-ID": 5,
+    "B-LOC": 6,
+    "B-LOC_PUB": 7,
+    "B-MEDICAL_ID": 8,
+    "B-MONEY": 9,
+    "B-ORG": 10,
+    "B-PER": 11,
+    "B-PHONE": 12,
+    "B-WATERMARK": 13,
+    "I-DATE": 14,
+    "I-DIAGNOSIS": 15,
+    "I-EMAIL": 16,
+    "I-HEALTH_FACILITY": 17,
+    "I-IBAN": 18,
+    "I-ID": 19,
+    "I-LOC": 20,
+    "I-LOC_PUB": 21,
+    "I-MEDICAL_ID": 22,
+    "I-MONEY": 23,
+    "I-ORG": 24,
+    "I-PER": 25,
+    "I-PHONE": 26,
+    "I-WATERMARK": 27,
+    "O": 28
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 514,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "tokenizer_class": "HerbertTokenizerFast",
+  "transformers_version": "4.57.6",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 50000
+}

examples/KNOWN_LIMITATIONS.md

@@ -0,0 +1,48 @@
+# Known limitations — real failure cases
+
+This general-purpose variant is strongest on **clean digital** text. Its main
+weakness is **heavily OCR-garbled names** (scanned / photographed documents).
+
+> 👉 For scanned / OCR'd documents, use the OCR-robust sibling
+> **[`lexedit/herbert-polish-legal-ner-ocr`](https://huggingface.co/lexedit/herbert-polish-legal-ner-ocr)**
+> — it catches several of the cases below that this model misses.
+
+All inputs are **synthetic** — fictional names, **no real or evaluation data**.
+Outputs are the model's **actual** predictions (quantized ONNX, recall-first PER
+threshold 0.2). Reproduce with [`known_limitations.py`](known_limitations.py).
+
+## Heavy OCR garble → name missed entirely
+
+When OCR destroys most of a name's recognizable structure, this variant can fail to
+flag it at all:
+
+| Input | Behind the garble | Model `PER` |
+|---|---|---|
+| `Pozwana ote tobodaaka wniosła sprzeciw.` | *Bożena Łobodzińska* | — (nothing) |
+| `Powód go Hea stawił się osobiście.` | *Igor Heliasz* | — (nothing) |
+| `Decyzję doręczono: ist yi daziak.` | *Justyna Idziak* | — (nothing) |
+| `Wniosek złożył wiar Żoją w dniu 5 maja.` | *Wiktor Żołądek* | — (nothing) |
+
+*(The OCR-robust variant catches part of the last one — `['Żoją']` — instead of
+missing it entirely.)*
+
+## Partial detection → residue leak
+
+| Input | Behind the garble | Model `PER` | Problem |
+|---|---|---|---|
+| `WIESEAW KUE stawił się na rozprawie.` | *Wiesław Kuc* | `['WIE', 'AW KUE']` | split into two spans → the gap (`SE`) leaks |
+
+## Other known behaviours
+
+- **Foreign / out-of-distribution names** (e.g. German, Dutch) are less reliable —
+  the model is tuned for Polish.
+- **Boundaries** can occasionally over- or under-extend.
+
+## Mitigations (recommended)
+
+1. **Use the OCR-robust variant for scanned documents** (route by document type).
+2. **Document-safety post-pass** — propagate masking from any clean mention of a
+   surname to its other inflected / OCR-variant mentions in the same document.
+3. **Checksum regex** for structured PII (PESEL, NIP, IBAN, …).
+4. **Lower the threshold on noisy/scanned documents.**
+5. **Human review** for high-stakes anonymisation.

examples/STRONG_CASES.md

@@ -0,0 +1,52 @@
+# Where the model shines — hard cases handled well
+
+Difficult inputs this general-purpose model gets **right**. All inputs are
+synthetic (fictional names); outputs are the model's **actual** predictions
+(quantized ONNX, recall-first PER threshold 0.2). Reproduce with
+[`strong_cases.py`](strong_cases.py).
+
+## Polish inflection (declension)
+
+Polish surnames change form by grammatical case; the model tags the full inflected
+surface, not just the nominative:
+
+| Input | Case | Model |
+|---|---|---|
+| `Pozew skierowano przeciwko Jerzemu Sekule.` | dative | `PER 'Jerzemu Sekule'` |
+| `Sprawa dotyczy akt Wojciecha Michalika.` | genitive | `PER 'Wojciecha Michalika'` |
+
+## Tricky surface forms
+
+| Input | Why it's hard | Model |
+|---|---|---|
+| `Powódka Anna Nowak-Kowalska złożyła wniosek.` | hyphenated double surname → one span | `PER 'Anna Nowak-Kowalska'` |
+| `Stawiła się ALEKSANDRA ŚWIDERSKA, pełnomocnik.` | ALL-CAPS header/signature style | `PER 'ALEKSANDRA ŚWIDERSKA'` |
+| `Z poważaniem,` / `Iga Mełech` / `radca prawny` | short female first + rare surname in a low-cue signature | `PER 'Iga Mełech'` |
+
+## Mild / moderate OCR noise
+
+Light scan corruption (missing diacritics, a few confusable characters) is still
+handled:
+
+| Input | Behind the garble | Model |
+|---|---|---|
+| `Pozwany Damian Zotadek wniósł odpowiedź na pozew.` | *Damian Żołądek* (lost diacritics) | `PER 'Damian Zotadek'` |
+| `Z powództwa Jiga Bękowiki przeciwko spółce.` | *Jaga Bętkowski* | `PER 'Jiga Bękowiki'` |
+| `Pełnomocnik: Ptek Gostoiski, adwokat.` | *Patryk Gostomski* | `PER 'Ptek Gostoiski'` |
+
+> **Heavy** OCR garble is this variant's weak spot — see
+> [`KNOWN_LIMITATIONS.md`](KNOWN_LIMITATIONS.md). For scanned documents prefer the
+> [OCR-robust variant](https://huggingface.co/lexedit/herbert-polish-legal-ner-ocr).
+
+## Multiple types + public/private distinction
+
+```
+Jan Kowalski, zam. ul. Słoneczna 5 w Krakowie, PESEL 02070803628.
+   PER      'Jan Kowalski'
+   LOC      'ul. Słoneczna 5'      ← private address (masked)
+   LOC_PUB  'Krakowie'             ← public city (kept by default)
+   ID       '02070803628'          ← national id (PESEL)
+```
+
+The model keeps the public place (`Krakowie`) visible while masking the private
+street address — so the legal context survives anonymisation.

examples/inference_onnx.py

@@ -0,0 +1,76 @@
+#!/usr/bin/env python3
+"""Self-contained ONNX inference for the HerBERT Polish legal NER model.
+No PyTorch needed.  pip install onnxruntime transformers numpy
+
+Run from the repo root:  python examples/inference_onnx.py
+"""
+import json
+from pathlib import Path
+
+import numpy as np
+import onnxruntime as ort
+from transformers import AutoTokenizer
+
+ROOT = Path(__file__).resolve().parent.parent
+PER_THRESHOLD = 0.2  # recall-first: flip a token to PER if summed PER prob >= this
+
+tok = AutoTokenizer.from_pretrained(str(ROOT))
+cfg = json.load(open(ROOT / "config.json", encoding="utf-8"))
+id2label = {int(k): v for k, v in cfg["id2label"].items()}
+label2id = cfg["label2id"]
+sess = ort.InferenceSession(str(ROOT / "onnx" / "model_quantized.onnx"))
+in_names = {i.name for i in sess.get_inputs()}
+
+
+def softmax(x):
+    e = np.exp(x - x.max(-1, keepdims=True))
+    return e / e.sum(-1, keepdims=True)
+
+
+def predict(text):
+    enc = tok(text, return_offsets_mapping=True, return_tensors="np",
+              truncation=True, max_length=512)
+    offsets = enc["offset_mapping"][0]
+    feeds = {"input_ids": enc["input_ids"].astype(np.int64),
+             "attention_mask": enc["attention_mask"].astype(np.int64)}
+    if "token_type_ids" in in_names:
+        feeds["token_type_ids"] = np.zeros_like(enc["input_ids"], dtype=np.int64)
+    probs = softmax(sess.run(None, feeds)[0][0])      # (seq, num_labels)
+    ids = probs.argmax(-1)
+    per_b, per_i = label2id["B-PER"], label2id["I-PER"]
+
+    spans, cur = [], None  # cur = [start, end, type]
+    for i, (s, e) in enumerate(offsets):
+        if s == e:                                    # special token
+            if cur:
+                spans.append(cur); cur = None
+            continue
+        label = id2label[int(ids[i])]
+        # recall-first override for persons
+        if label == "O" and probs[i, per_b] + probs[i, per_i] >= PER_THRESHOLD:
+            label = "I-PER" if (cur and cur[2] == "PER") else "B-PER"
+        if label == "O":
+            if cur:
+                spans.append(cur); cur = None
+            continue
+        tag, etype = label.split("-", 1)
+        if tag == "B" or cur is None or cur[2] != etype:
+            if cur:
+                spans.append(cur)
+            cur = [int(s), int(e), etype]
+        else:
+            cur[1] = int(e)
+    if cur:
+        spans.append(cur)
+    return [{"type": t, "start": a, "end": b, "text": text[a:b]} for a, b, t in spans]
+
+
+if __name__ == "__main__":
+    samples = [
+        "Pozwany Jan Kowalski, zam. ul. Słoneczna 5 w Krakowie, PESEL 02070803628.",
+        "Powódka Anna Nowak-Kowalska, e-mail a.nowak@example.pl, tel. 501 234 567.",
+    ]
+    for t in samples:
+        print("\n" + t)
+        for ent in predict(t):
+            print(f"   {ent['type']:8} [{ent['start']:>3}:{ent['end']:<3}] {ent['text']!r}")

examples/inference_pytorch.py

@@ -0,0 +1,32 @@
+#!/usr/bin/env python3
+"""PyTorch / transformers inference for the HerBERT Polish legal NER model.
+pip install torch transformers
+
+Run from the repo root:  python examples/inference_pytorch.py
+"""
+from pathlib import Path
+
+from transformers import pipeline
+
+ROOT = Path(__file__).resolve().parent.parent
+
+ner = pipeline(
+    "token-classification",
+    model=str(ROOT),
+    tokenizer=str(ROOT),
+    aggregation_strategy="first",  # group B-/I- subwords into whole entities
+)
+
+SAMPLES = [
+    "Pozwany Jan Kowalski, zam. ul. Słoneczna 5 w Krakowie, PESEL 02070803628.",
+    "Powódka Anna Nowak-Kowalska, e-mail a.nowak@example.pl, tel. 501 234 567.",
+]
+
+if __name__ == "__main__":
+    for text in SAMPLES:
+        print("\n" + text)
+        for ent in ner(text):
+            print(f"   {ent['entity_group']:8} [{ent['start']:>3}:{ent['end']:<3}] "
+                  f"{ent['word']!r}  ({ent['score']:.2f})")
+    # NOTE: for anonymisation, prefer a recall-first PER threshold (see
+    # examples/inference_onnx.py) over the arg-max grouping the pipeline uses.

examples/known_limitations.py

@@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+"""Reproduce the model's KNOWN FAILURE CASES (see KNOWN_LIMITATIONS.md).
+
+All inputs are SYNTHETIC — fictional names, no real personal data. The point is
+to show, honestly and reproducibly, where the model still misses or mislabels
+PERSON names so you can decide where extra safeguards (post-pass, review) are
+needed.  pip install onnxruntime transformers numpy
+
+Run from the repo root:  python examples/known_limitations.py
+"""
+import json
+from pathlib import Path
+
+import numpy as np
+import onnxruntime as ort
+from transformers import AutoTokenizer
+
+ROOT = Path(__file__).resolve().parent.parent
+PER_THRESHOLD = 0.2
+
+tok = AutoTokenizer.from_pretrained(str(ROOT))
+cfg = json.load(open(ROOT / "config.json", encoding="utf-8"))
+id2label = {int(k): v for k, v in cfg["id2label"].items()}
+label2id = cfg["label2id"]
+sess = ort.InferenceSession(str(ROOT / "onnx" / "model_quantized.onnx"))
+in_names = {i.name for i in sess.get_inputs()}
+
+
+def softmax(x):
+    e = np.exp(x - x.max(-1, keepdims=True)); return e / e.sum(-1, keepdims=True)
+
+
+def predict_per(text):
+    enc = tok(text, return_offsets_mapping=True, return_tensors="np", truncation=True, max_length=512)
+    offs = enc["offset_mapping"][0]
+    feeds = {"input_ids": enc["input_ids"].astype(np.int64),
+             "attention_mask": enc["attention_mask"].astype(np.int64)}
+    if "token_type_ids" in in_names:
+        feeds["token_type_ids"] = np.zeros_like(enc["input_ids"], dtype=np.int64)
+    probs = softmax(sess.run(None, feeds)[0][0])
+    ids = probs.argmax(-1)
+    pb, pi = label2id["B-PER"], label2id["I-PER"]
+    spans, cur = [], None
+    for i, (s, e) in enumerate(offs):
+        if s == e:
+            if cur: spans.append(cur); cur = None
+            continue
+        lab = id2label[int(ids[i])]
+        if lab == "O" and probs[i, pb] + probs[i, pi] >= PER_THRESHOLD:
+            lab = "I-PER" if cur else "B-PER"
+        if not lab.endswith("PER"):
+            if cur: spans.append(cur); cur = None
+            continue
+        if lab == "B-PER" or cur is None:
+            if cur: spans.append(cur)
+            cur = [int(s), int(e)]
+        else:
+            cur[1] = int(e)
+    if cur: spans.append(cur)
+    return [text[a:b] for a, b in spans]
+
+
+# (input, fictional clean name behind the garble, failure mode)
+CASES = [
+    # 1) Heavy OCR garble -> missed entirely
+    ("Pozwana ote tobodaaka wniosła sprzeciw.", "Bożena Łobodzińska", "heavy OCR garble -> MISSED"),
+    ("Powód go Hea stawił się osobiście.", "Igor Heliasz", "heavy OCR garble -> MISSED"),
+    ("Decyzję doręczono: ist yi daziak.", "Justyna Idziak", "heavy OCR garble -> MISSED"),
+    # 2) Partial detection -> residue leak
+    ("Wniosek złożył wiar Żoją w dniu 5 maja.", "Wiktor Żołądek", "partial -> only part caught (residue)"),
+    ("WIESEAW KUE stawił się na rozprawie.", "Wiesław Kuc", "ALL-CAPS garble -> fragmented (gap leaks)"),
+]
+
+if __name__ == "__main__":
+    print(f"PER threshold = {PER_THRESHOLD}\n")
+    for text, clean, mode in CASES:
+        per = predict_per(text)
+        print(f"# {mode}   (fictional: {clean})")
+        print(f"  input : {text}")
+        print(f"  model : PER = {per if per else '— (nothing)'}")
+        print()

examples/strong_cases.py

@@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+"""Hard cases the model handles WELL — the flip side of KNOWN_LIMITATIONS.md.
+
+All inputs are synthetic (fictional names). Outputs are the model's actual
+predictions (quantized ONNX, recall-first PER threshold 0.2). Reproduce:
+    python examples/strong_cases.py
+"""
+import json
+from pathlib import Path
+
+import numpy as np
+import onnxruntime as ort
+from transformers import AutoTokenizer
+
+ROOT = Path(__file__).resolve().parent.parent
+PER_THRESHOLD = 0.2
+
+tok = AutoTokenizer.from_pretrained(str(ROOT))
+cfg = json.load(open(ROOT / "config.json", encoding="utf-8"))
+id2label = {int(k): v for k, v in cfg["id2label"].items()}
+label2id = cfg["label2id"]
+sess = ort.InferenceSession(str(ROOT / "onnx" / "model_quantized.onnx"))
+in_names = {i.name for i in sess.get_inputs()}
+
+
+def softmax(x):
+    e = np.exp(x - x.max(-1, keepdims=True)); return e / e.sum(-1, keepdims=True)
+
+
+def predict(text):
+    enc = tok(text, return_offsets_mapping=True, return_tensors="np", truncation=True, max_length=512)
+    offs = enc["offset_mapping"][0]
+    feeds = {"input_ids": enc["input_ids"].astype(np.int64),
+             "attention_mask": enc["attention_mask"].astype(np.int64)}
+    if "token_type_ids" in in_names:
+        feeds["token_type_ids"] = np.zeros_like(enc["input_ids"], dtype=np.int64)
+    probs = softmax(sess.run(None, feeds)[0][0])
+    ids = probs.argmax(-1)
+    pb, pi = label2id["B-PER"], label2id["I-PER"]
+    spans, cur = [], None
+    for i, (s, e) in enumerate(offs):
+        if s == e:
+            if cur: spans.append(cur); cur = None
+            continue
+        lab = id2label[int(ids[i])]
+        if lab == "O" and probs[i, pb] + probs[i, pi] >= PER_THRESHOLD:
+            lab = "I-PER" if (cur and cur[2] == "PER") else "B-PER"
+        if lab == "O":
+            if cur: spans.append(cur); cur = None
+            continue
+        tag, et = lab.split("-", 1)
+        if tag == "B" or cur is None or cur[2] != et:
+            if cur: spans.append(cur)
+            cur = [int(s), int(e), et]
+        else:
+            cur[1] = int(e)
+    if cur: spans.append(cur)
+    return [{"type": t, "start": a, "end": b, "text": text[a:b]} for a, b, t in spans]
+
+
+CASES = [
+    ("Pozwany Damian Zotadek wniósł odpowiedź na pozew.", "OCR: missing diacritics (Żołądek)"),
+    ("Z powództwa Jiga Bękowiki przeciwko spółce.", "OCR: heavily garbled name (Jaga Bętkowski)"),
+    ("Pełnomocnik: Ptek Gostoiski, adwokat.", "OCR garble in a signature line (Patryk Gostomski)"),
+    ("Pozew skierowano przeciwko Jerzemu Sekule.", "inflected (dative) Polish name"),
+    ("Sprawa dotyczy akt Wojciecha Michalika.", "inflected (genitive) Polish name"),
+    ("Powódka Anna Nowak-Kowalska złożyła wniosek.", "hyphenated double surname (single span)"),
+    ("Stawiła się ALEKSANDRA ŚWIDERSKA, pełnomocnik.", "ALL-CAPS name (header/signature style)"),
+    ("Z poważaniem,\nIga Mełech\nradca prawny", "short female first + rare surname in a signature"),
+    ("Jan Kowalski, zam. ul. Słoneczna 5 w Krakowie, PESEL 02070803628.",
+     "private address (LOC) vs public city (LOC_PUB) + national ID"),
+]
+
+if __name__ == "__main__":
+    print(f"PER threshold = {PER_THRESHOLD}\n")
+    for text, note in CASES:
+        print(f"# {note}")
+        print(f"  input : {text.replace(chr(10), ' / ')}")
+        for e in predict(text):
+            print(f"     {e['type']:8} {e['text']!r}")
+        print()

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f19a470d0590f1620950b01338544d5acd494c2827a12fb87d9a99a38d4fa7bf
+size 495521716

onnx/model_quantized.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ed91aabec29e07d2d5c7f370a5463283d4d88acbf44dd1ceedd0bca67330b50
+size 125082875

requirements.txt

@@ -0,0 +1,6 @@
+# ONNX inference (no PyTorch needed)
+transformers>=4.40
+onnxruntime>=1.17
+numpy
+# For the PyTorch / transformers-pipeline example instead, also:
+# torch>=2.0

special_tokens_map.json

@@ -0,0 +1,8 @@
+{
+  "bos_token": "<s>",
+  "cls_token": "<s>",
+  "mask_token": "<mask>",
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "unk_token": "<unk>"
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [],
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "<s>",
+  "do_lowercase_and_remove_accent": false,
+  "extra_special_tokens": {},
+  "id2lang": null,
+  "lang2id": null,
+  "mask_token": "<mask>",
+  "model_max_length": 512,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "tokenizer_class": "HerbertTokenizer",
+  "unk_token": "<unk>"
+}