` per text block, plus `
`, ``, ``; footnote content omitted. One `\n` between blocks. The model never generates text; it emits per-token BIO labels (12 classes × begin/inside + outside = 25 label ids). - **Training data:** ~4,400 pages of scanned U.S. reporter volumes (16 volumes across 8 reporter series), OCR'd through a multi-engine pipeline; labels human-annotated (a ~680-page golden set) and model-seeded + human-reviewed (a ~3,900-page extension). Augmentations: random block-boundary drops (OCR segmentation robustness) and styling dropout (``/``-stripped copies). - **Training:** warm-started from a golden-set fine-tune; lr 3e-5, effective batch 16, bf16, best epoch by span-F1 on a mixed validation set, early stopping. ## Evaluation Span-level F1: exact (start, end, label) sets, gold and predictions normalized identically (whitespace/markup edge anchoring; the "normalized" column additionally ignores edge punctuation). No prediction post-processing. All evaluation volumes are unseen in training. | eval set | span-F1 strict | normalized | |---|---|---| | validation (3 volumes, human-reviewed gold) | 0.929 | 0.935 | | test (4 volumes, human-reviewed gold) | 0.937 | 0.950 | Generalization detail: on the validation volume from a reporter series entirely absent from training (Pacific 3d), span-F1 is 0.934. On the test set, the in-sample-reporter volume scores 0.979; the fully out-of-sample reporter series (South Eastern 2d, Bankruptcy Reporter) score 0.872–0.903. Per-class on validation (strict): party 0.990, attorneys 0.983, datefiled 0.981, author 0.968, court 0.963, separator 0.994, docketnumber 0.949, heading 0.938, judges 0.865, otherdate 0.842, history 0.815, disposition 0.801. **Caption classes** (the six fields composing a case caption — party, separator, docketnumber, court, datefiled, otherdate — typically the primary extraction targets), aggregated as micro-F1 over the six: | eval set | strict | normalized | gold spans | |---|---|---|---:| | validation | 0.972 | 0.973 | 627 | | test | 0.979 | 0.986 | 2,005 | Per class (strict, val / test): party 0.990 / 0.977 · separator 0.994 / 0.967 · docketnumber 0.949 / 0.968 · court 0.963 / 0.997 · datefiled 0.981 / 0.998 · otherdate 0.842 / 0.750 (rare secondary argued/decided dates; 12–28 gold spans per set). ## Usage ```python from transformers import (AutoModelForTokenClassification, AutoTokenizer) repo = "freelawproject/caselaw-block-tagger" # e.g. freelawproject/caselaw-block-tagger tok = AutoTokenizer.from_pretrained(repo) model = AutoModelForTokenClassification.from_pretrained(repo) text = ( "Jane ROE, Appellant,
\n" "v.
\n" "STATE of Example, Appellee.
\n" "No. 24-123
\n" "Court of Appeals of Example.
\n" "January 1, 2026
\n" "Affirmed.
" ) enc = tok(text, return_tensors="pt", return_offsets_mapping=True) logits = model(**{k: v for k, v in enc.items() if k != "offset_mapping"}).logits labels = [model.config.id2label[i] for i in logits.argmax(-1)[0].tolist()] ``` Decode BIO runs over the offset mapping to recover character spans. For best results segment the input at case boundaries (one case per sequence, ≤8k tokens) and strip footnote content — the model was trained that way. ## Limitations - Trained on U.S. reporter typography; other layout traditions (official state reporters, foreign case law) are untested. - `disposition` is the hardest class (two-tier convention: a final short ruling plus in-text holdings; 0.77 on test), and `judges` degrades on unfamiliar concur-line formats (0.58 on out-of-sample reporters); `otherdate` has thin support. - Citations are deliberately NOT a class — they are handled by a separate model. - Input must follow the canonical serialization above; markup tokens are label-masked during training, so unexpected markup degrades output.