--- language: en license: apache-2.0 library_name: transformers pipeline_tag: token-classification base_model: microsoft/deberta-v3-base tags: - token-classification - ner - phi - pii - privacy - healthcare - deidentification - security - compliance - synthetic-data - deberta-v3 widget: - text: "Patient John Smith (MRN: 001-23-4567) visited Boston Medical Center on 12/19/2025." - text: "Discharge summary reviewed by Dr. Emily Chen at Riverfront Clinic on 03/07/2025." model-index: - name: phi-span-detector-deberta-v3 results: - task: type: token-classification name: Token Classification dataset: name: Synthetic PHI span test set type: synthetic metrics: - name: Micro F1 type: f1 value: 0.6523 - name: Micro Precision type: precision value: 0.6657 - name: Micro Recall type: recall value: 0.6394 - name: Macro F1 type: f1 value: 0.6362 --- # PHI Span Detector (BIO NER) - Synthetic `phi-span-detector-deberta-v3` is a DeBERTa v3 token-classification model for detecting Protected Health Information (PHI) spans in clinical-note-like text and log-like text using BIO tagging. It is designed for privacy tooling workflows such as: - deterministic redaction pipelines - pre-log and post-log PHI guardrails - research prototypes for de-identification Recommended pipeline: 1. detect PHI spans 2. apply deterministic redaction 3. run a secondary leak-check gate before downstream use Companion model: [`bharathjanumpally/phi-leak-checker-deberta-v3`](https://huggingface.co/bharathjanumpally/phi-leak-checker-deberta-v3) ## Model at a glance - Task: token classification - Architecture: `DebertaV2ForTokenClassification` - Base model: `microsoft/deberta-v3-base` - Max sequence length: 512 - Labeling scheme: BIO - Training data: synthetic text only ## PHI label set The model predicts the following entity families: | Label | Meaning | | --- | --- | | `NAME` | patient or person names | | `DATE` | visit dates, birth dates, service dates | | `AGE` | age mentions that may be identifying in context | | `PHONE` | phone and callback numbers | | `EMAIL` | email addresses | | `ADDRESS` | street or mailing addresses | | `ID` | MRN, account, encounter, record, or similar identifiers | | `PROVIDER` | clinician or provider names | | `FACILITY` | hospitals, clinics, centers, departments | | `LOCATION` | city, state, and other place references | Token-level outputs use BIO labels from the model config: `O`, `B-*`, and `I-*` across the ten PHI families above. ## How the training data was built This model was trained on synthetic examples to keep the project openly shareable. High-level training recipe: 1. Generate synthetic clinical notes and log-like text with templates. 2. Insert PHI-like fields such as names, dates, IDs, facilities, phone numbers, and addresses. 3. Convert gold character spans into BIO token labels for token classification. This provides clean supervision without exposing real patient data, but it also means real-world formatting and writing styles may differ from training-time distributions. ## Evaluation The repository includes a full [`seqeval_report.txt`](./seqeval_report.txt). Key held-out results from that report are summarized below. ### Overall metrics | Metric | Value | | --- | ---: | | Micro precision | 0.6657 | | Micro recall | 0.6394 | | Micro F1 | 0.6523 | | Macro precision | 0.6583 | | Macro recall | 0.6224 | | Macro F1 | 0.6362 | | Weighted F1 | 0.6495 | ### Per-label metrics | Label | Precision | Recall | F1 | Support | | --- | ---: | ---: | ---: | ---: | | ADDRESS | 0.6652 | 0.6481 | 0.6565 | 233 | | AGE | 0.6758 | 0.3834 | 0.4893 | 386 | | DATE | 0.6553 | 0.6492 | 0.6522 | 1297 | | EMAIL | 0.6474 | 0.6455 | 0.6465 | 347 | | FACILITY | 0.6320 | 0.6494 | 0.6406 | 656 | | ID | 0.6652 | 0.6519 | 0.6585 | 451 | | LOCATION | 0.6600 | 0.6600 | 0.6600 | 350 | | NAME | 0.7810 | 0.7802 | 0.7806 | 1001 | | PHONE | 0.5358 | 0.5025 | 0.5186 | 595 | | PROVIDER | 0.6652 | 0.6537 | 0.6594 | 231 | ### Interpretation - Strongest label in the current report: `NAME` - Weakest labels in the current report: `PHONE` and `AGE` - The model is usable as a PHI span detector for research and tooling, but it should be paired with deterministic rules and internal evaluation before higher-stakes deployment ## Intended use Appropriate uses: - PHI span detection in research prototypes - de-identification pipelines when paired with deterministic redaction - zero-trust logging guardrails - preprocessing before a secondary PHI leak checker Not intended for: - medical diagnosis or treatment advice - sole control for HIPAA, GDPR, or other compliance decisions - unsupervised high-stakes production usage without internal validation ## Limitations and failure modes - The model was trained on synthetic text, so real clinical documentation may include unseen abbreviations, formatting quirks, shorthand, OCR noise, and edge cases. - Numeric strings may be over-flagged when they resemble IDs, dates, or phone numbers. - Some rare PHI patterns may be missed if they were not well represented in the synthetic templates. - Partial tokens and tokenizer boundary effects can require careful post-processing in downstream systems. - Label performance is uneven; current metrics suggest extra caution around `PHONE` and `AGE`. Recommended mitigations: - add regex backstops for structured entities like email, phone, and date - apply deterministic placeholder redaction after detection - run a second PHI leak-check model before downstream release - evaluate on an internal, policy-approved test set that matches your real document style - keep a human-review path for ambiguous or high-risk content ## Usage ### Transformers pipeline ```python from transformers import pipeline ner = pipeline( "token-classification", model="bharathjanumpally/phi-span-detector-deberta-v3", aggregation_strategy="simple", ) text = ( "Patient John Smith (MRN: 001-23-4567) visited " "Boston Medical Center on 12/19/2025." ) print(ner(text)) ``` ### AutoModel and AutoTokenizer ```python from transformers import AutoModelForTokenClassification, AutoTokenizer, pipeline model_id = "bharathjanumpally/phi-span-detector-deberta-v3" tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForTokenClassification.from_pretrained(model_id) ner = pipeline( "token-classification", model=model, tokenizer=tokenizer, aggregation_strategy="simple", ) print(ner("Call Jane Doe at 617-555-0182 before 04/14/2025.")) ``` ### Deterministic redaction example ```python from transformers import pipeline ner = pipeline( "token-classification", model="bharathjanumpally/phi-span-detector-deberta-v3", aggregation_strategy="simple", ) text = ( "Patient John Smith (MRN: 001-23-4567) visited " "Boston Medical Center on 12/19/2025." ) spans = ner(text) redacted = text for item in sorted(spans, key=lambda x: x["start"], reverse=True): label = item["entity_group"] redacted = redacted[: item["start"]] + f"[{label}]" + redacted[item["end"] :] print(spans) print(redacted) ``` ### Example output schema For downstream systems, a practical span schema is: ```json [ {"start": 8, "end": 18, "label": "NAME", "score": 0.97}, {"start": 25, "end": 36, "label": "ID", "score": 0.94}, {"start": 45, "end": 66, "label": "FACILITY", "score": 0.91}, {"start": 70, "end": 80, "label": "DATE", "score": 0.89} ] ``` ## Safety and privacy This model was trained on synthetic data and is published for research and tooling purposes. Do not send real PHI to public demos or public inference endpoints. Use private infrastructure, access controls, and organization-approved evaluation workflows for real deployments. ## Citation ```bibtex @misc{janumpally_phi_span_detector_2025, title = {PHI Span Detector (Synthetic)}, author = {Bharath Kumar Reddy Janumpally}, year = {2025}, publisher = {Hugging Face}, howpublished = {Model on Hugging Face} } ``` ## Contact If you use this model in a serious workflow, validate it against your own internal test cases and document the operating policy around false positives, false negatives, and escalation paths.