README.md

---
language: en
license: apache-2.0
library_name: transformers
pipeline_tag: token-classification
base_model: microsoft/deberta-v3-base
tags:
  - token-classification
  - ner
  - phi
  - pii
  - privacy
  - healthcare
  - deidentification
  - security
  - compliance
  - synthetic-data
  - deberta-v3
widget:
  - text: "Patient John Smith (MRN: 001-23-4567) visited Boston Medical Center on 12/19/2025."
  - text: "Discharge summary reviewed by Dr. Emily Chen at Riverfront Clinic on 03/07/2025."
model-index:
  - name: phi-span-detector-deberta-v3
    results:
      - task:
          type: token-classification
          name: Token Classification
        dataset:
          name: Synthetic PHI span test set
          type: synthetic
        metrics:
          - name: Micro F1
            type: f1
            value: 0.6523
          - name: Micro Precision
            type: precision
            value: 0.6657
          - name: Micro Recall
            type: recall
            value: 0.6394
          - name: Macro F1
            type: f1
            value: 0.6362
---

# PHI Span Detector (BIO NER) - Synthetic

`phi-span-detector-deberta-v3` is a DeBERTa v3 token-classification model for detecting Protected Health Information (PHI) spans in clinical-note-like text and log-like text using BIO tagging.

It is designed for privacy tooling workflows such as:

- deterministic redaction pipelines
- pre-log and post-log PHI guardrails
- research prototypes for de-identification

Recommended pipeline:

1. detect PHI spans
2. apply deterministic redaction
3. run a secondary leak-check gate before downstream use

Companion model:
[`bharathjanumpally/phi-leak-checker-deberta-v3`](https://huggingface.co/bharathjanumpally/phi-leak-checker-deberta-v3)

## Model at a glance

- Task: token classification
- Architecture: `DebertaV2ForTokenClassification`
- Base model: `microsoft/deberta-v3-base`
- Max sequence length: 512
- Labeling scheme: BIO
- Training data: synthetic text only

## PHI label set

The model predicts the following entity families:

| Label | Meaning |
| --- | --- |
| `NAME` | patient or person names |
| `DATE` | visit dates, birth dates, service dates |
| `AGE` | age mentions that may be identifying in context |
| `PHONE` | phone and callback numbers |
| `EMAIL` | email addresses |
| `ADDRESS` | street or mailing addresses |
| `ID` | MRN, account, encounter, record, or similar identifiers |
| `PROVIDER` | clinician or provider names |
| `FACILITY` | hospitals, clinics, centers, departments |
| `LOCATION` | city, state, and other place references |

Token-level outputs use BIO labels from the model config:

`O`, `B-*`, and `I-*` across the ten PHI families above.

## How the training data was built

This model was trained on synthetic examples to keep the project openly shareable.

High-level training recipe:

1. Generate synthetic clinical notes and log-like text with templates.
2. Insert PHI-like fields such as names, dates, IDs, facilities, phone numbers, and addresses.
3. Convert gold character spans into BIO token labels for token classification.

This provides clean supervision without exposing real patient data, but it also means real-world formatting and writing styles may differ from training-time distributions.

## Evaluation

The repository includes a full [`seqeval_report.txt`](./seqeval_report.txt). Key held-out results from that report are summarized below.

### Overall metrics

| Metric | Value |
| --- | ---: |
| Micro precision | 0.6657 |
| Micro recall | 0.6394 |
| Micro F1 | 0.6523 |
| Macro precision | 0.6583 |
| Macro recall | 0.6224 |
| Macro F1 | 0.6362 |
| Weighted F1 | 0.6495 |

### Per-label metrics

| Label | Precision | Recall | F1 | Support |
| --- | ---: | ---: | ---: | ---: |
| ADDRESS | 0.6652 | 0.6481 | 0.6565 | 233 |
| AGE | 0.6758 | 0.3834 | 0.4893 | 386 |
| DATE | 0.6553 | 0.6492 | 0.6522 | 1297 |
| EMAIL | 0.6474 | 0.6455 | 0.6465 | 347 |
| FACILITY | 0.6320 | 0.6494 | 0.6406 | 656 |
| ID | 0.6652 | 0.6519 | 0.6585 | 451 |
| LOCATION | 0.6600 | 0.6600 | 0.6600 | 350 |
| NAME | 0.7810 | 0.7802 | 0.7806 | 1001 |
| PHONE | 0.5358 | 0.5025 | 0.5186 | 595 |
| PROVIDER | 0.6652 | 0.6537 | 0.6594 | 231 |

### Interpretation

- Strongest label in the current report: `NAME`
- Weakest labels in the current report: `PHONE` and `AGE`
- The model is usable as a PHI span detector for research and tooling, but it should be paired with deterministic rules and internal evaluation before higher-stakes deployment

## Intended use

Appropriate uses:

- PHI span detection in research prototypes
- de-identification pipelines when paired with deterministic redaction
- zero-trust logging guardrails
- preprocessing before a secondary PHI leak checker

Not intended for:

- medical diagnosis or treatment advice
- sole control for HIPAA, GDPR, or other compliance decisions
- unsupervised high-stakes production usage without internal validation

## Limitations and failure modes

- The model was trained on synthetic text, so real clinical documentation may include unseen abbreviations, formatting quirks, shorthand, OCR noise, and edge cases.
- Numeric strings may be over-flagged when they resemble IDs, dates, or phone numbers.
- Some rare PHI patterns may be missed if they were not well represented in the synthetic templates.
- Partial tokens and tokenizer boundary effects can require careful post-processing in downstream systems.
- Label performance is uneven; current metrics suggest extra caution around `PHONE` and `AGE`.

Recommended mitigations:

- add regex backstops for structured entities like email, phone, and date
- apply deterministic placeholder redaction after detection
- run a second PHI leak-check model before downstream release
- evaluate on an internal, policy-approved test set that matches your real document style
- keep a human-review path for ambiguous or high-risk content

## Usage

### Transformers pipeline

```python
from transformers import pipeline

ner = pipeline(
    "token-classification",
    model="bharathjanumpally/phi-span-detector-deberta-v3",
    aggregation_strategy="simple",
)

text = (
    "Patient John Smith (MRN: 001-23-4567) visited "
    "Boston Medical Center on 12/19/2025."
)

print(ner(text))
```

### AutoModel and AutoTokenizer

```python
from transformers import AutoModelForTokenClassification, AutoTokenizer, pipeline

model_id = "bharathjanumpally/phi-span-detector-deberta-v3"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForTokenClassification.from_pretrained(model_id)

ner = pipeline(
    "token-classification",
    model=model,
    tokenizer=tokenizer,
    aggregation_strategy="simple",
)

print(ner("Call Jane Doe at 617-555-0182 before 04/14/2025."))
```

### Deterministic redaction example

```python
from transformers import pipeline

ner = pipeline(
    "token-classification",
    model="bharathjanumpally/phi-span-detector-deberta-v3",
    aggregation_strategy="simple",
)

text = (
    "Patient John Smith (MRN: 001-23-4567) visited "
    "Boston Medical Center on 12/19/2025."
)

spans = ner(text)

redacted = text
for item in sorted(spans, key=lambda x: x["start"], reverse=True):
    label = item["entity_group"]
    redacted = redacted[: item["start"]] + f"[{label}]" + redacted[item["end"] :]

print(spans)
print(redacted)
```

### Example output schema

For downstream systems, a practical span schema is:

```json
[
  {"start": 8, "end": 18, "label": "NAME", "score": 0.97},
  {"start": 25, "end": 36, "label": "ID", "score": 0.94},
  {"start": 45, "end": 66, "label": "FACILITY", "score": 0.91},
  {"start": 70, "end": 80, "label": "DATE", "score": 0.89}
]
```

## Safety and privacy

This model was trained on synthetic data and is published for research and tooling purposes. Do not send real PHI to public demos or public inference endpoints. Use private infrastructure, access controls, and organization-approved evaluation workflows for real deployments.

## Citation

```bibtex
@misc{janumpally_phi_span_detector_2025,
  title        = {PHI Span Detector (Synthetic)},
  author       = {Bharath Kumar Reddy Janumpally},
  year         = {2025},
  publisher    = {Hugging Face},
  howpublished = {Model on Hugging Face}
}
```

## Contact

If you use this model in a serious workflow, validate it against your own internal test cases and document the operating policy around false positives, false negatives, and escalation paths.