README.md

---
license: apache-2.0
datasets:
- ai4privacy/pii-masking-200k
- ai4privacy/pii-masking-400k
- ai4privacy/pii-masking-openpii-1m
- piimb/privy
- gretelai/gretel-pii-masking-en-v1
- nvidia/Nemotron-PII
language:
- ar
- bn
- de
- en
- es
- fr
- hi
- it
- ja
- ko
- nl
- pt
- te
- tr
- vi
- zh
base_model:
- openai/privacy-filter
- OpenMed/privacy-filter-multilingual-v2
pipeline_tag: token-classification
library_name: transformers
tags:
- token-classification
- pii
- multilingual
- openai-privacy-filter
- ner
---
# OpenMed Privacy Filter Multilingual v2 - ONNX

This is an [ONNX](https://onnx.ai/) export of [`OpenMed/privacy-filter-multilingual-v2`](https://huggingface.co/OpenMed/privacy-filter-multilingual-v2), converted for CPU inference with [`onnxruntime`](https://onnxruntime.ai/)'s `CPUExecutionProvider`. No fine-tuning was performed - this repo only changes the runtime format, not the model's weights or behavior.

## About the base model

All credit for the model itself goes to **[OpenMed](https://huggingface.co/OpenMed)**. From their model card: `privacy-filter-multilingual-v2` performs fine-grained PII extraction across **54 entity categories in 16 languages**, built on top of **OpenAI's Privacy Filter** (a Mixture-of-Experts transformer, 128 experts with top-4 routing per token) with a BIOES token-classification head, and extended with multilingual coverage and expanded entity types by OpenMed and the AI4Privacy community. See the [original model card](https://huggingface.co/OpenMed/privacy-filter-multilingual-v2) for the full entity list, training details, and citation.

This conversion exists only because no ONNX export was published alongside the original safetensors weights.

## What was done for this conversion

The checkpoint's `config.json` declares a custom `model_type`/architecture name (`openai_privacy_filter` / `OpenAIPrivacyFilterForTokenClassification`). Depending on your installed `transformers` version this may already be a recognized, first-class architecture (loadable directly via `AutoModelForTokenClassification`/`AutoTokenizer`) - that's what this export uses. If you're on an older `transformers` release without native support, do **not** substitute `transformers`' generic `gpt_oss` classes as a stand-in: despite matching tensor shapes (the checkpoint loads with zero missing/unexpected weight keys either way), the two architectures differ in a way that silently produces near-total false negatives - this model is **bidirectional** with sliding-window attention, while `gpt_oss` is causal/decoder-style. That mismatch doesn't error out; it just quietly predicts "O" (no entity) almost everywhere, while looking numerically self-consistent. If you hit that, upgrade `transformers` instead of working around it.

Two things don't just fall out of a plain `torch.onnx.export` call, either way:

1. **MoE routing.** The expert-routing module has two forward code paths: a data-dependent, per-batch Python loop (`.nonzero()`-driven "which experts got hit") used at runtime, which isn't traceable for export; and a fully vectorized dense computation over all 128 experts (weighted by mostly-zero routing weights), which is traceable and mathematically identical. This export monkeypatches the routing module to always use the dense path. Getting this exactly right matters: the routing weights arrive as a **sparse** `(num_tokens, top_k)` pair (`router_indices`, `routing_weights`), not a dense `(num_tokens, num_experts)` tensor, so they have to be scattered into a dense per-expert weight matrix first; and this checkpoint's gate/up projection uses a **concatenated** split (`gate, up = gate_up.chunk(2, dim=-1)`), not the interleaved split (`gate_up[..., ::2]`/`[..., 1::2]`) some sibling MoE architectures use. Either bug produces an export that's internally self-consistent (matches its own incorrect PyTorch reference to ~1e-5) but wrong relative to the true model - only comparing against a real forward pass on a known example (e.g. does it actually tag "John Smith" as a name?) catches it.
2. **Precision.** The checkpoint's `config.json` declares a default of bf16. Loading in that precision and exporting produces `Where` nodes with bf16 scalar operands that `onnxruntime`'s `CPUExecutionProvider` doesn't have a kernel for (a `NOT_IMPLEMENTED` error at session-load time, not export time). This export explicitly loads and exports in full **fp32**.

## Verification

The exported graph was validated against a native, unpatched PyTorch forward pass of the original checkpoint on the same inputs (both loaded fp32), running on `onnxruntime`'s `CPUExecutionProvider`:

- **Max absolute logit difference (ONNX vs. true native PyTorch): ~1.8e-5**
- Spot-checked against a plain-English sentence containing a name, phone number, and email - all three correctly tagged with their entity types and BIOES boundaries.

## Usage

```python
import numpy as np
import onnxruntime as ort
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("lemonade-sdk/openmed-privacy-filter-multilingual-v2-onnx")
session = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])

text = "My name is John Smith, call me at 555-123-4567 or email john@example.com."
encoded = tokenizer(text, return_tensors="np")
(logits,) = session.run(None, {
    "input_ids": encoded["input_ids"],
    "attention_mask": encoded["attention_mask"],
})  # logits: [batch, seq_len, 217] (BIOES over 54 entity types + "O")
```

No custom mask construction needed - the graph takes a plain 2D `input_ids` + `attention_mask`, same convention as a standard BERT-family token classifier.

## Using this with the Lemonade router

This repo is self-contained for [Lemonade](https://github.com/lemonade-sdk/lemonade)'s `onnxruntime` backend (via [`ort-server`](https://github.com/lemonade-sdk/ort-server)) - `model.onnx` + `model.onnx.data` + `tokenizer.json` + `config.json` + `manifest.json` all sit together at the repo root, so it can be registered directly as a checkpoint with no extra file staging.

**1. Register the classifier:**

```bash
curl -X POST http://localhost:13305/v1/pull -H "Content-Type: application/json" -d '{
  "model_name": "user.privacy-filter-ml-v2-onnx",
  "checkpoint": "lemonade-sdk/openmed-privacy-filter-multilingual-v2-onnx",
  "recipe": "onnxruntime"
}'
```

**2. Register the router policy** - a ready-to-use `collection.router` policy that routes to a local model whenever any of this classifier's 216 non-"O" labels crosses `min_score: 0.5`, and to a cloud model otherwise, is published separately at [`lemonade-sdk/pii_policy_openmed-privacy-filter-multilingual-v2-onnx`](https://huggingface.co/lemonade-sdk/pii_policy_openmed-privacy-filter-multilingual-v2-onnx):

```bash
hf download lemonade-sdk/pii_policy_openmed-privacy-filter-multilingual-v2-onnx --local-dir .
curl -X POST http://localhost:13305/v1/pull -H "Content-Type: application/json" \
  --data-binary @pii_policy_openmed-privacy-filter-multilingual-v2-onnx.json
```

The policy's `routing.candidates` (`Qwen3.5-0.8B-GGUF` local / `fireworks.kimi-k2p6` cloud) and `min_score` are starting points, not fixed requirements - swap either to whatever local/cloud models you have configured. On the Nemotron-PII benchmark (20,000 PII-bearing prompts), this policy scored a 0% leak rate (zero PII prompts routed to the cloud candidate).

For building your own routing policy from scratch, or a more sophisticated one (multiple classifiers, LLM-based routing, custom match logic) rather than adapting this one, see the [`lemonade-router-builder`](https://github.com/amd/skills/tree/main/skills/lemonade-router-builder) skill - it turns a natural-language description of routing intent into a valid `collection.router` policy JSON.

## License

Apache 2.0, inherited from the base model.

## Citation

Please cite the original model and its underlying work:

```bibtex
@misc{openmed-privacy-filter-v2,
  title  = {OpenMed Privacy Filter Multilingual v2},
  author = {OpenMed},
  year   = {2026},
  url    = {https://huggingface.co/OpenMed/privacy-filter-multilingual-v2}
}
```

See the [original model card](https://huggingface.co/OpenMed/privacy-filter-multilingual-v2) for the complete BibTeX entry crediting OpenMed, OpenAI, and the AI4Privacy contributors.