---
base_model: Qwen/Qwen3-4B-Instruct-2507
library_name: peft
license: apache-2.0
tags:
- lora
- peft
- pii
- pii-masking
- qwen3
---

# Qwen3-4B-Instruct-2507 — PII Masking (SFT LoRA)

LoRA adapter that fine-tunes [Qwen/Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507)
to detect personally identifiable information (PII) in text, replace it with `[PII]` tags, and
return the masked text wrapped in `<masked_output>...</masked_output>`.

## Results

Greedy exact-match on a 5,914-row held-out split (provably disjoint from training by input):

| Model | Exact-match | Relaxed-match | mean_reward\* |
|---|---|---|---|
| Base Qwen3-4B-Instruct-2507 | 0.409 | 0.577 | 0.472 |
| **This adapter (SFT)** | **0.932** | 0.976 | 0.939 |

\*`mean_reward` is the normalized reward `(exact·1.0 + pii_count·0.5 + format·0.1) / 1.6` (max 1.0).
The Prime Intellect verifier for this task reports the same three components **un-normalized**
(max 1.6), so e.g. a Prime reward of ~1.50 corresponds to ~0.94 here.

Format compliance is already ~1.00 on the base model; the adapter's gain is almost entirely
learning the exact `[PII]` segmentation convention (it cuts the relaxed-minus-exact gap from
0.168 to 0.045).

## Training

- **Method:** supervised fine-tuning, completion-only loss (loss on the gold masked answer + EOS only).
- **LoRA:** r=8, α=16, dropout 0, all-linear (q/k/v/o/gate/up/down projections).
- **Data:** 5,000 examples from [`AdamLucek/open-pii-masking-en-us-30k`](https://huggingface.co/datasets/AdamLucek/open-pii-masking-en-us-30k) (train split).
- **Schedule:** 2 epochs, lr 2e-4 cosine, bf16, gradient checkpointing.

The published adapter was extracted from the merged SFT checkpoint by a per-layer rank-8 SVD of
`(W_merged − W_base)`. Base + this adapter reproduces the merged model's held-out exact-match to
within bf16 rounding (0.9319 vs 0.9325).

## Usage

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-4B-Instruct-2507", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "ichetandhembre/Qwen3-4B-Instruct-2507-PII-SFT-LoRA")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")

SYSTEM = ("Replace all personally identifiable information (PII) in the text with [PII] tags. "
          "PII includes: names, dates, phone numbers, SSNs, account numbers, addresses, "
          "email addresses. Wrap the masked text in <masked_output>...</masked_output>.")
msgs = [{"role": "system", "content": SYSTEM},
        {"role": "user", "content": "Hi, this is John Smith, call me at 555-0123."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(ids, max_new_tokens=384, do_sample=False)[0][ids.shape[1]:],
                 skip_special_tokens=True))
```

Or serve base + adapter with vLLM (`enable_lora=True`, `max_lora_rank>=8`).