---
base_model: distilbert/distilbert-base-uncased-finetuned-sst-2-english
datasets:
- anmolshrivastav/scam-hum-india
language: en
license: apache-2.0
metrics:
- accuracy
- f1
pipeline_tag: text-classification
tags:
- text-classification
- scam-detection
- fraud-detection
- spam-detection
- distilbert
- india
library_name: transformers
---

# Scam/Spam Message Detector — India (DistilBERT, fine-tuned)

Fine-tuned version of [`distilbert-base-uncased-finetuned-sst-2-english`](https://huggingface.co/distilbert/distilbert-base-uncased-finetuned-sst-2-english)
for binary classification of scam/spam vs. legitimate ("ham") text messages, with a focus on Indian scam patterns
(lottery fraud, Aadhaar/KYC phishing, UPI/bank fraud, fake job offers, OTP-theft attempts).

## Labels
- `0` / `ham`: Legitimate message
- `1` / `spam`: Scam or spam message

## Training Data
- Base dataset: `scam_hum_india.csv`, real-world Indian SMS/message samples (telecom promos, government notices, casual messages), deduplicated.
- Augmented with ~200 additional synthetic examples covering underrepresented scam categories: lottery/prize fraud, Aadhaar/KYC phishing, UPI/bank fraud, fake job offers, and OTP-theft social engineering — added after identifying that the base dataset was skewed toward telecom promotional spam and missed these patterns.
- Class-weighted loss (`sklearn` `compute_class_weight="balanced"`) used during training to counter class imbalance (~1521 ham vs ~700 spam originally).

## Training Procedure
- Base model: `distilbert-base-uncased-finetuned-sst-2-english`
- Epochs: 3
- Learning rate: 2e-5
- Batch size: 16
- Weighted `CrossEntropyLoss` via custom `Trainer` subclass to address class imbalance

## Evaluation Results
| Metric | Score |
|---|---|
| Accuracy | 0.9971 |
| F1 | 0.9963 |
| Eval Loss | 0.0206 |

## Limitations
- Trained primarily on India-specific scam message patterns (Aadhaar, UPI, telecom, lottery in Indian Rupees); may not generalize well to other regions or currencies.
- English-tokenizer based — code-mixed Hindi/English (Hinglish) text may reduce accuracy.
- Subtle, low-signal scam messages (no amounts, brand names, or urgency cues) are harder to detect and may be misclassified as legitimate.
- A portion of spam examples are synthetically generated to patch category gaps; performance on real-world messages of these types should be validated further before production use.

## Usage
```python
from transformers import pipeline
clf = pipeline("text-classification", model="anmolshrivastav/distilbert-scam-detector-india")
clf("Congratulations! You've won 50000 rupees, click link to claim")
```


## Quantized ONNX Version

A dynamically quantized (INT8) ONNX version `model.onnx` is available in the `onnx/` subfolder for faster CPU inference with a smaller footprint. Also a `onnx/model_fp32.onnx` (unquantized ONNX version)

```python
from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer, pipeline

model = ORTModelForSequenceClassification.from_pretrained(
    "anmolshrivastav/distilbert-scam-detector-india", subfolder="onnx"
)
tokenizer = AutoTokenizer.from_pretrained(
    "anmolshrivastav/distilbert-scam-detector-india", subfolder="onnx"
)

clf = pipeline("text-classification", model=model, tokenizer=tokenizer)
clf("Congratulations! You've won 50000 rupees, click link to claim")
```