---
license: other
tags:
  - deepfake-detection
  - uncertainty-quantification
  - pytorch
---

# Deepfake Triage Plugin — Detector Checkpoints

Four proxy-architecture deepfake detector checkpoints trained for the
**model-agnostic uncertainty + explainability triage plugin** described in
the research proposal *"A Model-Agnostic Uncertainty and Explainability
Plugin for Generalizable Deepfake Detection"* (Bhumika Tewari, TBVL Lab,
IISER Bhopal). Each checkpoint is loaded by the plugin's FastAPI backend
(`backend/main.py` in [Anamitra-Sarkar/deepfake-triage-plugin](https://github.com/Anamitra-Sarkar/deepfake-triage-plugin))
and wrapped with MC-Dropout uncertainty estimation, Grad-CAM/attention
explainability, and a joint confidence+explanation-quality triage rule.

**Update (2026-07-24): retrained on the full corrected dataset.** An
earlier version of these checkpoints was trained on data affected by a bug
in `restore_crops_from_hf.py` that silently excluded ~750 real videos'
worth of already-extracted crops, leaving that run with an artificially
severe ~1:35 real:fake ratio instead of FF++'s actual ~1:6. That bug is
fixed; the checkpoints and metrics below are from a full retrain on the
corrected, complete dataset. The old real-class F1 numbers (xception 0.725,
sbi 0.673, vit 0.627, lsda 0.725 at video level) are superseded by the
numbers in this card, which are all equal or higher.

## Files

| File | Architecture (proxy backbone used) | Size |
|---|---|---|
| `best_xception.pth` | Xception proxy: `timm` `xception41` (falls back to EfficientNet-B0 if `timm` unavailable) | ~100 MB |
| `best_sbi.pth` | Self-Blended Images (SBI) proxy: EfficientNet-B4 | ~71 MB |
| `best_vit.pth` | UIA-ViT proxy: ViT-Base (`vit_base_patch16_224`) | ~343 MB |
| `best_lsda.pth` | LSDA proxy: EfficientNet-B0 + latent-space Gaussian noise injection during training | ~16 MB |

**Important scope note:** these are architecturally-diverse *proxy*
backbones standing in for the four architectures named in the research
proposal (Xception, SBI, UIA-ViT, LSDA) — they reproduce each paper's
general architecture family (CNN / augmentation-based CNN / Vision
Transformer / latent-augmented CNN) but **not** each paper's exact
published training recipe (e.g. SBI's self-blending augmentation
pipeline, UIA-ViT's patch-consistency loss, or LSDA's specific
latent-space augmentation method). Treat these as a working
proof-of-concept for the plugin architecture, not a reproduction of the
original papers' benchmark numbers.

## Training data

Real FaceForensics++ (c23) videos via the `xdxd003/ff-c23` Kaggle dataset
mirror — folder layout: `DeepFakeDetection`, `Deepfakes`, `Face2Face`,
`FaceShifter`, `FaceSwap`, `NeuralTextures` (fake) and `original` (real).

**This run used the full ~7000-video dataset** (all 6 fake methods +
the full real set), face-cropped via MTCNN and persisted to the
`Arko007/deepfake-ff-face-crops` HF dataset repo across multiple
preprocessing sessions (resumable, `processed_videos.txt`-tracked, no
video reprocessed twice). The held-out validation split used for the
metrics below has 11,570 frames across 1,049 videos (150 real / 899
fake) — consistent with FF++'s ~1:6 real:fake ratio, confirming the
corrected restore actually pulled in the full real class this time.

## Training setup (from `training/train_ddp.py` / the training notebook)

All 4 models: `--epochs 20 --patience 5` (early stopping on validation
loss), AdamW optimizer, `ReduceLROnPlateau` scheduler,
`BCEWithLogitsLoss(pos_weight=n_real/n_fake)` for class-imbalance
correction, plus a `WeightedRandomSampler` (per-class weight `1/n_class`)
during training.
- Xception / SBI / LSDA: `--batch_size 64 --lr 1e-4`
- ViT: `--batch_size 32 --lr 5e-5`

Train/val split: 85/15, **video-level** stratified (not frame-level — see
`split_samples()` in `train_ddp.py`), so frames from the same video never
leak across the split.

## Evaluation methodology and results (real, computed — not illustrative)

Computed by `training/evaluate_models.py`, which reconstructs the exact
held-out validation split (`seed=42`, `val_fraction=0.15`) and reports
accuracy, per-class precision/recall/F1, macro-F1, AUROC, and confusion
matrices, at both frame level and video level (video-level = mean
probability across a video's frames, since frames from the same video are
near-duplicates and accuracy alone is misleading under FF++'s class
imbalance).

**Caveat, stated plainly:** this held-out split was also used *during
training* for checkpoint selection (best validation loss / early
stopping). It is not a separate, from-scratch generalization test set.
Treat these numbers as trustworthy validation-time performance, not an
independent-test-set claim.

### Video-level metrics (the numbers that matter for real-world triage)

| Model | Accuracy | Real Precision | Real Recall | **Real F1** | Fake F1 | Macro F1 | AUROC |
|---|---|---|---|---|---|---|---|
| **xception** | 0.953 | 0.770 | 0.960 | **0.855** | 0.972 | 0.913 | 0.989 |
| **sbi** | 0.869 | 0.523 | 0.973 | **0.681** | 0.918 | 0.799 | 0.971 |
| **vit** | 0.871 | 0.529 | 0.900 | **0.667** | 0.920 | 0.793 | 0.942 |
| **lsda** | 0.924 | 0.662 | 0.953 | **0.781** | 0.954 | 0.868 | 0.978 |

### Frame-level metrics

| Model | Accuracy | Real F1 | Fake F1 | Macro F1 | AUROC |
|---|---|---|---|---|---|
| xception | 0.926 | 0.849 | 0.951 | 0.900 | 0.977 |
| sbi | 0.836 | 0.716 | 0.885 | 0.800 | 0.943 |
| vit | 0.836 | 0.692 | 0.888 | 0.790 | 0.913 |
| lsda | 0.885 | 0.776 | 0.923 | 0.849 | 0.955 |

**Reading these honestly:** accuracy alone would be misleading here (FF++
is fake-heavy) — that's why real-class F1 and AUROC are the headline
numbers. Xception is the strongest all-around (real F1 0.855, AUROC
0.989). SBI and ViT show the largest real-precision vs. real-recall gap
(they over-flag real videos as fake more often) but their AUROC (0.94-0.97)
shows the underlying probability ranking is still strongly separated —
that gap is a threshold-calibration property of those two architectures on
this data, not evidence the model failed to learn. LSDA sits in between.
No model's F1 collapsed under the class imbalance; the `pos_weight` +
`WeightedRandomSampler` combination held up.

Full machine-readable results (including confusion matrices) are in
`eval_results.json` in this repo.

## Live deployment verification (2026-07-24)

Both the FastAPI backend and the React frontend were deployed to Modal
(T4 GPU, CPU fallback if CUDA raises a `RuntimeError` mid-request) purely
to verify the full product end-to-end with these corrected checkpoints —
not a permanent hosting solution (the client's proposal asked for the
working product, not hosted infrastructure; the Modal deployment was
stopped again after verification).

- `/health`: `{"status": "healthy", "cuda_available": true, ...}`.
- `/detect` on a real (non-fake) FF++ validation frame: returned
  `is_fake: false`, `probability: 0.00069` (correctly confident this is
  real), `weights_source: "trained"` (confirms the real checkpoint loaded
  — not a silently-failed fallback to ImageNet weights), full triage
  response (entropy/stability/Grad-CAM heatmap) returned correctly.
- Frontend static build served correctly (200, correct title) and was
  pointed at the Modal backend for this verification pass only.

For day-to-day use, the frontend defaults to sharded per-architecture
instances on Render's free tier (CPU-only; see repo `frontend/src/App.jsx`
`MODEL_ENDPOINTS`).

## Uncertainty, explainability, and triage (implementation, not just checkpoints)

See `plugin_core/` in the repo:
- `uncertainty.py` — `MCDropoutPlugin` (stochastic forward passes → mean
  probability, variance, entropy), plus `calculate_ece` /
  `generate_reliability_data` for calibration analysis (not yet run
  against this labeled held-out split — the deployed UI's Calibration tab
  currently shows illustrative, not measured, calibration-curve numbers;
  the F1/precision/recall/AUROC numbers above ARE measured).
- `explainability.py` — Grad-CAM (CNN backbones) / saliency-based attention
  (ViT), with stability-under-perturbation and spatial-entropy quality
  metrics.
- `triage.py` — joint rule combining entropy, explanation stability, and
  borderline-probability checks into VERIFIED_SAFE / VERIFIED_FAKE /
  ESCALATE_TO_HUMAN.