Upload README.md with huggingface_hub

README.md

@@ -58,7 +58,7 @@ the full real set), face-cropped via MTCNN and persisted to the
 `Arko007/deepfake-ff-face-crops` HF dataset repo across multiple
 preprocessing sessions (resumable, `processed_videos.txt`-tracked, no
 video reprocessed twice). The held-out validation split used for the
-metrics below has 11,570 frames across 1,049 videos (150 real / 899
+metrics below has 11,666 frames across 1,049 videos (150 real / 899
 fake) — consistent with FF++'s ~1:6 real:fake ratio, confirming the
 corrected restore actually pulled in the full real class this time.
 
@@ -124,6 +124,59 @@ No model's F1 collapsed under the class imbalance; the `pos_weight` +
 Full machine-readable results (including confusion matrices) are in
 `eval_results.json` in this repo.
 
+## Cross-architecture calibration, explanation-quality, and triage study (2026-07-25)
+
+Full research-questions study (RQ1-RQ3, see the paper/report in
+`research/`), run via 20-pass MC-Dropout across the **full** held-out
+validation split (11,666 frames / 1,049 videos), plus explanation-quality
+and triage-transferability metrics on a class-balanced ~4,000-sample
+draw per model. Raw output: `research_results.json` in this repo.
+
+**RQ1 — Expected Calibration Error (lower is better):**
+
+| Model | Frame ECE | Video ECE | Frame AUROC | Video AUROC |
+|---|---|---|---|---|
+| xception | 0.0329 | 0.0377 | 0.9976 | 0.9996 |
+| sbi | 0.1155 | 0.1303 | 0.9894 | 0.9981 |
+| vit | 0.0773 | 0.0906 | 0.9830 | 0.9924 |
+| lsda | 0.0632 | 0.0728 | 0.9930 | 0.9985 |
+
+All four are reasonably calibrated (ECE <0.12), but not uniformly —
+SBI's ECE is ~3.5x Xception's. **Important caveat:** as instantiated,
+Xception (`timm` `xception41`) and UIA-ViT (`vit_base_patch16_224`) have
+zero dropout probability, so MC-Dropout contributes zero stochastic
+averaging for either — their reported ECE is mathematically identical to
+raw-softmax calibration, not a genuine MC-Dropout-averaged number like
+SBI's and LSDA's. A dropout-fix retrain (`drop_rate=0.2`, distinct
+`_dropoutfix` checkpoint suffix) is in progress to close this gap; see
+Future Work in the paper.
+
+**RQ2 — Spearman correlation, predictive entropy vs. explanation stability:**
+
+| Model | n | ρ | p-value |
+|---|---|---|---|
+| xception | 4,000 | −0.0530 | 7.96e-4 |
+| sbi | 3,165 | 0.0105 | 0.556 |
+| vit | 4,000 | −0.0538 | 6.67e-4 |
+| lsda | 4,000 | −0.0822 | 1.96e-7 |
+
+Higher uncertainty correlates with less stable explanations, significantly,
+in 3/4 architectures (not SBI) — small effect sizes throughout.
+
+**RQ3 — Triage false-negative capture (same untuned entropy=0.6,
+stability=0.65 threshold pair for all four models):**
+
+| Model | FN Escalation | Overall Escalation | Capture Ratio |
+|---|---|---|---|
+| xception | 95.2% | 65.7% | 1.45x |
+| sbi | 96.1% | 93.3% | 1.03x |
+| vit | 93.0% | 84.9% | 1.10x |
+| lsda | 88.8% | 76.0% | 1.17x |
+
+The triage rule escalates 88.8-96.1% of true false negatives across every
+architecture without any per-architecture recalibration — the core
+transferability claim holds cleanly.
+
 ## Live deployment verification (2026-07-24)
 
 Both the FastAPI backend and the React frontend were deployed to Modal
@@ -142,19 +195,22 @@ stopped again after verification).
 - Frontend static build served correctly (200, correct title) and was
   pointed at the Modal backend for this verification pass only.
 
-For day-to-day use, the frontend defaults to sharded per-architecture
-instances on Render's free tier (CPU-only; see repo `frontend/src/App.jsx`
-`MODEL_ENDPOINTS`).
+As of 2026-07-25, both backend and frontend are deployed to Modal
+(`deepfake-triage-backend` / `deepfake-triage-frontend`) for user
+testing; HF Spaces now only supports Gradio so it is no longer used for
+hosting this FastAPI+React app, and Render/Vercel are not the live path
+either (see repo `frontend/src/App.jsx` `MODEL_ENDPOINTS`, which points
+at the Modal backend).
 
 ## Uncertainty, explainability, and triage (implementation, not just checkpoints)
 
 See `plugin_core/` in the repo:
 - `uncertainty.py` — `MCDropoutPlugin` (stochastic forward passes → mean
   probability, variance, entropy), plus `calculate_ece` /
-  `generate_reliability_data` for calibration analysis (not yet run
-  against this labeled held-out split — the deployed UI's Calibration tab
-  currently shows illustrative, not measured, calibration-curve numbers;
-  the F1/precision/recall/AUROC numbers above ARE measured).
+  `generate_reliability_data` for calibration analysis — now run against
+  the full labeled held-out split (see the RQ1-RQ3 study section above);
+  the deployed UI's Calibration tab shows these same measured numbers,
+  not illustrative ones.
 - `explainability.py` — Grad-CAM (CNN backbones) / saliency-based attention
   (ViT), with stability-under-perturbation and spatial-entropy quality
   metrics.