---
title: X-CLIP Crime Classifier
emoji: 🎬
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: "6.12.0"
app_file: app.py
pinned: false
license: mit
short_description: X-CLIP fine-tuned on UCF-Crime with LLM descriptions
---

# Crime Detection with X-CLIP + LLM Descriptions

Fine-tunes [X-CLIP](https://huggingface.co/microsoft/xclip-base-patch16) on the
UCF-Crime dataset using rich scene descriptions generated by Google Gemini Flash,
instead of bare category labels. Multi-class classification over 14 categories
(13 crime types + Normal).

## Quick try

Click any of the 8 example videos on the **Live Prediction** tab. Each example is
labelled with one of three scenarios:

- 🟢 **All pass** — simple case where CLIP, X-CLIP, and our model all get it right
- 💡 **Ours only** — baselines fail, the fine-tuned model recovers it
- 🔴 **All fail** — open limitation (mostly visually ambiguous classes like Abuse / Robbery)

The **Method Comparison** tab runs all three side-by-side on the same video
(slower — ~45s on the free CPU tier because models are swapped sequentially).

## Method in one paragraph

Each of the 532 training videos gets a unique paragraph-long description from
Gemini Flash. These pairs are used to contrastively fine-tune X-CLIP's visual
encoder (last 3 transformer blocks + CCT + MIT). At inference, we average 5
hand-crafted descriptive prompts per class and classify test videos by cosine
similarity in the shared embedding space.

## Numbers (UCF-Crime Action Recognition test split, 168 videos)

| Method | Accuracy | F1 (macro) | AUC-ROC |
|---|---|---|---|
| CLIP zero-shot | 23.2% | 21.0% | 0.77 |
| X-CLIP zero-shot | 17.3% | 13.9% | 0.70 |
| **Ours (X-CLIP + LLM)** | **26.2%** | **22.1%** | **0.80** |

## Free CPU tier note

This Space runs on CPU. Inference takes roughly 30-60 seconds per video on the
Live Prediction tab, and ~2-3 minutes on the Method Comparison tab (3 models
loaded sequentially). GPU-backed inference (~2-5s per video) would require a
paid Space upgrade.

## Dataset & license

- **Dataset:** [UCF-Crime](https://webpages.uncc.edu/cchen62/dataset.html)
  (Sultani et al., CVPR 2018). Used here under its research-only license.
- **Example videos:** 8 clips from the public UCF-Crime test split.
- **Code:** MIT.

## Citation

If this project is useful to you, please cite the original UCF-Crime and X-CLIP papers:

```
@inproceedings{sultani2018real,
  title={Real-world anomaly detection in surveillance videos},
  author={Sultani, Waqas and Chen, Chen and Shah, Mubarak},
  booktitle={CVPR}, year={2018}
}

@inproceedings{ni2022expanding,
  title={Expanding language-image pretrained models for general video recognition},
  author={Ni, Bolin and others},
  booktitle={ECCV}, year={2022}
}
```