README.md

metadata

title: X-CLIP Crime Classifier
emoji: 🎬
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 6.12.0
app_file: app.py
pinned: false
license: mit
short_description: X-CLIP fine-tuned on UCF-Crime with LLM descriptions

Crime Detection with X-CLIP + LLM Descriptions

Fine-tunes X-CLIP on the UCF-Crime dataset using rich scene descriptions generated by Google Gemini Flash, instead of bare category labels. Multi-class classification over 14 categories (13 crime types + Normal).

Quick try

Click any of the 8 example videos on the Live Prediction tab. Each example is labelled with one of three scenarios:

• 🟢 All pass — simple case where CLIP, X-CLIP, and our model all get it right
• 💡 Ours only — baselines fail, the fine-tuned model recovers it
• 🔴 All fail — open limitation (mostly visually ambiguous classes like Abuse / Robbery)

The Method Comparison tab runs all three side-by-side on the same video (slower — ~45s on the free CPU tier because models are swapped sequentially).

Method in one paragraph

Each of the 532 training videos gets a unique paragraph-long description from Gemini Flash. These pairs are used to contrastively fine-tune X-CLIP's visual encoder (last 3 transformer blocks + CCT + MIT). At inference, we average 5 hand-crafted descriptive prompts per class and classify test videos by cosine similarity in the shared embedding space.

Numbers (UCF-Crime Action Recognition test split, 168 videos)

Method	Accuracy	F1 (macro)	AUC-ROC
CLIP zero-shot	23.2%	21.0%	0.77
X-CLIP zero-shot	17.3%	13.9%	0.70
Ours (X-CLIP + LLM)	26.2%	22.1%	0.80

Free CPU tier note

This Space runs on CPU. Inference takes roughly 30-60 seconds per video on the Live Prediction tab, and 2-3 minutes on the Method Comparison tab (3 models loaded sequentially). GPU-backed inference (2-5s per video) would require a paid Space upgrade.

Dataset & license

• Dataset: UCF-Crime (Sultani et al., CVPR 2018). Used here under its research-only license.
• Example videos: 8 clips from the public UCF-Crime test split.
• Code: MIT.

Citation

If this project is useful to you, please cite the original UCF-Crime and X-CLIP papers:

@inproceedings{sultani2018real,
  title={Real-world anomaly detection in surveillance videos},
  author={Sultani, Waqas and Chen, Chen and Shah, Mubarak},
  booktitle={CVPR}, year={2018}
}

@inproceedings{ni2022expanding,
  title={Expanding language-image pretrained models for general video recognition},
  author={Ni, Bolin and others},
  booktitle={ECCV}, year={2022}
}