Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,10 +1,60 @@
|
|
| 1 |
---
|
|
|
|
| 2 |
tags:
|
| 3 |
-
-
|
| 4 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
---
|
| 6 |
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
-
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
tags:
|
| 4 |
+
- pytorch
|
| 5 |
+
- image-classification
|
| 6 |
+
- robust-representation
|
| 7 |
+
- modular-ai
|
| 8 |
+
- dynamic-masking
|
| 9 |
+
- dinov2
|
| 10 |
+
- ifam
|
| 11 |
+
arxiv: 2506.08915
|
| 12 |
+
library_name: generic
|
| 13 |
---
|
| 14 |
|
| 15 |
+
# IFAM (metashift-k8) Model Checkpoint
|
| 16 |
+
|
| 17 |
+
This is the official pre-trained checkpoint of the **IFAM (Iterative Focus and Attention Masking)** framework, proposed in the paper **"Two-stage Vision Transformers and Hard Masking offer Robust Object Representations"** (accepted as an oral presentation at ICPR 2026).
|
| 18 |
+
|
| 19 |
+
- **Paper:** [Two-stage Vision Transformers and Hard Masking offer Robust Object Representations](https://arxiv.org/abs/2506.08915)
|
| 20 |
+
- **Repository:** [GitHub - ananthu-aniraj/ifam](https://github.com/ananthu-aniraj/ifam)
|
| 21 |
+
|
| 22 |
+
## Model Description
|
| 23 |
+
IFAM model trained on the Metashifts dataset with 8 parts (K=8).
|
| 24 |
+
|
| 25 |
+
The IFAM framework is a two-stage approach:
|
| 26 |
+
1. **Stage 1 (Selector):** Processes the full image to discover object parts and identify task-relevant regions.
|
| 27 |
+
2. **Stage 2 (Predictor):** Restricts its receptive field to the selected regions using input attention masking, preventing spurious background details from affecting the classification.
|
| 28 |
+
|
| 29 |
+
## Usage
|
| 30 |
+
|
| 31 |
+
To use this model, ensure you have the `ifam` repository cloned and the dependencies installed. You can load this model directly from the Hugging Face Hub:
|
| 32 |
+
|
| 33 |
+
```python
|
| 34 |
+
import torch
|
| 35 |
+
import sys
|
| 36 |
+
# Append the path to the ifam repository
|
| 37 |
+
sys.path.append("/path/to/ifam")
|
| 38 |
+
|
| 39 |
+
from models import FullTwoStageModelDoubleClassify
|
| 40 |
+
|
| 41 |
+
# Load the model directly from Hugging Face Hub
|
| 42 |
+
model = FullTwoStageModelDoubleClassify.from_pretrained("ananthu-aniraj/ifam-metashift-k8")
|
| 43 |
+
model.eval()
|
| 44 |
+
|
| 45 |
+
# Example: Run inference
|
| 46 |
+
# x = torch.rand(1, 3, 224, 224)
|
| 47 |
+
# with torch.no_grad():
|
| 48 |
+
# outputs = model(x)
|
| 49 |
+
```
|
| 50 |
+
|
| 51 |
+
## Citation
|
| 52 |
+
If you use this model in your research, please cite:
|
| 53 |
+
```bibtex
|
| 54 |
+
@inproceedings{aniraj2026ifam,
|
| 55 |
+
title={Two-stage Vision Transformers and Hard Masking offer Robust Object Representations},
|
| 56 |
+
author={Aniraj, Ananthu and Dantas, Cassio F. and Ienco, Dino and Marcos, Diego},
|
| 57 |
+
booktitle={International Conference on Pattern Recognition (ICPR)},
|
| 58 |
+
year={2026}
|
| 59 |
+
}
|
| 60 |
+
```
|