Add AffKernel IIT-AFF checkpoint (R50vd stride-2 + deep supervision, seed 42) and model card

README.md

@@ -0,0 +1,171 @@
+---
+license: apache-2.0
+library_name: pytorch
+pipeline_tag: image-segmentation
+tags:
+  - affordance-segmentation
+  - instance-segmentation
+  - object-detection
+  - robotics
+  - manipulation
+  - rt-detr
+  - real-time
+metrics:
+  - name: Weighted F-measure (beta^2=1)
+    type: f-measure
+    value: 0.8685
+---
+
+# AffKernel: IIT-AFF affordance segmentation (R50vd, stride-2 + deep supervision)
+
+Single-pass, NMS-free affordance segmentation. AffKernel couples RT-DETR object
+queries with a CondInst-style per-query dynamic-convolution kernel that decodes
+per-object affordance masks from one shared high-resolution affordance map. A
+detector tells a robot *what* an object is; AffKernel also tells it *where* to
+act on it, at a latency a control loop can afford.
+
+- **Code:** https://github.com/anh0001/affkernel
+- **Architecture:** RT-DETR-R50vd + affordance branch with stride-2 readout and deep supervision
+- **Training data:** IIT-AFF (Nguyen et al., IROS 2017), 6,184 train images
+- **Weights:** last-epoch (epoch 72) EMA parameters, fp32, 174 MB
+
+## Results
+
+Evaluated on the IIT-AFF test split (2,651 images) with the Margolin weighted
+F-measure.
+
+| Metric | Value |
+|---|---:|
+| `F_beta^w` (beta^2 = 1), this checkpoint | **0.8685** |
+| `F_beta^w` (beta^2 = 0.3), this checkpoint | 0.8582 |
+| `F_beta^w` (beta^2 = 1), mean of 3 seeds | 0.8675 ± 0.0009 |
+| Mask quality on detected instances | 0.8933 |
+| Latency, 640x640, fp32, RTX 6000 Ada | 23.8 ms median |
+| Throughput | 41.7 img/s (42 FPS) |
+
+Against published baselines on the same benchmark: Mask R-CNN 0.844 at 45 ms,
+deterministic Swin-T 0.883 at 42 ms, Bayesian Swin-T deep ensemble 0.906 at
+roughly 1015 ms. AffKernel beats the Mask R-CNN baseline at about half its
+latency and trails the deterministic Swin-T by 1.55 points at 57% of its
+latency, with fully deterministic single-pass inference.
+
+**On seed selection.** This is seed 42, which is both the primary anchor seed
+used throughout the paper and the highest scoring of the three seeds trained
+(seed 7: 0.8668, seed 123: 0.8673). IIT-AFF provides no validation split, so
+the epoch was fixed a priori (last epoch, EMA weights) rather than selected on
+the test set.
+
+**On the beta convention.** Two conventions circulate in this literature. The
+AffordanceNet lineage reports `beta^2 = 0.3`; recent transformer baselines
+report `beta^2 = 1`. Both are given above so that comparisons can be made at a
+matched convention. Do not compare a `beta^2 = 1` number against a
+`beta^2 = 0.3` number.
+
+## Usage
+
+```bash
+git clone https://github.com/anh0001/affkernel.git
+cd affkernel
+pip install -r requirements.txt
+
+pip install huggingface_hub
+hf download anhrisn/affkernel-iit-aff \
+  affkernel_iit_r50vd_stride2_deepsup_seed42.pth --local-dir weights/
+
+python tools/infer.py \
+  -c configs/rtdetr/rtdetr_r50vd_6x_iit_v3_stride2_deepsup.yml \
+  -r weights/affkernel_iit_r50vd_stride2_deepsup_seed42.pth \
+  --input path/to/image.jpg \
+  --output outputs/prediction.png
+```
+
+Reproduce the reported metric (requires the IIT-AFF dataset; see
+[`docs/datasets.md`](https://github.com/anh0001/affkernel/blob/main/docs/datasets.md)):
+
+```bash
+python tools/decompose_fbw_gap.py \
+  -c configs/rtdetr/rtdetr_r50vd_6x_iit_v3_stride2_deepsup.yml \
+  -r weights/affkernel_iit_r50vd_stride2_deepsup_seed42.pth --beta2 1.0
+```
+
+### Checkpoint format
+
+A single file containing the EMA weights only:
+
+```python
+{"ema": {"module": <OrderedDict of 760 tensors>, "updates": 111312}}
+```
+
+The evaluation path uses the EMA parameters, so the raw (non-EMA) weights and
+the optimizer state from training are not included. Do not repack this into a
+bare `{"model": ...}` dict: under an EMA-enabled config the solver would then
+evaluate a freshly initialised EMA module and score near zero.
+
+The first run downloads ImageNet-pretrained ResNet-50vd backbone weights from
+the RT-DETR release artefacts, so it needs network access. For an offline
+deployment, set `PResNet.pretrained: False` in the config.
+
+## Classes
+
+**Objects (10):** bowl, tvm, pan, hammer, knife, cup, drill, racket, spatula, bottle
+
+**Affordances (9):** contain, cut, display, engine, grasp, hit, pound, support, w-grasp
+
+## Intended use and limitations
+
+Intended for research on affordance perception and perception-guided grasping.
+
+- Trained on a closed vocabulary of 10 object and 9 affordance classes; it will
+  not generalise to unseen categories.
+- IIT-AFF is a tabletop dataset. Performance under other viewpoints, lighting
+  or clutter regimes is untested.
+- The residual error is dominated by **missed detections**, not by mask quality.
+  On instances the detector does fire on, mask quality (0.893) already exceeds
+  the deterministic Swin-T baseline's overall score.
+- Latency was measured at fp32 on an RTX 6000 Ada. fp16 and TensorRT are
+  untested; they would be expected to help but are not characterised here.
+- Not validated for safety-critical deployment. A predicted grasp region is a
+  perception cue, not a guarantee of a safe grasp.
+
+## License and attribution
+
+These weights are released under Apache-2.0, matching the licence of the
+RT-DETR components and the ImageNet-pretrained backbone they were initialised
+from. The AffKernel repository's own source contributions are MIT licensed; see
+[`THIRD_PARTY_LICENSES.md`](https://github.com/anh0001/affkernel/blob/main/THIRD_PARTY_LICENSES.md).
+
+The **IIT-AFF dataset is not redistributed** here or in the code repository. It
+states no licence; its authors request citation of the original paper. Obtain
+it from https://sites.google.com/site/iitaffdataset/.
+
+## Citation
+
+```bibtex
+@article{risnumawan2026affkernel,
+  title   = {AffKernel: Single-Pass Affordance Segmentation with Per-Query
+             Dynamic Convolution for Real-Time Robotic Manipulation},
+  author  = {Risnumawan, Anhar},
+  journal = {IEEE Access},
+  year    = {2026},
+  note    = {Under review}
+}
+
+@inproceedings{nguyen2017object,
+  title     = {Object-Based Affordances Detection with Convolutional Neural
+               Networks and Dense Conditional Random Fields},
+  author    = {Nguyen, Anh and Kanoulas, Dimitrios and Caldwell, Darwin G. and
+               Tsagarakis, Nikos G.},
+  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and
+               Systems (IROS)},
+  year      = {2017}
+}
+
+@inproceedings{lv2024detrs,
+  title     = {DETRs Beat YOLOs on Real-time Object Detection},
+  author    = {Lv, Wenyu and Zhao, Yian and Xu, Shangliang and Wei, Jinman and
+               Wang, Guanzhong and Cui, Cheng and Du, Yuning and Dang, Qingqing
+               and Liu, Yi},
+  booktitle = {CVPR},
+  year      = {2024}
+}
+```

affkernel_iit_r50vd_stride2_deepsup_seed42.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e32f2412f1218a02ea084aae385a9d28a1f6dea5ac6d82fd4f092c4234eba6b
+size 174431423