AffKernel: IIT-AFF affordance segmentation (R50vd, stride-2, full model)
Released weights for AffKernel: single-pass, NMS-free affordance segmentation. RT-DETR object queries generate per-query dynamic convolution kernels that decode per-object affordance masks from one shared high-resolution map.
This card describes the artifacts only. Method, results, ablations, deployment benchmarks, protocol caveats, licensing detail and citation all live in the code repository and are deliberately not repeated here: https://github.com/anh0001/affkernel (reproduction guide, dataset setup)
What this checkpoint is
IIT-AFF, 72 epochs, seed 42, last-epoch EMA weights, fp32.
F_beta^w (beta^2 = 1) |
0.8685 |
F_beta^w (beta^2 = 0.3) |
0.8582 |
This is one seed, not the headline mean: the three-seed figure is
0.8675 ± 0.0009, and seed 42 is the highest of the three (7: 0.8668,
123: 0.8673). Both beta conventions are given because a beta^2 = 1 number
must never be compared against a beta^2 = 0.3 one. Read the accuracy as
exploratory rather than confirmatory: IIT-AFF ships no validation split, so the
same test set also informed the recipe. Full reasoning in the repository README.
Load it
hf download anhrisn/affkernel-iit-aff \
affkernel_iit_r50vd_stride2_deepsup_seed42.pth --local-dir weights/
Inference, evaluation and deployment commands are in the repository README.
Files
| File | What it is |
|---|---|
affkernel_iit_r50vd_stride2_deepsup_seed42.pth |
The model. fp32 EMA weights, 174 MB. |
backbone_fp16.plan |
Optional TensorRT fp16 backbone-only engine, 46.4 MiB. Not portable, not usable on its own. |
deepsup in the filename is the historical identifier for the auxiliary readout
losses used during training; it is kept so the documented commands keep working.
Checkpoint format
A single file holding the EMA weights only:
{"ema": {"module": <OrderedDict of 760 tensors>, "updates": 111312}}
The evaluation path reads the EMA parameters, so raw weights and optimizer state
are not included. Do not repack this into a bare {"model": ...} dict: under
an EMA-enabled config the solver would then evaluate a freshly initialised EMA
module and score near zero.
The first run fetches the ImageNet-pretrained ResNet-50vd backbone from the
RT-DETR release artefacts, so it needs network access. Set
PResNet.pretrained: False for an offline deployment.
backbone_fp16.plan
The dynamic-kernel affordance head cannot be exported to ONNX, but the backbone
is plain convolutions and is the largest single term of the forward pass
(17.7 ms to 5.3 ms). The encoder, decoder and affordance head stay in PyTorch,
CUDA-graphed against the engine's output buffers. Built on Jetson AGX Orin
(64 GB), JetPack 6.2 (L4T R36.4.7), TensorRT 10.3.0 / CUDA 12.6; input images
(1, 3, 640, 640) fp16 fixed; outputs feat0..feat3 fp16.
Engine files are not portable. A
.planis compiled for one specific GPU, TensorRT version and input shape. On any other device, JetPack/TensorRT version or input size it will fail to deserialize, or load and behave incorrectly. It is published here only as a convenience for setups identical to the above. If yours differs in any respect, build your own — one command, a few minutes, and it is the supported path. See Deployment.
Intended use and limitations
Research on affordance perception and perception-guided grasping.
- Closed vocabulary: 10 object and 9 affordance classes, listed in
docs/datasets.md. It will not generalise to unseen categories. - IIT-AFF is a tabletop dataset. Other viewpoints, lighting or clutter regimes are untested.
- The residual error is dominated by missed detections rather than by mask quality.
- Not validated for safety-critical deployment. A predicted grasp region is a perception cue, not a guarantee of a safe grasp.
License
Weights are Apache-2.0, matching the RT-DETR components and the ImageNet-pretrained backbone they were initialised from. The IIT-AFF dataset is not redistributed here; obtain it from https://sites.google.com/site/iitaffdataset/. Full attribution and the citation BibTeX are in the repository.