README.md

metadata

title: FoundationPose 6-DoF
emoji: 📦
colorFrom: green
colorTo: blue
sdk: gradio
sdk_version: 6.27.0
app_file: app.py
short_description: 6-DoF pose of novel objects from RGB-D plus a CAD model
python_version: '3.12'
startup_duration_timeout: 45m
pinned: false
license: other

FoundationPose — 6-DoF pose estimation of novel objects

FoundationPose (NVIDIA, CVPR 2024) estimates the full 6-DoF pose of an object it has never seen during training, given

• an RGB image,
• the aligned depth map,
• the object's CAD model,
• the camera intrinsics (fx, fy, cx, cy), and
• a 2D bounding box around the object.

The demo reproduces the model-based registration path of the reference implementation: 252 pose hypotheses are sampled on an icosphere (42 viewpoints × 6 in-plane rotations), the translation is initialised from the box centre and the median depth inside the box, every hypothesis is refined for 5 passes by the refiner network, and the scorer network ranks them.

The published checkpoints (nvidia/foundationpose) are ONNX; they are converted to PyTorch with onnx2torch at startup, so the whole pipeline runs natively on ZeroGPU. Rendering uses nvdiffrast.

Credits and licences

• Weights: NVIDIA Open Model License — https://huggingface.co/nvidia/foundationpose
• Reference code: NVlabs/FoundationPose
• Example scenes and CAD models: YCB-Video / BOP, MIT licence, © 2017 University of Washington Robotics and State Estimation Lab — one of FoundationPose's own evaluation datasets.