--- title: FoundationPose 6-DoF emoji: 📦 colorFrom: green colorTo: blue sdk: gradio sdk_version: 6.27.0 app_file: app.py short_description: 6-DoF pose of novel objects from RGB-D plus a CAD model python_version: "3.12" startup_duration_timeout: 45m pinned: false license: other --- # FoundationPose — 6-DoF pose estimation of novel objects [FoundationPose](https://arxiv.org/abs/2312.08344) (NVIDIA, CVPR 2024) estimates the full 6-DoF pose of an object it has never seen during training, given * an **RGB image**, * the **aligned depth map**, * the object's **CAD model**, * the **camera intrinsics** (`fx, fy, cx, cy`), and * a **2D bounding box** around the object. The demo reproduces the model-based *registration* path of the reference implementation: 252 pose hypotheses are sampled on an icosphere (42 viewpoints × 6 in-plane rotations), the translation is initialised from the box centre and the median depth inside the box, every hypothesis is refined for 5 passes by the refiner network, and the scorer network ranks them. The published checkpoints ([`nvidia/foundationpose`](https://huggingface.co/nvidia/foundationpose)) are ONNX; they are converted to PyTorch with [onnx2torch](https://github.com/ENOT-AutoDL/onnx2torch) at startup, so the whole pipeline runs natively on ZeroGPU. Rendering uses [nvdiffrast](https://github.com/NVlabs/nvdiffrast). ## Credits and licences * Weights: NVIDIA Open Model License — * Reference code: [NVlabs/FoundationPose](https://github.com/NVlabs/FoundationPose) * Example scenes and CAD models: [YCB-Video / BOP](https://huggingface.co/datasets/bop-benchmark/ycbv), MIT licence, © 2017 University of Washington Robotics and State Estimation Lab — one of FoundationPose's own evaluation datasets.