Add GeoRay model card
Browse filesDocument GeoRay architecture, results, installation, inference, limitations, licensing, and citation.
README.md
ADDED
|
@@ -0,0 +1,118 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: facebook/VGGT-1B
|
| 4 |
+
tags:
|
| 5 |
+
- georay
|
| 6 |
+
- satellite-imagery
|
| 7 |
+
- satellite-photogrammetry
|
| 8 |
+
- remote-sensing
|
| 9 |
+
- 3d-reconstruction
|
| 10 |
+
- multi-view-stereo
|
| 11 |
+
- rational-polynomial-camera
|
| 12 |
+
- digital-surface-model
|
| 13 |
+
- pytorch
|
| 14 |
+
- research
|
| 15 |
+
arxiv: 2608.29680
|
| 16 |
+
---
|
| 17 |
+
|
| 18 |
+
# GeoRay
|
| 19 |
+
|
| 20 |
+
**Gauge-Aware Feed-Forward Satellite 3D Reconstruction in the Geodetic Frame**
|
| 21 |
+
|
| 22 |
+
[Paper](https://arxiv.org/abs/2608.29680) · [Code](https://github.com/HIT-SIRS/GeoRay) · [Issues](https://github.com/HIT-SIRS/GeoRay/issues)
|
| 23 |
+
|
| 24 |
+
GeoRay is a feed-forward framework for reconstructing dense satellite surface height directly in the absolute geodetic frame under rational polynomial cameras (RPCs). It combines RPC height-ray reasoning, gauge-aware relief–datum separation, and uncertainty-calibrated monocular/multi-view fusion.
|
| 25 |
+
|
| 26 |
+
## Model details
|
| 27 |
+
|
| 28 |
+
- **Architecture:** frozen VGGT-1B and MoGe-2 backbones with lightweight GeoRay adaptation, relief, datum, calibration, and uncertainty modules
|
| 29 |
+
- **Inputs:** three-view satellite image patches and forward RPC coefficients
|
| 30 |
+
- **Outputs:** absolute surface height and predictive uncertainty; optional longitude/latitude/height back-projection
|
| 31 |
+
- **Framework:** PyTorch 2.7.x
|
| 32 |
+
- **Code license:** Apache-2.0
|
| 33 |
+
- **Checkpoint file:** `georay_ckpt.pt`
|
| 34 |
+
- **Checkpoint SHA-256:** `7bbe1fc5f04d471d1ecd843be94b73571d8e6aded9a407965277cf37a1a3f330`
|
| 35 |
+
|
| 36 |
+
The released checkpoint contains GeoRay's trainable modules. Frozen backbone weights are obtained separately and remain subject to their upstream licenses.
|
| 37 |
+
|
| 38 |
+
## Results
|
| 39 |
+
|
| 40 |
+
On 26 held-out US3D tiles, GeoRay reports:
|
| 41 |
+
|
| 42 |
+
| Metric | Result |
|
| 43 |
+
|:--|--:|
|
| 44 |
+
| Absolute MAE ↓ | **2.99 m** |
|
| 45 |
+
| Coverage ↑ | **91.9%** |
|
| 46 |
+
| Completeness-aware accuracy gain ↑ | **+46.4 points** over the strongest compliant feed-forward baseline |
|
| 47 |
+
| Model-forward time ↓ | **24.0 s / tile** in the reported setting |
|
| 48 |
+
|
| 49 |
+
See the [paper](https://arxiv.org/abs/2608.29680) for the complete evaluation protocol, cross-dataset and cross-city results, and baseline definitions.
|
| 50 |
+
|
| 51 |
+
## Installation
|
| 52 |
+
|
| 53 |
+
```bash
|
| 54 |
+
git clone https://github.com/HIT-SIRS/GeoRay.git
|
| 55 |
+
cd GeoRay
|
| 56 |
+
conda create -n georay python=3.11 -y
|
| 57 |
+
conda activate georay
|
| 58 |
+
|
| 59 |
+
# Install the CUDA-enabled PyTorch 2.7.x build appropriate for your system first.
|
| 60 |
+
pip install -r requirements.txt
|
| 61 |
+
pip install -e .
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
Follow the repository [installation guide](https://github.com/HIT-SIRS/GeoRay#installation) to prepare the pinned VGGT code and external frozen weights.
|
| 65 |
+
|
| 66 |
+
## Download
|
| 67 |
+
|
| 68 |
+
After this model repository is published, download the checkpoint with:
|
| 69 |
+
|
| 70 |
+
```bash
|
| 71 |
+
hf download hit-sirs/GeoRay georay_ckpt.pt --local-dir checkpoints/GeoRay
|
| 72 |
+
export GEORAY_CKPT="$PWD/checkpoints/GeoRay/georay_ckpt.pt"
|
| 73 |
+
```
|
| 74 |
+
|
| 75 |
+
## Inference
|
| 76 |
+
|
| 77 |
+
```bash
|
| 78 |
+
python scripts/infer.py \
|
| 79 |
+
--ckpt "$GEORAY_CKPT" \
|
| 80 |
+
--config configs/train.yaml \
|
| 81 |
+
--input /path/to/tile.tar \
|
| 82 |
+
--out_dir outputs/tile
|
| 83 |
+
|
| 84 |
+
python scripts/export_dsm.py \
|
| 85 |
+
--infer_out outputs/tile \
|
| 86 |
+
--out_dir outputs/tile \
|
| 87 |
+
--input /path/to/tile.tar
|
| 88 |
+
```
|
| 89 |
+
|
| 90 |
+
The input WebDataset format and RPC requirements are documented in the [GeoRay README](https://github.com/HIT-SIRS/GeoRay#data-format).
|
| 91 |
+
|
| 92 |
+
## Intended use
|
| 93 |
+
|
| 94 |
+
GeoRay is intended for research on satellite multi-view 3D reconstruction, absolute-frame surface-height estimation, uncertainty-aware photogrammetry, and related geospatial studies.
|
| 95 |
+
|
| 96 |
+
## Limitations
|
| 97 |
+
|
| 98 |
+
- The released model expects the preprocessing and three-view WebDataset structure documented by the project.
|
| 99 |
+
- Accuracy is not guaranteed for unseen sensors, geographic regions, RPC conventions, or height datums; validate outputs before downstream use.
|
| 100 |
+
- The exported product is a predicted digital surface model, not a surveyed digital terrain model.
|
| 101 |
+
- Predictive uncertainty is a model estimate and must not be treated as a formal safety guarantee.
|
| 102 |
+
- Inference requires an NVIDIA GPU and separately obtained frozen backbone weights.
|
| 103 |
+
|
| 104 |
+
## Third-party components
|
| 105 |
+
|
| 106 |
+
GeoRay source code and its lightweight checkpoint are released under Apache-2.0. VGGT-1B, MoGe-2, and other third-party components are governed by their own licenses. In particular, users must review the applicable VGGT weight license before use or redistribution.
|
| 107 |
+
|
| 108 |
+
## Citation
|
| 109 |
+
|
| 110 |
+
```bibtex
|
| 111 |
+
@article{dong2026georay,
|
| 112 |
+
title = {GeoRay: Gauge-Aware Feed-Forward Satellite 3D Reconstruction in the Geodetic Frame},
|
| 113 |
+
author = {Dong, Zhe and Wu, Wanqing and Sun, Yuzhe and Jiang, Haochen and Ma, Yuchen and Ren, Lecheng and Liu, Tianzhu and Gu, Yanfeng},
|
| 114 |
+
journal = {arXiv preprint arXiv:2608.29680},
|
| 115 |
+
year = {2026}
|
| 116 |
+
}
|
| 117 |
+
```
|
| 118 |
+
|