File size: 4,302 Bytes
4e189b9 f37790f 9d58440 4e189b9 9d58440 f37790f 4e189b9 f37790f 4e189b9 c737d2b f37790f c737d2b f37790f 9d58440 f37790f 9d58440 4e189b9 f37790f 4e189b9 f37790f 9d58440 f37790f 9d58440 f37790f 4e189b9 f37790f 9d58440 f37790f 9d58440 f37790f 9d58440 f37790f 9d58440 f37790f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 | ---
pipeline_tag: object-detection
license: apache-2.0
base_model: PekingU/rtdetr_r50vd_coco_o365
library_name: zeromodels
tags:
- keras
- zeromodels
- rt-detr
- detr
- object-detection
- arxiv:2304.08069
- pytorch
- jax
- tf
---
## ***See [our collection](https://huggingface.co/collections/zeromodels/rt-detr-v1-and-v2-6a8eaf859a918a955e683778) for all versions of RT-DETR.***
# Run RT-DETR with Keras 3: JAX, PyTorch, or TensorFlow
[](https://github.com/IMvision12/ZeroModels) [](https://imvision12.github.io/ZeroModels/rt_detr/) [](https://huggingface.co/collections/zeromodels/rt-detr-v1-and-v2-6a8eaf859a918a955e683778)
# zeromodels/rtdetr-r50vd-coco-o365
Paper: [DETRs Beat YOLOs on Real-time Object Detection (arXiv:2304.08069)](https://arxiv.org/abs/2304.08069) · [HF Papers](https://huggingface.co/papers/2304.08069)
RT-DETR was the first DETR-style detector to beat YOLO on the real-time speed/accuracy tradeoff. It pairs a ResNet-vd backbone with a hybrid encoder that decouples intra-scale attention from cross-scale fusion, then feeds IoU-aware selected queries into a deformable decoder. It is NMS-free: a fixed set of queries, constant inference cost, no NMS threshold to tune.
For more details on the model, please go to PekingU's original [model card](https://huggingface.co/PekingU/rtdetr_r50vd_coco_o365).
Pure-**Keras 3** conversion of [`PekingU/rtdetr_r50vd_coco_o365`](https://huggingface.co/PekingU/rtdetr_r50vd_coco_o365) for [zeromodels](https://github.com/IMvision12/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
This is an **object detection** checkpoint (`RTDETRDetect`) on COCO (ResNet-50-vd).
## ✨ Quick start
```python
import os
os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
from PIL import Image
from zeromodels.models.rt_detr import RTDETRDetect, RTDETRImageProcessor
model = RTDETRDetect.from_weights("zeromodels/rtdetr-r50vd-coco-o365")
processor = RTDETRImageProcessor.from_weights("zeromodels/rtdetr-r50vd-coco-o365")
image = Image.open("your_image.jpg").convert("RGB")
inputs = processor(image)
output = model(inputs["pixel_values"], training=False)
results = processor.post_process_object_detection(
output, threshold=0.5, target_sizes=[(image.height, image.width)]
)[0]
for score, name, box in zip(
results["scores"], results["label_names"], results["boxes"]
):
print(f"{name}: {float(score):.3f} {box}")
```
Load any RT-DETR v1 variant the same way with `from_weights("zeromodels/<variant>")`:
| Variant | Hub | Backbone |
|---|---|---|
| `rtdetr-r18vd` | [`zeromodels/rtdetr-r18vd`](https://huggingface.co/zeromodels/rtdetr-r18vd) | ResNet-18-vd |
| `rtdetr-r18vd-coco-o365` | [`zeromodels/rtdetr-r18vd-coco-o365`](https://huggingface.co/zeromodels/rtdetr-r18vd-coco-o365) | ResNet-18-vd (COCO+O365) |
| `rtdetr-r34vd` | [`zeromodels/rtdetr-r34vd`](https://huggingface.co/zeromodels/rtdetr-r34vd) | ResNet-34-vd |
| `rtdetr-r50vd` | [`zeromodels/rtdetr-r50vd`](https://huggingface.co/zeromodels/rtdetr-r50vd) | ResNet-50-vd |
| `rtdetr-r50vd-coco-o365` | [`zeromodels/rtdetr-r50vd-coco-o365`](https://huggingface.co/zeromodels/rtdetr-r50vd-coco-o365) | ResNet-50-vd (COCO+O365) |
| `rtdetr-r101vd` | [`zeromodels/rtdetr-r101vd`](https://huggingface.co/zeromodels/rtdetr-r101vd) | ResNet-101-vd |
| `rtdetr-r101vd-coco-o365` | [`zeromodels/rtdetr-r101vd-coco-o365`](https://huggingface.co/zeromodels/rtdetr-r101vd-coco-o365) | ResNet-101-vd (COCO+O365) |
## Tips
- Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
- `RTDETRImageProcessor` keeps `do_normalize=False` by default (rescaled `[0, 1]` input, matching upstream).
- See [RT-DETR docs](https://imvision12.github.io/ZeroModels/rt_detr/) and [Loading Weights](https://imvision12.github.io/ZeroModels/loading_weights/).
- Community / upstream safetensors still work via the `hf:` prefix, e.g. `RTDETRDetect.from_weights("hf:PekingU/rtdetr_r50vd_coco_o365")`.
## Special Thanks
A huge thank you to the RT-DETR authors (Baidu / PekingU) for creating and releasing these models.
License: Apache 2.0.
|