File size: 4,274 Bytes
ef44b9c d1c3c85 1433cc7 ef44b9c 1433cc7 ef44b9c d1c3c85 ef44b9c d1c3c85 ef44b9c 6ba5bc9 d1c3c85 6ba5bc9 d1c3c85 1433cc7 d1c3c85 1ca6f58 d1c3c85 1433cc7 d1c3c85 ef44b9c d1c3c85 ef44b9c d1c3c85 1433cc7 d1c3c85 1433cc7 d1c3c85 ef44b9c d1c3c85 1433cc7 d1c3c85 1433cc7 d1c3c85 1433cc7 d1c3c85 1433cc7 d1c3c85 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 | ---
pipeline_tag: image-segmentation
license: mit
base_model: facebook/mask2former-swin-tiny-coco-panoptic
library_name: zeromodels
tags:
- keras
- zeromodels
- mask2former
- panoptic-segmentation
- image-segmentation
- arxiv:2112.01527
- pytorch
- jax
- tf
---
## ***See [our collection](https://huggingface.co/collections/zeromodels/mask2former-6a8eaf66faaf81a53d54fa03) for all versions of Mask2Former.***
# Run Mask2Former with Keras 3: JAX, PyTorch, or TensorFlow
[](https://github.com/IMvision12/ZeroModels) [](https://imvision12.github.io/ZeroModels/mask2former/) [](https://huggingface.co/collections/zeromodels/mask2former-6a8eaf66faaf81a53d54fa03)
# zeromodels/mask2former-swin-tiny-coco-panoptic
Paper: [Masked-attention Mask Transformer for Universal Image Segmentation (arXiv:2112.01527)](https://arxiv.org/abs/2112.01527) · [HF Papers](https://huggingface.co/papers/2112.01527)
Mask2Former improves MaskFormer with masked attention in the transformer decoder, restricting cross-attention to predicted mask regions for sharper boundaries and stronger universal segmentation.
For more details on the model, please go to the upstream [model card](https://huggingface.co/facebook/mask2former-swin-tiny-coco-panoptic).
Pure-**Keras 3** conversion of [`facebook/mask2former-swin-tiny-coco-panoptic`](https://huggingface.co/facebook/mask2former-swin-tiny-coco-panoptic) for [zeromodels](https://github.com/IMvision12/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
This is a **panoptic** checkpoint (`Mask2FormerUniversalSegment`) (trained for panoptic; architecture is universal).
## ✨ Quick start
```python
import os
os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
from PIL import Image
from zeromodels.models.mask2former import Mask2FormerUniversalSegment, Mask2FormerImageProcessor
model = Mask2FormerUniversalSegment.from_weights("zeromodels/mask2former-swin-tiny-coco-panoptic")
processor = Mask2FormerImageProcessor.from_weights("zeromodels/mask2former-swin-tiny-coco-panoptic")
image = Image.open("your_image.jpg").convert("RGB")
output = model(processor(image)["pixel_values"], training=False)
result = processor.post_process_panoptic_segmentation(
output, target_size=(image.height, image.width)
)
print(result["segmentation"].shape)
```
Load any Mask2Former variant the same way with `from_weights("zeromodels/<variant>")`:
| Variant | Hub | Task |
|---|---|---|
| `mask2former-swin-tiny-coco-instance` | [`zeromodels/mask2former-swin-tiny-coco-instance`](https://huggingface.co/zeromodels/mask2former-swin-tiny-coco-instance) | instance |
| `mask2former-swin-small-coco-instance` | [`zeromodels/mask2former-swin-small-coco-instance`](https://huggingface.co/zeromodels/mask2former-swin-small-coco-instance) | instance |
| `mask2former-swin-base-coco-instance` | [`zeromodels/mask2former-swin-base-coco-instance`](https://huggingface.co/zeromodels/mask2former-swin-base-coco-instance) | instance |
| `mask2former-swin-large-coco-instance` | [`zeromodels/mask2former-swin-large-coco-instance`](https://huggingface.co/zeromodels/mask2former-swin-large-coco-instance) | instance |
| `mask2former-swin-tiny-coco-panoptic` | [`zeromodels/mask2former-swin-tiny-coco-panoptic`](https://huggingface.co/zeromodels/mask2former-swin-tiny-coco-panoptic) | panoptic |
| `mask2former-swin-tiny-ade-semantic` | [`zeromodels/mask2former-swin-tiny-ade-semantic`](https://huggingface.co/zeromodels/mask2former-swin-tiny-ade-semantic) | semantic |
## Tips
- Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
- The task suffix is what the checkpoint was trained for; post-process accordingly.
- See [Mask2Former docs]({DOCS_URL}) and [Loading Weights](https://imvision12.github.io/ZeroModels/loading_weights/).
- Community / upstream weights: `Mask2FormerUniversalSegment.from_weights("hf:facebook/mask2former-swin-tiny-coco-panoptic")`.
## Special Thanks
A huge thank you to the Facebook AI Research Mask2Former authors for creating and releasing these models.
License: MIT.
|