README.md

---
pipeline_tag: image-segmentation
license: other
license_link: https://github.com/NVlabs/SegFormer/blob/master/LICENSE
base_model: nvidia/segformer-b4-finetuned-cityscapes-1024-1024
library_name: zeromodels
tags:
- keras
- zeromodels
- segformer
- semantic-segmentation
- image-segmentation
- cityscapes
- arxiv:2105.15203
- pytorch
- jax
- tf
---

## ***See [our collection](https://huggingface.co/collections/zeromodels/segformer-6a8eaf78872e857fd77bad81) for all versions of SegFormer.***

# Run SegFormer with Keras 3: JAX, PyTorch, or TensorFlow

[![GitHub](https://img.shields.io/badge/GitHub-ZeroModels-black?logo=github)](https://github.com/IMvision12/ZeroModels) [![Docs](https://img.shields.io/badge/Docs-SegFormer-blue)](https://imvision12.github.io/ZeroModels/segformer/) [![Collection](https://img.shields.io/badge/HF-SegFormer%20collection-yellow)](https://huggingface.co/collections/zeromodels/segformer-6a8eaf78872e857fd77bad81)

# zeromodels/segformer_b4_cityscapes_1024

Paper: [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers (arXiv:2105.15203)](https://arxiv.org/abs/2105.15203) · [HF Papers](https://huggingface.co/papers/2105.15203)

SegFormer pairs a hierarchical transformer encoder (MiT) with a lightweight all-MLP decoder. The encoder produces features at four scales, and the decoder upsamples, concatenates, and projects them. Sequence-reduction attention and no positional encoding keep it efficient and resolution-flexible.

For more details on the model, please go to NVIDIA's original [model card](https://huggingface.co/nvidia/segformer-b4-finetuned-cityscapes-1024-1024).

Pure-**Keras 3** conversion of [`nvidia/segformer-b4-finetuned-cityscapes-1024-1024`](https://huggingface.co/nvidia/segformer-b4-finetuned-cityscapes-1024-1024) for [zeromodels](https://github.com/IMvision12/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.

This is a **semantic segmentation** checkpoint (`SegFormerSemanticSegment`) for **Cityscapes** (19 classes, MiT-B4, 1024px).

## ✨ Quick start

```python
import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
from zeromodels.models.segformer import (
    SegFormerSemanticSegment,
    SegFormerImageProcessor,
)

model = SegFormerSemanticSegment.from_weights("zeromodels/segformer_b4_cityscapes_1024")
processor = SegFormerImageProcessor.from_weights("zeromodels/segformer_b4_cityscapes_1024")

image = Image.open("your_image.jpg").convert("RGB")
output = model(processor(image)["pixel_values"], training=False)
result = processor.post_process_semantic_segmentation(
    output, target_size=(image.height, image.width)
)
print(result["unique_classes"], result["class_names"])
```

Load any SegFormer variant the same way with `from_weights("zeromodels/<variant>")`:

| Variant | Hub | Dataset | Res |
|---|---|---|---|
| `segformer_b0_ade_512` | [`zeromodels/segformer_b0_ade_512`](https://huggingface.co/zeromodels/segformer_b0_ade_512) | ADE20K | 512 |
| `segformer_b1_ade_512` | [`zeromodels/segformer_b1_ade_512`](https://huggingface.co/zeromodels/segformer_b1_ade_512) | ADE20K | 512 |
| `segformer_b2_ade_512` | [`zeromodels/segformer_b2_ade_512`](https://huggingface.co/zeromodels/segformer_b2_ade_512) | ADE20K | 512 |
| `segformer_b3_ade_512` | [`zeromodels/segformer_b3_ade_512`](https://huggingface.co/zeromodels/segformer_b3_ade_512) | ADE20K | 512 |
| `segformer_b4_ade_512` | [`zeromodels/segformer_b4_ade_512`](https://huggingface.co/zeromodels/segformer_b4_ade_512) | ADE20K | 512 |
| `segformer_b5_ade_640` | [`zeromodels/segformer_b5_ade_640`](https://huggingface.co/zeromodels/segformer_b5_ade_640) | ADE20K | 640 |
| `segformer_b0_cityscapes_768` | [`zeromodels/segformer_b0_cityscapes_768`](https://huggingface.co/zeromodels/segformer_b0_cityscapes_768) | Cityscapes | 768 |
| `segformer_b0_cityscapes_1024` | [`zeromodels/segformer_b0_cityscapes_1024`](https://huggingface.co/zeromodels/segformer_b0_cityscapes_1024) | Cityscapes | 1024 |
| `segformer_b1_cityscapes_1024` | [`zeromodels/segformer_b1_cityscapes_1024`](https://huggingface.co/zeromodels/segformer_b1_cityscapes_1024) | Cityscapes | 1024 |
| `segformer_b2_cityscapes_1024` | [`zeromodels/segformer_b2_cityscapes_1024`](https://huggingface.co/zeromodels/segformer_b2_cityscapes_1024) | Cityscapes | 1024 |
| `segformer_b3_cityscapes_1024` | [`zeromodels/segformer_b3_cityscapes_1024`](https://huggingface.co/zeromodels/segformer_b3_cityscapes_1024) | Cityscapes | 1024 |
| `segformer_b4_cityscapes_1024` | [`zeromodels/segformer_b4_cityscapes_1024`](https://huggingface.co/zeromodels/segformer_b4_cityscapes_1024) | Cityscapes | 1024 |
| `segformer_b5_cityscapes_1024` | [`zeromodels/segformer_b5_cityscapes_1024`](https://huggingface.co/zeromodels/segformer_b5_cityscapes_1024) | Cityscapes | 1024 |

## Tips

- Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
- Prefer `SegFormerImageProcessor.from_weights(...)` so the resize matches the variant (ADE B5 is 640; Cityscapes is often 1024).
- See [SegFormer docs](https://imvision12.github.io/ZeroModels/segformer/) and [Loading Weights](https://imvision12.github.io/ZeroModels/loading_weights/).
- Community / upstream safetensors still work via the `hf:` prefix, e.g. `SegFormerSemanticSegment.from_weights("hf:nvidia/segformer-b4-finetuned-cityscapes-1024-1024")`.

## Special Thanks

A huge thank you to the NVIDIA SegFormer authors for creating and releasing these models.

License: see the [NVIDIA SegFormer LICENSE](https://github.com/NVlabs/SegFormer/blob/master/LICENSE) (Hub tag: `other`).