---
license: mit
base_model: ESFM/ESFM_s_wm_pre
tags:
- weather-forecasting
- climate
- earth-system
- station-data
- weather-5k
- missing-data
- pytorch
---

# Model Card for ESFM/ESFM_s_wm_w5k_lt24h

ESFM small checkpoint directly finetuned for 24-hour forecasting on the imputed Weather-5K global station benchmark with variable and spatial masking.

> **Checkpoint selection:** Direct 24-hour forecasts on the documented Weather-5K station layout and variables.

## Model Details

- **Developed by:** The ESFM research team, with the full contributor and author lists linked below.
- **Shared by:** [ESFM on Hugging Face](https://huggingface.co/ESFM)
- **Model type:** Deterministic Weather-5K station checkpoint; modified 3D Swin-UNet encoder-decoder
- **Model size:** Approximately 115 million parameters
- **Masking protocol:** Station variable and spatial masking
- **Ensemble members:** 1
- **Forecast lead time:** 24 hours
- **License:** MIT
- **Repository:** https://huggingface.co/ESFM/ESFM_s_wm_w5k_lt24h

### Model Sources

- **Code:** https://github.com/swiss-ai/ESFM
- **Paper:** https://arxiv.org/abs/2605.00850
- **Project page:** https://swiss-ai.github.io/ESFM/

The paper is currently available as an arXiv preprint.

## Uses

### Direct Use

Direct 24-hour forecasts on the documented Weather-5K station layout and variables.

### Downstream Use

Day-ahead station benchmark research and adaptation to related imputed station products.

### Out-of-Scope Use

Do not generate 24-hour forecasts by rolling the 6-hour Weather-5K model when comparing this released checkpoint; this model is direct-trained. It is not an operational station forecast service.

## Bias, Risks, and Limitations

Weather-5K includes ERA5-based infilling and its evaluation differs from raw-station settings. Sea-level pressure is reported as comparatively weak, and benchmark comparisons require accounting for different input histories.

All ESFM checkpoints are research artifacts. Validate forecasts for the target variables, stations or regions, seasons, lead times, missingness pattern, and decision context. Do not use the model as the sole basis for safety-critical decisions.

## How to Get Started

The checkpoint is not packaged as a Hugging Face Transformers from_pretrained model. Construct ESFM with the matching config and load the state dictionary. The released notebook demonstrates checkpoint download and architecture construction.

```python
from huggingface_hub import hf_hub_download

model_name = "ESFM_s_wm_w5k_lt24h"
weights_path = hf_hub_download(
    repo_id=f"ESFM/{model_name}",
    filename=f"{model_name}.safetensors",
)
print(weights_path)
```

Use `configs/config_ESFM_s_wm_w5k_lt24h.yaml` with locally preprocessed Weather-5K data and the matching inference script.

Clone the implementation first:

```bash
git clone https://github.com/swiss-ai/ESFM.git
cd ESFM
```

## Training Details

### Training Data

Weather-5K hourly data from 5,672 NCEI stations over 2014-2023. Short gaps are interpolated and remaining gaps are filled from ERA5; the manuscript uses 2014-2021 for training and 2023 for testing.

Preprocessing and the exact variable registry are documented in the [ESFM repository](https://github.com/swiss-ai/ESFM), [preprocessing repository](https://github.com/swiss-ai/SwissClim_data_processing_scripts), and [preprint](https://arxiv.org/abs/2605.00850).

### Training Procedure

Initialized independently from the masked ERA5 lineage and finetuned directly for a 24-hour target for 20,000 steps on 16 GPUs. The station protocol uses variable and spatial masking.

- **Nominal architecture:** ESFM small, approximately 115M parameters
- **Software environment:** PyTorch/Lightning in the released NVIDIA PhysicsNeMo 25.03 container; lightning==2.5.1 is pinned in the Dockerfile
- **Training regime:** Lightning `precision="32-true"` with FP32 parameters and optimizer state; selected model forward operations use CUDA BF16 autocasting through `torch.autocast(dtype=torch.bfloat16)`.

## Evaluation

The manuscript reports that ESFM achieves the strongest overall 24-hour MAE among the listed Weather-5K benchmark models, while specialized models remain stronger for some variables. Numerical tables remain in the linked preprint.

Detailed numerical results are intentionally not copied into this card.

## Technical Specifications

ESFM uses variable-specific tokenization, axial attention across variables, perceiver aggregation, a 3D Swin-UNet backbone, and a decoder queried at target pressure levels. Missing patches are represented by learnable NaN tokens. Resolution-specific tokenizers and station mapping are enabled for the relevant sparse-data configs. The ensemble checkpoint additionally applies member-conditioned AdaLN-Zero after the backbone.

## Environmental Impact

- **Hardware type:** NVIDIA GH200 systems with four GPUs per node. This experiment was run on four nodes, totaling 16 GPUs.
- **Total training time:** 18 hours
- **Compute location:** Training used CSCS Alps infrastructure.

## Citation

```bibtex
@misc{ozdemir2026esfm,
  title={Earth System Foundation Model (ESFM): A unified framework for heterogeneous data integration and forecasting},
  author={Firat Ozdemir and Yun Cheng and Salman Mohebi and Fanny Lehmann and Simon Adamov and Zhenyi Zhang and Leonardo Trentini and Dana Grund and Oliver Fuhrer and Torsten Hoefler and Siddhartha Mishra and Sebastian Schemm and Benedikt Soja and Mathieu Salzmann},
  year={2026},
  eprint={2605.00850},
  archivePrefix={arXiv},
  primaryClass={physics.ao-ph},
  url={https://arxiv.org/abs/2605.00850}
}
```

## More Information

- [ESFM code and configurations](https://github.com/swiss-ai/ESFM)
- [Current ESFM preprint](https://arxiv.org/abs/2605.00850)
- [Data preprocessing scripts](https://github.com/swiss-ai/SwissClim_data_processing_scripts)
- [Project page](https://swiss-ai.github.io/ESFM/)

## Model Card Contact

Firat Ozdemir: `firat.ozdemir@sdsc.ethz.ch`