project-oceania/timm-eva02-large-m38m-ft-planktonzilla

EVA02-Large fine-tuned on Planktonzilla-17M

Supervised plankton image classifier released with the paper Planktonzilla: Multimodal dataset and models for understanding plankton ecosystems (Contreras, Valenzuela, Martí & Sanchez-Pi, 2026). The paper's controlled supervised-vs-CLIP comparison finds that fully supervised classifiers match or exceed CLIP-style image–text training on the same backbone; this EVA02-Large model is the supervised classifier released for direct use.

• Backbone: eva02_large_patch14_448 (EVA02-Large, patch 14, 448×448), mim_m38m pretraining
• Head: single-label linear classifier (TimmWrapperForImageClassification)
• Classes: 385, labelled by full taxonomic lineage (e.g. animalia annelida polychaeta phyllodocida tomopteridae)
• Trained on: the plankton subset of Planktonzilla-17M
• Input: 3×448×448, bicubic, crop_mode=squash; mean [0.4815, 0.4578, 0.4082], std [0.2686, 0.2613, 0.2758]

Results (validation split)

Metric	Value
Top-1 accuracy	0.9499
Macro-F1	0.8861
Macro precision	0.9033
Macro recall	0.8785

Usage

from transformers import AutoModelForImageClassification, AutoImageProcessor
from PIL import Image

repo = "project-oceania/timm-eva02-large-m38m-ft-planktonzilla"
model = AutoModelForImageClassification.from_pretrained(repo)
processor = AutoImageProcessor.from_pretrained(repo)

image = Image.open("plankton.jpg").convert("RGB")
inputs = processor(image, return_tensors="pt")
logits = model(**inputs).logits
pred = logits.argmax(-1).item()
print(model.config.label_names[pred])

Intended use & limitations

Intended for taxonomic classification of plankton imagery similar to the thirteen imaging systems consolidated in Planktonzilla-17M. Predictions are limited to the 385 taxa seen in training; only a small fraction of labels reach species level, so read predictions at the deepest valid taxonomic rank. Out-of-distribution imagers and long-tail taxa degrade accuracy. Not a sole basis for ecological or biodiversity decisions without expert review.

Citation

If you use Planktonzilla in your research, please cite as:

A. G. Contreras Montanares, L. Valenzuela, L. Martí, and N. Sanchez‑Pi, Planktonzilla: Multimodal dataset and models for understanding plankton ecosystems, Inria Chile Research Center, Tech. Rep., May 2026, doi: 10.48550/arXiv.2606.00080, arXiv: 2606.00080 [cs.CV]. url: https://arxiv.org/abs/2606.00080

@techreport{contrerasmontanares:hal-05621003,
  title         = {Planktonzilla: {M}ultimodal dataset and models for understanding plankton ecosystems},
  author        = {Contreras Montanares, Alan Gerson and Valenzuela, Luis and Mart{\'i}, Luis and Sanchez-Pi, Nayat},
  year          = 2026,
  month         = {May},
  keywords      = {Explainable AI; XAI ; Plankton Classification ; CLIPS ; Multimodal Classification},
  eprinttype    = {arxiv},
  hal_id        = {hal-05621003},
  hal_version   = {v1},
  eprint        = {2606.00080},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2606.00080},
  doi           = {10.48550/arXiv.2606.00080},
  institution   = {Inria Chile Research Center},
}

Model produced by Project OcéanIA / Inria Chile Research Center. Code: Inria-Chile/planktonzilla.