Add files using upload-large-folder tool

README.md

@@ -0,0 +1,191 @@
+---
+license: cc-by-4.0
+library_name: timm
+pipeline_tag: image-feature-extraction
+tags:
+  - image-feature-extraction
+  - re-identification
+  - metric-learning
+  - wildlife
+  - dinov2
+  - gorilla
+datasets:
+  - gorilla-watch/Gorilla-SPAC-Wild
+model-index:
+- name: GorillaWatch-DINOv2-Giant
+  results:
+    - task:
+        type: image-feature-extraction
+        name: facial gorilla re-identification
+      dataset:
+        type: gorilla-watch/Gorilla-SPAC-Wild
+        name: Gorilla-SPAC-Wild
+        config: face_with_body
+        split: test
+      metrics:
+        - name: Micro Accuracy
+          type: accuracy
+          value: 0.5554
+        - name: Macro Accuracy
+          type: accuracy
+          value: 0.4629
+        - name: Tracklet Micro Accuracy
+          type: accuracy
+          value: 0.6121
+        - name: Tracklet Macro Accuracy
+          type: accuracy
+          value: 0.4451
+    - task:
+        type: image-feature-extraction
+        name: facial gorilla re-identification
+      dataset:
+        type: gorilla-watch/Gorilla-Zoo-Berlin
+        name: Gorilla-Zoo-Berlin
+        config: face_with_body
+        split: test
+      metrics:
+        - name: Micro Accuracy
+          type: accuracy
+          value: 0.7657
+        - name: Macro Accuracy
+          type: accuracy
+          value: 0.759
+        - name: Tracklet Micro Accuracy
+          type: accuracy
+          value: 0.8218
+        - name: Tracklet Macro Accuracy
+          type: accuracy
+          value: 0.8044
+---
+
+# GorillaWatch-DINOv2-Giant
+
+Gorilla re-identification model from **[GorillaWatch: An Automated System for In-the-Wild Gorilla Re-Identification and Population Monitoring](https://arxiv.org/abs/2512.07776)** (WACV 2026). Further project details can be found [here](https://gorilla-watch.github.io/).
+
+A `vit_giant_patch14_dinov2.lvd142m` DINOv2 backbone fine-tuned with hard-mining triplet loss on
+[Gorilla-SPAC-Wild](https://huggingface.co/datasets/gorilla-watch/Gorilla-SPAC-Wild), projecting to a
+**256-dimensional embedding**. Identification is done by k-NN retrieval against a
+gallery of embeddings, not by classification. The model has no fixed identity vocabulary, to enable
+generalisation to individuals unseen during training.
+
+| | |
+|---|---|
+| Backbone | `vit_giant_patch14_dinov2.lvd142m` |
+| Input resolution | 518×518 |
+| Embedding dimension | 256 |
+| Parameters | 1136.9M |
+| Training data | Gorilla-SPAC-Wild (`face_with_body`) |
+
+## Preprocessing
+
+> [!IMPORTANT]
+> This model does **not** use timm's default DINOv2 transform. It expects a **square resize**
+> (which only preserves the aspect ratio when the input images are already squared, which is the case in our datasets) and normalization with **mean = std = 0.5**, not the ImageNet
+> statistics reported in the backbone's `default_cfg`. Using timm's default transform produces
+> incorrect embeddings.
+
+```python
+transforms.Compose([
+    transforms.Resize((518, 518)),
+    transforms.ToTensor(),
+    transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]),
+])
+```
+
+`modeling.py` in this repository exposes this as `model.get_transform()`.
+
+## Usage
+
+Here we provide a minimal setup to use the model for feature extraction. 
+
+Requires `torch`, `timm`, `safetensors`, `huggingface_hub` and `torchvision`.
+
+```python
+import sys, torch
+from huggingface_hub import snapshot_download
+from PIL import Image
+
+# Fetch weights, config and the self-contained modeling.py in one go
+local_dir = snapshot_download("gorilla-watch/GorillaWatch-DINOv2-Giant")
+sys.path.insert(0, local_dir)
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+from modeling import load_model
+
+model = load_model(local_dir, device=device)   # already in eval mode
+transform = model.get_transform()
+
+image = Image.open("gorilla.png").convert("RGB")
+with torch.no_grad():
+    embedding = model(transform(image).unsqueeze(0).to(model.device))  # (1, 256)
+```
+
+`load_model` also accepts the repo id directly (`load_model("gorilla-watch/GorillaWatch-DINOv2-Giant")`) if you would rather not
+manage a local directory.
+
+Identity assignment uses **k-NN with k=5 under Euclidean distance** against a gallery of embeddings.
+The paper's protocol masks out gallery entries from the same encounter (same camera on the same
+date) to avoid trivially easy matches. The full evaluation code can be found in our [GitHub Repo](https://github.com/gorilla-watch/gorillawatch).
+
+## Training
+
+Fine-tuned from the upstream `vit_giant_patch14_dinov2.lvd142m` DINOv2 checkpoint.
+
+| Hyperparameter | Value |
+|---|---|
+| Loss | Online triplet, hard mining, Euclidean, margin 0.647 |
+| Optimizer | AdamW (β=0.9/0.999, ε=1e-7) |
+| Learning rate | 1.9e-7, cosine annealing to 1e-7 |
+| Batch size | 8 (effective 48 via 6 gradient accumulation steps) |
+| Regularization | L2 = 0.0059, L2-SP = 1.3e-5 |
+| Epochs | 100 max, best-validation-loss checkpoint retained |
+| Precision | AMP (fp16 autocast, fp32 master weights) |
+| Seed | 42 |
+
+The code used to train these models can be found in our [Github Repository](https://github.com/gorilla-watch/gorillawatch).
+
+## Results
+
+k-NN retrieval accuracy (k=5, Euclidean distance). Gallery entries from the same encounter (same camera on the same date) are masked out, so every match is made across encounters. Macro accuracy averages over identities and is the harder number: it weights rarely-seen individuals equally with frequently-seen ones.
+
+### In-domain: Gorilla-SPAC-Wild
+
+Test split of [Gorilla-SPAC-Wild](https://huggingface.co/datasets/gorilla-watch/Gorilla-SPAC-Wild), the distribution the model was fine-tuned on.
+
+| Protocol | Micro accuracy | Macro accuracy |
+|---|---|---|
+| Per image | 0.5554 | 0.4629 |
+| Per tracklet (average pooling) | 0.6121 | 0.4451 |
+
+### Out-of-distribution: Gorilla-Zoo-Berlin
+
+[Gorilla-Zoo-Berlin](https://huggingface.co/datasets/gorilla-watch/Gorilla-Zoo-Berlin) is a **zero-shot domain-transfer test**: the model is applied to footage recorded in the Berlin Zoo, with no fine-tuning on it, so enclosure, lighting, camera hardware and the individuals themselves are all unseen. The numbers are still higher, since the amount of individuals is much lower than in the SPAC dataset. This evaluation clearly shows that the model is able to generalize to new, unseen populations.
+
+| Protocol | Micro accuracy | Macro accuracy |
+|---|---|---|
+| Per image | 0.7657 | 0.7590 |
+| Per tracklet (average pooling) | 0.8218 | 0.8044 |
+
+
+## Provenance
+
+These weights are bit-identical conversions from the `.pth` files created in the training process. They were converted to the `model.safetensors` format for better integration with HuggingFace.
+
+## License
+
+This model is released under the **CC-BY-4.0 License**.
+
+## Citation
+
+```bibtex
+@inproceedings{GorillaWatch2026,
+  title={GorillaWatch: An Automated System for In-the-Wild Gorilla Re-Identification and Population Monitoring},
+  booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
+  author={Maximilian Schall and Felix Leonard Kn\"ofel and Noah Elias K\"onig and Jan Jonas Kubeler and Maximilian von Klinski and Joan Wilhelm Linnemann and Xiaoshi Liu and Iven Jelle Schlegelmilch and Ole Woyciniuk and Alexandra Schild and Dante Wasmuht and Magdalena Bermejo Espinet and German Illera Basas and Gerard de Melo},
+  year={2026},
+  archivePrefix={arXiv},
+  eprint={2512.07776}
+}
+```
+
+## Acknowledgements
+The project on which this report is based was funded by the Federal Ministry of Research, Technology and Space under the funding code “KI-Servicezentrum Berlin-Brandenburg” 16IS22092. We acknowledge the support of Sabine Plattner African Charities (SPAC) for their funding to this research. We are grateful to Zoo Berlin for their expert assistance and facility access. This collaboration enabled the development of AI tools capable of being deployed in the wild to directly support gorilla conservation. The responsibility for the content of this publication remains with the authors.

config.json

@@ -0,0 +1,24 @@
+{
+  "backbone_name": "vit_giant_patch14_dinov2.lvd142m",
+  "img_size": 518,
+  "embedding_size": 256,
+  "embedding_id": "linear",
+  "pool_mode": "none",
+  "dropout_p": 0.0,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "resize_mode": "square",
+  "interpolation": "bilinear",
+  "timm_version": "1.0.15",
+  "torch_version": "2.9.1+cu128",
+  "source_checkpoint": "vit_giant_patch14_dinov2.lvd142m_fine_tuned.pth",
+  "source_sha256": "dc6807a766a7a8d070afdc9d05f25bebc3256203d9eff6985f573dc529c98009"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:17a06000a498afafeaff3247c337669ea438f7a446117d06e83629c307157527
+size 4547548872

modeling.py

@@ -0,0 +1,255 @@
+"""Standalone model definition for the published GorillaWatch re-ID checkpoints.
+
+This file is intentionally self-contained: it is copied verbatim into every
+HuggingFace model repository so the weights are usable without installing
+anything from the GorillaWatch source tree.
+
+The forward pass mirrors ``TimmWrapper`` in
+``gorillawatch/src/gorillawatch/model/basemodel.py`` exactly.
+
+Usage::
+
+    from modeling import load_model
+
+    model = load_model("gorilla-watch/GorillaWatch-DINOv2-Large")
+    tf = model.get_transform()
+    embeddings = model(tf(image).unsqueeze(0))   # (1, 256)
+"""
+
+import json
+from dataclasses import asdict, dataclass, field, fields
+from pathlib import Path
+from typing import Any, Optional, Sequence, Union
+
+import torch
+import torch.nn as nn
+
+CONFIG_NAME = "config.json"
+WEIGHTS_NAME = "model.safetensors"
+
+# Fields of GorillaWatchConfig that determine the module topology. Everything
+# else in the config is preprocessing or provenance metadata.
+ARCH_FIELDS = (
+    "backbone_name",
+    "img_size",
+    "embedding_size",
+    "embedding_id",
+    "pool_mode",
+    "dropout_p",
+)
+
+try:  # optional; only needed for the idiomatic from_pretrained path
+    from huggingface_hub import PyTorchModelHubMixin
+
+    _HUB_MIXIN_AVAILABLE = True
+except ImportError:  # pragma: no cover - exercised only in minimal envs
+    _HUB_MIXIN_AVAILABLE = False
+
+    class PyTorchModelHubMixin:  # type: ignore[no-redef]
+        """No-op stand-in when huggingface_hub is not installed."""
+
+
+@dataclass
+class GorillaWatchConfig:
+    """Everything needed to rebuild a published checkpoint and preprocess for it.
+
+    The training pipeline stores none of this in the ``.pth`` file, which is the
+    whole reason this dataclass exists. ``img_size`` in particular is not
+    recoverable from the checkpoint filename and silently changes the
+    positional-embedding shape.
+    """
+
+    backbone_name: str
+    img_size: int = 518
+    embedding_size: int = 256
+    embedding_id: str = "linear"
+    pool_mode: str = "none"
+    dropout_p: float = 0.0
+
+    # Preprocessing. NOTE: mean/std are 0.5, *not* the ImageNet statistics that
+    # timm's DINOv2 default_cfg reports. Using timm's default transform against
+    # these weights produces wrong embeddings.
+    image_mean: Sequence[float] = field(default_factory=lambda: [0.5, 0.5, 0.5])
+    image_std: Sequence[float] = field(default_factory=lambda: [0.5, 0.5, 0.5])
+    resize_mode: str = "square"  # Resize((S, S)); requires squared input to preserve aspect ratio
+    interpolation: str = "bilinear"
+
+    timm_version: Optional[str] = None
+    torch_version: Optional[str] = None
+    source_checkpoint: Optional[str] = None
+    source_sha256: Optional[str] = None
+
+    def to_dict(self) -> dict[str, Any]:
+        return asdict(self)
+
+    @classmethod
+    def from_dict(cls, data: dict[str, Any]) -> "GorillaWatchConfig":
+        known = {f.name for f in fields(cls)}
+        unknown = set(data) - known
+        if unknown:
+            raise ValueError(
+                f"Unrecognised keys in config: {sorted(unknown)}. "
+                "Refusing to load rather than silently ignoring architecture settings."
+            )
+        return cls(**data)
+
+    def save(self, path: Union[str, Path]) -> None:
+        Path(path).write_text(json.dumps(self.to_dict(), indent=2) + "\n")
+
+    @classmethod
+    def load(cls, path: Union[str, Path]) -> "GorillaWatchConfig":
+        return cls.from_dict(json.loads(Path(path).read_text()))
+
+    @property
+    def arch_kwargs(self) -> dict[str, Any]:
+        return {name: getattr(self, name) for name in ARCH_FIELDS}
+
+
+def build_transform(
+    img_size: int = 518,
+    image_mean: Sequence[float] = (0.5, 0.5, 0.5),
+    image_std: Sequence[float] = (0.5, 0.5, 0.5),
+):
+    """The exact eval transform used by the training pipeline.
+
+    Mirrors ``get_transform`` in
+    ``gorillawatch/src/gorillawatch/data_hf/data_loading.py``.
+    """
+    from torchvision import transforms
+
+    return transforms.Compose(
+        [
+            transforms.Resize((img_size, img_size)),
+            transforms.ToTensor(),
+            transforms.Normalize(mean=list(image_mean), std=list(image_std)),
+        ]
+    )
+
+
+def _build_embedding_layer(embedding_id: str, feature_dim: int, embedding_dim: int) -> nn.Module:
+    if embedding_id == "linear":
+        return nn.Linear(feature_dim, embedding_dim)
+    if embedding_id == "identity":
+        return nn.Identity()
+    raise NotImplementedError(
+        f"embedding_id={embedding_id!r} is not supported by the published wrapper. "
+        "All released checkpoints use 'linear'."
+    )
+
+
+class GorillaWatchViT(nn.Module, PyTorchModelHubMixin):
+    """DINOv2 ViT backbone with a linear projection to a 256-d re-ID embedding.
+
+    Constructor arguments mirror ``config.json`` one-for-one so that
+    ``from_pretrained`` round-trips without losing fields.
+    """
+
+    def __init__(
+        self,
+        backbone_name: str,
+        img_size: int = 518,
+        embedding_size: int = 256,
+        embedding_id: str = "linear",
+        pool_mode: str = "none",
+        dropout_p: float = 0.0,
+        image_mean: Optional[Sequence[float]] = None,
+        image_std: Optional[Sequence[float]] = None,
+        resize_mode: str = "square",
+        interpolation: str = "bilinear",
+        pretrained: bool = False,
+        **_ignored: Any,
+    ) -> None:
+        super().__init__()
+
+        if pool_mode != "none":
+            raise NotImplementedError(
+                f"pool_mode={pool_mode!r} is not supported by the published wrapper. "
+                "All released checkpoints are ViTs trained with pool_mode='none'."
+            )
+
+        import timm
+
+        # pretrained=False by default: the fine-tuned weights overwrite the
+        # upstream DINOv2 weights anyway, so downloading them first is pure
+        # waste (4.5 GB for the giant).
+        self.model = timm.create_model(
+            backbone_name,
+            pretrained=pretrained,
+            drop_rate=0.0,
+            img_size=img_size,
+        )
+        self.num_features = self.model.num_features
+        self.embedding_layer = _build_embedding_layer(
+            embedding_id, self.num_features, embedding_size
+        )
+
+        self.backbone_name = backbone_name
+        self.img_size = img_size
+        self.embedding_size = embedding_size
+        self.embedding_id = embedding_id
+        self.pool_mode = pool_mode
+        self.dropout_p = dropout_p
+        self.image_mean = list(image_mean) if image_mean is not None else [0.5, 0.5, 0.5]
+        self.image_std = list(image_std) if image_std is not None else [0.5, 0.5, 0.5]
+        self.resize_mode = resize_mode
+        self.interpolation = interpolation
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        x = self.model.forward_features(x)
+        x = self.model.forward_head(x, pre_logits=True)
+        if x.dim() == 3:  # VisionTransformer: take the CLS token
+            x = x[:, 0, :]
+        return self.embedding_layer(x)
+
+    @property
+    def device(self) -> torch.device:
+        return next(self.parameters()).device
+
+    def get_transform(self):
+        return build_transform(self.img_size, self.image_mean, self.image_std)
+
+    @classmethod
+    def from_config(cls, config: GorillaWatchConfig, pretrained: bool = False) -> "GorillaWatchViT":
+        return cls(pretrained=pretrained, **config.to_dict())
+
+
+def load_model(
+    model_id_or_path: Union[str, Path],
+    device: Union[str, torch.device] = "cpu",
+    revision: Optional[str] = None,
+    token: Optional[str] = None,
+    cache_dir: Optional[str] = None,
+) -> GorillaWatchViT:
+    """Load a published checkpoint from a local directory or the HuggingFace Hub.
+
+    This is the guaranteed-stable loader; it does not depend on
+    ``PyTorchModelHubMixin`` behaviour, which varies across huggingface_hub
+    versions.
+    """
+    from safetensors.torch import load_file
+
+    local = Path(model_id_or_path)
+    if local.is_dir():
+        config_path = local / CONFIG_NAME
+        weights_path = local / WEIGHTS_NAME
+        missing = [p.name for p in (config_path, weights_path) if not p.exists()]
+        if missing:
+            raise FileNotFoundError(f"{local} is missing {missing}")
+    else:
+        from huggingface_hub import hf_hub_download
+
+        download_kwargs = dict(
+            repo_id=str(model_id_or_path),
+            repo_type="model",
+            revision=revision,
+            token=token,
+            cache_dir=cache_dir,
+        )
+        config_path = Path(hf_hub_download(filename=CONFIG_NAME, **download_kwargs))
+        weights_path = Path(hf_hub_download(filename=WEIGHTS_NAME, **download_kwargs))
+
+    config = GorillaWatchConfig.load(config_path)
+    model = GorillaWatchViT.from_config(config)
+    state_dict = load_file(str(weights_path))
+    model.load_state_dict(state_dict, strict=True)
+    return model.to(device).eval()