| ---
|
| title: "ViT Base Patch16 384 – GI Endoscopy Classifier"
|
| tags:
|
| - vision-transformer
|
| - vit
|
| - image-classification
|
| - medical-imaging
|
| - gastrointestinal
|
| - pytorch
|
| - timm
|
| library_name: timm
|
| license: other
|
| language: en
|
| pipeline_tag: image-classification
|
| ---
|
|
|
| ## ViT Base (384px) for GI Endoscopy
|
| 23-class gastrointestinal endoscopy classifier built on ViT Base Patch16 384. Trained with timm, strong augmentation (MixUp + Albumentations), and evaluated with TTA. Includes traced TorchScript weights `vit_best_traced.pt` for drop-in PyTorch inference.
|
|
|
| ### Highlights
|
| - Val: **92.18%** (epoch 21 best); Test (TTA): **93.25% Acc / 92.19% Prec / 93.25% Rec / 92.59% F1**
|
| - Robust pipeline: MixUp, CoarseDropout, and TTA for generalization
|
| - Ready to serve: TorchScript traced weights; standard 384x384 ImageNet normalization
|
| - Scope: 23 GI classes; input RGB 384x384
|
|
|
| ### Performance
|
| | Split | Accuracy | Precision | Recall | F1 | Notes |
|
| | --- | --- | --- | --- | --- | --- |
|
| | Validation (best) | 92.18% | – | – | – | Best at epoch 21 |
|
| | Test (TTA) | 93.25% | 92.19% | 93.25% | 92.59% | 800 batches, TTA enabled |
|
|
|
| ### Model Card
|
| - Backbone: `vit_base_patch16_384` (≈86.1M params)
|
| - Classes: 23
|
| - Input: 384x384 RGB, mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]
|
| - Training recipe:
|
| - Epochs: 25; batch size: 2
|
| - LR: 1e-5 base with warmup schedule
|
| - Augment: MixUp; Albumentations with CoarseDropout
|
| - Eval: TTA on validation/test; best checkpoint saved each epoch (best at epoch 21)
|
| - Data scale (from logs): 7,463 train, 1,599 val, 1,600 test images across 23 classes
|
|
|
| ### Inference (PyTorch TorchScript)
|
| ```python
|
| import torch
|
| from PIL import Image
|
| from torchvision import transforms
|
|
|
| model = torch.jit.load("vit_best_traced.pt")
|
| model.eval()
|
|
|
| preprocess = transforms.Compose([
|
| transforms.Resize((384, 384)),
|
| transforms.ToTensor(),
|
| transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
|
| ])
|
|
|
| img = Image.open("path/to/image.jpg").convert("RGB")
|
| tensor = preprocess(img).unsqueeze(0)
|
| with torch.no_grad():
|
| probs = model(tensor).softmax(dim=1)
|
| pred = probs.argmax(dim=1).item()
|
|
|
| print("Predicted class index:", pred)
|
| ```
|
|
|
| ### Training Log (key checkpoints)
|
| - Epoch 8: Val Acc 90.56%
|
| - Epoch 12: Val Acc 91.62%
|
| - Epoch 16: Val Acc 91.74%
|
| - Epoch 18: Val Acc 92.06%
|
| - Epoch 20: Val Acc 92.12%
|
| - Epoch 21: **Val Acc 92.18%** (best)
|
| - Test (TTA): Acc 93.25%, Precision 92.19%, Recall 93.25%, F1 92.59%
|
|
|
| ### Files
|
| - `vit_best_traced.pt` — TorchScript traced weights for ViT Base Patch16 384 (best checkpoint)
|
|
|
| ### Notes & Responsible Use
|
| - Trainer: custom `AdvancedMemoryEfficientTrainer` with MixUp and Albumentations TTA.
|
| - Warnings during training: Albumentations recommends `Affine` over `ShiftScaleRotate`; CoarseDropout args adjusted. Metrics unaffected.
|
| - Medical context: research-only; not a regulated medical device. Validate clinically and keep a human in the loop.
|
|
|
| ### Reproduce / Fine-tune Tips
|
| - Keep preprocessing (384 resize + ImageNet normalization) consistent to match reported metrics.
|
| - Start from `vit_base_patch16_384` in timm; use MixUp and similar augmentations.
|
| - Enable TTA for best accuracy; disable for faster inference.
|
|
|
| ### Changelog
|
| - 2025-12-29: Initial public release with traced weights and metrics.
|
|
|
