Create README.md

README.md

@@ -0,0 +1,101 @@
+---
+license: mit
+language:
+- en
+pipeline_tag: image-to-text
+tags:
+- image-captioning
+- pytorch
+- lstm
+- computer-vision
+- nlp
+- flickr8k
+metrics:
+- bleu
+- rouge
+- meteor
+---
+
+# Image Caption Generator (ResNet50 + LSTM)
+
+A ResNet50 (frozen, transfer learning) + LSTM decoder model that generates natural-language captions for images. Trained on [Flickr8k](https://www.kaggle.com/datasets/adityajn105/flickr8k).
+
+- **Full project code, training pipeline, and documentation:** https://github.com/adhamashraf7788/Image-Caption-Generator
+- **Live interactive demo (Hugging Face Space):** https://huggingface.co/spaces/AdhamAshraf/image_caption_generator
+
+## Files in this repo
+
+```
+vocab.json                             # vocabulary (shared across both checkpoints)
+base_resnet_lstm/
+├── best_model.pt                      # baseline checkpoint
+└── config.yaml                        # baseline training config
+resnet_lstm_regularized/
+├── best_model.pt                      # regularized checkpoint (recommended -- best results)
+└── config.yaml                        # regularized training config
+```
+
+Two checkpoints are provided:
+
+| Checkpoint | BLEU-4 (beam-3) | Notes |
+|---|---|---|
+| `base_resnet_lstm/best_model.pt` | 0.1364 | Initial baseline |
+| `resnet_lstm_regularized/best_model.pt` | **0.1557** | Added LSTM output dropout, weight decay, gradient clipping — recommended |
+
+Both checkpoints share the same `vocab.json` (identical vocabulary, 2,662 tokens).
+
+## Architecture
+
+```
+Image → ResNet50 (frozen, ImageNet-pretrained) → 2048-d feature
+      → Linear(2048 → 256) projection
+      → fed as first input step to a 1-layer LSTM (hidden_dim=512)
+      → LSTM generates caption word-by-word (beam search recommended, width 3)
+```
+
+Full architecture, preprocessing, and training details: see the [GitHub README](https://github.com/adhamashraf7788/Image-Caption-Generator#architecture).
+
+## How to use
+
+Requires the inference code from the [GitHub repo](https://github.com/adhamashraf7788/Image-Caption-Generator) (`src/inference/predict.py` and its dependencies) — these checkpoints are not standalone `transformers`-compatible weights, they're plain PyTorch `state_dict`s wrapped with config metadata.
+
+```python
+from huggingface_hub import hf_hub_download
+from src.inference.predict import Predictor  # from the GitHub repo's src/
+
+checkpoint_path = hf_hub_download(
+    repo_id="AdhamAshraf/image-caption-generator",
+    filename="resnet_lstm_regularized/best_model.pt",
+)
+vocab_path = hf_hub_download(
+    repo_id="AdhamAshraf/image-caption-generator",
+    filename="vocab.json",
+)
+
+predictor = Predictor(checkpoint_path=checkpoint_path, vocab_path=vocab_path, device="cpu")
+caption = predictor.predict("path/to/image.jpg")
+print(caption)
+```
+
+## Training data
+
+[Flickr8k](https://www.kaggle.com/datasets/adityajn105/flickr8k) — 8,091 images, 5 human-written reference captions each. Split 80/10/10 (by image, not caption, to avoid leakage) using a fixed seed.
+
+## Evaluation results (test set, 810 images)
+
+| Metric | Baseline + greedy | Baseline + beam-3 | Regularized + greedy | **Regularized + beam-3** |
+|---|---|---|---|---|
+| BLEU-1 | 0.5127 | 0.5240 | 0.5444 | **0.5517** |
+| BLEU-4 | 0.1221 | 0.1364 | 0.1435 | **0.1557** |
+| ROUGE-L | 0.4177 | 0.4265 | 0.4434 | **0.4527** |
+| METEOR | 0.3266 | 0.3267 | 0.3480 | **0.3528** |
+
+Full evaluation methodology, qualitative examples, and failure-case analysis: see the [GitHub README](https://github.com/adhamashraf7788/Image-Caption-Generator#evaluation-metrics-and-results).
+
+## Limitations
+
+- Trained on a small (8k image) dataset; struggles with image content/styles underrepresented in Flickr8k (predominantly people, dogs, and outdoor scenes).
+- Even the regularized model still shows some overfitting past its best epoch.
+- Generated captions are sometimes fluent but not fully grounded in image-specific detail.
+
+See the [GitHub README's Limitations section](https://github.com/adhamashraf7788/Image-Caption-Generator#known-limitations--next-steps) for a full discussion, including a documented failure case and how regularization + beam search improved it.