Image-to-Text
Transformers
Safetensors
Portuguese
vision-encoder-decoder
image-text-to-text
Eval Results (legacy)
Instructions to use laicsiifes/swin-distilbertimbau with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use laicsiifes/swin-distilbertimbau with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "image-to-text" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # 'pip install "transformers<5.0.0' from transformers import pipeline pipe = pipeline("image-to-text", model="laicsiifes/swin-distilbertimbau")# Load model directly from transformers import AutoTokenizer, AutoModelForMultimodalLM tokenizer = AutoTokenizer.from_pretrained("laicsiifes/swin-distilbertimbau") model = AutoModelForMultimodalLM.from_pretrained("laicsiifes/swin-distilbertimbau", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -27,10 +27,10 @@ as encoder and the checkpoints of the [DistilBERTimbau](https://huggingface.co/a
|
|
| 27 |
The encoder checkpoints come from Swin Trasnformer version pre-trained on ImageNet-1k at resolution 224x224.
|
| 28 |
|
| 29 |
The code used for training and evaluation is available at: https://github.com/laicsiifes/ved-transformer-caption-ptbr. In this work, Swin-DistilBERTimbau
|
| 30 |
-
was trained together with its buddy [Swin-GPorTuguese](https://huggingface.co/laicsiifes/swin-gpt2-flickr30k-pt-br).
|
| 31 |
|
| 32 |
-
Other models evaluated didn't achieve performance as high as Swin-DistilBERTimbau and Swin-GPorTuguese, namely: DeiT-BERTimbau,
|
| 33 |
-
DeiT-DistilBERTimbau, DeiT-GPorTuguese, Swin-BERTimbau, ViT-BERTimbau, ViT-DistilBERTimbau and ViT-GPorTuguese.
|
| 34 |
|
| 35 |
## 🧑💻 How to Get Started with the Model
|
| 36 |
|
|
@@ -65,7 +65,7 @@ The evaluation metrics Cider-D, BLEU@4, ROUGE-L, METEOR and BERTScore are abbrev
|
|
| 65 |
|Model|Training|Evaluation|C|B@4|RL|M|BS|
|
| 66 |
|:---:|:------:|:--------:|:-----:|:----:|:-----:|:----:|:-------:|
|
| 67 |
|Swin-DistilBERTimbau|Flickr30K Portuguese|Flickr30K Portuguese|66.73|24.65|39.98|44.71|72.30|
|
| 68 |
-
|Swin-GPorTuguese|Flickr30K Portuguese|Flickr30K Portuguese|64.71|23.15|39.39|44.36|71.70|
|
| 69 |
|
| 70 |
## 📋 BibTeX entry and citation info
|
| 71 |
|
|
|
|
| 27 |
The encoder checkpoints come from Swin Trasnformer version pre-trained on ImageNet-1k at resolution 224x224.
|
| 28 |
|
| 29 |
The code used for training and evaluation is available at: https://github.com/laicsiifes/ved-transformer-caption-ptbr. In this work, Swin-DistilBERTimbau
|
| 30 |
+
was trained together with its buddy [Swin-GPorTuguese-2](https://huggingface.co/laicsiifes/swin-gpt2-flickr30k-pt-br).
|
| 31 |
|
| 32 |
+
Other models evaluated didn't achieve performance as high as Swin-DistilBERTimbau and Swin-GPorTuguese-2, namely: DeiT-BERTimbau,
|
| 33 |
+
DeiT-DistilBERTimbau, DeiT-GPorTuguese-2, Swin-BERTimbau, ViT-BERTimbau, ViT-DistilBERTimbau and ViT-GPorTuguese-2.
|
| 34 |
|
| 35 |
## 🧑💻 How to Get Started with the Model
|
| 36 |
|
|
|
|
| 65 |
|Model|Training|Evaluation|C|B@4|RL|M|BS|
|
| 66 |
|:---:|:------:|:--------:|:-----:|:----:|:-----:|:----:|:-------:|
|
| 67 |
|Swin-DistilBERTimbau|Flickr30K Portuguese|Flickr30K Portuguese|66.73|24.65|39.98|44.71|72.30|
|
| 68 |
+
|Swin-GPorTuguese-2|Flickr30K Portuguese|Flickr30K Portuguese|64.71|23.15|39.39|44.36|71.70|
|
| 69 |
|
| 70 |
## 📋 BibTeX entry and citation info
|
| 71 |
|