Update README.md
Browse files
README.md
CHANGED
|
@@ -9,6 +9,31 @@ tags:
|
|
| 9 |
- llava
|
| 10 |
- lora
|
| 11 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
|
| 13 |
### Training hyperparameters
|
| 14 |
The following hyperparameters were used during training:
|
|
|
|
| 9 |
- llava
|
| 10 |
- lora
|
| 11 |
---
|
| 12 |
+
<style>
|
| 13 |
+
.img-responsive {
|
| 14 |
+
width: 50%;
|
| 15 |
+
height: auto;
|
| 16 |
+
}
|
| 17 |
+
</style>
|
| 18 |
+
|
| 19 |
+
### Conclusion
|
| 20 |
+
While significantly better at understanding and describing emotions and details in images compared to LLaVA-1.5-7b-hf, the fine-tuned model struggles with recognizing text.
|
| 21 |
+
|
| 22 |
+
### Train Loss
|
| 23 |
+

|
| 24 |
+
|
| 25 |
+
### Test
|
| 26 |
+
A comparative analysis of emoji in prompts, differents between the original model and its fine-tuned counterpart. </br>
|
| 27 |
+
Original Model:https://huggingface.co/llava-hf/llava-1.5-7b-hf/</br>
|
| 28 |
+
<img src="./images/original-01.JPG" alt="meme01" class="img-responsive">
|
| 29 |
+
<img src="./images/original-02.JPG" alt="meme02" class="img-responsive">
|
| 30 |
+
<img src="./images/original-03.JPG" alt="meme03" class="img-responsive">
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
Fine-tuned Lora Model:https://huggingface.co/REILX/llava-1.5-7b-hf-meme-lora</br>
|
| 34 |
+
<img src="./images/lora-01.JPG" alt="meme01" class="img-responsive">
|
| 35 |
+
<img src="./images/lora-02.JPG" alt="meme02" class="img-responsive">
|
| 36 |
+
<img src="./images/lora-03.JPG" alt="meme03" class="img-responsive">
|
| 37 |
|
| 38 |
### Training hyperparameters
|
| 39 |
The following hyperparameters were used during training:
|