chaxjli commited on
Commit
3d3d240
Β·
verified Β·
1 Parent(s): 40784df

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -50
README.md CHANGED
@@ -33,54 +33,16 @@ Building on these insights, we introduce U-MARVEL (Universal Multimodal Retrieva
33
 
34
  - [U-MARVEL-Qwen2VL-7B-Instruct](https://huggingface.co/TencentBAC/U-MARVEL-Qwen2VL-7B-Instruct) πŸ€—
35
  - [U-MARVEL-Qwen3VL-4B-Instruct](https://huggingface.co/TencentBAC/U-MARVEL-Qwen3VL-4B-Instruct) πŸ€—
36
- - Inference code available at: [U-MARVEL-inference](https://github.com/chaxjli/U-MARVEL)
37
- ## Requirements
38
 
39
- To install requirements:
40
-
41
- ```setup
42
- pip install -r requirements.txt
43
- ```
44
-
45
- ### Data Preparation
46
-
47
- Download Qwen2-VL-7B and place it in `./checkpoints/hf_models/Qwen2-VL-7B-Instruct`
48
-
49
- Download Qwen3-VL-4B and place it in `./checkpoints/hf_models/Qwen3-VL-4B-Instruct`
50
-
51
- For NLI dataset, please refer to [link](https://huggingface.co/datasets/princeton-nlp/datasets-for-simcse)
52
-
53
- For multimodal instruction tuning datset, please refer to [M-BEIR](https://huggingface.co/datasets/TIGER-Lab/M-BEIR)
54
-
55
- After downloading all of them, organize the data as follows in `./data`
56
-
57
- ```
58
- β”œβ”€β”€ data
59
- β”‚ β”œβ”€β”€ M-BEIR
60
- β”‚ β”œβ”€β”€ nli_for_simcse.csv
61
- β”‚ β”œβ”€β”€ rerank_data_for_training
62
- β”‚ β”œβ”€β”€ flickr
63
- β”‚ β”œβ”€β”€ coco
64
- β”‚ β”œβ”€β”€ sharegpt4v
65
- β”‚ β”œβ”€β”€ Urban1K
66
- β”‚ β”œβ”€β”€ circo
67
- β”‚ β”œβ”€β”€ genecis
68
- β”‚ β”œβ”€β”€ vist
69
- β”‚ β”œβ”€β”€ visdial
70
- β”‚ β”œβ”€β”€ ccneg
71
- β”‚ β”œβ”€β”€ sugar-crepe
72
- β”‚ β”œβ”€β”€ MSVD
73
- β”‚ └── msrvtt
74
- ```
75
-
76
- ## Evaluation
77
-
78
- To evaluate our model on M-BEIR, run:
79
 
80
  ```bash
81
- python scripts/vtools_eval_mbeir_model.py # Evaluate locally
82
- sh scripts/eval_mbeir_global.sh # Evaluate globally
83
- sh scripts/eval_zeroshot.sh # Evaluate zero-shot
 
84
  ```
85
 
86
  ## Model Performance
@@ -104,10 +66,10 @@ Many thanks to the code bases from **[LamRA](https://github.com/Code-kunkun/LamR
104
  ## Citation
105
  If you use this code for your research or project, please cite:
106
  ```latex
107
- @article{li2025umarvel,
108
- title={U-MARVEL: Unveiling Key Factors for Universal Multimodal Retrieval via Embedding Learning with MLLMs},
109
- author={Li, Xiaojie and Li, Chu and Chen, Shi-Zhe and Chen, Xi},
110
- journal={arXiv preprint arXiv:2507.14902},
111
- year={2025}
112
  }
113
  ```
 
33
 
34
  - [U-MARVEL-Qwen2VL-7B-Instruct](https://huggingface.co/TencentBAC/U-MARVEL-Qwen2VL-7B-Instruct) πŸ€—
35
  - [U-MARVEL-Qwen3VL-4B-Instruct](https://huggingface.co/TencentBAC/U-MARVEL-Qwen3VL-4B-Instruct) πŸ€—
36
+ - Code available at: [U-MARVEL](https://github.com/chaxjli/U-MARVEL)
37
+ ## πŸš€ Demo
38
 
39
+ To get started, first create a virtual environment and install the required dependencies:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
40
 
41
  ```bash
42
+ conda create -n u-marvel python=3.9 -y
43
+ conda activate u-marvel
44
+ pip install -r requirements_qwen2_vl.txt
45
+ python demo.py
46
  ```
47
 
48
  ## Model Performance
 
66
  ## Citation
67
  If you use this code for your research or project, please cite:
68
  ```latex
69
+ @inproceedings{li2026umarvel,
70
+ title={U-{MARVEL}: Unveiling Key Factors for Universal Multimodal Retrieval via Embedding Learning with {MLLM}s},
71
+ author={Xiaojie Li and Chu Li and Shi-Zhe Chen and Xi Chen},
72
+ booktitle={The Fourteenth International Conference on Learning Representations},
73
+ year={2026}
74
  }
75
  ```