Instructions to use psp-dada/LLaVA-v1.5-13B-SENTINEL with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use psp-dada/LLaVA-v1.5-13B-SENTINEL with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="psp-dada/LLaVA-v1.5-13B-SENTINEL")

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("psp-dada/LLaVA-v1.5-13B-SENTINEL", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use psp-dada/LLaVA-v1.5-13B-SENTINEL with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "psp-dada/LLaVA-v1.5-13B-SENTINEL"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "psp-dada/LLaVA-v1.5-13B-SENTINEL",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/psp-dada/LLaVA-v1.5-13B-SENTINEL

SGLang

How to use psp-dada/LLaVA-v1.5-13B-SENTINEL with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "psp-dada/LLaVA-v1.5-13B-SENTINEL" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "psp-dada/LLaVA-v1.5-13B-SENTINEL",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "psp-dada/LLaVA-v1.5-13B-SENTINEL" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "psp-dada/LLaVA-v1.5-13B-SENTINEL",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use psp-dada/LLaVA-v1.5-13B-SENTINEL with Docker Model Runner:
```
docker model run hf.co/psp-dada/LLaVA-v1.5-13B-SENTINEL
```

LLaVA-v1.5-13B-SENTINEL / README.md

psp-dada

Update README.md

3376763 verified 3 months ago

preview code

Raw

History Blame Contribute Delete

5.06 kB

	---
	license: apache-2.0
	datasets:
	- psp-dada/SENTINEL
	language:
	- en
	base_model:
	- liuhaotian/llava-v1.5-13b
	pipeline_tag: image-text-to-text
	library_name: transformers
	tags:
	- PEFT
	- lora
	- bf16
	- llava
	---

	# Model Card for ``psp-dada/LLaVA-v1.5-13B-SENTINEL`` \| ICCV 2025 \| SENTINEL:<br>Mitigating Object Hallucinations via Sentence-Level Early Intervention <!-- omit in toc -->

	<a href='https://arxiv.org/abs/2507.12455'>
	<img src='https://img.shields.io/badge/Paper-Arxiv-purple'></a>
	<a href='https://huggingface.co/datasets/psp-dada/SENTINEL'>
	<img src='https://img.shields.io/badge/Datasets-HF-Green'></a>
	<a href='https://github.com/pspdada/SENTINEL'>
	<img src='https://img.shields.io/badge/Github-Repo-orange'></a>
	<a href='https://huggingface.co/papers/2507.12455'>
	<img src='https://img.shields.io/badge/Discussion-HF-blue'></a>
	<a href='https://github.com/pspdada/SENTINEL/blob/main/LICENSE'>
	<img src='https://img.shields.io/badge/LICENSE-Apache_2.0-yellow'></a>

	<a href='https://modelscope.cn/datasets/pspdada/SENTINEL'>
	<img src='https://img.shields.io/badge/Datasets-🤖ModelScope-pink'></a>
	<a href='https://modelscope.cn/collections/pspdada/SENTINEL'>
	<img src='https://img.shields.io/badge/Models-🤖ModelScope-red'></a>

	## 🎊 News <!-- omit in toc -->

	- [2025.07.30] 🔍 Our work has been featured and explained by 52CV, check it out [here](https://mp.weixin.qq.com/s/Sfr1wdUCkeOLmj7NVWNUnw).
	- [2025.07.21] 📖 All code, data, and models are released!
	- [2025.06.26] 🎉 Our SENTINEL is accepted by ICCV 2025!

	## 🚀 Overview <!-- omit in toc -->

	SENTINEL introduces an automatic, sentence‑level early intervention strategy to prevent and mitigate object hallucinations in multimodal large language models (MLLMs). Key advantages:

	- Annotation‑free: No human labeling required.

	- Model-agnostic: Compatible with any MLLM architecture.

	- Efficient: Lightweight LoRA fine‑tuning.

	## 🔑 Key Features

	- 🧠 Early intervention halts hallucination propagation. We find that hallucinations of MLLMs predominantly arise in early sentences and propagate through the rest of the output. SENTINEL interrupts this chain early to maximize mitigation.
	<table align="center">
	<p align="center">
	<img src="https://github.com/pspdada/SENTINEL/raw/main/docs/figures/figure2.png" width="80%" />
	</p>
	</table>

	- 🔍 In-domain contextual preference learning without human labels. SENTINEL constructs hallucinated/factual samples via detector cross-validation and builds context-aware preference data without relying on proprietary LLMs or manual annotations.
	<table align="center">
	<p align="center">
	<img src="https://github.com/pspdada/SENTINEL/raw/main/docs/figures/figure3.png" width="80%" />
	</p>
	</table>

	- 💡 Context matters: rich coherence drives robustness. By prioritizing context-coherent positive samples over hallucinated ones, SENTINEL significantly boosts generalization.
	<table align="center">
	<p align="center">
	<img src="https://github.com/pspdada/SENTINEL/raw/main/docs/figures/figure4.png" width="80%" />
	</p>
	</table>

	- ♻️ Iterative contextual bootstrapping for diverse hallucination-free contexts. Our pipeline dynamically grows non-hallucinated contexts and expands coverage across varied scenes, improving robustness across generations.
	<table align="center">
	<p align="center">
	<img src="https://github.com/pspdada/SENTINEL/raw/main/docs/figures/figure5.png" width="80%" />
	</p>
	</table>

	- 📊 State-of-the-art results across benchmarks.
	SENTINEL achieves up to 92% reduction in hallucinations and outperforms prior SOTA methods across Object HalBench, AMBER, and HallusionBench, while maintaining or improving general task performance.
	<table align="center">
	<p align="center">
	<img src="https://github.com/pspdada/SENTINEL/raw/main/docs/figures/table1.png" width="80%" />
	</p>
	</table>

	## How to use

	This model is a PEFT (LoRA) adapter. You first need to load the base model (`liuhaotian/llava-v1.5-13b`) and then load this adapter on top of it. Please follow the official repo of [LLaVA](https://github.com/haotian-liu/LLaVA) for more instructions.

	For the details of this model, please refer to the [documentation](https://github.com/pspdada/SENTINEL?tab=readme-ov-file#-model-weights) of the GitHub repo.

	## 📝 Citation

	If you find our model/code/data/paper helpful, please consider citing our papers 📝 and starring us ⭐️！

	```bibtex
	@inproceedings{SENTINEL_2025,
	title = {Mitigating Object Hallucinations via Sentence-Level Early Intervention},
	author = {Peng, Shangpin and Yang, Senqiao and Jiang, Li and Tian, Zhuotao},
	booktitle = {Proceedings of the IEEE International Conference on Computer Vision},
	pages = {635--646},
	year = {2025}
	}
	```

	## 📧 Contact us <!-- omit in toc -->

	If you have any questions, comments, or suggestions, please do not hesitate to submit an issue or PR to help advance research in this area.