Instructions to use toshi456/llava-jp-1.3b-v1.1-pretrain with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use toshi456/llava-jp-1.3b-v1.1-pretrain with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="toshi456/llava-jp-1.3b-v1.1-pretrain")

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("toshi456/llava-jp-1.3b-v1.1-pretrain", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use toshi456/llava-jp-1.3b-v1.1-pretrain with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "toshi456/llava-jp-1.3b-v1.1-pretrain"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "toshi456/llava-jp-1.3b-v1.1-pretrain",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/toshi456/llava-jp-1.3b-v1.1-pretrain

SGLang

How to use toshi456/llava-jp-1.3b-v1.1-pretrain with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "toshi456/llava-jp-1.3b-v1.1-pretrain" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "toshi456/llava-jp-1.3b-v1.1-pretrain",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "toshi456/llava-jp-1.3b-v1.1-pretrain" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "toshi456/llava-jp-1.3b-v1.1-pretrain",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use toshi456/llava-jp-1.3b-v1.1-pretrain with Docker Model Runner:
```
docker model run hf.co/toshi456/llava-jp-1.3b-v1.1-pretrain
```

toshi456 commited on Apr 20, 2024

Commit

3c3b439

verified ·

1 Parent(s): 1e2ceea

Update README.md

Browse files

Files changed (1) hide show

README.md +25 -0

README.md CHANGED Viewed

@@ -1,3 +1,28 @@
 ---
 license: apache-2.0
 ---

 ---
 license: apache-2.0
+datasets:
+- turing-motors/LLaVA-Pretrain-JA
+language:
+- ja
 ---
+# LLaVA-JP Model Card
+This is a pretrained checkpoint, you can use it to instruct tune your multimodal models.
+Check out the instructions [here](https://github.com/tosiyuki/LLaVA-JP)
+## Model details
+**Model type:**
+LLaVA-JP is a vision-language model that can converse about input images.<br>
+This model is an LVLM model trained using [google/siglip-so400m-patch14-384](https://huggingface.co/google/siglip-so400m-patch14-384) as the image encoder and [llm-jp/llm-jp-1.3b-v1.0](https://huggingface.co/llm-jp/llm-jp-1.3b-v1.0) as the text decoder. supports the input of 768 x 768 high resolution images by scaling_on_scales method.
+## Training dataset
+- [LLaVA-Pretrain-JA](https://huggingface.co/datasets/turing-motors/LLaVA-Pretrain-JA)
+## Acknowledgement
+- [LLaVA](https://llava-vl.github.io/)
+- [LLM-jp](https://llm-jp.nii.ac.jp/)
+- [scaling_on_scales](https://github.com/bfshi/scaling_on_scales/tree/master)
+## License
+Apache-2.0