Instructions to use Gopal2002/donut_visual_zeon with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Gopal2002/donut_visual_zeon with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Gopal2002/donut_visual_zeon")

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("Gopal2002/donut_visual_zeon")
model = AutoModelForMultimodalLM.from_pretrained("Gopal2002/donut_visual_zeon", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Gopal2002/donut_visual_zeon with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Gopal2002/donut_visual_zeon"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Gopal2002/donut_visual_zeon",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Gopal2002/donut_visual_zeon

SGLang

How to use Gopal2002/donut_visual_zeon with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Gopal2002/donut_visual_zeon" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Gopal2002/donut_visual_zeon",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Gopal2002/donut_visual_zeon" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Gopal2002/donut_visual_zeon",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Gopal2002/donut_visual_zeon with Docker Model Runner:
```
docker model run hf.co/Gopal2002/donut_visual_zeon
```

Gopal2002 commited on Jan 4, 2024

Commit

0965a5a

1 Parent(s): a6304f5

Upload model

Browse files

Files changed (3) hide show

config.json +2 -5
generation_config.json +1 -1
model.safetensors +2 -2

config.json CHANGED Viewed

@@ -1,6 +1,5 @@
 {
-  "_commit_hash": "6b3e8ef8087609eff23423ea1b5d7c3ae8b90402",
-  "_name_or_path": "nielsr/donut-base",
   "architectures": [
     "VisionEncoderDecoderModel"
   ],
@@ -83,7 +82,6 @@
     "top_p": 1.0,
     "torch_dtype": null,
     "torchscript": false,
-    "transformers_version": "4.26.1",
     "typical_p": 1.0,
     "use_bfloat16": false,
     "use_cache": true,
@@ -180,7 +178,6 @@
     "top_p": 1.0,
     "torch_dtype": null,
     "torchscript": false,
-    "transformers_version": "4.26.1",
     "typical_p": 1.0,
     "use_absolute_embeddings": false,
     "use_bfloat16": false,
@@ -191,5 +188,5 @@
   "pad_token_id": 1,
   "tie_word_embeddings": false,
   "torch_dtype": "float32",
-  "transformers_version": null
 }

 {
+  "_name_or_path": "Gopal2002/donut_visual_zeon",
   "architectures": [
     "VisionEncoderDecoderModel"
   ],
     "top_p": 1.0,
     "torch_dtype": null,
     "torchscript": false,
     "typical_p": 1.0,
     "use_bfloat16": false,
     "use_cache": true,
     "top_p": 1.0,
     "torch_dtype": null,
     "torchscript": false,
     "typical_p": 1.0,
     "use_absolute_embeddings": false,
     "use_bfloat16": false,
   "pad_token_id": 1,
   "tie_word_embeddings": false,
   "torch_dtype": "float32",
+  "transformers_version": "4.35.2"
 }

generation_config.json CHANGED Viewed

@@ -6,5 +6,5 @@
   "forced_eos_token_id": 2,
   "max_length": 8,
   "pad_token_id": 1,
-  "transformers_version": "4.26.1"
 }

   "forced_eos_token_id": 2,
   "max_length": 8,
   "pad_token_id": 1,
+  "transformers_version": "4.35.2"
 }

model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6e990e15f61827a00577fce5c13c174be76763133c51a25a997751923e0c1664
-size 809070744

 version https://git-lfs.github.com/spec/v1
+oid sha256:fe4c2cbb9729fd4201f3e9c24f857447103e8026ce3668badc1d8eb129d742f2
+size 809095320