Instructions to use mradermacher/UniME-V2-Qwen2VL-2B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mradermacher/UniME-V2-Qwen2VL-2B-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("mradermacher/UniME-V2-Qwen2VL-2B-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use mradermacher/UniME-V2-Qwen2VL-2B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/UniME-V2-Qwen2VL-2B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/UniME-V2-Qwen2VL-2B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/UniME-V2-Qwen2VL-2B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/UniME-V2-Qwen2VL-2B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mradermacher/UniME-V2-Qwen2VL-2B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf mradermacher/UniME-V2-Qwen2VL-2B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mradermacher/UniME-V2-Qwen2VL-2B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mradermacher/UniME-V2-Qwen2VL-2B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/mradermacher/UniME-V2-Qwen2VL-2B-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use mradermacher/UniME-V2-Qwen2VL-2B-GGUF with Ollama:
```
ollama run hf.co/mradermacher/UniME-V2-Qwen2VL-2B-GGUF:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use mradermacher/UniME-V2-Qwen2VL-2B-GGUF with Docker Model Runner:
```
docker model run hf.co/mradermacher/UniME-V2-Qwen2VL-2B-GGUF:Q4_K_M
```

Lemonade

How to use mradermacher/UniME-V2-Qwen2VL-2B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mradermacher/UniME-V2-Qwen2VL-2B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.UniME-V2-Qwen2VL-2B-GGUF-Q4_K_M

List all available models

lemonade list

Atomic Chat

mradermacher commited on Oct 16, 2025

Commit

1456a10

verified ·

1 Parent(s): b1650a5

uploaded from nico1

Browse files

Files changed (13) hide show

.gitattributes +12 -0
UniME-V2-Qwen2VL-2B.IQ4_XS.gguf +3 -0
UniME-V2-Qwen2VL-2B.Q2_K.gguf +3 -0
UniME-V2-Qwen2VL-2B.Q3_K_L.gguf +3 -0
UniME-V2-Qwen2VL-2B.Q3_K_M.gguf +3 -0
UniME-V2-Qwen2VL-2B.Q3_K_S.gguf +3 -0
UniME-V2-Qwen2VL-2B.Q4_K_M.gguf +3 -0
UniME-V2-Qwen2VL-2B.Q4_K_S.gguf +3 -0
UniME-V2-Qwen2VL-2B.Q5_K_M.gguf +3 -0
UniME-V2-Qwen2VL-2B.Q5_K_S.gguf +3 -0
UniME-V2-Qwen2VL-2B.Q6_K.gguf +3 -0
UniME-V2-Qwen2VL-2B.Q8_0.gguf +3 -0
UniME-V2-Qwen2VL-2B.f16.gguf +3 -0

.gitattributes CHANGED Viewed

@@ -35,3 +35,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 UniME-V2-Qwen2VL-2B.mmproj-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
 UniME-V2-Qwen2VL-2B.mmproj-f16.gguf filter=lfs diff=lfs merge=lfs -text

 *tfevents* filter=lfs diff=lfs merge=lfs -text
 UniME-V2-Qwen2VL-2B.mmproj-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
 UniME-V2-Qwen2VL-2B.mmproj-f16.gguf filter=lfs diff=lfs merge=lfs -text
+UniME-V2-Qwen2VL-2B.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+UniME-V2-Qwen2VL-2B.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+UniME-V2-Qwen2VL-2B.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+UniME-V2-Qwen2VL-2B.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+UniME-V2-Qwen2VL-2B.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+UniME-V2-Qwen2VL-2B.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+UniME-V2-Qwen2VL-2B.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+UniME-V2-Qwen2VL-2B.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+UniME-V2-Qwen2VL-2B.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+UniME-V2-Qwen2VL-2B.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+UniME-V2-Qwen2VL-2B.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+UniME-V2-Qwen2VL-2B.f16.gguf filter=lfs diff=lfs merge=lfs -text

UniME-V2-Qwen2VL-2B.IQ4_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:874ded46dbc6f19966254ae52e0c86b7b18db33f456e31a7ea945919427e8979
+size 902182048

UniME-V2-Qwen2VL-2B.Q2_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b3b52e8bc69cf98422663f0743d51db46bdaf9c84d51a4cbd27c168197dc1756
+size 676304032

UniME-V2-Qwen2VL-2B.Q3_K_L.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9d19bc942b4b0ad770c61bbe20591ce289d9c880391b8f44c9b8061110123ded
+size 880161952

UniME-V2-Qwen2VL-2B.Q3_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ec6394645a1e32a4123ce556e6909748d9a2ad5174d68d7801e72f6830d200dd
+size 824177824

UniME-V2-Qwen2VL-2B.Q3_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dfe33cccdb01897ae030f3915d757b3c928ee14a70da3ed59ec2a8fd3b5e8c79
+size 760943776

UniME-V2-Qwen2VL-2B.Q4_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:771050d12bb2bc49055b996e9b5602754e5ed4eb6591f6f584ed5862242e2783
+size 986047648

UniME-V2-Qwen2VL-2B.Q4_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d2dee0435d043f9c006df28dc830a3d9adbfecf58f82f0ff0ef9ff0465b3cf96
+size 940311712

UniME-V2-Qwen2VL-2B.Q5_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ab86c9f0c31ba0b7a61052d4b423f1a191cdd73141874c37e39a6438656c4c82
+size 1125049504

UniME-V2-Qwen2VL-2B.Q5_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cf52d5d938a6cae2dc4d4de9fabf98263a88f1053e4464a5ccc2ec6204802de7
+size 1098728608

UniME-V2-Qwen2VL-2B.Q6_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8238d857f639b0500f2c8c74a55ada18f9987265837eb586bd856a02171d0dfa
+size 1272738976

UniME-V2-Qwen2VL-2B.Q8_0.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:288cb3929c61c52ceb68d11b14b1916e4ec3af3856c8df6dd7b2d287fd273107
+size 1646572192

UniME-V2-Qwen2VL-2B.f16.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4a92ff49627d4472981374c71f0f316beb25145a04c0400d546dc0a48a8cc0cf
+size 3093668512