Instructions to use mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF:Q4_K_M

Use Docker

docker model run hf.co/mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF with Ollama:
```
ollama run hf.co/mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF with Docker Model Runner:
```
docker model run hf.co/mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF:Q4_K_M
```

Lemonade

How to use mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mradermacher/Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Fourier-Qwen2.5-VL-3B-0.67-i1-GGUF-Q4_K_M

List all available models

lemonade list

Atomic Chat

mradermacher commited on May 20

Commit

d7af8f2

verified ·

1 Parent(s): 6e12095

uploaded from nico1

Browse files

Files changed (25) hide show

.gitattributes +24 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-IQ1_M.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-IQ1_S.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-IQ2_M.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-IQ2_S.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-IQ2_XS.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-IQ2_XXS.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-IQ3_M.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-IQ3_S.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-IQ3_XS.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-IQ3_XXS.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-IQ4_NL.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-IQ4_XS.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-Q2_K.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-Q2_K_S.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-Q3_K_L.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-Q3_K_M.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-Q3_K_S.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-Q4_0.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-Q4_1.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-Q4_K_M.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-Q4_K_S.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-Q5_K_M.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-Q5_K_S.gguf +3 -0
Fourier-Qwen2.5-VL-3B-0.67.i1-Q6_K.gguf +3 -0

.gitattributes CHANGED Viewed

@@ -34,3 +34,27 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 Fourier-Qwen2.5-VL-3B-0.67.imatrix.gguf filter=lfs diff=lfs merge=lfs -text

 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 Fourier-Qwen2.5-VL-3B-0.67.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Fourier-Qwen2.5-VL-3B-0.67.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text

Fourier-Qwen2.5-VL-3B-0.67.i1-IQ1_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:83f328381fb7202e1c76a5e5b03f006f2a80f88cac23aebde55f7970f21e0ae7
+size 850026688

Fourier-Qwen2.5-VL-3B-0.67.i1-IQ1_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e7228c683d630beb27193bface9b517cb903845b327e92734c683279e86c407f
+size 791093440

Fourier-Qwen2.5-VL-3B-0.67.i1-IQ2_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:18d3d91d9d0d241a619ef5857c642da7798488e8f2153760cd9c5a672021c661
+size 1140515008

Fourier-Qwen2.5-VL-3B-0.67.i1-IQ2_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:091474ba30297dcb0051461146c8db3697277be29e61d96374c37621dc576dbb
+size 1061937344

Fourier-Qwen2.5-VL-3B-0.67.i1-IQ2_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7c18e1e26c84a763059ebe65fdcb6ed4aadc536945d68c68d7b332eab30141f5
+size 1031545024

Fourier-Qwen2.5-VL-3B-0.67.i1-IQ2_XXS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9d0df86e07844330015ea0eb4798fd66abbc061f62db3ffc6084440ed1509bf3
+size 948248768

Fourier-Qwen2.5-VL-3B-0.67.i1-IQ3_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dc5f7c5342e3045cfa8a0edbd146f75608be43419f8fb8b6a49cdf1943ec7060
+size 1488894144

Fourier-Qwen2.5-VL-3B-0.67.i1-IQ3_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0de644e1a646c1eee1d8589f75778cdac88a633e201ab519757f86361ad41e03
+size 1456863424

Fourier-Qwen2.5-VL-3B-0.67.i1-IQ3_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d2e56754a247c07332a01e6a977a23d0fec105d201d354f3fd3ca24f86a688bc
+size 1391835328

Fourier-Qwen2.5-VL-3B-0.67.i1-IQ3_XXS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2846f0f1e7efed625cb9836e7c2e2720b6ff846c6d1517b16af4cb96c79cb806
+size 1282826432

Fourier-Qwen2.5-VL-3B-0.67.i1-IQ4_NL.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:18a53633cb1452985478148d339e5d2b303908577c8980c1145b82eed4e13169
+size 1825208512

Fourier-Qwen2.5-VL-3B-0.67.i1-IQ4_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e98f96ed3bfeddc1cb20df579b9978f733f895914ab1e7f53b4b11a4f971e0c6
+size 1739094208

Fourier-Qwen2.5-VL-3B-0.67.i1-Q2_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0d2f05cf4baf34d164662980533ee6ec79de84184c9f9f31ac880f304a8fa441
+size 1274755264

Fourier-Qwen2.5-VL-3B-0.67.i1-Q2_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1d032780b54083d60c165987df2dd5f2b681a6aad017f07a0328a6ab2b81f798
+size 1198127296

Fourier-Qwen2.5-VL-3B-0.67.i1-Q3_K_L.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9f62ab263e50b7756dab5dc1fb6586cd3ce40b9c8e2773dfd897106466a7706b
+size 1707391168

Fourier-Qwen2.5-VL-3B-0.67.i1-Q3_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bc1130af591091cfab030dc065632b132c05576eef5fe9ee4cf1010db656b19d
+size 1590474944

Fourier-Qwen2.5-VL-3B-0.67.i1-Q3_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:50fdce58053202d67e16a676ef21f394d14633fc864bd2dcf0600dcbedde9e91
+size 1454356672

Fourier-Qwen2.5-VL-3B-0.67.i1-Q4_0.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4c939b8b172883c2ae9d9f1a2213c84c186676cf747f919529d350be629d904e
+size 1828485312

Fourier-Qwen2.5-VL-3B-0.67.i1-Q4_1.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:145346ab533c0fa6d4a9b861a49e105e0aac463404b5c8234cac851498b542ef
+size 1996257472

Fourier-Qwen2.5-VL-3B-0.67.i1-Q4_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fddfda1ca83b8b4e62c5e8728918d6a54ac72d4f3c20c60450dff60a7e2af5fe
+size 1929902272

Fourier-Qwen2.5-VL-3B-0.67.i1-Q4_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:48ad19c7831e76a17355ffc90e095f1b6bc75d2372eea56ae230e43ac8f9d562
+size 1834383552

Fourier-Qwen2.5-VL-3B-0.67.i1-Q5_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:541cbc93382451c93add6d7b9153b74c38d1db8307dd569e55a89ed4f44f6a62
+size 2224814272

Fourier-Qwen2.5-VL-3B-0.67.i1-Q5_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e77f17bb18465994a6b415551648b300dc3a035d1aa09af920c0fb8ddfde7e87
+size 2169665728

Fourier-Qwen2.5-VL-3B-0.67.i1-Q6_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f846f622ee9c214e26706472e1ea01cec60c50117256252c94dbdbd9c9141ef8
+size 2538158272