Instructions to use mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF:Q4_K_M

Use Docker

docker model run hf.co/mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF with Ollama:
```
ollama run hf.co/mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF with Docker Model Runner:
```
docker model run hf.co/mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF:Q4_K_M
```

Lemonade

How to use mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mradermacher/QiMing-Gemma-3-Socratic-4b-i1-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.QiMing-Gemma-3-Socratic-4b-i1-GGUF-Q4_K_M

List all available models

lemonade list

Atomic Chat

mradermacher commited on Aug 15, 2025

Commit

77651e8

verified ·

1 Parent(s): 7d6e8b6

uploaded from nico1

Browse files

Files changed (25) hide show

.gitattributes +24 -0
QiMing-Gemma-3-Socratic-4b.i1-IQ1_M.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-IQ1_S.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-IQ2_M.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-IQ2_S.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-IQ2_XS.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-IQ2_XXS.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-IQ3_M.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-IQ3_S.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-IQ3_XS.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-IQ3_XXS.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-IQ4_NL.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-IQ4_XS.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-Q2_K.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-Q2_K_S.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-Q3_K_L.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-Q3_K_M.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-Q3_K_S.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-Q4_0.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-Q4_1.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-Q4_K_M.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-Q4_K_S.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-Q5_K_M.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-Q5_K_S.gguf +3 -0
QiMing-Gemma-3-Socratic-4b.i1-Q6_K.gguf +3 -0

.gitattributes CHANGED Viewed

@@ -34,3 +34,27 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 QiMing-Gemma-3-Socratic-4b.imatrix.gguf filter=lfs diff=lfs merge=lfs -text

 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 QiMing-Gemma-3-Socratic-4b.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+QiMing-Gemma-3-Socratic-4b.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text

QiMing-Gemma-3-Socratic-4b.i1-IQ1_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8e99d3be023dc59a42c1f329f724c03f9289c02b242bc09241e6af70fe9774df
+size 1199571456

QiMing-Gemma-3-Socratic-4b.i1-IQ1_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:50096f06f5e52165140e64e1c3fdc304efb9b6f80f8fa17c46ce408dfbd5f606
+size 1133093376

QiMing-Gemma-3-Socratic-4b.i1-IQ2_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b882f5106bd3df60bed293cfd6be22ad021341cd105e240b45eb33b24e1ab49d
+size 1537982976

QiMing-Gemma-3-Socratic-4b.i1-IQ2_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4e2db08821a64c5444952bb83fc23a0af0a4060700f483802f8bbaf28eb499b1
+size 1449345536

QiMing-Gemma-3-Socratic-4b.i1-IQ2_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8701de2fca5a556d478915682b78b052133fffe7e181497138b98b124d9930d1
+size 1404576256

QiMing-Gemma-3-Socratic-4b.i1-IQ2_XXS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1528fa291a008dee93b05e342972f0b160e905af1d26c4e9e09b681942f7b8eb
+size 1310368256

QiMing-Gemma-3-Socratic-4b.i1-IQ3_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:93367586643812bc3003726ebe6fd0a5a08ce3f032a5c78d2b10df02b572166f
+size 1986803456

QiMing-Gemma-3-Socratic-4b.i1-IQ3_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:019944a11972d4e141f226b58349f433fe347dcddbf8ac4a764146fcf97e86e8
+size 1937364736

QiMing-Gemma-3-Socratic-4b.i1-IQ3_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9ad02479d83a8d7e79ee12a7d44413480ef0109e2babfefd1097c1a1b06e78e1
+size 1863390976

QiMing-Gemma-3-Socratic-4b.i1-IQ3_XXS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:516cd1dac27d3ac4796172f24d8478d66b924138a66cd46df18e1d6d06cf19d0
+size 1689453056

QiMing-Gemma-3-Socratic-4b.i1-IQ4_NL.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9ce61395f6249dd466b6c147a5508ad242c317ff877098721897d74037ea9607
+size 2363512576

QiMing-Gemma-3-Socratic-4b.i1-IQ4_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:15660f51aac01bab133ce8a0ced5084e11a6586e91574748162af7cbe94bc436
+size 2263242496

QiMing-Gemma-3-Socratic-4b.i1-Q2_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:23143b3fd447793845c3380f0b2b438ee1984a8521ab3ec3282b778dd59f09b8
+size 1729165056

QiMing-Gemma-3-Socratic-4b.i1-Q2_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:439cadbf888b688e2c2b9cbe0939fb770d8400d89065bf1c89617475b3fec446
+size 1636062976

QiMing-Gemma-3-Socratic-4b.i1-Q3_K_L.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4ab2c2903e7c8701d03a3688c7b1cf4f59b63c2bc27b662f324d0e17c2db72d8
+size 2236086016

QiMing-Gemma-3-Socratic-4b.i1-Q3_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bfd8c8fba626c8ffa546dcb96372cd9b58f95f249181b5c91cd8fbee72415a99
+size 2098460416

QiMing-Gemma-3-Socratic-4b.i1-Q3_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:68b06438da8f676bb1e44c8d6dfa7f1da28514eba24f96f5906015c166275322
+size 1937364736

QiMing-Gemma-3-Socratic-4b.i1-Q4_0.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fa091e9070128ff62d447cb14c4ff32e3b6829b12afa76bf0823e777066ecc3a
+size 2370066176

QiMing-Gemma-3-Socratic-4b.i1-Q4_1.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:554b00f194e1de1804d03ff406d0cfcb706f8348b1221647de65d8a6c1425c82
+size 2564052736

QiMing-Gemma-3-Socratic-4b.i1-Q4_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:351907cdc768d606194140a9706bb47d74ded3f326da5f5f6a90bcaeaee4702d
+size 2489894656

QiMing-Gemma-3-Socratic-4b.i1-Q4_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:325b110a49ec663eeae89ac7502537ca5605dab2e9160827645d038ae9a2aee7
+size 2377930496

QiMing-Gemma-3-Socratic-4b.i1-Q5_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:839277874f60572c661b8d7bca64e22a84150ba0c808b6f951ab5909e4d24a40
+size 2829698816

QiMing-Gemma-3-Socratic-4b.i1-Q5_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1818de70b023fca0d1c07451d50fcacdd3521a55d85dfccf751a4932f93fd5af
+size 2764592896

QiMing-Gemma-3-Socratic-4b.i1-Q6_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e239554bd16788c00803949dcb29c0efe1aa637445772b1f19de2c6b1b8ba2b3
+size 3190740736