Instructions to use chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF with jev-style:

pip install jev-style
# GGUF builds score through llama.cpp: build the jev-score binary once
hf download chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF build_jev_score.sh jev_score.cpp --local-dir jev-score
export JEV_SCORE_BIN=$(sh jev-score/build_jev_score.sh /path/to/llama.cpp | tail -n 1)

from jev_style import JevStyle, noul, choice

js = JevStyle.from_pretrained("chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF")
out = js.decide("I was charged twice for one order.", {
    "billing": noul("This message is about billing."),
    "team": choice("Which team should handle it?", ["billing", "shipping", "tech"]),
})
print(out["answers"]["team"]["choice"])

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M

Use Docker

docker model run hf.co/chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M

Ollama
How to use chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF with Ollama:
```
ollama run hf.co/chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF with Docker Model Runner:
```
docker model run hf.co/chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M
```

Lemonade

How to use chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Jev-Style-Qwen3.5-2B-Decision-v2-GGUF-Q4_K_M

List all available models

lemonade list

Atomic Chat

chaoliangUNSW commited on 10 days ago

Commit

e9a1bbe

verified ·

1 Parent(s): f34035e

Map current GGUF filenames to validated build names in quantization_summary.json

Browse files

Files changed (2) hide show

SHA256SUMS.json +2 -2
evaluation/quantization_summary.json +8 -4

SHA256SUMS.json CHANGED Viewed

@@ -84,8 +84,8 @@
     "sha256": "2a4e163354b0d0d31bf5fcdfcfbcf726fb152fb2f6419432ddddaec3b475a921"
   },
   "evaluation/quantization_summary.json": {
-    "bytes": 5465,
-    "sha256": "45c350f225f9a403d036d96a80ef14ca9cd89bb9c25af82617071833846eefd6"
   },
   "evaluation/calibration_batch_parity.json": {
     "bytes": 147,

     "sha256": "2a4e163354b0d0d31bf5fcdfcfbcf726fb152fb2f6419432ddddaec3b475a921"
   },
   "evaluation/quantization_summary.json": {
+    "bytes": 5844,
+    "sha256": "9ec0233027766c2104450fa45d2a7c4cfea66fcde786d5ad699691e81d9f2030"
   },
   "evaluation/calibration_batch_parity.json": {
     "bytes": 147,

evaluation/quantization_summary.json CHANGED Viewed

@@ -4,7 +4,8 @@
   "metric": "real-label task-macro accuracy; teacher-reference decisions excluded from this macro",
   "variants": {
     "Q4_K_M": {
-      "filename": "Jev-Style-v2-Q4_K_M-Calibrated.gguf",
       "weight_bytes": 1274388384,
       "sha256": "c697d3b29d07fdd37b6ebeb5c98066f4c31632162adca6258db23f75184eb0c4",
       "argmax_agreement": 0.914,
@@ -54,7 +55,8 @@
       }
     },
     "Q8_0": {
-      "filename": "Jev-Style-v2-Q8_0-Calibrated.gguf",
       "weight_bytes": 2012004256,
       "sha256": "5c2aa0d35b24a27f03228b2c62ebaaebd9b5b785844634d4217278d822751494",
       "argmax_agreement": 0.992,
@@ -97,7 +99,8 @@
       }
     },
     "BF16": {
-      "filename": "Jev-Style-v2-BF16-Calibrated.gguf",
       "weight_bytes": 3775700896,
       "sha256": "8baa111eec6e30a5c9e97d559b53127a19ef30171e8aed26673153b4f77adcbb",
       "argmax_agreement": 0.996,
@@ -146,5 +149,6 @@
         "passed": true
       }
     }
-  }
 }

   "metric": "real-label task-macro accuracy; teacher-reference decisions excluded from this macro",
   "variants": {
     "Q4_K_M": {
+      "filename": "Jev-Style-v2-Calibrated-Q4_K_M.gguf",
+      "build_filename": "Jev-Style-v2-Q4_K_M-Calibrated.gguf",
       "weight_bytes": 1274388384,
       "sha256": "c697d3b29d07fdd37b6ebeb5c98066f4c31632162adca6258db23f75184eb0c4",
       "argmax_agreement": 0.914,
       }
     },
     "Q8_0": {
+      "filename": "Jev-Style-v2-Calibrated-Q8_0.gguf",
+      "build_filename": "Jev-Style-v2-Q8_0-Calibrated.gguf",
       "weight_bytes": 2012004256,
       "sha256": "5c2aa0d35b24a27f03228b2c62ebaaebd9b5b785844634d4217278d822751494",
       "argmax_agreement": 0.992,
       }
     },
     "BF16": {
+      "filename": "Jev-Style-v2-Calibrated-BF16.gguf",
+      "build_filename": "Jev-Style-v2-BF16-Calibrated.gguf",
       "weight_bytes": 3775700896,
       "sha256": "8baa111eec6e30a5c9e97d559b53127a19ef30171e8aed26673153b4f77adcbb",
       "argmax_agreement": 0.996,
         "passed": true
       }
     }
+  },
+  "filename_note": "filename is the current published name; build_filename is the name used when the file was validated (renamed 2026-09-24 for Ollama quant tags; bytes and sha256 unchanged)."
 }