Text Generation
GGUF
jev-style
English
decision-model
classification
calibration
qwen3.5
single-prefill
conversational
Instructions to use chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- jev-style
How to use chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF with jev-style:
pip install jev-style # GGUF builds score through llama.cpp: build the jev-score binary once hf download chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF build_jev_score.sh jev_score.cpp --local-dir jev-score export JEV_SCORE_BIN=$(sh jev-score/build_jev_score.sh /path/to/llama.cpp | tail -n 1)
from jev_style import JevStyle, noul, choice js = JevStyle.from_pretrained("chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF") out = js.decide("I was charged twice for one order.", { "billing": noul("This message is about billing."), "team": choice("Which team should handle it?", ["billing", "shipping", "tech"]), }) print(out["answers"]["team"]["choice"]) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M
Use Docker
docker model run hf.co/chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M
- Ollama
How to use chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF with Ollama:
ollama run hf.co/chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF with Docker Model Runner:
docker model run hf.co/chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M
- Lemonade
How to use chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Jev-Style-Qwen3.5-2B-Decision-v2-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Download SHA256SUMS.json from chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF: direct link, hf CLI and curl.
- Browser
- Download file 4.64 kB
-
https://huggingface.co/chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF/resolve/main/SHA256SUMS.json
- Command line
-
hf download hf://chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF/SHA256SUMS.json
-
curl -L -o SHA256SUMS.json https://huggingface.co/chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF/resolve/main/SHA256SUMS.json
4.64 kB
| { | |
| "Jev-Style-v2-Calibrated-Q8_0.calibration.json": { | |
| "bytes": 1807, | |
| "sha256": "7f3ae96ebf0271483365b8941124f3be9521e9e3ed4c3d62240e51c9b416ebb4" | |
| }, | |
| "Jev-Style-v2-Calibrated-Q8_0.gguf": { | |
| "bytes": 2012004256, | |
| "sha256": "5c2aa0d35b24a27f03228b2c62ebaaebd9b5b785844634d4217278d822751494" | |
| }, | |
| "LICENSE": { | |
| "bytes": 11343, | |
| "sha256": "50cbab8a892c5f2993b8c7351a99182507472def3b1374558308605d99b86b32" | |
| }, | |
| "README.md": { | |
| "bytes": 12565, | |
| "sha256": "081e42c5a4b989a746cc99aabeeb1a1d83adb861affc601dd5aee8a22904c5c9" | |
| }, | |
| "evaluation/baseline_sensitivity.json": { | |
| "bytes": 2015, | |
| "sha256": "aadc4566710a397044edbfb9c353ee2e0baf14a6deb5e30f4234eabd66294026" | |
| }, | |
| "evaluation/data_manifest.json": { | |
| "bytes": 10680, | |
| "sha256": "8d03d4bba9400ffe062b248733213a6e67a3d6c187340e9aa796c319f2feef23" | |
| }, | |
| "evaluation/deployment.json": { | |
| "bytes": 963, | |
| "sha256": "a49d48a3c5cd3173dfc6badb11d33a3e22120e9a559b8e4ccbb2bd91653bc757" | |
| }, | |
| "evaluation/http_smoke.json": { | |
| "bytes": 542, | |
| "sha256": "b33c0ea9f7a6cf49bba573b98dcf966c1dbf033ef21e6b9a92c8bc217a46a9f3" | |
| }, | |
| "evaluation/laya_source.json": { | |
| "bytes": 97, | |
| "sha256": "b12ecce8dca6afad9e43c31c2385be255514d78856009ef9a6222bc955ca0757" | |
| }, | |
| "evaluation/laya_typed_source.json": { | |
| "bytes": 113, | |
| "sha256": "c2c2bc4931bb489636c5471f25f2fa31c3baca347c0465f119b8039aac1d3619" | |
| }, | |
| "evaluation/reference_comparison.json": { | |
| "bytes": 27042, | |
| "sha256": "36f56a490fc67e774cbb25b38d4dc9b5e3a42e1f65fcd748af092e127a98eff8" | |
| }, | |
| "gguf_logits.cpp": { | |
| "bytes": 3001, | |
| "sha256": "a29f2dbd608f6448a6d440995baed130c8ae35c1a2bed0e653f5da0d32b6b067" | |
| }, | |
| "jev_decision_client.py": { | |
| "bytes": 4113, | |
| "sha256": "7d02a340b80326ceefc6d20378bdf7c05cb5087b8ee243394eebd4c43eb47b36" | |
| }, | |
| "requirements.txt": { | |
| "bytes": 64, | |
| "sha256": "7265d12f38635b9f592b0e2d62b1cc0a9788ebaa2f37d84eea2212ae16952d7c" | |
| }, | |
| "evaluation/chart_data.json": { | |
| "bytes": 4588, | |
| "sha256": "0db9855803eb514792a906ff04b1d9ccdcba81babeedcffdd9ae34131f8b04f1" | |
| }, | |
| "figures/calibration.svg": { | |
| "bytes": 28967, | |
| "sha256": "8433aa3a878362e977defe21ccc4400d63c3eba07afa7725535c6fa7fb93d0fc" | |
| }, | |
| "figures/robustness.png": { | |
| "bytes": 125258, | |
| "sha256": "41c317f25853b88452c8515cddf8773fbcbd3d354ec718d90ab6bc01716bd4a0" | |
| }, | |
| "figures/benchmark.png": { | |
| "bytes": 188910, | |
| "sha256": "5a3a782cf203a586a8ffa7bef3006a79906ad4553b8548af4b62a63f6723a6dc" | |
| }, | |
| "figures/benchmark.svg": { | |
| "bytes": 22570, | |
| "sha256": "187ee46cfdccc078fa5725e9668b28dad2456b9920d6e1bc26f6ec133a9d5e0a" | |
| }, | |
| "figures/robustness.svg": { | |
| "bytes": 11821, | |
| "sha256": "1f54a0e66a5bcceafad85595f91db1a0ed5f0eb6b4c45b726bafd8a941f53600" | |
| }, | |
| "figures/calibration.png": { | |
| "bytes": 183790, | |
| "sha256": "2a4e163354b0d0d31bf5fcdfcfbcf726fb152fb2f6419432ddddaec3b475a921" | |
| }, | |
| "evaluation/quantization_summary.json": { | |
| "bytes": 5844, | |
| "sha256": "9ec0233027766c2104450fa45d2a7c4cfea66fcde786d5ad699691e81d9f2030" | |
| }, | |
| "evaluation/calibration_batch_parity.json": { | |
| "bytes": 147, | |
| "sha256": "1f819d59b9ebf6ffb79953832c3f7ae4564d7bae7fa68209b6583a9d92db2bf5" | |
| }, | |
| "Jev-Style-v2-Calibrated-Q4_K_M.gguf": { | |
| "bytes": 1274388384, | |
| "sha256": "c697d3b29d07fdd37b6ebeb5c98066f4c31632162adca6258db23f75184eb0c4" | |
| }, | |
| "Jev-Style-v2-Calibrated-Q4_K_M.calibration.json": { | |
| "bytes": 1824, | |
| "sha256": "4a5abb23d86ddd44b8b6b1054b48a0a9c40560cc0ee3edd24ae6be8e623ad5f1" | |
| }, | |
| "evaluation/q4_k_m_deployment.json": { | |
| "bytes": 941, | |
| "sha256": "b67e128b7e46287a84ce487171882b4b8ece888c28cfa92d09e72fefec298bcc" | |
| }, | |
| "evaluation/q4_k_m_http_smoke.json": { | |
| "bytes": 694, | |
| "sha256": "5a0e2b0696e3aa918aff6e8ad2ebf1532091edf81a10815bf3add221c36932d3" | |
| }, | |
| "Jev-Style-v2-Calibrated-BF16.gguf": { | |
| "bytes": 3775700896, | |
| "sha256": "8baa111eec6e30a5c9e97d559b53127a19ef30171e8aed26673153b4f77adcbb" | |
| }, | |
| "Jev-Style-v2-Calibrated-BF16.calibration.json": { | |
| "bytes": 1840, | |
| "sha256": "e5eb54d405d213c751ba61b334a8370d03332c915d9c49725da3b7a6ee295109" | |
| }, | |
| "evaluation/bf16_deployment.json": { | |
| "bytes": 946, | |
| "sha256": "19cbc98b2c892509f804b4b758ab5009dbcd5fef9807b4cbedb4f7114a4efab2" | |
| }, | |
| "evaluation/bf16_http_smoke.json": { | |
| "bytes": 690, | |
| "sha256": "d729e4dd7ab5d425e5eb031ca1dc92d74302a7971d9aeeba782c15e4904a5f59" | |
| }, | |
| "template": { | |
| "bytes": 91, | |
| "sha256": "244d5cfdeb97383239af17d352fc32c2123257b2eecda8fd8285a7cd29770648" | |
| }, | |
| "params": { | |
| "bytes": 31, | |
| "sha256": "208f455e3685ae841dea00e45a06d3b7fd226a0fbf88e6f2cd1077ff8e6b36af" | |
| } | |
| } | |