Instructions to use Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF") model = AutoModelForMultimodalLM.from_pretrained("Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M
Use Docker
docker model run hf.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M
- SGLang
How to use Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Ollama
How to use Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF with Ollama:
ollama run hf.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF with Docker Model Runner:
docker model run hf.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M
- Lemonade
How to use Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
can't run in ollama 0.18.2
Error: 500 Internal Server Error: unable to load model: C:\Users\Owner.ollama\models\blobs\sha256-51ce67b6936e98b60abc4f61af79c3f5edd1610871818a917df55baae06d631b
C:\Users\Owner>ollama --version
ollama version is 0.18.2
Perhaps it's the Ollama model, Ollama tends to not like direct GGUF Qwen models with image support. [there is a similar issue with Qwen3-VL] {I believe some models run in Ollama's internal "new engine"}
$> ollama --version
ollama version is 0.18.0
$> ollama run fredrezones55/qwen3.5-opus:27b "In a sentence please discribe what this image is saying." ./s-going-soup.png
The user wants a sentence description of the image's meaning.
1. **Analyze the image:**
* **Text:** "That's it, you're going in the soup" in a gothic font.
* **Visuals:** A calico cat holding a wooden spoon, looking angry/aggressive. Next to it is a bubbling
cauldron over a fire.
* **Context:** This is a meme. The phrase "go in the soup" usually means failure or ruin, but here it's
taken literally in a witchy/horror context (like the cat is a witch brewing something). The cat looks like it's
about to cook someone or something.
2. **Determine the meaning:**
* The cat is declaring that someone (or something) is about to be put into a cauldron.
* It's a threat.
* It combines the idiom "go in the soup" (meaning to fail or get into trouble) with a literal, scary
interpretation involving a witch's cauldron.
3. **Draft sentences:**
* A cat is threatening to put someone in a soup pot.
* The image shows a cat holding a spoon next to a cauldron, saying "That's it, you're going in the soup,"
implying a threat to cook someone.
* This meme depicts an angry cat threatening to throw someone into a bubbling cauldron, playing on the
phrase "going in the soup."
4. **Refine for the user's request ("In a sentence please discribe what this image is saying"):**
* The user wants to know *what the image is saying*. This implies interpreting the message.
* The message is a threat.
* "The image features an angry cat holding a wooden spoon next to a bubbling cauldron, delivering the
threat, 'That's it, you're going in the soup,' suggesting that someone is about to be cooked or punished
severely." -> A bit long.
* "An angry cat holding a wooden spoon threatens to throw someone into a bubbling cauldron with the caption
'That's it, you're going in the soup.'" -> Accurate and concise.
5. **Final Polish:** Let's focus on the humorous/threatening aspect.
* "This meme shows an angry cat holding a spoon next to a cauldron, threateningly declaring that someone is
about to be cooked in the soup."
Let's try to capture the specific "saying" part.
"The image conveys a humorous threat where an angry cat, acting like a witch, declares that someone is about to be
thrown into a bubbling cauldron."
Let's go with a straightforward description of the action and text.
"The image depicts an angry cat holding a wooden spoon next to a bubbling cauldron, threateningly stating, 'That's
it, you're going in the soup.'"
Wait, the prompt asks "what this image is saying". It's asking for the *message*.
The message is a threat.
"The image is a meme depicting an angry cat threatening to cook someone in a cauldron, playing on the idiom 'going
in the soup' which usually means failing or getting into trouble."
Let's stick to a simple descriptive sentence that captures the essence.
"This image is a meme featuring an angry cat holding a spoon next to a bubbling cauldron, threateningly declaring
that someone is about to be put 'in the soup.'"
Let's make it punchy.
"An angry cat holding a wooden spoon threatens to throw someone into a bubbling cauldron, declaring, 'That's it,
you're going in the soup.'"
This covers the visual elements and the text.
Let's try one more angle. Is it referring to the movie *Harry Potter*? No, not specifically. Is it a specific cat
meme? It looks like a generic "angry cat" meme template but edited.
Okay, the best sentence is:
**"An angry cat holding a wooden spoon threatens to throw someone into a bubbling cauldron, declaring, 'That's it,
you're going in the soup.'"**
An angry cat holding a wooden spoon threatens to throw someone into a bubbling cauldron, declaring, "That's it,
you're going in the soup."
[Slightly over thought huh.]
$> ollama run fredrezones55/qwen3.5-opus:27b "Hello?"
Think:
Let me think through this carefully.
The question/task: Hello?
My reasoning: The most appropriate response is a friendly greeting. I'm here to help you with whatever you have in
mind!
Hello! π
It looks like you may have just said "Hello?" or perhaps you intended to ask a question but didn't include it yet.
I'm here to help you with whatever you have in mind!
Whether you have:
- A question I can answer
- A problem you'd like me to solve
- Something you'd like to discuss
- A task you need help completing
Please feel free to share what's on your mind, and I'll do my best to assist you.
I can find the v1 on ollama.com, but is there a v2 uploaded yet?
