Instructions to use prism-ml/Ternary-Bonsai-27B-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Inference
- HuggingChat
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use prism-ml/Ternary-Bonsai-27B-gguf with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf prism-ml/Ternary-Bonsai-27B-gguf:F16 # Run inference directly in the terminal: llama cli -hf prism-ml/Ternary-Bonsai-27B-gguf:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf prism-ml/Ternary-Bonsai-27B-gguf:F16 # Run inference directly in the terminal: llama cli -hf prism-ml/Ternary-Bonsai-27B-gguf:F16
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf prism-ml/Ternary-Bonsai-27B-gguf:F16 # Run inference directly in the terminal: ./llama-cli -hf prism-ml/Ternary-Bonsai-27B-gguf:F16
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf prism-ml/Ternary-Bonsai-27B-gguf:F16 # Run inference directly in the terminal: ./build/bin/llama-cli -hf prism-ml/Ternary-Bonsai-27B-gguf:F16
Use Docker
docker model run hf.co/prism-ml/Ternary-Bonsai-27B-gguf:F16
- LM Studio
- Jan
- vLLM
How to use prism-ml/Ternary-Bonsai-27B-gguf with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "prism-ml/Ternary-Bonsai-27B-gguf" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "prism-ml/Ternary-Bonsai-27B-gguf", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/prism-ml/Ternary-Bonsai-27B-gguf:F16
- Ollama
How to use prism-ml/Ternary-Bonsai-27B-gguf with Ollama:
ollama run hf.co/prism-ml/Ternary-Bonsai-27B-gguf:F16
- Unsloth Desktop
- Pi
How to use prism-ml/Ternary-Bonsai-27B-gguf with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf prism-ml/Ternary-Bonsai-27B-gguf:F16
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "prism-ml/Ternary-Bonsai-27B-gguf:F16" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use prism-ml/Ternary-Bonsai-27B-gguf with Docker Model Runner:
docker model run hf.co/prism-ml/Ternary-Bonsai-27B-gguf:F16
- Lemonade
How to use prism-ml/Ternary-Bonsai-27B-gguf with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull prism-ml/Ternary-Bonsai-27B-gguf:F16
Run and chat with the model
lemonade run user.Ternary-Bonsai-27B-gguf-F16
List all available models
lemonade list
- Hermes Agent
How to use prism-ml/Ternary-Bonsai-27B-gguf with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf prism-ml/Ternary-Bonsai-27B-gguf:F16
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default prism-ml/Ternary-Bonsai-27B-gguf:F16
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use prism-ml/Ternary-Bonsai-27B-gguf with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf prism-ml/Ternary-Bonsai-27B-gguf:F16
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "prism-ml/Ternary-Bonsai-27B-gguf:F16" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Appreciation Post
I'm gonna just leave it here that this model works better than gemini 3.1 pro in coding tasks at the moment.
I see a lot of people being all sorts of weird about llama.cpp implementation and I ask them to download the prebuilt versions if they can't build the repo properly and don't mess with dials like it's a normal linear quant.
keep on rocking and please do higher tier models if possible.
nb
I wouldn't say better than Pro 3.1 in any of my experiences - but I will say, it's "ideas" are better, just execution is definitely much lower.
In asking it to help with ideas for events/story line to close out the last chapter in a children's game I'm making (A dog that steals food from it's owner in various parts of the home) it decided that the owner is actually an evil veterinarian running experiments on the dog, because he had lost his own daughter, and was attempting to find out how to cure his depression via .. dog stuff? Then essentially, the ending was a "Then the vet offs himself in guilt, and the dog watches the sunrise" - it was much more of a twist than I expected, but obviously also a wee bit too dark for a children's game lol
I wouldn't say better than Pro 3.1 in any of my experiences - but I will say, it's "ideas" are better, just execution is definitely much lower.
In asking it to help with ideas for events/story line to close out the last chapter in a children's game I'm making (A dog that steals food from it's owner in various parts of the home) it decided that the owner is actually an evil veterinarian running experiments on the dog, because he had lost his own daughter, and was attempting to find out how to cure his depression via .. dog stuff? Then essentially, the ending was a "Then the vet offs himself in guilt, and the dog watches the sunrise" - it was much more of a twist than I expected, but obviously also a wee bit too dark for a children's game lol
it's as good as you can make it, it literally zero shotted a problem that pro 3.1 kept making into a buggy code for 10 days (on and off) and it was so infuriating that I just gave up. it might not be as great in agentic coding with every environment, but that's a given since this model wasn't created with agentic coding in mind.
but as a stand alone model on its own, it's absolutely top tier.
I'm breezing through code tasks with it. very stable compared to the quantization it went through.
just had a very very productive session with it designing something very out of distribution. literally sitting here after just taking a breather. lol.
also yes! it's great for creative writing, you might want to give it a proper system prompt for that so it knows what constraints it's working against, pretty sure it won't disappoint with a proper system prompt in place.
I wouldn't say better than Pro 3.1 in any of my experiences - but I will say, it's "ideas" are better, just execution is definitely much lower.
In asking it to help with ideas for events/story line to close out the last chapter in a children's game I'm making (A dog that steals food from it's owner in various parts of the home) it decided that the owner is actually an evil veterinarian running experiments on the dog, because he had lost his own daughter, and was attempting to find out how to cure his depression via .. dog stuff? Then essentially, the ending was a "Then the vet offs himself in guilt, and the dog watches the sunrise" - it was much more of a twist than I expected, but obviously also a wee bit too dark for a children's game lol
it's as good as you can make it, it literally zero shotted a problem that pro 3.1 kept making into a buggy code for 10 days (on and off) and it was so infuriating that I just gave up. it might not be as great in agentic coding with every environment, but that's a given since this model wasn't created with agentic coding in mind.
but as a stand alone model on its own, it's absolutely top tier.
I'm breezing through code tasks with it. very stable compared to the quantization it went through.
just had a very very productive session with it designing something very out of distribution. literally sitting here after just taking a breather. lol.also yes! it's great for creative writing, you might want to give it a proper system prompt for that so it knows what constraints it's working against, pretty sure it won't disappoint with a proper system prompt in place.
I just want to tell you gemini 3.6 flash is better for coding than gemini 3.1 pro
I wouldn't say better than Pro 3.1 in any of my experiences - but I will say, it's "ideas" are better, just execution is definitely much lower.
In asking it to help with ideas for events/story line to close out the last chapter in a children's game I'm making (A dog that steals food from it's owner in various parts of the home) it decided that the owner is actually an evil veterinarian running experiments on the dog, because he had lost his own daughter, and was attempting to find out how to cure his depression via .. dog stuff? Then essentially, the ending was a "Then the vet offs himself in guilt, and the dog watches the sunrise" - it was much more of a twist than I expected, but obviously also a wee bit too dark for a children's game lol
it's as good as you can make it, it literally zero shotted a problem that pro 3.1 kept making into a buggy code for 10 days (on and off) and it was so infuriating that I just gave up. it might not be as great in agentic coding with every environment, but that's a given since this model wasn't created with agentic coding in mind.
but as a stand alone model on its own, it's absolutely top tier.
I'm breezing through code tasks with it. very stable compared to the quantization it went through.
just had a very very productive session with it designing something very out of distribution. literally sitting here after just taking a breather. lol.also yes! it's great for creative writing, you might want to give it a proper system prompt for that so it knows what constraints it's working against, pretty sure it won't disappoint with a proper system prompt in place.
I just want to tell you gemini 3.6 flash is better for coding than gemini 3.1 pro
sadly for google and happily for me, my days of feeding my data to google for mediocre results is behind me.