Instructions to use Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL
# Run inference directly in the terminal:
llama cli -hf Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL
# Run inference directly in the terminal:
llama cli -hf Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL
# Run inference directly in the terminal:
./llama-cli -hf Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL

Use Docker

docker model run hf.co/Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL

LM Studio
Jan

vLLM

How to use Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL

Ollama
How to use Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF with Ollama:
```
ollama run hf.co/Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL
```
Unsloth Desktop

How to use Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF with Docker Model Runner:
```
docker model run hf.co/Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL
```

Lemonade

How to use Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL

Run and chat with the model

lemonade run user.Treebeard-Qwen3.6-35B-A3B-GGUF-UD-Q5_K_XL

List all available models

lemonade list

Hermes Agent

How to use Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF:UD-Q5_K_XL" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Treebeard

Linux package for Qwen3.6-35B-A3B: one Q5_K_XL GGUF (~26.6 GB), optional F16 vision projector, Qwen tokenizer/metadata, platform runtimes (portable CPU, Intel SYCL, NVIDIA CUDA), installer, and local llama-server with /v1/chat/completions on loopback by default.

Package line on the card: release 0.1.1 / runtime 0.1.0-rc.3 (launcher pkg4) unless a newer archive is listed in this repo. Weights are commodity open Qwen GGUF; this repo is packaging + runtime evidence.

Site (evidence tables): https://newjordan.github.io/treebeard/
GitHub: https://github.com/newjordan/treebeard

Install on Linux

About 26.7 GB download (resumable). Installer checks file SHA-256. Roughly 32 GB system or unified memory.

curl -fsSL https://raw.githubusercontent.com/newjordan/treebeard/main/install.sh | bash
~/.local/bin/treebeard doctor
~/.local/bin/treebeard serve

curl http://127.0.0.1:8093/health

curl http://127.0.0.1:8093/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "treebeard",
    "messages": [{"role": "user", "content": "Write a haiku about old forests."}],
    "max_tokens": 96,
    "chat_template_kwargs": {"enable_thinking": false}
  }'

Vision projector (+0.9 GB):

curl -fsSL https://raw.githubusercontent.com/newjordan/treebeard/main/install.sh | \
  bash -s -- --multimodal

Useful installer flags: --backend, --install-dir, --bin-dir, --allow-low-memory.

Platforms

Backend	Platform	Host dependency	Validation row
CPU	Linux x86_64	glibc 2.35+	Ryzen 9 5950X package chat + tool-call smoke
Intel SYCL	Linux x86_64	oneAPI 2026, Level Zero	Arc Pro B70 agent + serving evidence
NVIDIA CUDA	Linux ARM64	CUDA 13, cuBLAS, driver	GB10 agent + native-bench + package smoke

NVIDIA GPU acceleration in this package line is the ARM64 GB10 path. x86_64 NVIDIA hosts use the portable CPU runtime unless a later package adds a CUDA x86 build.

Agent Bench freeze (named run)

Field	Value
Test	tool-eval-bench 2.1.0 public 69
Score	94/100 (130/138); 63 pass / 4 partial / 2 fail; 0 request errors
Shape	np=1 · c=262144 · temp 0 · thinking off · seed 42 · commit `8b3259b`
Hardware	Intel Arc Pro B70 and NVIDIA GB10
Artifacts	result.json, nvidia-result.json

Report UI: https://newjordan.github.io/treebeard/
This is a checksum-pinned freeze for a named package/runtime. It is the score in those result files, not a guarantee for every local install or quant.

Control A/B (2026-07-28, stock Q5, Arc Pro B70)

Clean upstream SYCL binary versus Treebeard package binary + package env. Same weights: Q5_K_XL sha256 25233af7642e3a91bd52cc4aeefdbd4a117479088e06cf1aea5b6bedb443c506.

Test	Shape	Control	Treebeard	Artifact
tool-eval-bench 69	np=1 · c=262144 · seed 42	91/100 (125/138)	91/100 (126/138)	GitHub agent-bench-ab REPORT
ho-pack-v1.1	np=1 · seed 42	42/46	42/46	same dir `heldout/`
sequential tg_p50	np=1 · c=32768 · 5×2 prompts	77.1 t/s	88.9 t/s	single-agent-ab
12-agent ABA p50/agent	np=12 · n_predict=96	6.88	26.33	base-vs-package-aba

Multi-slot p50 is concurrent capacity. Sequential tg is single-stream. Full notes: RELEASE-20260728.md.

Other measurements

Test	Result	Artifact
llama-bench pp4096 (GB10)	2,422.325 tok/s	evidence/nvidia / package BENCHMARKS
llama-bench tg128 (GB10)	59.614 tok/s	same
Q8_0 12-col latency (Blackwell)	31.49% lower	docs/BENCHMARKS.md
Q8_0 MoE-down latency (Blackwell)	4.01% lower	same
12-slot aggregate serving (B70)	194.023 tok/s	docs/BENCHMARKS.md
CPU package chat smoke (5950X, c=4096)	9.30 tok/s	evidence/cpu-linux-x86_64
CPU package tool-call smoke	7.40 tok/s	same

CPU numbers are smoke at 4,096 context, not a cross-hardware promise. Detail and scope: docs/BENCHMARKS.md.

Commands

treebeard serve       # default: start local API
treebeard doctor      # platform selection + launch command
treebeard verify      # verify installed files
treebeard status      # health endpoint
treebeard report      # public report URL
treebeard help

Variable	Default	Meaning
`TREEBEARD_PROFILE`	`quality`	`quality`, `throughput`, or `custom`
`TREEBEARD_BACKEND`	`auto`	`auto`, `cpu`, `sycl`, or `cuda`
`TREEBEARD_CONTEXT`	backend/profile	Total context tokens
`TREEBEARD_PARALLEL`	backend/profile	Server slots
`TREEBEARD_HOST`	`127.0.0.1`	Bind address
`TREEBEARD_PORT`	`8093`	Port
`TREEBEARD_MULTIMODAL`	`0`	Load installed F16 projector
`TREEBEARD_REASONING`	`off`	`off`, `bounded`, or `unrestricted`
`TREEBEARD_REASONING_BUDGET`	64 GPU / 16 CPU	Bounded thinking tokens
`TREEBEARD_SPECULATION`	`off`	`off`, `ngram`, `mtp`, or `hybrid`
`TREEBEARD_VERIFY`	`once`	`once`, `always`, or `never`

Default GPU quality profile: 1 slot, 262144 context (shape used for the freeze). Throughput profile: 12 slots (serving saturation). Reasoning defaults off. TREEBEARD_SPECULATION is opt-in; no speed claim attached on this card.

TREEBEARD_CONTEXT=8192 TREEBEARD_PORT=8080 treebeard serve
TREEBEARD_PROFILE=throughput treebeard serve
TREEBEARD_MULTIMODAL=1 TREEBEARD_CONTEXT=32768 treebeard serve

Full package layout

This HF repo is the package (Git LFS for the GGUF). After clone/download:

bash ./verify.sh
bash ./treebeard doctor
bash ./treebeard serve

Contents include the Q5 GGUF, optional mmproj-F16.gguf, tokenizer/config, CPU/SYCL/CUDA runtime archives, install.sh / treebeard / run.sh, profiles, SHA256SUMS, and evidence under evidence/. See docs/PACKAGE.md.

Provenance (package pin)

Integration commit (package line): 6a6dc2def952fe5e9b2da81e638968653b6be3db
Release baseline: 3799687213eddbdc1389994f110fac0fa01f3e36
NVIDIA patch SHA-256: c1e0780c96432059ea7a517f6ab2db935f1083da065ed0a9009a00d944c3415f
Base model: Qwen/Qwen3.6-35B-A3B metadata revision 995ad96e
GGUF source: unsloth/Qwen3.6-35B-A3B-GGUF revision a483e9e6

Security and license

API binds to loopback by default. Non-loopback bind is an unauthenticated inference surface unless you add TLS auth and firewall rules.

Model/tokenizer: Apache-2.0. llama.cpp-derived runtimes: MIT. See LICENSE, LICENSE-RUNTIME, NOTICE.md. Unofficial packaging; not an official Qwen, Unsloth, NVIDIA, Intel, Hugging Face, or llama.cpp product.

Downloads last month: 206

GGUF

Model size

35B params

Architecture

qwen35moe

Hardware compatibility

5-bit

Model tree for Frosty40/Treebeard-Qwen3.6-35B-A3B-GGUF

Base model

Qwen/Qwen3.6-35B-A3B

Quantized

(872)

this model