Instructions to use mattjoyce/antares-1b-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use mattjoyce/antares-1b-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mattjoyce/antares-1b-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mattjoyce/antares-1b-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mattjoyce/antares-1b-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mattjoyce/antares-1b-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mattjoyce/antares-1b-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf mattjoyce/antares-1b-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mattjoyce/antares-1b-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mattjoyce/antares-1b-GGUF:Q4_K_M

Use Docker

docker model run hf.co/mattjoyce/antares-1b-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use mattjoyce/antares-1b-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mattjoyce/antares-1b-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mattjoyce/antares-1b-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/mattjoyce/antares-1b-GGUF:Q4_K_M

Ollama
How to use mattjoyce/antares-1b-GGUF with Ollama:
```
ollama run hf.co/mattjoyce/antares-1b-GGUF:Q4_K_M
```
Unsloth Desktop

How to use mattjoyce/antares-1b-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mattjoyce/antares-1b-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "mattjoyce/antares-1b-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use mattjoyce/antares-1b-GGUF with Docker Model Runner:
```
docker model run hf.co/mattjoyce/antares-1b-GGUF:Q4_K_M
```

Lemonade

How to use mattjoyce/antares-1b-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mattjoyce/antares-1b-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.antares-1b-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use mattjoyce/antares-1b-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mattjoyce/antares-1b-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default mattjoyce/antares-1b-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use mattjoyce/antares-1b-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mattjoyce/antares-1b-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "mattjoyce/antares-1b-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

antares-1b-GGUF / stability.svg

mattjoyce

run-stability chart

c7fd470 verified about 2 months ago

preview code

Raw

History Blame

3.95 kB

	<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 760 300" font-family="system-ui, -apple-system, 'Segoe UI', sans-serif">
	<style>
	.surface { fill: #fcfcfb; }
	.ink { fill: #0b0b0b; }
	.ink2 { fill: #52514e; }
	.muted { fill: #898781; }
	.grid { stroke: #e1e0d9; }
	.axis { stroke: #c3c2b7; }
	.s1f { fill: #2a78d6; } .s1s { stroke: #2a78d6; }
	.s2f { fill: #eb6834; } .s2s { stroke: #eb6834; }
	@media (prefers-color-scheme: dark) {
	.surface { fill: #1a1a19; }
	.ink { fill: #ffffff; }
	.ink2 { fill: #c3c2b7; }
	.grid { stroke: #2c2c2a; }
	.axis { stroke: #383835; }
	.s1f { fill: #3987e5; } .s1s { stroke: #3987e5; }
	.s2f { fill: #d95926; } .s2s { stroke: #d95926; }
	}
	</style>
	<rect class="surface" x="0" y="0" width="760" height="300" rx="8"/>

	<text class="ink" x="16" y="27" font-size="15" font-weight="600">Same file, three benchmark runs — spread by rung</text>

	<!-- legend -->
	<circle class="s1f" cx="559" cy="23" r="5"/>
	<text class="ink2" x="569" y="27" font-size="12">static</text>
	<rect class="s2f" x="623" y="17" width="10" height="10" transform="rotate(45 628 22)"/>
	<text class="ink2" x="636" y="27" font-size="12">imatrix</text>

	<!-- gridlines -->
	<line class="grid" x1="258.3" y1="48" x2="258.3" y2="248" stroke-width="1"/>
	<line class="grid" x1="386.5" y1="48" x2="386.5" y2="248" stroke-width="1"/>
	<line class="grid" x1="514.8" y1="48" x2="514.8" y2="248" stroke-width="1"/>
	<line class="grid" x1="643" y1="48" x2="643" y2="248" stroke-width="1"/>
	<line class="axis" x1="130" y1="248" x2="740" y2="248" stroke-width="1.5"/>
	<text class="muted" x="258.3" y="266" font-size="11" text-anchor="middle">0.160</text>
	<text class="muted" x="386.5" y="266" font-size="11" text-anchor="middle">0.165</text>
	<text class="muted" x="514.8" y="266" font-size="11" text-anchor="middle">0.170</text>
	<text class="muted" x="643" y="266" font-size="11" text-anchor="middle">0.175</text>
	<text class="muted" x="435" y="284" font-size="11" text-anchor="middle">File F1, Phase A (three identical runs per rung)</text>

	<!-- row labels -->
	<g class="ink" font-size="12">
	<text x="120" y="88" text-anchor="end">Q5_K_M</text>
	<text x="120" y="136" text-anchor="end">Q4_K_M</text>
	<text x="120" y="184" text-anchor="end">Q4_K_M-imat</text>
	<text x="120" y="232" text-anchor="end">IQ4_XS</text>
	</g>

	<!-- Q5_K_M: .1720 .1738 .1711 \| mean .1723 -->
	<line class="s1s" x1="574" y1="74" x2="574" y2="94" stroke-width="2.5"/>
	<g class="s1f" opacity="0.65">
	<circle cx="566" cy="84" r="5.5"/><circle cx="612" cy="84" r="5.5"/><circle cx="543" cy="84" r="5.5"/>
	</g>
	<!-- Q4_K_M: .1602 .1608 .1621 \| mean .1610 -->
	<line class="s1s" x1="284" y1="122" x2="284" y2="142" stroke-width="2.5"/>
	<g class="s1f" opacity="0.65">
	<circle cx="263" cy="132" r="5.5"/><circle cx="279" cy="132" r="5.5"/><circle cx="312" cy="132" r="5.5"/>
	</g>
	<!-- Q4_K_M-imat: .1679 .1659 .1663 \| mean .1667 -->
	<line class="s2s" x1="430" y1="170" x2="430" y2="190" stroke-width="2.5"/>
	<g class="s2f" opacity="0.65">
	<rect x="456" y="175" width="10" height="10" transform="rotate(45 461 180)"/>
	<rect x="405" y="175" width="10" height="10" transform="rotate(45 410 180)"/>
	<rect x="415" y="175" width="10" height="10" transform="rotate(45 420 180)"/>
	</g>
	<!-- IQ4_XS: .1742 .1588 .1613 \| mean .1648 -->
	<line class="s2s" x1="381" y1="218" x2="381" y2="238" stroke-width="2.5"/>
	<g class="s2f" opacity="0.65">
	<rect x="618" y="223" width="10" height="10" transform="rotate(45 623 228)"/>
	<rect x="222" y="223" width="10" height="10" transform="rotate(45 227 228)"/>
	<rect x="287" y="223" width="10" height="10" transform="rotate(45 292 228)"/>
	</g>
	<text class="ink2" x="720" y="212" font-size="12" text-anchor="end">IQ4_XS spread: 0.0154 ≈ 8× the K-quant noise</text>
	</svg>