Instructions to use Anomly/Qwen2.5-0.5B-Instruct-bposit8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Anomly/Qwen2.5-0.5B-Instruct-bposit8 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Anomly/Qwen2.5-0.5B-Instruct-bposit8
# Run inference directly in the terminal:
llama cli -hf Anomly/Qwen2.5-0.5B-Instruct-bposit8

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Anomly/Qwen2.5-0.5B-Instruct-bposit8
# Run inference directly in the terminal:
llama cli -hf Anomly/Qwen2.5-0.5B-Instruct-bposit8

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Anomly/Qwen2.5-0.5B-Instruct-bposit8
# Run inference directly in the terminal:
./llama-cli -hf Anomly/Qwen2.5-0.5B-Instruct-bposit8

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Anomly/Qwen2.5-0.5B-Instruct-bposit8
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Anomly/Qwen2.5-0.5B-Instruct-bposit8

Use Docker

docker model run hf.co/Anomly/Qwen2.5-0.5B-Instruct-bposit8

LM Studio
Jan
Ollama
How to use Anomly/Qwen2.5-0.5B-Instruct-bposit8 with Ollama:
```
ollama run hf.co/Anomly/Qwen2.5-0.5B-Instruct-bposit8
```
Unsloth Desktop

How to use Anomly/Qwen2.5-0.5B-Instruct-bposit8 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Anomly/Qwen2.5-0.5B-Instruct-bposit8

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Anomly/Qwen2.5-0.5B-Instruct-bposit8"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Anomly/Qwen2.5-0.5B-Instruct-bposit8 with Docker Model Runner:
```
docker model run hf.co/Anomly/Qwen2.5-0.5B-Instruct-bposit8
```

Lemonade

How to use Anomly/Qwen2.5-0.5B-Instruct-bposit8 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Anomly/Qwen2.5-0.5B-Instruct-bposit8

Run and chat with the model

lemonade run user.Qwen2.5-0.5B-Instruct-bposit8-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use Anomly/Qwen2.5-0.5B-Instruct-bposit8 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Anomly/Qwen2.5-0.5B-Instruct-bposit8

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Anomly/Qwen2.5-0.5B-Instruct-bposit8

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Anomly/Qwen2.5-0.5B-Instruct-bposit8 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Anomly/Qwen2.5-0.5B-Instruct-bposit8

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Anomly/Qwen2.5-0.5B-Instruct-bposit8" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen2.5-0.5B-Instruct-bposit8

Qwen/Qwen2.5-0.5B-Instruct converted to b-posit8 (32-code blocks with a power-of-two scale, 8-bit posit codes, es = 2) for the exact profile of INVAR: every matmul accumulates in a 256-bit quire with one rounding, so a deterministic runtime produces bit-identical activations and logits on x86, CUDA and aarch64, and independent reference implementations reproduce a served answer from these weights and the token ids.

File: Qwen2.5-0.5B-Instruct-bposit8.gguf (0.52 GB), SHA-256 7a56889a1eff44063abad894c8a6270debd10dba010556b8c1488cf2c8492651
Format: GGUF general.file_type 42 (b-posit8), llama-cpp-et fork (deterministic backend)
Spec: docs/EXACT-PROFILE-SPEC.md in the INVAR repo; conformance fixture and tokenizer vectors in go/crverify/testdata
Licence: this is a re-quantised copy of Qwen/Qwen2.5-0.5B-Instruct; the upstream licence (apache-2.0) applies unchanged, including any use restrictions and attribution requirements.

Run it: invar serve --model Qwen2.5-0.5B-Instruct-bposit8.gguf --binary llama-cli --spot-check --spot-check-units, then invar verify worldline.jsonl --model Qwen2.5-0.5B-Instruct-bposit8.gguf --binary llama-cli --spot-check --units --reexec.

Why the Hub says "not able to determine the quantization variants"

The Hub's GGUF viewer recognises the quantization types shipped in upstream llama.cpp (Q4_K_M, Q8_0, F16, MXFP4 and so on) from the file name. b-posit8 is a new tensor type (GGUF tensor type 43, file type 42) that is not in that list yet, so the viewer cannot label it; the file itself is a valid GGUF and the metadata viewer reads it fine. Storage is 8 bits per weight plus one signed 8-bit power-of-two scale per 32 weights (8.25 bits per weight), comparable to Q8_0. It runs only on the deterministic llama.cpp fork that INVAR uses, not on stock llama.cpp or Ollama; that is the point of the profile, and the spec is public so other runtimes can implement it.

Downloads last month: 93

GGUF

Model size

0.5B params

Architecture

qwen2

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Anomly/Qwen2.5-0.5B-Instruct-bposit8

Base model

Qwen/Qwen2.5-0.5B

Finetuned

Qwen/Qwen2.5-0.5B-Instruct

Quantized

(278)

this model