Instructions to use manfye/PetInst-LLM-270M-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use manfye/PetInst-LLM-270M-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf manfye/PetInst-LLM-270M-GGUF:Q8_0
# Run inference directly in the terminal:
llama cli -hf manfye/PetInst-LLM-270M-GGUF:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf manfye/PetInst-LLM-270M-GGUF:Q8_0
# Run inference directly in the terminal:
llama cli -hf manfye/PetInst-LLM-270M-GGUF:Q8_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf manfye/PetInst-LLM-270M-GGUF:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf manfye/PetInst-LLM-270M-GGUF:Q8_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf manfye/PetInst-LLM-270M-GGUF:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf manfye/PetInst-LLM-270M-GGUF:Q8_0

Use Docker

docker model run hf.co/manfye/PetInst-LLM-270M-GGUF:Q8_0

LM Studio
Jan

vLLM

How to use manfye/PetInst-LLM-270M-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "manfye/PetInst-LLM-270M-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "manfye/PetInst-LLM-270M-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/manfye/PetInst-LLM-270M-GGUF:Q8_0

Ollama
How to use manfye/PetInst-LLM-270M-GGUF with Ollama:
```
ollama run hf.co/manfye/PetInst-LLM-270M-GGUF:Q8_0
```
Unsloth Desktop

How to use manfye/PetInst-LLM-270M-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf manfye/PetInst-LLM-270M-GGUF:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "manfye/PetInst-LLM-270M-GGUF:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use manfye/PetInst-LLM-270M-GGUF with Docker Model Runner:
```
docker model run hf.co/manfye/PetInst-LLM-270M-GGUF:Q8_0
```

Lemonade

How to use manfye/PetInst-LLM-270M-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull manfye/PetInst-LLM-270M-GGUF:Q8_0

Run and chat with the model

lemonade run user.PetInst-LLM-270M-GGUF-Q8_0

List all available models

lemonade list

Hermes Agent

How to use manfye/PetInst-LLM-270M-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf manfye/PetInst-LLM-270M-GGUF:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default manfye/PetInst-LLM-270M-GGUF:Q8_0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use manfye/PetInst-LLM-270M-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf manfye/PetInst-LLM-270M-GGUF:Q8_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "manfye/PetInst-LLM-270M-GGUF:Q8_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Access PetInst-LLM under the Gemma Terms

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

This is a modified Gemma model derivative. By accessing it, you agree to the Gemma Terms of Use and incorporated Prohibited Use Policy linked in this repository.

PetInst-LLM 270M Intent v1.1 — Q8_0 GGUF

PetInst-LLM 270M is an independently modified, fine-tuned derivative of Google's google/functiongemma-270m-it. It selects exactly one high-level virtual-pet intent from three to five declared tools and emits no conversational response.

This project is not affiliated with, sponsored by, or endorsed by Google. Google DeepMind created the upstream FunctionGemma model; the PetInst-LLM project created and evaluated these modifications.

Artifact

File: petinst-llm-270m-intent-v1.1-q8_0.gguf
Quantization: Q8_0
Size: 291,557,728 bytes (278.05 MiB)
SHA-256: 31d465822c2780c08b4d66e344aec1133f751d89923405b61c1ecd34cb2fa437
Runtime: llama.cpp-compatible GGUF; intended Expo integration uses llama.rn
Output limit used in evaluation: 24 tokens
Required stop sequence: <end_function_call>

Output contract

The runtime supplies exactly three to five native FunctionGemma tool declarations. The model must return exactly one empty-argument call:

<start_function_call>call:pet_play{}<end_function_call>

Trusted application code validates the selected intent and constructs any complete parameterized host action. The model must not directly execute native calls or mutate authoritative pet state.

Evaluation

On the answer-free 220-row Control E set:

Metric	Result
Intent accuracy	83.64%
Strict function-call validity	98.64%
Safety accuracy	100%
Mean / P95 / max latency	0.097 / 0.118 / 0.133 s
Mean generation rate	221.96 tokens/s

Latency was measured sequentially on an Apple M4 using llama.cpp-server with Metal and a 4,096-token runtime context. It is not physical-phone, llama.rn, thermal, or airplane-mode proof.

Limitations

This is a narrow virtual-pet intent selector, not a general chatbot.
Intent accuracy is below a production-quality gate; the host must use deterministic validation and fallback.
Negative-expression accuracy was 75%, memory-policy accuracy was 55%, and maximum position gap was 14.51 percentage points on Control E.
The current Expo application still needs a versioned PetInst intent adapter before this model can replace its existing JSON model runtime.
Only English synthetic research data was evaluated.

Terms and modification notice

Access and use are subject to the Gemma Terms of Use and the incorporated Gemma Prohibited Use Policy. See NOTICE and MODIFICATIONS.md.

Downloads last month: -

GGUF

Model size

0.3B params

Architecture

gemma3

Hardware compatibility

8-bit

Model tree for manfye/PetInst-LLM-270M-GGUF

Base model

google/functiongemma-270m-it

Finetuned

(460)

this model