Instructions to use ApollonLabs/Heliactis-1-4B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use ApollonLabs/Heliactis-1-4B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use ApollonLabs/Heliactis-1-4B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ApollonLabs/Heliactis-1-4B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ApollonLabs/Heliactis-1-4B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M

Ollama
How to use ApollonLabs/Heliactis-1-4B-GGUF with Ollama:
```
ollama run hf.co/ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M
```
Unsloth Desktop

How to use ApollonLabs/Heliactis-1-4B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use ApollonLabs/Heliactis-1-4B-GGUF with Docker Model Runner:
```
docker model run hf.co/ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M
```

Lemonade

How to use ApollonLabs/Heliactis-1-4B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Heliactis-1-4B-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use ApollonLabs/Heliactis-1-4B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use ApollonLabs/Heliactis-1-4B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "ApollonLabs/Heliactis-1-4B-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

apollonlabsai commited on 13 days ago

Commit

00404a6

verified ·

1 Parent(s): 1dbb94c

Explain the name Heliactis

Browse files

Files changed (1) hide show

README.md +149 -145

README.md CHANGED Viewed

@@ -1,145 +1,149 @@
----
-license: apache-2.0
-base_model: XHToken/Spark-X2.5-4B
-language:
-- el
-- en
-library_name: llama.cpp
-pipeline_tag: text-generation
-tags:
-- gguf
-- greek
-- quantized
-- tool-calling
----
-<p align="center">
-  <img src="https://huggingface.co/ApollonLabs/Heliactis-1-4B-GGUF/resolve/main/insignia.svg" width="280" alt="ApollonLabsAI — Towards the Sun">
-</p>
-# Heliactis-1-4B — GGUF
-Quantized builds of **Heliactis 1**, an Apollon Labs fine-tune of
-[`XHToken/Spark-X2.5-4B`](https://huggingface.co/XHToken/Spark-X2.5-4B),
-for `llama.cpp`, LM Studio, Ollama and anything else that reads GGUF.
-**What it is for:** Greek that does not fall apart, code that stays intact, and
-**tool calling that knows when to stay quiet.**
-## Files
-| File | Size | Gate |
-|---|---|---|
-| `Heliactis-1-4B-F16.gguf` | 7.66 GiB | passed |
-| `Heliactis-1-4B-Q8_0.gguf` | 4.07 GiB | passed |
-| `Heliactis-1-4B-Q6_K.gguf` | 3.15 GiB | passed (imatrix) |
-| `Heliactis-1-4B-Q5_K_M.gguf` | 2.77 GiB | passed (imatrix) |
-| `Heliactis-1-4B-Q4_K_M.gguf` | 2.42 GiB | passed (imatrix) · **the default**, runs on 6 GB VRAM or plain CPU |
-Every file listed here went through the full gate below. Nothing is shipped unmeasured.
-## Usage
-```bash
-llama-server -m Heliactis-1-4B-Q4_K_M.gguf --jinja -c 8192 \
-             --cache-type-k q8_0 --cache-type-v q8_0
-```
-**Context window: 1,048,576 tokens**, inherited unchanged from the base
-(`max_position_embeddings`; 27 of 36 layers use a 512-token sliding window, which keeps
-the KV cache small). `-c 8192` above is only a default that fits small GPUs: raise it as
-far as your memory allows. The KV cache at `q8_0` costs 14,976 bytes per token, about
-14.6 GiB for the full window. Long-context recall was **not re-measured** for this release;
-see the limitations below.
-The chat template is embedded in the GGUF. The model was trained and evaluated
-with **thinking disabled**. Call it that way. `--jinja` is required for tool
-calling, otherwise the toolbox never reaches the model.
-### Identity
-This model was **not trained on any self-identity data**. If identity matters,
-set it in the system prompt:
-```
-You are Heliactis 1, a Greek/English assistant built by Apollon Labs
-on top of Spark-X2.5-4B.
-```
-## A base-model defect, reduced
-On `Spark-X2.5-4B`, 502 vocabulary entries begin with an orphan UTF-8 byte. When the model
-emits one, the output breaks and `llama-server` answers HTTP 500, even on an ordinary
-translation prompt. We [reported it upstream](https://huggingface.co/XHToken/Spark-X2.5-4B/discussions/25).
-Heliactis 1 was trained against those tokens: at the worst position we measured, their
-probability fell from **12.7%** (our unreleased intermediate fine-tune, the starting point of
-this training) to **0.17%**. Rarer, not impossible.
-**Block them at inference** to close this route (`ban-ids.json` ships in this repo):
-```bash
-llama-server -m Heliactis-1-4B-Q4_K_M.gguf --jinja \
-  $(python -c "import json;print(' '.join(f'-l {i}-inf' for i in json.load(open('ban-ids.json'))))")
-```
-Via the API, per request: `"logit_bias": [[<id>, false], ...]` for the same ids.
-A system prompt does not fix it. Full reproduction, training trade-off and a second
-route that blocking does not close: [DEFECT.md](DEFECT.md).
-## Measured results
-Instrument: 12-part gate, thinking disabled, `--jinja`, KV `q8_0`, `-c 8192`.
-Each value comes from the gate's own results files.
-| Test | Q4_K_M | Q5_K_M | Q6_K | Q8_0 | F16 | bar |
-|---|---|---|---|---|---|---|
-| Tool abstention | 15/15 | 15/15 | 15/15 | 15/15 | 15/15 | ≥ 9 |
-| Overall tool score | 45/45 | 45/45 | 45/45 | 45/45 | 45/45 | ≥ 28 |
-| Tool calls, Greek | 10/10 | 10/10 | 10/10 | 10/10 | 10/10 | ≥ 7 |
-| Tool calls, English | 20/20 | 20/20 | 20/20 | 20/20 | 20/20 | ≥ 17.5 |
-| HumanEval pass@1 | 0.7439 | 0.7622 | 0.7805 | 0.7622 | 0.7866 | ≥ 0.6952 |
-| Code fencing | 60/60 | 57/60 | 58/60 | 59/60 | 58/60 | ≥ 53 |
-| Capability eval | 52/60 | 53/60 | 55/60 | 54/60 | 54/60 | ≥ 51 |
-| Greek character validity | 0.9996 | 0.9983 | 0.9974 | 0.9974 | 0.9968 | ≥ 0.9943 |
-| Greek vocabulary | 0.9553 | 0.9536 | 0.9528 | 0.9549 | 0.9568 | ≥ 0.949 |
-| Greek meaning | 0.76 | 0.79 | 0.78 | 0.79 | 0.80 | ≥ 0.70 |
-| Median answer length (tokens) | 242 | 251.5 | 256.5 | 253.5 | 254.5 | ≤ 266 |
-| Truncated answers | 3 | 5 | 2 | 2 | 3 | ≤ 9 |
-### Honest limitations
-- **The orphan-byte defect is reduced, not removed.** See above; block the ids to close this route, and see the Lao note below.
-- **Do not use this model for Lao.** A second route to the same broken output exists: the model
-  can emit a token that ends mid-character and then not finish it. Blocking the 502 ids does not
-  stop this. In our test (greedy, one prompt per language) a Lao paragraph ended in HTTP 500
-  on this model and on our unreleased intermediate fine-tune, but not on the base model; Thai passed on all three.
-- **Lao and Thai got worse.** The 502 blocked tokens are pieces of those scripts.
-  Perplexity vs our unreleased intermediate fine-tune, fp32, 5 short outside texts (151 tokens,
-  so the noise is large): English 17.09 → 16.81 · code 18.44 → 19.30 · Greek 21.13 → 27.13 ·
-  mixed languages 56.73 → 67.90 · Lao/Thai 56.49 → 78.85. The Greek line is one sentence;
-  the gate's Greek vocabulary test is the measure we trust, and it passed.
-- **Greek meaning (0.76 on Q4_K_M)** has a wide ±0.08 band. It detects collapse, not small changes.
-- **Abstention was measured on N=15 prompts.** Evidence, not proof.
-- **Long context was not re-measured for this release.** The architecture and KV cost
-  are the base model's (14,976 bytes/token at q8_0; 9 of 36 layers full attention).
-  An unreleased earlier fine-tune on the same base recalled 3/3 two-hop facts at 231k tokens.
-  We do not carry that number over as a claim for this model.
-- **24 of the 554 code training examples use Greek identifiers.** Known, not yet fixed.
-## Training
-LoRA (r=16, alpha=32). Stage 1: SFT on 2014 examples (1212 Greek, 554 code,
-248 tool-calling; 742 of the 866 toolbox rows are decoys that teach abstention).
-Dataset sha256: `cd58f73ad7fbb339de87a3bef22af6c2ff996b971f829978c78b4f70703f26e6`.
-Stage 2: on-policy unlikelihood on the 502 tokens (856 of the model's own answers,
-weight 10), with the same SFT set alongside to hold the rest in place.
-imatrix: 100 × 512-token chunks of Greek text.
-## License and attribution
-Apache 2.0, inherited from the base model `XHToken/Spark-X2.5-4B`.
-Derivative work by Apollon Labs.
----
-*Apollon Labs — Towards the Sun.*

+---
+license: apache-2.0
+base_model: XHToken/Spark-X2.5-4B
+language:
+- el
+- en
+library_name: llama.cpp
+pipeline_tag: text-generation
+tags:
+- gguf
+- greek
+- quantized
+- tool-calling
+---
+<p align="center">
+  <img src="https://huggingface.co/ApollonLabs/Heliactis-1-4B-GGUF/resolve/main/insignia.svg" width="280" alt="ApollonLabsAI — Towards the Sun">
+</p>
+# Heliactis-1-4B — GGUF
+> **Heliactis** (Greek *ἡλιακτίς*, today *ηλιακτίδα*) means **"sunbeam"**: a name we formed from
+> *hḗlios* ("sun") and *aktís* ("ray"). The Heliactis models are the rays; the larger **Helios**
+> models are the sun itself. The lab is named after **Apollo** (*Apollon*), the Greek god of light.
+Quantized builds of **Heliactis 1**, an Apollon Labs fine-tune of
+[`XHToken/Spark-X2.5-4B`](https://huggingface.co/XHToken/Spark-X2.5-4B),
+for `llama.cpp`, LM Studio, Ollama and anything else that reads GGUF.
+**What it is for:** Greek that does not fall apart, code that stays intact, and
+**tool calling that knows when to stay quiet.**
+## Files
+| File | Size | Gate |
+|---|---|---|
+| `Heliactis-1-4B-F16.gguf` | 7.66 GiB | passed |
+| `Heliactis-1-4B-Q8_0.gguf` | 4.07 GiB | passed |
+| `Heliactis-1-4B-Q6_K.gguf` | 3.15 GiB | passed (imatrix) |
+| `Heliactis-1-4B-Q5_K_M.gguf` | 2.77 GiB | passed (imatrix) |
+| `Heliactis-1-4B-Q4_K_M.gguf` | 2.42 GiB | passed (imatrix) · **the default**, runs on 6 GB VRAM or plain CPU |
+Every file listed here went through the full gate below. Nothing is shipped unmeasured.
+## Usage
+```bash
+llama-server -m Heliactis-1-4B-Q4_K_M.gguf --jinja -c 8192 \
+             --cache-type-k q8_0 --cache-type-v q8_0
+```
+**Context window: 1,048,576 tokens**, inherited unchanged from the base
+(`max_position_embeddings`; 27 of 36 layers use a 512-token sliding window, which keeps
+the KV cache small). `-c 8192` above is only a default that fits small GPUs: raise it as
+far as your memory allows. The KV cache at `q8_0` costs 14,976 bytes per token, about
+14.6 GiB for the full window. Long-context recall was **not re-measured** for this release;
+see the limitations below.
+The chat template is embedded in the GGUF. The model was trained and evaluated
+with **thinking disabled**. Call it that way. `--jinja` is required for tool
+calling, otherwise the toolbox never reaches the model.
+### Identity
+This model was **not trained on any self-identity data**. If identity matters,
+set it in the system prompt:
+```
+You are Heliactis 1, a Greek/English assistant built by Apollon Labs
+on top of Spark-X2.5-4B.
+```
+## A base-model defect, reduced
+On `Spark-X2.5-4B`, 502 vocabulary entries begin with an orphan UTF-8 byte. When the model
+emits one, the output breaks and `llama-server` answers HTTP 500, even on an ordinary
+translation prompt. We [reported it upstream](https://huggingface.co/XHToken/Spark-X2.5-4B/discussions/25).
+Heliactis 1 was trained against those tokens: at the worst position we measured, their
+probability fell from **12.7%** (our unreleased intermediate fine-tune, the starting point of
+this training) to **0.17%**. Rarer, not impossible.
+**Block them at inference** to close this route (`ban-ids.json` ships in this repo):
+```bash
+llama-server -m Heliactis-1-4B-Q4_K_M.gguf --jinja \
+  $(python -c "import json;print(' '.join(f'-l {i}-inf' for i in json.load(open('ban-ids.json'))))")
+```
+Via the API, per request: `"logit_bias": [[<id>, false], ...]` for the same ids.
+A system prompt does not fix it. Full reproduction, training trade-off and a second
+route that blocking does not close: [DEFECT.md](DEFECT.md).
+## Measured results
+Instrument: 12-part gate, thinking disabled, `--jinja`, KV `q8_0`, `-c 8192`.
+Each value comes from the gate's own results files.
+| Test | Q4_K_M | Q5_K_M | Q6_K | Q8_0 | F16 | bar |
+|---|---|---|---|---|---|---|
+| Tool abstention | 15/15 | 15/15 | 15/15 | 15/15 | 15/15 | ≥ 9 |
+| Overall tool score | 45/45 | 45/45 | 45/45 | 45/45 | 45/45 | ≥ 28 |
+| Tool calls, Greek | 10/10 | 10/10 | 10/10 | 10/10 | 10/10 | ≥ 7 |
+| Tool calls, English | 20/20 | 20/20 | 20/20 | 20/20 | 20/20 | ≥ 17.5 |
+| HumanEval pass@1 | 0.7439 | 0.7622 | 0.7805 | 0.7622 | 0.7866 | ≥ 0.6952 |
+| Code fencing | 60/60 | 57/60 | 58/60 | 59/60 | 58/60 | ≥ 53 |
+| Capability eval | 52/60 | 53/60 | 55/60 | 54/60 | 54/60 | ≥ 51 |
+| Greek character validity | 0.9996 | 0.9983 | 0.9974 | 0.9974 | 0.9968 | ≥ 0.9943 |
+| Greek vocabulary | 0.9553 | 0.9536 | 0.9528 | 0.9549 | 0.9568 | ≥ 0.949 |
+| Greek meaning | 0.76 | 0.79 | 0.78 | 0.79 | 0.80 | ≥ 0.70 |
+| Median answer length (tokens) | 242 | 251.5 | 256.5 | 253.5 | 254.5 | ≤ 266 |
+| Truncated answers | 3 | 5 | 2 | 2 | 3 | ≤ 9 |
+### Honest limitations
+- **The orphan-byte defect is reduced, not removed.** See above; block the ids to close this route, and see the Lao note below.
+- **Do not use this model for Lao.** A second route to the same broken output exists: the model
+  can emit a token that ends mid-character and then not finish it. Blocking the 502 ids does not
+  stop this. In our test (greedy, one prompt per language) a Lao paragraph ended in HTTP 500
+  on this model and on our unreleased intermediate fine-tune, but not on the base model; Thai passed on all three.
+- **Lao and Thai got worse.** The 502 blocked tokens are pieces of those scripts.
+  Perplexity vs our unreleased intermediate fine-tune, fp32, 5 short outside texts (151 tokens,
+  so the noise is large): English 17.09 → 16.81 · code 18.44 → 19.30 · Greek 21.13 → 27.13 ·
+  mixed languages 56.73 → 67.90 · Lao/Thai 56.49 → 78.85. The Greek line is one sentence;
+  the gate's Greek vocabulary test is the measure we trust, and it passed.
+- **Greek meaning (0.76 on Q4_K_M)** has a wide ±0.08 band. It detects collapse, not small changes.
+- **Abstention was measured on N=15 prompts.** Evidence, not proof.
+- **Long context was not re-measured for this release.** The architecture and KV cost
+  are the base model's (14,976 bytes/token at q8_0; 9 of 36 layers full attention).
+  An unreleased earlier fine-tune on the same base recalled 3/3 two-hop facts at 231k tokens.
+  We do not carry that number over as a claim for this model.
+- **24 of the 554 code training examples use Greek identifiers.** Known, not yet fixed.
+## Training
+LoRA (r=16, alpha=32). Stage 1: SFT on 2014 examples (1212 Greek, 554 code,
+248 tool-calling; 742 of the 866 toolbox rows are decoys that teach abstention).
+Dataset sha256: `cd58f73ad7fbb339de87a3bef22af6c2ff996b971f829978c78b4f70703f26e6`.
+Stage 2: on-policy unlikelihood on the 502 tokens (856 of the model's own answers,
+weight 10), with the same SFT set alongside to hold the rest in place.
+imatrix: 100 × 512-token chunks of Greek text.
+## License and attribution
+Apache 2.0, inherited from the base model `XHToken/Spark-X2.5-4B`.
+Derivative work by Apollon Labs.
+---
+*Apollon Labs — Towards the Sun.*