Instructions to use Maelstrome/lora-wave-session with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Maelstrome/lora-wave-session with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("unsloth/gemma-4-e2b-it-unsloth-bnb-4bit")
model = PeftModel.from_pretrained(base_model, "Maelstrome/lora-wave-session")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Maelstrome/lora-wave-session with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Maelstrome/lora-wave-session:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Maelstrome/lora-wave-session:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Maelstrome/lora-wave-session:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Maelstrome/lora-wave-session:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Maelstrome/lora-wave-session:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Maelstrome/lora-wave-session:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Maelstrome/lora-wave-session:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Maelstrome/lora-wave-session:Q4_K_M

Use Docker

docker model run hf.co/Maelstrome/lora-wave-session:Q4_K_M

LM Studio
Jan

vLLM

How to use Maelstrome/lora-wave-session with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Maelstrome/lora-wave-session"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Maelstrome/lora-wave-session",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Maelstrome/lora-wave-session:Q4_K_M

Ollama
How to use Maelstrome/lora-wave-session with Ollama:
```
ollama run hf.co/Maelstrome/lora-wave-session:Q4_K_M
```
Unsloth Desktop

How to use Maelstrome/lora-wave-session with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Maelstrome/lora-wave-session:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Maelstrome/lora-wave-session:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Maelstrome/lora-wave-session with Docker Model Runner:
```
docker model run hf.co/Maelstrome/lora-wave-session:Q4_K_M
```

Lemonade

How to use Maelstrome/lora-wave-session with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Maelstrome/lora-wave-session:Q4_K_M

Run and chat with the model

lemonade run user.lora-wave-session-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use Maelstrome/lora-wave-session with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Maelstrome/lora-wave-session:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Maelstrome/lora-wave-session:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Maelstrome/lora-wave-session with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Maelstrome/lora-wave-session:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Maelstrome/lora-wave-session:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Maelstrome commited on May 14

Commit

ae87a6f

verified ·

1 Parent(s): dd32ea4

Upload README.md with huggingface_hub

Browse files

Files changed (1) hide show

README.md +31 -2

README.md CHANGED Viewed

@@ -31,6 +31,20 @@ A unified LoRA adapter on top of **Gemma 4 E2B Instruct** that handles three str
 All three surfaces emit strict JSON, no markdown, no analysis voice, in patient-facing tone.
 ## Sibling runs
 This is the **rank-16 / 3-epoch RTX 5080** training of the WAVE corpus. The rank-32 / 1-epoch A100 sibling lives at [`Maelstrome/lora-wave-session-r32`](https://huggingface.co/Maelstrome/lora-wave-session-r32) and wins on every probability metric on the same frozen 428-row test split:
@@ -53,7 +67,9 @@ Trained for the WAVE app, a wellness/reflection tool — not a medical device, n
 🌊 **Interactive demo:** [`Maelstrome/lora-wave-session-demo`](https://huggingface.co/spaces/Maelstrome/lora-wave-session-demo) — Gradio Space with surface-specific example prompts.
-## Quickstart (PEFT + Unsloth)
 ```python
 from unsloth import FastModel
@@ -76,7 +92,20 @@ tok = AutoTokenizer.from_pretrained("unsloth/gemma-4-E2B-it")
 model = PeftModel.from_pretrained(base, "Maelstrome/lora-wave-session")
 ```
-For a one-file 4-bit GGUF deployable with llama.cpp / Ollama / wllama, see [`Maelstrome/lora-wave-session-gguf`](https://huggingface.co/Maelstrome/lora-wave-session-gguf).
 ## Example prompts

 All three surfaces emit strict JSON, no markdown, no analysis voice, in patient-facing tone.
+## Repository layout
+This repo is the single home for the r16 fine-tune. Everything lives here:
+| Path | What | When to use |
+|---|---|---|
+| `adapter_model.safetensors` + `adapter_config.json` (root) | LoRA adapter (~100 MB) | `peft.PeftModel.from_pretrained` / Unsloth `FastModel` — pairs with the upstream `unsloth/gemma-4-E2B-it` base |
+| `tokenizer.json`, `tokenizer_config.json`, `chat_template.jinja`, `processor_config.json` (root) | Gemma 4 tokenizer + chat template | required for any inference path |
+| [`gguf/`](./tree/main/gguf) | Q4_K_M GGUF (~3.27 GB, single file) + Ollama Modelfile | llama.cpp / Ollama / LM Studio |
+> The `Maelstrome/lora-wave-session-gguf` sibling is **deprecated** — its contents were moved into this repo's `gguf/` subdirectory. The rank-32 variant has the same layout at [`Maelstrome/lora-wave-session-r32`](https://huggingface.co/Maelstrome/lora-wave-session-r32).
+>
+> Note: the GGUF here is a **single 3.27 GB file**, not pre-split. It works directly with llama.cpp/Ollama but **will not load in [wllama](https://github.com/ngxson/wllama)** because it exceeds the 2 GB ArrayBuffer per-file limit. If you want to run this rank-16 build in the browser, split it first with `llama-gguf-split --split-max-size 512M`. The rank-32 sibling is pre-split in its `gguf/` subdir if you'd rather just use that.
 ## Sibling runs
 This is the **rank-16 / 3-epoch RTX 5080** training of the WAVE corpus. The rank-32 / 1-epoch A100 sibling lives at [`Maelstrome/lora-wave-session-r32`](https://huggingface.co/Maelstrome/lora-wave-session-r32) and wins on every probability metric on the same frozen 428-row test split:
 🌊 **Interactive demo:** [`Maelstrome/lora-wave-session-demo`](https://huggingface.co/spaces/Maelstrome/lora-wave-session-demo) — Gradio Space with surface-specific example prompts.
+## Quickstart
+### PEFT + Unsloth (CUDA, server-side)
 ```python
 from unsloth import FastModel
 model = PeftModel.from_pretrained(base, "Maelstrome/lora-wave-session")
 ```
+### Ollama (via the GGUF in `gguf/`)
+```bash
+ollama create wave-r16 -f - <<EOF
+FROM hf://Maelstrome/lora-wave-session/gguf/gemma-4-e2b-it.Q4_K_M.gguf
+EOF
+ollama run wave-r16
+```
+### llama.cpp directly
+```bash
+llama-cli -hf Maelstrome/lora-wave-session:gguf/gemma-4-e2b-it.Q4_K_M.gguf --jinja
+```
 ## Example prompts