Instructions to use mhndayesh/gemma-4-E2B-security-expert-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use mhndayesh/gemma-4-E2B-security-expert-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Use Docker

docker model run hf.co/mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use mhndayesh/gemma-4-E2B-security-expert-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mhndayesh/gemma-4-E2B-security-expert-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mhndayesh/gemma-4-E2B-security-expert-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Ollama
How to use mhndayesh/gemma-4-E2B-security-expert-GGUF with Ollama:
```
ollama run hf.co/mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M
```
Unsloth Desktop

How to use mhndayesh/gemma-4-E2B-security-expert-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use mhndayesh/gemma-4-E2B-security-expert-GGUF with Docker Model Runner:
```
docker model run hf.co/mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M
```

Lemonade

How to use mhndayesh/gemma-4-E2B-security-expert-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.gemma-4-E2B-security-expert-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use mhndayesh/gemma-4-E2B-security-expert-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use mhndayesh/gemma-4-E2B-security-expert-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

gemma-4-E2B-security-expert (GGUF)

A ~2B (E2B) model that writes secure code by default — an application-security FactBank is baked into its chat-template and fires inside the inference engine to steer the model off insecure-by-default patterns. Weights untouched; no external RAG. This repo ships two editions (see below).

🔗 Full project: github.com/mhndayesh/experts-models

Two editions — pick one (files in this repo)

edition	file	when to use	notes
thinking-OFF (default)	`gemma-4-E2B-security-expert-Q4_K_M.gguf`	production, speed, reliability	0 empty answers, fast, secure. No reasoning trace. The safe default.
thinking-ON	`gemma-4-E2B-security-expert-thinking-Q4_K_M.gguf`	you want a visible reasoning trace	Shows its work and is strong on most tasks, *but ~10% of the hardest-reasoning prompts can run long and return a blank* answer** (fails safe — it does not emit insecure code; just retry). Needs a large context + generous output budget.

Both editions carry the same bank. They differ only in whether chain-of-thought is on. We ship both so you know exactly what you're getting — a landmine fact reverses a trained habit, and with thinking on the model sometimes over-reasons back toward its prior, so the thinking edition trades some reliability for a reasoning trace. Thinking-ON has a known llama.cpp Gemma-4 fix applied (the generation prompt opens the thought channel) + strong authority framing so facts hold; the residual ~10% is inherent to reasoning-on.

Settings that work (baked into the Hub editions; set these yourself on llama.cpp)

Sampling (gemma-native): temperature 1.0, top_k 64, top_p 0.95, min_p 0.01. The min_p 0.01 floor is important — it stops the low-temperature repetition loop that otherwise yields empty answers.
Context length: ≥ 16k, 32k recommended (contextLength 32768 is baked). The bank injects ~3,400 tokens per query, so a small context truncates the answer — especially with thinking on.
Output budget (thinking-ON): allow generous max_tokens (reasoning traces run ~2–3k tokens); too small a cap truncates before the answer.

What it does

Given a normal coding request it silently corrects insecure defaults, e.g. torch.load(weights_only=True) (post-cutoff), XXE resolve_entities=False, yaml.safe_load, secrets over random, os.environ creds, parameterized SQL, ast.literal_eval over eval, constant-time HMAC compare. You don't name the vulnerability — the bank is indexed by the coding task, so it fires on benign prompts.

Coverage — what's in the bank

258 concepts (254 CWE + 4 door groups) → 3,984 language/framework variants (organized by weakness, not a short lib list).
10+ languages: Python, Java, JavaScript, C/C++, Swift, Go, C#, Rust, Ruby, and more.
Frameworks: Android, iOS/SwiftUI/WKWebView, Flask, Django, Express, Node.js, ASP.NET/.NET, Spring, Java EE, Rails, Laravel.
7 mined sources: MITRE CWE, GitHub CodeQL, SAST (Bandit+gosec), OWASP MASTG, RustSec, NIST/RFC/Mozilla crypto-net, OWASP — each fact quote-grounded then adversarially correctness-audited (~3.8% wrong/stale removed).

How the three sizes compare — and vs a cloud model (hand-scored)

External SecurityEval (s2e-lab, MSR 2022, 121 Python CWE tasks). Every arm below is the shipped baked GGUF, thinking-OFF, served on my own llama-server with identical scoring; DeepSeek-V4 (deepseek-v4-flash) run on the same prompts in no-thinking mode (thinking disabled) — to match the thinking-off editions here (its reasoning mode could score higher; this is an apples-to-apples, thinking-off comparison). Table is the common pattern-checkable subset (21 tasks) where every arm is machine-judgeable.

model (baked, thinking-off)	secure / 21
e2b base	11
e2b + bank	13
12b base	17
12b + bank	17
26b base	17
26b + bank	19 — best
DeepSeek-V4 (cloud)	14

Reading it: bigger model → more secure. The bank lifts each model and helps most where the base is weak

on specific weaknesses — XXE (CWE-611): 12b 5→6/6, 26b 4→5/5, plus deserialization, weak-random, hardcoded-creds. 12b+bank and 26b+bank beat the cloud model; e2b+bank is about even with it. On each model's own full judgeable set the bank adds +2–3 (e2b 15→18, 12b 29→31, 26b 30→33).

Honest note: these are the in-template (prompt-only) baked editions. The FactBank retrieval method reaches higher when run as a served loop with a HyDE draft-key + authority framing (e2b 14→25 on the 28-task subset) — that draft-key is the big lever for benign security prompts and is not in the baked template. The numbers above are what the shipped GGUFs actually do.

How to run

LM Studio (Hub editions — the full bank is delivered via model.yaml)

The bank template is 4.2 MB — over LM Studio's ~980 KB raw-GGUF cap, so for LM Studio use the Hub virtual models (they hand the full template to the engine, no cap), which also carry the settings above:

thinking-OFF: https://lmstudio.ai/mhndayesh/gemma-4-e2b-security-expert
thinking-ON: https://lmstudio.ai/mhndayesh/gemma-4-e2b-security-expert-thinking

llama.cpp / any OpenAI-compatible server (this GGUF, full bank embedded)

# thinking-OFF (recommended)
llama-server -m gemma-4-E2B-security-expert-Q4_K_M.gguf --jinja --ctx-size 32768 --port 8080
# thinking-ON
llama-server -m gemma-4-E2B-security-expert-thinking-Q4_K_M.gguf --jinja --ctx-size 32768 --port 8080

Sampling: temperature 1.0, top_k 64, top_p 0.95, min_p 0.01. Don't quantize the KV cache on small models. Exposes an OpenAI-compatible API at /v1.

Limitations

Scoped to secure coding. Outside it, base model.
Supplies knowledge, not reasoning — still misses some multi-step fixes.
thinking-ON: ~10% of hard prompts run long / blank (fails safe). Use thinking-OFF for reliability.
Keyword + faceted retrieval, not embeddings — an unusual phrasing can miss. Research-grade; review output.

Provenance & license

Base: lmstudio-community/gemma-4-E2B-it-GGUF (weights untouched). Only tokenizer.chat_template is rewritten to embed an inverted-index retriever + the bank (factbank.version 0.4.0).
License: Google Gemma Terms of Use (license: gemma). Fact bank = FactBank-project content; mined sources keep their own licenses.
Source & method: github.com/mhndayesh/experts-models.