Instructions to use mhndayesh/gemma-4-E2B-security-expert-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use mhndayesh/gemma-4-E2B-security-expert-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Use Docker

docker model run hf.co/mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use mhndayesh/gemma-4-E2B-security-expert-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mhndayesh/gemma-4-E2B-security-expert-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mhndayesh/gemma-4-E2B-security-expert-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Ollama
How to use mhndayesh/gemma-4-E2B-security-expert-GGUF with Ollama:
```
ollama run hf.co/mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M
```
Unsloth Desktop

How to use mhndayesh/gemma-4-E2B-security-expert-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use mhndayesh/gemma-4-E2B-security-expert-GGUF with Docker Model Runner:
```
docker model run hf.co/mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M
```

Lemonade

How to use mhndayesh/gemma-4-E2B-security-expert-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.gemma-4-E2B-security-expert-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use mhndayesh/gemma-4-E2B-security-expert-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use mhndayesh/gemma-4-E2B-security-expert-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "mhndayesh/gemma-4-E2B-security-expert-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

mhndayesh commited on Jul 18

Commit

e475aff

verified ·

1 Parent(s): 0326284

Upload README.md with huggingface_hub

Browse files

Files changed (1) hide show

README.md +66 -100

README.md CHANGED Viewed

@@ -14,125 +14,91 @@ tags:
   - secure-coding
   - appsec
   - cwe
-  - retrieval
-  - edge
 ---
 # gemma-4-E2B-security-expert (GGUF)
-**A ~2B edge model (`gemma-4-E2B-it`) that writes *secure code by default* — via an
-application-security *FactBank* baked into its chat-template.** When you ask for code, a searchable
-bank of security "landmine facts" fires **inside the inference engine** and steers the model off the
-insecure-by-default pattern it would otherwise emit. **Weights are untouched**; there is no external
-RAG service, no fine-tuning. Runs on a laptop.
-> A stock 2B model writes insecure code about half the time — so do much larger models. The bank
-> supplies the *security knowledge* the weights lack, so the small model behaves like one taught to
-> code securely. On the results below, this 2B **beats DeepSeek-V4** at secure-code generation.
-> 🔗 **Full project — all experts, methodology, transcripts, benchmarks:**
-> **[github.com/mhndayesh/experts-models](https://github.com/mhndayesh/experts-models)**
-## What it does
-Given a normal coding request, it silently corrects the common insecure defaults, e.g.:
-| you ask for… | base 2B tends to write | **this model writes** |
-|---|---|---|
-| load a PyTorch checkpoint | `torch.load(path)` | `torch.load(path, weights_only=True)` *(post-cutoff default)* |
-| parse an XML file | `etree.parse(f)` | `XMLParser(resolve_entities=False, no_network=True)` *(XXE)* |
-| load a YAML config | `yaml.load(...)` | `yaml.safe_load(...)` |
-| generate a token / nonce | `random.random()` | `secrets.token_hex()` / `os.urandom` |
-| a DB credential | `password = "hunter2"` | `os.environ.get("DB_PASSWORD")` |
-| run a SQL query with user input | string-concatenated SQL | parameterized query |
-| evaluate a user expression | `eval(...)` | `ast.literal_eval(...)` |
-| compare an HMAC signature | `a == b` | constant-time compare |
-You **don't have to name the vulnerability** — the bank is indexed by the *coding task*
-("verify a signed URL", "load a model", "parse untrusted XML"), so it fires on benign prompts.
-## Coverage — what's in the bank
-- **258 security concepts** (backed by the **MITRE CWE** taxonomy) → **3,984 language/framework
-  variants** — the same weakness written the way it actually appears in each ecosystem. (This bank is
-  organized by **weakness/CWE**, not by a short library list — it spans the whole secure-coding surface.)
-- **10+ languages:** Python, Java, JavaScript, C / C++, Swift, Go, C#, Rust, Ruby, and more.
-- **Frameworks & platforms:** Android, iOS / SwiftUI / WKWebView, Flask, Django, Express, Node.js,
-  ASP.NET / .NET, Spring, Java EE, Rails, Laravel.
-- **Mined from 7 permissive security sources:** MITRE **CWE**, GitHub **CodeQL** queries, SAST rules
-  (**Bandit** + **gosec**), OWASP **MASTG** (mobile), **RustSec** advisories, **NIST / RFC / Mozilla**
-  crypto-&-network guidance, and **OWASP**. Every fact was quote-grounded to its source, then put
-  through an **adversarial correctness audit** (a second model tried to refute each fact; ~3.8%
-  wrong/stale were removed — grounding proves a fact is *real*, the audit checks it's *right*).
-## Benchmarks — hand-scored (the project rule is to read outputs, not trust a scorer)
-### External benchmark: SecurityEval (s2e-lab, MSR 2022)
-121 Python CWE-tagged insecure-by-default function-completion tasks; complete each stub securely.
-| arm | secure (pattern-checkable subset, 28 tasks) |
-|---|---|
-| e2b **base** (2B) | 14 / 28 |
-| **e2b + this bank** | **25 / 28  (+11, 0 regressions)** |
-Standout: **XXE (CWE-611) 0/6 → 6/6**, plus wins on deserialization, hardcoded creds, code-injection,
-and weak-random. (The rest of the 121 are rarer CWEs / cases no pattern resolves; no covered CWE
-regressed.)
-### Vs. a strong cloud model — same 24 tasks, identical scoring
-The cloud baseline is **DeepSeek-V4 (`deepseek-v4-flash`)**, run on the exact same prompts.
-| model | secure |
 |---|---|
-| e2b base (2B) | 12 / 24 |
-| **DeepSeek-V4 (`deepseek-v4-flash`, cloud)** | 13 / 24 |
-| **e2b + this bank (2B)** | **22 / 24** |
-**The 2B + bank *strictly dominates* the cloud model** on this subset: secure on **9 tasks where
-DeepSeek is not**, and secure on **every task DeepSeek got right** (0 losses). Scale doesn't supply
-this knowledge; the bank does.
-> **Honest caveats.** Hand-scored on the pattern-checkable subset (the rest weren't machine-judgeable).
-> `deepseek-v4-flash` is a strong, large cloud model but not literally GPT-5 / Claude-Opus (those
-> weren't callable in the harness). Numbers are a secure-code delta, not a security guarantee.
 ## How to run
-### llama.cpp / `llama-server` — recommended, full bank (the bank is embedded in this GGUF)
 ```bash
-llama-server -m gemma-4-E2B-security-expert-Q4_K_M.gguf --jinja --port 8080 --ctx-size 8192
 ```
-Then query normally — the bank fires automatically for security-relevant requests. Exposes an
-**OpenAI-compatible** API at `http://localhost:8080/v1`, so any OpenAI SDK / server client works
-unchanged. **Sampling — Gemma-native:** `temperature 1.0, top_k 64, top_p 0.95, min_p 0.01` (the
-`min_p` floor prevents reasoning-loop empty answers). Don't quantize the KV cache on a model this
-small. Ships **thinking-OFF** — the correct config for a baked expert (chain-of-thought reverts a
-habit-reversal fact to the model's trained prior).
-### LM Studio — use the Hub model (delivers the full bank)
-The bank template is **4.18 MB**, over LM Studio's ~980 KB raw-GGUF metadata cap — so if you load
-*this GGUF* directly in LM Studio the template is silently truncated and the bank won't fire. For LM
-Studio, install the **virtual-model** edition, which hands the full template to LM Studio's own engine
-(no cap):
-**➡ https://lmstudio.ai/mhndayesh/gemma-4-e2b-security-expert** — open it and click *"Use Model in
-LM Studio"* (or `lms get mhndayesh/gemma-4-e2b-security-expert`).
-Both editions carry the **same** bank; only the delivery differs. This GGUF is for llama.cpp and
-OpenAI-compatible servers; the Hub edition is for LM Studio.
 ## Limitations
-- **Scoped to application-security / secure coding.** Outside that, it's the base model.
-- **Supplies knowledge, not reasoning.** A 2B still misses some multi-step fixes (e.g. stripping
-  newlines for log-injection).
-- **Keyword + faceted retrieval, not embeddings.** A wholly unexpected phrasing can miss; there is no
-  "I don't know" signal — a miss looks like a normal answer.
-- **Grounded & audited, but research-grade** — review generated code; this is an assistant, not a
-  substitute for review.
-- Hand-scored security tests, not a general coding benchmark.
 ## Provenance & license
-- **Base:** `lmstudio-community/gemma-4-E2B-it-GGUF` (Q4_K_M, weights untouched). This model = that
-  GGUF with `tokenizer.chat_template` rewritten to embed an inverted-index retriever + the bank
-  (`factbank.version 0.4.0`). The standalone template ships alongside as `chat-template.jinja`.
-- **License:** Google **Gemma Terms of Use** (`license: gemma`) — a gemma-4 derivative. The fact bank
-  is FactBank-project content (repo `LICENSE`); mined sources keep their own licenses.
 - **Source & method:** [github.com/mhndayesh/experts-models](https://github.com/mhndayesh/experts-models).

   - secure-coding
   - appsec
   - cwe
+  - reasoning
 ---
 # gemma-4-E2B-security-expert (GGUF)
+**A ~2B (E2B) model that writes *secure code by default* — an application-security *FactBank* is baked into
+its chat-template and fires **inside the inference engine** to steer the model off insecure-by-default
+patterns.** Weights untouched; no external RAG. This repo ships **two editions** (see below).
+> 🔗 **Full project:** [github.com/mhndayesh/experts-models](https://github.com/mhndayesh/experts-models)
+## Two editions — pick one (files in this repo)
+| edition | file | when to use | notes |
+|---|---|---|---|
+| **thinking-OFF** (default) | `gemma-4-E2B-security-expert-Q4_K_M.gguf` | production, speed, reliability | 0 empty answers, fast, secure. No reasoning trace. **The safe default.** |
+| **thinking-ON** | `gemma-4-E2B-security-expert-thinking-Q4_K_M.gguf` | you want a visible reasoning trace | Shows its work and is strong on most tasks, **but ~10% of the hardest-reasoning prompts can run long and return a *blank* answer** (fails safe — it does not emit insecure code; just retry). Needs a large context + generous output budget. |
+Both editions carry the **same** bank. They differ only in whether chain-of-thought is on. We ship both so
+you know exactly what you're getting — a landmine fact *reverses a trained habit*, and with thinking on the
+model sometimes over-reasons back toward its prior, so the thinking edition trades some reliability for a
+reasoning trace. Thinking-ON has a **known llama.cpp Gemma-4 fix applied** (the generation prompt opens the
+thought channel) + **strong authority framing** so facts hold; the residual ~10% is inherent to reasoning-on.
+## Settings that work (baked into the Hub editions; set these yourself on llama.cpp)
+- **Sampling (gemma-native):** `temperature 1.0, top_k 64, top_p 0.95, min_p 0.01`. The **`min_p 0.01`** floor
+  is important — it stops the low-temperature repetition loop that otherwise yields empty answers.
+- **Context length:** **≥ 16k, 32k recommended** (`contextLength 32768` is baked). The bank injects ~3,400
+  tokens per query, so a small context truncates the answer — especially with thinking on.
+- **Output budget (thinking-ON):** allow **generous `max_tokens`** (reasoning traces run ~2–3k tokens); too
+  small a cap truncates before the answer.
+## What it does
+Given a normal coding request it silently corrects insecure defaults, e.g. `torch.load(weights_only=True)`
+(post-cutoff), XXE `resolve_entities=False`, `yaml.safe_load`, `secrets` over `random`, `os.environ` creds,
+parameterized SQL, `ast.literal_eval` over `eval`, constant-time HMAC compare. You **don't name the
+vulnerability** — the bank is indexed by the coding task, so it fires on benign prompts.
+## Coverage — what's in the bank
+- **258 CWE concepts → 3,984 language/framework variants** (organized by weakness, not a short lib list).
+- **10+ languages:** Python, Java, JavaScript, C/C++, Swift, Go, C#, Rust, Ruby, and more.
+- **Frameworks:** Android, iOS/SwiftUI/WKWebView, Flask, Django, Express, Node.js, ASP.NET/.NET, Spring,
+  Java EE, Rails, Laravel.
+- **7 mined sources:** MITRE **CWE**, GitHub **CodeQL**, SAST (**Bandit**+**gosec**), OWASP **MASTG**,
+  **RustSec**, **NIST/RFC/Mozilla** crypto-net, **OWASP** — each fact quote-grounded then adversarially
+  correctness-audited (~3.8% wrong/stale removed).
+## Benchmarks (E2B, hand-scored — read the outputs, don't trust a scorer)
+External **SecurityEval** (s2e-lab, MSR 2022, 121 Python CWE tasks), pattern-checkable subset:
+| arm | secure |
 |---|---|
+| base (2B) | 14/28 |
+| **+ this bank** | **25/28  (+11, 0 regressions; XXE 0/6→6/6)** |
+And vs a strong cloud model on the same 24 tasks: base 12 · **DeepSeek-V4 (`deepseek-v4-flash`) 13** ·
+**+bank 22** — the 2B+bank *strictly dominates* the cloud model (9 wins, 0 losses). These are FactBank
+retrieval measurements; the shipped GGUFs bake the same bank in-engine.
 ## How to run
+### LM Studio (Hub editions — the full bank is delivered via model.yaml)
+The bank template is 4.2 MB — over LM Studio's ~980 KB raw-GGUF cap, so for LM Studio use the **Hub virtual
+models** (they hand the full template to the engine, no cap), which also carry the settings above:
+- **thinking-OFF:** https://lmstudio.ai/mhndayesh/gemma-4-e2b-security-expert
+- **thinking-ON:** https://lmstudio.ai/mhndayesh/gemma-4-e2b-security-expert-thinking
+### llama.cpp / any OpenAI-compatible server (this GGUF, full bank embedded)
 ```bash
+# thinking-OFF (recommended)
+llama-server -m gemma-4-E2B-security-expert-Q4_K_M.gguf --jinja --ctx-size 32768 --port 8080
+# thinking-ON
+llama-server -m gemma-4-E2B-security-expert-thinking-Q4_K_M.gguf --jinja --ctx-size 32768 --port 8080
 ```
+Sampling: `temperature 1.0, top_k 64, top_p 0.95, min_p 0.01`. Don't quantize the KV cache on small models.
+Exposes an OpenAI-compatible API at `/v1`.
 ## Limitations
+- **Scoped to secure coding.** Outside it, base model.
+- **Supplies knowledge, not reasoning** — still misses some multi-step fixes.
+- **thinking-ON:** ~10% of hard prompts run long / blank (fails safe). Use thinking-OFF for reliability.
+- **Keyword + faceted retrieval, not embeddings** — an unusual phrasing can miss. Research-grade; review output.
 ## Provenance & license
+- **Base:** `lmstudio-community/gemma-4-E2B-it-GGUF` (weights untouched). Only `tokenizer.chat_template` is rewritten to embed an
+  inverted-index retriever + the bank (`factbank.version 0.4.0`).
+- **License:** Google **Gemma Terms of Use** (`license: gemma`). Fact bank = FactBank-project content; mined
+  sources keep their own licenses.
 - **Source & method:** [github.com/mhndayesh/experts-models](https://github.com/mhndayesh/experts-models).