File size: 2,630 Bytes
27e800b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 | # Quantara Sentinel v6 — local install
Two quants, pick by memory bandwidth rather than raw size:
| file | size | for |
|---|---|---|
| `quantara-sentinel-v6.Q4_K_M.gguf` | 4.92 GB | RTX 4060 laptop (8 GB VRAM) — fits entirely in VRAM |
| `quantara-sentinel-v6.Q8_0.gguf` | 8.54 GB | Legion Go S / Z1E (32 GB unified) and desktops |
The 4060 (~256 GB/s GDDR6) runs Q4_K_M roughly 2x faster than the Go S
(~120 GB/s LPDDR5X) *because it fits*. The moment a model exceeds 8 GB the 4060
falls off a PCIe cliff and the Go S — unified memory, no cliff — wins outright.
So: Q4_K_M on the 4060, Q8_0 on the Go S. The Go S is SteamOS, so Vulkan works
properly but root is immutable — use a static llama.cpp build in `$HOME` or
distrobox, never `steamos-readonly disable`.
## Install
ollama create quantara-sentinel -f Modelfile.Q4_K_M
ollama run quantara-sentinel
The Modelfile is hand-written and that is deliberate. This model uses Cisco
Foundation-Sec's native grammar (`<|system|>` / `<|user|>` / `<|assistant|>`,
turns closing on `<|end_of_text|>`), NOT llama-3.1's. Unsloth reports
"No Ollama template mapping found" for this base and writes no Modelfile at all.
Ship the llama-3.1 template here and every local user gets the runaway output
that cost this project two rebuilds.
Do not change the SYSTEM line. The model was trained under it and behaves
differently without it.
## What it is
Llama-Quantara-Sentinel-8B — a Quantara fine-tune of Cisco's
Foundation-Sec-8B-Instruct, built on Llama 3.1. Defensive scope: it explains and
fixes, it does not weaponize.
## Verified before release
- Safety: all 6 gates pass. 0 judged weaponization leaks across 46 red-team
probes, 100% refusal (regex and LLM judge agree), 0% over-refusal on both
benign and dual-use, 0 runaway responses.
- Terminates on its own native eos, unpatched.
## Known weaknesses — read these
- **Phishing false negatives.** 4 of 31 real phishing samples were cleared as
legitimate (cap-0185, cap-0193, cap-0200, cap-0211). Do not use verdicts as
the only gate on user-reported mail.
- **ATT&CK technique mis-mapping.** It cites a real technique ID from the wrong
tactic often enough to matter. Treat cited technique IDs as a starting point.
- **Canonical-fact grounding is uneven.** Zerologon improved sharply this
release (59.5% grounded, 2.4% fact errors) but BlueKeep regressed (66.7% /
12.8%). Verify vulnerability specifics against a primary source.
- Anything time-sensitive — CVE details, current versions — should be checked
against a live feed. Paste the advisory in rather than asking from memory.
|