--- license: apache-2.0 base_model: fdtn-ai/antares-1b base_model_relation: quantized pipeline_tag: text-generation language: - en tags: - gguf - llama.cpp - security - vulnerability-detection - agentic - terminal-agent - granite - imatrix - benchmarked --- # Antares-1B GGUF — the measured ladder Nine quantizations of [fdtn-ai/antares-1b](https://huggingface.co/fdtn-ai/antares-1b) (Cisco Foundation AI's security SLM — a CWE localizer that explores a repository through a terminal and submits a ranked list of suspect files; a fine-tune of [ibm-granite/granite-4.0-1b](https://huggingface.co/ibm-granite/granite-4.0-1b)). **Every file here was benchmarked before publishing** on the model's own vulnerability-localization benchmark — the full 499-task set, detection (Phase A) *and* clean-tree false-alarm rate (Phase B), with three-run error bars on six of the nine rungs — and every file's sha256 is published. As of 2026-08-02, none of the dozen-plus community GGUF repos of this model we surveyed publish per-file hashes or measurements, and we found no imatrix builds among them. ![Quant ladder: File F1 vs file size](ladder.png) ## Which file do I want? | You want | Take | Why | |----------|------|-----| | Best quality per GB | **`antares-1b-Q5_K_M.gguf`** | 0.1723 ± 0.0014 vs the BF16 anchor: Δ −0.002 ± 0.004, at 36% of the size. This recommendation is robust to every noise story on this card | | Smallest safe file | **`antares-1b-Q4_K_M-imat.gguf`** | Recovers half of static Q4's quality gap at *identical file size*, and is the best retriever measured (three-run bars both ways) — also robust | | Tightest VRAM, eyes open | `antares-1b-IQ4_XS.gguf` | Good mean, but the widest run-to-run spread measured (span 0.0154, n=3) — *not* a robust recommendation; see the table note | | Reference / re-quantizing | `antares-1b-BF16.gguf` | The conversion baseline every claim is measured against | | ❌ Not these | `Q3_K_M`, `Q2_K` | Included to document the cliff: Q3 loses 4× the F1 with 51% abstention; Q2 cannot emit one valid tool call. They are data, not deployables | **Retriever view** — if you wire Antares as a cheap filter ahead of a stronger judge, the deciding metric is hit@all, not F1: `Q4_K_M-imat` is the best measured (0.350 ± 0.017, n=3, vs static Q4's 0.331); IQ4_XS is the worst healthy rung (0.287 ± 0.016) because it submits half as many files. ## The measured ladder File F1 on the 499-task benchmark (Phase A), abstain rate, and Phase B true-negative rate on clean trees. Rungs 0–6 measured 2026-07-30/31; rungs 7–8 measured 2026-08-01; BF16 and Q8_0 re-measured ×3 on 2026-08-02. | # | File | imatrix | Size | % of BF16 | File F1 | abstain | TNR | |--:|------|:-:|-----:|----------:|--------:|--------:|----:| | 0 | BF16 | — | 3.68 GB | 100% | 0.1740 ± 0.0074 | 0.9% | 0.006 | | 1 | Q8_0 | — | 1.96 GB | 53% | 0.1741 ± 0.0035 | 1.6% | 0.014 | | 2 | Q6_K | — | 1.51 GB | 41% | 0.1622 | 1.4% | 0.030 | | 3 | Q5_K_M | — | 1.32 GB | 36% | **0.1723 ± 0.0014** | 1.2% | 0.020 | | 4 | Q4_K_M | — | 1.14 GB | 31% | 0.1610 ± 0.0010 | 3.7% | 0.022 | | 7 | **Q4_K_M-imat** | ✓ | 1.14 GB | 31% | **0.1667 ± 0.0011** | 2.7% | 0.036 | | 8 | IQ4_XS | ✓ | 1.05 GB | 28% | 0.1648 ± 0.0083 | 2.7% | 0.034 | | 5 | Q3_K_M | — | 0.95 GB | 26% | 0.0402 | 51.3% | 0.539* | | 6 | Q2_K | — | 0.77 GB | 21% | 0.0000 | 100% | 1.000* | **Table notes:** - ± values are 1 sd over three full benchmark runs; every triple is published here: BF16 0.1735/0.1669/0.1817 · Q8_0 0.1721/0.1720/0.1781 · Q5_K_M 0.1720/0.1738/0.1711 · Q4_K_M 0.1602/0.1608/0.1621 · Q4_K_M-imat 0.1679/0.1659/0.1663 · IQ4_XS 0.1742/0.1588/0.1613. (Each sd carries 2 degrees of freedom — read spans, not third decimals.) - **Run-to-run noise is rung-dependent.** K-quant triples span ≤ 0.003; BF16's own three runs span 0.0148 and IQ4_XS's 0.0154. Cells without ± (Q6_K, Q3, Q2, and all single-run TNR cells) carry doubt at whatever their rung's scale turns out to be — **rungs are not rank-ordered within noise**. Q6_K's dip below both neighbors is a single-run cell inside that noise, not a measured effect. - **A correction we're proud of:** this table's first draft showed single-run Q8_0 (0.1798) *above* single-run BF16 (0.1670). Re-running both ×3 dissolved the gap entirely (both means 0.174, Δ = 0.000 ± 0.005) — a high draw had met a low draw. We re-measured our most interesting number before believing it; treat anyone's single-run cells (ours included) with the same skepticism. - TNR cells are single runs of a binary outcome over 499 clean trees: binomial se is ±0.006–0.007 at these rates (Q5's Phase-B triple confirms it: 0.020/0.006/0.014). TNR differences between healthy rungs are within noise. - \*Q3/Q2's "good" TNR is a Goodhart artifact — a model that abstains on everything never false-alarms. Pair TNR with the detection column, always. - **IQ4_XS**: the widest spread on the board at n=3 — suggestive, not proven, but treat any single-run IQ4_XS benchmark (ours or anyone's) as uninformative. - **None of these rungs is a stand-alone merge gate.** Healthy rungs flag something on 96–99% of *clean* trees (Phase B). Treat output as leads for a human or a stronger model — which matches the upstream guidance ("a lead to verify, not proof"). - Token-level quant-damage metrics (KLD / top-token agreement vs BF16) were not measured — known future work; these numbers are task-level. ![Run-to-run spread by rung](stability.png) ## We measure ≈0.174 — and the serving stack is not the reason we miss 0.209 Cisco's model card reports 0.209 (three-run mean, their internal pipeline). Across **seven full-corpus BF16 runs** — two serving engines (llama.cpp and vLLM agree within 0.004 on the same weights), three sampler configurations, and a fresh ×3 repeat on 2026-08-02 — we measure **0.1735 ± 0.0058**, and no run of the seven exceeded 0.1817: the published figure is not reachable by run-to-run luck. (The ladder table's BF16 cell is the ×3 subset run under the exact ladder protocol; this pooled family is the anchor.) We did not reproduce the published figure, and the serving stack is *exonerated* as the cause. The residual is consistent with either a reference-pipeline difference (the model card's usage snippet renders prompts differently than the published runner) or a checkpoint-labeling difference (the leaderboard's SFT row is 0.188) — neither verifiable from published artifacts. **All rung claims in this card are relative to this repo's own BF16 anchor**, which makes them internally consistent regardless of the absolute-scale question. ## Serving — the traps that silently ruin results Measurement configuration (what these numbers were produced with): - llama.cpp `llama-server`, 4 slots × 32k ctx, **raw `/v1/completions`** — the agent harness formats its own prompts using the model's granite chat markers; the GGUF's embedded chat template is **not** applied during measurement. - Server-side sampler pins: `--top-k 0 --top-p 1.0 --min-p 0.0`; the harness supplies `temperature 0.3, top_p 1.0` per the published protocol. Pin the server: llama.cpp otherwise defaults `top_k=40, top_p=0.95, min_p=0.05` for fields a client omits — a different model than the one measured. Traps: 1. **Ollama's `/v1/completions` is not raw.** It wraps prompts in the GGUF's chat template — no error, degraded results. If you must use Ollama, set an identity template (`TEMPLATE {{ .Prompt }}`) and verify with a token-count sentinel (tokenize your prompt separately; compare `prompt_tokens`). 2. **Double-BOS.** The tokenizer adds BOS; if your client also prepends one, quality drops silently. The sentinel check above catches this too. ## imatrix provenance `antares-1b.imatrix` (567 chunks, final PPL 12.52) was computed from `calibration.txt` (included, 968 KB): 5 generic CWE-class prompts in agent framing, 40 agent transcripts from an out-of-corpus repository run, and 68 multi-language source files (C/C++, Python). Decontamination method, stated plainly: the corpus was **constructed from sources that exclude the benchmark corpus** and spot-checked by string search (0/25 sampled eval-task CWE descriptions found); no automated n-gram decontamination pass was run. Both imatrix rungs were quantized from the hash-verified BF16 below. One honest caveat: this corpus is a single point in a large space of reasonable calibration choices, no second corpus was tried, and the imatrix rungs' scores are conditional on it. ## Provenance & reproducibility - **Upstream:** `fdtn-ai/antares-1b` @ revision `10417eb35641b32e7141157db19c76eb545193b6`. - **Conversion:** llama.cpp `f5b9bd3`, `convert_hf_to_gguf.py` → BF16. The `granitemoehybrid` class converts to plain `granite`: the checkpoint is dense (363 tensors, zero experts, zero state-space tensors — verified by counting, not by trusting the label). - **Quantization:** `llama-quantize` (same commit); imatrix rungs via `llama-imatrix -m BF16 -f calibration.txt -ngl 99` then `llama-quantize --imatrix antares-1b.imatrix {Q4_K_M|IQ4_XS}`. - **Changes vs upstream:** format conversion and quantization only — no fine-tuning, no re-training, no merges. - **IQ4_XS requires a llama.cpp new enough for IQ quants** (any 2024+ build; the measured server was `f5b9bd3`). - **K-quants are not byte-reproducible across CPU architectures** — the same command on ARM and x86 produces files differing in ~0.016% of bytes (weights sitting on a quantization-level boundary land on different sides depending on accumulation order). "Q4_K_M" is a recipe, not a file. These are the measured bytes: ``` e8b3c75677beb1044e64cdefe05cf151ab22c6e0a4e119d338876ee2b580bbac antares-1b-BF16.gguf e9f27ab7c232536f74721f79917beefdd0fbfcdc0baf313b86ea82701c40d51c antares-1b-Q8_0.gguf 057b5523f25f428d2f78e59c5760638f93b54fee871dea8651b0fc20cba0e6a9 antares-1b-Q6_K.gguf 675106237244c22e5b69d90fc756a18ac81e09547579f67607373cb59948f4f1 antares-1b-Q5_K_M.gguf 9fe52cdd2036e165a9d355a2ea488f248fd0fd4839cdfff0a5b41ebf5f790a85 antares-1b-Q4_K_M.gguf e51582e2c4b1f796aff3a8318b372bd46e2efab0fb4b31a6d01f53f05dd71687 antares-1b-Q4_K_M-imat.gguf f1eb03cb07e0d0a80a3d09037acb3dcd5b85032f01525b71a88cadff92132612 antares-1b-IQ4_XS.gguf 269eaf3b34efcce471277e7634c2dca8c12665d78657c00c14036812208dccbf antares-1b-Q3_K_M.gguf 04570f582bcdd1b2310d5edae67433c89ff9d46bd7629590be14fcfc3736698b antares-1b-Q2_K.gguf 8eb4e617ce8a7921171fc7e40a6ab991c03d036058ea8cec69dd3a970780f196 antares-1b.imatrix ``` - Benchmark: the model's own vulnerability-localization benchmark, full task set. One task references a deleted GitHub repository, so the denominator is 499, not 500 — held identical on every rung (a fixed client timeout was raised so the three largest repos couldn't time out on slow rungs only). - Measurement hygiene, every session: corpus raw-verified from disk before any GPU time; a prompt-fidelity sentinel (tokenize count == `prompt_tokens`) after every server start; orphaned benchmark containers swept between runs; model bytes hash-gated against the sums below before every measurement. - Hardware: RTX 5070 Ti 16 GB, Ubuntu 24.04, CUDA sm_120 build, one runtime across every rung. - Full methodology, raw per-run aggregates, and the harness scripts: *(links land here when the companion repo and writeups go public).* ## Intended use & limitations This is a 1.8B-parameter localizer, not an authority. It ranks *files that deserve a look* for a stated weakness class; it does not explain, patch, or prove anything. On its own benchmark the best run finds a relevant file about one time in three, and it flags something on nearly every clean tree — so wire it as a triage assistant feeding a human or a stronger reviewer, never as an autonomous blocker. Measured on its own benchmark corpus only; behaviour elsewhere is uncharacterized beyond one field test. ## License Apache-2.0, inherited from [fdtn-ai/antares-1b](https://huggingface.co/fdtn-ai/antares-1b) (a full copy is in `LICENSE`; upstream ships no NOTICE file). The upstream repository is gated — if you want the original safetensors, the CLI tooling, or the benchmark, go through their gate; this repo exists to publish *measured* quantizations, not to route around upstream. If you are the upstream team and want anything here changed, open a discussion.