README.md

---
license: apache-2.0
base_model: fdtn-ai/antares-1b
base_model_relation: quantized
pipeline_tag: text-generation
language:
  - en
tags:
  - gguf
  - llama.cpp
  - security
  - vulnerability-detection
  - agentic
  - terminal-agent
  - granite
  - imatrix
  - benchmarked
---

# Antares-1B GGUF — the measured ladder

Nine quantizations of [fdtn-ai/antares-1b](https://huggingface.co/fdtn-ai/antares-1b)
(Cisco Foundation AI's security SLM — a CWE localizer that explores a repository
through a terminal and submits a ranked list of suspect files; a fine-tune of
[ibm-granite/granite-4.0-1b](https://huggingface.co/ibm-granite/granite-4.0-1b)).

**Every file here was benchmarked before publishing** on the model's own
vulnerability-localization benchmark — the full 499-task set, detection (Phase A)
*and* clean-tree false-alarm rate (Phase B), with three-run error bars on six of
the nine rungs — and every file's sha256 is published. As of 2026-08-02, none
of the dozen-plus community GGUF repos of this model we surveyed publish per-file
hashes or measurements, and we found no imatrix builds among them.

![Quant ladder: File F1 vs file size](ladder.png)

## Which file do I want?

| You want | Take | Why |
|----------|------|-----|
| Best quality per GB | **`antares-1b-Q5_K_M.gguf`** | 0.1723 ± 0.0014 vs the BF16 anchor: Δ −0.002 ± 0.004, at 36% of the size. This recommendation is robust to every noise story on this card |
| Smallest safe file | **`antares-1b-Q4_K_M-imat.gguf`** | Recovers half of static Q4's quality gap at *identical file size*, and is the best retriever measured (three-run bars both ways) — also robust |
| Tightest VRAM, eyes open | `antares-1b-IQ4_XS.gguf` | Good mean, but the widest run-to-run spread measured (span 0.0154, n=3) — *not* a robust recommendation; see the table note |
| Reference / re-quantizing | `antares-1b-BF16.gguf` | The conversion baseline every claim is measured against |
| ❌ Not these | `Q3_K_M`, `Q2_K` | Included to document the cliff: Q3 loses 4× the F1 with 51% abstention; Q2 cannot emit one valid tool call. They are data, not deployables |

**Retriever view** — if you wire Antares as a cheap filter ahead of a stronger judge,
the deciding metric is hit@all, not F1: `Q4_K_M-imat` is the best measured
(0.350 ± 0.017, n=3, vs static Q4's 0.331); IQ4_XS is the worst healthy rung
(0.287 ± 0.016) because it submits half as many files.

## The measured ladder

File F1 on the 499-task benchmark (Phase A), abstain rate, and Phase B true-negative
rate on clean trees. Rungs 0–6 measured 2026-07-30/31; rungs 7–8 measured 2026-08-01;
BF16 and Q8_0 re-measured ×3 on 2026-08-02.

| # | File | imatrix | Size | % of BF16 | File F1 | abstain | TNR |
|--:|------|:-:|-----:|----------:|--------:|--------:|----:|
| 0 | BF16 | — | 3.68 GB | 100% | 0.1740 ± 0.0074 | 0.9% | 0.006 |
| 1 | Q8_0 | — | 1.96 GB | 53% | 0.1741 ± 0.0035 | 1.6% | 0.014 |
| 2 | Q6_K | — | 1.51 GB | 41% | 0.1622 | 1.4% | 0.030 |
| 3 | Q5_K_M | — | 1.32 GB | 36% | **0.1723 ± 0.0014** | 1.2% | 0.020 |
| 4 | Q4_K_M | — | 1.14 GB | 31% | 0.1610 ± 0.0010 | 3.7% | 0.022 |
| 7 | **Q4_K_M-imat** | ✓ | 1.14 GB | 31% | **0.1667 ± 0.0011** | 2.7% | 0.036 |
| 8 | IQ4_XS | ✓ | 1.05 GB | 28% | 0.1648 ± 0.0083 | 2.7% | 0.034 |
| 5 | Q3_K_M | — | 0.95 GB | 26% | 0.0402 | 51.3% | 0.539* |
| 6 | Q2_K | — | 0.77 GB | 21% | 0.0000 | 100% | 1.000* |

**Table notes:**

- ± values are 1 sd over three full benchmark runs; every triple is published here:
  BF16 0.1735/0.1669/0.1817 · Q8_0 0.1721/0.1720/0.1781 · Q5_K_M 0.1720/0.1738/0.1711
  · Q4_K_M 0.1602/0.1608/0.1621 · Q4_K_M-imat 0.1679/0.1659/0.1663 · IQ4_XS
  0.1742/0.1588/0.1613. (Each sd carries 2 degrees of freedom — read spans, not
  third decimals.)
- **Run-to-run noise is rung-dependent.** K-quant triples span ≤ 0.003; BF16's own
  three runs span 0.0148 and IQ4_XS's 0.0154. Cells without ± (Q6_K, Q3, Q2, and all
  single-run TNR cells) carry doubt at whatever their rung's scale turns out to be —
  **rungs are not rank-ordered within noise**. Q6_K's dip below both neighbors is a
  single-run cell inside that noise, not a measured effect.
- **A correction we're proud of:** this table's first draft showed single-run Q8_0
  (0.1798) *above* single-run BF16 (0.1670). Re-running both ×3 dissolved the gap
  entirely (both means 0.174, Δ = 0.000 ± 0.005) — a high draw had met a low draw. We
  re-measured our most interesting number before believing it; treat anyone's
  single-run cells (ours included) with the same skepticism.
- TNR cells are single runs of a binary outcome over 499 clean trees: binomial se is
  ±0.006–0.007 at these rates (Q5's Phase-B triple confirms it: 0.020/0.006/0.014).
  TNR differences between healthy rungs are within noise.
- \*Q3/Q2's "good" TNR is a Goodhart artifact — a model that abstains on everything
  never false-alarms. Pair TNR with the detection column, always.
- **IQ4_XS**: the widest spread on the board at n=3 — suggestive, not proven, but
  treat any single-run IQ4_XS benchmark (ours or anyone's) as uninformative.

- **None of these rungs is a stand-alone merge gate.** Healthy rungs flag something
  on 96–99% of *clean* trees (Phase B). Treat output as leads for a human or a
  stronger model — which matches the upstream guidance ("a lead to verify, not
  proof").
- Token-level quant-damage metrics (KLD / top-token agreement vs BF16) were not
  measured — known future work; these numbers are task-level.

![Run-to-run spread by rung](stability.png)

## We measure ≈0.174 — and the serving stack is not the reason we miss 0.209

Cisco's model card reports 0.209 (three-run mean, their internal pipeline). Across
**seven full-corpus BF16 runs** — two serving engines (llama.cpp and vLLM agree within
0.004 on the same weights), three sampler configurations, and a fresh ×3 repeat on
2026-08-02 — we measure **0.1735 ± 0.0058**, and no run of the seven exceeded 0.1817:
the published figure is not reachable by run-to-run luck. (The ladder table's BF16
cell is the ×3 subset run under the exact ladder protocol; this pooled family is the
anchor.) We did not reproduce the published figure, and the serving stack is
*exonerated* as the cause. The residual is consistent with either a reference-pipeline
difference (the model card's usage snippet renders prompts differently than the
published runner) or a checkpoint-labeling difference (the leaderboard's SFT row is
0.188) — neither verifiable from published artifacts. **All rung claims in this card
are relative to this repo's own BF16 anchor**, which makes them internally consistent
regardless of the absolute-scale question.

## Serving — the traps that silently ruin results

Measurement configuration (what these numbers were produced with):

- llama.cpp `llama-server`, 4 slots × 32k ctx, **raw `/v1/completions`** — the agent
  harness formats its own prompts using the model's granite chat markers; the GGUF's
  embedded chat template is **not** applied during measurement.
- Server-side sampler pins: `--top-k 0 --top-p 1.0 --min-p 0.0`; the harness supplies
  `temperature 0.3, top_p 1.0` per the published protocol. Pin the server: llama.cpp
  otherwise defaults `top_k=40, top_p=0.95, min_p=0.05` for fields a client omits —
  a different model than the one measured.

Traps:

1. **Ollama's `/v1/completions` is not raw.** It wraps prompts in the GGUF's chat
   template — no error, degraded results. If you must use Ollama, set an
   identity template (`TEMPLATE {{ .Prompt }}`) and verify with a token-count sentinel
   (tokenize your prompt separately; compare `prompt_tokens`).
2. **Double-BOS.** The tokenizer adds BOS; if your client also prepends one, quality
   drops silently. The sentinel check above catches this too.

## imatrix provenance

`antares-1b.imatrix` (567 chunks, final PPL 12.52) was computed from `calibration.txt`
(included, 968 KB): 5 generic CWE-class prompts in agent framing, 40 agent transcripts
from an out-of-corpus repository run, and 68 multi-language source files (C/C++,
Python). Decontamination method, stated plainly: the corpus was **constructed from
sources that exclude the benchmark corpus** and spot-checked by string search (0/25
sampled eval-task CWE descriptions found); no automated n-gram decontamination pass
was run. Both imatrix rungs were quantized from the hash-verified BF16 below.
One honest caveat: this corpus is a single point in a large space of reasonable
calibration choices, no second corpus was tried, and the imatrix rungs' scores are
conditional on it.

## Provenance & reproducibility

- **Upstream:** `fdtn-ai/antares-1b` @ revision `10417eb35641b32e7141157db19c76eb545193b6`.
- **Conversion:** llama.cpp `f5b9bd3`, `convert_hf_to_gguf.py` → BF16. The
  `granitemoehybrid` class converts to plain `granite`: the checkpoint is dense
  (363 tensors, zero experts, zero state-space tensors — verified by counting, not
  by trusting the label).
- **Quantization:** `llama-quantize` (same commit); imatrix rungs via
  `llama-imatrix -m BF16 -f calibration.txt -ngl 99` then
  `llama-quantize --imatrix antares-1b.imatrix <BF16> <out> {Q4_K_M|IQ4_XS}`.
- **Changes vs upstream:** format conversion and quantization only — no fine-tuning,
  no re-training, no merges.
- **IQ4_XS requires a llama.cpp new enough for IQ quants** (any 2024+ build; the
  measured server was `f5b9bd3`).
- **K-quants are not byte-reproducible across CPU architectures** — the same command
  on ARM and x86 produces files differing in ~0.016% of bytes (weights sitting on a
  quantization-level boundary land on different sides depending on accumulation
  order). "Q4_K_M" is a recipe, not a file. These are the measured bytes:

```
e8b3c75677beb1044e64cdefe05cf151ab22c6e0a4e119d338876ee2b580bbac  antares-1b-BF16.gguf
e9f27ab7c232536f74721f79917beefdd0fbfcdc0baf313b86ea82701c40d51c  antares-1b-Q8_0.gguf
057b5523f25f428d2f78e59c5760638f93b54fee871dea8651b0fc20cba0e6a9  antares-1b-Q6_K.gguf
675106237244c22e5b69d90fc756a18ac81e09547579f67607373cb59948f4f1  antares-1b-Q5_K_M.gguf
9fe52cdd2036e165a9d355a2ea488f248fd0fd4839cdfff0a5b41ebf5f790a85  antares-1b-Q4_K_M.gguf
e51582e2c4b1f796aff3a8318b372bd46e2efab0fb4b31a6d01f53f05dd71687  antares-1b-Q4_K_M-imat.gguf
f1eb03cb07e0d0a80a3d09037acb3dcd5b85032f01525b71a88cadff92132612  antares-1b-IQ4_XS.gguf
269eaf3b34efcce471277e7634c2dca8c12665d78657c00c14036812208dccbf  antares-1b-Q3_K_M.gguf
04570f582bcdd1b2310d5edae67433c89ff9d46bd7629590be14fcfc3736698b  antares-1b-Q2_K.gguf
8eb4e617ce8a7921171fc7e40a6ab991c03d036058ea8cec69dd3a970780f196  antares-1b.imatrix
```

- Benchmark: the model's own vulnerability-localization benchmark, full task set. One
  task references a deleted GitHub repository, so the denominator is 499, not 500 —
  held identical on every rung (a fixed client timeout was raised so the three
  largest repos couldn't time out on slow rungs only).
- Measurement hygiene, every session: corpus raw-verified from disk before any GPU
  time; a prompt-fidelity sentinel (tokenize count == `prompt_tokens`) after every
  server start; orphaned benchmark containers swept between runs; model bytes
  hash-gated against the sums below before every measurement.
- Hardware: RTX 5070 Ti 16 GB, Ubuntu 24.04, CUDA sm_120 build, one runtime across
  every rung.
- Full methodology, raw per-run aggregates, and the harness scripts: *(links land
  here when the companion repo and writeups go public).*

## Intended use & limitations

This is a 1.8B-parameter localizer, not an authority. It ranks *files that deserve a
look* for a stated weakness class; it does not explain, patch, or prove anything. On
its own benchmark the best run finds a relevant file about one time in three, and it
flags something on nearly every clean tree — so wire it as a triage assistant feeding
a human or a stronger reviewer, never as an autonomous blocker. Measured on its own
benchmark corpus only; behaviour elsewhere is uncharacterized beyond one field test.

## License

Apache-2.0, inherited from [fdtn-ai/antares-1b](https://huggingface.co/fdtn-ai/antares-1b)
(a full copy is in `LICENSE`; upstream ships no NOTICE file). The upstream repository
is gated — if you want the original safetensors, the CLI tooling, or the benchmark,
go through their gate; this repo exists to publish *measured* quantizations, not to
route around upstream. If you are the upstream team and want anything here changed,
open a discussion.