🚢 Bosun now runs locally — official GGUF builds are up.
Bosun is a programmable relational judge: give it a rule + a pair of findings, get one calibrated score in [0,1]. You program it with a
sentence. Now you can run it on a laptop, CPU, or Apple Silicon via llama.cpp.
Models:
- 🟦 Bosun-XS (0.6B) → https://huggingface.co/Hanno-Labs/bosun-xs-GGUF
- 🟩 Bosun-4B → https://huggingface.co/Hanno-Labs/bosun-4b-GGUF
Each ships f16 / Q8_0 / Q4_K_M. We validated every quant per-pair against the transformers reference — Q8_0 lands within ~0.001–0.01 — and verified XS end-to-end on a Mac through stock llama-server. Steering survives quantization: flip the rule, the score flips.
⚠️ One gotcha: don't use llama.cpp's --rerank mode. It silently drops your <Instruct> and returns degenerate, instruction-blind scores —
it looks like it works but can't steer. Use the completion + logits path (read sigmoid(yes − no) at the final token); each repo's
README has a copy-paste example. Thanks to Frederick Wood for the careful report that prompted these builds.
Apache-2.0. The full models + LoRA source: Hanno-Labs/bosun-xs / bosun-4b. More on the thesis — the missing judgment layer for agent memory — in the launch post: https://hannolabs.ai/field-notes/introducing-bosun
