---
license: apache-2.0
base_model: Qwen/Qwen3.5-2B
language:
- en
library_name: transformers
tags:
- decision-model
- system-one
- jev
- jev-alternative
- typed-decisions
- self-hosted
- jevbench
- calibration
---

# JevK5-2B v0.2: the small JevK5

JevK5-2B is the 2-billion-parameter version of [JevK5](https://huggingface.co/alibiserikbay/JevK5),
an Apache-2.0 open alternative to TypeSafe's Jev for typed decisions. It reads a document and a
yes/no (`noul`), choice, or score question and returns **a probability for every option in one
forward pass, with nothing generated**. It is not affiliated with TypeSafe AI.

It was trained on exactly the same data, the same way, as JevK5 v0.2 (4B), on the smaller
Qwen3.5-2B. It keeps most of JevK5's accuracy at half the memory: good at reading a document and
deciding, weaker on arithmetic, multi-step reasoning, and general knowledge.

- **Base:** Qwen3.5-2B, with a LoRA (rank 16, attention and linear-attention projections) merged
  into the weights
- **Readout:** a softmax over the answer letters' next-token logits, divided by one calibration
  temperature (`jevk5_config.json`, T = 1.42)
- **Memory:** 3.5 GB of bf16 weights, about 3.8 GB at peak on a 4,000-token document; a 2.0 GB
  GGUF (Q8_0) is in [JevK5-GGUF](https://huggingface.co/alibiserikbay/JevK5-GGUF)
- **Speed:** about 9 ms per decision on an H100 with the [JevK5 runtime](https://github.com/allebee/jevk5)
- **License:** Apache-2.0

## Results

**Held-out teacher questions.** 362 questions from three business domains that training never saw
(residential leases, public-sector permits, manufacturing QC). This is the set used to choose
between models and to fit the temperature.

| | JevK5-2B | JevK5 (4B) |
|---|---:|---:|
| All held-out teacher questions | 0.751 | 0.804 |
| Extraction | 0.91 | 0.94 |
| Routing | 0.97 | 0.89 |
| Trade-offs | 0.88 | 0.88 |
| Judging answers | 0.70 | 0.73 |
| Multi-step lookups | 0.67 | 0.76 |
| Ambiguous questions | 0.59 | 0.76 |
| Dates and numbers | 0.42 | 0.58 |

Each question family has only 21-42 items, so the per-family rows are rough.

**JevBench's 231 public decisions**, through JevBench's own runner and scoring. Reported only;
never used to choose anything. These are our runs, not official results: JevK5-2B has not been
submitted to JevBench.

| Split | n | JevK5-2B | JevK5 (4B) |
|---|---:|---:|---:|
| easy | 48 | 1.000 | 1.000 |
| original (standard) | 72 | 0.806 | 0.958 |
| hard (public half) | 111 | 0.604 | 0.739 |
| hard-tier calibration error (ECE) | | 0.071 | 0.066 |
| distance to gold on probability items (TVD) | 10 | 0.241 | 0.196 |

## How it was trained

The same data and recipe as JevK5 v0.2. Qwen3.6-27B, with thinking on, wrote realistic documents
with hard typed questions across 17 business domains and answered every question twice,
independently; a question was kept only when both answers matched the intended one. Training used
3,272 of those questions plus as many human-labelled items from MMLU-Pro, WANLI, MultiNLI, BoolQ,
banking77, ARC and CommonsenseQA: cross-entropy on the option-letter logits, 2 epochs, learning rate
3e-5. The temperature was fitted on the held-out teacher questions above.

We also tried, at 2B, distilling JevK5's own option probabilities, three option orders per
question, 3,400 extra replay questions, and a higher learning rate with a larger adapter. None
improved the held-out teacher questions, so this is the plain recipe.

**No JevBench item, public or held out, and no output of Jev was used for training, tuning or
selection.** Two corrections, shared with JevK5, are described in `CHANGELOG.md` in the
[repository](https://github.com/allebee/jevk5):
- Our hand-written calibration set echoed a public JevBench instruction and one public item's rule
  wording. Both are rewritten.
- The 940 MMLU-Pro items in the training data came from MMLU-Pro's test split, since it has no
  training split. JevK5 (4B), trained on the same items, scores 5.5 points higher on them than on
  MMLU-Pro items it never saw; the 2B has not been measured. The next version uses no test split.

## Known weak spots

- Dates, amounts and multi-step lookups: the largest gap to the 4B model.
- The standard tier drops to 0.806 from the 4B's 0.958.
- Not measured on JevBench's sealed decisions, where JevK5 (4B) drops to 33.1% and every one-pass
  model struggles. Expect the same or worse here.
- English only. Inputs over 16,384 tokens are refused, not cut.

## Use

With the JevK5 runtime on an NVIDIA GPU:

```bash
pip install "jevk5[fast] @ git+https://github.com/allebee/jevk5@v0.2.0"
```

```python
from jevk5 import JevK5

model = JevK5("alibiserikbay/JevK5-2B")
model.decide(
    "I was billed twice for order #4411. Please refund the duplicate charge today.",
    {"type": "choice", "instructions": "Which team should handle this?",
     "criteria": {"billing": "Payments and refunds", "tech": "Bugs", "sales": "New purchases"}},
)
# {'type': 'choice', 'choice': 'billing', 'confidence': 0.99, ...}
```

On other GPUs, on a Mac, or on a CPU, use the GGUF build with llama.cpp: see
[JevK5-GGUF](https://huggingface.co/alibiserikbay/JevK5-GGUF).

## Credits

Qwen3.5-2B and Qwen3.6-27B by the Qwen team (Apache-2.0). The one-pass readout and prompt come
from SemIf by TheoLeeCJ (MIT). Evaluated with JevBench (github.com/fstandhartinger/jevbench, MIT).
Not affiliated with TypeSafe AI or Jev.