EXPLANATION.md

# How this model works

A short explainer for why Qevi-2B answers questions the way it does. Assumes no background.
For usage, see the [model card](README.md).

> **Where the idea came from.** [TypeSafe](https://typesafe.ai) published the case for models that
> return typed, calibrated values instead of generated text, in their System One model **Jev**.
> Jev works on text; this project asked whether the same trick works on images, and began life
> named JEVI (*Jev for images*). It is an independent implementation, not affiliated with
> TypeSafe and not using their method or code.

## What a normal vision model does

Show a model a photo and ask "is there a ladder?" and it *writes* an answer: it picks the most
likely next word, appends that word to its own input, runs the whole model again for the next
word, and repeats. A thirteen-word reply means thirteen-plus passes through 2.1 billion
parameters, to communicate what is really one bit of information.

Ask ten questions about the same photo and it does that ten times over, re-encoding the photo
from scratch each time. The photo is ~1,000 tokens and the question is maybe 20, so almost all
of that work is repeated for nothing.

## What we do instead

**1. Read the answer, don't generate it.**

Instruction-tuned models follow a rigid script. After your question ends, there is a specific
position where the model is about to write the first word of its reply. We call it the *sentinel*.

```
<|im_start|>user
<|vision_start|>[image tokens]<|vision_end|>
Statement: There is a ladder in this image.
Is this statement true of the image? Answer Yes or No.<|im_end|>
<|im_start|>assistant
                     ^ the sentinel: the model's opinion already exists here
```

At that position the model has already computed a score for every word in its vocabulary. The
score for `Yes` and the score for `No` are sitting right there. We take those two numbers and
stop. No generation, no parsing, no chance of it replying in an unexpected format.

**2. Ask everything at once.**

Because nothing is being generated, we can lay the image down once and append every question
after it, then use the attention mask to enforce three rules:

- every question can see the image
- every question can see itself
- **no question can see any other question**

Each question therefore behaves exactly as if it had been asked alone. We verified this: shuffle
the question order and the outputs are bitwise identical. Packing doesn't quietly change answers.

The payoff is that the expensive part (encoding the image, ~500 ms) happens once, while each
extra question costs about 5 ms. Thirty questions run ~24x faster than asking them one at a time.

## What the fine-tune changed

The base model could already do this — reading logits works on a stock checkpoint. We fine-tuned
all 2.13 billion parameters against exactly that objective: cross-entropy over the candidate
answer logits, nothing else. Train-time and test-time behaviour are identical, which is rarer
than it sounds.

One epoch over 85,500 questions on 28,500 images, ~5 GPU-hours on two consumer cards. Results:

| | Base 2B | Qevi-2B |
|---|---|---|
| Accuracy, domains seen in training | 0.855 | **0.977** |
| Accuracy, 12 domains never trained on | 0.745 | **0.889** |
| Calibration error (held-out, lower better) | 0.160 | **0.054** |

The interesting number is the second row. It improved *more* on domains it had never seen
(+14.4) than on ones it trained on (+12.2), which is the evidence it learned a transferable
skill rather than memorising 28,500 images.

## What this costs you

Every question needs a finite answer set declared up front. The model cannot caption an image,
describe it freely, or answer something you didn't anticipate. Free-form generation still works
(we checked) but comes out about 44% shorter than the base model, having been trained on
one-word answers.

For classification, moderation, triage, inspection and tagging, that trade is usually correct.
For open-ended description, use the base model.

## Honest caveats

- Two of 31 domains got **worse**: German traffic signs (0.834 vs 0.879) and heavily pixelated
  car models (0.474 vs 0.501, where both models are near the floor).
- One epoch, one seed, no ablations — there are no error bars on any of these numbers.
- The baseline in every comparison is the base model run through the *same* readout, which
  isolates the effect of fine-tuning. It is not a comparison against the base model used
  conversationally.
- English prompt templates only.

See the [model card](README.md) for the full evaluation, per-domain results and limitations.