How this model works
A short explainer for why Qevi-2B answers questions the way it does. Assumes no background. For usage, see the model card.
Where the idea came from. TypeSafe published the case for models that return typed, calibrated values instead of generated text, in their System One model Jev. Jev works on text; this project asked whether the same trick works on images, and began life named JEVI (Jev for images). It is an independent implementation, not affiliated with TypeSafe and not using their method or code.
What a normal vision model does
Show a model a photo and ask "is there a ladder?" and it writes an answer: it picks the most likely next word, appends that word to its own input, runs the whole model again for the next word, and repeats. A thirteen-word reply means thirteen-plus passes through 2.1 billion parameters, to communicate what is really one bit of information.
Ask ten questions about the same photo and it does that ten times over, re-encoding the photo from scratch each time. The photo is ~1,000 tokens and the question is maybe 20, so almost all of that work is repeated for nothing.
What we do instead
1. Read the answer, don't generate it.
Instruction-tuned models follow a rigid script. After your question ends, there is a specific position where the model is about to write the first word of its reply. We call it the sentinel.
<|im_start|>user
<|vision_start|>[image tokens]<|vision_end|>
Statement: There is a ladder in this image.
Is this statement true of the image? Answer Yes or No.<|im_end|>
<|im_start|>assistant
^ the sentinel: the model's opinion already exists here
At that position the model has already computed a score for every word in its vocabulary. The
score for Yes and the score for No are sitting right there. We take those two numbers and
stop. No generation, no parsing, no chance of it replying in an unexpected format.
2. Ask everything at once.
Because nothing is being generated, we can lay the image down once and append every question after it, then use the attention mask to enforce three rules:
- every question can see the image
- every question can see itself
- no question can see any other question
Each question therefore behaves exactly as if it had been asked alone. We verified this: shuffle the question order and the outputs are bitwise identical. Packing doesn't quietly change answers.
The payoff is that the expensive part (encoding the image, ~500 ms) happens once, while each extra question costs about 5 ms. Thirty questions run ~24x faster than asking them one at a time.
What the fine-tune changed
The base model could already do this — reading logits works on a stock checkpoint. We fine-tuned all 2.13 billion parameters against exactly that objective: cross-entropy over the candidate answer logits, nothing else. Train-time and test-time behaviour are identical, which is rarer than it sounds.
One epoch over 85,500 questions on 28,500 images, ~5 GPU-hours on two consumer cards. Results:
| Base 2B | Qevi-2B | |
|---|---|---|
| Accuracy, domains seen in training | 0.855 | 0.977 |
| Accuracy, 12 domains never trained on | 0.745 | 0.889 |
| Calibration error (held-out, lower better) | 0.160 | 0.054 |
The interesting number is the second row. It improved more on domains it had never seen (+14.4) than on ones it trained on (+12.2), which is the evidence it learned a transferable skill rather than memorising 28,500 images.
What this costs you
Every question needs a finite answer set declared up front. The model cannot caption an image, describe it freely, or answer something you didn't anticipate. Free-form generation still works (we checked) but comes out about 44% shorter than the base model, having been trained on one-word answers.
For classification, moderation, triage, inspection and tagging, that trade is usually correct. For open-ended description, use the base model.
Honest caveats
- Two of 31 domains got worse: German traffic signs (0.834 vs 0.879) and heavily pixelated car models (0.474 vs 0.501, where both models are near the floor).
- One epoch, one seed, no ablations — there are no error bars on any of these numbers.
- The baseline in every comparison is the base model run through the same readout, which isolates the effect of fine-tuning. It is not a comparison against the base model used conversationally.
- English prompt templates only.
See the model card for the full evaluation, per-domain results and limitations.