Typed decisions, on your laptop, in one forward pass. The bigger one.
GitHub · PyPI · TinyJev 0.6B · Examples
Typed decisions, on your laptop, in one forward pass. The bigger one.
GitHub · PyPI · TinyJev 0.6B · Examples
**TinyJev 0.6B** | 596M, 1.2 GB | 440 (88.0%) | 59% @ 98.0% | 85 | 🤗 [AnkitAI/TinyJev-0.6B](https://huggingface.co/AnkitAI/TinyJev-0.6B) |
| **TinyJev 4B** | 4.0B, 8.0 GB | 474 (94.8%) | 87% @ 99.1% | 628 | 🤗 [AnkitAI/TinyJev-4B](https://huggingface.co/AnkitAI/TinyJev-4B) |
OD-500 is correct answers out of 500. Gate 0.85 is the share of decisions answered on its own at
confidence ≥ 0.85, and how often those were right. Calibration (ECE 0.071 vs 0.022), coverage at 2%
error (63% vs 92%) and transfer-v4 dev (0.625 vs 0.762) are on the benchmark page. Load either with
`tinyjev.load("TinyJev-0.6B")` or `tinyjev.load("TinyJev-4B")`.
Both rows are fp16. Loading with `quantize=8` keeps the same weights in half the memory and changes
almost nothing: the 0.6B scores 440 at 90 ms, the 4B 473 at 845 ms, one answer in 500 different from
fp16. The gate is the number that matters in production; the rest of the queue goes to a person or a
bigger model. On the same 500: Kev-0.8B 463, Claude Opus 5.5 496, the same Qwen3-0.6B weights read
through letter logits with no head 354.
## Measured
On OpenDecision's Original Choice 500, a suite of 25 domains that was not in the training data, with the same 500 inputs for every model:
| Model | Correct / 500 | Handled alone at confidence ≥ 0.85 |
|---|---:|---:|
| Claude Opus 5.5 (cloud, self-reported probabilities) | 496 | 477 at 100.0% |
| **TinyJev-4B** (fp16) | **474** | **437 at 99.1%** |
| TinyJev-4B at `quantize=8` | 473 | 437 at 99.1% |
| Kev-0.8B (raw logits) | 463 | 186 at 100.0% |
| TinyJev-0.6B | 440 | 296 at 98.0% |
353/375 on dev, 121/125 on holdout, 95% CI 0.928–0.966, ECE 0.022, Brier 0.071, coverage at 2% error 92%.
On Kev's transfer-v4 dev it scores 0.762 against 0.625 for the 0.6B (Kev-4B, a full fine-tune, 0.790).
Every case, every probability, the coverage curves and all the baseline rows:
[benchmarks/opendecision](https://github.com/ankit-aglawe/tinyjev/tree/main/benchmarks/opendecision).
Known weakness, measured: statement-form yes/no checks. On a 21-item check list about one support
email it scores 18 (the 0.6B: 14, it says yes to everything); on hard negatives, statements about a
topic the text mentions but does not support, 81%. Training data for that is the next experiment.
## Use it
```bash
pip install 'tinyjev[mlx]' # Apple Silicon
pip install 'tinyjev[torch]' # everything else
```
```python
import tinyjev
agent = tinyjev.load("TinyJev-4B", quantize=8) # 4.5 GB in memory; drop quantize for fp16, 8 GB
agent.predict({
"state": "Shoes arrived two weeks late and in the wrong size. Also I see two charges on my card.",
"questions": {
"team": {"type": "choice", "instructions": "Which team should handle this?",
"criteria": {"returns": "Exchanges, refunds, wrong or damaged items",
"shipping": "Delivery status, delays, lost packages",
"billing": "Charges, invoices, payment problems"}},
"escalate": {"type": "noul", "instructions": "Does this need urgent human attention?"},
"anger": {"type": "score", "instructions": "How angry is the customer?",
"criteria": ["calm", "frustrated", "very angry"]},
}})
```
Eight bits changed one answer in 500 on the held-out suite. Serve it over HTTP, speaking the System
One request shape:
```bash
tinyjev serve TinyJev-4B --quantize 8 # POST /v1/systemone on 127.0.0.1:8077
```
## What is in this repo
`AutoModel.from_pretrained("AnkitAI/TinyJev-4B")` loads the backbone on its own, a standard
`Qwen3Model` in fp16 with the LoRA already merged. The decision head lives in `head.safetensors`, and
`tinyjev` is what turns hidden states into calibrated answers.
## How it was built, and how it scores
Qwen3-4B-Base with a pointer head, LoRA r16 at lr 5e-5, seed 2, two epochs on the public
`jaredpalmer/kev-suites` decision-v7 training split (12,576 records), Kev's study runner on one H100
for 45 minutes, the adapter merged back into the base. No held-out transfer source was used in
training. Temperature 1.0 at inference; the calibration figures above are at raw logits.
| | transfer-v4 dev | decision-v7 dev | OpenDecision 500 |
| --- | --- | --- | --- |
| TinyJev-4B | 0.762 | 0.859 | 474 / 500 |
| TinyJev-0.6B | 0.625 | | 440 / 500 |
## Support the Project
If this model is useful in your work, you can support independent research:
