---
license: apache-2.0
base_model:
  - Qwen/Qwen3-0.6B
library_name: pytorch
tags:
  - jev
  - mini-jev
  - decision-model
  - agent
  - agentic-ai
  - tool-selection
  - tool-routing
  - model-routing
  - function-calling
  - agent-control
  - local-inference
  - open-weights
  - qwen
---

# mini-Jev

**A Qwen3-0.6B-based decision model for tool selection, routing, and agent-control experiments.**

mini-Jev is an experimental decision model designed to make structured control decisions inside an agent loop:

```text
state + available candidates → probabilities + selected decision
```

The current release:
- Uses [Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B) as a backbone.
- Keeps the Qwen backbone frozen.
- Adds a lightweight trained decision head (~1.1 MB) to score candidate actions directly.
- Performs candidate scoring and selection; it is not designed for text generation.
- Serves as an experimental baseline and demo for fast, structured agent decision-making.

> **Note on naming:** The model implementation was originally developed under the internal name **ODM Mini v1**. That name remains in class names (`ODMMiniModel`), loader files (`odm_mini.py`), and `config.json` for backwards compatibility.

---

## 1. What is mini-Jev?

AI agents frequently encounter situations where they must make small, structured decisions rather than generate free-form text. Common examples include:

- **Tool selection:** Deciding which tool or external integration to invoke.
- **Function routing:** Selecting the specific API or method to execute next.
- **Next-action prediction:** Choosing between continuing investigation, asking for clarification, or concluding.
- **Continue / finish decisions:** Deciding whether an assigned task is satisfied or requires another step.
- **Model routing:** Directing incoming tasks to appropriate candidate downstream models.

Rather than invoking a large generative language model for every routing and control decision, mini-Jev explores using a small, specialized decision model to evaluate candidate options directly over agent state.

*mini-Jev is an experimental baseline and is not intended for unmonitored production use.*

---

## 2. Current model

**Current release: Qwen3-0.6B-based mini-Jev v1**

### Architecture

```text
Frozen Qwen3-0.6B → candidate representation → lightweight decision head → grouped softmax
```

1. **Backbone:** The base model is [`Qwen/Qwen3-0.6B`](https://huggingface.co/Qwen/Qwen3-0.6B), which remains completely frozen. The weights in this repository contain only the trained decision head; the Qwen backbone is downloaded separately by the loader.
2. **Representation pooling:** State text and candidate descriptions are passed through the model. Only candidate-description tokens are mean-pooled. Candidate IDs, prompt formatting headers, state tokens, and padding tokens are excluded from the pooling mask.
3. **Decision head:** A lightweight two-layer MLP:
   - `Linear(1024, 256)`
   - `GELU`
   - `Linear(256, 1)`
   - Total head parameters: **262,657** (~1.1 MB).
4. **Scoring:** The resulting scalar logits are normalized across candidate options using a grouped softmax to yield probabilities, a selected candidate, confidence, and decision margin.

---

## 3. Quick start

### Installation

```bash
pip install torch transformers safetensors huggingface_hub
```

Download the loader module:

```bash
hf download samatv256/mini-Jev odm_mini.py --local-dir .
```

### Basic usage

```python
from odm_mini import ODMMiniModel

model = ODMMiniModel.from_pretrained("samatv256/mini-Jev")

choice = model.predict_choice(
    state={"user_request": "Find the weather in Boston."},
    candidates=[
        {
            "id": "weather.lookup",
            "description": "Look up the current weather.",
        },
        {
            "id": "calendar.list",
            "description": "List calendar events.",
        },
    ],
)

print("Selected:", choice.selected)
print("Probabilities:", choice.probabilities)
```

### Full snapshot download and usage

You can also download all repository files to a local directory before loading:

```python
import sys
from pathlib import Path
from huggingface_hub import snapshot_download

release_dir = Path(
    snapshot_download(
        repo_id="samatv256/mini-Jev",
        allow_patterns=[
            "README.md",
            "model.safetensors",
            "config.json",
            "odm_mini.py",
            "LICENSE",
        ],
    )
)
sys.path.insert(0, str(release_dir))

from odm_mini import ODMMiniModel

model = ODMMiniModel.from_pretrained(release_dir)

state = {
    "user_request": "Find the current weather in Boston.",
    "available_context": "The user has not provided weather data.",
}
candidates = [
    {
        "id": "weather.lookup",
        "description": "Look up current weather for a specified city.",
    },
    {
        "id": "calendar.list",
        "description": "List upcoming calendar events for the user.",
    },
    {
        "id": "control.finish",
        "description": "Finish because the request has already been completed.",
    },
]

choice = model.predict_choice(state=state, candidates=candidates)
print("Selected candidate:", choice.selected)
print("Probabilities:", choice.probabilities)
print("Confidence:", choice.confidence)
print("Decision margin:", choice.decision_margin)
print("Latency (ms):", choice.latency_ms)
```

`ODMMiniModel.from_pretrained()` accepts either the Hugging Face repo ID or a local directory path. CUDA runs in BF16 by default; CPU runs in FP32.

---

## 4. Example use cases

mini-Jev can be used for several structured agent-control patterns:

- **Tool selection:** Choosing the right tool from a list of available integrations.
- **Function routing:** Determining which endpoint or handler to call.
- **Next-action prediction:** Selecting intermediate steps in a reasoning or execution plan.
- **Agent-control research:** Investigating small-footprint models for local agent decision-making.
- **Model-routing experiments:** Evaluating routing policies between different candidate models. *(Note: Model routing is a conceptual application of the candidate-selection interface, not a validated production routing capability.)*

### Conceptual example

```text
State:
User wants to find the weather in Boston.

Candidates:
- web_search
- weather_tool
- calculator
- finish

Decision:
weather_tool
```

---

## 5. Jev Decisions v1

Researchers interested in training and evaluating decision models can explore the [Jev Decisions v1 dataset](https://huggingface.co/datasets/samatv256/jev-decisions-v1).

- **Dataset card:** [https://huggingface.co/datasets/samatv256/jev-decisions-v1](https://huggingface.co/datasets/samatv256/jev-decisions-v1)
- **Description:** A public dataset containing nearly 12 million canonical agent-decision records (over 6.2M choice-eligible decisions across 1.06M task trajectories) with explicit candidate sets, chosen actions, and eligibility flags.

> **Important:** The current mini-Jev Qwen3-0.6B baseline was **NOT trained on Jev Decisions v1**. It is provided as an open project resource for community research.

---

## 6. Current verified results

The results below reflect the public Qwen3-0.6B baseline checkpoint:

### Training configuration

- **Training data:** 50,000 synthetic decision examples
- **Backbone:** Frozen `Qwen/Qwen3-0.6B`
- **Trained components:** DecisionHead weights only (grouped cross-entropy objective)
- **Checkpoint:** Seed 41, epoch 4
- **Temperature:** Fixed at 1.0; maximum softmax output is reported as confidence.

### Synthetic held-out evaluation

| Metric | Result |
|---|---:|
| Semantic Choice accuracy | **72.97%** |
| Stress Choice accuracy | **67.64%** |
| Counterfactual pair consistency | **67.12%** |

- **Latency measurements:** On an NVIDIA GH200 using BF16 backbone inference and a shared-prefix KV-cache path, total latency was approximately **76–85 ms** for contexts of 256–1,024 state tokens across 3–16 candidates (measured range: 76.11–82.36 ms). Candidate representation extraction reached 831.3 candidates/second at batch size 512.

### Real shadow-agent evaluation

In an offline evaluation on real agent executions across **75 multi-step trajectories and 243 decisions**, the baseline achieved:

- **Action accuracy:** **27.98%**
- **Controller agreement:** **26.34%**

These metrics are reported to transparently show the significant transfer gap between synthetic single-step decisions and dynamic, multi-step agent trajectories.

---

## 7. Limitations

> **mini-Jev v1 is an experimental research prototype and is not ready for unmonitored production agent control.**

- **Limited real-agent transfer:** While performance on synthetic single-step benchmarks is moderate (~73%), accuracy drops substantially (~28%) in real multi-step agent environments.
- **Premature completion:** The model exhibits a known failure mode of high-confidence premature `finish` decisions. After making partial progress, it frequently over-indexes on successful intermediate receipts and chooses `control.finish` before completing remaining steps.
- **Uncalibrated confidence:** Grouped softmax probabilities indicate relative preference among the provided candidates, not calibrated real-world uncertainty.
- **No autonomous control:** Do not use this baseline as the sole decision-maker for consequential actions or mission-critical workflows without a human or supervising controller in the loop.

---

## 8. Public project links

- **Dataset:** [Jev Decisions v1](https://huggingface.co/datasets/samatv256/jev-decisions-v1)
- **Collection:** [mini-Jev Hugging Face Collection](https://huggingface.co/collections/samatv256/mini-jev-small-decision-models-for-ai-agents-6ab3e559bed2124120973013)

Future work will continue exploring improved decision models and training on larger real agent/tool-use datasets.

---

## Released files and license

- `model.safetensors` — Trained DecisionHead weights only (262,657 parameters, ~1.1 MB)
- `config.json` — Architecture specification and backbone reference
- `odm_mini.py` — Inference loader module
- `README.md` — Model card and usage documentation
- `LICENSE` — Apache License 2.0

The mini-Jev decision head and loader code are released under the Apache-2.0 License. The separately downloaded Qwen3-0.6B backbone is governed by its own license terms from the Qwen team.