Update README and model card for mini-Jev baseline

README.md

@@ -4,50 +4,125 @@ base_model:
   - Qwen/Qwen3-0.6B
 library_name: pytorch
 tags:
+  - jev
+  - mini-jev
+  - decision-model
   - agent
   - agentic-ai
-  - decision-model
   - tool-selection
-  - system-one
+  - tool-routing
+  - model-routing
+  - function-calling
+  - agent-control
+  - local-inference
   - open-weights
   - qwen
 ---
 
-# ODM Mini v1
+# mini-Jev
+
+**A Qwen3-0.6B-based decision model for tool selection, routing, and agent-control experiments.**
+
+mini-Jev is an experimental decision model designed to make structured control decisions inside an agent loop:
+
+```text
+state + available candidates → probabilities + selected decision
+```
+
+The current release:
+- Uses [Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B) as a backbone.
+- Keeps the Qwen backbone frozen.
+- Adds a lightweight trained decision head (~1.1 MB) to score candidate actions directly.
+- Performs candidate scoring and selection; it is not designed for text generation.
+- Serves as an experimental baseline and demo for fast, structured agent decision-making.
+
+> **Note on naming:** The model implementation was originally developed under the internal name **ODM Mini v1**. That name remains in class names (`ODMMiniModel`), loader files (`odm_mini.py`), and `config.json` for backwards compatibility.
 
-**Open-weight local Choice decision head for agent tool selection, built on frozen Qwen3-0.6B.**
+---
+
+## 1. What is mini-Jev?
+
+AI agents frequently encounter situations where they must make small, structured decisions rather than generate free-form text. Common examples include:
+
+- **Tool selection:** Deciding which tool or external integration to invoke.
+- **Function routing:** Selecting the specific API or method to execute next.
+- **Next-action prediction:** Choosing between continuing investigation, asking for clarification, or concluding.
+- **Continue / finish decisions:** Deciding whether an assigned task is satisfied or requires another step.
+- **Model routing:** Directing incoming tasks to appropriate candidate downstream models.
+
+Rather than invoking a large generative language model for every routing and control decision, mini-Jev explores using a small, specialized decision model to evaluate candidate options directly over agent state.
+
+*mini-Jev is an experimental baseline and is not intended for unmonitored production use.*
+
+---
 
-ODM Mini takes an agent state plus candidate action descriptions and directly ranks the candidates. It produces candidate scores, grouped-softmax probabilities, a selected action, confidence, decision margin, and measured latency. It does **not** generate text.
+## 2. Current model
 
-## Model architecture
+**Current release: Qwen3-0.6B-based mini-Jev v1**
+
+### Architecture
 
 ```text
-Frozen Qwen3-0.6B
-→ candidate-description mean pooling
-→ Linear(1024,256)
-→ GELU
-→ Linear(256,1)
-→ grouped softmax
+Frozen Qwen3-0.6B → candidate representation → lightweight decision head → grouped softmax
 ```
 
-Only description-token hidden states are pooled. The candidate ID, prompt headers, state tokens, and padding are excluded from the pooling mask. The Qwen backbone remains frozen.
+1. **Backbone:** The base model is [`Qwen/Qwen3-0.6B`](https://huggingface.co/Qwen/Qwen3-0.6B), which remains completely frozen. The weights in this repository contain only the trained decision head; the Qwen backbone is downloaded separately by the loader.
+2. **Representation pooling:** State text and candidate descriptions are passed through the model. Only candidate-description tokens are mean-pooled. Candidate IDs, prompt formatting headers, state tokens, and padding tokens are excluded from the pooling mask.
+3. **Decision head:** A lightweight two-layer MLP:
+   - `Linear(1024, 256)`
+   - `GELU`
+   - `Linear(256, 1)`
+   - Total head parameters: **262,657** (~1.1 MB).
+4. **Scoring:** The resulting scalar logits are normalized across candidate options using a grouped softmax to yield probabilities, a selected candidate, confidence, and decision margin.
 
-## Size and required backbone
+---
 
-The ODM-specific trained DecisionHead is approximately **1.1 MB**. The complete runnable model is substantially larger because it additionally requires the separately downloaded [`Qwen/Qwen3-0.6B`](https://huggingface.co/Qwen/Qwen3-0.6B) backbone. This repository does not redistribute Qwen weights.
+## 3. Quick start
 
-## Installation and usage
+### Installation
 
 ```bash
 pip install torch transformers safetensors huggingface_hub
 ```
 
-The code below downloads this five-file release, imports its deterministic loader without `trust_remote_code=True`, separately obtains Qwen3-0.6B, and performs a Choice:
+Download the loader module:
+
+```bash
+hf download samatv256/mini-Jev odm_mini.py --local-dir .
+```
+
+### Basic usage
+
+```python
+from odm_mini import ODMMiniModel
+
+model = ODMMiniModel.from_pretrained("samatv256/mini-Jev")
+
+choice = model.predict_choice(
+    state={"user_request": "Find the weather in Boston."},
+    candidates=[
+        {
+            "id": "weather.lookup",
+            "description": "Look up the current weather.",
+        },
+        {
+            "id": "calendar.list",
+            "description": "List calendar events.",
+        },
+    ],
+)
+
+print("Selected:", choice.selected)
+print("Probabilities:", choice.probabilities)
+```
+
+### Full snapshot download and usage
+
+You can also download all repository files to a local directory before loading:
 
 ```python
 import sys
 from pathlib import Path
-
 from huggingface_hub import snapshot_download
 
 release_dir = Path(
@@ -66,7 +141,7 @@ sys.path.insert(0, str(release_dir))
 
 from odm_mini import ODMMiniModel
 
-model = ODMMiniModel.from_pretrained("samatv256/mini-Jev")
+model = ODMMiniModel.from_pretrained(release_dir)
 
 state = {
     "user_request": "Find the current weather in Boston.",
@@ -88,28 +163,69 @@ candidates = [
 ]
 
 choice = model.predict_choice(state=state, candidates=candidates)
-print("selected candidate:", choice.selected)
-print("probabilities:", choice.probabilities)
-print("confidence:", choice.confidence)
-print("decision margin:", choice.decision_margin)
-print("latency (ms):", choice.latency_ms)
+print("Selected candidate:", choice.selected)
+print("Probabilities:", choice.probabilities)
+print("Confidence:", choice.confidence)
+print("Decision margin:", choice.decision_margin)
+print("Latency (ms):", choice.latency_ms)
 ```
 
-`ODMMiniModel.from_pretrained()` accepts either the Hub model ID or a local snapshot directory. On CUDA it defaults to BF16 backbone inference; on CPU it defaults to FP32. The DecisionHead is always evaluated in FP32.
+`ODMMiniModel.from_pretrained()` accepts either the Hugging Face repo ID or a local directory path. CUDA runs in BF16 by default; CPU runs in FP32.
+
+---
 
-## Training
+## 4. Example use cases
 
-- 50,000 synthetic training decisions
-- frozen `Qwen/Qwen3-0.6B` backbone
-- DecisionHead-only training
-- grouped cross-entropy objective
-- selected seed: **41**
+mini-Jev can be used for several structured agent-control patterns:
 
-The selected head is the seed-41 epoch-4 checkpoint. Temperature remains at `1.0`; its maximum grouped-softmax output is reported as confidence, not as a universally calibrated probability.
+- **Tool selection:** Choosing the right tool from a list of available integrations.
+- **Function routing:** Determining which endpoint or handler to call.
+- **Next-action prediction:** Selecting intermediate steps in a reasoning or execution plan.
+- **Agent-control research:** Investigating small-footprint models for local agent decision-making.
+- **Model-routing experiments:** Evaluating routing policies between different candidate models. *(Note: Model routing is a conceptual application of the candidate-selection interface, not a validated production routing capability.)*
 
-## Evaluation
+### Conceptual example
 
-Synthetic held-out evaluation:
+```text
+State:
+User wants to find the weather in Boston.
+
+Candidates:
+- web_search
+- weather_tool
+- calculator
+- finish
+
+Decision:
+weather_tool
+```
+
+---
+
+## 5. Jev Decisions v1
+
+Researchers interested in training and evaluating decision models can explore the [Jev Decisions v1 dataset](https://huggingface.co/datasets/samatv256/jev-decisions-v1).
+
+- **Dataset card:** [https://huggingface.co/datasets/samatv256/jev-decisions-v1](https://huggingface.co/datasets/samatv256/jev-decisions-v1)
+- **Description:** A public dataset containing nearly 12 million canonical agent-decision records (over 6.2M choice-eligible decisions across 1.06M task trajectories) with explicit candidate sets, chosen actions, and eligibility flags.
+
+> **Important:** The current mini-Jev Qwen3-0.6B baseline was **NOT trained on Jev Decisions v1**. It is provided as an open project resource for community research.
+
+---
+
+## 6. Current verified results
+
+The results below reflect the public Qwen3-0.6B baseline checkpoint:
+
+### Training configuration
+
+- **Training data:** 50,000 synthetic decision examples
+- **Backbone:** Frozen `Qwen/Qwen3-0.6B`
+- **Trained components:** DecisionHead weights only (grouped cross-entropy objective)
+- **Checkpoint:** Seed 41, epoch 4
+- **Temperature:** Fixed at 1.0; maximum softmax output is reported as confidence.
+
+### Synthetic held-out evaluation
 
 | Metric | Result |
 |---|---:|
@@ -117,29 +233,45 @@ Synthetic held-out evaluation:
 | Stress Choice accuracy | **67.64%** |
 | Counterfactual pair consistency | **67.12%** |
 
-Performance measurements used an NVIDIA GH200, BF16 backbone inference, and the shared-prefix KV-cache serving path. For short and medium contexts of approximately 256–1,024 state tokens, total latency was approximately **76–85 ms** for **3–16 candidates** (measured range: 76.11–82.36 ms). Offline representation extraction reached **831.3 candidates/second** at candidate batch size 512 while caching 430,072 candidates. These local measurements are hardware- and workload-specific and are not a direct speed comparison with Jev or any hosted service.
+- **Latency measurements:** On an NVIDIA GH200 using BF16 backbone inference and a shared-prefix KV-cache path, total latency was approximately **76–85 ms** for contexts of 256–1,024 state tokens across 3–16 candidates (measured range: 76.11–82.36 ms). Candidate representation extraction reached 831.3 candidates/second at batch size 512.
+
+### Real shadow-agent evaluation
 
-## Limitations and intended use
+In an offline evaluation on real agent executions across **75 multi-step trajectories and 243 decisions**, the baseline achieved:
 
-> **ODM Mini v1 is a research/hackathon prototype and is not ready for unmonitored production agent control.**
+- **Action accuracy:** **27.98%**
+- **Controller agreement:** **26.34%**
 
-In a real shadow-agent evaluation covering **75 multi-step trajectories and 243 decisions**, ODM Mini achieved:
+These metrics are reported to transparently show the significant transfer gap between synthetic single-step decisions and dynamic, multi-step agent trajectories.
 
-- action accuracy: **27.98%**
-- controller agreement: **26.34%**
+---
 
-The major known failure is **high-confidence premature completion on intermediate multi-step trajectories**. After partial progress, the model can over-index on successful receipts and choose `control.finish` before remaining steps have been completed. Training used synthetic, static decision snapshots, so the published metrics should not be assumed to transfer to arbitrary tools, domains, or agent loops.
+## 7. Limitations
 
-Recommended uses are research, offline evaluation, single-step Choice experiments, and shadow-mode analysis with an independent controller. Do not use this checkpoint as the sole decision-maker for consequential actions or autonomous production control.
+> **mini-Jev v1 is an experimental research prototype and is not ready for unmonitored production agent control.**
 
-## Released files
+- **Limited real-agent transfer:** While performance on synthetic single-step benchmarks is moderate (~73%), accuracy drops substantially (~28%) in real multi-step agent environments.
+- **Premature completion:** The model exhibits a known failure mode of high-confidence premature `finish` decisions. After making partial progress, it frequently over-indexes on successful intermediate receipts and chooses `control.finish` before completing remaining steps.
+- **Uncalibrated confidence:** Grouped softmax probabilities indicate relative preference among the provided candidates, not calibrated real-world uncertainty.
+- **No autonomous control:** Do not use this baseline as the sole decision-maker for consequential actions or mission-critical workflows without a human or supervising controller in the loop.
 
-- `model.safetensors` — ODM Mini DecisionHead tensors only
-- `config.json` — architecture and backbone reference
-- `odm_mini.py` — inference-only loader
-- `README.md` — model card and working example
-- `LICENSE` — Apache License 2.0
+---
+
+## 8. Public project links
+
+- **Dataset:** [Jev Decisions v1](https://huggingface.co/datasets/samatv256/jev-decisions-v1)
+- **Collection:** [mini-Jev Hugging Face Collection](https://huggingface.co/collections/samatv256/mini-jev-small-decision-models-for-ai-agents-6ab3e559bed2124120973013)
+
+Future work will continue exploring improved decision models and training on larger real agent/tool-use datasets.
 
-## License
+---
+
+## Released files and license
+
+- `model.safetensors` — Trained DecisionHead weights only (262,657 parameters, ~1.1 MB)
+- `config.json` — Architecture specification and backbone reference
+- `odm_mini.py` — Inference loader module
+- `README.md` — Model card and usage documentation
+- `LICENSE` — Apache License 2.0
 
-ODM Mini-specific code and weights in this repository are released under Apache-2.0. The separately downloaded Qwen backbone is governed by its own repository terms.
+The mini-Jev decision head and loader code are released under the Apache-2.0 License. The separately downloaded Qwen3-0.6B backbone is governed by its own license terms from the Qwen team.