Model card: OpenJev names in the text; note the paper's earlier name
Browse files
README.md
CHANGED
|
@@ -24,9 +24,9 @@ datasets:
|
|
| 24 |
|
| 25 |
**Jun Huang\*, Xin Ren\*** 路 University of Electronic Science and Technology of China 路 \*Equal contribution
|
| 26 |
|
| 27 |
-
**OpenJev**
|
| 28 |
|
| 29 |
-
**A local decision model: typed questions in, calibrated probabilities out, in one forward pass.**
|
| 30 |
the `POST /v1/systemone` request shape (choice, yes/no and score questions over a free-form state), for general
|
| 31 |
decisions and for browser-agent steps (*which operation? which element?*). It runs on your own GPU: no API key,
|
| 32 |
no per-call cost, nothing generated.
|
|
@@ -64,21 +64,21 @@ wev serve --model alanhuangya/OpenJev-1.7B --port 8009 # drop-in POST /v1/syst
|
|
| 64 |
## Results
|
| 65 |
|
| 66 |
Test splits, held out from training; every other model was run on the same requests and scored the same way
|
| 67 |
-
(per-question accuracy; `scripts/compare.py`). For
|
| 68 |
repository README).
|
| 69 |
|
| 70 |
**General typed decisions**
|
| 71 |
|
| 72 |
| model | kev decision-v7 | kev transfer-v4 | typed-decisions |
|
| 73 |
|---|---|---|---|
|
| 74 |
-
| **
|
| 75 |
| Kev-4B | **88.2** | **82.1** | 65.1 |
|
| 76 |
| Kev-8B | 88.1 | 76.8 | 62.7 |
|
| 77 |
| Laya (typed-decisions) | 65.7 | 62.8 | 76.8 |
|
| 78 |
| Laya | 64.3 | 63.7 | 36.2 |
|
| 79 |
|
| 80 |
-
|
| 81 |
-
do not, so on that column they are generalists. kev decision-v7 is Kev's own training suite (
|
| 82 |
its train split); transfer-v4 is out-of-domain for every model here.
|
| 83 |
|
| 84 |
**Browser steps** (Mind2Web test split: websites unseen in training, jev-ultrafast request format; step success =
|
|
@@ -86,13 +86,13 @@ operation and target element both right)
|
|
| 86 |
|
| 87 |
| model | step success | operation |
|
| 88 |
|---|---|---|
|
| 89 |
-
| **
|
| 90 |
| Kev-4B | 21.2 | 35.7 |
|
| 91 |
| Kev-8B | 19.0 | 73.3 |
|
| 92 |
| Laya (typed-decisions) | 0.7 | 13.1 |
|
| 93 |
| Laya | 0.0 | 2.5 |
|
| 94 |
|
| 95 |
-
873 requests; 11 exceed the context
|
| 96 |
|
| 97 |
NNetNav test split (live-web steps, DONE judged by an LLM): step success 61.0, DONE recall
|
| 98 |
80.4, premature DONE 9.4.
|
|
@@ -136,7 +136,8 @@ check those terms before any commercial use.
|
|
| 136 |
|
| 137 |
## Citation
|
| 138 |
|
| 139 |
-
Jun Huang and Xin Ren contributed equally (University of Electronic Science and Technology of China).
|
|
|
|
| 140 |
|
| 141 |
```bibtex
|
| 142 |
@misc{huang2026wev,
|
|
|
|
| 24 |
|
| 25 |
**Jun Huang\*, Xin Ren\*** 路 University of Electronic Science and Technology of China 路 \*Equal contribution
|
| 26 |
|
| 27 |
+
**OpenJev** is an open-source, local alternative to Jev's System One decision API: same request shape, your own GPU, no API key.
|
| 28 |
|
| 29 |
+
**A local decision model: typed questions in, calibrated probabilities out, in one forward pass.** OpenJev-1.7B answers
|
| 30 |
the `POST /v1/systemone` request shape (choice, yes/no and score questions over a free-form state), for general
|
| 31 |
decisions and for browser-agent steps (*which operation? which element?*). It runs on your own GPU: no API key,
|
| 32 |
no per-call cost, nothing generated.
|
|
|
|
| 64 |
## Results
|
| 65 |
|
| 66 |
Test splits, held out from training; every other model was run on the same requests and scored the same way
|
| 67 |
+
(per-question accuracy; `scripts/compare.py`). For OpenJev-4B and OpenJev-8B, two candidates each were read on test (see the
|
| 68 |
repository README).
|
| 69 |
|
| 70 |
**General typed decisions**
|
| 71 |
|
| 72 |
| model | kev decision-v7 | kev transfer-v4 | typed-decisions |
|
| 73 |
|---|---|---|---|
|
| 74 |
+
| **OpenJev-1.7B** | 81.1 | 65.5 | **79.5** |
|
| 75 |
| Kev-4B | **88.2** | **82.1** | 65.1 |
|
| 76 |
| Kev-8B | 88.1 | 76.8 | 62.7 |
|
| 77 |
| Laya (typed-decisions) | 65.7 | 62.8 | 76.8 |
|
| 78 |
| Laya | 64.3 | 63.7 | 36.2 |
|
| 79 |
|
| 80 |
+
OpenJev-1.7B trains on 80% of the typed-decisions train split, like the Laya (typed-decisions) specialist; Kev and Laya
|
| 81 |
+
do not, so on that column they are generalists. kev decision-v7 is Kev's own training suite (OpenJev-1.7B also trains on
|
| 82 |
its train split); transfer-v4 is out-of-domain for every model here.
|
| 83 |
|
| 84 |
**Browser steps** (Mind2Web test split: websites unseen in training, jev-ultrafast request format; step success =
|
|
|
|
| 86 |
|
| 87 |
| model | step success | operation |
|
| 88 |
|---|---|---|
|
| 89 |
+
| **OpenJev-1.7B** | **68.2** | **88.1** |
|
| 90 |
| Kev-4B | 21.2 | 35.7 |
|
| 91 |
| Kev-8B | 19.0 | 73.3 |
|
| 92 |
| Laya (typed-decisions) | 0.7 | 13.1 |
|
| 93 |
| Laya | 0.0 | 2.5 |
|
| 94 |
|
| 95 |
+
873 requests; 11 exceed the context OpenJev-1.7B is evaluated with and count as wrong for it.
|
| 96 |
|
| 97 |
NNetNav test split (live-web steps, DONE judged by an LLM): step success 61.0, DONE recall
|
| 98 |
80.4, premature DONE 9.4.
|
|
|
|
| 136 |
|
| 137 |
## Citation
|
| 138 |
|
| 139 |
+
Jun Huang and Xin Ren contributed equally (University of Electronic Science and Technology of China). The paper
|
| 140 |
+
describes OpenJev under its earlier name, wev.
|
| 141 |
|
| 142 |
```bibtex
|
| 143 |
@misc{huang2026wev,
|