alanhuangya commited on
Commit
7418ae8
路
verified 路
1 Parent(s): c8f9e9c

Model card: OpenJev names in the text; note the paper's earlier name

Browse files
Files changed (1) hide show
  1. README.md +10 -9
README.md CHANGED
@@ -24,9 +24,9 @@ datasets:
24
 
25
  **Jun Huang\*, Xin Ren\*** 路 University of Electronic Science and Technology of China 路 \*Equal contribution
26
 
27
- **OpenJev** (project *wev*) is an open-source, local alternative to Jev's System One decision API: same request shape, your own GPU, no API key.
28
 
29
- **A local decision model: typed questions in, calibrated probabilities out, in one forward pass.** `wev-1.7b` answers
30
  the `POST /v1/systemone` request shape (choice, yes/no and score questions over a free-form state), for general
31
  decisions and for browser-agent steps (*which operation? which element?*). It runs on your own GPU: no API key,
32
  no per-call cost, nothing generated.
@@ -64,21 +64,21 @@ wev serve --model alanhuangya/OpenJev-1.7B --port 8009 # drop-in POST /v1/syst
64
  ## Results
65
 
66
  Test splits, held out from training; every other model was run on the same requests and scored the same way
67
- (per-question accuracy; `scripts/compare.py`). For wev-4b and wev-8b, two candidates each were read on test (see the
68
  repository README).
69
 
70
  **General typed decisions**
71
 
72
  | model | kev decision-v7 | kev transfer-v4 | typed-decisions |
73
  |---|---|---|---|
74
- | **wev-1.7b** | 81.1 | 65.5 | **79.5** |
75
  | Kev-4B | **88.2** | **82.1** | 65.1 |
76
  | Kev-8B | 88.1 | 76.8 | 62.7 |
77
  | Laya (typed-decisions) | 65.7 | 62.8 | 76.8 |
78
  | Laya | 64.3 | 63.7 | 36.2 |
79
 
80
- `wev-1.7b` trains on 80% of the typed-decisions train split, like the Laya (typed-decisions) specialist; Kev and Laya
81
- do not, so on that column they are generalists. kev decision-v7 is Kev's own training suite (`wev-1.7b` also trains on
82
  its train split); transfer-v4 is out-of-domain for every model here.
83
 
84
  **Browser steps** (Mind2Web test split: websites unseen in training, jev-ultrafast request format; step success =
@@ -86,13 +86,13 @@ operation and target element both right)
86
 
87
  | model | step success | operation |
88
  |---|---|---|
89
- | **wev-1.7b** | **68.2** | **88.1** |
90
  | Kev-4B | 21.2 | 35.7 |
91
  | Kev-8B | 19.0 | 73.3 |
92
  | Laya (typed-decisions) | 0.7 | 13.1 |
93
  | Laya | 0.0 | 2.5 |
94
 
95
- 873 requests; 11 exceed the context `wev-1.7b` is evaluated with and count as wrong for it.
96
 
97
  NNetNav test split (live-web steps, DONE judged by an LLM): step success 61.0, DONE recall
98
  80.4, premature DONE 9.4.
@@ -136,7 +136,8 @@ check those terms before any commercial use.
136
 
137
  ## Citation
138
 
139
- Jun Huang and Xin Ren contributed equally (University of Electronic Science and Technology of China).
 
140
 
141
  ```bibtex
142
  @misc{huang2026wev,
 
24
 
25
  **Jun Huang\*, Xin Ren\*** 路 University of Electronic Science and Technology of China 路 \*Equal contribution
26
 
27
+ **OpenJev** is an open-source, local alternative to Jev's System One decision API: same request shape, your own GPU, no API key.
28
 
29
+ **A local decision model: typed questions in, calibrated probabilities out, in one forward pass.** OpenJev-1.7B answers
30
  the `POST /v1/systemone` request shape (choice, yes/no and score questions over a free-form state), for general
31
  decisions and for browser-agent steps (*which operation? which element?*). It runs on your own GPU: no API key,
32
  no per-call cost, nothing generated.
 
64
  ## Results
65
 
66
  Test splits, held out from training; every other model was run on the same requests and scored the same way
67
+ (per-question accuracy; `scripts/compare.py`). For OpenJev-4B and OpenJev-8B, two candidates each were read on test (see the
68
  repository README).
69
 
70
  **General typed decisions**
71
 
72
  | model | kev decision-v7 | kev transfer-v4 | typed-decisions |
73
  |---|---|---|---|
74
+ | **OpenJev-1.7B** | 81.1 | 65.5 | **79.5** |
75
  | Kev-4B | **88.2** | **82.1** | 65.1 |
76
  | Kev-8B | 88.1 | 76.8 | 62.7 |
77
  | Laya (typed-decisions) | 65.7 | 62.8 | 76.8 |
78
  | Laya | 64.3 | 63.7 | 36.2 |
79
 
80
+ OpenJev-1.7B trains on 80% of the typed-decisions train split, like the Laya (typed-decisions) specialist; Kev and Laya
81
+ do not, so on that column they are generalists. kev decision-v7 is Kev's own training suite (OpenJev-1.7B also trains on
82
  its train split); transfer-v4 is out-of-domain for every model here.
83
 
84
  **Browser steps** (Mind2Web test split: websites unseen in training, jev-ultrafast request format; step success =
 
86
 
87
  | model | step success | operation |
88
  |---|---|---|
89
+ | **OpenJev-1.7B** | **68.2** | **88.1** |
90
  | Kev-4B | 21.2 | 35.7 |
91
  | Kev-8B | 19.0 | 73.3 |
92
  | Laya (typed-decisions) | 0.7 | 13.1 |
93
  | Laya | 0.0 | 2.5 |
94
 
95
+ 873 requests; 11 exceed the context OpenJev-1.7B is evaluated with and count as wrong for it.
96
 
97
  NNetNav test split (live-web steps, DONE judged by an LLM): step success 61.0, DONE recall
98
  80.4, premature DONE 9.4.
 
136
 
137
  ## Citation
138
 
139
+ Jun Huang and Xin Ren contributed equally (University of Electronic Science and Technology of China). The paper
140
+ describes OpenJev under its earlier name, wev.
141
 
142
  ```bibtex
143
  @misc{huang2026wev,