---
base_model:
- Qwen/Qwen3.5-4B
- Qwen/Qwen3.5-9B
- Qwen/Qwen3.5-35B-A3B
library_name: transformers
language:
- en
- zh
tags:
- decision-model
- dynamic-depth
- structured-output
---
# APUS-OpenJev-v1
Technical Report
**Decision models with selectable compute depth.**
[Model weights](https://huggingface.co/apus-ailab/APUS-OpenJev-v1/tree/main) · [Architecture](ARCHITECTURE.md) · [Evaluation data](https://huggingface.co/datasets/apus-ailab/APUS-OpenJev-Eval-Frozen80)
## 1. Introduction
We introduce **APUS-OpenJev-v1**, a family of decision models for browser agents and business workflows. Given a task, context, and candidate actions, the model returns a scored choice that an application can execute. The family provides 4B, 9B, and 35B-A3B variants with a common decision interface.
**Language-native decisions.** Task definitions and candidate meanings arrive through natural language, allowing one shared model to handle changing decision spaces without a separate business-category classifier.
**Compute matched to the decision.** The architecture combines selectable execution depth with [candidate-aware output projection](#candidate-aware-output-projection). Applications choose `effort="low"` or `effort="high"`; the native low path pairs an earlier trained exit with a readout that computes scores only for the supplied candidate labels on supported output heads. This reduces both executed depth and unnecessary output-head work while retaining the model's shared language output space.
**Joint post-training.** Decision learning is coordinated across computation depths. Both execution paths learn from reference decisions, while the complete path supplies a distribution-level learning signal for the shorter path. This trains useful early decisions rather than relying on an untrained intermediate representation. The [architecture guide](ARCHITECTURE.md) explains the design and its trade-offs.
**Decision-oriented inference.** The runtime scores candidate labels and maps them back to application actions. This removes the need to generate a structured answer token by token for bounded-choice tasks. Choose `effort="low"` or `effort="high"` to balance compute cost and decision quality.
On our frozen 80-question development panel, **APUS-OpenJev 35B-A3B achieves 88.75% and 9B achieves 85.0% accuracy**, compared with **82.5% for the Jev API**. The optimized 9B decision service reports **25.58 ms median HTTP latency** in the run snapshot below. The 85.0% result uses `9B-3000`; the optimized latency measurement uses `9B-5949`. Accuracy and latency are reported with their evaluation settings below. On this development panel, the full-depth 35B-A3B result is **6.25 percentage points higher** than the recorded Jev API baseline; 9B is **2.50 points higher**.
*Figure 1. Valid and correct decisions / all 1,000 questions on the expanded development panel. Historical APUS checkpoint-5949 base + adapter runs at full depth; Laya uses typed full input. Jev has 972 valid results (28 unavailable results remain in the denominator). The published 9B uses checkpoint-3000; this chart is not a 1,000-question validation of the merged releases. [Chart data](accuracy-1000-data.json).*
## 2. Evaluation Results
### Decision quality · 80-question release regression
| Model | Correct / total | Accuracy |
| --- | ---: | ---: |
| [**APUS-OpenJev 35B-A3B**](https://huggingface.co/apus-ailab/APUS-OpenJev-v1-35B-A3B) | **71 /80** | **88.75%** |
| [**APUS-OpenJev 9B**](https://huggingface.co/apus-ailab/APUS-OpenJev-v1-9B) | **68 /80** | **85.00%** |
| [APUS-OpenJev 4B](https://huggingface.co/apus-ailab/APUS-OpenJev-v1-4B) | 66 /80 | 82.50% |
| Jev API | 66 /80 | 82.50% |
| Laya · typed configuration, complete input | 55 /80 | 68.75% |
All models are compared on the same frozen question identities and reference labels. Laya's complete-input typed configuration reproduces 55/80 across three runs; its complete-input English configuration scores 54/80. Repetition establishes consistency on these questions, not additional independent evidence. These panel differences do not establish broad or statistically significant superiority.
The 35B-A3B row uses **checkpoint-5949, merged BF16, all 40 layers**. The bundled standalone runtime reproduces 71/80 on the same panel. Candidate probabilities are not numerically equivalent to the source adapter; see [35B evaluation evidence](https://huggingface.co/apus-ailab/APUS-OpenJev-v1-35B-A3B/blob/main/evaluation/standalone-frozen80.json) and [runtime details](https://huggingface.co/apus-ailab/APUS-OpenJev-v1-35B-A3B/blob/main/RUNTIME.md).
### Results by task
*Figure 2. Correct answers and accuracy for each task family. APUS models use the full compute budget; Laya uses the complete-input typed configuration. Each family contains 16 questions, so one answer changes accuracy by 6.25 percentage points.*
The 4B model leads this panel's Browser subset, while 9B's gains come from principle-based judgments and natural language inference. Laya's corrected Browser result is 4/16. The task breakdown and source hashes are recorded in [subset-chart-data.json](subset-chart-data.json).
### Response latency
*Figure 3. 9B · effort: high · vLLM HTTP API full-response latency · one RTX PRO 6000 Blackwell 96GB GPU. The supplied run snapshot reports 333/333 successful requests and 0% errors. This is full-response latency, not streaming first-token latency.*
| Serving path | P50 | P95 | P99 | Source |
| --- | ---: | ---: | ---: | --- |
| **APUS-OpenJev 9B · high** | **25.58 ms** | **222.36 ms** | **275.58 ms** | Supplied run snapshot |
| [**APUS-OpenJev 4B**](https://huggingface.co/apus-ailab/APUS-OpenJev-v1-4B) | **18.00 ms** | **100.06 ms** | **154.32 ms** | Reported P50; ratio-derived P95/P99 |
| **Jev / TypeSafe official API** | **280.80 ms** | **376.56 ms** | **830.45 ms** | Supplied pooled 240-request results |
The 4B P50 uses the updated reported result of **18 ms**. Its P95/P99 apply the supplied cross-session ratios of **45% / 56%** to the corresponding 9B high percentiles, rounded to two decimals. The 9B snapshot reports **333/333 successful requests and 0% errors**; these request counts are not attributed to 4B.
The chart distinguishes the supplied 333-request snapshot from the independently archived three-run C1 validation: **25.34 / 222.83 / 276.78 ms** median per-run P50/P95/P99, with **3,120/3,120** successful serial requests on an RTX PRO 6000 96GB. The exact 333-request raw trace is not part of that archive. Accuracy uses 80 fixed questions; replay counts do not increase the number of independent questions.
The optimized path is the candidate-scoring `/decide` service. The standard Chat Completions launcher below has a separate generation path. The 4B bars use the reported P50 and ratio-derived P95, as shown above; they are not a new request-log replay in this session. Laya local-pipeline and Jev public-API measurements remain labeled in the chart; different network and processing boundaries do not establish a model speedup ratio. See [measurement sources](latency-vllm-data.json).
**TypeSafe official API — updated three-round reference.**
| Run | Mean HTTP response time | Accuracy |
| --- | ---: | ---: |
| Round 1 | 298.83 ms | 68/80 · 85.00% |
| Round 2 | 303.55 ms | 66/80 · 82.50% |
| Round 3 | 305.57 ms | 65/80 · 81.25% |
| **Overall** | **302.65 ms** | **199/240 · 82.92%** |
The updated screenshot reports **P50 280.80 ms, P95 376.56 ms and P99 830.45 ms**, computed from all **240 HTTP 200 responses pooled across three rounds**, rather than averaging per-round percentiles. Mean response time remains **302.65 ms**. Timing includes the public-network overhead from Runpod to the official service; 80 questions repeated three times provide a sample-specific tail-latency reference. The earlier task-accuracy charts retain their original single-run results; this table records the separate three-round evaluation.
Network reference, same Runpod host: Google ICMP ping averaged 31.97 ms (about 30 ms; 5 probes). TypeSafe did not respond to ICMP ping; TCP port 443 connection averaged 181.55 ms (10 probes), and TLS handshake averaged 184.57 ms (10 new connections). The API test used 240 consecutive calls across three rounds. TCP/TLS probes were separate; their timings are not subtracted from each API request. Source: supplied measurement screenshot.
### Selectable effort and HTTP serving
Select `effort="high"` for decision quality or `effort="low"` for a smaller compute budget in the native effort runtime. The same 4B checkpoint scores **82.50% / 76.25%** at high / low: a **6.25 percentage-point** difference (**7.58% relative**).
The separate **9B · effort: high · vLLM HTTP API** reference reports **25.58 / 222.36 / 275.58 ms** P50/P95/P99 on **one RTX PRO 6000 Blackwell 96GB GPU**. These are complete HTTP responses. The published standard vLLM launcher runs the full-depth model; the figure does not imply that it exposes a low-effort HTTP route. [Measurement sources](effort-tradeoff-data.json).
### Download the models
[APUS-OpenJev-v1 Collection](https://huggingface.co/collections/apus-ailab/apus-openjev-v1-6ab1ee888eb002fcdd3a2825)
Standalone repositories use standard root-level model files. This repository and its legacy download paths remain available.
- **4B · checkpoint-5949 · BF16**: [https://huggingface.co/apus-ailab/APUS-OpenJev-v1-4B](https://huggingface.co/apus-ailab/APUS-OpenJev-v1-4B)
- **9B · checkpoint-3000 · BF16**: [https://huggingface.co/apus-ailab/APUS-OpenJev-v1-9B](https://huggingface.co/apus-ailab/APUS-OpenJev-v1-9B)
- **35B-A3B · checkpoint-5949 · BF16**: [https://huggingface.co/apus-ailab/APUS-OpenJev-v1-35B-A3B](https://huggingface.co/apus-ailab/APUS-OpenJev-v1-35B-A3B)
## 3. Evaluation Set
*Figure 4. Dataset composition, source datasets, and target capabilities. Parent groups identify related examples; equal question counts do not imply equal task difficulty or production traffic.*
| Task family | Questions | Decision evaluated |
| --- | ---: | --- |
| Browser / Mind2Web | 16 | Select an action from a static page state |
| HelpSteer3 | 16 | Check a response against a supplied principle |
| BoolQ | 16 | Answer a binary question from evidence |
| MNLI | 16 | Distinguish entailment, neutral, and contradiction |
| Score / GoEmotions | 16 | Judge whether an individual attribute applies |
The panel contains **80 questions from 79 parent groups** and is released as `validation`. It has informed development and model selection. Score examples evaluate a specified emotion category independently; detailed composition is available in the dataset statistics. Browser evaluation covers offline action selection, not complete website tasks.
The [dataset](https://huggingface.co/datasets/apus-ailab/APUS-OpenJev-Eval-Frozen80) includes JSONL/Parquet, frozen identifiers, provenance, schema documentation, and verification code. It is a separate private repository with its own access permissions.
## 4. Decision Encoding
A request carries the task, supporting context, and 2–16 candidate descriptions. The runtime assigns request-local short labels, evaluates the legal candidate set, and returns the selected candidate ID with relative scores. Application code assembles the response. Candidate meanings can change between requests without adding a fixed business-category classifier.
A legal output can still be the wrong decision. Candidate probabilities are not calibrated confidence, and business thresholds require validation. See the [runtime contract](9B-3000/RUNTIME.md) for the supported request format.
### Candidate-aware output projection
Compute depth and output width address two different costs. The native low-effort path combines an earlier exit with **candidate-row projection**: it selects the language output weights for the request's candidate labels before computing their scores. This retains the shared language output space while avoiding scores for unrelated vocabulary entries in that path.
| Execution path | Output-head computation |
| --- | --- |
| Native `effort="low"` | Candidate rows first; full-head fallback for unsupported output heads |
| Native `effort="high"` | Full vocabulary, then candidate selection |
| Current vLLM decision / Chat service | Full vocabulary; decision gateway extracts candidate scores afterward |
The optimization reduces output-head work, not the context-processing backbone. Current vLLM latency results do not include candidate-only output-head execution. See the [Technical Report](TECHNICAL_REPORT.md#6-performance-where-the-savings-can-come-from) for the mechanism and execution boundaries.
## 5. Minimal Inference
The root [meta.yaml](meta.yaml) catalogs model variants and their configuration hashes. Include it when downloading a variant; the repository and existing model paths are unchanged. You can also use [download_model.py](download_model.py) with `--variant 9B-3000 --local-dir ./APUS-OpenJev-v1`; it selects and checks the variant at one fixed Hub revision. Hub download statistics count requests to selected metadata files, not unique users or completed weight transfers. Older subfolder-only clients can bypass this metadata.
The repository is a model-family bundle. Choose a subdirectory containing complete BF16 weights and the reference runtime:
| Variant | Model directory | Suggested use |
| --- | --- | --- |
| **9B** | `9B-3000/` | Quality-focused evaluation |
| **4B** | `4B-5949/` | Smaller parameter footprint |
An additional 9B research variant is listed in the [artifact manifest](bundle-manifest.json). Load a model subdirectory, not the repository root.
Use a CUDA-capable PyTorch environment. Select the model directory to download:
```bash
python -m pip install huggingface_hub
hf auth login
hf download apus-ailab/APUS-OpenJev-v1 \
--include "meta.yaml" "9B-3000/*" --local-dir ./APUS-OpenJev-v1
cd ./APUS-OpenJev-v1/9B-3000
python -m pip install -r requirements.txt
python examples.py . --device cuda:0 --effort high
```
For 4B, download `4B-5949/*` and enter that directory. The included runtime implements the budget control; ordinary Transformers loading does not enable it automatically. Text generation should use `high`. See [inference documentation](9B-3000/RUNTIME.md) and [examples](9B-3000/examples.py).
### Deploy with vLLM
The [single-file launcher](deployment/serve_vllm.py) downloads the merged **9B-5949** weights from **`apus-ailab/APUS-OpenJev-v1`** and starts a standard OpenAI-compatible vLLM server. It resolves the requested revision to a fixed commit before downloading and prints the resolved model identity.
On Linux with Python 3.12 and an NVIDIA GPU with sufficient memory (the existing 9B serving configuration was tested on RTX PRO 6000 96GB):
```bash
python3.12 -m venv .venv
source .venv/bin/activate
pip install vllm==0.29.0 transformers==5.17.0 huggingface-hub==1.32.0 openai==3.16.2 ninja==1.13.2
# Blackwell: use the CUDA 13 compiler used in the GPU verification.
pip install nvidia-cuda-nvcc==13.4.92 nvidia-cuda-crt==13.4.92 nvidia-cuda-cccl==13.3.4.3.1
export CUDA_HOME="$(python -c 'import sysconfig; print(sysconfig.get_path("purelib") + "/nvidia/cu13")')"
export PATH="$CUDA_HOME/bin:$PATH"
"$CUDA_HOME/bin/nvcc" --version
curl -fL https://huggingface.co/apus-ailab/APUS-OpenJev-v1/resolve/main/deployment/serve_vllm.py -o serve_vllm.py
CUDA_VISIBLE_DEVICES=0 python serve_vllm.py
# Optional: --revision <40-character-commit-SHA>
# Inspect the command without downloading or using a GPU:
# python serve_vllm.py --dry-run
```
The first launch downloads approximately 18 GB and compiles GPU kernels; allow several minutes before sending requests. Check readiness with `curl -f http://127.0.0.1:8000/health`.
In a second terminal using the same environment:
```python
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="local")
response = client.chat.completions.create(
model="APUS-OpenJev-v1-9B",
messages=[{"role": "user", "content":
"Task: find the shipping policy. Choose one visible link: "
"A: Shipping and returns. B: Add to cart. C: Sign in. "
"Reply with only A, B, or C."}],
temperature=0,
max_tokens=16,
extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(response.choices[0].message.content) # A
```
Keep `enable_thinking=False` for these short decision outputs. The original example without this setting reached its 16-token limit before producing a decision; the corrected example returned `A` in the real GPU check.
**GPU verification:** anonymous download from this public repository into a fresh cache, followed by 240/240 successful Chat Completions requests (80 fixed questions replayed three times), with no invalid answers or truncated completions. Local HTTP P50/P95/P99: **45.47 / 191.36 / 232.29 ms**. Test environment: Linux, Python 3.12.3, RTX PRO 6000 Blackwell 96GB, vLLM 0.29.0, PyTorch 2.13.0, Transformers 5.17.0, and the CUDA 13 compiler specified above. This reused an installed Python environment; a clean OS installation was not tested.
This example returns standard chat-completion text through `choices[0].message.content`. It demonstrates **full-depth text serving**, not the specialized candidate-probability `/decide` gateway or `effort` routing. It does not enforce the three candidate labels; application-specific decision serving requires the corresponding prompt/compiler and output constraints. The server binds to localhost by default. Blackwell JIT compilation requires a compatible CUDA toolchain; the launcher disables the FlashInfer sampler as in the tested engine configuration.
## 6. Reproducing the Evaluation
Use the fixed dataset revision `7a63c3d6df2bdc6c8e82dfc566800dc318f0c36b`, preserve question and candidate order, and record the model revision, runtime, dtype, and compute budget. Keep reference labels out of the model input. Report decision accuracy separately from latency, and state precisely where timing starts and ends.
The release records fixed source revisions and per-file hashes. Model files were downloaded and verified, with GPU checks on the source packages; this family bundle preserves those model bytes. Subsequent model-card changes do not represent new training or evaluation. Detailed merge results and diagnostic examples remain in the [9B evaluation records](9B-3000/merged-evaluation.json) and [4B evaluation records](4B-5949/merged-evaluation.json). BF16 merging changed some candidate probabilities, so calibration and routing thresholds must be revalidated.
## 7. License
Original APUS-OpenJev-v1 code and documentation contributed in this release are licensed under the [MIT License](LICENSE). Qwen-derived model weights and inherited code retain their applicable [Apache 2.0 license and notices](9B-3000/LICENSE); see the [license scope](LICENSE_NOTICES.md) for all variants. Dataset licenses are documented separately.
## 8. Citation
```bibtex
@misc{apusopenjev2026,
title = {APUS-OpenJev-v1: Decision Models with Selectable Compute Depth},
author = {gumpcheng and zhangxu and {APUS AI-LAB}},
year = {2026},
url = {https://huggingface.co/apus-ailab/APUS-OpenJev-v1}
}
```
## 9. Contact
Visit the [APUS official website](https://www.apusai.com) or [APUS AI Lab on Hugging Face](https://huggingface.co/apus-ailab). For model questions and feedback, open a discussion in this model repository's [Community tab](https://huggingface.co/apus-ailab/APUS-OpenJev-v1/discussions).
**Authors:** [gumpcheng](https://cryptosun2049.github.io/) ([https://huggingface.co/xDAN2099](https://huggingface.co/xDAN2099)), zhangxu, [APUS AI-LAB](https://github.com/APUS-AI-Lab).