# Runtime guide and evaluation boundaries

Use **40-layer `high`** as the primary mode of APUS-Reflex-35B-A3B checkpoint-5949. The **20-layer `low`** path is an explicitly selected experimental decision exit. The model is distributed as merged BF16 weights and needs no separate LoRA adapter.

## Portable runtime acceptance

**Passed: 160/160 decisions identical; maximum logit and candidate-probability differences 0.**

The standalone runtime passed all 80 questions at both 20 and 40 layers against the native merged reference. The strict probability-difference gate of 1e-6 passed; all 60 sensitive checkpoint parameters retained their exact serialized precision. The tested process imported no project-local jev, Swift, or PEFT modules. See [comparison](evaluation/standalone-comparison.json), [Frozen80 results](evaluation/standalone-frozen80.json), and [text execution smoke](evaluation/text-smoke.json). TYPE high returned `red shoes`; low is not recommended for text generation. GPU validation used the local release artifacts before upload; repository file identities are verified separately.

## Download and environment

Download the independent repository root, then install its pinned dependencies:

```bash
python -m pip install huggingface_hub
hf download apus-ailab/APUS-Reflex-35B-A3B --local-dir ./APUS-Reflex-35B-A3B
cd APUS-Reflex-35B-A3B
python -m pip install -r requirements.txt
python examples.py . --device cuda:0 --effort high
```

Pin `hf download --revision <commit>` for repeatable use. The portable loader reads a local snapshot and its `depth_config.json`. It checks **Transformers 5.16.1**, uses BF16 by default, and selects the MoE model class from the supplied configuration. Its source has no project-local Swift/PEFT dependency.

The native merged GPU evaluation used a **96 GB RTX PRO 6000**. Budgeting **75 GB or more GPU memory** for BF16 is a rough planning estimate, not a measured minimum: long inputs, generation, batch size and the serving stack change memory use. The 20-layer exit still loads the full model; it is not a smaller weight package or a promise of half the memory use.

## Decision interface

Run from the downloaded model directory so that the bundled `openjet_runtime` package is importable:

```python
from openjet_runtime import OpenJet

runtime = OpenJet.from_pretrained(".", device="cuda:0", dtype="bfloat16")
request = {
    "id": "support-731",
    "group_id": "support-731",
    "primitive": "choice",
    "state": "Order 731 was delivered. The customer confirms that the issue is resolved.",
    "instructions": "Select the next support action.",
    "criteria": [
        {"id": "close", "description": "Close the resolved support ticket."},
        {"id": "refund", "description": "Refund an undelivered order."},
    ],
}
result = runtime.decide(request, effort="high")
print(result)
```

- `choice` accepts 2–16 distinct candidate IDs with nonempty descriptions.
- `id`, `group_id`, `state` and `instructions` must be nonempty strings.
- `noul` and `score_level` use the runtime's canonical yes/no candidates; `score_level` is not an aggregate ordinal-score API.
- The runtime uses the checkpoint's `jev.dynamic.prompt.v2` contract and a no-thinking chat template. It validates single-token candidate labels before scoring.
- Inputs longer than the runtime's configured limit (at most 8,192 prompt tokens) are rejected without truncation.
- The response maps candidate IDs to normalized candidate probabilities; `choice` also returns the selected ID.

To examine the experimental exit, explicitly pass `effort="low"` or run `python examples.py . --effort low`. There is no confidence-based automatic choice between the exits. Use high for text generation; `python examples.py . --effort high --text` also exercises the slow TYPE reference path.

The bundled loader uses `Qwen3_5MoeForConditionalGeneration` and preserves the checkpoint precision of `linear_attn.A_log` and `linear_attn.norm.weight`. Casting these parameters to BF16 can change decisions. Use the bundled loader to reproduce the published results; its 60 sensitive parameters are checked against the serialized checkpoint in [dtype validation](evaluation/dtype-validation.json).

## Merged-model evaluation

| Model and execution path | Depth | Frozen80 correct | Accuracy |
| --- | ---: | ---: | ---: |
| Checkpoint-5949 merged BF16, native | 40 | 71 / 80 | 88.75% |
| Checkpoint-5949 merged BF16, native experimental exit | 20 | 63 / 80 | 78.75% |

These results are from a completed GPU evaluation of the new merged model. Frozen80 is a reused engineering development panel covering Browser, HelpSteer3, BoolQ, MNLI and attribute decisions. It is not an independent blind benchmark, a calibrated correctness estimate, or a browser end-to-end task-success measurement. The source adapter's larger evaluation panels are not evidence for this merged artifact.

The native adapter-versus-merged comparison recorded maximum absolute candidate-probability changes of approximately **0.14943 at 40 layers** and **0.17818 at 20 layers**. Both probability-equivalence gates failed. CPU merge arithmetic verification and the merged model's accuracy do not cancel that result. Re-evaluate downstream calibration and decision thresholds on the merged model; do not transfer adapter thresholds on an assumption of numerical equivalence.

## Supported scope

The 40-layer model is the primary release path. The 20-layer exit is experimental; source-checkpoint development evidence showed substantial weakness in shallow Browser/TYPE behavior. A shallow-exit decision result does not establish that the exit is suitable for browser text generation.

The reference runtime is single-request and text-only. It rejects multimodal placeholders; image/video inference was not validated for this release. This release does not claim automatic depth routing, calibrated probabilities, production concurrency or latency targets, vLLM/SGLang parity, or end-to-end browser execution. Candidate probabilities describe relative preference among the supplied candidates, not the probability that an action will succeed.

## 中文说明

正式主模式是 **40 层 high**；**20 层 low** 是手动指定的实验性决策出口，仍需加载完整权重。portable 独立验收已通过：160/160决策与native merged一致，logit与概率最大差为0；60个敏感参数逐值匹配checkpoint。

本 merged checkpoint-5949 已完成独立 native GPU Frozen80 评估：40 层 71/80（88.75%），20 层 63/80（78.75%）。这是重复使用的工程开发面板，不能解释为独立盲测、校准后的正确率或浏览器端到端成功率。

原 adapter 与 merged 的候选概率最大绝对差分别约为 0.14943（40 层）和 0.17818（20 层），概率等价门均未通过。需针对 merged 重新验证概率校准和业务阈值，不可直接迁移 adapter 阈值。当前仅验证文本场景；自动路由、多模态、生产并发性能和其他推理引擎的等价性均不在本次验证范围。

**Authors:** gumpcheng ([xDAN2099](https://huggingface.co/xDAN2099)), zhangxu, [APUS AI-LAB](https://github.com/APUS-AI-Lab).