Instructions to use apus-ailab/APUS-Reflex-35B-A3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use apus-ailab/APUS-Reflex-35B-A3B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="apus-ailab/APUS-Reflex-35B-A3B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("apus-ailab/APUS-Reflex-35B-A3B")
model = AutoModelForMultimodalLM.from_pretrained("apus-ailab/APUS-Reflex-35B-A3B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use apus-ailab/APUS-Reflex-35B-A3B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "apus-ailab/APUS-Reflex-35B-A3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "apus-ailab/APUS-Reflex-35B-A3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/apus-ailab/APUS-Reflex-35B-A3B

SGLang

How to use apus-ailab/APUS-Reflex-35B-A3B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "apus-ailab/APUS-Reflex-35B-A3B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "apus-ailab/APUS-Reflex-35B-A3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "apus-ailab/APUS-Reflex-35B-A3B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "apus-ailab/APUS-Reflex-35B-A3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use apus-ailab/APUS-Reflex-35B-A3B with Docker Model Runner:
```
docker model run hf.co/apus-ailab/APUS-Reflex-35B-A3B
```

APUS-Reflex-35B-A3B / RUNTIME.md

gump2049

docs(brand): unify APUS Reflex naming across model cards and references

f52c2a4 verified 3 days ago

preview code

Raw History Blame Contribute Delete

7.18 kB

Runtime guide and evaluation boundaries

Use 40-layer high as the primary mode of APUS-Reflex-35B-A3B checkpoint-5949. The 20-layer low path is an explicitly selected experimental decision exit. The model is distributed as merged BF16 weights and needs no separate LoRA adapter.

Portable runtime acceptance

Passed: 160/160 decisions identical; maximum logit and candidate-probability differences 0.

The standalone runtime passed all 80 questions at both 20 and 40 layers against the native merged reference. The strict probability-difference gate of 1e-6 passed; all 60 sensitive checkpoint parameters retained their exact serialized precision. The tested process imported no project-local jev, Swift, or PEFT modules. See comparison, Frozen80 results, and text execution smoke. TYPE high returned red shoes; low is not recommended for text generation. GPU validation used the local release artifacts before upload; repository file identities are verified separately.

Download and environment

Download the independent repository root, then install its pinned dependencies:

python -m pip install huggingface_hub
hf download apus-ailab/APUS-Reflex-35B-A3B --local-dir ./APUS-Reflex-35B-A3B
cd APUS-Reflex-35B-A3B
python -m pip install -r requirements.txt
python examples.py . --device cuda:0 --effort high

Pin hf download --revision <commit> for repeatable use. The portable loader reads a local snapshot and its depth_config.json. It checks Transformers 5.16.1, uses BF16 by default, and selects the MoE model class from the supplied configuration. Its source has no project-local Swift/PEFT dependency.

The native merged GPU evaluation used a 96 GB RTX PRO 6000. Budgeting 75 GB or more GPU memory for BF16 is a rough planning estimate, not a measured minimum: long inputs, generation, batch size and the serving stack change memory use. The 20-layer exit still loads the full model; it is not a smaller weight package or a promise of half the memory use.

Decision interface

Run from the downloaded model directory so that the bundled openjet_runtime package is importable:

from openjet_runtime import OpenJet

runtime = OpenJet.from_pretrained(".", device="cuda:0", dtype="bfloat16")
request = {
    "id": "support-731",
    "group_id": "support-731",
    "primitive": "choice",
    "state": "Order 731 was delivered. The customer confirms that the issue is resolved.",
    "instructions": "Select the next support action.",
    "criteria": [
        {"id": "close", "description": "Close the resolved support ticket."},
        {"id": "refund", "description": "Refund an undelivered order."},
    ],
}
result = runtime.decide(request, effort="high")
print(result)

choice accepts 2–16 distinct candidate IDs with nonempty descriptions.
id, group_id, state and instructions must be nonempty strings.
noul and score_level use the runtime's canonical yes/no candidates; score_level is not an aggregate ordinal-score API.
The runtime uses the checkpoint's jev.dynamic.prompt.v2 contract and a no-thinking chat template. It validates single-token candidate labels before scoring.
Inputs longer than the runtime's configured limit (at most 8,192 prompt tokens) are rejected without truncation.
The response maps candidate IDs to normalized candidate probabilities; choice also returns the selected ID.

To examine the experimental exit, explicitly pass effort="low" or run python examples.py . --effort low. There is no confidence-based automatic choice between the exits. Use high for text generation; python examples.py . --effort high --text also exercises the slow TYPE reference path.

The bundled loader uses Qwen3_5MoeForConditionalGeneration and preserves the checkpoint precision of linear_attn.A_log and linear_attn.norm.weight. Casting these parameters to BF16 can change decisions. Use the bundled loader to reproduce the published results; its 60 sensitive parameters are checked against the serialized checkpoint in dtype validation.

Merged-model evaluation

Model and execution path	Depth	Frozen80 correct	Accuracy
Checkpoint-5949 merged BF16, native	40	71 / 80	88.75%
Checkpoint-5949 merged BF16, native experimental exit	20	63 / 80	78.75%

These results are from a completed GPU evaluation of the new merged model. Frozen80 is a reused engineering development panel covering Browser, HelpSteer3, BoolQ, MNLI and attribute decisions. It is not an independent blind benchmark, a calibrated correctness estimate, or a browser end-to-end task-success measurement. The source adapter's larger evaluation panels are not evidence for this merged artifact.

The native adapter-versus-merged comparison recorded maximum absolute candidate-probability changes of approximately 0.14943 at 40 layers and 0.17818 at 20 layers. Both probability-equivalence gates failed. CPU merge arithmetic verification and the merged model's accuracy do not cancel that result. Re-evaluate downstream calibration and decision thresholds on the merged model; do not transfer adapter thresholds on an assumption of numerical equivalence.

Supported scope

The 40-layer model is the primary release path. The 20-layer exit is experimental; source-checkpoint development evidence showed substantial weakness in shallow Browser/TYPE behavior. A shallow-exit decision result does not establish that the exit is suitable for browser text generation.

The reference runtime is single-request and text-only. It rejects multimodal placeholders; image/video inference was not validated for this release. This release does not claim automatic depth routing, calibrated probabilities, production concurrency or latency targets, vLLM/SGLang parity, or end-to-end browser execution. Candidate probabilities describe relative preference among the supplied candidates, not the probability that an action will succeed.

中文说明

正式主模式是 40 层 high；20 层 low 是手动指定的实验性决策出口，仍需加载完整权重。portable 独立验收已通过：160/160决策与native merged一致，logit与概率最大差为0；60个敏感参数逐值匹配checkpoint。

本 merged checkpoint-5949 已完成独立 native GPU Frozen80 评估：40 层 71/80（88.75%），20 层 63/80（78.75%）。这是重复使用的工程开发面板，不能解释为独立盲测、校准后的正确率或浏览器端到端成功率。

原 adapter 与 merged 的候选概率最大绝对差分别约为 0.14943（40 层）和 0.17818（20 层），概率等价门均未通过。需针对 merged 重新验证概率校准和业务阈值，不可直接迁移 adapter 阈值。当前仅验证文本场景；自动路由、多模态、生产并发性能和其他推理引擎的等价性均不在本次验证范围。

Authors: gumpcheng (xDAN2099), zhangxu, APUS AI-LAB.