README.md

---
license: apache-2.0
base_model: Qwen/Qwen3.5-4B
base_model_relation: adapter
language:
- zh
- en
tags:
- qwen3.5
- peft
- lora
- qlora
- bitsandbytes
- decision-model
- jev
- structured-decisions
- prefill-only
---

# Qwen3.5-4B-Jev

**A locally fine-tuned, prefill-only decision model: state + typed questions → Choice, Score, and Noul.**

This release contains the **LoRA adapter, FP32 scalar decision head, processor, calibrated temperatures,
and standalone inference code**. It loads the pinned official Qwen3.5-4B base in **NF4 4-bit** with
double quantization and BF16 computation. The base weights are downloaded separately.

**[jev-benchmark](https://github.com/haxudev/jev-benchmark): 99/100 reference agreement,
10/10 context-contrast pairs passed.**
See the [evaluation report](https://github.com/haxudev/jev-benchmark/blob/main/results/qwen35-4b-report.md)
and [raw results](evaluation/jev-benchmark.json). Results reflect this specific 100-item Chinese
implicit-intent diagnostic, not general-purpose accuracy.

This is an independent Jev-like model, not an official TypeSafe model or a reproduction of its weights.

## How it works

```text
state + instructions + complete criteria + candidate
  → Qwen3.5-4B (frozen NF4 base + language LoRA)
  → last valid hidden state → FP32 scalar head
  → per-question softmax / calibrated temperature
  → typed response assembled in code
```

- **Choice:** candidate distribution and argmax.
- **Score:** ordered-level distribution and expected level, starting at 0.
- **Noul:** probability of the true interpretation from false/true candidate scoring.
- **Confidence:** `1 − H(p)/log(K)`, a concentration statistic, not a calibrated probability of correctness.
- No response-token decoding, generated JSON, or self-reported numeric probabilities.

The complete criteria are included in each candidate input. This differs from
[LLM2Jev](https://github.com/Yinsongxu/LLM2Jev)'s independent yes/no scoring and normalization.
The typed API design follows [TypeSafe's primitives](https://docs.typesafe.ai/introduction).

## Quick start

Validated on **Linux aarch64 / NVIDIA GB10 / CUDA 13 / Python 3.12**.
Use a CUDA-enabled PyTorch build compatible with your platform, then install `requirements.txt`.
The pinned inference stack is Transformers 5.15.0, PEFT 0.20.0, bitsandbytes 0.50.2 and FLA 0.5.2.
Other GPU/platform combinations have not been validated for this release.

```bash
hf download xuhaodev/Qwen3.5-4B-Jev --local-dir Qwen3.5-4B-Jev
python -m pip install -r Qwen3.5-4B-Jev/requirements.txt
```

```python
import sys
sys.path.insert(0, "Qwen3.5-4B-Jev")
from qwen_jev import JevModel

model = JevModel.from_pretrained("Qwen3.5-4B-Jev")
# Optional: base_path="/path/to/the/pinned/Qwen3.5-4B"

result = model.predict(
    state="订单已经付款。仓库明确记录：尚未发货。",
    questions={
        "status": {
            "type": "choice",
            "instructions": "订单的发货状态是什么？",
            "criteria": {"pending": "尚未发货", "shipped": "已经发货"},
        },
        "paid": {"type": "noul", "instructions": "订单是否已经付款？"},
        "progress": {
            "type": "score",
            "instructions": "按订单进度判级。",
            "criteria": ["未付款", "已付款未发货", "已发货"],
        },
    },
)
print(result)
```

Use this loader rather than a generic `AutoModelForCausalLM` pipeline. The adapter is attached to the
multimodal backbone and needs the scalar head and matching template to reproduce the evaluated model.
No `trust_remote_code` auto-execution is required: the downloaded Python modules are explicitly imported.
For reproducibility, pass a fixed Hub `revision` when downloading.

The API model identifier remains **`qwen35-4b-jev-v1`**; the public repository name is **Qwen3.5-4B-Jev**.
String and JSON text states are supported. This release validates **text only**.

## Training

| Setting | Value |
|---|---|
| Base | [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) |
| Base revision | `851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a` |
| Quantization | NF4 4-bit, double quantization, BF16 compute |
| Adapter | language LoRA rank 16, alpha 32, dropout 0.05 |
| Trainable parameters | 32,467,456 |
| Training questions | 6,000: 2,449 semantic references + 3,551 program-verifiable rules |
| Dev / calibration / internal test | 600 / 600 / 600 questions, grouped source separation |
| Training | 2 epochs, 750 optimizer updates, accumulation 16 questions |
| Learning rate | LoRA 1e-5, scalar head 5e-5 |
| Objective | candidate CE; Score ordinal loss; verified equivalent-view consistency |
| Selected checkpoint | epoch 2, Dev family × primitive macro NLL |

Semantic data were recovered from historical reviewed synthetic training records produced using
GPT-6 Luna, Grok 4.7, and GPT-6 Sol. Labels are **teacher references**, not new human annotations.
Program-rule labels are computed from generated facts, with balanced family/label sampling.
Source groups are weighted inversely to their question count. Related views stay within the same split.
Raw training records are not distributed in this model repository.

The public benchmark was excluded from training, checkpoint selection and calibration. The release
was hash-locked before a single 100-request final benchmark run; no weights or calibration were adjusted
after seeing that result. A state-only exact-hash exclusion check and cross-split semantic near-duplicate
audit were performed. Public benchmark exposure during base pretraining cannot be ruled out.

## Evaluation

Benchmark: [haxudev/jev-benchmark](https://github.com/haxudev/jev-benchmark), v1.0.0,
commit [`d6308af`](https://github.com/haxudev/jev-benchmark/tree/d6308af55b0331f56558203606ec8f1057ca61a6).
Run completed 2026-10-01 21:40 UTC / 2026-10-02 local time.

| Metric | Result |
|---|---:|
| Overall reference agreement | **99/100 (99%)** |
| Marriage subset | **50/50** |
| Girlfriend-hint subset | **49/50** |
| Same-final-utterance context pairs (both correct) | **10/10** |
| Mean / P50 / P95 full-decision latency | **635 / 634 / 650 ms** |

Latency is from one sequential, loaded-model, loopback HTTP run on GB10, including SDK overhead;
P95 uses linear interpolation. It is not a concurrent-throughput or cold-start measurement.
Only `love-073` differs from the reference: predicted B (0.51463), reference D (0.47288).

The independently held-out **internal** test achieved 99.0% agreement over 600 questions:
97.93% on 290 semantic teacher references and 100% on 310 program-rule items.
Overall calibrated NLL=0.02248 and Brier=0.01135. Program families share templates across splits;
these numbers do not establish unseen-template or broad task generalization.

Temperatures were fitted on the separate design-distribution calibration set:
Choice=1.91865, Noul=2.37021, Score=2.02511. Calibration improved internal NLL/Brier but did not improve
every metric (ECE increased from 0.00673 to 0.00963).

## Scope and limits

- Per-candidate input budget: **4,096 tokens**; per-question expanded budget: **32,768 tokens**.
- Training main inputs were short (maximum candidate length 671 tokens). The 4K limit is an execution
  budget, not comprehensive long-context quality validation.
- Nine ~7,800-token controlled padded probes passed; this does not establish general 8K capability.
- Vision structure is retained, but image decision quality is not validated in this text-first release.
- Normalized-entropy confidence is not official Jev confidence equivalence or a correctness guarantee.
- The 100-item benchmark is synthetic, single-author and domain-specific; 99% is reference agreement.
- Original adapter/head/calibration hashes are recorded in `provenance.json`; public packaging changes
  only portable configuration paths and inference imports.

## License and references

Model adapter, decision head and included inference code: **Apache-2.0**.
Base model: Qwen3.5-4B, Apache-2.0. The benchmark dataset is CC BY 4.0, authored by haxudev;
evaluation outputs and links are included with attribution, without redistributing the full dataset.

- [Model repository](https://huggingface.co/xuhaodev/Qwen3.5-4B-Jev)
- [Benchmark and report](https://github.com/haxudev/jev-benchmark)
- [TypeSafe introduction](https://docs.typesafe.ai/introduction)
- [LLM2Jev](https://github.com/Yinsongxu/LLM2Jev)
- [LocalJev](https://github.com/githubnext/localjev)