Hugging Face's logo

vllm-sr
/
Decision-1.0-Nox-4B

Zero-Shot Classification
Safetensors
PyTorch
decision1
multilingual
decision-model
classification
qwen3_5
custom_code
Model card Files
xet
 Community
2
Decision-1.0-Nox-4B / TASKS.md
Xunzhuo's picture
Xunzhuo
Update public comparison roster; preserve model weights and benchmark scores
46505c7 verified
preview code
|
Raw History Blame
9.45 kB

All 54 tasks

Accuracy (%) on the same requested rows. Bold marks a Decision-family result strictly above every external open or untuned reference; Jev and the other Decision models do not set that threshold. Ties are not bold. Each task uses its full requested denominator. These task rows are diagnostic; the headline retains its declared within-panel weights.

Decisions · 10 tasks
Task n Jev Lux Nox Kev-9B Kev-4B Qwen3.5-9B Decider Qwen3.5-4B Sol Kev-0.8B Qwen3.5-2B Laya · English Laya · Multilingual
News classification 128 85.16 88.28 85.16 87.50 87.50 85.94 86.72 84.38 83.59 86.72 80.47 91.41 89.84
Boolean constraints 64 100.00 95.31 93.75 95.31 71.88 98.44 71.88 62.50 50.00 81.25 37.50 43.75 39.06
Entity classification 112 96.43 96.43 96.43 98.21 98.21 96.43 98.21 97.32 95.54 96.43 92.86 83.93 57.14
Intent routing 64 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 96.88 81.25 81.25
Evidence placement 96 30.21 61.46 100.00 50.00 41.67 79.17 8.33 72.92 98.96 33.33 9.38 95.83 54.17
Ordered rubric 64 100.00 100.00 89.06 96.88 100.00 84.38 84.38 90.62 65.62 59.38 81.25 18.75 12.50
Relation composition 96 56.25 61.46 52.08 51.04 62.50 36.46 54.17 51.04 37.50 44.79 48.96 25.00 35.42
Scoped evidence 96 89.58 98.96 78.12 65.62 47.92 55.21 47.92 51.04 79.17 10.42 36.46 37.50 36.46
State tracking 96 33.33 33.33 35.42 38.54 34.38 31.25 29.17 28.12 27.08 31.25 25.00 23.96 29.17
In / out of menu 64 100.00 96.88 100.00 84.38 75.00 71.88 59.38 60.94 100.00 57.81 62.50 64.06 37.50
Composition · 10 tasks
Task n Jev Lux Nox Kev-9B Kev-4B Qwen3.5-9B Decider Qwen3.5-4B Sol Kev-0.8B Qwen3.5-2B Laya · English Laya · Multilingual
Record identity 80 76.25 71.25 53.75 45.00 50.00 57.50 56.25 50.00 50.00 50.00 46.25 47.50 50.00
Capacity assignment 80 68.75 71.25 46.25 50.00 50.00 50.00 51.25 50.00 47.50 50.00 50.00 63.75 50.00
Constraint assignment 80 55.00 43.75 41.25 27.50 32.50 25.00 23.75 23.75 26.25 20.00 21.25 22.50 27.50
Intent routing · EN 120 91.67 90.83 91.67 86.67 87.50 88.33 90.00 91.67 90.83 83.33 70.83 74.17 70.83
Intent routing · ZH 120 88.33 86.67 87.50 85.83 86.67 86.67 88.33 86.67 87.50 85.83 74.17 44.17 73.33
Multiset reconciliation 80 58.75 26.25 28.75 37.50 28.75 20.00 23.75 27.50 28.75 25.00 27.50 26.25 32.50
Ordered service loss 80 42.50 20.00 30.00 25.00 20.00 20.00 22.50 21.25 25.00 27.50 20.00 20.00 17.50
Conflicting rule closure 80 66.25 36.25 33.75 27.50 33.75 27.50 26.25 25.00 25.00 27.50 27.50 18.75 20.00
Temporal exclusion 80 41.25 21.25 42.50 28.75 45.00 36.25 42.50 31.25 41.25 21.25 28.75 12.50 28.75
Transaction recovery 80 75.00 51.25 62.50 43.75 51.25 35.00 41.25 26.25 38.75 32.50 23.75 23.75 18.75
Reading · 3 tasks
Task n Jev Lux Nox Kev-9B Kev-4B Qwen3.5-9B Decider Qwen3.5-4B Sol Kev-0.8B Qwen3.5-2B Laya · English Laya · Multilingual
Yes / no reading 160 92.50 90.62 86.25 91.25 88.75 84.38 91.88 83.75 83.12 74.38 65.00 69.38 69.38
Reading · EN 160 96.88 91.25 73.12 83.12 75.62 98.75 93.12 93.12 70.00 63.75 83.75 38.12 36.25
Reading · ZH 160 96.25 88.75 70.62 81.25 74.38 91.88 91.25 91.25 70.00 58.75 81.25 28.75 28.12
Inference · 4 tasks
Task n Jev Lux Nox Kev-9B Kev-4B Qwen3.5-9B Decider Qwen3.5-4B Sol Kev-0.8B Qwen3.5-2B Laya · English Laya · Multilingual
Contextual reasoning 120 86.67 83.33 70.83 68.33 70.83 64.17 66.67 60.83 69.17 42.50 56.67 30.00 20.00
Answerability 120 90.83 93.33 88.33 78.33 81.67 74.17 84.17 81.67 83.33 66.67 82.50 57.50 55.00
Textual entailment 120 82.50 90.00 89.17 90.83 89.17 81.67 90.83 80.00 88.33 77.50 64.17 72.50 72.50
Scientific inference 120 99.17 96.67 96.67 96.67 96.67 98.33 95.83 96.67 95.83 88.33 85.83 95.00 81.67
Transfer · 27 tasks
Task n Jev Lux Nox Kev-9B Kev-4B Qwen3.5-9B Decider Qwen3.5-4B Sol Kev-0.8B Qwen3.5-2B Laya · English Laya · Multilingual
Buried emotion 20 70.00 40.00 20.00 65.00 55.00 40.00 85.00 45.00 20.00 40.00 55.00 60.00 35.00
Buried paraphrase 20 95.00 85.00 75.00 80.00 75.00 80.00 75.00 75.00 70.00 60.00 70.00 55.00 70.00
Buried entailment 20 95.00 90.00 90.00 95.00 90.00 95.00 90.00 75.00 75.00 85.00 65.00 60.00 55.00
Buried offensive-language detection 20 55.00 35.00 45.00 80.00 65.00 45.00 80.00 40.00 35.00 60.00 35.00 70.00 80.00
Combined policy conditions 32 96.88 90.62 75.00 68.75 78.12 53.12 53.12 59.38 78.12 40.62 62.50 62.50 43.75
Policy exceptions 32 100.00 96.88 96.88 87.50 96.88 59.38 56.25 75.00 71.88 65.62 40.62 43.75 37.50
Policy negation 32 90.62 87.50 87.50 90.62 87.50 62.50 62.50 53.12 71.88 71.88 46.88 53.12 50.00
Authorization contrast 40 100.00 100.00 97.50 100.00 100.00 100.00 100.00 97.50 50.00 97.50 62.50 67.50 50.00
Deadline contrast 40 92.50 87.50 70.00 87.50 75.00 77.50 30.00 52.50 40.00 45.00 25.00 35.00 22.50
Emotion 80 67.50 61.25 40.00 67.50 65.00 52.50 86.25 65.00 22.50 60.00 67.50 62.50 56.25
MMLU 80 88.75 77.50 61.25 75.00 68.75 73.75 62.50 66.25 52.50 51.25 53.75 22.50 27.50
MMLU-Pro 200 84.00 53.00 37.00 53.00 45.50 53.50 37.50 45.00 24.50 22.50 28.00 11.00 11.50
Paraphrase 80 87.50 91.25 85.00 81.25 81.25 88.75 77.50 83.75 80.00 58.75 70.00 86.25 75.00
Question entailment 80 91.25 93.75 91.25 95.00 91.25 90.00 87.50 87.50 83.75 81.25 63.75 80.00 73.75
Science questions 80 100.00 100.00 98.75 100.00 100.00 100.00 98.75 98.75 97.50 96.25 96.25 90.00 72.50
Offensive-language detection 80 76.25 73.75 75.00 86.25 85.00 83.75 88.75 77.50 75.00 72.50 83.75 81.25 82.50
Evidence control · age eligibility 10 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 90.00 50.00
Evidence control · authorization 10 100.00 100.00 100.00 100.00 100.00 100.00 80.00 100.00 50.00 100.00 50.00 70.00 50.00
Evidence control · deadline 10 100.00 80.00 60.00 70.00 60.00 80.00 30.00 50.00 30.00 40.00 30.00 30.00 30.00
Evidence control · late fee 10 100.00 90.00 90.00 100.00 100.00 90.00 70.00 80.00 80.00 90.00 70.00 40.00 40.00
Evidence control · quantity limit 10 100.00 100.00 80.00 100.00 100.00 70.00 80.00 70.00 70.00 100.00 20.00 60.00 30.00
Evidence control · return window 10 50.00 60.00 40.00 70.00 70.00 60.00 60.00 40.00 40.00 80.00 50.00 60.00 40.00
Evidence control · shipping delay 10 90.00 90.00 40.00 90.00 80.00 80.00 80.00 60.00 80.00 100.00 30.00 30.00 30.00
Evidence control · sla response 10 100.00 70.00 40.00 100.00 100.00 50.00 30.00 50.00 40.00 100.00 50.00 50.00 30.00
Evidence control · spend threshold 10 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 90.00 100.00 100.00 50.00 50.00
Evidence control · volume discount 10 100.00 100.00 80.00 100.00 100.00 100.00 90.00 100.00 90.00 100.00 20.00 20.00 30.00
Evidence control · warranty claim 10 90.00 40.00 70.00 80.00 100.00 60.00 40.00 50.00 50.00 50.00 50.00 40.00 30.00