Hugging Face's logo

vllm-sr
/
Decision-1.0-Nox-4B

Zero-Shot Classification
Safetensors
PyTorch
decision1
multilingual
decision-model
classification
qwen3_5
custom_code
Model card Files
xet
 Community
2
Decision-1.0-Nox-4B / TASKS.md
Xunzhuo's picture
Xunzhuo
Publish qualified Nox Choice semantics and latest Decision comparison
e2f752a verified
preview code
|
Raw History Blame
9.3 kB

All 54 tasks

Accuracy (%) on the same requested rows. Bold marks a Decision-family result strictly above every external open or untuned reference; Jev and the other Decision models do not set that threshold. Ties are not bold. Each task uses its full requested denominator. These task rows are diagnostic; the headline retains its declared within-panel weights.

Decisions · 10 tasks
Task n Lux Nox Kev-9B Kev-4B Qwen3.5-9B Decider Qwen3.5-4B Sol Kev-0.8B Qwen3.5-2B Laya · English Laya · Multilingual Jev
News classification 128 88.28 85.16 87.50 87.50 85.94 86.72 84.38 83.59 86.72 80.47 91.41 89.84 85.16
Boolean constraints 64 95.31 93.75 95.31 71.88 98.44 71.88 62.50 50.00 81.25 37.50 43.75 39.06 100.00
Entity classification 112 96.43 96.43 98.21 98.21 96.43 98.21 97.32 95.54 96.43 92.86 83.93 57.14 96.43
Intent routing 64 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 96.88 81.25 81.25 100.00
Evidence placement 96 61.46 100.00 50.00 41.67 79.17 8.33 72.92 98.96 33.33 9.38 95.83 54.17 30.21
Ordered rubric 64 100.00 89.06 96.88 100.00 84.38 84.38 90.62 65.62 59.38 81.25 18.75 12.50 100.00
Relation composition 96 61.46 52.08 51.04 62.50 36.46 54.17 51.04 37.50 44.79 48.96 25.00 35.42 56.25
Scoped evidence 96 98.96 78.12 65.62 47.92 55.21 47.92 51.04 79.17 10.42 36.46 37.50 36.46 89.58
State tracking 96 33.33 35.42 38.54 34.38 31.25 29.17 28.12 27.08 31.25 25.00 23.96 29.17 33.33
In / out of menu 64 96.88 100.00 84.38 75.00 71.88 59.38 60.94 100.00 57.81 62.50 64.06 37.50 100.00
Composition · 10 tasks
Task n Lux Nox Kev-9B Kev-4B Qwen3.5-9B Decider Qwen3.5-4B Sol Kev-0.8B Qwen3.5-2B Laya · English Laya · Multilingual Jev
Record identity 80 71.25 53.75 45.00 50.00 57.50 56.25 50.00 50.00 50.00 46.25 47.50 50.00 76.25
Capacity assignment 80 71.25 46.25 50.00 50.00 50.00 51.25 50.00 47.50 50.00 50.00 63.75 50.00 68.75
Constraint assignment 80 43.75 41.25 27.50 32.50 25.00 23.75 23.75 26.25 20.00 21.25 22.50 27.50 55.00
Intent routing · EN 120 90.83 91.67 86.67 87.50 88.33 90.00 91.67 90.83 83.33 70.83 74.17 70.83 91.67
Intent routing · ZH 120 86.67 87.50 85.83 86.67 86.67 88.33 86.67 87.50 85.83 74.17 44.17 73.33 88.33
Multiset reconciliation 80 26.25 28.75 37.50 28.75 20.00 23.75 27.50 28.75 25.00 27.50 26.25 32.50 58.75
Ordered service loss 80 20.00 30.00 25.00 20.00 20.00 22.50 21.25 25.00 27.50 20.00 20.00 17.50 42.50
Conflicting rule closure 80 36.25 33.75 27.50 33.75 27.50 26.25 25.00 25.00 27.50 27.50 18.75 20.00 66.25
Temporal exclusion 80 21.25 42.50 28.75 45.00 36.25 42.50 31.25 41.25 21.25 28.75 12.50 28.75 41.25
Transaction recovery 80 51.25 62.50 43.75 51.25 35.00 41.25 26.25 38.75 32.50 23.75 23.75 18.75 75.00
Reading · 3 tasks
Task n Lux Nox Kev-9B Kev-4B Qwen3.5-9B Decider Qwen3.5-4B Sol Kev-0.8B Qwen3.5-2B Laya · English Laya · Multilingual Jev
Yes / no reading 160 90.62 86.25 91.25 88.75 84.38 91.88 83.75 83.12 74.38 65.00 69.38 69.38 92.50
Reading · EN 160 91.25 73.12 83.12 75.62 98.75 93.12 93.12 70.00 63.75 83.75 38.12 36.25 96.88
Reading · ZH 160 88.75 70.62 81.25 74.38 91.88 91.25 91.25 70.00 58.75 81.25 28.75 28.12 96.25
Inference · 4 tasks
Task n Lux Nox Kev-9B Kev-4B Qwen3.5-9B Decider Qwen3.5-4B Sol Kev-0.8B Qwen3.5-2B Laya · English Laya · Multilingual Jev
Contextual reasoning 120 83.33 70.83 68.33 70.83 64.17 66.67 60.83 69.17 42.50 56.67 30.00 20.00 86.67
Answerability 120 93.33 88.33 78.33 81.67 74.17 84.17 81.67 83.33 66.67 82.50 57.50 55.00 90.83
Textual entailment 120 90.00 89.17 90.83 89.17 81.67 90.83 80.00 88.33 77.50 64.17 72.50 72.50 82.50
Scientific inference 120 96.67 96.67 96.67 96.67 98.33 95.83 96.67 95.83 88.33 85.83 95.00 81.67 99.17
Transfer · 27 tasks
Task n Lux Nox Kev-9B Kev-4B Qwen3.5-9B Decider Qwen3.5-4B Sol Kev-0.8B Qwen3.5-2B Laya · English Laya · Multilingual Jev
Buried emotion 20 40.00 25.00 65.00 55.00 40.00 85.00 45.00 20.00 40.00 55.00 60.00 35.00 70.00
Buried paraphrase 20 85.00 75.00 80.00 75.00 80.00 75.00 75.00 70.00 60.00 70.00 55.00 70.00 95.00
Buried entailment 20 90.00 90.00 95.00 90.00 95.00 90.00 75.00 75.00 85.00 65.00 60.00 55.00 95.00
Buried offensive-language detection 20 35.00 45.00 80.00 65.00 45.00 80.00 40.00 35.00 60.00 35.00 70.00 80.00 55.00
Combined policy conditions 32 90.62 75.00 68.75 78.12 53.12 53.12 59.38 78.12 40.62 62.50 62.50 43.75 96.88
Policy exceptions 32 96.88 96.88 87.50 96.88 59.38 56.25 75.00 71.88 65.62 40.62 43.75 37.50 100.00
Policy negation 32 87.50 87.50 90.62 87.50 62.50 62.50 53.12 71.88 71.88 46.88 53.12 50.00 90.62
Authorization contrast 40 100.00 97.50 100.00 100.00 100.00 100.00 97.50 50.00 97.50 62.50 67.50 50.00 100.00
Deadline contrast 40 87.50 70.00 87.50 75.00 77.50 30.00 52.50 40.00 45.00 25.00 35.00 22.50 92.50
Emotion 80 61.25 60.00 67.50 65.00 52.50 86.25 65.00 22.50 60.00 67.50 62.50 56.25 67.50
MMLU 80 77.50 61.25 75.00 68.75 73.75 62.50 66.25 52.50 51.25 53.75 22.50 27.50 88.75
MMLU-Pro 200 53.00 37.00 53.00 45.50 53.50 37.50 45.00 24.50 22.50 28.00 11.00 11.50 84.00
Paraphrase 80 91.25 85.00 81.25 81.25 88.75 77.50 83.75 80.00 58.75 70.00 86.25 75.00 87.50
Question entailment 80 93.75 91.25 95.00 91.25 90.00 87.50 87.50 83.75 81.25 63.75 80.00 73.75 91.25
Science questions 80 100.00 98.75 100.00 100.00 100.00 98.75 98.75 97.50 96.25 96.25 90.00 72.50 100.00
Offensive-language detection 80 73.75 75.00 86.25 85.00 83.75 88.75 77.50 75.00 72.50 83.75 81.25 82.50 76.25
Evidence control · age eligibility 10 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 90.00 50.00 100.00
Evidence control · authorization 10 100.00 100.00 100.00 100.00 100.00 80.00 100.00 50.00 100.00 50.00 70.00 50.00 100.00
Evidence control · deadline 10 80.00 60.00 70.00 60.00 80.00 30.00 50.00 30.00 40.00 30.00 30.00 30.00 100.00
Evidence control · late fee 10 90.00 90.00 100.00 100.00 90.00 70.00 80.00 80.00 90.00 70.00 40.00 40.00 100.00
Evidence control · quantity limit 10 100.00 80.00 100.00 100.00 70.00 80.00 70.00 70.00 100.00 20.00 60.00 30.00 100.00
Evidence control · return window 10 60.00 40.00 70.00 70.00 60.00 60.00 40.00 40.00 80.00 50.00 60.00 40.00 50.00
Evidence control · shipping delay 10 90.00 40.00 90.00 80.00 80.00 80.00 60.00 80.00 100.00 30.00 30.00 30.00 90.00
Evidence control · sla response 10 70.00 40.00 100.00 100.00 50.00 30.00 50.00 40.00 100.00 50.00 50.00 30.00 100.00
Evidence control · spend threshold 10 100.00 100.00 100.00 100.00 100.00 100.00 100.00 90.00 100.00 100.00 50.00 50.00 100.00
Evidence control · volume discount 10 100.00 80.00 100.00 100.00 100.00 90.00 100.00 90.00 100.00 20.00 20.00 30.00 100.00
Evidence control · warranty claim 10 40.00 70.00 80.00 100.00 60.00 40.00 50.00 50.00 50.00 50.00 40.00 30.00 90.00