---
library_name: qarar
tags:
  - arabic
  - decision
  - classification
  - calibrated
  - typed-decisions
  - choice
  - noul
  - score
license: apache-2.0
language:
  - ar
  - en
base_model: silma-ai/silma-embedding-matryoshka-v0.1
---

# QARAR

An **Arabic-first typed-decision** model. Given one state (an Arabic message, record
or document) and a typed question, QARAR returns a **calibrated probability
distribution over the options supplied with the question** — it never generates text.

Three question types are supported:

| type | meaning | options |
|---|---|---|
| `choice` | pick one option | your named options |
| `score` | place the state on an ordered scale | your ordered levels |
| `noul` | yes/no question about the state | `false`, `true` |

The model also returns a **confidence** and an **abstention flag**, so a caller can
trade coverage for precision.

## Architecture

QARAR is a single self-contained model: a multilingual encoder **and** a lightweight
typed-decision head, released together as one architecture (`model.safetensors`).

- Encoder: `silma-ai/silma-embedding-matryoshka-v0.1` (768-d, 12 layers).
- Decision head: projects the state, question and candidate definitions; four
  decision blocks attend from the question to the state; candidate/level definitions
  are encoded by the same encoder and scored against the question.
- The state is encoded **once**; questions are independent (adding or reordering
  questions cannot change an answer).
- Calibration: per-`(type × option-count)` temperature.
- Abstention: per-type threshold.

## Install

```bash
pip install qarar
```

## Usage

```python
from qarar import load            # published alias; `from nomeda import load` also works
m = load("nomeda-lab/qarar")

out = m.predict(
    state="أنا اتخصم مني مرتين على فاتورة مارس، عايز أرجّع الفلوس النهاردة.",
    questions={
        "intent": {
            "type": "choice",
            "instructions": "ماذا يريد العميل؟",
            "criteria": {
                "refund": "يريد استرداد المال",
                "cancel": "يريد إلغاء الطلب",
                "info":   "يطلب معلومات",
            },
        },
        "needs_human": {
            "type": "noul",
            "instructions": "هل يحتاج الأمر تدخل موظف؟",
            "criteria": {"true": "يحتاج تدخلًا بشريًا", "false": "يمكن التعامل آليًا"},
        },
    },
)
print(out["answers"]["intent"]["choice"], out["answers"]["intent"]["answer_confidence"])
```

## Evaluation

Measured on the public **Arabic Decision Benchmark**
(`nomeda-lab/arabic-decision-benchmark`, 5,635 questions / 22 tasks), against two
external typed-decision systems.

| slice | n | **QARAR** | JEV-1.13 | Laya |
|---|---:|---:|---:|---:|
| Native-Arabic tasks | 3,474 | **0.724 / 0.671** | 0.696 / 0.591 | 0.454 / 0.386 |
| Workflow tasks | 2,161 | 0.626 / 0.449 | **0.875 / 0.747** | 0.426 / 0.230 |
| **Whole benchmark** | **5,635** | 0.687 / 0.560 | **0.765 / 0.669** | 0.443 / 0.308 |

(accuracy / macro-F1)

| dimension | leader | values |
|---|---|---|
| Calibration (ECE, lower better) | **QARAR** | **0.035** · 0.072 · 0.337 |
| Selective precision @ 20% coverage | **QARAR** | **0.955** · 0.898 · 0.570 |
| Latency (CPU-matched) | **QARAR** | **13.9** · 10.0 q/s (QARAR · Laya) |

QARAR is the **best-calibrated** system and gives the **highest selective precision at
low coverage** (monotone risk–coverage); it leads on native Arabic and is
self-hosted. JEV leads on the workflow tasks and on whole-benchmark accuracy.

### Per-task accuracy

| task | n | **QARAR** | JEV-1.13 | Laya |
|---|---:|---:|---:|---:|
| massive_ar / scenario | 484 | **0.870** | 0.758 | 0.498 |
| egy hate speech / hate | 326 | **0.862** | 0.828 | 0.537 |
| arbanking77 / intent | 337 | **0.816** | 0.807 | 0.457 |
| massive_ar / intent | 410 | 0.780 | **0.829** | 0.571 |
| egy fake reviews / sentiment | 279 | **0.746** | 0.674 | 0.355 |
| egy fake reviews / rating | 270 | 0.685 | **0.722** | 0.156 |
| arsarcasm / sentiment | 169 | **0.686** | 0.639 | 0.331 |
| egy fake reviews / authenticity | 470 | **0.670** | 0.491 | 0.487 |
| arsarcasm / sarcasm | 250 | 0.628 | **0.672** | 0.556 |
| shein reviews / rating | 76 | **0.539** | 0.263 | 0.145 |
| stance | 403 | 0.489 | **0.643** | 0.489 |
| openjev / mailroom | 233 | 0.923 | **0.991** | 0.236 |
| openjev / sponsor segment | 225 | 0.280 | **0.964** | 0.267 |
| openjev / phone extraction | 57 | 0.825 | **0.965** | 0.789 |
| openjev / email selection | 165 | 0.897 | **0.945** | 0.552 |
| openjev / silent failure | 240 | 0.504 | **0.921** | 0.479 |
| openjev / entity alignment | 227 | 0.687 | **0.872** | 0.643 |
| openjev / ir decision | 233 | 0.597 | **0.850** | 0.189 |
| openjev / amount extraction | 164 | 0.817 | **0.848** | 0.598 |
| openjev / citation control | 216 | 0.329 | **0.792** | 0.375 |
| openjev / browser drone | 181 | 0.580 | **0.768** | 0.475 |
| openjev / context retention | 220 | 0.700 | **0.750** | 0.450 |

## Limitations

- **Closed-label.** QARAR selects among the options you supply; it degrades on tasks
  and labels outside its training coverage. It is not a zero-shot label-invention
  system.
- Ordinal `score` decisions are the weakest type.
- It does not generate text and does not execute tools.

## License

Apache-2.0. Encoder: `silma-ai/silma-embedding-matryoshka-v0.1` (Apache-2.0).
See `LICENSES.yaml`.