Update README.md

README.md

@@ -1,199 +1,635 @@
 ---
 library_name: transformers
-tags: []
+pipeline_tag: text-classification
+language:
+  - en
+base_model:
+  - answerdotai/ModernBERT-base
+datasets:
+  - yasserrmd/enterprise-reflex-dataset
+  - yasserrmd/enterprise-reflex-v1-hard-dataset
+tags:
+  - enterprise-ai
+  - agentic-ai
+  - system-1
+  - system-2
+  - action-ranking
+  - action-selection
+  - tool-selection
+  - dynamic-actions
+  - selective-prediction
+  - abstention
+  - no-action
+  - enterprise-agents
+  - modernbert
+  - workflow-routing
+  - decision-model
 ---
 
-# Model Card for Model ID
+# Enterprise Reflex V1
 
-<!-- Provide a quick summary of what the model is/does. -->
+**Model:** `yasserrmd/enterprise-reflex-v1`  
+**Base Model:** `answerdotai/ModernBERT-base`  
+**Language:** English  
+**Architecture:** Dynamic enterprise action scorer  
+**Status:** Research Prototype / Pre-Production Candidate  
 
+Enterprise Reflex V1 is a lightweight enterprise decision model designed to operate as a **System-1 action-ranking layer** in front of larger reasoning models, agent frameworks, and enterprise automation systems.
 
+Given a request, structured enterprise state, optional context, and a dynamic runtime set of candidate actions, the model ranks the available actions, can abstain with `NO_ACTION`, and uses calibrated confidence to decide whether the request should stay in the fast System-1 path or be escalated to a more capable System-2 model or a human.
 
-## Model Details
+Enterprise Reflex is **not a generative LLM** and **not a fixed intent classifier**. Candidate actions are supplied dynamically at inference time, allowing the model to score actions it was not explicitly trained to recognize by identifier alone.
 
-### Model Description
+---
 
-<!-- Provide a longer summary of what this model is. -->
+## Why Enterprise Reflex
 
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+Enterprise agent systems increasingly expose large action spaces: create or update records, approve or reject workflow steps, search internal systems, trigger notifications, route work, invoke APIs, execute business operations, or abstain when no safe action is available.
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+Sending every request directly to a large reasoning model can be unnecessarily expensive and slow. Enterprise Reflex is designed to act as a compact decision layer:
 
-### Model Sources [optional]
+```text
+Request + Enterprise State + Context + Candidate Actions
+                         |
+                         v
+                 Enterprise Reflex
+                         |
+             +-----------+-----------+
+             |                       |
+             v                       v
+     Confident System-1      Low confidence / NO_ACTION
+          decision                    |
+             |                        v
+             v                System-2 / Human Review
+          Execute
+```
 
-<!-- Provide the basic links for the model. -->
+The objective is not to replace larger reasoning models. It is to reduce how often they are required.
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+---
 
-## Uses
+## What Changed in V1
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+V1 extends the original Enterprise Reflex prototype with targeted hard-training data focused on failure modes observed in V0.
 
-### Direct Use
+The V1 hard-training set emphasizes:
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+- same-domain sibling-action discrimination,
+- counterfactual state changes,
+- policy and workflow constraints,
+- semantic cross-domain collisions,
+- improved `NO_ACTION` boundaries,
+- unseen and renamed action identifiers.
 
-[More Information Needed]
+The training strategy retained broad V0 enterprise coverage while adding targeted V1 hard examples so the model could improve difficult routing behavior without losing general enterprise performance.
 
-### Downstream Use [optional]
+### Training Configuration
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+| Item | Value |
+|---|---:|
+| Base model | `answerdotai/ModernBERT-base` |
+| Training samples | **553,632 pairwise examples** |
+| Epochs | **2** |
+| Total training steps | **23,068** |
+| Warmup steps | **1,384** |
+| Maximum sequence length | **384** |
+| Objective | Binary request-action compatibility |
+| Best-model metric | Validation F1 |
+| Reported hardware | NVIDIA A100 |
 
-[More Information Needed]
+---
 
-### Out-of-Scope Use
+## Input Representation
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+Enterprise Reflex scores each candidate action against a compact serialized request representation.
 
-[More Information Needed]
+### Request Side
 
-## Bias, Risks, and Limitations
+```json
+{
+  "request": "Release the approved supplier payment",
+  "domain": "Finance",
+  "state": {
+    "payment_approved": true,
+    "invoice_matched": true
+  },
+  "context": {}
+}
+```
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+### Candidate Action Side
 
-[More Information Needed]
+```json
+{
+  "name": "finance.release_payment",
+  "description": "Release an approved and validated supplier payment.",
+  "family": "EXECUTE",
+  "domain": "Finance"
+}
+```
 
-### Recommendations
+Each candidate is scored independently. Compatibility margins are then calibrated and normalized across the runtime candidate set. `NO_ACTION` is added as an explicit abstention candidate.
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+---
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+## Evaluation
 
-## How to Get Started with the Model
+V1 was evaluated at three levels:
 
-Use the code below to get started with the model.
+1. pairwise request-action classification,
+2. grouped candidate ranking,
+3. a manually designed 100-case hard stress test.
 
-[More Information Needed]
+These evaluations represent different levels of difficulty and should be interpreted separately.
 
-## Training Details
+---
 
-### Training Data
+## Pairwise Test Results
+
+### V0 Test Distribution
+
+| Metric | Result |
+|---|---:|
+| Accuracy | **97.86%** |
+| Precision | **97.97%** |
+| Recall | **83.69%** |
+| F1 | **90.27%** |
+| ROC AUC | **99.03%** |
+| Average Precision | **94.85%** |
+
+### V1 Hard Test Distribution
+
+| Metric | Result |
+|---|---:|
+| Accuracy | **94.07%** |
+| Precision | **85.31%** |
+| Recall | **79.00%** |
+| F1 | **82.04%** |
+| ROC AUC | **96.88%** |
+| Average Precision | **89.24%** |
+
+### Combined Test Distribution
+
+| Metric | Result |
+|---|---:|
+| Accuracy | **97.67%** |
+| Precision | **97.01%** |
+| Recall | **83.36%** |
+| F1 | **89.67%** |
+| ROC AUC | **98.94%** |
+| Average Precision | **94.57%** |
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+---
 
-[More Information Needed]
+## Grouped Action-Ranking Results
 
-### Training Procedure
+Grouped evaluation measures whether the correct action is ranked highest within the full runtime candidate set.
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+### V0 Grouped Test
 
-#### Preprocessing [optional]
+| Metric | Result |
+|---|---:|
+| Groups | **5,948** |
+| Top-1 | **98.30%** |
+| Top-3 | **99.98%** |
+| MRR | **0.9913** |
+| `NO_ACTION` Precision | **94.02%** |
+| `NO_ACTION` Recall | **97.32%** |
+| `NO_ACTION` F1 | **95.64%** |
 
-[More Information Needed]
+### V1 Hard Grouped Test
 
+| Metric | Result |
+|---|---:|
+| Groups | **500** |
+| Top-1 | **88.60%** |
+| Top-3 | **99.40%** |
+| MRR | **0.9370** |
+| `NO_ACTION` Precision | **84.75%** |
+| `NO_ACTION` Recall | **90.09%** |
+| `NO_ACTION` F1 | **87.34%** |
 
-#### Training Hyperparameters
+### Combined Grouped Test
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+| Metric | Result |
+|---|---:|
+| Groups | **6,448** |
+| Top-1 | **97.55%** |
+| Top-3 | **99.94%** |
+| MRR | **0.9871** |
+| `NO_ACTION` Precision | **93.05%** |
+| `NO_ACTION` Recall | **96.58%** |
+| `NO_ACTION` F1 | **94.78%** |
 
-#### Speeds, Sizes, Times [optional]
+The V1 hard grouped split is intentionally more difficult than the broad V0 evaluation and should not be treated as the same distribution.
 
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+---
 
-[More Information Needed]
+## Selective System-1 / System-2 Routing
 
-## Evaluation
+V1 uses calibrated confidence to determine whether a decision should remain in System-1 or be escalated.
+
+For the reported run:
+
+- **Selected System-2 threshold:** `0.87`
+- **Validation System-1 coverage:** **73.71%**
+- **Validation System-1 accuracy:** **99.02%**
+
+`NO_ACTION` is always treated as a System-2 route.
+
+This threshold is calibrated on the validation distribution and should be recalibrated for any materially different deployment domain.
+
+---
+
+## 100-Case Manual Hard Stress Test
+
+A separate manual suite of **100 hard enterprise cases** was used to stress behavior outside the easier validation distribution.
+
+### Overall Results
+
+| Metric | Result |
+|---|---:|
+| Total cases | **100** |
+| Correct | **85** |
+| Incorrect | **15** |
+| Raw Top-1 accuracy | **85.00%** |
+| System-1 handled | **66%** |
+| System-2 routed | **34%** |
+| System-1 accuracy | **87.88%** |
+| Unsafe System-1 failures | **8** |
+
+### Performance by Category
+
+| Category | Tests | Accuracy |
+|---|---:|---:|
+| Cross-domain | 12 | **83.33%** |
+| `NO_ACTION` | 21 | **100.00%** |
+| Policy constraint | 8 | **12.50%** |
+| Sibling action | 32 | **87.50%** |
+| State sensitive | 22 | **90.91%** |
+| Unseen action name | 5 | **100.00%** |
+
+The manual stress test is intentionally adversarial and significantly harder than the standard grouped benchmark.
+
+---
+
+## What V1 Improved
+
+Compared with V0 hard-test behavior, V1 improved both hard-decision accuracy and autonomous coverage.
+
+Observed improvements include:
+
+- hard Top-1 accuracy increased from approximately **80% to 85%**,
+- System-1 coverage increased from approximately **53% to 66%**,
+- state-sensitive decisions improved substantially,
+- unseen action-name generalization remained strong,
+- `NO_ACTION` behavior improved on the manual hard suite,
+- broad V0 enterprise ranking performance remained largely intact.
+
+The result supports the core Enterprise Reflex design: a lightweight model can perform useful dynamic enterprise action ranking while routing uncertain cases to a larger reasoner.
+
+---
+
+## Current Limitation: Policy-Constrained Execution
+
+The dominant V1 weakness is policy-sensitive action validity.
+
+Several hard cases were semantically understood but executed incorrectly because state or policy should have blocked the action.
+
+Observed failure patterns include:
+
+- releasing a payment without required approval,
+- provisioning privileged access without security approval,
+- deleting logs under legal or retention hold,
+- cancelling an order after a workflow state that prohibits cancellation,
+- granting physical access before mandatory induction is complete.
+
+This indicates that V1 is currently stronger at answering:
+
+> Which action best matches this request?
+
+than:
+
+> Is this action actually permitted under the current enterprise state and policy?
+
+For this reason, V1 should not be used as the sole authority for autonomous high-impact enterprise execution.
+
+---
+
+## Intended Use
+
+Enterprise Reflex V1 is suitable for research and controlled enterprise-agent experiments such as:
 
-<!-- This section describes the evaluation protocols and provides the results. -->
+- action ranking,
+- dynamic tool selection,
+- top-k tool narrowing,
+- System-1 / System-2 routing,
+- agent handoff,
+- workflow recommendation,
+- shadow-mode decision analysis,
+- human-in-the-loop action suggestions,
+- enterprise action-space reduction before LLM reasoning.
 
-### Testing Data, Factors & Metrics
+---
 
-#### Testing Data
+## Not Recommended For
 
-<!-- This should link to a Dataset Card if possible. -->
+V1 is not recommended as the sole decision layer for:
 
-[More Information Needed]
+- autonomous financial transactions,
+- privileged-access provisioning,
+- destructive security actions,
+- compliance-sensitive deletion,
+- irreversible workflow actions,
+- legal or regulatory decisions,
+- production execution without deterministic authorization and policy enforcement.
 
-#### Factors
+---
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+## Recommended Production Architecture
+
+Enterprise Reflex should be combined with deterministic controls.
+
+```text
+Request + State + Candidate Actions
+                |
+                v
+       Enterprise Reflex
+                |
+                v
+        Ranked Action
+                |
+                v
+ Policy / Authorization Engine
+         /              \
+      Allowed          Blocked
+        |                 |
+        v                 v
+     Execute        System-2 / Human
+```
+
+The learned model provides decision intelligence. Authorization, policy, entitlement, retention, approval, and other hard enterprise controls should remain deterministic whenever possible.
 
-[More Information Needed]
+---
 
-#### Metrics
+## Example Usage
+
+```python
+import json
+import torch
+import numpy as np
+
+from transformers import (
+    AutoTokenizer,
+    AutoModelForSequenceClassification,
+)
+
+MODEL_ID = "yasserrmd/enterprise-reflex-v1"
+SYSTEM2_THRESHOLD = 0.87
+MAX_LENGTH = 384
+
+# Load the runtime calibration values saved with the model if available.
+# The reported experiment used a calibrated temperature determined from
+# the combined validation distribution.
+TEMPERATURE = 1.0
+
+tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
+model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID)
+
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+model.to(device)
+model.eval()
+
+
+def softmax_np(x):
+    x = np.asarray(x, dtype=np.float64)
+    x = x - np.max(x)
+    e = np.exp(x)
+    return e / e.sum()
+
+
+def request_text(request, domain="enterprise", state=None, context=None):
+    return json.dumps(
+        {
+            "request": request,
+            "domain": domain,
+            "state": state or {},
+            "context": context or {},
+        },
+        ensure_ascii=False,
+        sort_keys=True,
+    )
+
+
+def action_text(action, default_domain="enterprise"):
+    return json.dumps(
+        {
+            "name": action.get("name", ""),
+            "description": action.get("description", ""),
+            "family": action.get("family", "OTHER"),
+            "domain": action.get("domain", default_domain),
+        },
+        ensure_ascii=False,
+        sort_keys=True,
+    )
+
+
+def rank_actions(request, candidate_actions, state=None, context=None, domain="enterprise"):
+    actions = list(candidate_actions)
+
+    actions.append(
+        {
+            "name": "NO_ACTION",
+            "description": "None of the available actions safely or correctly satisfy the request.",
+            "family": "ABSTAIN",
+            "domain": domain,
+        }
+    )
+
+    left = request_text(
+        request,
+        domain=domain,
+        state=state,
+        context=context,
+    )
+
+    enc = tokenizer(
+        [left] * len(actions),
+        [action_text(a, default_domain=domain) for a in actions],
+        padding=True,
+        truncation=True,
+        max_length=MAX_LENGTH,
+        return_tensors="pt",
+    ).to(device)
+
+    with torch.no_grad():
+        logits = model(**enc).logits
+
+    margins = (logits[:, 1] - logits[:, 0]).float().cpu().numpy()
+    probabilities = softmax_np(margins / TEMPERATURE)
+    order = np.argsort(-probabilities)
+
+    ranked = [
+        {
+            "action": actions[int(i)]["name"],
+            "probability": float(probabilities[int(i)]),
+        }
+        for i in order
+    ]
+
+    top = ranked[0]
+
+    system2_required = (
+        top["action"] == "NO_ACTION"
+        or top["probability"] < SYSTEM2_THRESHOLD
+    )
+
+    return {
+        "decision": top["action"],
+        "confidence": top["probability"],
+        "system2_required": system2_required,
+        "ranked_actions": ranked,
+    }
+```
+
+### Example
+
+```python
+result = rank_actions(
+    request="Release the supplier payment",
+    domain="Finance",
+    state={
+        "invoice_matched": True,
+        "payment_approved": True,
+    },
+    candidate_actions=[
+        {
+            "name": "finance.release_payment",
+            "description": "Release an approved and validated supplier payment.",
+            "family": "EXECUTE",
+            "domain": "Finance",
+        },
+        {
+            "name": "finance.create_invoice",
+            "description": "Create an invoice record.",
+            "family": "CREATE",
+            "domain": "Finance",
+        },
+    ],
+)
+
+print(result)
+```
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+---
 
-[More Information Needed]
+## Calibration Note
 
-### Results
+The reported confidence threshold was selected for the reported V1 experiment.
 
-[More Information Needed]
+For a new deployment:
 
-#### Summary
+1. collect domain-specific validation data,
+2. calibrate temperature,
+3. determine an acceptable System-1 error rate,
+4. select the confidence threshold for that environment,
+5. validate policy-sensitive and destructive actions separately.
 
+Do not assume that `0.87` is appropriate for every enterprise domain.
 
+---
 
-## Model Examination [optional]
+## Research Status
 
-<!-- Relevant interpretability work for the model goes here -->
+Enterprise Reflex V1 should currently be considered a:
 
-[More Information Needed]
+**Research Prototype / Pre-Production Candidate**
 
-## Environmental Impact
+The model has demonstrated:
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+- strong broad enterprise action ranking,
+- useful dynamic action selection,
+- effective abstention,
+- high Top-3 retrieval quality,
+- improved state-sensitive behavior,
+- improved System-1 coverage,
+- promising generalization to unseen action identifiers.
 
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+It has not yet demonstrated sufficient reliability for unrestricted autonomous enterprise execution.
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
+The primary V2 research target is **policy-aware action validity and confident wrong-action suppression**.
 
-## Technical Specifications [optional]
+---
 
-### Model Architecture and Objective
+## V2 Direction
 
-[More Information Needed]
+The next iteration should focus less on generic enterprise volume and more on targeted safety and state-validity examples:
 
-### Compute Infrastructure
+- policy counterfactuals,
+- approval-sensitive actions,
+- retention and legal-hold constraints,
+- authorization and entitlement state,
+- workflow-state legality,
+- semantic domain collisions,
+- hard negatives mined from confident V1 failures,
+- valid-action vs `NO_ACTION` boundary cases.
 
-[More Information Needed]
+A likely architectural extension is to separate:
 
-#### Hardware
+1. semantic action suitability,
+2. state/policy validity,
 
-[More Information Needed]
+before producing the final action confidence.
 
-#### Software
+---
 
-[More Information Needed]
+## Datasets
 
-## Citation [optional]
+### Enterprise Reflex Dataset
 
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+`yasserrmd/enterprise-reflex-dataset`
 
-**BibTeX:**
+Broad enterprise action-ranking data used for V0 and retained in V1 training.
 
-[More Information Needed]
+### Enterprise Reflex V1 Hard Dataset
 
-**APA:**
+`yasserrmd/enterprise-reflex-v1-hard-dataset`
 
-[More Information Needed]
+Targeted hard-training examples covering sibling actions, counterfactual state, policy constraints, cross-domain collisions, `NO_ACTION`, and unseen action names.
 
-## Glossary [optional]
+---
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+## Limitations
 
-[More Information Needed]
+- English only in V1.
+- Text and structured state only.
+- No multimodal input.
+- No deterministic policy engine is embedded in the model.
+- Confidence calibration is distribution-dependent.
+- The hard-test suite is manually constructed and relatively small.
+- Policy-sensitive action validity remains the main weakness.
+- The model may still produce high-confidence incorrect actions.
+- Reported metrics should not be interpreted as production-safety guarantees.
 
-## More Information [optional]
+---
+
+## Responsible Use
 
-[More Information Needed]
+Enterprise Reflex is intended to assist enterprise decision routing, not replace enterprise authorization, policy, compliance, or human accountability.
 
-## Model Card Authors [optional]
+High-impact actions should remain subject to deterministic controls and appropriate human or System-2 review.
+
+---
 
-[More Information Needed]
+## Author
+
+**Mohamed Yasser**  
+Hugging Face: `yasserrmd`  
+GitHub: `yasserrmd`
+
+---
 
-## Model Card Contact
+## Version
 
-[More Information Needed]
+**Enterprise Reflex V1**  
+Research Prototype / Pre-Production Candidate