| --- |
| license: apache-2.0 |
| library_name: transformers |
| tags: |
| - laya |
| - typed-decisions |
| - structured-decisions |
| metrics: |
| - accuracy |
| - brier_score |
| --- |
| # Laya fine-tuned on Typed Decisions |
|
|
| This derivative checkpoint fine-tunes the open-source Laya base model by Convai Innovations for typed decision-making sample sets. |
|
|
| - Base model: https://huggingface.co/convaiinnovations/laya |
| - Source code: https://github.com/NandhaKishorM/laya |
| - Source repository owner: NandhaKishorM |
| - Upstream package author: Convai Innovations |
| - License: Apache-2.0 |
|
|
| Fine-tuned on 1,200 training cases (6,000 decisions) from LocalLLaMA/typed-decisions using two NVIDIA T4 GPUs for four epochs. |
|
|
| ## Official held-out evaluation |
|
|
| 400 cases / 2,000 decisions. |
|
|
| | Metric | Result | |
| |---|---:| |
| | Accuracy | 0.769 | |
| | Soft accuracy | 0.5068 | |
| | Brier score | 0.0694 | |
| | ECE | 0.2150 | |
| | Score MAE | 0.2442 | |
| | Within one level | 0.9912 | |
| | p50 latency | 116.1 ms/case | |
| | p95 latency | 153.8 ms/case | |
|
|
| Accuracy 0.769 vs TypeSafe Jev 1.13.0 baseline 0.727 and teacher self-agreement 0.735. |
|
|
| Per workflow: agent trace 0.746, customer service 0.776, invoice processing 0.806, security incidents 0.748. |
|
|
| ```python |
| import laya |
| agent = laya.Agent("Ankit1106/laya-typed-decisions") |
| result = agent.predict(state, questions) |
| ``` |
|
|
| Full metrics are in `laya_benchmark_report.json`. This is a fine-tuned derivative checkpoint, not an ownership claim over Laya. |
|
|
