File size: 1,413 Bytes
66172d2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | ---
license: apache-2.0
library_name: transformers
tags:
- laya
- typed-decisions
- structured-decisions
metrics:
- accuracy
- brier_score
---
# Laya fine-tuned on Typed Decisions
This derivative checkpoint fine-tunes the open-source Laya base model by Convai Innovations for typed decision-making sample sets.
- Base model: https://huggingface.co/convaiinnovations/laya
- Source code: https://github.com/NandhaKishorM/laya
- Source repository owner: NandhaKishorM
- Upstream package author: Convai Innovations
- License: Apache-2.0
Fine-tuned on 1,200 training cases (6,000 decisions) from LocalLLaMA/typed-decisions using two NVIDIA T4 GPUs for four epochs.
## Official held-out evaluation
400 cases / 2,000 decisions.
| Metric | Result |
|---|---:|
| Accuracy | 0.769 |
| Soft accuracy | 0.5068 |
| Brier score | 0.0694 |
| ECE | 0.2150 |
| Score MAE | 0.2442 |
| Within one level | 0.9912 |
| p50 latency | 116.1 ms/case |
| p95 latency | 153.8 ms/case |
Accuracy 0.769 vs TypeSafe Jev 1.13.0 baseline 0.727 and teacher self-agreement 0.735.
Per workflow: agent trace 0.746, customer service 0.776, invoice processing 0.806, security incidents 0.748.
```python
import laya
agent = laya.Agent("Ankit1106/laya-typed-decisions")
result = agent.predict(state, questions)
```
Full metrics are in `laya_benchmark_report.json`. This is a fine-tuned derivative checkpoint, not an ownership claim over Laya.
|