Single-file ONNX (fixes from_pretrained), better model card

README.md

@@ -5,6 +5,7 @@ language:
 library_name: onnx
 pipeline_tag: zero-shot-classification
 base_model: jhu-clsp/ettin-encoder-150m
+base_model_relation: finetune
 tags:
 - intent-classification
 - intent-detection
@@ -22,6 +23,19 @@ tags:
 - ettin
 - laya
 - distillation
+- knowledge-distillation
+- onnxruntime
+- zero-shot
+- nlu
+- intent-router
+- semantic-router
+- llm-router
+- open-intent-detection
+- out-of-distribution-detection
+- customer-service
+- banking
+- e-commerce
+- edge
 datasets:
 - clinc_oos
 - PolyAI/banking77
@@ -48,13 +62,21 @@ model-index:
     - type: recall
       value: 0.986
       name: Out-of-scope recall
+    - type: accuracy
+      value: 0.943
+      name: Routing accuracy, menus of 20 to 45 intents
+    - type: accuracy
+      value: 0.92
+      name: Routing accuracy, 6 unseen domains
 ---
 
-# Laya Intent Router 150M (ONNX, int8)
+# Laya Intent Router 150M: zero-shot intent classification on CPU (ONNX, int8)
+
+**A 150M-parameter zero-shot intent detection model with out-of-scope detection, 160 ms p95 on 4 CPU threads.** You give it a user message and a list of intents written in plain English. It tells you which intent the message belongs to, or that it belongs to none of them.
 
-**Zero-shot intent classification that runs on a CPU in about 60 ms.** You give it a message and a list of intents written in plain English. It tells you which intent the message belongs to, or that it belongs to none of them.
+No training. No labelled data. You write the intents when you call it, and you can change them on every request. It's a small ONNX file you run with `onnxruntime` and `numpy`, no GPU and no torch.
 
-No training. No labelled data. You write the intents when you call it, and you can change them on every request.
+On a held-out set of 1,636 messages it gets **94.9% routing accuracy and 98.6% out-of-scope recall**, up from 72.8% and 68.2% for the original [Laya](https://huggingface.co/convaiinnovations/laya).
 
 ```
 "i want to cancel my order 88213"         -> cancel_order     0.97
@@ -66,6 +88,15 @@ No training. No labelled data. You write the intents when you call it, and you c
 
 That last line is why I built this. The original Laya sent `qwewqeqw` to `order_not_received` with 0.73 confidence. This one puts 0.98 on none.
 
+## Use it for
+
+- **Chatbot and voicebot intent routing**, where the menu of intents changes per flow or per customer.
+- **Out-of-scope detection**: knowing when a message fits none of your intents, so you can fall back to a human, an LLM or an "I didn't get that".
+- **A cheap semantic router in front of an LLM**, when you want a decision in 150 ms on CPU instead of a full LLM call.
+- **Customer support triage** for e-commerce, banking, insurance, telecom and SaaS.
+
+It's built for short English customer messages. It isn't a general text classifier for long documents, and it only speaks English.
+
 ## Try it
 
 ```bash
@@ -113,7 +144,7 @@ A SaaS support bot, 4 intents, nothing fine-tuned:
 
 The intents were `reset_password: "User can't log in or forgot their password"`, `billing: "Questions about invoices, charges or refunds"`, `bug_report: "Something in the app is broken or showing an error"` and `talk_to_human: "User wants to speak to a real person"`. That's the whole setup.
 
-## How good is it
+## Benchmarks: 94.9% routing accuracy, 98.6% out-of-scope recall
 
 I tested it on 1,636 hand-written messages across 10 routing setups: e-commerce, retail banking, insurance and telecom, plus an adversarial set of near-duplicates, typos, slang and "don't cancel, just tell me where it is" style traps. **Insurance and telecom were never seen in training.**
 
@@ -132,7 +163,7 @@ So you get the big fine-tuned model's accuracy at a third of its latency and hal
 
 It also holds up when I poke at it. A perturbation test (shuffled intent order, removed correct intent, distractor intents, rewritten messages, gibberish) scores 0.92 averaged over 3 seeds, against 0.73 for the original Laya.
 
-## Big intent lists
+## Big intent lists (up to 148 intents)
 
 The model reads everything in one 512-token window, so it can only see so many intents at once. For lists longer than 4, a small embedder (`bge-small-en-v1.5`, bundled in `shortlist/`) picks the 4 closest intents first and the router decides between those and "none".
 
@@ -170,7 +201,7 @@ Everything trained locally on a 32 GB M2 Pro. Training code: [github.com/vrajnot
 
 | file | what |
 |---|---|
-| `laya.onnx`, `laya.onnx.data` | the router (int8 weights) |
+| `laya.onnx` | the router, one file with int8 weights (302 MB) |
 | `tokenizer.json`, `rl_agent_config.json` | tokenizer, prompt format, calibrated temperatures, threshold |
 | `shortlist/` | bge-small-en-v1.5 ONNX embedder for long intent lists |
 | `router.py` | the whole inference code, one file, no torch |

laya.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:53e526532f19fff929506f3bffbd4faa0f6fdb684fa5842a2376e973e4091808
-size 2467583
+oid sha256:f4e29e431074a059977293de52408c4e18ce7f6e38d6b7b77e7e3e9406d67f34
+size 302276803

laya.onnx.data

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:17cfafb27b977cd0537a61da24edecad8d8f50eac2d0eb9f3e14e5bc3116cf9c
-size 299825152