blue-machines/Canine-S-sentence-intent-v1

CANINE-S Sentence Intent v1

blue-machines/Canine-S-sentence-intent-v1 is a sentence-level intent classifier.

• Base model: google/canine-s (~133M)
• Task: 7-class utterance intent
• Runtime: TorchScript INT8 for fast CPU — ONNX Runtime is not supported for CANINE
• Input format: a single bare sentence / utterance (no dialogue tags like [assistant] / [user])
• Output: intent class label + confidence (softmax max probability)

Intent labels

id	label
0	provide_info
1	affirm
2	deny
3	correction
4	question
5	clarify_request
6	unclear

Files

File	Role
model.pt	Deploy (fast CPU) — TorchScript dynamic INT8
model_int8.pt	Same INT8 TorchScript (explicit name)
model_fp32.pt	TorchScript FP32 reference
model.safetensors	Original PyTorch weights
cpu_bench.json	Local CPU latency numbers
label_map.json	Label id ↔ name map
tokenizer_config.json	CANINE char tokenizer config
config.json	Transformer config

Why not ONNX / ORT INT8? CANINE’s local-attention + molecule path still yields illegal MatMul shapes in ONNX Runtime. For very fast CPU use TorchScript INT8.

CPU latency (batch=1, pad to 192, 8 threads)

Runtime	p50 latency	Approx QPS
TorchScript INT8 (model.pt)	24.7 ms	40.5
TorchScript FP32 (model_fp32.pt)	40.3 ms	24.8

INT8 is ~1.63× faster on CPU and ~2.3× smaller on disk (221 MB vs 508 MB).

Recommended confidence threshold

Optimal threshold (selected for performance + coverage): 0.55

Serving rule:

probs = softmax(intent_logits)
pred  = argmax(probs)
conf  = max(probs)
if conf < 0.55:
    pred = unclear   # safe abstain

Selection rule: among thresholds with coverage ≥ 0.97, maximize selective accuracy (tie-break: lower false-intent rate). This keeps most examples committed while cutting false specific intents.

Test-set metrics (sentence-direct test)

Variant	Accuracy	Macro-F1	False-intent	Coverage	Selective acc
FP32 raw (no gate)	0.9379	0.9365	0.0547	1.0000	0.9379
Deploy raw (no gate)	0.9379	0.9365	0.0547	1.0000	0.9379
Deploy @ thr=0.55	0.9329	0.9333	0.0410	0.9715	0.9528

Input / output contract

Input: pass one individual sentence (or a batch of independent sentences). Do not wrap with dialogue markers.

Valid examples:

• Haan, ye sahi hai. Proceed karo.
• What is the minimum balance required?
• Nahi, maine ye payment nahi kiya.
• Sorry I meant March not April.

Tensors

Name	Type	Shape	Notes
input_ids	int64	[batch, seq]	pad/truncate to 192
attention_mask	int64	[batch, seq]	1 = real, 0 = pad
logits / intent_logits	float	[batch, 7]	apply softmax for confidence

Inference snippet

import json
from pathlib import Path

import numpy as np
from transformers import AutoTokenizer, CanineTokenizer

MODEL_DIR = Path(".")  # or huggingface_hub.snapshot_download(...)
MAX_LEN = 192
CONF_THRESHOLD = 0.55  # optimal gate (coverage>=0.97, max selective acc)
UNCLEAR_ID = 6

# tokenizer — Character-level CANINE tokenizer (no WordPiece vocab).
tokenizer = CanineTokenizer.from_pretrained(MODEL_DIR)

with open(MODEL_DIR / "label_map.json", encoding="utf-8") as f:
    maps = json.load(f)
id2intent = {int(k): v for k, v in maps["head4_intent"]["id2label"].items()}

import torch

# Fast CPU path: INT8 TorchScript (model.pt == model_int8.pt)
session = torch.jit.load(str(MODEL_DIR / "model.pt"), map_location="cpu")
session.eval()
torch.set_num_threads(8)  # tune; too many threads can hurt

def predict_intent(sentence: str) -> dict:
    """Pass a single bare sentence; returns label + confidence."""
    enc = tokenizer(
        sentence,
        return_tensors="pt",
        truncation=True,
        max_length=MAX_LEN,
        padding="max_length",
    )
    with torch.inference_mode():
        logits = session(
            enc["input_ids"].to(torch.int64),
            enc["attention_mask"].to(torch.int64),
        )[0].cpu().numpy()

    x = logits.astype(np.float64)
    x = x - x.max()
    probs = np.exp(x)
    probs = probs / probs.sum()
    pred_id = int(probs.argmax())
    conf = float(probs.max())
    abstained = conf < CONF_THRESHOLD
    if abstained:
        pred_id = UNCLEAR_ID
    return {
        "text": sentence,
        "intent": id2intent[pred_id],
        "confidence": conf,
        "abstained": abstained,
        "threshold": CONF_THRESHOLD,
        "probs": {id2intent[i]: float(probs[i]) for i in range(7)},
    }

for s in [
    "Haan, ye sahi hai. Proceed karo.",
    "What is the minimum balance required?",
    "Nahi, maine ye payment nahi kiya.",
]:
    print(predict_intent(s))

Notes

• Intent-only model (no LID / LSD heads).
• Trained on sentence-direct data (Gemini synth + Muthoot user turns + noisy unclear), without [assistant]/[user] packing.