superskillret-onnx-int8

An ONNX INT8 export of an embedding encoder for agent skill retrieval. Built for CPU inference, for environments that need to run skill routing without a GPU (local dev machines, edge, CI).

Item	Value
Architecture	`Qwen3Model` (hidden 1024, 28 layers, max position 32,768)
Format	ONNX, INT8 quantization — single `model.onnx`, 598 MB
Paired index	`ThakiCloud/superskillret-index`
Released	2026-05-11

Usage

import onnxruntime as ort
from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("ThakiCloud/superskillret-onnx-int8")
sess = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])

enc = tok(["슬랙에 오늘 배포 결과 올려줘"], return_tensors="np", padding=True)  # "post today's deploy results to Slack"
out = sess.run(None, {k: v for k, v in enc.items() if k in {i.name for i in sess.get_inputs()}})

The pooling and normalization method must match whatever built the index (follow the build config of superskillret-index exactly).

⛔ Read this first — what we do not know

This checkpoint is the output of an early superskillret generation and has never been re-evaluated on the SkillRet benchmark released afterward. This repo has no citable retrieval-accuracy figure.
How much INT8 quantization costs in retrieval quality was also not measured. There is no fp32 control to compare against, so do not read this as "no loss."
If you need accuracy right now, use ThakiCloud/SKILLRET-Embedding-0.6B (same size class, trained and evaluated on the SkillRet benchmark), and reach for this repo only when you have a CPU-only constraint.