nerualdreming/ruaccent-omograph-int8

RUAccent homograph models in int8

The two large homograph models of RUAccent (ruaccent/accentuator at b78ae5ea1e62beaf138bed1865cd8c3b0b5ca855, MIT), stored in int8 so that they make the same choices as the fp32 originals.

folder	upstream model	fp32	here
big_poetry/	nn/nn_omograph/big_poetry (BERT-base cross-encoder)	713.5 MB	181.6 MB
turbo3.1/	nn/nn_omograph/turbo3.1 (DeBERTa, 6 layers)	359.3 MB	92.3 MB

Each folder is a drop-in replacement for the upstream one: same inputs (input_ids, attention_mask, plus token_type_ids for big_poetry), same logits output, same tokenizer files.

How

Weight-only int8 (convert.py): every 2-D fp32 weight of a MatMul, and the embedding tables read by Gather, is stored as symmetric int8 with an fp32 scale per output channel (per row for embeddings) and restored in the graph by Cast + Mul. Activations stay fp32. It runs on any opset and any ONNX Runtime execution provider.

ONNX Runtime's dynamic quantization (the recipe of upstream tiny2) was tried first and rejected: quantizing the activations of every MatMul flips homograph choices against fp32 (see below).

Checked against fp32

The reading of a homograph is the variant with the highest probability for the context marked as in RUAccent (... <w>замок</w> ... paired with з+амок / зам+ок).

	213 homographs in 772 sentences of real Russian text: same choice as fp32	33 sentences with a known reading: correct	largest probability shift there
big_poetry weight-only int8 (this repo)	213 / 213	29 / 33 (fp32: 29)	0.071
turbo3.1 weight-only int8 (this repo)	211 / 213 (both on near ties in fp32)	28 / 33 (fp32: 28)	0.005
big_poetry dynamic int8, per channel	208 / 213	29 / 33	0.59
turbo3.1 dynamic int8, per channel	208 / 213	28 / 33	—
upstream tiny2 (dynamic int8)	—	24 / 33	—

On CPU (one variant scored): tiny2 0.7 ms, turbo3.1 4.3 ms, big_poetry 8.8 ms.

Use

from huggingface_hub import snapshot_download
d = snapshot_download("nerualdreming/ruaccent-omograph-int8", allow_patterns=["big_poetry/*"])

Then point RUAccent's nn/nn_omograph/big_poetry at d/big_poetry, or load model.onnx with ONNX Runtime directly.

License MIT, as upstream; copyright in the weights stays with the RUAccent authors.