sevenreasons/laya-onnx-fp16

Laya ONNX FP16

FP16 ONNX Runtime export of convaiinnovations/laya.

Selected numerically sensitive ops remain FP32.

Inputs

input_ids       int64 [batch, sequence_length]
attention_mask  int64 [batch, sequence_length]
marker_pos      int64 [batch, num_options]
marker_mask     bool  [batch, num_options]
qtype           int64 [batch]

qtype: 0=choice, 1=score, 2=noul

Outputs

logits      [batch, num_options]
act_logits  [batch, 2]

Dynamic batch size, sequence length, and option count.

Validation

Numerical fidelity

Compared against the original PyTorch model:

Max logits diff:       0.00416
Mean logits diff:      0.00115
Max action-prob diff:  0.0

Tested option counts: 2, 3, 5, 8, 10.

End-to-end classification

300 hardcoded real-world-style classification cases were evaluated with the original PyTorch model and this ONNX export using identical inputs and inference settings.

                    PyTorch       ONNX
Overall accuracy    251/300       251/300
                    83.7%         83.7%

Brier score         0.1295        0.1295
Log loss            0.4121        0.4121

All 300 cases produced the same final classifications.

Inference on the same hardware:

PyTorch             47.66 s
ONNX Runtime        42.24 s

PyTorch             6.29 decisions/s
ONNX Runtime        7.10 decisions/s

License

Apache-2.0