rishhh/verisci-claim-verifier-dense-adapted-seed123

VeriSci Claim Verifier Dense-Retrieval-Adapted Candidate

Dense-retrieval-adapted VeriSci classifier for scientific claim verification over SUPPORTS, REFUTES, and NOT_ENOUGH_INFO.

Promotion Status

Not promoted. This checkpoint was trained and evaluated successfully, but it regressed against rishhh/verisci-claim-verifier-retrieval-adapted-seed123 on both the derived test split and the dense/hybrid retrieval-grounded SciFact validation gate.

Training Data

• allenai/scifact_entailment train split with gold evidence sentences.
• allenai/scifact train claims converted into gold evidence snippets, BM25-selected evidence snippets, and BM25-retrieved hard NOT_ENOUGH_INFO negatives.
• andreiaalexa/scifact-relevance-pairs title/train hard negatives.

Derived Test

Accuracy: 0.8935 Macro F1: 0.8567

Full SciFact Validation Gate

Top-1 end-to-end accuracy: 0.5711 Top-1 end-to-end macro F1: 0.5536 Top-5 decisive accuracy: 0.6356 Top-5 decisive macro F1: 0.5935

Dense/Hybrid Retrieval-Grounded Gate

Using rishhh/verisci-scifact-e5-retriever, hybrid alpha 0.75, and guarded top-5 decisive aggregation:

• Accuracy: 0.6756
• Macro F1: 0.6426

The current promoted retrieval-adapted verifier reaches 0.6911 accuracy and 0.6529 macro F1 under the same dense/hybrid guarded gate.

See evaluation/eval_summary.json for full metrics and limitations.

Responsible Use

This model is not a medical device, not a substitute for peer review, and should not be used for clinical, legal, or public-policy decisions without expert review.