How to interpret the score?
#4
by Jr92 - opened
Hi,
I am trying the rerank-xsmall model. I can see even for 2 same sentences the score is under 0.25.
So, I was wondering if I am interpreting it incorrectly.
I was trying "What a day. What a day"
Thanks
Hey @Jr92 ,
our reranker is mostly designed to work like a search engine and in this iteration is not mean for STS tasks. This means you should design your query to be a question. E.g. "Who wrote 'To Kill a Mockingbird'?" + "'To Kill a Mockingbird' is a novel by Harper Lee published in 1960. It was immediately successful, winning the Pulitzer Prize, and has become a classic of modern American literature.". We're aware that this is a downfall and are working to fix this in V2.
Have a nice day!
juliuslipp changed discussion status to closed