nomic-ai

/

nomic-embed-text-v1

like 581

Follow

Nomic AI 1.22k

Just tested this, and I'm getting identical performance on NanoMSMARCO using Sentence Transformers using:

• trust_remote_code=True on main
• trust_remote_code=True on this PR
• No trust_remote_code on this PR, i.e. with native transformers

My only comment is that the last of the 3 gets 2 warnings:

Unrecognized keys in `rope_parameters` for 'rope_type'='dynamic': {'rope_theta'}
model.safetensors: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 547M/547M [00:05<00:00, 94.0MB/s]
Loading weights: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 112/112 [00:00<00:00, 10455.19it/s]
Unrecognized keys in `rope_parameters` for 'rope_type'='dynamic': {'rope_theta'}
0.6025776932803806

But I assume this is possible for the frankenconfig that supports both old and new. Nice work @AntonV

• Tom Aarsen

Before I forget it, the warning should be resolved now btw. Was not due to the config but something missing in our codebase