Update README.md

176cf7f verified 6 months ago

30.2 kB

	---
	language:
	- en
	license: apache-2.0
	tags:
	- sentence-transformers
	- sentence-similarity
	- feature-extraction
	- loss:MatryoshkaLoss
	- loss:MultipleNegativesRankingLoss
	datasets:
	- sentence-transformers/squad
	- sentence-transformers/trivia-qa-triplet
	- sentence-transformers/all-nli
	- sentence-transformers/pubmedqa
	- sentence-transformers/hotpotqa
	- sentence-transformers/miracl
	- sentence-transformers/mr-tydi
	- sentence-transformers/s2orc
	- nthakur/swim-ir-monolingual
	- sentence-transformers/paq
	- tomaarsen/natural-questions-hard-negatives
	pipeline_tag: sentence-similarity
	library_name: sentence-transformers
	metrics:
	- cosine_accuracy@1
	- cosine_accuracy@3
	- cosine_accuracy@5
	- cosine_accuracy@10
	- cosine_precision@1
	- cosine_precision@3
	- cosine_precision@5
	- cosine_precision@10
	- cosine_recall@1
	- cosine_recall@3
	- cosine_recall@5
	- cosine_recall@10
	- cosine_ndcg@10
	- cosine_mrr@10
	- cosine_map@100
	model-index:
	- name: SSE Retrieval MRL 0.9999
	results:
	- task:
	type: information-retrieval
	name: Information Retrieval
	dataset:
	name: NanoClimateFEVER
	type: NanoClimateFEVER
	metrics:
	- type: cosine_accuracy@1
	value: 0.2
	name: Cosine Accuracy@1
	- type: cosine_accuracy@3
	value: 0.48
	name: Cosine Accuracy@3
	- type: cosine_accuracy@5
	value: 0.54
	name: Cosine Accuracy@5
	- type: cosine_accuracy@10
	value: 0.68
	name: Cosine Accuracy@10
	- type: cosine_precision@1
	value: 0.2
	name: Cosine Precision@1
	- type: cosine_precision@3
	value: 0.18
	name: Cosine Precision@3
	- type: cosine_precision@5
	value: 0.128
	name: Cosine Precision@5
	- type: cosine_precision@10
	value: 0.102
	name: Cosine Precision@10
	- type: cosine_recall@1
	value: 0.10166666666666666
	name: Cosine Recall@1
	- type: cosine_recall@3
	value: 0.24166666666666667
	name: Cosine Recall@3
	- type: cosine_recall@5
	value: 0.2733333333333334
	name: Cosine Recall@5
	- type: cosine_recall@10
	value: 0.39233333333333337
	name: Cosine Recall@10
	- type: cosine_ndcg@10
	value: 0.299751347194741
	name: Cosine Ndcg@10
	- type: cosine_mrr@10
	value: 0.36113492063492053
	name: Cosine Mrr@10
	- type: cosine_map@100
	value: 0.23438514328438953
	name: Cosine Map@100
	- task:
	type: information-retrieval
	name: Information Retrieval
	dataset:
	name: NanoDBPedia
	type: NanoDBPedia
	metrics:
	- type: cosine_accuracy@1
	value: 0.66
	name: Cosine Accuracy@1
	- type: cosine_accuracy@3
	value: 0.84
	name: Cosine Accuracy@3
	- type: cosine_accuracy@5
	value: 0.84
	name: Cosine Accuracy@5
	- type: cosine_accuracy@10
	value: 0.9
	name: Cosine Accuracy@10
	- type: cosine_precision@1
	value: 0.66
	name: Cosine Precision@1
	- type: cosine_precision@3
	value: 0.5666666666666667
	name: Cosine Precision@3
	- type: cosine_precision@5
	value: 0.52
	name: Cosine Precision@5
	- type: cosine_precision@10
	value: 0.44400000000000006
	name: Cosine Precision@10
	- type: cosine_recall@1
	value: 0.07827093153121195
	name: Cosine Recall@1
	- type: cosine_recall@3
	value: 0.16032236337443734
	name: Cosine Recall@3
	- type: cosine_recall@5
	value: 0.20952091065849757
	name: Cosine Recall@5
	- type: cosine_recall@10
	value: 0.29831579691724436
	name: Cosine Recall@10
	- type: cosine_ndcg@10
	value: 0.5493340697005651
	name: Cosine Ndcg@10
	- type: cosine_mrr@10
	value: 0.7491666666666665
	name: Cosine Mrr@10
	- type: cosine_map@100
	value: 0.4246657246617055
	name: Cosine Map@100
	- task:
	type: information-retrieval
	name: Information Retrieval
	dataset:
	name: NanoFEVER
	type: NanoFEVER
	metrics:
	- type: cosine_accuracy@1
	value: 0.46
	name: Cosine Accuracy@1
	- type: cosine_accuracy@3
	value: 0.76
	name: Cosine Accuracy@3
	- type: cosine_accuracy@5
	value: 0.82
	name: Cosine Accuracy@5
	- type: cosine_accuracy@10
	value: 0.92
	name: Cosine Accuracy@10
	- type: cosine_precision@1
	value: 0.46
	name: Cosine Precision@1
	- type: cosine_precision@3
	value: 0.25333333333333335
	name: Cosine Precision@3
	- type: cosine_precision@5
	value: 0.17199999999999996
	name: Cosine Precision@5
	- type: cosine_precision@10
	value: 0.09599999999999997
	name: Cosine Precision@10
	- type: cosine_recall@1
	value: 0.43666666666666665
	name: Cosine Recall@1
	- type: cosine_recall@3
	value: 0.7166666666666667
	name: Cosine Recall@3
	- type: cosine_recall@5
	value: 0.7866666666666667
	name: Cosine Recall@5
	- type: cosine_recall@10
	value: 0.8866666666666667
	name: Cosine Recall@10
	- type: cosine_ndcg@10
	value: 0.6808214594769284
	name: Cosine Ndcg@10
	- type: cosine_mrr@10
	value: 0.6318253968253967
	name: Cosine Mrr@10
	- type: cosine_map@100
	value: 0.6105163447649364
	name: Cosine Map@100
	- task:
	type: information-retrieval
	name: Information Retrieval
	dataset:
	name: NanoFiQA2018
	type: NanoFiQA2018
	metrics:
	- type: cosine_accuracy@1
	value: 0.32
	name: Cosine Accuracy@1
	- type: cosine_accuracy@3
	value: 0.48
	name: Cosine Accuracy@3
	- type: cosine_accuracy@5
	value: 0.58
	name: Cosine Accuracy@5
	- type: cosine_accuracy@10
	value: 0.62
	name: Cosine Accuracy@10
	- type: cosine_precision@1
	value: 0.32
	name: Cosine Precision@1
	- type: cosine_precision@3
	value: 0.22666666666666666
	name: Cosine Precision@3
	- type: cosine_precision@5
	value: 0.17600000000000002
	name: Cosine Precision@5
	- type: cosine_precision@10
	value: 0.10200000000000001
	name: Cosine Precision@10
	- type: cosine_recall@1
	value: 0.1861904761904762
	name: Cosine Recall@1
	- type: cosine_recall@3
	value: 0.3212936507936508
	name: Cosine Recall@3
	- type: cosine_recall@5
	value: 0.38946031746031745
	name: Cosine Recall@5
	- type: cosine_recall@10
	value: 0.4546825396825397
	name: Cosine Recall@10
	- type: cosine_ndcg@10
	value: 0.3743730832469537
	name: Cosine Ndcg@10
	- type: cosine_mrr@10
	value: 0.4197142857142857
	name: Cosine Mrr@10
	- type: cosine_map@100
	value: 0.3162051518688468
	name: Cosine Map@100
	- task:
	type: information-retrieval
	name: Information Retrieval
	dataset:
	name: NanoHotpotQA
	type: NanoHotpotQA
	metrics:
	- type: cosine_accuracy@1
	value: 0.64
	name: Cosine Accuracy@1
	- type: cosine_accuracy@3
	value: 0.88
	name: Cosine Accuracy@3
	- type: cosine_accuracy@5
	value: 0.94
	name: Cosine Accuracy@5
	- type: cosine_accuracy@10
	value: 0.96
	name: Cosine Accuracy@10
	- type: cosine_precision@1
	value: 0.64
	name: Cosine Precision@1
	- type: cosine_precision@3
	value: 0.41999999999999993
	name: Cosine Precision@3
	- type: cosine_precision@5
	value: 0.296
	name: Cosine Precision@5
	- type: cosine_precision@10
	value: 0.15999999999999998
	name: Cosine Precision@10
	- type: cosine_recall@1
	value: 0.32
	name: Cosine Recall@1
	- type: cosine_recall@3
	value: 0.63
	name: Cosine Recall@3
	- type: cosine_recall@5
	value: 0.74
	name: Cosine Recall@5
	- type: cosine_recall@10
	value: 0.8
	name: Cosine Recall@10
	- type: cosine_ndcg@10
	value: 0.7020829772895696
	name: Cosine Ndcg@10
	- type: cosine_mrr@10
	value: 0.7678571428571429
	name: Cosine Mrr@10
	- type: cosine_map@100
	value: 0.6273248247260853
	name: Cosine Map@100
	- task:
	type: information-retrieval
	name: Information Retrieval
	dataset:
	name: NanoMSMARCO
	type: NanoMSMARCO
	metrics:
	- type: cosine_accuracy@1
	value: 0.24
	name: Cosine Accuracy@1
	- type: cosine_accuracy@3
	value: 0.46
	name: Cosine Accuracy@3
	- type: cosine_accuracy@5
	value: 0.52
	name: Cosine Accuracy@5
	- type: cosine_accuracy@10
	value: 0.6
	name: Cosine Accuracy@10
	- type: cosine_precision@1
	value: 0.24
	name: Cosine Precision@1
	- type: cosine_precision@3
	value: 0.1533333333333333
	name: Cosine Precision@3
	- type: cosine_precision@5
	value: 0.10400000000000001
	name: Cosine Precision@5
	- type: cosine_precision@10
	value: 0.06000000000000001
	name: Cosine Precision@10
	- type: cosine_recall@1
	value: 0.24
	name: Cosine Recall@1
	- type: cosine_recall@3
	value: 0.46
	name: Cosine Recall@3
	- type: cosine_recall@5
	value: 0.52
	name: Cosine Recall@5
	- type: cosine_recall@10
	value: 0.6
	name: Cosine Recall@10
	- type: cosine_ndcg@10
	value: 0.4132396978554854
	name: Cosine Ndcg@10
	- type: cosine_mrr@10
	value: 0.35374603174603175
	name: Cosine Mrr@10
	- type: cosine_map@100
	value: 0.373289844122511
	name: Cosine Map@100
	- task:
	type: information-retrieval
	name: Information Retrieval
	dataset:
	name: NanoNFCorpus
	type: NanoNFCorpus
	metrics:
	- type: cosine_accuracy@1
	value: 0.38
	name: Cosine Accuracy@1
	- type: cosine_accuracy@3
	value: 0.56
	name: Cosine Accuracy@3
	- type: cosine_accuracy@5
	value: 0.6
	name: Cosine Accuracy@5
	- type: cosine_accuracy@10
	value: 0.76
	name: Cosine Accuracy@10
	- type: cosine_precision@1
	value: 0.38
	name: Cosine Precision@1
	- type: cosine_precision@3
	value: 0.34666666666666673
	name: Cosine Precision@3
	- type: cosine_precision@5
	value: 0.29600000000000004
	name: Cosine Precision@5
	- type: cosine_precision@10
	value: 0.24599999999999997
	name: Cosine Precision@10
	- type: cosine_recall@1
	value: 0.0338546319021278
	name: Cosine Recall@1
	- type: cosine_recall@3
	value: 0.06462469800035843
	name: Cosine Recall@3
	- type: cosine_recall@5
	value: 0.07727799038239798
	name: Cosine Recall@5
	- type: cosine_recall@10
	value: 0.10829423267139048
	name: Cosine Recall@10
	- type: cosine_ndcg@10
	value: 0.298189605225764
	name: Cosine Ndcg@10
	- type: cosine_mrr@10
	value: 0.48890476190476195
	name: Cosine Mrr@10
	- type: cosine_map@100
	value: 0.10911000304853699
	name: Cosine Map@100
	- task:
	type: information-retrieval
	name: Information Retrieval
	dataset:
	name: NanoNQ
	type: NanoNQ
	metrics:
	- type: cosine_accuracy@1
	value: 0.24
	name: Cosine Accuracy@1
	- type: cosine_accuracy@3
	value: 0.52
	name: Cosine Accuracy@3
	- type: cosine_accuracy@5
	value: 0.62
	name: Cosine Accuracy@5
	- type: cosine_accuracy@10
	value: 0.7
	name: Cosine Accuracy@10
	- type: cosine_precision@1
	value: 0.24
	name: Cosine Precision@1
	- type: cosine_precision@3
	value: 0.1733333333333333
	name: Cosine Precision@3
	- type: cosine_precision@5
	value: 0.124
	name: Cosine Precision@5
	- type: cosine_precision@10
	value: 0.07400000000000001
	name: Cosine Precision@10
	- type: cosine_recall@1
	value: 0.23
	name: Cosine Recall@1
	- type: cosine_recall@3
	value: 0.5
	name: Cosine Recall@3
	- type: cosine_recall@5
	value: 0.6
	name: Cosine Recall@5
	- type: cosine_recall@10
	value: 0.69
	name: Cosine Recall@10
	- type: cosine_ndcg@10
	value: 0.46521648817123007
	name: Cosine Ndcg@10
	- type: cosine_mrr@10
	value: 0.39922222222222226
	name: Cosine Mrr@10
	- type: cosine_map@100
	value: 0.4028459782678049
	name: Cosine Map@100
	- task:
	type: information-retrieval
	name: Information Retrieval
	dataset:
	name: NanoQuoraRetrieval
	type: NanoQuoraRetrieval
	metrics:
	- type: cosine_accuracy@1
	value: 0.86
	name: Cosine Accuracy@1
	- type: cosine_accuracy@3
	value: 0.98
	name: Cosine Accuracy@3
	- type: cosine_accuracy@5
	value: 0.98
	name: Cosine Accuracy@5
	- type: cosine_accuracy@10
	value: 1.0
	name: Cosine Accuracy@10
	- type: cosine_precision@1
	value: 0.86
	name: Cosine Precision@1
	- type: cosine_precision@3
	value: 0.37999999999999995
	name: Cosine Precision@3
	- type: cosine_precision@5
	value: 0.23599999999999993
	name: Cosine Precision@5
	- type: cosine_precision@10
	value: 0.12399999999999999
	name: Cosine Precision@10
	- type: cosine_recall@1
	value: 0.7706666666666666
	name: Cosine Recall@1
	- type: cosine_recall@3
	value: 0.932
	name: Cosine Recall@3
	- type: cosine_recall@5
	value: 0.9453333333333334
	name: Cosine Recall@5
	- type: cosine_recall@10
	value: 0.9626666666666668
	name: Cosine Recall@10
	- type: cosine_ndcg@10
	value: 0.9094074101386184
	name: Cosine Ndcg@10
	- type: cosine_mrr@10
	value: 0.9122222222222223
	name: Cosine Mrr@10
	- type: cosine_map@100
	value: 0.8846858964622123
	name: Cosine Map@100
	- task:
	type: information-retrieval
	name: Information Retrieval
	dataset:
	name: NanoSCIDOCS
	type: NanoSCIDOCS
	metrics:
	- type: cosine_accuracy@1
	value: 0.46
	name: Cosine Accuracy@1
	- type: cosine_accuracy@3
	value: 0.62
	name: Cosine Accuracy@3
	- type: cosine_accuracy@5
	value: 0.68
	name: Cosine Accuracy@5
	- type: cosine_accuracy@10
	value: 0.76
	name: Cosine Accuracy@10
	- type: cosine_precision@1
	value: 0.46
	name: Cosine Precision@1
	- type: cosine_precision@3
	value: 0.29333333333333333
	name: Cosine Precision@3
	- type: cosine_precision@5
	value: 0.252
	name: Cosine Precision@5
	- type: cosine_precision@10
	value: 0.162
	name: Cosine Precision@10
	- type: cosine_recall@1
	value: 0.09666666666666668
	name: Cosine Recall@1
	- type: cosine_recall@3
	value: 0.18166666666666664
	name: Cosine Recall@3
	- type: cosine_recall@5
	value: 0.26066666666666666
	name: Cosine Recall@5
	- type: cosine_recall@10
	value: 0.33466666666666667
	name: Cosine Recall@10
	- type: cosine_ndcg@10
	value: 0.33808831519730853
	name: Cosine Ndcg@10
	- type: cosine_mrr@10
	value: 0.5508571428571427
	name: Cosine Mrr@10
	- type: cosine_map@100
	value: 0.260404942677937
	name: Cosine Map@100
	- task:
	type: information-retrieval
	name: Information Retrieval
	dataset:
	name: NanoArguAna
	type: NanoArguAna
	metrics:
	- type: cosine_accuracy@1
	value: 0.14
	name: Cosine Accuracy@1
	- type: cosine_accuracy@3
	value: 0.5
	name: Cosine Accuracy@3
	- type: cosine_accuracy@5
	value: 0.56
	name: Cosine Accuracy@5
	- type: cosine_accuracy@10
	value: 0.7
	name: Cosine Accuracy@10
	- type: cosine_precision@1
	value: 0.14
	name: Cosine Precision@1
	- type: cosine_precision@3
	value: 0.16666666666666669
	name: Cosine Precision@3
	- type: cosine_precision@5
	value: 0.11200000000000002
	name: Cosine Precision@5
	- type: cosine_precision@10
	value: 0.07
	name: Cosine Precision@10
	- type: cosine_recall@1
	value: 0.14
	name: Cosine Recall@1
	- type: cosine_recall@3
	value: 0.5
	name: Cosine Recall@3
	- type: cosine_recall@5
	value: 0.56
	name: Cosine Recall@5
	- type: cosine_recall@10
	value: 0.7
	name: Cosine Recall@10
	- type: cosine_ndcg@10
	value: 0.41047352977721935
	name: Cosine Ndcg@10
	- type: cosine_mrr@10
	value: 0.3192777777777777
	name: Cosine Mrr@10
	- type: cosine_map@100
	value: 0.33248820268587403
	name: Cosine Map@100
	- task:
	type: information-retrieval
	name: Information Retrieval
	dataset:
	name: NanoSciFact
	type: NanoSciFact
	metrics:
	- type: cosine_accuracy@1
	value: 0.54
	name: Cosine Accuracy@1
	- type: cosine_accuracy@3
	value: 0.6
	name: Cosine Accuracy@3
	- type: cosine_accuracy@5
	value: 0.66
	name: Cosine Accuracy@5
	- type: cosine_accuracy@10
	value: 0.74
	name: Cosine Accuracy@10
	- type: cosine_precision@1
	value: 0.54
	name: Cosine Precision@1
	- type: cosine_precision@3
	value: 0.21333333333333332
	name: Cosine Precision@3
	- type: cosine_precision@5
	value: 0.14400000000000002
	name: Cosine Precision@5
	- type: cosine_precision@10
	value: 0.08199999999999999
	name: Cosine Precision@10
	- type: cosine_recall@1
	value: 0.505
	name: Cosine Recall@1
	- type: cosine_recall@3
	value: 0.58
	name: Cosine Recall@3
	- type: cosine_recall@5
	value: 0.645
	name: Cosine Recall@5
	- type: cosine_recall@10
	value: 0.735
	name: Cosine Recall@10
	- type: cosine_ndcg@10
	value: 0.6175889955513287
	name: Cosine Ndcg@10
	- type: cosine_mrr@10
	value: 0.5933015873015873
	name: Cosine Mrr@10
	- type: cosine_map@100
	value: 0.5823752505606269
	name: Cosine Map@100
	- task:
	type: information-retrieval
	name: Information Retrieval
	dataset:
	name: NanoTouche2020
	type: NanoTouche2020
	metrics:
	- type: cosine_accuracy@1
	value: 0.6530612244897959
	name: Cosine Accuracy@1
	- type: cosine_accuracy@3
	value: 0.9183673469387755
	name: Cosine Accuracy@3
	- type: cosine_accuracy@5
	value: 0.9591836734693877
	name: Cosine Accuracy@5
	- type: cosine_accuracy@10
	value: 1.0
	name: Cosine Accuracy@10
	- type: cosine_precision@1
	value: 0.6530612244897959
	name: Cosine Precision@1
	- type: cosine_precision@3
	value: 0.6394557823129251
	name: Cosine Precision@3
	- type: cosine_precision@5
	value: 0.6244897959183674
	name: Cosine Precision@5
	- type: cosine_precision@10
	value: 0.5551020408163265
	name: Cosine Precision@10
	- type: cosine_recall@1
	value: 0.04446978335433603
	name: Cosine Recall@1
	- type: cosine_recall@3
	value: 0.12883713641764533
	name: Cosine Recall@3
	- type: cosine_recall@5
	value: 0.20234901450308018
	name: Cosine Recall@5
	- type: cosine_recall@10
	value: 0.3514245193484443
	name: Cosine Recall@10
	- type: cosine_ndcg@10
	value: 0.602875180920439
	name: Cosine Ndcg@10
	- type: cosine_mrr@10
	value: 0.7852283770651117
	name: Cosine Mrr@10
	- type: cosine_map@100
	value: 0.4539105214909128
	name: Cosine Map@100
	- task:
	type: nano-beir
	name: Nano BEIR
	dataset:
	name: NanoBEIR mean
	type: NanoBEIR_mean
	metrics:
	- type: cosine_accuracy@1
	value: 0.4456200941915227
	name: Cosine Accuracy@1
	- type: cosine_accuracy@3
	value: 0.6614128728414441
	name: Cosine Accuracy@3
	- type: cosine_accuracy@5
	value: 0.7153218210361068
	name: Cosine Accuracy@5
	- type: cosine_accuracy@10
	value: 0.7953846153846154
	name: Cosine Accuracy@10
	- type: cosine_precision@1
	value: 0.4456200941915227
	name: Cosine Precision@1
	- type: cosine_precision@3
	value: 0.30867608581894296
	name: Cosine Precision@3
	- type: cosine_precision@5
	value: 0.24496075353218213
	name: Cosine Precision@5
	- type: cosine_precision@10
	value: 0.17516169544740973
	name: Cosine Precision@10
	- type: cosine_recall@1
	value: 0.2448809607419091
	name: Cosine Recall@1
	- type: cosine_recall@3
	value: 0.416698296045084
	name: Cosine Recall@3
	- type: cosine_recall@5
	value: 0.47766217176956105
	name: Cosine Recall@5
	- type: cosine_recall@10
	value: 0.5626192632271503
	name: Cosine Recall@10
	- type: cosine_ndcg@10
	value: 0.5124186276727808
	name: Cosine Ndcg@10
	- type: cosine_mrr@10
	value: 0.5640352719842516
	name: Cosine Mrr@10
	- type: cosine_map@100
	value: 0.43170829450941384
	name: Cosine Map@100
	---

	![SSE](assets/SSE_Logo.png)

	If you would like to know more details:

	[SSE Technical Article](https://huggingface.co/blog/RikkaBotan/stable-static-embedding-technical-report)


	![SSE](assets/SSE_comp_en.png)

	(a) Retrieval performance (nDCG@10) across NanoBEIR English tasks. (b) Mean nDCG@10 vs. inference speed (QPS: queries per second) measured on TREC-COVID and Quora using an Intel® Core™ Ultra 7 265K (3.90 GHz) with batch size 32.


	# 🩵 SSE: Stable Static Embedding for Retrieval MRL 🩵
	### A lightweight, faster and powerful embedding model

	Performance Snapshot
	Our SSE model achieves NDCG@10 = 0.5124 on NanoBEIR — slightly outperforming the popular `static-retrieval-mrl-en-v1` (0.5032) while using half the dimensions (512 vs 1024)! 💫 Plus, we're ~2× faster in retrieval thanks to our compact 512D embeddings and Separable Dynamic Tanh.

	\| Model \| NanoBEIR NDCG@10 \| Dimensions \| Parameters \| Speed Advantage \| License \|
	\|-------\|------------------\|------------\|------------\|-----------------\|---------\|
	\| SSE Retrieval MRL \| 0.5124 ✨ \| 512 \| ~16M 🪽 \| ~2x faster retrieval (ultra-efficient!) \| Apache 2.0 \|
	\| `static-retrieval-mrl-en-v1` \| 0.5032 \| 1024 \| ~33M \| baseline \| Apache 2.0 \|

	---

	## 🩵 Why Choose SSE Retrieval MRL? 🩵

	✅ Higher NDCG@10 than all comparable small models (<35M params)
	✅ Only ~16M parameters — 27% smaller than MiniLM-L6 (22M) and 52% smaller than BGE-small (33M)
	✅ 512D native output — richer than 1024D models, yet half the size of static-retrieval-mrl-en-v1
	✅ Matryoshka-ready — smoothly truncate to 256D/128D/64D/32D with graceful degradation
	✅ Apache 2.0 licensed — free for commercial & personal use
	✅ CPU-optimized — runs faster on edge devices & modest hardware

	---

	## 🩵 Model Details 🩵

	\| Property \| Value \|
	\|----------\|-------\|
	\| Model Type \| Sentence Transformer (SSE architecture) \|
	\| Max Sequence Length \| ∞ tokens \|
	\| Output Dimension \| 512 (with Matryoshka truncation down to 32D!) \|
	\| Similarity Function \| Cosine Similarity \|
	\| Language \| English \|
	\| License \| Apache 2.0 \|

	```python
	SentenceTransformer(
	(0): SSE(
	(embedding): EmbeddingBag(30522, 512, mode='mean')
	(dyt): SeparableDyT()
	)
	)
	```

	![Architecture](assets/SSE_Architecture.png)

	---

	## 🩵 Mathematical formulations 🩵

	Dynamic Tanh Normalization (DyT) enables magnitude-adaptive gradient flow for static embeddings. For input dimension x, DyT computes
	$$
	y_k = c_k \tanh(a_k x_k + b_k)
	$$
	with learnable parameters. The gradient of x is:

	$$
	\frac{\partial y_k}{\partial x_k} = c_k a_k \, \mathrm{sech}^2(a_k x_k + b_k).
	$$

	For saturated dimensions \|x\| > 1
	$$
	\|a_i x_i + b_i\| \gg 1
	$$
	yields exponential decay
	$$
	\mathrm{sech}^2(z) \sim 4e^{-2\|z\|}
	$$
	suppressing gradients as
	$$
	\partial y_i / \partial x_i \to 0
	$$
	For non-saturated dimensions \|x\| << 1 ,
	$$
	\mathrm{sech}^2(z) \approx 1
	$$
	preserves near-constant gradients
	$$
	\partial y_j / \partial x_j \approx c_j a_j
	$$
	This magnitude-dependent gating attenuates learning signals from noisy, large-magnitude dimensions while maintaining full gradient flow for stable, informative dimensions—providing implicit regularization that enhances generalization without explicit hyperparameters.

	---

	## 🩵 Evaluation Results (NanoBEIR) 🩵

	\| Dataset \| NDCG@10 \| MRR@10 \| MAP@100 \|
	\|---------\|---------\|--------\|---------\|
	\| NanoBEIR Mean \| 0.5124 ✨ \| 0.5640 \| 0.4317 \|
	\| NanoClimateFEVER \| 0.2998 \| 0.3611 \| 0.2344 \|
	\| NanoDBPedia \| 0.5493 \| 0.7492 \| 0.4247 \|
	\| NanoFEVER \| 0.6808 \| 0.6318 \| 0.6105 \|
	\| NanoFiQA2018 \| 0.3744 \| 0.4197 \| 0.3162 \|
	\| NanoHotpotQA \| 0.7021 \| 0.7679 \| 0.6273 \|
	\| NanoMSMARCO \| 0.4132 \| 0.3537 \| 0.3733 \|
	\| NanoNFCorpus \| 0.2982 \| 0.4889 \| 0.1091 \|
	\| NanoNQ \| 0.4652 \| 0.3992 \| 0.4028 \|
	\| NanoQuoraRetrieval \| 0.9094 ✨ \| 0.9122 \| 0.8847 \|
	\| NanoSCIDOCS \| 0.3381 \| 0.5509 \| 0.2604 \|
	\| NanoArguAna \| 0.4105 \| 0.3193 \| 0.3325 \|
	\| NanoSciFact \| 0.6176 \| 0.5933 \| 0.5824 \|
	\| NanoTouche2020 \| 0.6029 \| 0.7852 \| 0.4539 \|

	> Top performance on community-based retrieval (Quora) and scientific fact verification!

	---

	## 🩵 How to use? 🩵

	```python
	import torch
	from sentence_transformers import SentenceTransformer

	# load (remote code enabled)
	model = SentenceTransformer(
	"RikkaBotan/stable-static-embedding-fast-retrieval-mrl-en",
	trust_remote_code=True,
	device="cuda" if torch.cuda.is_available() else "cpu",
	)

	# inference
	sentences = [
	"Stable Static embedding is interesting.",
	"SSE works without attention."
	]

	with torch.no_grad():
	embeddings = model.encode(
	sentences,
	convert_to_tensor=True,
	normalize_embeddings=True,
	batch_size=32
	)

	# cosine similarity
	# cosine_sim = embeddings[0] @ embeddings[1].T
	cosine_sim = model.similarity(embeddings, embeddings)

	print("embeddings shape:", embeddings.shape)
	print("cosine similarity matrix:")
	print(cosine_sim)
	```
	---
	## 🩵 Retrieval usage 🩵

	```python
	import torch
	from sentence_transformers import SentenceTransformer

	# load (remote code enabled)
	model = SentenceTransformer(
	"RikkaBotan/stable-static-embedding-fast-retrieval-mrl-en",
	trust_remote_code=True,
	device="cuda" if torch.cuda.is_available() else "cpu",
	)

	# inference
	query = "What is Stable Static Embedding?"
	sentences = [
	"SSE: Stable Static embedding works without attention.",
	"Stable Static Embedding is a fast embedding method designed for retrieval tasks.",
	"Static embeddings are often compared with transformer-based sentence encoders.",
	"I cooked pasta last night while listening to jazz music.",
	"Large language models are commonly trained using next-token prediction objectives.",
	"Instruction tuning improves the ability of LLMs to follow human-written prompts.",
	]


	with torch.no_grad():
	embeddings = model.encode(
	[query] + sentences,
	convert_to_tensor=True,
	normalize_embeddings=True,
	batch_size=32
	)

	print("embeddings shape:", embeddings.shape)

	# cosine similarity
	similarities = model.similarity(embeddings[0], embeddings[1:])
	for i, similarity in enumerate(similarities[0].tolist()):
	print(f"{similarity:.05f}: {sentences[i]}")
	```

	---

	## 🩵 Training Hyperparameters 🩵

	#### Non-Default Hyperparameters

	- `eval_strategy`: steps
	- `per_device_train_batch_size`: 512
	- `gradient_accumulation_steps`: 8
	- `learning_rate`: 0.1
	- `adam_beta2`: 0.9999
	- `adam_epsilon`: 1e-10
	- `num_train_epochs`: 1
	- `lr_scheduler_type`: cosine
	- `warmup_ratio`: 0.1
	- `bf16`: True
	- `dataloader_num_workers`: 4
	- `batch_sampler`: no_duplicates

	---

	## 🩵 Training Datasets 🩵

	We learned from 14 datasets:

	\| Dataset \|
	\|---------\|
	\| `squad` \|
	\| `trivia_qa` \|
	\| `allnli` \|
	\| `pubmedqa` \|
	\| `hotpotqa` \|
	\| `miracl` \|
	\| `mr_tydi` \|
	\| `msmarco` \|
	\| `msmarco_10m` \|
	\| `msmarco_hard` \|
	\| `mldr` \|
	\| `s2orc` \|
	\| `swim_ir` \|
	\| `paq` \|
	\| `nq` \|
	\| `scidocs` \|

	All trained with MatryoshkaLoss* — learning representations at multiple scales like Russian nesting dolls!*

	## 🩵 Training results 🩵

	![loss](assets/SSE_loss.png)

	![ndcg](assets/SSE_ndcg.png)

	## 🩵 About me 🩵

	Japanese independent researcher having shy and pampered personality. Twin-tail hair is a charm point. Interested in nlp. Usually using python and C.

	X(Twitter):
	https://twitter.com/peony__snow

	![Logo](assets/RikkaBotan_Logo.png)

	## 🩵 Acknowledgements 🩵

	The author acknowledge the support of Saldra, Witness and Lumina Logic Minds for providing computational resources used in this work.

	I thank the developers of sentence-transformers, python and pytorch.

	I thank all the researchers for their efforts to date.

	I thank Japan's high standard of education.

	And most of all, thank you for your interest in this repository.

	## 🩵 Citation 🩵

	### BibTeX

	#### Sentence Transformers
	```bibtex
	@inproceedings{reimers-2019-sentence-bert,
	title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
	author = "Reimers, Nils and Gurevych, Iryna",
	booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
	month = "11",
	year = "2019",
	publisher = "Association for Computational Linguistics",
	url = "https://arxiv.org/abs/1908.10084",
	}
	```

	#### MatryoshkaLoss
	```bibtex
	@misc{kusupati2024matryoshka,
	title={Matryoshka Representation Learning},
	author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
	year={2024},
	eprint={2205.13147},
	archivePrefix={arXiv},
	primaryClass={cs.LG}
	}
	```

	#### MultipleNegativesRankingLoss
	```bibtex
	@misc{henderson2017efficient,
	title={Efficient Natural Language Response Suggestion for Smart Reply},
	author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
	year={2017},
	eprint={1705.00652},
	archivePrefix={arXiv},
	primaryClass={cs.CL}
	}
	```