Hugging Face's logo

steerrec
/
bge-reranker-v2-m3-query-note

Text Ranking
sentence-transformers
Safetensors
xlm-roberta
cross-encoder
reranker
Generated from Trainer
dataset_size:769572
loss:BinaryCrossEntropyLoss
Eval Results (legacy)
text-embeddings-inference
Model card Files
xet
 Community
bge-reranker-v2-m3-query-note / README.md
TheKostins's picture
TheKostins
Add new CrossEncoder model
0b58985 verified
preview code
|
Raw
History Blame Contribute Delete
33.2 kB
metadata 
tags:
  - sentence-transformers
  - cross-encoder
  - reranker
  - generated_from_trainer
  - dataset_size:769572
  - loss:BinaryCrossEntropyLoss
base_model: BAAI/bge-reranker-v2-m3
pipeline_tag: text-ranking
library_name: sentence-transformers
metrics:
  - map
  - mrr@10
  - ndcg@10
model-index:
  - name: CrossEncoder based on BAAI/bge-reranker-v2-m3
    results:
      - task:
          type: cross-encoder-reranking
          name: Cross Encoder Reranking
        dataset:
          name: query note test
          type: query_note_test
        metrics:
          - type: map
            value: 0.45654587062161356
            name: Map
          - type: mrr@10
            value: 0.5283685579312956
            name: Mrr@10
          - type: ndcg@10
            value: 0.527108528024005
            name: Ndcg@10
      - task:
          type: cross-encoder-reranking
          name: Cross Encoder Reranking
        dataset:
          name: query note val
          type: query_note_val
        metrics:
          - type: map
            value: 0.45724051914340746
            name: Map
          - type: mrr@10
            value: 0.5246903296465979
            name: Mrr@10
          - type: ndcg@10
            value: 0.5328170850876233
            name: Ndcg@10

CrossEncoder based on BAAI/bge-reranker-v2-m3

This is a Cross Encoder model finetuned from BAAI/bge-reranker-v2-m3 using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

  • Model Type: Cross Encoder
  • Base model: BAAI/bge-reranker-v2-m3
  • Maximum Sequence Length: 512 tokens
  • Number of Output Labels: 1 label
  • Supported Modality: Text

Model Sources

Full Model Architecture 

CrossEncoder(
  (0): Transformer({'transformer_task': 'sequence-classification', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'logits'}}, 'module_output_name': 'scores', 'architecture': 'XLMRobertaForSequenceClassification'})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("steerrec/bge-reranker-v2-m3-query-note")
# Get scores for pairs of inputs
pairs = [
    ['infp男和infj女', '谁懂啊啊啊‼️💗INFP×INFJ相处好戳我\n绿老头和小蝴蝶都是高敏易碎人群，消息秒回啊，朋友圈点赞啊，这种细微且偏爱的行为，都能让对方感到安心和减少内耗。\n小蝴蝶的话，有事情就不要藏着掖着，对绿老头也不要有过多的试探行为，有问题就直接问，直接说，只要把事情拖到情绪都上来了，那这问题就更难解决了。因为绿老头只是看起来淡然，但其实他们的内心是很火热躁动的，他们只是不好意思主动表达。绿老头得学会调整，明知道小蝴蝶容易想多内耗，那就多主动表现情绪，不要让小蝴蝶觉得你对他淡淡的，要是能偶尔打个只球，明确的表达一下内心，那就更好了。#心理[话题]# #mbti[话题]# #infp[话题]# #infp日常[话题]# #infp和infj[话题]# #INFJ[话题]# #infj的世界[话题]# #我的日常[话题]# #温暖治愈[话题]# '],
    ['电动车远光灯刺眼反击', '电动车在这些情况下是全责哦！注意了哦！\n骑电动车一定要小心这些情况哦！[暗中观察R]#交通事故理赔[话题]# #交通事故[话题]# #法律求助[话题]# #电动车[话题]# #法律咨询[话题]# #交通安全[话题]# '],
    ['电动车远光灯刺眼反击', '支付宝上这个骑行险有用吗\n#车险[话题]# #电动车骑行险[话题]# #支付宝[话题]# '],
    ['牛奶品牌排名', '注意❗注意❗杯子到货，买就送📢\n杯子已经到货啦📣\n依然还是35.9到手2瓶4斤装的鲜奶，在额外赠送2个mini玻璃杯哦~\n顺丰冷链发货，包邮到家📦\n\xa0#乍甸牛奶[话题]#\xa0\xa0#云南游[话题]#\xa0\xa0#鲜奶[话题]#\xa0\xa0#可爱杯子[话题]#\xa0\xa0#杯子分享[话题]#\xa0\xa0#杯子控必入系列[话题]#\xa0\xa0#我就是个杯子控[话题]#\xa0\xa0#杯子[话题]#\xa0\xa0#鲜奶酸奶怎么挑[话题]#\xa0\xa0#鲜奶推荐[话题]#\xa0\xa0#鲜牛乳[话题]#\xa0\xa0#乍甸牛奶福利种草官[话题]#\xa0\xa0#乍甸牛奶也很好喝[话题]#\xa0\xa0#乍甸鲜奶[话题]#\xa0\t\n'],
    ['mbti人格', 'INFP小蝴蝶🦋请谨慎破防😅\n感觉本infp的确心灵上有些许脆弱，在面对朋友或者家人，别人的一句批评或者不认同，会影响我一天的心情～感觉这个习惯跟刻在骨子里一样，一边安慰自己，却一边焦虑😮\u200d💨很难想象有时候自己却很开朗乐观，其实内心很脆弱，一点就破⊙﹏⊙\n\t\n内容纯属娱乐🌚如有雷同纯属巧合🌚\n请大家对号入座哈哈哈哈😂#MBTI16型人格[话题]# #mbti梗图[话题]# #infp精神世界[话题]# #infp日常[话题]# '],
]
scores = model.predict(pairs)
print(scores)
# [0.3757 0.2134 0.1656 0.179  0.272 ]

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'infp男和infj女',
    [
        '谁懂啊啊啊‼️💗INFP×INFJ相处好戳我\n绿老头和小蝴蝶都是高敏易碎人群，消息秒回啊，朋友圈点赞啊，这种细微且偏爱的行为，都能让对方感到安心和减少内耗。\n小蝴蝶的话，有事情就不要藏着掖着，对绿老头也不要有过多的试探行为，有问题就直接问，直接说，只要把事情拖到情绪都上来了，那这问题就更难解决了。因为绿老头只是看起来淡然，但其实他们的内心是很火热躁动的，他们只是不好意思主动表达。绿老头得学会调整，明知道小蝴蝶容易想多内耗，那就多主动表现情绪，不要让小蝴蝶觉得你对他淡淡的，要是能偶尔打个只球，明确的表达一下内心，那就更好了。#心理[话题]# #mbti[话题]# #infp[话题]# #infp日常[话题]# #infp和infj[话题]# #INFJ[话题]# #infj的世界[话题]# #我的日常[话题]# #温暖治愈[话题]# ',
        '电动车在这些情况下是全责哦！注意了哦！\n骑电动车一定要小心这些情况哦！[暗中观察R]#交通事故理赔[话题]# #交通事故[话题]# #法律求助[话题]# #电动车[话题]# #法律咨询[话题]# #交通安全[话题]# ',
        '支付宝上这个骑行险有用吗\n#车险[话题]# #电动车骑行险[话题]# #支付宝[话题]# ',
        '注意❗注意❗杯子到货，买就送📢\n杯子已经到货啦📣\n依然还是35.9到手2瓶4斤装的鲜奶，在额外赠送2个mini玻璃杯哦~\n顺丰冷链发货，包邮到家📦\n\xa0#乍甸牛奶[话题]#\xa0\xa0#云南游[话题]#\xa0\xa0#鲜奶[话题]#\xa0\xa0#可爱杯子[话题]#\xa0\xa0#杯子分享[话题]#\xa0\xa0#杯子控必入系列[话题]#\xa0\xa0#我就是个杯子控[话题]#\xa0\xa0#杯子[话题]#\xa0\xa0#鲜奶酸奶怎么挑[话题]#\xa0\xa0#鲜奶推荐[话题]#\xa0\xa0#鲜牛乳[话题]#\xa0\xa0#乍甸牛奶福利种草官[话题]#\xa0\xa0#乍甸牛奶也很好喝[话题]#\xa0\xa0#乍甸鲜奶[话题]#\xa0\t\n',
        'INFP小蝴蝶🦋请谨慎破防😅\n感觉本infp的确心灵上有些许脆弱，在面对朋友或者家人，别人的一句批评或者不认同，会影响我一天的心情～感觉这个习惯跟刻在骨子里一样，一边安慰自己，却一边焦虑😮\u200d💨很难想象有时候自己却很开朗乐观，其实内心很脆弱，一点就破⊙﹏⊙\n\t\n内容纯属娱乐🌚如有雷同纯属巧合🌚\n请大家对号入座哈哈哈哈😂#MBTI16型人格[话题]# #mbti梗图[话题]# #infp精神世界[话题]# #infp日常[话题]# ',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Evaluation

Metrics

Cross Encoder Reranking

Metric query_note_test query_note_val
map 0.4565 0.4572
mrr@10 0.5284 0.5247
ndcg@10 0.5271 0.5328

Training Details

Training Dataset

Unnamed Dataset

  • Size: 769,572 training samples
  • Columns: sentence1, sentence2, and label
  • Approximate statistics based on the first 100 samples:
    sentence1 sentence2 label
    type string string float
    modality text text
    details
    • min: 5 tokens
    • mean: 7.92 tokens
    • max: 13 tokens
    • min: 18 tokens
    • mean: 208.42 tokens
    • max: 512 tokens
    • min: 0.0
    • mean: 0.0
    • max: 0.0
  • Samples:
    sentence1 sentence2 label
    infp男和infj女 谁懂啊啊啊‼️💗INFP×INFJ相处好戳我
    绿老头和小蝴蝶都是高敏易碎人群，消息秒回啊，朋友圈点赞啊，这种细微且偏爱的行为，都能让对方感到安心和减少内耗。
    小蝴蝶的话，有事情就不要藏着掖着，对绿老头也不要有过多的试探行为，有问题就直接问，直接说，只要把事情拖到情绪都上来了，那这问题就更难解决了。因为绿老头只是看起来淡然，但其实他们的内心是很火热躁动的，他们只是不好意思主动表达。绿老头得学会调整，明知道小蝴蝶容易想多内耗，那就多主动表现情绪，不要让小蝴蝶觉得你对他淡淡的，要是能偶尔打个只球，明确的表达一下内心，那就更好了。#心理[话题]# #mbti[话题]# #infp[话题]# #infp日常[话题]# #infp和infj[话题]# #INFJ[话题]# #infj的世界[话题]# #我的日常[话题]# #温暖治愈[话题]#     		0.0
    电动车远光灯刺眼反击 电动车在这些情况下是全责哦！注意了哦！
    骑电动车一定要小心这些情况哦！[暗中观察R]#交通事故理赔[话题]# #交通事故[话题]# #法律求助[话题]# #电动车[话题]# #法律咨询[话题]# #交通安全[话题]#     		0.0
    电动车远光灯刺眼反击 支付宝上这个骑行险有用吗
    #车险[话题]# #电动车骑行险[话题]# #支付宝[话题]#     		0.0
  • Loss: BinaryCrossEntropyLoss with these parameters:
    {
    "activation_fn": "torch.nn.modules.linear.Identity",
    "pos_weight": null
}

Training Hyperparameters

Non-Default Hyperparameters

  • per_device_train_batch_size: 32
  • learning_rate: 0.0001
  • weight_decay: 0.01
  • num_train_epochs: 1
  • lr_scheduler_type: cosine
  • warmup_ratio: 0.05
  • seed: 3407
  • bf16: True

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • prediction_loss_only: True
  • per_device_train_batch_size: 32
  • per_device_eval_batch_size: 8
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 1
  • eval_accumulation_steps: None
  • torch_empty_cache_steps: None
  • learning_rate: 0.0001
  • weight_decay: 0.01
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1.0
  • num_train_epochs: 1
  • max_steps: -1
  • lr_scheduler_type: cosine
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.05
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 3407
  • data_seed: None
  • jit_mode_eval: False
  • use_ipex: False
  • bf16: True
  • fp16: False
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • parallelism_config: None
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch_fused
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: None
  • hub_always_push: False
  • hub_revision: None
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • include_for_metrics: []
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: False
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • use_liger_kernel: False
  • liger_kernel_config: None
  • eval_use_gather_object: False
  • average_tokens_across_devices: False
  • prompts: None
  • batch_sampler: batch_sampler
  • multi_dataset_batch_sampler: proportional
  • router_mapping: {}
  • learning_rate_mapping: {}

Training Logs

Click to expand
Epoch Step Training Loss query_note_test_ndcg@10 query_note_val_ndcg@10
-1 -1 - 0.5206 -
0.7048 16950 0.6932 - -
0.7069 17000 0.5323 - -
0.7089 17050 0.5209 - -
0.7110 17100 0.5281 - -
0.7131 17150 0.5267 - -
0.7152 17200 0.5124 - -
0.7173 17250 0.5199 - -
0.7193 17300 0.5189 - -
0.7214 17350 0.5059 - -
0.7235 17400 0.5326 - -
0.7256 17450 0.5159 - -
0.7277 17500 0.5246 - -
0.7297 17550 0.5128 - -
0.7318 17600 0.5078 - -
0.7339 17650 0.4966 - -
0.7360 17700 0.506 - -
0.7380 17750 0.499 - -
0.7401 17800 0.5069 - -
0.7422 17850 0.5387 - -
0.7443 17900 0.5124 - -
0.7464 17950 0.522 - -
0.7484 18000 0.5103 - -
0.7505 18050 0.5217 - -
0.7526 18100 0.4939 - -
0.7547 18150 0.5151 - -
0.7568 18200 0.4804 - -
0.7588 18250 0.4969 - -
0.7609 18300 0.5277 - -
0.7630 18350 0.5143 - -
0.7651 18400 0.5063 - -
0.7672 18450 0.4899 - -
0.7692 18500 0.5144 - -
0.7713 18550 0.528 - -
0.7734 18600 0.5032 - -
0.7755 18650 0.4956 - -
0.7775 18700 0.5144 - -
0.7796 18750 0.5145 - -
0.7817 18800 0.4971 - -
0.7838 18850 0.5188 - -
0.7859 18900 0.501 - -
0.7879 18950 0.4892 - -
0.7900 19000 0.4752 - -
0.7921 19050 0.4984 - -
0.7942 19100 0.5001 - -
0.7963 19150 0.4809 - -
0.7983 19200 0.5085 - -
0.8004 19250 0.5122 - -
0.8025 19300 0.5122 - -
0.8046 19350 0.4909 - -
0.8067 19400 0.5341 - -
0.8087 19450 0.5147 - -
0.8108 19500 0.5095 - -
0.8129 19550 0.4945 - -
0.8150 19600 0.4971 - -
0.8170 19650 0.4967 - -
0.8191 19700 0.5108 - -
0.8212 19750 0.4983 - -
0.8233 19800 0.5154 - -
0.8254 19850 0.5214 - -
0.8274 19900 0.4953 - -
0.8295 19950 0.5079 - -
0.8316 20000 0.5252 - -
0.8337 20050 0.4966 - -
0.8358 20100 0.492 - -
0.8378 20150 0.5065 - -
0.8399 20200 0.4825 - -
0.8420 20250 0.4879 - -
0.8441 20300 0.5351 - -
0.8462 20350 0.4904 - -
0.8482 20400 0.5141 - -
0.8503 20450 0.5146 - -
0.8524 20500 0.508 - -
0.8545 20550 0.5271 - -
0.8565 20600 0.5057 - -
0.8586 20650 0.4757 - -
0.8607 20700 0.5151 - -
0.8628 20750 0.486 - -
0.8649 20800 0.4908 - -
0.8669 20850 0.5287 - -
0.8690 20900 0.5223 - -
0.8711 20950 0.5086 - -
0.8732 21000 0.5066 - -
0.8753 21050 0.5042 - -
0.8773 21100 0.5032 - -
0.8794 21150 0.5123 - -
0.8815 21200 0.4825 - -
0.8836 21250 0.5222 - -
0.8857 21300 0.5044 - -
0.8877 21350 0.5034 - -
0.8898 21400 0.5193 - -
0.8919 21450 0.4975 - -
0.8940 21500 0.4754 - -
0.8960 21550 0.5209 - -
0.8981 21600 0.5024 - -
0.9002 21650 0.5206 - -
0.9023 21700 0.5032 - -
0.9044 21750 0.5264 - -
0.9064 21800 0.499 - -
0.9085 21850 0.4967 - -
0.9106 21900 0.491 - -
0.9127 21950 0.5056 - -
0.9148 22000 0.4996 - -
0.9168 22050 0.4994 - -
0.9189 22100 0.5254 - -
0.9210 22150 0.5034 - -
0.9231 22200 0.5123 - -
0.9252 22250 0.4956 - -
0.9272 22300 0.5194 - -
0.9293 22350 0.474 - -
0.9314 22400 0.4842 - -
0.9335 22450 0.4914 - -
0.9356 22500 0.4925 - -
0.9376 22550 0.4938 - -
0.9397 22600 0.5086 - -
0.9418 22650 0.457 - -
0.9439 22700 0.5185 - -
0.9459 22750 0.5268 - -
0.9480 22800 0.4872 - -
0.9501 22850 0.5048 - -
0.9522 22900 0.5103 - -
0.9543 22950 0.5236 - -
0.9563 23000 0.5049 - -
0.9584 23050 0.5041 - -
0.9605 23100 0.5066 - -
0.9626 23150 0.5206 - -
0.9647 23200 0.4732 - -
0.9667 23250 0.4881 - -
0.9688 23300 0.5099 - -
0.9709 23350 0.5226 - -
0.9730 23400 0.5322 - -
0.9751 23450 0.4993 - -
0.9771 23500 0.4856 - -
0.9792 23550 0.4727 - -
0.9813 23600 0.5093 - -
0.9834 23650 0.5073 - -
0.9854 23700 0.5153 - -
0.9875 23750 0.4979 - -
0.9896 23800 0.4961 - -
0.9917 23850 0.5093 - -
0.9938 23900 0.4811 - -
0.9958 23950 0.5008 - -
0.9979 24000 0.5151 - -
1.0 24050 0.5318 - 0.5328
-1 -1 - 0.5271 -

Training Time

  • Training: 57.2 minutes

Framework Versions

  • Python: 3.12.3
  • Sentence Transformers: 5.5.1
  • Transformers: 4.56.2
  • PyTorch: 2.11.0+cu128
  • Accelerate: 1.13.0
  • Datasets: 4.3.0
  • Tokenizers: 0.22.2

Additional Resources

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}