Add new CrossEncoder model

README.md

@@ -0,0 +1,516 @@
+---
+language:
+- en
+tags:
+- sentence-transformers
+- cross-encoder
+- reranker
+- generated_from_trainer
+- dataset_size:1990000
+- loss:MarginMSELoss
+base_model: microsoft/MiniLM-L12-H384-uncased
+datasets:
+- sentence-transformers/msmarco
+pipeline_tag: text-ranking
+library_name: sentence-transformers
+metrics:
+- map
+- mrr@10
+- ndcg@10
+model-index:
+- name: CrossEncoder based on microsoft/MiniLM-L12-H384-uncased
+  results:
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoMSMARCO R100
+      type: NanoMSMARCO_R100
+    metrics:
+    - type: map
+      value: 0.5676
+      name: Map
+    - type: mrr@10
+      value: 0.5587
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.6364
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoNFCorpus R100
+      type: NanoNFCorpus_R100
+    metrics:
+    - type: map
+      value: 0.3529
+      name: Map
+    - type: mrr@10
+      value: 0.6198
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.4158
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoNQ R100
+      type: NanoNQ_R100
+    metrics:
+    - type: map
+      value: 0.7124
+      name: Map
+    - type: mrr@10
+      value: 0.727
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.7557
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-nano-beir
+      name: Cross Encoder Nano BEIR
+    dataset:
+      name: NanoBEIR R100 mean
+      type: NanoBEIR_R100_mean
+    metrics:
+    - type: map
+      value: 0.5443
+      name: Map
+    - type: mrr@10
+      value: 0.6352
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.6026
+      name: Ndcg@10
+---
+
+# CrossEncoder based on microsoft/MiniLM-L12-H384-uncased
+
+This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model finetuned from [microsoft/MiniLM-L12-H384-uncased](https://huggingface.co/microsoft/MiniLM-L12-H384-uncased) on the [msmarco](https://huggingface.co/datasets/sentence-transformers/msmarco) dataset using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Cross Encoder
+- **Base model:** [microsoft/MiniLM-L12-H384-uncased](https://huggingface.co/microsoft/MiniLM-L12-H384-uncased) <!-- at revision 44acabbec0ef496f6dbc93adadea57f376b7c0ec -->
+- **Maximum Sequence Length:** 512 tokens
+- **Number of Output Labels:** 1 label
+- **Training Dataset:**
+    - [msmarco](https://huggingface.co/datasets/sentence-transformers/msmarco)
+- **Language:** en
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Documentation:** [Cross Encoder Documentation](https://www.sbert.net/docs/cross_encoder/usage/usage.html)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)
+- **Hugging Face:** [Cross Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=cross-encoder)
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import CrossEncoder
+
+# Download from the 🤗 Hub
+model = CrossEncoder("alantang2025/reranker-MiniLM-L12-H384-uncased-msmarco-margin-mse")
+# Get scores for pairs of texts
+pairs = [
+    ['what is mcafee sidewinder control center', 'McAfee Firewall Enterprise Control Center. McAfee Firewall Enterprise Control Center (CommandCenterâ\x84¢) provides a central interface for simplifying the management of multiple McAfee Firewall Enterprise (Sidewinder®) appliances.'],
+    ['where is cork ie', 'Cork, Irish Corcaigh (â\x80\x9cMarshâ\x80\x9d), seaport and seat of County Cork, in the province of Munster, Ireland. It is located at the head of Cork Harbour on the River Lee. Cork is, after Dublin, the Irish republicâ\x80\x99s second largest conurbation. The city is administratively independent of the county.'],
+    ['what is a embedded computer system', 'Embedded Computer Systems. An embedded system is a special-purpose system in which the computer is completely encapsulated by the device it controls. Unlike a general-purpose computer, such as a personal computer, an embedded system performs pre-defined tasks, usually with very specific requirements.'],
+    ['who is kennedy space center named after', "Cecil replies: Sure, but let's get our facts straight: they didn't change the name of the space center, they changed the name of the cape â\x80\x94 i.e., the land under the space center (or under part of it, anyway). The NASA launch facility continues to be known as the John F. Kennedy Space Center. The whole confusing business got started back on November 27, 1963, shortly after JFK's assassination, when Lyndon Johnson was casting about for a suitable memorial for the slain president."],
+    ['how to calculate protein requirement for a horse', 'Although protein is listed as a percentage on feed tags, the National Research Councilâ\x80\x99s (NRC) latest recommendation for horses lists the protein requirement unit in grams. To understand how much protein to feed a horse, we need to do some math and further reading.Many people think that a 30 percent protein feed is way too much for a horse to handle. If one looks a little closer, the tag of Hubbard Lifeâ\x80\x99s 30% Supplement indicates that a 1,100-pound horse should consume one pound per day.Thirty percent of one pound is 136 grams of protein.verage mixed hay is about 17 percent protein. Feeding 15 pounds of hay alone can deliver 1,157 grams of protein to a horse. Understanding the protein content and feeding rates of all the feed sources a horse has access to is key to achieving the proper protein ratio.'],
+]
+scores = model.predict(pairs)
+print(scores.shape)
+# (5,)
+
+# Or rank different texts based on similarity to a single text
+ranks = model.rank(
+    'what is mcafee sidewinder control center',
+    [
+        'McAfee Firewall Enterprise Control Center. McAfee Firewall Enterprise Control Center (CommandCenterâ\x84¢) provides a central interface for simplifying the management of multiple McAfee Firewall Enterprise (Sidewinder®) appliances.',
+        'Cork, Irish Corcaigh (â\x80\x9cMarshâ\x80\x9d), seaport and seat of County Cork, in the province of Munster, Ireland. It is located at the head of Cork Harbour on the River Lee. Cork is, after Dublin, the Irish republicâ\x80\x99s second largest conurbation. The city is administratively independent of the county.',
+        'Embedded Computer Systems. An embedded system is a special-purpose system in which the computer is completely encapsulated by the device it controls. Unlike a general-purpose computer, such as a personal computer, an embedded system performs pre-defined tasks, usually with very specific requirements.',
+        "Cecil replies: Sure, but let's get our facts straight: they didn't change the name of the space center, they changed the name of the cape â\x80\x94 i.e., the land under the space center (or under part of it, anyway). The NASA launch facility continues to be known as the John F. Kennedy Space Center. The whole confusing business got started back on November 27, 1963, shortly after JFK's assassination, when Lyndon Johnson was casting about for a suitable memorial for the slain president.",
+        'Although protein is listed as a percentage on feed tags, the National Research Councilâ\x80\x99s (NRC) latest recommendation for horses lists the protein requirement unit in grams. To understand how much protein to feed a horse, we need to do some math and further reading.Many people think that a 30 percent protein feed is way too much for a horse to handle. If one looks a little closer, the tag of Hubbard Lifeâ\x80\x99s 30% Supplement indicates that a 1,100-pound horse should consume one pound per day.Thirty percent of one pound is 136 grams of protein.verage mixed hay is about 17 percent protein. Feeding 15 pounds of hay alone can deliver 1,157 grams of protein to a horse. Understanding the protein content and feeding rates of all the feed sources a horse has access to is key to achieving the proper protein ratio.',
+    ]
+)
+# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Cross Encoder Reranking
+
+* Datasets: `NanoMSMARCO_R100`, `NanoNFCorpus_R100` and `NanoNQ_R100`
+* Evaluated with [<code>CrossEncoderRerankingEvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderRerankingEvaluator) with these parameters:
+  ```json
+  {
+      "at_k": 10,
+      "always_rerank_positives": true
+  }
+  ```
+
+| Metric      | NanoMSMARCO_R100     | NanoNFCorpus_R100    | NanoNQ_R100          |
+|:------------|:---------------------|:---------------------|:---------------------|
+| map         | 0.5676 (+0.0780)     | 0.3529 (+0.0919)     | 0.7124 (+0.2928)     |
+| mrr@10      | 0.5587 (+0.0812)     | 0.6198 (+0.1200)     | 0.7270 (+0.3003)     |
+| **ndcg@10** | **0.6364 (+0.0960)** | **0.4158 (+0.0907)** | **0.7557 (+0.2551)** |
+
+#### Cross Encoder Nano BEIR
+
+* Dataset: `NanoBEIR_R100_mean`
+* Evaluated with [<code>CrossEncoderNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderNanoBEIREvaluator) with these parameters:
+  ```json
+  {
+      "dataset_names": [
+          "msmarco",
+          "nfcorpus",
+          "nq"
+      ],
+      "rerank_k": 100,
+      "at_k": 10,
+      "always_rerank_positives": true
+  }
+  ```
+
+| Metric      | Value                |
+|:------------|:---------------------|
+| map         | 0.5443 (+0.1542)     |
+| mrr@10      | 0.6352 (+0.1672)     |
+| **ndcg@10** | **0.6026 (+0.1473)** |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### msmarco
+
+* Dataset: [msmarco](https://huggingface.co/datasets/sentence-transformers/msmarco) at [9e329ed](https://huggingface.co/datasets/sentence-transformers/msmarco/tree/9e329ed2e649c9d37b0d91dd6b764ff6fe671d83)
+* Size: 1,990,000 training samples
+* Columns: <code>score</code>, <code>query</code>, <code>positive</code>, and <code>negative</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | score                                                               | query                                                                                           | positive                                                                                         | negative                                                                                          |
+  |:--------|:--------------------------------------------------------------------|:------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------|
+  | type    | float                                                               | string                                                                                          | string                                                                                           | string                                                                                            |
+  | details | <ul><li>min: -1.47</li><li>mean: 13.31</li><li>max: 22.71</li></ul> | <ul><li>min: 10 characters</li><li>mean: 33.44 characters</li><li>max: 105 characters</li></ul> | <ul><li>min: 75 characters</li><li>mean: 355.82 characters</li><li>max: 952 characters</li></ul> | <ul><li>min: 85 characters</li><li>mean: 338.69 characters</li><li>max: 1197 characters</li></ul> |
+* Samples:
+  | score                           | query                                                    | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | negative                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
+  |:--------------------------------|:---------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>20.983789285024006</code> | <code>what is a french bulldog</code>                    | <code>The French Bulldog is a small breed of domestic dog. Frenchies were the result in the 1800s of a cross between bulldog ancestors imported from England and local ratters in Paris, France.In the UK, they moved up to become the fourth most popular registered dog by 2014.he modern French Bulldog breed descends directly from the dogs of the Molossians, an ancient Greek tribe.</code>                                                                                                                                                                                 | <code>The many faces of Duke Dog: Bulldog, human mascot, statue and cartoon. In all likelihood, James Madison University is the only college or university in the country whose athletic teams draw their nickname from the name of the school's president.</code>                                                                                                                                                                                                                                                                                                                                                     |
+  | <code>20.930927753448486</code> | <code>difference between hostname and domain name</code> | <code>hostname is the name given to the end-point (the machine in question) and will be used to identify it over DNS if that is configured. domain is the name given to the 'network' it will be required to reach the network from an external point (like the Internet)</code>                                                                                                                                                                                                                                                                                                   | <code>Determining the domain of a function. Determine the domains of functions according to various considerations. 1  How to determine the domain of a radical function (example) (Video). 2  How to determine the domain of algebraic functions (examples) (Video). 3  Domain of algebraic functions (Exercise).</code>                                                                                                                                                                                                                                                                                              |
+  | <code>12.55810809135437</code>  | <code>will sod come back if overwatered</code>           | <code>Root Death. Pull up pieces of the sod in different areas after the sod has established for a few weeks. Healthy sod that gets the right amount of water will resist the pull because the roots are healthy and growing. Overwatered sod will come right up because its roots are dying or dead.f the soil is wet and mushy, you're watering too much. Let the sod dry out until the top 2 inches of soil are dry and crumbly. Washington State University recommends early morning watering, which allows soil and grass blades to dry out thoroughly during the day.</code> | <code>THE ADVANTAGES OF HYDRO SEEDING! Hydroseeding is a fast, cost effective way to have a new lawn that will turn your neighbors green with envy. Hydroseeding costs only a little more than old fashioned methods using dry seeding techniques combined with a messy straw mulch.ydro Seeding mulch adds to the humus content of a lawn as it decomposes. The bacterial action of straw will leach nitrogen from the soil as it decomposes. ADVANTAGES OF HYDROSEEDING OVER SOD Sod is a good solution to the need for a new lawn. It is expensive. Sod generally costs 3 to 5 times more than hydroseeding.</code> |
+* Loss: [<code>MarginMSELoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#marginmseloss) with these parameters:
+  ```json
+  {
+      "activation_fn": "torch.nn.modules.linear.Identity"
+  }
+  ```
+
+### Evaluation Dataset
+
+#### msmarco
+
+* Dataset: [msmarco](https://huggingface.co/datasets/sentence-transformers/msmarco) at [9e329ed](https://huggingface.co/datasets/sentence-transformers/msmarco/tree/9e329ed2e649c9d37b0d91dd6b764ff6fe671d83)
+* Size: 10,000 evaluation samples
+* Columns: <code>score</code>, <code>query</code>, <code>positive</code>, and <code>negative</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | score                                                               | query                                                                                          | positive                                                                                         | negative                                                                                         |
+  |:--------|:--------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------|
+  | type    | float                                                               | string                                                                                         | string                                                                                           | string                                                                                           |
+  | details | <ul><li>min: -1.29</li><li>mean: 13.64</li><li>max: 22.17</li></ul> | <ul><li>min: 11 characters</li><li>mean: 33.82 characters</li><li>max: 98 characters</li></ul> | <ul><li>min: 55 characters</li><li>mean: 360.68 characters</li><li>max: 990 characters</li></ul> | <ul><li>min: 57 characters</li><li>mean: 340.77 characters</li><li>max: 964 characters</li></ul> |
+* Samples:
+  | score                           | query                                                 | positive                                                                                                                                                                                                                                                                                                                   | negative                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
+  |:--------------------------------|:------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>18.650261163711548</code> | <code>what is mcafee sidewinder control center</code> | <code>McAfee Firewall Enterprise Control Center. McAfee Firewall Enterprise Control Center (CommandCenter™) provides a central interface for simplifying the management of multiple McAfee Firewall Enterprise (Sidewinder®) appliances.</code>                                                                         | <code>JSC's Mission Control — now formally known as the Christopher C. Kraft, Jr. Mission Control Center — has helped plan, support and operate every NASA human spaceflight mission since 1965.</code>                                                                                                                                                                                                                                                                                                 |
+  | <code>14.571923971176147</code> | <code>where is cork ie</code>                         | <code>Cork, Irish Corcaigh (“Marsh”), seaport and seat of County Cork, in the province of Munster, Ireland. It is located at the head of Cork Harbour on the River Lee. Cork is, after Dublin, the Irish republic’s second largest conurbation. The city is administratively independent of the county.</code>       | <code>suberin. (biochemistry). A fatty substance found in many plant cell walls, especially cork.</code>                                                                                                                                                                                                                                                                                                                                                                                                    |
+  | <code>18.947856426239014</code> | <code>what is a embedded computer system</code>       | <code>Embedded Computer Systems. An embedded system is a special-purpose system in which the computer is completely encapsulated by the device it controls. Unlike a general-purpose computer, such as a personal computer, an embedded system performs pre-defined tasks, usually with very specific requirements.</code> | <code>Confidence votes 13.4K. It means that an application (a software, a computer program) is not compatible (it doesn't work well or doesn't work at all)with the operating system running on a computer. Applications are software programs that you can download onto your computer and some mobile phones. A computer game, for example, is an application. An MP3 player system … on your computer or phone is also an application.. + 12 others found this useful. Ashley Reeves. Answered.</code> |
+* Loss: [<code>MarginMSELoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#marginmseloss) with these parameters:
+  ```json
+  {
+      "activation_fn": "torch.nn.modules.linear.Identity"
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `learning_rate`: 8e-06
+- `num_train_epochs`: 1
+- `warmup_ratio`: 0.1
+- `seed`: 12
+- `bf16`: True
+- `dataloader_num_workers`: 4
+- `load_best_model_at_end`: True
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 8e-06
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 1
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 12
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: True
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 4
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: True
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `tp_size`: 0
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: proportional
+- `router_mapping`: {}
+- `learning_rate_mapping`: {}
+
+</details>
+
+### Training Logs
+| Epoch     | Step       | Training Loss | Validation Loss | NanoMSMARCO_R100_ndcg@10 | NanoNFCorpus_R100_ndcg@10 | NanoNQ_R100_ndcg@10  | NanoBEIR_R100_mean_ndcg@10 |
+|:---------:|:----------:|:-------------:|:---------------:|:------------------------:|:-------------------------:|:--------------------:|:--------------------------:|
+| -1        | -1         | -             | -               | 0.0185 (-0.5220)         | 0.1835 (-0.1415)          | 0.0374 (-0.4632)     | 0.0798 (-0.3756)           |
+| 0.0000    | 1          | 183.8208      | -               | -                        | -                         | -                    | -                          |
+| 0.0322    | 4000       | 148.9381      | -               | -                        | -                         | -                    | -                          |
+| 0.0643    | 8000       | 27.4436       | -               | -                        | -                         | -                    | -                          |
+| 0.0965    | 12000      | 8.3597        | -               | -                        | -                         | -                    | -                          |
+| 0.1286    | 16000      | 6.374         | -               | -                        | -                         | -                    | -                          |
+| 0.1608    | 20000      | 5.4584        | 4.7464          | 0.6619 (+0.1215)         | 0.3947 (+0.0696)          | 0.7165 (+0.2158)     | 0.5910 (+0.1356)           |
+| 0.1930    | 24000      | 4.9963        | -               | -                        | -                         | -                    | -                          |
+| 0.2251    | 28000      | 4.6626        | -               | -                        | -                         | -                    | -                          |
+| 0.2573    | 32000      | 4.3576        | -               | -                        | -                         | -                    | -                          |
+| 0.2894    | 36000      | 4.1884        | -               | -                        | -                         | -                    | -                          |
+| 0.3216    | 40000      | 4.0283        | 4.4428          | 0.6594 (+0.1190)         | 0.3927 (+0.0676)          | 0.7435 (+0.2428)     | 0.5985 (+0.1432)           |
+| 0.3538    | 44000      | 3.8617        | -               | -                        | -                         | -                    | -                          |
+| 0.3859    | 48000      | 3.734         | -               | -                        | -                         | -                    | -                          |
+| 0.4181    | 52000      | 3.5985        | -               | -                        | -                         | -                    | -                          |
+| 0.4503    | 56000      | 3.6228        | -               | -                        | -                         | -                    | -                          |
+| 0.4824    | 60000      | 3.4607        | 3.4983          | 0.6285 (+0.0881)         | 0.4059 (+0.0809)          | 0.7429 (+0.2423)     | 0.5924 (+0.1371)           |
+| 0.5146    | 64000      | 3.4429        | -               | -                        | -                         | -                    | -                          |
+| 0.5467    | 68000      | 3.3256        | -               | -                        | -                         | -                    | -                          |
+| 0.5789    | 72000      | 3.2512        | -               | -                        | -                         | -                    | -                          |
+| 0.6111    | 76000      | 3.2302        | -               | -                        | -                         | -                    | -                          |
+| 0.6432    | 80000      | 3.1449        | 3.0647          | 0.6427 (+0.1023)         | 0.4070 (+0.0820)          | 0.7428 (+0.2421)     | 0.5975 (+0.1421)           |
+| 0.6754    | 84000      | 3.1304        | -               | -                        | -                         | -                    | -                          |
+| 0.7075    | 88000      | 3.0615        | -               | -                        | -                         | -                    | -                          |
+| 0.7397    | 92000      | 3.0513        | -               | -                        | -                         | -                    | -                          |
+| 0.7719    | 96000      | 3.0657        | -               | -                        | -                         | -                    | -                          |
+| **0.804** | **100000** | **2.9726**    | **3.0527**      | **0.6364 (+0.0960)**     | **0.4158 (+0.0907)**      | **0.7557 (+0.2551)** | **0.6026 (+0.1473)**       |
+| 0.8362    | 104000     | 2.9537        | -               | -                        | -                         | -                    | -                          |
+| 0.8683    | 108000     | 2.9453        | -               | -                        | -                         | -                    | -                          |
+| 0.9005    | 112000     | 2.8856        | -               | -                        | -                         | -                    | -                          |
+| 0.9327    | 116000     | 2.9301        | -               | -                        | -                         | -                    | -                          |
+| 0.9648    | 120000     | 2.8464        | 2.9493          | 0.6475 (+0.1070)         | 0.4105 (+0.0855)          | 0.7406 (+0.2399)     | 0.5995 (+0.1442)           |
+| 0.9970    | 124000     | 2.8968        | -               | -                        | -                         | -                    | -                          |
+| -1        | -1         | -             | -               | 0.6364 (+0.0960)         | 0.4158 (+0.0907)          | 0.7557 (+0.2551)     | 0.6026 (+0.1473)           |
+
+* The bold row denotes the saved checkpoint.
+
+### Framework Versions
+- Python: 3.12.9
+- Sentence Transformers: 5.1.2
+- Transformers: 4.51.3
+- PyTorch: 2.6.0
+- Accelerate: 1.11.0
+- Datasets: 3.6.0
+- Tokenizers: 0.21.4
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### MarginMSELoss
+```bibtex
+@misc{hofstätter2021improving,
+    title={Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation},
+    author={Sebastian Hofstätter and Sophia Althammer and Michael Schröder and Mete Sertkan and Allan Hanbury},
+    year={2021},
+    eprint={2010.02666},
+    archivePrefix={arXiv},
+    primaryClass={cs.IR}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,34 @@
+{
+  "architectures": [
+    "BertForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 384,
+  "id2label": {
+    "0": "LABEL_0"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "label2id": {
+    "LABEL_0": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "sentence_transformers": {
+    "activation_fn": "torch.nn.modules.activation.Sigmoid",
+    "version": "5.1.2"
+  },
+  "torch_dtype": "float32",
+  "transformers_version": "4.51.3",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:543fc56633f20a9f46725e1d69d7fa8d5a66f395bc0b8c7895dd224b4a911ec3
+size 133464836

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}