Add new SentenceTransformer model

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+    "word_embedding_dimension": 1024,
+    "pooling_mode_cls_token": true,
+    "pooling_mode_mean_tokens": false,
+    "pooling_mode_max_tokens": false,
+    "pooling_mode_mean_sqrt_len_tokens": false,
+    "pooling_mode_weightedmean_tokens": false,
+    "pooling_mode_lasttoken": false,
+    "include_prompt": true
+}

README.md

@@ -0,0 +1,706 @@
+---
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- dense
+- generated_from_trainer
+- dataset_size:160
+- loss:MatryoshkaLoss
+- loss:MultipleNegativesRankingLoss
+base_model: Snowflake/snowflake-arctic-embed-l
+widget:
+- source_sentence: What is the total cost to process 68,000 images based on the provided
+    token calculations?
+  sentences:
+  - 'One way to think about these models is an extension of the chain-of-thought prompting
+    trick, first explored in the May 2022 paper Large Language Models are Zero-Shot
+    Reasoners.
+
+    This is that trick where, if you get a model to talk out loud about a problem
+    it’s solving, you often get a result which the model would not have achieved otherwise.
+
+    o1 takes this process and further bakes it into the model itself. The details
+    are somewhat obfuscated: o1 models spend “reasoning tokens” thinking through the
+    problem that are not directly visible to the user (though the ChatGPT UI shows
+    a summary of them), then outputs a final result.'
+  - 'Each photo would need 260 input tokens and around 100 output tokens.
+
+    260 * 68,000 = 17,680,000 input tokens
+
+    17,680,000 * $0.0375/million = $0.66
+
+    100 * 68,000 = 6,800,000 output tokens
+
+    6,800,000 * $0.15/million = $1.02
+
+    That’s a total cost of $1.68 to process 68,000 images. That’s so absurdly cheap
+    I had to run the numbers three times to confirm I got it right.
+
+    How good are those descriptions? Here’s what I got from this command:
+
+    llm -m gemini-1.5-flash-8b-latest describe -a IMG_1825.jpeg'
+  - 'So far, I think they’re a net positive. I’ve used them on a personal level to
+    improve my productivity (and entertain myself) in all sorts of different ways.
+    I think people who learn how to use them effectively can gain a significant boost
+    to their quality of life.
+
+    A lot of people are yet to be sold on their value! Some think their negatives
+    outweigh their positives, some think they are all hot air, and some even think
+    they represent an existential threat to humanity.
+
+    They’re actually quite easy to build
+
+    The most surprising thing we’ve learned about LLMs this year is that they’re actually
+    quite easy to build.'
+- source_sentence: What model of MacBook Pro does the author own, and what is its
+    storage capacity?
+  sentences:
+  - 'On the other hand, as software engineers we are better placed to take advantage
+    of this than anyone else. We’ve all been given weird coding interns—we can use
+    our deep knowledge to prompt them to solve coding problems more effectively than
+    anyone else can.
+
+    The ethics of this space remain diabolically complex
+
+    In September last year Andy Baio and I produced the first major story on the unlicensed
+    training data behind Stable Diffusion.
+
+    Since then, almost every major LLM (and most of the image generation models) have
+    also been trained on unlicensed data.'
+  - 'My personal laptop is a 64GB M2 MacBook Pro from 2023. It’s a powerful machine,
+    but it’s also nearly two years old now—and crucially it’s the same laptop I’ve
+    been using ever since I first ran an LLM on my computer back in March 2023 (see
+    Large language models are having their Stable Diffusion moment).
+
+    That same laptop that could just about run a GPT-3-class model in March last year
+    has now run multiple GPT-4 class models! Some of my notes on that:'
+  - 'I think people who complain that LLM improvement has slowed are often missing
+    the enormous advances in these multi-modal models. Being able to run prompts against
+    images (and audio and video) is a fascinating new way to apply these models.
+
+    Voice and live camera mode are science fiction come to life
+
+    The audio and live video modes that have started to emerge deserve a special mention.
+
+    The ability to talk to ChatGPT first arrived in September 2023, but it was mostly
+    an illusion: OpenAI used their excellent Whisper speech-to-text model and a new
+    text-to-speech model (creatively named tts-1) to enable conversations with the
+    ChatGPT mobile apps, but the actual model just saw text.'
+- source_sentence: How do longer inputs enhance the problem-solving capabilities of
+    an LLM?
+  sentences:
+  - 'This remains astonishing to me. I thought a model with the capabilities and output
+    quality of GPT-4 needed a datacenter class server with one or more $40,000+ GPUs.
+
+    These models take up enough of my 64GB of RAM that I don’t run them often—they
+    don’t leave much room for anything else.
+
+    The fact that they run at all is a testament to the incredible training and inference
+    performance gains that we’ve figured out over the past year. It turns out there
+    was a lot of low-hanging fruit to be harvested in terms of model efficiency. I
+    expect there’s still more to come.'
+  - 'Now add a walrus: Prompt engineering in DALL-E 3
+
+    32.8k
+
+    41.2k
+
+
+
+    Web LLM runs the vicuna-7b Large Language Model entirely in your browser, and
+    it’s very impressive
+
+    32.5k
+
+    38.2k
+
+
+
+    ChatGPT can’t access the internet, even though it really looks like it can
+
+    30.5k
+
+    34.2k
+
+
+
+    Stanford Alpaca, and the acceleration of on-device large language model development
+
+    29.7k
+
+    35.7k
+
+
+
+    Run Llama 2 on your own Mac using LLM and Homebrew
+
+    27.9k
+
+    33.6k
+
+
+
+    Midjourney 5.1
+
+    26.7k
+
+    33.4k
+
+
+
+    Think of language models like ChatGPT as a “calculator for words”
+
+    25k
+
+    31.8k
+
+
+
+    Multi-modal prompt injection image attacks against GPT-4V
+
+    23.7k
+
+    27.4k'
+  - 'Longer inputs dramatically increase the scope of problems that can be solved
+    with an LLM: you can now throw in an entire book and ask questions about its contents,
+    but more importantly you can feed in a lot of example code to help the model correctly
+    solve a coding problem. LLM use-cases that involve long inputs are far more interesting
+    to me than short prompts that rely purely on the information already baked into
+    the model weights. Many of my tools were built using this pattern.'
+- source_sentence: What is the cost of OpenAI's most expensive model, o1, in terms
+    of $/mTok?
+  sentences:
+  - 'Today $30/mTok gets you OpenAI’s most expensive model, o1. GPT-4o is $2.50 (12x
+    cheaper than GPT-4) and GPT-4o mini is $0.15/mTok—200x cheaper than GPT-4, nearly
+    7x cheaper than GPT-3.5 and massively more capable than that model.
+
+    Other model providers charge even less. Anthropic’s Claude 3 Haiku (from March,
+    but still their cheapest model) is $0.25/mTok. Google’s Gemini 1.5 Flash is $0.075/mTok
+    and their Gemini 1.5 Flash 8B is $0.0375/mTok—that’s 27x cheaper than GPT-3.5
+    Turbo last year.
+
+    I’ve been tracking these pricing changes under my llm-pricing tag.'
+  - 'Gemini 1.5 Pro also illustrated one of the key themes of 2024: increased context
+    lengths. Last year most models accepted 4,096 or 8,192 tokens, with the notable
+    exception of Claude 2.1 which accepted 200,000. Today every serious provider has
+    a 100,000+ token model, and Google’s Gemini series accepts up to 2 million.'
+  - 'Nothing yet from Anthropic or Meta but I would be very surprised if they don’t
+    have their own inference-scaling models in the works. Meta published a relevant
+    paper Training Large Language Models to Reason in a Continuous Latent Space in
+    December.
+
+    Was the best currently available LLM trained in China for less than $6m?
+
+    Not quite, but almost! It does make for a great attention-grabbing headline.
+
+    The big news to end the year was the release of DeepSeek v3—dropped on Hugging
+    Face on Christmas Day without so much as a README file, then followed by documentation
+    and a paper the day after that.'
+- source_sentence: What features does Google’s Gemini offer in relation to audio input
+    and output?
+  sentences:
+  - 'Your browser does not support the audio element.
+
+
+    OpenAI aren’t the only group with a multi-modal audio model. Google’s Gemini also
+    accepts audio input, and the Google Gemini apps can speak in a similar way to
+    ChatGPT now. Amazon also pre-announced voice mode for Amazon Nova, but that’s
+    meant to roll out in Q1 of 2025.
+
+    Google’s NotebookLM, released in September, took audio output to a new level by
+    producing spookily realistic conversations between two “podcast hosts” about anything
+    you fed into their tool. They later added custom instructions, so naturally I
+    turned them into pelicans:
+
+
+
+    Your browser does not support the audio element.'
+  - 'Against this photo of butterflies at the California Academy of Sciences:
+
+
+
+    A shallow dish, likely a hummingbird or butterfly feeder, is red.  Pieces of orange
+    slices of fruit are visible inside the dish.
+
+    Two butterflies are positioned in the feeder, one is a dark brown/black butterfly
+    with white/cream-colored markings.  The other is a large, brown butterfly with
+    patterns of lighter brown, beige, and black markings, including prominent eye
+    spots. The larger brown butterfly appears to be feeding on the fruit.'
+  - 'I also gave a bunch of talks and podcast appearances. I’ve started habitually
+    turning my talks into annotated presentations—here are my best from 2023:
+
+
+    Prompt injection explained, with video, slides, and a transcript
+
+    Catching up on the weird world of LLMs
+
+    Making Large Language Models work for you
+
+    Open questions for AI engineering
+
+    Embeddings: What they are and why they matter
+
+    Financial sustainability for open source projects at GitHub Universe
+
+
+    And in podcasts:
+
+
+
+    What AI can do for you on the Theory of Change
+
+
+    Working in public on Path to Citus Con
+
+
+    LLMs break the internet on the Changelog
+
+
+    Talking Large Language Models on Rooftop Ruby
+
+
+    Thoughts on the OpenAI board situation on Newsroom Robots'
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+metrics:
+- cosine_accuracy@1
+- cosine_accuracy@3
+- cosine_accuracy@5
+- cosine_accuracy@10
+- cosine_precision@1
+- cosine_precision@3
+- cosine_precision@5
+- cosine_precision@10
+- cosine_recall@1
+- cosine_recall@3
+- cosine_recall@5
+- cosine_recall@10
+- cosine_ndcg@10
+- cosine_mrr@10
+- cosine_map@100
+model-index:
+- name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-l
+  results:
+  - task:
+      type: information-retrieval
+      name: Information Retrieval
+    dataset:
+      name: Unknown
+      type: unknown
+    metrics:
+    - type: cosine_accuracy@1
+      value: 1.0
+      name: Cosine Accuracy@1
+    - type: cosine_accuracy@3
+      value: 1.0
+      name: Cosine Accuracy@3
+    - type: cosine_accuracy@5
+      value: 1.0
+      name: Cosine Accuracy@5
+    - type: cosine_accuracy@10
+      value: 1.0
+      name: Cosine Accuracy@10
+    - type: cosine_precision@1
+      value: 1.0
+      name: Cosine Precision@1
+    - type: cosine_precision@3
+      value: 0.33333333333333326
+      name: Cosine Precision@3
+    - type: cosine_precision@5
+      value: 0.20000000000000004
+      name: Cosine Precision@5
+    - type: cosine_precision@10
+      value: 0.10000000000000002
+      name: Cosine Precision@10
+    - type: cosine_recall@1
+      value: 1.0
+      name: Cosine Recall@1
+    - type: cosine_recall@3
+      value: 1.0
+      name: Cosine Recall@3
+    - type: cosine_recall@5
+      value: 1.0
+      name: Cosine Recall@5
+    - type: cosine_recall@10
+      value: 1.0
+      name: Cosine Recall@10
+    - type: cosine_ndcg@10
+      value: 1.0
+      name: Cosine Ndcg@10
+    - type: cosine_mrr@10
+      value: 1.0
+      name: Cosine Mrr@10
+    - type: cosine_map@100
+      value: 1.0
+      name: Cosine Map@100
+---
+
+# SentenceTransformer based on Snowflake/snowflake-arctic-embed-l
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Snowflake/snowflake-arctic-embed-l](https://huggingface.co/Snowflake/snowflake-arctic-embed-l). It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [Snowflake/snowflake-arctic-embed-l](https://huggingface.co/Snowflake/snowflake-arctic-embed-l) <!-- at revision d8fb21ca8d905d2832ee8b96c894d3298964346b -->
+- **Maximum Sequence Length:** 512 tokens
+- **Output Dimensionality:** 1024 dimensions
+- **Similarity Function:** Cosine Similarity
+<!-- - **Training Dataset:** Unknown -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
+  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+  (2): Normalize()
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("Maximgolubov/legal-ft-2")
+# Run inference
+queries = [
+    "What features does Google\u2019s Gemini offer in relation to audio input and output?",
+]
+documents = [
+    'Your browser does not support the audio element.\n\nOpenAI aren’t the only group with a multi-modal audio model. Google’s Gemini also accepts audio input, and the Google Gemini apps can speak in a similar way to ChatGPT now. Amazon also pre-announced voice mode for Amazon Nova, but that’s meant to roll out in Q1 of 2025.\nGoogle’s NotebookLM, released in September, took audio output to a new level by producing spookily realistic conversations between two “podcast hosts” about anything you fed into their tool. They later added custom instructions, so naturally I turned them into pelicans:\n\n\nYour browser does not support the audio element.',
+    'Against this photo of butterflies at the California Academy of Sciences:\n\n\nA shallow dish, likely a hummingbird or butterfly feeder, is red.  Pieces of orange slices of fruit are visible inside the dish.\nTwo butterflies are positioned in the feeder, one is a dark brown/black butterfly with white/cream-colored markings.  The other is a large, brown butterfly with patterns of lighter brown, beige, and black markings, including prominent eye spots. The larger brown butterfly appears to be feeding on the fruit.',
+    'I also gave a bunch of talks and podcast appearances. I’ve started habitually turning my talks into annotated presentations—here are my best from 2023:\n\nPrompt injection explained, with video, slides, and a transcript\nCatching up on the weird world of LLMs\nMaking Large Language Models work for you\nOpen questions for AI engineering\nEmbeddings: What they are and why they matter\nFinancial sustainability for open source projects at GitHub Universe\n\nAnd in podcasts:\n\n\nWhat AI can do for you on the Theory of Change\n\nWorking in public on Path to Citus Con\n\nLLMs break the internet on the Changelog\n\nTalking Large Language Models on Rooftop Ruby\n\nThoughts on the OpenAI board situation on Newsroom Robots',
+]
+query_embeddings = model.encode_query(queries)
+document_embeddings = model.encode_document(documents)
+print(query_embeddings.shape, document_embeddings.shape)
+# [1, 1024] [3, 1024]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(query_embeddings, document_embeddings)
+print(similarities)
+# tensor([[0.6603, 0.0568, 0.0967]])
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Information Retrieval
+
+* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
+
+| Metric              | Value   |
+|:--------------------|:--------|
+| cosine_accuracy@1   | 1.0     |
+| cosine_accuracy@3   | 1.0     |
+| cosine_accuracy@5   | 1.0     |
+| cosine_accuracy@10  | 1.0     |
+| cosine_precision@1  | 1.0     |
+| cosine_precision@3  | 0.3333  |
+| cosine_precision@5  | 0.2     |
+| cosine_precision@10 | 0.1     |
+| cosine_recall@1     | 1.0     |
+| cosine_recall@3     | 1.0     |
+| cosine_recall@5     | 1.0     |
+| cosine_recall@10    | 1.0     |
+| **cosine_ndcg@10**  | **1.0** |
+| cosine_mrr@10       | 1.0     |
+| cosine_map@100      | 1.0     |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### Unnamed Dataset
+
+* Size: 160 training samples
+* Columns: <code>sentence_0</code> and <code>sentence_1</code>
+* Approximate statistics based on the first 160 samples:
+  |         | sentence_0                                                                         | sentence_1                                                                           |
+  |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
+  | type    | string                                                                             | string                                                                               |
+  | details | <ul><li>min: 11 tokens</li><li>mean: 20.01 tokens</li><li>max: 36 tokens</li></ul> | <ul><li>min: 38 tokens</li><li>mean: 133.65 tokens</li><li>max: 214 tokens</li></ul> |
+* Samples:
+  | sentence_0                                                        | sentence_1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
+  |:------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>What years are listed in the provided context?</code>       | <code>Disclosures<br>Colophon<br>©<br>2002<br>2003<br>2004<br>2005<br>2006<br>2007<br>2008<br>2009<br>2010<br>2011<br>2012<br>2013<br>2014<br>2015<br>2016<br>2017<br>2018<br>2019<br>2020<br>2021<br>2022<br>2023<br>2024<br>2025<br>2026</code>                                                                                                                                                                                                                                                                                                                                                                                      |
+  | <code>What is the range of years mentioned in the context?</code> | <code>Disclosures<br>Colophon<br>©<br>2002<br>2003<br>2004<br>2005<br>2006<br>2007<br>2008<br>2009<br>2010<br>2011<br>2012<br>2013<br>2014<br>2015<br>2016<br>2017<br>2018<br>2019<br>2020<br>2021<br>2022<br>2023<br>2024<br>2025<br>2026</code>                                                                                                                                                                                                                                                                                                                                                                                      |
+  | <code>What model do I run on my iPhone?</code>                    | <code>I run a bunch of them on my laptop. I run Mistral 7B (a surprisingly great model) on my iPhone. You can install several different apps to get your own, local, completely private LLM. My own LLM project provides a CLI tool for running an array of different models via plugins.<br>You can even run them entirely in your browser using WebAssembly and the latest Chrome!<br>Hobbyists can build their own fine-tuned models<br>I said earlier that building an LLM was still out of reach of hobbyists. That may be true for training from scratch, but fine-tuning one of those models is another matter entirely.</code> |
+* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
+  ```json
+  {
+      "loss": "MultipleNegativesRankingLoss",
+      "matryoshka_dims": [
+          768,
+          512,
+          256,
+          128,
+          64
+      ],
+      "matryoshka_weights": [
+          1,
+          1,
+          1,
+          1,
+          1
+      ],
+      "n_dims_per_step": -1
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 10
+- `per_device_eval_batch_size`: 10
+- `num_train_epochs`: 10
+- `multi_dataset_batch_sampler`: round_robin
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 10
+- `per_device_eval_batch_size`: 10
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 5e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1
+- `num_train_epochs`: 10
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: None
+- `warmup_ratio`: None
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `enable_jit_checkpoint`: False
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `use_cpu`: False
+- `seed`: 42
+- `data_seed`: None
+- `bf16`: False
+- `fp16`: False
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: -1
+- `ddp_backend`: None
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `parallelism_config`: None
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch_fused
+- `optim_args`: None
+- `group_by_length`: False
+- `length_column_name`: length
+- `project`: huggingface
+- `trackio_space_id`: trackio
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `hub_revision`: None
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `include_num_input_tokens_seen`: no
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `liger_kernel_config`: None
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: True
+- `use_cache`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: round_robin
+- `router_mapping`: {}
+- `learning_rate_mapping`: {}
+
+</details>
+
+### Training Logs
+| Epoch | Step | cosine_ndcg@10 |
+|:-----:|:----:|:--------------:|
+| 1.0   | 16   | 0.975          |
+| 2.0   | 32   | 0.975          |
+| 3.0   | 48   | 0.9815         |
+| 3.125 | 50   | 0.9815         |
+| 4.0   | 64   | 1.0            |
+| 5.0   | 80   | 1.0            |
+| 6.0   | 96   | 1.0            |
+| 6.25  | 100  | 1.0            |
+| 7.0   | 112  | 1.0            |
+| 8.0   | 128  | 1.0            |
+| 9.0   | 144  | 1.0            |
+| 9.375 | 150  | 1.0            |
+| 10.0  | 160  | 1.0            |
+
+
+### Framework Versions
+- Python: 3.12.12
+- Sentence Transformers: 5.2.2
+- Transformers: 5.1.0
+- PyTorch: 2.9.0+cu128
+- Accelerate: 1.12.0
+- Datasets: 4.5.0
+- Tokenizers: 0.22.2
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### MatryoshkaLoss
+```bibtex
+@misc{kusupati2024matryoshka,
+    title={Matryoshka Representation Learning},
+    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
+    year={2024},
+    eprint={2205.13147},
+    archivePrefix={arXiv},
+    primaryClass={cs.LG}
+}
+```
+
+#### MultipleNegativesRankingLoss
+```bibtex
+@misc{henderson2017efficient,
+    title={Efficient Natural Language Response Suggestion for Smart Reply},
+    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
+    year={2017},
+    eprint={1705.00652},
+    archivePrefix={arXiv},
+    primaryClass={cs.CL}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,29 @@
+{
+  "add_cross_attention": false,
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": null,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "eos_token_id": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 1024,
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "is_decoder": false,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "tie_word_embeddings": true,
+  "transformers_version": "5.1.0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

config_sentence_transformers.json

@@ -0,0 +1,14 @@
+{
+  "__version__": {
+    "sentence_transformers": "5.2.2",
+    "transformers": "5.1.0",
+    "pytorch": "2.9.0+cu128"
+  },
+  "prompts": {
+    "query": "Represent this sentence for searching relevant passages: ",
+    "document": ""
+  },
+  "default_prompt_name": null,
+  "model_type": "SentenceTransformer",
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a60694cbfe998eb77858133d34f7e42566ea36f9f661e58644ad60d835dd8625
+size 1336413792

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+    "max_seq_length": 512,
+    "do_lower_case": false
+}

tokenizer_config.json

@@ -0,0 +1,22 @@
+{
+  "backend": "tokenizers",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "is_local": false,
+  "mask_token": "[MASK]",
+  "max_length": 512,
+  "model_max_length": 512,
+  "pad_to_multiple_of": null,
+  "pad_token": "[PAD]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}