Add new SentenceTransformer model

1_Pooling/config.json

@@ -0,0 +1,12 @@
+{
+    "word_embedding_dimension": 768,
+    "pooling_mode_cls_token": false,
+    "pooling_mode_mean_tokens": false,
+    "pooling_mode_max_tokens": false,
+    "pooling_mode_mean_sqrt_len_tokens": false,
+    "pooling_mode_weightedmean_tokens": false,
+    "pooling_mode_lasttoken": false,
+    "pooling_mode_lmk": true,
+    "lmk_token_id": -1,
+    "include_prompt": true
+}

README.md

@@ -0,0 +1,542 @@
+---
+language:
+- en
+license: apache-2.0
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- dense
+- generated_from_trainer
+- dataset_size:90000
+- loss:CachedMultipleNegativesRankingLoss
+base_model: microsoft/mpnet-base
+widget:
+- source_sentence: how do i change the password for the app store?
+  sentences:
+  - Tap Settings > [your name] > Password & Security. Tap Change Password. Enter your
+    current password or device passcode, then enter a new password and confirm the
+    new password. Tap Change or Change Password.
+  - Unless others attending the funeral know you for wearing bow ties, we recommend
+    sticking to a necktie for funerals. We love bow ties, but they are statement pieces
+    that can be a distraction at a funeral. ... Solid black bow ties, black tone-on-tone
+    bow ties or dark solid color bow ties are the most appropriate choice.
+  - 'Convert fraction (ratio) 5 / 35 Answer: 14.285714285714%'
+- source_sentence: how many molecules are present in one mole of a substance?
+  sentences:
+  - A year is divided into 12 months in the modern-day Gregorian calendar. The months
+    are either 28, 29, 30, or 31 days long. Calendar with 12 months.
+  - Latex is considered one of the best materials for mattress construction due to
+    its natural softness, breathability and resiliency. Latex is most commonly found
+    in luxury mattresses with high price points, and may be used in both the comfort
+    layer and support core.
+  - 'The mole allows scientists to calculate the number of elementary entities (usually
+    atoms or molecules) in a certain mass of a given substance. Avogadro''s number
+    is an absolute number: there are 6.022×1023 elementary entities in 1 mole. This
+    can also be written as 6.022×1023 mol-1.'
+- source_sentence: how is baking powder and baking soda different?
+  sentences:
+  - While both products appear similar, they're certainly not the same. Baking soda
+    is sodium bicarbonate, which requires an acid and a liquid to become activated
+    and help baked goods rise. Conversely, baking powder includes sodium bicarbonate,
+    as well as an acid. It only needs a liquid to become activated.
+  - Donna Sheridan is dead, long live her hotel on the made-up Greek island of Kalokairi.
+    You can rest assured that Streep, who still appears in new scenes in the movie,
+    was fine with the decision — and according to director Ol Parker, it wasn't because
+    she had other scheduling obligations.
+  - Gyan (Sanskrit), a Sanskrit word that roughly translates to 'knowledge' in English.
+- source_sentence: what are the differences between food webs and food chains?
+  sentences:
+  - FOOD WEBS show how plants and animals are connected in many ways to help them
+    all survive. FOOD CHAINS follow just one path of energy as animals find food.
+  - 'The following instructions apply to making a PDF text-searchable in Adobe Acrobat
+    Professional or Standard: Click on Tools > Text Recognition > In This File. The
+    Recognize Text popup box opens. Select All pages, then click OK.'
+  - Most sets are worth less than the $5 retail, though some like Civil War can command
+    over $10, and the Aliens set of three are on eBay stores for $50. The rare US
+    version of the action fleet Aliens jump ship has been asking over $70.
+- source_sentence: can you get period pains while pregnant?
+  sentences:
+  - The most confusing part about SACs is that your SAC score DOES NOT go towards
+    your study score, at least not directly. In fact, your SAC score is merely used
+    to rank you in your class.
+  - For an uncomplicated UTI that occurs when you're otherwise healthy, your doctor
+    may recommend a shorter course of treatment, such as taking an antibiotic for
+    one to three days. But whether this short course of treatment is enough to treat
+    your infection depends on your particular symptoms and medical history.
+  - 'Some women will experience more cramping as they start to go into menopause.
+    Pregnancy: Early in pregnancy, you may experience mild or light cramping. These
+    cramps will probably feel like the light cramps you get during your period, but
+    they''ll be in your lower stomach or lower back.'
+datasets:
+- sentence-transformers/gooaq
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+metrics:
+- cosine_accuracy@1
+- cosine_accuracy@3
+- cosine_accuracy@5
+- cosine_accuracy@10
+- cosine_precision@1
+- cosine_precision@3
+- cosine_precision@5
+- cosine_precision@10
+- cosine_recall@1
+- cosine_recall@3
+- cosine_recall@5
+- cosine_recall@10
+- cosine_ndcg@10
+- cosine_mrr@10
+- cosine_map@100
+model-index:
+- name: MPNet base with Landmark Pooling trained on GooAQ triplets using CachedMultipleNegativesRankingLoss
+    with GradCache
+  results:
+  - task:
+      type: information-retrieval
+      name: Information Retrieval
+    dataset:
+      name: gooaq dev
+      type: gooaq-dev
+    metrics:
+    - type: cosine_accuracy@1
+      value: 0.7581
+      name: Cosine Accuracy@1
+    - type: cosine_accuracy@3
+      value: 0.896
+      name: Cosine Accuracy@3
+    - type: cosine_accuracy@5
+      value: 0.9329
+      name: Cosine Accuracy@5
+    - type: cosine_accuracy@10
+      value: 0.9641
+      name: Cosine Accuracy@10
+    - type: cosine_precision@1
+      value: 0.7581
+      name: Cosine Precision@1
+    - type: cosine_precision@3
+      value: 0.29866666666666664
+      name: Cosine Precision@3
+    - type: cosine_precision@5
+      value: 0.18658
+      name: Cosine Precision@5
+    - type: cosine_precision@10
+      value: 0.09641000000000001
+      name: Cosine Precision@10
+    - type: cosine_recall@1
+      value: 0.7581
+      name: Cosine Recall@1
+    - type: cosine_recall@3
+      value: 0.896
+      name: Cosine Recall@3
+    - type: cosine_recall@5
+      value: 0.9329
+      name: Cosine Recall@5
+    - type: cosine_recall@10
+      value: 0.9641
+      name: Cosine Recall@10
+    - type: cosine_ndcg@10
+      value: 0.8654027716772075
+      name: Cosine Ndcg@10
+    - type: cosine_mrr@10
+      value: 0.8332606746031707
+      name: Cosine Mrr@10
+    - type: cosine_map@100
+      value: 0.8349243908994848
+      name: Cosine Map@100
+---
+
+# MPNet base with Landmark Pooling trained on GooAQ triplets using CachedMultipleNegativesRankingLoss with GradCache
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [microsoft/mpnet-base](https://huggingface.co/microsoft/mpnet-base) on the [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [microsoft/mpnet-base](https://huggingface.co/microsoft/mpnet-base) <!-- at revision 6996ce1e91bd2a9c7d7f61daec37463394f73f09 -->
+- **Maximum Sequence Length:** 512 tokens
+- **Output Dimensionality:** 768 dimensions
+- **Similarity Function:** Cosine Similarity
+- **Training Dataset:**
+    - [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq)
+- **Language:** en
+- **License:** apache-2.0
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'splitter_type': 'variable', 'splitter_granularity': [32, 64, 128, 256], 'lmk_token_id': 2, 'architecture': 'MPNetModel'})
+  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'pooling_mode_lmk': True, 'lmk_token_id': -1, 'include_prompt': True})
+  (2): Normalize()
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("tomaarsen/mpnet-base-gooaq-cmnrl-1024bs-lmk-gran-per-batch")
+# Run inference
+queries = [
+    "can you get period pains while pregnant?",
+]
+documents = [
+    "Some women will experience more cramping as they start to go into menopause. Pregnancy: Early in pregnancy, you may experience mild or light cramping. These cramps will probably feel like the light cramps you get during your period, but they'll be in your lower stomach or lower back.",
+    "For an uncomplicated UTI that occurs when you're otherwise healthy, your doctor may recommend a shorter course of treatment, such as taking an antibiotic for one to three days. But whether this short course of treatment is enough to treat your infection depends on your particular symptoms and medical history.",
+    'The most confusing part about SACs is that your SAC score DOES NOT go towards your study score, at least not directly. In fact, your SAC score is merely used to rank you in your class.',
+]
+query_embeddings = model.encode_query(queries)
+document_embeddings = model.encode_document(documents)
+print(query_embeddings.shape, document_embeddings.shape)
+# [1, 768] [3, 768]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(query_embeddings, document_embeddings)
+print(similarities)
+# tensor([[ 0.7430,  0.1732, -0.0864]])
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Information Retrieval
+
+* Dataset: `gooaq-dev`
+* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
+
+| Metric              | Value      |
+|:--------------------|:-----------|
+| cosine_accuracy@1   | 0.7581     |
+| cosine_accuracy@3   | 0.896      |
+| cosine_accuracy@5   | 0.9329     |
+| cosine_accuracy@10  | 0.9641     |
+| cosine_precision@1  | 0.7581     |
+| cosine_precision@3  | 0.2987     |
+| cosine_precision@5  | 0.1866     |
+| cosine_precision@10 | 0.0964     |
+| cosine_recall@1     | 0.7581     |
+| cosine_recall@3     | 0.896      |
+| cosine_recall@5     | 0.9329     |
+| cosine_recall@10    | 0.9641     |
+| **cosine_ndcg@10**  | **0.8654** |
+| cosine_mrr@10       | 0.8333     |
+| cosine_map@100      | 0.8349     |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### gooaq
+
+* Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
+* Size: 90,000 training samples
+* Columns: <code>question</code> and <code>answer</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | question                                                                          | answer                                                                              |
+  |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                              |
+  | details | <ul><li>min: 8 tokens</li><li>mean: 11.83 tokens</li><li>max: 20 tokens</li></ul> | <ul><li>min: 15 tokens</li><li>mean: 61.75 tokens</li><li>max: 185 tokens</li></ul> |
+* Samples:
+  | question                                                            | answer                                                                                                                                                                                                                                                                                                                                       |
+  |:--------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>how long does halifax take to transfer mortgage funds?</code> | <code>Bear in mind that the speed of application will vary depending on your own personal circumstances and the lender's present day-to-day performance. In some cases, applications can be approved by the lender within 24 hours, while some can take weeks or even months.</code>                                                         |
+  | <code>can you get a false pregnancy test?</code>                    | <code>In very rare cases, you can have a false-positive result. This means you're not pregnant but the test says you are. You could have a false-positive result if you have blood or protein in your pee. Certain drugs, such as tranquilizers, anticonvulsants, hypnotics, and fertility drugs, could cause false-positive results.</code> |
+  | <code>are ahead of its time?</code>                                 | <code>Definition of ahead of one's/its time : too advanced or modern to be understood or appreciated during the time when one lives or works As a director, he was ahead of his time.</code>                                                                                                                                                 |
+* Loss: [<code>CachedMultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedmultiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 20.0,
+      "similarity_fct": "cos_sim",
+      "mini_batch_size": 64,
+      "gather_across_devices": false
+  }
+  ```
+
+### Evaluation Dataset
+
+#### gooaq
+
+* Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
+* Size: 10,000 evaluation samples
+* Columns: <code>question</code> and <code>answer</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | question                                                                          | answer                                                                             |
+  |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                             |
+  | details | <ul><li>min: 8 tokens</li><li>mean: 11.93 tokens</li><li>max: 25 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 61.2 tokens</li><li>max: 128 tokens</li></ul> |
+* Samples:
+  | question                                                         | answer                                                                                                                                                                                                                                                                                                                                                        |
+  |:-----------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>should you take ibuprofen with high blood pressure?</code> | <code>In general, people with high blood pressure should use acetaminophen or possibly aspirin for over-the-counter pain relief. Unless your health care provider has said it's OK, you should not use ibuprofen, ketoprofen, or naproxen sodium. If aspirin or acetaminophen doesn't help with your pain, call your doctor.</code>                           |
+  | <code>how old do you have to be to work in sc?</code>            | <code>The general minimum age of employment for South Carolina youth is 14, although the state allows younger children who are performers to work in show business. If their families are agricultural workers, children younger than age 14 may also participate in farm labor.</code>                                                                       |
+  | <code>how to write a topic proposal for a research paper?</code> | <code>['Write down the main topic of your paper. ... ', 'Write two or three short sentences under the main topic that explain why you chose that topic. ... ', 'Write a thesis sentence that states the angle and purpose of your research paper. ... ', 'List the items you will cover in the body of the paper that support your thesis statement.']</code> |
+* Loss: [<code>CachedMultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedmultiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 20.0,
+      "similarity_fct": "cos_sim",
+      "mini_batch_size": 64,
+      "gather_across_devices": false
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 1024
+- `per_device_eval_batch_size`: 1024
+- `learning_rate`: 8e-05
+- `num_train_epochs`: 1
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0.1
+- `bf16`: True
+- `batch_sampler`: no_duplicates
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 1024
+- `per_device_eval_batch_size`: 1024
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 8e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 1
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: None
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0.1
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `enable_jit_checkpoint`: False
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `use_cpu`: False
+- `seed`: 42
+- `data_seed`: None
+- `bf16`: True
+- `fp16`: False
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: -1
+- `ddp_backend`: None
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `parallelism_config`: None
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch_fused
+- `optim_args`: None
+- `group_by_length`: False
+- `length_column_name`: length
+- `project`: huggingface
+- `trackio_space_id`: trackio
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `hub_revision`: None
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `include_num_input_tokens_seen`: no
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `liger_kernel_config`: None
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: True
+- `use_cache`: False
+- `prompts`: None
+- `batch_sampler`: no_duplicates
+- `multi_dataset_batch_sampler`: proportional
+- `router_mapping`: {}
+- `learning_rate_mapping`: {}
+
+</details>
+
+### Training Logs
+| Epoch  | Step | Training Loss | Validation Loss | gooaq-dev_cosine_ndcg@10 |
+|:------:|:----:|:-------------:|:---------------:|:------------------------:|
+| -1     | -1   | -             | -               | 0.2180                   |
+| 0.0114 | 1    | 6.3995        | -               | -                        |
+| 0.0568 | 5    | 5.8563        | -               | -                        |
+| 0.1023 | 9    | -             | 1.1550          | 0.6291                   |
+| 0.1136 | 10   | 2.4891        | -               | -                        |
+| 0.1705 | 15   | 1.0676        | -               | -                        |
+| 0.2045 | 18   | -             | 0.5168          | 0.7867                   |
+| 0.2273 | 20   | 0.7828        | -               | -                        |
+| 0.2841 | 25   | 0.6288        | -               | -                        |
+| 0.3068 | 27   | -             | 0.3813          | 0.8246                   |
+| 0.3409 | 30   | 0.5358        | -               | -                        |
+| 0.3977 | 35   | 0.4802        | -               | -                        |
+| 0.4091 | 36   | -             | 0.3268          | 0.8399                   |
+| 0.4545 | 40   | 0.4491        | -               | -                        |
+| 0.5114 | 45   | 0.4258        | 0.2941          | 0.8490                   |
+| 0.5682 | 50   | 0.4255        | -               | -                        |
+| 0.6136 | 54   | -             | 0.2762          | 0.8572                   |
+| 0.625  | 55   | 0.4056        | -               | -                        |
+| 0.6818 | 60   | 0.3826        | -               | -                        |
+| 0.7159 | 63   | -             | 0.2675          | 0.8580                   |
+| 0.7386 | 65   | 0.3455        | -               | -                        |
+| 0.7955 | 70   | 0.3722        | -               | -                        |
+| 0.8182 | 72   | -             | 0.2540          | 0.8631                   |
+| 0.8523 | 75   | 0.3543        | -               | -                        |
+| 0.9091 | 80   | 0.3519        | -               | -                        |
+| 0.9205 | 81   | -             | 0.2502          | 0.8633                   |
+| 0.9659 | 85   | 0.3422        | -               | -                        |
+| -1     | -1   | -             | -               | 0.8654                   |
+
+
+### Framework Versions
+- Python: 3.11.6
+- Sentence Transformers: 5.3.0.dev0
+- Transformers: 5.0.1.dev0
+- PyTorch: 2.10.0+cu126
+- Accelerate: 1.12.0
+- Datasets: 4.3.0
+- Tokenizers: 0.22.2
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### CachedMultipleNegativesRankingLoss
+```bibtex
+@misc{gao2021scaling,
+    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
+    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
+    year={2021},
+    eprint={2101.06983},
+    archivePrefix={arXiv},
+    primaryClass={cs.LG}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,24 @@
+{
+  "architectures": [
+    "MPNetModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "dtype": "float32",
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "mpnet",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "relative_attention_num_buckets": 32,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.0.1.dev0",
+  "vocab_size": 30527
+}

config_sentence_transformers.json

@@ -0,0 +1,14 @@
+{
+  "model_type": "SentenceTransformer",
+  "__version__": {
+    "sentence_transformers": "5.3.0.dev0",
+    "transformers": "5.0.1.dev0",
+    "pytorch": "2.10.0+cu126"
+  },
+  "prompts": {
+    "query": "",
+    "document": ""
+  },
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:121aaf57c238f0ea23bd124d70702137b9fc460a86cc4959990e6ba78ccb1707
+size 437967648

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.LandmarkTransformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,12 @@
+{
+    "max_seq_length": 512,
+    "do_lower_case": false,
+    "splitter_type": "variable",
+    "splitter_granularity": [
+        32,
+        64,
+        128,
+        256
+    ],
+    "lmk_token_id": 2
+}

tokenizer_config.json

@@ -0,0 +1,16 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "cls_token": "<s>",
+  "do_lower_case": true,
+  "eos_token": "</s>",
+  "is_local": false,
+  "mask_token": "<mask>",
+  "model_max_length": 512,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "MPNetTokenizer",
+  "unk_token": "[UNK]"
+}