Add new SentenceTransformer model

1_Pooling/config.json

@@ -0,0 +1,5 @@
+{
+    "embedding_dimension": 768,
+    "pooling_mode": "mean",
+    "include_prompt": true
+}

README.md

@@ -0,0 +1,81 @@
+---
+language:
+- pt
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- portuguese
+- financial
+- cvm
+- bertimbau
+base_model: neuralmind/bert-base-portuguese-cased
+license: mit
+---
+
+# cvm-bertimbau-sentence-transformer
+
+Fine-tuned [BERTimbau](https://huggingface.co/neuralmind/bert-base-portuguese-cased)
+sentence transformer for dense retrieval over Brazilian public company filings (CVM ITR/DFP).
+Part of the [CVM Filing Intelligence System](https://github.com/conderafael/cvm-intelligence).
+
+## Training
+
+| Parameter | Value |
+|---|---|
+| Base model | `neuralmind/bert-base-portuguese-cased` |
+| Loss | `MultipleNegativesRankingLoss` |
+| Training pairs | 14,500 (adjacent same-section chunk pairs from 686 CVM filings) |
+| Epochs | 10 |
+| Batch size | 16 (effective 64 with gradient accumulation ×4) |
+| Mixed precision | fp16 |
+| Max sequence length | 256 tokens |
+| Hardware | NVIDIA RTX A1000 (6 GB VRAM), ~2 hours |
+| Initial loss | 2.201 (step 50) |
+| Final loss | 0.115 (step 2,270) |
+
+Training data: 97,138 management commentary chunks from 49 B3 large-cap companies
+(Petrobras, Vale, Itaú, Bradesco, Ambev, etc.), 2022–2025. Pairs are adjacent
+paragraphs within the same section of the same filing.
+
+## Retrieval Results
+
+Evaluated on 94 synthetic queries over the 97,138-chunk corpus
+(dense-only configuration):
+
+| Metric | Value |
+|---|---|
+| Recall@5 | 0.057 |
+| Recall@10 | 0.071 |
+| MRR | 0.100 |
+| NDCG@10 | 0.063 |
+
+**Note:** The model underperforms BM25 on query–document retrieval because it was
+fine-tuned with doc–doc contrastive pairs. Query–doc performance improves significantly
+with GPL (Generative Pseudo Labeling) fine-tuning using synthetic query–chunk pairs.
+
+## Usage
+
+```python
+from sentence_transformers import SentenceTransformer
+
+model = SentenceTransformer("conderafael/cvm-bertimbau-sentence-transformer")
+
+# Encode a single passage
+embeddings = model.encode(["Receita líquida cresceu 12% no trimestre"])
+
+# Encode a batch
+texts = [
+    "O EBITDA ajustado atingiu R$ 4,2 bilhões no 3T24.",
+    "A Companhia mantém posição conservadora de hedge cambial.",
+]
+embeddings = model.encode(texts, normalize_embeddings=True)
+print(embeddings.shape)  # (2, 768)
+```
+
+## Limitations
+
+- Trained on Portuguese-language financial filings only; degrades on other domains.
+- Max sequence length 256 tokens; longer passages are truncated.
+- Query-time performance is below doc-time performance due to training objective mismatch
+  (doc–doc pairs vs. query–doc retrieval).

config.json

@@ -0,0 +1,35 @@
+{
+  "add_cross_attention": false,
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": null,
+  "classifier_dropout": null,
+  "directionality": "bidi",
+  "dtype": "float32",
+  "eos_token_id": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "is_decoder": false,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "output_past": true,
+  "pad_token_id": 0,
+  "pooler_fc_size": 768,
+  "pooler_num_attention_heads": 12,
+  "pooler_num_fc_layers": 3,
+  "pooler_size_per_head": 128,
+  "pooler_type": "first_token_transform",
+  "tie_word_embeddings": true,
+  "transformers_version": "5.5.4",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 29794
+}

config_sentence_transformers.json

@@ -0,0 +1,14 @@
+{
+  "__version__": {
+    "pytorch": "2.11.0+cu130",
+    "sentence_transformers": "5.4.1",
+    "transformers": "5.5.4"
+  },
+  "default_prompt_name": null,
+  "model_type": "SentenceTransformer",
+  "prompts": {
+    "document": "",
+    "query": ""
+  },
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:715058b832103f95a9f0f24233329cc42eec5896cbde3ad2eae5646a2d4f8714
+size 435714880

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.base.modules.transformer.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.sentence_transformer.modules.pooling.Pooling"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,10 @@
+{
+    "transformer_task": "feature-extraction",
+    "modality_config": {
+        "text": {
+            "method": "forward",
+            "method_output_name": "last_hidden_state"
+        }
+    },
+    "module_output_name": "token_embeddings"
+}

tokenizer_config.json

@@ -0,0 +1,21 @@
+{
+  "backend": "tokenizers",
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "is_local": true,
+  "mask_token": "[MASK]",
+  "max_length": 256,
+  "model_max_length": 256,
+  "pad_to_multiple_of": null,
+  "pad_token": "[PAD]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}