Jpcosta90 commited on
Commit
61dfe51
·
verified ·
1 Parent(s): e8dc2e1

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +90 -0
README.md ADDED
@@ -0,0 +1,90 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: pt
3
+ license: mit
4
+ tags:
5
+ - document-understanding
6
+ - document-retrieval
7
+ - metric-learning
8
+ - siamese-network
9
+ - internvl
10
+ - cosdoc
11
+ datasets:
12
+ - LA-CDIP
13
+ metrics:
14
+ - eer
15
+ model-index:
16
+ - name: CosDoc (Attention nq=2)
17
+ results:
18
+ - task:
19
+ type: document-retrieval
20
+ dataset:
21
+ name: LA-CDIP
22
+ type: la-cdip
23
+ metrics:
24
+ - type: eer
25
+ value: 0.0107
26
+ ---
27
+
28
+ # CosDoc — Attention nq=2
29
+
30
+ **CosDoc** is a visual document embedding model trained with supervised metric learning
31
+ and hard-example selection via a Reinforcement Learning professor network.
32
+
33
+ Pooler variant: **Attention nq=2** — Multi-query attention pooler, num_queries=2.
34
+
35
+ ## Architecture
36
+
37
+ | Component | Value |
38
+ |---|---|
39
+ | Backbone | InternVL3-2B (`OpenGVLab/InternVL3-2B`) |
40
+ | Cut layer | 27 |
41
+ | Pooler | attention (num_queries=2) |
42
+ | Embedding dim | 1536 |
43
+ | Loss | Sub-Center CosFace (m=0.35, s=32, k=3) |
44
+ | Embedding prompt | `<image> Analyze this document` |
45
+
46
+ ## Performance (LA-CDIP, full validation pairs)
47
+
48
+ | Dataset | EER |
49
+ |---|---|
50
+ | LA-CDIP (5-fold CV) | **1.07%** |
51
+
52
+ Source run: `Sprint3b_S0_subcenter_cosface_seed42_noinit_nq2_fase1_E10`
53
+
54
+ ## Usage
55
+
56
+ ```python
57
+ import torch
58
+ from huggingface_hub import hf_hub_download
59
+ from cavl_doc.models.backbone_loader import load_model
60
+ from cavl_doc.models.modeling_cavl import build_cavl_model
61
+
62
+ device = "cuda" if torch.cuda.is_available() else "cpu"
63
+
64
+ # Download fine-tuned weights
65
+ ckpt_path = hf_hub_download(repo_id="Jpcosta90/cosdoc-nq2", filename="best_model.pt")
66
+ ckpt = torch.load(ckpt_path, map_location=device, weights_only=False)
67
+ cfg = ckpt["config"]
68
+
69
+ backbone, _, tokenizer, _, _ = load_model("InternVL3-2B")
70
+ model = build_cavl_model(
71
+ backbone=backbone,
72
+ cut_layer=cfg["cut_layer"],
73
+ pooler_type=cfg["pooler_type"],
74
+ num_queries=cfg.get("num_queries", 1),
75
+ )
76
+ model.pool.load_state_dict(ckpt["siam_pool"])
77
+ model.head.load_state_dict(ckpt["siam_head"])
78
+ model.eval().to(device)
79
+ ```
80
+
81
+ ## Citation
82
+
83
+ ```bibtex
84
+ @misc{cosdoc2026,
85
+ title = {CosDoc: Cosine-Margin Document Embeddings with RL-guided Hard Mining},
86
+ author = {Costa, João Paulo},
87
+ year = {2026},
88
+ url = {https://huggingface.co/Jpcosta90/cosdoc-nq2}
89
+ }
90
+ ```