Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,62 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
pipeline_tag: token-classification
|
| 4 |
+
tags:
|
| 5 |
+
- ner
|
| 6 |
+
- gliner
|
| 7 |
+
- data-use
|
| 8 |
+
- tiered
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# gliner-datause-displacement-withnonmention
|
| 12 |
+
|
| 13 |
+
Fine-tune of `urchade/gliner_large-v2.1` for data-use mention extraction with a
|
| 14 |
+
**single `DATA_MENTION` class**, trained on
|
| 15 |
+
[`rafmacalaba/data-use-mentions-tiered`](https://huggingface.co/datasets/rafmacalaba/data-use-mentions-tiered)
|
| 16 |
+
— the tiered copy of `rafmacalaba/data-use-mentions` where Luna/classifier-judged
|
| 17 |
+
T3 (non-mention) and junk spans are **untagged hard negatives** (text stays,
|
| 18 |
+
span removed). The extractor owns the mention boundary only (T1 evidential ∪
|
| 19 |
+
T2 declaration vs T3/junk); specificity detail is recovered downstream by the
|
| 20 |
+
multitask SFT model.
|
| 21 |
+
|
| 22 |
+
## Labels
|
| 23 |
+
- `DATA_MENTION` — a real data mention that carries an analytic or
|
| 24 |
+
declarative use (T1 evidential ∪ T2 declaration)
|
| 25 |
+
|
| 26 |
+
## Training
|
| 27 |
+
- base model: `urchade/gliner_large-v2.1`
|
| 28 |
+
- dataset: `rafmacalaba/datause-displacement-reviewed` (gliner_reviewed_nm config)
|
| 29 |
+
- epochs: 5
|
| 30 |
+
- learning rate: 5e-06
|
| 31 |
+
- batch size: 16
|
| 32 |
+
- precision: bf16
|
| 33 |
+
- checkpoint selection: **val span-F0.5** (post-hoc sweep of epoch
|
| 34 |
+
checkpoints; eval_loss was explicitly not used)
|
| 35 |
+
|
| 36 |
+
## Evaluation (tiered holdout)
|
| 37 |
+
|
| 38 |
+
Gold = T1∪T2 spans; a true-FP cluster matching a dropped T3/junk span
|
| 39 |
+
counts as a **T3 leak** (lower is better). Label-agnostic Hungarian
|
| 40 |
+
matching, jaccard >= 0.5 — identical to prior data-use-mentions evals.
|
| 41 |
+
|
| 42 |
+
| thr | tp | fp | fn | precision | recall | f0.5 | f1 | t3_leak | t3_leak% |
|
| 43 |
+
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
| 44 |
+
| 0.10 | 254 | 440 | 66 | 0.3660 | 0.7937 | 0.4102 | 0.5010 | 47 | 10.7% |
|
| 45 |
+
| 0.20 | 250 | 351 | 70 | 0.4160 | 0.7812 | 0.4589 | 0.5429 | 45 | 12.8% |
|
| 46 |
+
| 0.30 | 242 | 289 | 78 | 0.4557 | 0.7562 | 0.4951 | 0.5687 | 44 | 15.2% |
|
| 47 |
+
| 0.40 | 233 | 226 | 87 | 0.5076 | 0.7281 | 0.5404 | 0.5982 | 40 | 17.7% |
|
| 48 |
+
| 0.50 | 217 | 177 | 103 | 0.5508 | 0.6781 | 0.5723 | 0.6078 | 34 | 19.2% |
|
| 49 |
+
| 0.60 | 177 | 89 | 143 | 0.6654 | 0.5531 | 0.6395 | 0.6041 | 23 | 25.8% |
|
| 50 |
+
| 0.70 | 125 | 39 | 195 | 0.7622 | 0.3906 | 0.6404 | 0.5165 | 16 | 41.0% |
|
| 51 |
+
|
| 52 |
+
**Best F0.5**: 0.6404 (thr=0.7)
|
| 53 |
+
**Best F1**: 0.6078 (thr=0.5)
|
| 54 |
+
|
| 55 |
+
Full per-doc predictions (raw scores, gold spans with tier decisions): `holdout_predictions.jsonl` on this repo.
|
| 56 |
+
|
| 57 |
+
## Corpus breakdown (holdout, best F0.5)
|
| 58 |
+
|
| 59 |
+
| corpus | examples | spans | thr | precision | recall | f0.5 | f1 |
|
| 60 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 61 |
+
| prwp | 0 | 0 | 0.10 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
|
| 62 |
+
| fcv | 0 | 0 | 0.10 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
|