| --- |
| license: apache-2.0 |
| pipeline_tag: token-classification |
| tags: |
| - ner |
| - gliner |
| - data-use |
| - tiered |
| --- |
| |
| # gliner-datause-probe |
|
|
| Fine-tune of `urchade/gliner_large-v2.1` for data-use mention extraction with a |
| **single `DATA_MENTION` class**, trained on |
| [`rafmacalaba/data-use-mentions-tiered`](https://huggingface.co/datasets/rafmacalaba/data-use-mentions-tiered) |
| — the tiered copy of `rafmacalaba/data-use-mentions` where Luna/classifier-judged |
| T3 (non-mention) and junk spans are **untagged hard negatives** (text stays, |
| span removed). The extractor owns the mention boundary only (T1 evidential ∪ |
| T2 declaration vs T3/junk); specificity detail is recovered downstream by the |
| multitask SFT model. |
| |
| ## Labels |
| - `DATA_MENTION` — a real data mention that carries an analytic or |
| declarative use (T1 evidential ∪ T2 declaration) |
| |
| ## Training |
| - base model: `urchade/gliner_large-v2.1` |
| - dataset: `rafmacalaba/data-use-mentions-tiered` (gliner_tiered config) |
| - epochs: 5 |
| - learning rate: 5e-06 |
| - batch size: 16 |
| - precision: bf16 |
| - checkpoint selection: **val span-F0.5** (post-hoc sweep of epoch |
| checkpoints; eval_loss was explicitly not used) |
| |
| ## Evaluation (tiered holdout) |
| |
| Gold = T1∪T2 spans; a true-FP cluster matching a dropped T3/junk span |
| counts as a **T3 leak** (lower is better). Label-agnostic Hungarian |
| matching, jaccard >= 0.5 — identical to prior data-use-mentions evals. |
| |
| | thr | tp | fp | fn | precision | recall | f0.5 | f1 | t3_leak | t3_leak% | |
| | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | |
| | 0.10 | 16812 | 9110 | 353 | 0.6486 | 0.9794 | 0.6956 | 0.7804 | 2829 | 31.1% | |
| | 0.20 | 16664 | 6850 | 501 | 0.7087 | 0.9708 | 0.7491 | 0.8193 | 2335 | 34.1% | |
| | 0.30 | 16505 | 5607 | 660 | 0.7464 | 0.9615 | 0.7814 | 0.8404 | 2002 | 35.7% | |
| | 0.40 | 16275 | 4623 | 890 | 0.7788 | 0.9482 | 0.8076 | 0.8552 | 1741 | 37.7% | |
| | 0.50 | 15934 | 3701 | 1231 | 0.8115 | 0.9283 | 0.8325 | 0.8660 | 1498 | 40.5% | |
| | 0.60 | 15195 | 2735 | 1970 | 0.8475 | 0.8852 | 0.8548 | 0.8659 | 1203 | 44.0% | |
| | 0.70 | 13793 | 1808 | 3372 | 0.8841 | 0.8036 | 0.8667 | 0.8419 | 876 | 48.4% | |
| |
| **Best F0.5**: 0.8667 (thr=0.7) |
| **Best F1**: 0.8660 (thr=0.5) |
| |
| ## Corpus breakdown (holdout, best F0.5) |
| |
| | corpus | examples | spans | thr | precision | recall | f0.5 | f1 | |
| | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | |
| | prwp | 7758 | 12359 | 0.70 | 0.8897 | 0.7777 | 0.8648 | 0.8299 | |
| | fcv | 7574 | 9693 | 0.70 | 0.8744 | 0.8542 | 0.8703 | 0.8642 | |
| |
