README.md

---
license: cc-by-nc-nd-4.0
tags:
- cybersecurity
- cti
- yara
- contrastive-learning
- sentence-similarity
- dual-encoder
library_name: transformers
base_model: sentence-transformers/all-mpnet-base-v2
---

# FALCON dual-encoder — YARA / `all-mpnet-base-v2`

Contrastive encoder fine-tuned to map CTI text and YARA rules into a shared embedding space.
Backbone: `sentence-transformers/all-mpnet-base-v2`.

## Test-set metrics

| split | recall@1 | F1 | threshold | diag mean | off-diag mean |
|---|---:|---:|---:|---:|---:|
| pretrained | 0.7325 | 0.2365 | 0.6850 | 0.7315 | 0.4856 |
| run_0 | 0.9498 | 0.9214 | 0.6746 | 0.8456 | 0.0140 |
| run_1 | 0.9509 | 0.9392 | 0.6951 | 0.9041 | 0.0207 |
| run_2 | 0.9509 | 0.9351 | 0.6849 | 0.8838 | 0.0123 |
| run_3 | 0.9509 | 0.9336 | 0.6979 | 0.9185 | 0.0234 |
| run_4 | 0.9509 | 0.9412 | 0.7060 | 0.9625 | 0.0046 |

## Training

Symmetric InfoNCE / NT-Xent over in-batch negatives. Best checkpoint selected by validation loss.

- **Run 0** — batch=16, epochs=5, lr=2e-05, schedule=constant, T=0.05
- **Run 1** — batch=50, epochs=10, lr=2e-05, schedule=constant, T=0.05
- **Run 2** — batch=70, epochs=30, lr=2e-05, schedule=constant, T=0.05
- **Run 3** — batch=128, epochs=30, lr=5e-05, schedule=warmup_cosine, T=0.05
- **Run 4** — batch=70, epochs=50, lr=2e-05, schedule=constant, T=0.07

## Loading

```python
from transformers import AutoModel, AutoTokenizer
tok   = AutoTokenizer.from_pretrained("shaswatamitra/falcon-yara-dual-all-mpnet-base-v2", subfolder='rule')
model = AutoModel.from_pretrained("shaswatamitra/falcon-yara-dual-all-mpnet-base-v2", subfolder='rule')
```


_Dual-encoder layout: this repo has `rule/` (encodes YARA rules) and `cti/` (encodes CTI text) subfolders. Load each with `subfolder=...`._

## Citation

```bibtex
@article{mitra2025falcon,
  title={FALCON: Autonomous Cyber Threat Intelligence Mining with LLMs for IDS Rule Generation},
  author={Mitra, Shaswata and Bazarov, Azim and Duclos, Martin and Mittal, Sudip and Piplai, Aritran and Rahman, Md Rayhanur and Zieglar, Edward and Rahimi, Shahram},
  journal={arXiv preprint arXiv:2508.18684},
  year={2025}
}
```