Publish the clean-split retrain as revision clean-2026-08-21, and say which revision the numbers describe
Browse files
README.md
CHANGED
|
@@ -41,8 +41,8 @@ language:
|
|
| 41 |
> documents during training, so no honest score for *them* exists. Publishing
|
| 42 |
> the clean adapter as a new revision is pending.
|
| 43 |
> 2. **This was the harshest of the three corrections, and the reason is volume.**
|
| 44 |
-
> Document isolation cut training from
|
| 45 |
-
> pool — because a single Act yields hundreds of same-document pairs. The two
|
| 46 |
> runs are not remotely volume-matched, so the drop from +48 % to +28.1 % is
|
| 47 |
> *not* a measure of "what the leak was worth". The honest finding is that
|
| 48 |
> **this corpus does not contain enough distinct Acts to support both a clean
|
|
@@ -127,6 +127,31 @@ string matching. It no longer "more than doubles"; that phrasing belonged to the
|
|
| 127 |
figure. By pair kind, the gain holds across both dominant types: adjacent-section
|
| 128 |
0.185 → 0.242 (n = 1,488) and heading↔section 0.172 → 0.205 (n = 489).
|
| 129 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 130 |
## Usage
|
| 131 |
|
| 132 |
Pull the adapter and run it through the Quanfire framework, which applies the LoRA over
|
|
|
|
| 41 |
> documents during training, so no honest score for *them* exists. Publishing
|
| 42 |
> the clean adapter as a new revision is pending.
|
| 43 |
> 2. **This was the harshest of the three corrections, and the reason is volume.**
|
| 44 |
+
> Document isolation cut training from **32,432** pairs to **3,052** — 90.6 % of
|
| 45 |
+
> the pool — because a single Act yields hundreds of same-document pairs. The two
|
| 46 |
> runs are not remotely volume-matched, so the drop from +48 % to +28.1 % is
|
| 47 |
> *not* a measure of "what the leak was worth". The honest finding is that
|
| 48 |
> **this corpus does not contain enough distinct Acts to support both a clean
|
|
|
|
| 127 |
figure. By pair kind, the gain holds across both dominant types: adjacent-section
|
| 128 |
0.185 → 0.242 (n = 1,488) and heading↔section 0.172 → 0.205 (n = 489).
|
| 129 |
|
| 130 |
+
## Which weights should I use?
|
| 131 |
+
|
| 132 |
+
Two revisions are published. They share the recipe and differ in **what can be said about
|
| 133 |
+
them**:
|
| 134 |
+
|
| 135 |
+
| Revision | Trained on | Has a valid score? |
|
| 136 |
+
|---|---|---|
|
| 137 |
+
| `main` (default) | 32,432 pairs — the full mined pool | ❌ **No.** It trained on the evaluation documents, so no clean held-out set exists for it *within this corpus* |
|
| 138 |
+
| `clean-2026-08-21` | 3,052 pairs — document-isolated split | ✅ **Yes** — every figure in *Results* above is this adapter |
|
| 139 |
+
|
| 140 |
+
```bash
|
| 141 |
+
hf download quanfire-ai/embed-statute-en --revision clean-2026-08-21 --local-dir embed-statute-en-clean
|
| 142 |
+
```
|
| 143 |
+
|
| 144 |
+
**Which one to take.** If you need a number you can cite or audit, take
|
| 145 |
+
`clean-2026-08-21` — it is the one the Results section describes. `main` saw
|
| 146 |
+
10x the training data and may well be the stronger retriever in practice, but "may
|
| 147 |
+
well be" is precisely the kind of claim this card no longer makes.
|
| 148 |
+
|
| 149 |
+
**We have not swapped the default**, and the reason is worth stating: doing so would trade a
|
| 150 |
+
plausibly-stronger model for a measurable one with no evidence that the trade is good.
|
| 151 |
+
Settling it properly needs a head-to-head of the two adapters on a corpus *neither* of them
|
| 152 |
+
trained on. That is planned, and until it runs, both revisions stay up and this section stays
|
| 153 |
+
honest about which is which.
|
| 154 |
+
|
| 155 |
## Usage
|
| 156 |
|
| 157 |
Pull the adapter and run it through the Quanfire framework, which applies the LoRA over
|