quanfire-dev commited on
Commit
678b1ee
·
verified ·
1 Parent(s): 5d3b90c

Publish the clean-split retrain as revision clean-2026-08-21, and say which revision the numbers describe

Browse files
Files changed (1) hide show
  1. README.md +27 -2
README.md CHANGED
@@ -41,8 +41,8 @@ language:
41
  > documents during training, so no honest score for *them* exists. Publishing
42
  > the clean adapter as a new revision is pending.
43
  > 2. **This was the harshest of the three corrections, and the reason is volume.**
44
- > Document isolation cut training from 33,252 pairs to **3,052** — 91 % of the
45
- > pool — because a single Act yields hundreds of same-document pairs. The two
46
  > runs are not remotely volume-matched, so the drop from +48 % to +28.1 % is
47
  > *not* a measure of "what the leak was worth". The honest finding is that
48
  > **this corpus does not contain enough distinct Acts to support both a clean
@@ -127,6 +127,31 @@ string matching. It no longer "more than doubles"; that phrasing belonged to the
127
  figure. By pair kind, the gain holds across both dominant types: adjacent-section
128
  0.185 → 0.242 (n = 1,488) and heading↔section 0.172 → 0.205 (n = 489).
129
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
130
  ## Usage
131
 
132
  Pull the adapter and run it through the Quanfire framework, which applies the LoRA over
 
41
  > documents during training, so no honest score for *them* exists. Publishing
42
  > the clean adapter as a new revision is pending.
43
  > 2. **This was the harshest of the three corrections, and the reason is volume.**
44
+ > Document isolation cut training from **32,432** pairs to **3,052** — 90.6 % of
45
+ > the pool — because a single Act yields hundreds of same-document pairs. The two
46
  > runs are not remotely volume-matched, so the drop from +48 % to +28.1 % is
47
  > *not* a measure of "what the leak was worth". The honest finding is that
48
  > **this corpus does not contain enough distinct Acts to support both a clean
 
127
  figure. By pair kind, the gain holds across both dominant types: adjacent-section
128
  0.185 → 0.242 (n = 1,488) and heading↔section 0.172 → 0.205 (n = 489).
129
 
130
+ ## Which weights should I use?
131
+
132
+ Two revisions are published. They share the recipe and differ in **what can be said about
133
+ them**:
134
+
135
+ | Revision | Trained on | Has a valid score? |
136
+ |---|---|---|
137
+ | `main` (default) | 32,432 pairs — the full mined pool | ❌ **No.** It trained on the evaluation documents, so no clean held-out set exists for it *within this corpus* |
138
+ | `clean-2026-08-21` | 3,052 pairs — document-isolated split | ✅ **Yes** — every figure in *Results* above is this adapter |
139
+
140
+ ```bash
141
+ hf download quanfire-ai/embed-statute-en --revision clean-2026-08-21 --local-dir embed-statute-en-clean
142
+ ```
143
+
144
+ **Which one to take.** If you need a number you can cite or audit, take
145
+ `clean-2026-08-21` — it is the one the Results section describes. `main` saw
146
+ 10x the training data and may well be the stronger retriever in practice, but "may
147
+ well be" is precisely the kind of claim this card no longer makes.
148
+
149
+ **We have not swapped the default**, and the reason is worth stating: doing so would trade a
150
+ plausibly-stronger model for a measurable one with no evidence that the trade is good.
151
+ Settling it properly needs a head-to-head of the two adapters on a corpus *neither* of them
152
+ trained on. That is planned, and until it runs, both revisions stay up and this section stays
153
+ honest about which is which.
154
+
155
  ## Usage
156
 
157
  Pull the adapter and run it through the Quanfire framework, which applies the LoRA over