Quanfire Legal Embedding — embed-legal-en (Supreme Court judgments)

⚠️ Measurement correction — 2026-08-21

The headline this card used to carry (+76.2 % Recall@1) was measured on a contaminated evaluation split. The corrected figure is +59.7 %.

The bug: the training filter dropped a pair only when its positive was in the held-out set, and pairs drawn from a held-out judgment were never excluded at all. Adjacent-passage pairs from the same judgment therefore sat on both sides of the split — the model was partly trained on the documents it was scored on.

The adapter was retrained and rescored on a split that isolates at the judgment-document level (1,379 documents held out, 48,304 pairs dropped from the pool). Every in-distribution number in Results below is the clean measurement. This was the best-surviving of the three corrections we pushed that day: the gain lost about a quarter of its magnitude and kept its direction, its significance and its shape.

Two things a reader deserves to know:

1. The clean figures are for a retrain (legal-indic-e1c), not for the weight file currently in this repo. The published weights saw the evaluation documents during training, so no honest score for them exists. Publishing the clean adapter as a new revision is pending.
2. The runs are not volume-matched — 46,115 training pairs clean against 92,419 before. Do not read the difference as a clean measure of "what the leak was worth".

The out-of-origin transfer result is unaffected in kind — it is scored on a different corpus, so the split bug cannot reach it — but the figures below belong to the withdrawn adapter and have not been recomputed for the retrain.

Fix: without_held_out() in quanfire-multilingual-embedding (commits 66470fe, 6fe7e6b, 3deaf8d).

A retrieval adapter for English-language Indian Supreme Court judgment text. It is a LoRA adaptation over a frozen intfloat/multilingual-e5-small (MIT) base — a 2.4 MB adapter, 384-dimensional normalized vectors, max_length 256 — trained only on statutory public-domain judgment text.

This is not a from-scratch model, and it is not a general "Indian legal" model. It is a specialist: on judgment-to-judgment retrieval it is markedly stronger than the base; on other legal registers (statutes) it is not — and this card shows you both, measured.

• Framework & code: github.com/quanfire-ai/quanfire-multilingual-embedding (Apache-2.0)
• PyPI: pip install quanfire-multilingual-embedding
• Weights licence: Apache-2.0 (see Licence & provenance — the training text is statutory public domain, so no share-alike floor applies)
• Internal run: legal-indic-e1 · base e5-small · rank 32 / alpha 64, LoRA on query,value · 589,824 adapter params · 1 epoch, lr 1e-4, batch 256 (bf16, CUDA) · the clean re-measure reported below is legal-indic-e1c: identical configuration, retrained on a document-isolated split

What it is for

Retrieving and ranking passages of English Supreme Court judgments — case-law search, judgment-to-judgment similarity, semantic retrieval over a judgment corpus. It embeds a query and a passage into the same 384-d space; cosine similarity ranks.

Scope — read this before you use it

	Validated?
English Supreme Court judgment retrieval	✅ Yes — +59.7 % Recall@1 on a clean, document-isolated split; measured below
Statutory / bare-act text, FAQs, other legal registers	❌ No — transfer tested, came back flat (see below)
Non-English legal text (Hindi, Tamil, …)	❌ No — the model and its training data are English-only

The gain this adapter provides is judgment-specific. If your text is statutes, contracts, or non-English legal material, use the base model or a purpose-built model — this one will not help there, and we measured that rather than assuming it.

Results (held-out, scored on CUDA)

In-distribution — 2,000 held-out Supreme Court judgment pairs. The published base is the only honest baseline; the adapter is scored on the same held-out pairs.

Metric	base e5-small	clean retrain (legal-indic-e1c)	change	withdrawn figure
Recall@1	0.3090	0.4935	+59.7 %	0.545 / +76.2 %
Recall@5	0.4940	0.7180	+45.3 %	0.772
Recall@10	0.5670	0.7910	+39.5 %	0.829
MRR	0.3988	0.5996	+50.4 %	0.647
nDCG@10	0.4326	0.6408	+48.1 %	0.687

The base column is unchanged — the contamination only ever inflated the adapter, so the correction is confined to one column.

The Recall@1 95 % confidence intervals are disjoint (base [0.2891, 0.3296] → clean retrain [0.4716, 0.5154]), so the gain is not sampling noise. It holds where it is hardest: on the low-lexical-overlap bucket (<0.3, pure-semantic matches, no shared words to lean on) Recall@1 rises 0.156 → 0.290 (+85.7 %; the withdrawn figure was 0.325).

Out-of-origin transfer — 1,578 English statutory adjacency pairs (a different legal register: bare-act sections and regulatory FAQs, origin-walled from the judgment training corpus). This is the honest generalization test, and it is flat. These two rows are the withdrawn adapter's numbers and have not been recomputed for the clean retrain; the split bug could not reach them (different corpus), but a retrained adapter is a different adapter:

Metric	base e5-small	embed-legal-en
Recall@1	0.036	0.036 (−1.8 %)
nDCG@10	0.219	0.208

The instrument is informative, not degenerate — the base model has real signal on it (high-overlap Recall@5 0.76). The adapter simply does not improve statutory retrieval. We publish this row on purpose: the in-distribution gain is a judgment-domain result, not a "legal English" result, and the difference is exactly what a buyer needs to know. For statutory text, use the purpose-built sibling embed-statute-en.

Which weights should I use?

Two revisions are published. They share the recipe and differ in what can be said about them:

Revision	Trained on	Has a valid score?
main (default)	92,419 pairs — the full mined pool	❌ No. It trained on the evaluation documents, so no clean held-out set exists for it within this corpus
clean-2026-08-21	46,115 pairs — document-isolated split	✅ Yes — every figure in Results above is this adapter

hf download quanfire-ai/embed-legal-en --revision clean-2026-08-21 --local-dir embed-legal-en-clean

Which one to take. If you need a number you can cite or audit, take clean-2026-08-21 — it is the one the Results section describes. main saw 2x the training data and may well be the stronger retriever in practice, but "may well be" is precisely the kind of claim this card no longer makes.

We have not swapped the default, and the reason is worth stating: doing so would trade a plausibly-stronger model for a measurable one with no evidence that the trade is good. Settling it properly needs a head-to-head of the two adapters on a corpus neither of them trained on. That is planned, and until it runs, both revisions stay up and this section stays honest about which is which.

Usage

Pull the adapter and run it through the Quanfire framework, which applies the LoRA over the frozen base and produces normalized embeddings:

pip install 'quanfire-multilingual-embedding[neural]'

hf download quanfire-ai/embed-legal-en --local-dir embed-legal-en

As an HTTP embeddings service (OpenAI-compatible POST /v1/embeddings):

qfme serve --adapter embed-legal-en --port 8000

curl -s localhost:8000/v1/embeddings \
  -H 'content-type: application/json' \
  -d '{"input": ["Whether the appellant was denied a fair hearing under Article 21."]}'

In-process, as a search pipeline:

from multilingual_embedding.pipelines.search import SemanticSearchPipeline

pipe = SemanticSearchPipeline.from_adapter("embed-legal-en")
pipe.index([
    "The conviction under Section 302 is set aside for want of corroboration.",
    "Bail is granted subject to the appellant surrendering the passport.",
    "The writ petition challenges the vires of the impugned notification.",
])
for hit in pipe.search("appeal against a murder conviction", top_k=3):
    print(hit.rank, round(hit.score, 3), hit.text)

Vectors are L2-normalized float32 (dimension 384). The model is symmetric (empty prefixes), so input_type is not required. Exact (brute-force cosine) search is the intended regime up to ~10⁵–10⁶ vectors; add an ANN index beyond that.

Licence & provenance

Weights: Apache-2.0. Use them commercially and redistribute them freely, with attribution. There is no share-alike obligation, because — unlike a Wikipedia-derived model — every input to this adapter is public-domain text:

Source	Role	Licence / status
Indian Supreme Court judgment text (official court portals)	training corpus	Public domain — Copyright Act 1957, §52(1)(q) places judgment text outside copyright
intfloat/multilingual-e5-small	frozen base checkpoint	MIT

What was removed, and why it matters. §52(1)(q) frees the text of the judgment — it does not free the reporter-written headnote / syllabus, which is separately copyrightable editorial matter. The training corpus was built by extracting each judgment with a layout-faithful parser, dropping page furniture, and excising the headnote span between the coram line and the start of the reported judgment body. Only the statutory-public-domain judgment text was trained on. The base checkpoint is frozen and unmodified (MIT); the adapter is a separate set of weights over it.

The framework source code is Apache-2.0 (separate from these weights).

Limitations

• A LoRA adapter over a published checkpoint — not an independently pretrained model.
• English only. It does not embed Hindi, Tamil or other Indic legal text; a cross-lingual Indian-legal model is separate work requiring parallel legal data.
• Judgment-specific. Transfer to statutory / bare-act text was tested and is flat (table above). Do not rely on it outside judgment-style text without your own evaluation. It is evaluated only on held-out Indian Supreme Court judgment pairs from the same distribution as its training data; transfer to other jurisdictions or legal systems is not measured and should not be assumed.
• Exact cosine search is the intended regime up to ~10⁵–10⁶ vectors.

Citation

Quanfire Legal Embedding — embed-legal-en (internal run legal-indic-e1).
Quanfire, 2026. https://github.com/quanfire-ai/quanfire-multilingual-embedding