overthelex/fi-legal-citation-grounded-12b

Finnish Legal Citation-Grounded 12B

A gemma-4-12B-it model fine-tuned to answer Finnish legal questions grounded in retrieved court-decision chunks: it cites every claim with a [doc:ID] tag, attributes each citation to the court's holding rather than an overruled or dissenting passage, pins the right paragraph (kohta), and refuses when the sources do not support an answer.

This is a collaboration: the Finnish corpus, dataset schema, and judge definitions are Niko Hannolainen's (nikohann/fi-legal-sft, built from a 264M-triple Finnish/EU legal knowledge graph); the teacher pass, judge stack, and student training were run by overthelex on 8×H100. It ports the method behind overthelex/ua-legal-citation-grounded-14b from Ukrainian to Finnish.

Method

Reverse-QA distillation with a multi-stage reject filter:

• Teacher: google/gemma-4-31B-it writes [doc:ID]-cited answers over retrieved chunks (served under vLLM with --reasoning-parser gemma4 + enable_thinking=false).
• Judge stack (only judge-passing answers survive to training):
  1. membership — every cited id must resolve to a chunk in context
  2. attribution — a citation on an overruled/dissent chunk must be marked as a losing position in the same sentence (turns same-case overruled/dissent passages into hard negatives)
  3. kohta-precision — a cited paragraph number must fall inside the chunk's official range
  4. factual-consistency — an LLM pass (31B) checks each claim against its cited chunk
• Student: LoRA (r=64) on gemma-4-12B-it, seq 8192, 2 epochs, 8×H100 DDP, stratified to a real-usage court-family mix.

Evaluation

On 150 held-back examples (unseen question/context; see caveat), against the unfinetuned base:

metric	base gemma-4-12B-it	this model
Citation coverage	100.0%	97.2%
In-context rate	99.7%	98.5%
Membership-clean	97.9%	97.2%
Attribution-clean (cites holding, not overruled/dissent)	97.2%	99.3%
Kohta-precision	97.2%	100.0%
Refusal recall	62.5%	100.0%
Over-refusal	0.0%	2.8%
Degenerate outputs	0	0

The base model already cites in-context well, but it refuses correctly only 62.5% of the time (it answers even when the sources do not support it) and cites the losing position more often. Read as error rates, the fine-tune removes about three quarters of the attribution errors (2.8% → 0.7%) and every paragraph-precision error (2.8% → 0%), and takes refusal recall from 62.5% to 100% — better calibrated, not just more citation-happy, at a small cost in raw coverage because it now refuses when it should.

Caveat: these are in-distribution held-back examples (unseen question/context on decisions the model saw other examples from), not a separate held-out-decisions test set. Read them as in-distribution grounding quality, not cross-decision generalization.

Attribution-label caveat (provisional number): attribution-clean is scored against the corpus kumottu (overruled) markers, which are being corrected. The current marker over-flags lower instances — it flagged every lower instance in a document rather than only the one named in the operative part (1,257 of 2,986 marked decisions carry more than one flag; 194 have an operative part that expressly leaves a lower judgment standing). So a share of the attribution rejects are false rejects, and 99.3% is partly conformance to labels that are wrong in places. This will be re-measured on the corrected labels; treat it as provisional.

Serving

Serve with vLLM and read the answer from the content field:

vllm serve <this-model> --reasoning-parser gemma4 --chat-template chat_template.jinja

Pass chat_template_kwargs={"enable_thinking": false} per request (Gemma-4 has a thinking channel that must be suppressed). Inference guardrail: strip any [doc:ID] not present in the context you passed (a few percent of citations can be out-of-context even after training).

License

Derivative of google/gemma-4-12B-it, governed by the Gemma Terms of Use.