Add inference_contract

inference_contract/INFERENCE.md

@@ -0,0 +1,95 @@
+# Inference contract: FlowX Semantic Mapper
+
+Prompt version: `mapper_sys_v1`.
+
+This model was trained against a **frozen inference contract**: an exact system
+prompt, an exact user-turn format, a fixed decode setting, and a fixed output
+schema. Reproduce all four or the outputs drift. Do not edit the prompt or schema;
+the weights are trained against them.
+
+Files in this directory:
+
+- [`prompt_mapper_sys_v1.txt`](./prompt_mapper_sys_v1.txt): the system prompt, verbatim.
+- [`schema_mapper_v1.json`](./schema_mapper_v1.json): JSON Schema for the output object (structural / semantic / governance).
+
+Referenced from the repo root:
+
+- `concept_taxonomy.yaml`: the 252-concept controlled vocabulary the `concepts` field is drawn from.
+
+---
+
+## 1. System prompt (verbatim)
+
+The exact contents of [`prompt_mapper_sys_v1.txt`](./prompt_mapper_sys_v1.txt):
+
+```
+You are a legal and regulatory ontology extractor.
+Extract structured tags from document chunks. Output ONLY valid JSON.
+```
+
+Send it as the `system` turn. No trailing whitespace, no extra lines.
+
+## 2. User turn format
+
+One regulatory/legal text chunk per request, wrapped exactly like this:
+
+```
+Extract ontology from this chunk:
+
+CHUNK:
+<the regulatory text>
+```
+
+- The literal header `Extract ontology from this chunk:`, a blank line, then
+  `CHUNK:`, a newline, then the raw chunk text.
+- One chunk per call. The model was trained on single-chunk turns; do not batch
+  multiple clauses into one user turn.
+- Pass the chunk verbatim (the source language is fine: EN, FR, DE, RO). Do not
+  pre-summarize or translate it.
+
+## 3. Decode settings
+
+| Setting | Value | Why |
+| --- | --- | --- |
+| `enable_thinking` | **`False`** | Qwen3 is a thinking model, but the adapter was trained on pure JSON with no reasoning block. Leaving thinking on yields an empty or malformed object. |
+| `temperature` | **`0` (greedy)** | The task is deterministic extraction; sampling only adds drift. |
+| `max_new_tokens` | **~1024** | A full three-facet object fits comfortably; 1024 leaves headroom for long hierarchies. |
+| stop | end-of-turn | The model emits a single JSON object and stops. |
+
+Apply the chat template with `add_generation_prompt=True, enable_thinking=False`.
+
+## 4. Output
+
+A single JSON object with three top-level facets (`structural`, `semantic`,
+`governance`), conforming to [`schema_mapper_v1.json`](./schema_mapper_v1.json).
+Parse it strictly. On held-out data JSON validity is 1.00 and all three facets are
+present 1.00, so a parse failure means the contract above was not reproduced (most
+often `enable_thinking` left at its default).
+
+## 5. The controlled concept vocabulary (required for `semantic.concepts`)
+
+`semantic.concepts` is **not** free text. It is drawn from a **252-concept
+controlled taxonomy** shipped as `concept_taxonomy.yaml` at the repo root (6
+categories: money, rights_waived, time_renewal, lease, insurance, data; each entry
+has an `id`, a `definition`, and a `primary_domain`).
+
+This is the model's central design choice. Open free-text concepts (1062 unique in
+the first corpus, 88% of them singletons) were unlearnable and unmeasurable;
+collapsing to 252 canonical ids made the `concepts` facet both learnable and
+scoreable (F1 0.24 → 0.54). At integration time you should:
+
+- Treat any concept id **not** present in `concept_taxonomy.yaml` as
+  out-of-vocabulary and drop or flag it. The model targets the controlled set but
+  can still surface an occasional near-miss id.
+- Use the taxonomy `id` as the join key into your policy layer.
+
+`domain_tags`, by contrast, are a **free snake_case** vocabulary and are expected
+to be noisier / less consistent than `concepts`.
+
+## 6. Downstream
+
+The `governance.escalation_trigger` (`if <condition> THEN escalate`) and
+`policy_references` (`PDP.<domain>.<rule>`) are consumed by the sibling
+**[`flowxai/sentinel-gate`](https://huggingface.co/flowxai/sentinel-gate)**
+escalation model: Mapper tags a chunk → policy layer → Sentinel decides
+DECIDE vs ESCALATE. Keep the field names stable so the pipeline lines up.

inference_contract/prompt_mapper_sys_v1.txt

@@ -0,0 +1,2 @@
+You are a legal and regulatory ontology extractor.
+Extract structured tags from document chunks. Output ONLY valid JSON.

inference_contract/schema_mapper_v1.json

@@ -0,0 +1,96 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "$id": "https://huggingface.co/flowxai/semantic-mapper/inference_contract/schema_mapper_v1.json",
+  "title": "FlowX Semantic Mapper output (schema_mapper_v1)",
+  "description": "The single JSON object the Semantic Mapper emits for one regulatory/legal text chunk. Three facets: structural, semantic, governance. Prompt version mapper_sys_v1.",
+  "type": "object",
+  "additionalProperties": false,
+  "required": ["structural", "semantic", "governance"],
+  "properties": {
+    "structural": {
+      "type": "object",
+      "description": "Where the chunk sits in its source document.",
+      "additionalProperties": false,
+      "required": ["source_id", "hierarchy", "document_type"],
+      "properties": {
+        "source_id": {
+          "type": "string",
+          "description": "Stable identifier for the chunk, typically an uppercased normalization of the citation path (e.g. US_CA_INS_790_03_h_2)."
+        },
+        "hierarchy": {
+          "type": "array",
+          "description": "Ordered flat array of alternating [level, value] pairs from the outermost container down to the leaf (e.g. [\"state_code\",\"california_insurance_code\",\"section\",\"790.03\",\"subdivision\",\"h\",\"paragraph\",\"2\"]).",
+          "items": { "type": "string" },
+          "minItems": 2
+        },
+        "document_type": {
+          "type": "string",
+          "description": "Snake_case document class, e.g. state_insurance_code, eu_regulation, federal_regulation, labor_code, adr_agreement."
+        }
+      }
+    },
+    "semantic": {
+      "type": "object",
+      "description": "What the chunk is about.",
+      "additionalProperties": false,
+      "required": ["domain_tags", "concepts", "entities"],
+      "properties": {
+        "domain_tags": {
+          "type": "array",
+          "description": "Free snake_case topic tags (open vocabulary). Less consistent than concepts by design.",
+          "items": { "type": "string" }
+        },
+        "concepts": {
+          "type": "array",
+          "description": "Concept ids drawn from the 252-concept controlled taxonomy (concept_taxonomy.yaml, shipped at repo root). Values SHOULD be in-vocabulary; out-of-vocabulary ids are treated as errors by downstream consumers.",
+          "items": { "type": "string" }
+        },
+        "entities": {
+          "type": "object",
+          "description": "The core actor/action/object triple plus a constraint map.",
+          "additionalProperties": false,
+          "required": ["actor", "action", "object", "constraint"],
+          "properties": {
+            "actor": {
+              "type": "string",
+              "description": "Who the obligation/right falls on (e.g. insurer, employer, carrier, credit_institution)."
+            },
+            "action": {
+              "type": "string",
+              "description": "What must/may/must-not be done (verb phrase)."
+            },
+            "object": {
+              "type": "string",
+              "description": "What the action is performed on (e.g. claim_communication, personal_data, hazmat_package)."
+            },
+            "constraint": {
+              "type": "object",
+              "description": "Free key/value map of qualifying conditions (e.g. {\"condition\":\"reasonably_prompt\"}, {\"deadline_days\":\"30\"}). Keys and values are snake_case strings.",
+              "additionalProperties": { "type": "string" }
+            }
+          }
+        }
+      }
+    },
+    "governance": {
+      "type": "object",
+      "description": "How the chunk maps to FlowX policy and escalation.",
+      "additionalProperties": false,
+      "required": ["policy_references", "escalation_trigger"],
+      "properties": {
+        "policy_references": {
+          "type": "array",
+          "description": "Policy ids in the form PDP.<domain>.<rule> (e.g. PDP.insurance.claims_settlement).",
+          "items": {
+            "type": "string",
+            "pattern": "^PDP\\.[a-z_]+\\.[a-z0-9_]+$"
+          }
+        },
+        "escalation_trigger": {
+          "type": "string",
+          "description": "A single guard clause in the form 'if <condition> THEN escalate' consumed downstream by the Sentinel Gate."
+        }
+      }
+    }
+  }
+}