Upload dataset_inventory.md

dataset_inventory.md

@@ -1,61 +1,6 @@
-# Contract Drafting Assistant — Dataset Inventory
+# Contract Drafting Assistant Dataset Inventory
 
-## 1. Clause Retrieval & Understanding Datasets
-
-### ACORD (theatticusproject/acord)
-- **Type**: Clause retrieval benchmark
-- **Size**: 114 queries, 126,662 query-clause pairs, 1-5 star relevance
-- **Categories**: Limitation of Liability, Indemnification, Affirmative Covenants, Restrictive Covenants, Term & Termination, Governing Law, etc.
-- **Use**: Clause retrieval index, training reranker, fallback position mining
-- **License**: CC-BY-4.0
-- **Status**: Available on HF Hub
-
-### CUAD (theatticusproject/cuad-qa)
-- **Type**: Extractive QA over contracts
-- **Size**: 510 contracts, 13,000+ labels, 41 clause categories
-- **Use**: Clause extraction, clause classification training data, verifier pass
-- **License**: CC-BY-4.0
-- **Status**: Available on HF Hub
-
-### ContractNLI (kiddothe2b/contract-nli)
-- **Type**: Natural Language Inference on contract clauses
-- **Size**: ~10K examples (contractnli_a + contractnli_b)
-- **Schema**: premise (clause), hypothesis (statement), label (contradiction/entailment/neutral)
-- **Use**: Clause entailment verification, risk flag detection, internal consistency checking
-- **License**: CC-BY-NC-SA-4.0
-- **Status**: Available on HF Hub
-
-## 2. General Legal Benchmarks
-
-### LegalBench (nguha/legalbench)
-- **Type**: Multi-task legal benchmark
-- **Size**: 322 configs/subsets
-- **Relevant subsets**:
-  - `consumer_contracts_qa`: contract + question + answer
-  - ContractNLI classification subsets (via MTEB)
-  - `cuad` QA subset
-- **Use**: Evaluation rubric development, clause QA, drafting instruction following
-- **License**: CC-BY-4.0
-- **Status**: Available on HF Hub
-
-## 3. Raw Contract Corpora
-
-### Legal Contracts (albertvillanova/legal_contracts)
-- **Type**: Raw contract text corpus
-- **Size**: ~100K-1M contracts, 2GB
-- **Schema**: text (full contract)
-- **Use**: Clause extraction, language model pretraining/fine-tuning, retrieval corpus
-- **Status**: Available on HF Hub
-
-### Contract Clause Sample Dataset (asapworks/Contract_Clause_SampleDataset)
-- **Type**: Clause-level labeled dataset
-- **Size**: Small (~100 rows)
-- **Schema**: file, clause_type, clause_text, ticker, filing_date, exhibit_type
-- **Clause types**: payment_terms, dispute_resolution, warranty, assignment, duration, termination
-- **Use**: Seed clause library, few-shot examples
-- **Status**: Available on HF Hub
-
-## 4. Selected Datasets for This System
+## Selected Datasets
 
 | Purpose | Dataset | Split/Config |
 |---------|---------|-------------|
@@ -63,12 +8,10 @@
 | Clause retrieval benchmark | theatticusproject/acord | train/test |
 | Clause extraction labels | theatticusproject/cuad-qa | train |
 | Clause verification | kiddothe2b/contract-nli | contractnli_a |
-| Evaluation & rubrics | nguha/legalbench | consumer_contracts_qa, contractnli subsets |
+| Evaluation rubrics | nguha/legalbench | consumer_contracts_qa |
 | Seed clause library | asapworks/Contract_Clause_SampleDataset | train |
 
-## 5. Gaps & Synthetic Data Needs
-
-- **No dedicated clause generation dataset** exists for SaaS/MSA/NDA/SOW/DPA types
-- **No playbook/fallback position dataset** exists publicly
-- **No contract drafting instruction dataset** exists
-- We will synthesize playbook rules, fallback positions, and drafting checklists from domain knowledge and ACORD/CUAD clause patterns.
+## Gaps
+- No dedicated clause generation dataset for SaaS/MSA/NDA/SOW/DPA
+- No playbook/fallback position dataset exists publicly
+- Will synthesize from domain knowledge + ACORD/CUAD patterns