GSMS-B commited on
Commit
af16dcc
Β·
verified Β·
1 Parent(s): 5348deb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +167 -11
README.md CHANGED
@@ -1,23 +1,179 @@
1
  ---
 
 
 
 
2
  tags:
 
 
 
 
 
 
 
 
3
  - gguf
4
  - llama.cpp
 
 
5
  - unsloth
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
 
7
  ---
8
 
9
- # Indian-Legal-Qwen2.5-1.5B-GGUF : GGUF
 
 
 
 
 
 
 
 
 
10
 
11
- This model was finetuned and converted to GGUF format using [Unsloth](https://github.com/unslothai/unsloth).
12
 
13
- **Example usage**:
14
- - For text only LLMs: `llama-cli -hf GSMS-B/Indian-Legal-Qwen2.5-1.5B-GGUF --jinja`
15
- - For multimodal models: `llama-mtmd-cli -hf GSMS-B/Indian-Legal-Qwen2.5-1.5B-GGUF --jinja`
16
 
17
- ## Available Model files:
18
- - `qwen2.5-1.5b-instruct.Q4_K_M.gguf`
 
 
 
 
 
19
 
20
- ## Ollama
21
- An Ollama Modelfile is included for easy deployment.
22
- This was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth)
23
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
1
  ---
2
+ language:
3
+ - en
4
+ license: apache-2.0
5
+ base_model: unsloth/Qwen2.5-1.5B-Instruct-bnb-4bit
6
  tags:
7
+ - legal
8
+ - indian-law
9
+ - BNS
10
+ - BNSS
11
+ - BSA
12
+ - criminal-law
13
+ - qwen
14
+ - qwen2.5
15
  - gguf
16
  - llama.cpp
17
+ - ollama
18
+ - qlora
19
  - unsloth
20
+ - domain-adaptation
21
+ - instruction-tuning
22
+ - question-answering
23
+ - law
24
+ - india
25
+ datasets:
26
+ - GSMS-B/Indian-Legal-QA-BNS-BNSS-BSA
27
+ pipeline_tag: text-generation
28
+ ---
29
+
30
+ # βš–οΈπŸ‰ Indian Legal Qwen 2.5 β€” 1.5B (GGUF)
31
+
32
+ <p align="center">
33
+ <img src="https://img.shields.io/badge/Base%20Model-Qwen%202.5%201.5B-6366F1?style=for-the-badge" alt="Base Model"/>
34
+ <img src="https://img.shields.io/badge/Type-GGUF%20Quantized-A855F7?style=for-the-badge" alt="Type"/>
35
+ <img src="https://img.shields.io/badge/Domain-Indian%20Criminal%20Law-DC2626?style=for-the-badge" alt="Domain"/>
36
+ <img src="https://img.shields.io/badge/Method-QLoRA-2563EB?style=for-the-badge" alt="Method"/>
37
+ <img src="https://img.shields.io/badge/Acts-BNS%20%7C%20BNSS%20%7C%20BSA-16A34A?style=for-the-badge" alt="Acts"/>
38
+ <img src="https://img.shields.io/badge/License-Apache%202.0-F59E0B?style=for-the-badge" alt="License"/>
39
+ </p>
40
+
41
+ > 🟑 **This is the GGUF-quantized version** β€” for CPU inference via Ollama or llama.cpp. For full-precision inference see the [Merged Model](https://huggingface.co/GSMS-B/Indian-Legal-Qwen2.5-1.5B) Β· For lightweight adapter loading see the [Adapter](https://huggingface.co/GSMS-B/Indian-Legal-Qwen2.5-1.5B-Adapter).
42
+
43
+ ---
44
+
45
+ ## πŸ“– Model Description
46
+
47
+ **Indian Legal Qwen 2.5 β€” 1.5B (GGUF)** is a quantized, CPU-friendly version of [`GSMS-B/Indian-Legal-Qwen2.5-1.5B`](https://huggingface.co/GSMS-B/Indian-Legal-Qwen2.5-1.5B), a domain-adapted model fine-tuned using QLoRA on a structured question-answer dataset covering all **1,059 sections** of India's three landmark 2023 criminal justice reform acts:
48
+
49
+ | Act | Full Name | Replaces | Sections |
50
+ |---|---|---|---|
51
+ | πŸ“• **BNS 2023** | Bharatiya Nyaya Sanhita | IPC 1860 | 358 |
52
+ | πŸ“— **BNSS 2023** | Bharatiya Nagarik Suraksha Sanhita | CrPC 1973 | 531 |
53
+ | πŸ“˜ **BSA 2023** | Bharatiya Sakshya Adhiniyam | Indian Evidence Act 1872 | 170 |
54
+
55
+ Trained on **6,354 instruction-format QA pairs** β€” 6 question types per section covering definitions, scenarios, legal elements, exceptions, and consequences β€” giving it broad, structured coverage of India's reformed criminal law framework. As the smallest model in the family, this GGUF build is ideal for fast, fully offline CPU inference.
56
+
57
+ ---
58
+
59
+ ## πŸ”— Model Family β€” Qwen 2.5 1.5B
60
+
61
+ | Variant | Repo | Best For |
62
+ |---|---|---|
63
+ | 🟒 **Merged** | [GSMS-B/Indian-Legal-Qwen2.5-1.5B](https://huggingface.co/GSMS-B/Indian-Legal-Qwen2.5-1.5B) | Out-of-the-box inference, Gradio / API deployment |
64
+ | πŸ”΅ **LoRA Adapter** | [GSMS-B/Indian-Legal-Qwen2.5-1.5B-Adapter](https://huggingface.co/GSMS-B/Indian-Legal-Qwen2.5-1.5B-Adapter) | Lightweight loading on top of base model |
65
+ | 🟑 **GGUF (this repo)** | `GSMS-B/Indian-Legal-Qwen2.5-1.5B-GGUF` | CPU inference via Ollama / llama.cpp |
66
+
67
+ ---
68
+
69
+ ## πŸš€ Quick Start
70
+
71
+ ### πŸ’» Run with Ollama
72
+ ```bash
73
+ ollama run hf.co/GSMS-B/Indian-Legal-Qwen2.5-1.5B-GGUF
74
+ ```
75
+
76
+ ### βš™οΈ Run with llama.cpp
77
+ ```bash
78
+ ./llama-cli \
79
+ -hf GSMS-B/Indian-Legal-Qwen2.5-1.5B-GGUF \
80
+ -p "What is a Zero FIR under BNSS 2023?" \
81
+ -n 300 \
82
+ --temp 0.1
83
+ ```
84
+
85
+ ### 🐍 Run with llama-cpp-python
86
+ ```python
87
+ from llama_cpp import Llama
88
+
89
+ llm = Llama.from_pretrained(
90
+ repo_id="GSMS-B/Indian-Legal-Qwen2.5-1.5B-GGUF",
91
+ filename="*.gguf",
92
+ )
93
+
94
+ SYSTEM = "You are an expert legal assistant specializing in Indian criminal law β€” BNS, BNSS, and BSA 2023."
95
+
96
+ response = llm.create_chat_completion(
97
+ messages=[
98
+ {"role": "system", "content": SYSTEM},
99
+ {"role": "user", "content": "What is a Zero FIR under BNSS 2023?"}
100
+ ],
101
+ temperature=0.1,
102
+ max_tokens=300
103
+ )
104
+
105
+ print(response["choices"][0]["message"]["content"])
106
+ ```
107
+
108
+ ---
109
+
110
+ ## 🎯 Recommended Use Cases
111
+
112
+ > ⚠️ **Important Note:** This model has been domain-adapted on structured QA data and works best as a **component in a larger pipeline** rather than a standalone answer engine. Direct usage without retrieval context may produce incomplete or imprecise answers on complex legal queries.
113
+
114
+ ### βœ… Where this model excels
115
+
116
+ | Use Case | πŸ’‘ How to Use |
117
+ |---|---|
118
+ | πŸ” **RAG Pipeline** | Pair with a BM25 or vector retriever over BNS/BNSS/BSA texts; feed retrieved sections as context for grounded, citation-backed answers |
119
+ | πŸ€– **Legal Chatbot Backend** | Use as the generation backbone of a legal assistant app with a ChromaDB / FAISS document store |
120
+ | πŸ“š **Legal Education Tool** | Build interactive Q&A apps for law students and practitioners learning the 2023 criminal justice reforms |
121
+ | πŸ”Ž **Section Lookup Assistant** | Combine with a section index to surface the exact BNS / BNSS / BSA provision relevant to a given situation |
122
+ | πŸ’» **Offline / Edge Deployment** | Smallest model in the family, runnable on consumer CPUs without a GPU β€” ideal for local apps, kiosks, or low-resource environments |
123
+ | πŸ“ **Structured Legal Summarization** | Summarize individual sections when the section text is supplied as input context |
124
+ | πŸ›οΈ **Legal NLP Research** | Benchmark Indian criminal law understanding across model families (Qwen vs Llama) |
125
+ | βš–οΈ **Comparative Law Analysis** | Highlight differences between old acts (IPC/CrPC/IEA) and their 2023 replacements |
126
+
127
+ ### ❌ Not recommended for
128
+ - Standalone legal advice without a retrieval component
129
+ - High-stakes legal decisions without qualified human review
130
+ - Jurisdictions or acts outside BNS / BNSS / BSA 2023
131
+
132
+ ---
133
+
134
+ ## πŸ‹οΈ Training Details
135
+
136
+ | Property | Value |
137
+ |---|---|
138
+ | πŸ€– Base model | `unsloth/Qwen2.5-1.5B-Instruct-bnb-4bit` |
139
+ | πŸ”§ Fine-tuning method | QLoRA |
140
+ | πŸŽ›οΈ LoRA rank | 64 |
141
+ | πŸŽ›οΈ LoRA alpha | 128 |
142
+ | 🧩 Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
143
+ | πŸ“Š Training data | 6,354 QA pairs β€” 1,059 sections Γ— 6 question types |
144
+ | πŸ” Epochs | 3 |
145
+ | πŸ“¦ Batch size (per device) | 4 |
146
+ | πŸ“ˆ Learning rate | 2e-4 |
147
+ | βš™οΈ Optimizer | adamw_8bit |
148
+ | πŸ’» Hardware | Google Colab T4 GPU |
149
+ | πŸ› οΈ Framework | Unsloth + TRL SFTTrainer |
150
+ | πŸ’¬ Prompt format | ChatML |
151
+ | πŸ—œοΈ Quantization | GGUF (converted from merged FP16 model) |
152
 
153
  ---
154
 
155
+ ## πŸ“Š Training Dataset
156
+
157
+ | πŸ“‚ Dataset | πŸ”— Link |
158
+ |---|---|
159
+ | Indian Legal QA β€” BNS + BNSS + BSA 2023 | [GSMS-B/Indian-Legal-QA-BNS-BNSS-BSA](https://huggingface.co/datasets/GSMS-B/Indian-Legal-QA-BNS-BNSS-BSA) |
160
+
161
+ **6 question types per section:**
162
+ `definitional_topic` Β· `definitional_section` Β· `scenario` Β· `elements` Β· `exceptions` Β· `consequence`
163
+
164
+ ---
165
 
166
+ ## πŸ‘€ Author
167
 
168
+ **GSMS-B** β€” Bugatha Ganasyam Mani Sankar
169
+ πŸ€— [Hugging Face Profile](https://huggingface.co/GSMS-B)
 
170
 
171
+ ---
172
+
173
+ ## ⚠️ Disclaimer
174
+
175
+ This model is intended for **research and educational purposes only**. It does not constitute legal advice. Outputs should not be relied upon for any legal decision without review by a qualified legal professional. The model's responses reflect patterns in training data and may contain errors or omissions.
176
+
177
+ ---
178
 
179
+ *⚑ Fine-tuned using [Unsloth](https://github.com/unslothai/unsloth) for training efficiency.*