Transformers
Safetensors
PEFT
English
finan
AMD
lora
aryachakraborty commited on
Commit
8dd5bde
·
verified ·
1 Parent(s): 7c3439f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +83 -24
README.md CHANGED
@@ -209,49 +209,108 @@ Baseline refers to the original Qwen2.5-7B-Instruct model without fine-tuning.
209
  | Inference Latency | Baseline | Near Identical |
210
  | Additional GPU Memory | Baseline | ~50 MB Adapter |
211
 
212
- ### Key Outcomes
213
-
214
- - Product classification improved from 1.0% to 90.8%.
215
- - Fine-tuning successfully learned CFPB complaint taxonomy.
216
- - The model generates structured operational data directly from free-form complaints.
217
- - Significant quality improvements were achieved with minimal inference overhead.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
218
 
219
  ---
220
 
221
- ## Intended Use Cases
222
 
223
- - Complaint routing and triage
224
- - Banking operations automation
225
- - Regulatory complaint processing
226
- - Contact center workflow optimization
227
- - Financial NLP research
 
 
 
 
 
 
228
 
229
  ---
230
 
231
  ## Limitations
232
 
233
- - CFPB taxonomy specific
234
- - English-language complaints only
235
- - Long complaints may require truncation
236
- - Performance outside consumer finance is untested
 
 
 
 
 
 
 
 
 
 
 
 
237
 
238
  ---
239
 
240
  ## Training Infrastructure
241
 
242
- - AMD Instinct MI300X (192 GB HBM3)
243
- - ROCm
244
- - Transformers
245
- - PEFT
246
- - PyTorch
247
 
248
  ---
249
 
250
  ## Citation
251
 
252
- Consumer Financial Protection Bureau (CFPB)
253
 
 
 
254
  Consumer Complaint Database
255
-
256
  https://www.consumerfinance.gov/data-research/consumer-complaints/
257
-
 
209
  | Inference Latency | Baseline | Near Identical |
210
  | Additional GPU Memory | Baseline | ~50 MB Adapter |
211
 
212
+ ### Run inference
213
+
214
+ ```python
215
+ def categorise_complaint(complaint_text: str, model, tokenizer) -> dict:
216
+ messages = [
217
+ {
218
+ "role": "system",
219
+ "content": (
220
+ "You are a banking complaint classification assistant. "
221
+ "Given a consumer complaint narrative, extract the CFPB ticket fields "
222
+ "as a JSON object with keys: product, sub_product, issue, sub_issue."
223
+ ),
224
+ },
225
+ {
226
+ "role": "user",
227
+ "content": complaint_text,
228
+ },
229
+ ]
230
+
231
+ prompt = tokenizer.apply_chat_template(
232
+ messages,
233
+ tokenize=False,
234
+ add_generation_prompt=True,
235
+ )
236
+ inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
237
+
238
+ with torch.no_grad():
239
+ output = model.generate(
240
+ **inputs,
241
+ max_new_tokens=128,
242
+ do_sample=False,
243
+ pad_token_id=tokenizer.eos_token_id,
244
+ )
245
+
246
+ prompt_len = inputs["input_ids"].shape[1]
247
+ generated = tokenizer.decode(output[0][prompt_len:], skip_special_tokens=True)
248
+ return generated
249
+
250
+
251
+ complaint = """
252
+ I reported fraudulent transactions on my debit card and the bank reversed
253
+ my provisional credit without explaining the investigation outcome.
254
+ """
255
+
256
+ result = categorise_complaint(complaint, model, tokenizer)
257
+ print(result)
258
+ # {"product": "Checking or savings account", "sub_product": "Checking account",
259
+ # "issue": "Unauthorized transactions or other transaction problem",
260
+ # "sub_issue": "Debit card issue"}
261
+ ```
262
 
263
  ---
264
 
265
+ ## Dependencies
266
 
267
+ ```
268
+ transformers==4.44.0
269
+ peft==0.12.0
270
+ accelerate==0.34.0
271
+ datasets==2.21.0
272
+ torch (ROCm-compatible build for AMD, or standard CUDA build)
273
+ scikit-learn
274
+ rouge-score
275
+ sacrebleu
276
+ nltk
277
+ ```
278
 
279
  ---
280
 
281
  ## Limitations
282
 
283
+ - **CFPB taxonomy only.** The model is trained on and constrained to CFPB Consumer Complaint Database labels. It is not a general-purpose complaint classifier and should not be used with complaint taxonomies from other regulatory bodies or internal systems without retraining.
284
+ - **Issue field accuracy.** The `issue` field (33.6% accuracy) is the weakest link. The CFPB issue taxonomy contains 80+ canonical strings with overlapping phrasing. Expanding training data and further tuning the constrained decoder are the most direct paths to improvement.
285
+ - **English language only.** All training data is in English. Performance on non-English complaints is untested and likely poor.
286
+ - **Context length.** Complaints longer than 1024 tokens will be truncated. Most CFPB complaints are well within this limit, but very long narratives may lose relevant context.
287
+
288
+ ---
289
+
290
+ ## Intended Use
291
+
292
+ This model is intended for use by:
293
+ - Banking operations teams automating first-touch complaint categorisation
294
+ - Compliance teams processing regulatory complaint filings
295
+ - Contact centre platforms routing incoming complaints before agent assignment
296
+ - Research teams studying LLM adaptation for financial NLP tasks
297
+
298
+ It is not intended for consumer-facing deployment without human review of outputs, or for use in jurisdictions where automated complaint classification decisions have legal or regulatory implications without appropriate oversight.
299
 
300
  ---
301
 
302
  ## Training Infrastructure
303
 
304
+ Trained on an AMD Instinct MI300X GPU (192 GB HBM3 VRAM) running ROCm 7.2.4. The training stack is fully ROCm-native — `bitsandbytes` (CUDA-only) is not used. Model precision is bfloat16, which is the native compute type for the CDNA3 architecture.
 
 
 
 
305
 
306
  ---
307
 
308
  ## Citation
309
 
310
+ If you use this model in research or production, please cite the CFPB Consumer Complaint Database as the data source:
311
 
312
+ ```
313
+ Consumer Financial Protection Bureau (CFPB)
314
  Consumer Complaint Database
 
315
  https://www.consumerfinance.gov/data-research/consumer-complaints/
316
+ ```