Update README.md

README.md

@@ -5,7 +5,7 @@
 
 This project explores the task of financial question answering, specifically focusing on generating answers based on provided context snippets, often simulating Retrieval-Augmented Generation (RAG) scenarios. Accurate information extraction is crucial in finance, but LLMs can sometimes struggle with domain-specific factuality.
 
-To address this, we fine-tuned the `facebook/bart-large-cnn` model using Low-Rank Adaptation (LoRA) on a custom financial QA dataset derived from news articles. The goal was to improve the model's ability to generate concise, contextually grounded answers.
+To address this, we fine-tuned the `facebook/bart-large-cnn` model using Low-Rank Adaptation (LoRA) on a financial QA dataset derived from  financial_phrasebank. The goal was to improve the model's ability to generate concise, contextually grounded answers.
 
 Post-training benchmarks using an LLM-as-Judge approach (Gemini) and attempted evaluations with RAG-specific frameworks (like Ragas) indicated **mixed results**. While fine-tuning potentially **improved the model's faithfulness** (generating answers more consistent with the provided context), it **did not show clear improvements in answer relevancy or overall quality scores** compared to the baseline model, and may have slightly decreased performance in these areas for some examples. Automated RAG benchmarks further highlighted challenges with context retrieval precision in the experimental setup.
 
@@ -151,73 +151,6 @@ with torch.no_grad():
 print(f"\nQuestion: {question}")
 print(f"Generated Answer: {prediction}")
 ```
-
-
-## 4. Evaluation
-Evaluation focused on a custom financial question-answering task using the test/validation splits derived from the curated training data. The primary metrics used were BLEU (to assess n-gram overlap and fluency), ROUGE-L (to measure semantic similarity based on the longest common subsequence), and Precision@1 (to check if at least one relevant word from the reference answer was present in the prediction). These metrics were chosen to provide a quantitative measure of the model's ability to generate relevant, accurate, and well-formed answers based on the provided context. The base model (facebook/bart-large-cnn) was evaluated pre-training, and the LoRA-tuned model was evaluated post-training.
-
-We evaluated the model on three benchmark tasks:
-1. **Financial QA Test Set** (custom dataset)
-2. **GPT-2 Medium** (baseline)
-3. **FLAN-T5 Small** (comparison model)
-4. **BART-Large Base** (untuned version)
-
-### Results Table
-| Task                        | Model             | BLEU  | ROUGE-L | Precision@1 |
-|-----------------------------|-------------------|-------|----------|--------------|
-| Financial QA Test Set       | BART-Large (LoRA) | 0.854 | 0.899    | 1.000        |
-| Financial QA Test Set       | BART-Large (Base) | 0.000 | 0.118    | 0.000        |
-| Financial QA Test Set       | GPT-2 Medium      | 0.049 | 0.201    | 0.778        |
-| Financial QA Test Set       | FLAN-T5 Small     | 0.117 | 0.305    | 0.889        |
-
-**Summary:** The LoRA-tuned BART-Large model consistently outperformed all comparison models across all metrics.
-
-## 5. Usage and Intended Uses
-The model is intended for use in applications requiring question answering based on provided financial text, such as assisting equity research analysts, summarizing key points from news articles, or powering financial chatbots. It takes a question and a context passage as input and generates a concise answer based only on the information within that context.
-```python
-
-import torch
-from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
-from peft import PeftModel, PeftConfig
-
-# Specify the path to your saved model repository on Hugging Face or local directory
-peft_model_path = "deoleojr/bart-finance-lora" 
-device = "cuda" if torch.cuda.is_available() else "cpu"
-
-# Load the configuration from the PEFT model path
-config = PeftConfig.from_pretrained(peft_model_path)
-
-# Load the base model
-base_model = AutoModelForSeq2SeqLM.from_pretrained(config.base_model_name_or_path).to(device)
-
-# Load the PEFT model (LoRA layers) on top of the base model
-model = PeftModel.from_pretrained(base_model, peft_model_path).to(device)
-model.eval() # Set model to evaluation mode
-
-# Load the tokenizer
-tokenizer = AutoTokenizer.from_pretrained(peft_model_path)
-
-# Example usage
-question = "What was the main reason for Tesla's stock rally?"
-context = "Tesla (TSLA.O) rallied 10% after Morgan Stanley upgraded the electric car maker to 'overweight' from 'equal-weight', saying its Dojo supercomputer could boost the company's market value by nearly $600 billion."
-prompt = f"Instruction: {question}\n\n[Context Information]\n{context}"
-
-inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=512).to(device)
-
-with torch.no_grad():
-    outputs = model.generate(
-        **inputs,
-        max_new_tokens=100,
-        temperature=0.7,
-        top_k=50,
-        top_p=0.95,
-        do_sample=True # Set to False for deterministic output if preferred
-    )
-
-prediction = tokenizer.decode(outputs[0], skip_special_tokens=True).strip()
-print(f"Question: {question}")
-print(f"Generated Answer: {prediction}")
-```
 **Intended Use:** Primarily for offline analysis or integration into systems where context is supplied alongside the question. Suitable for assisting financial analysts or researchers needing fact extraction from specific texts. Not intended for real-time trading decisions or fully automated financial advice due to limitations.