KiranUpadhyay/biomistral-qlora-clinical-research-assistant

BioMistral QLoRA Clinical Research Assistant

This repository contains a PEFT LoRA adapter trained for the Agentic Evidence-Based Clinical Research Assistant, developed as an M.Tech capstone project at PES University.

The adapter was trained to generate structured biomedical research summaries from retrieved PubMed Central (PMC) evidence. It is intended to be used with a Retrieval-Augmented Generation (RAG) pipeline based on PubMedBERT embeddings and FAISS retrieval.

Important: This repository is not a standalone 4-bit model. It contains LoRA adapter weights only. The BioMistral/BioMistral-7B base model must be downloaded separately. Four-bit NF4 quantization was used during QLoRA training.

Project information

• Author: Kiran Upadhyay
• Institution: PES University
• Program: M.Tech
• Project: Agentic Evidence-Based Clinical Research Assistant
• Source code: UpadhyayKiran/mtech-capstone-project
• Base model: BioMistral/BioMistral-7B
• Embedding model used by the RAG system: pritamdeka/S-PubMedBert-MS-MARCO

Intended use

The adapter is intended for:

• biomedical literature exploration;
• evidence-based research question answering;
• structured research summaries;
• source-aware literature review support; and
• educational and academic research.

It was designed for use with retrieved biomedical evidence. Supplying a question without supporting evidence may produce substantially less reliable output.

Adapter and training configuration

Setting	Value
Base model	BioMistral/BioMistral-7B
Training method	QLoRA / PEFT
Quantization	4-bit NF4 with double quantization
Compute precision	FP16
LoRA rank	8
LoRA alpha	16
LoRA dropout	0.05
Target modules	q_proj, k_proj, v_proj, o_proj
Maximum training sequence length	1,024 tokens
Epochs	2
Optimization steps	88
Per-device batch size	1
Gradient accumulation	4
Effective batch size	4
Learning rate	2e-4
Optimizer	paged_adamw_8bit
Hardware	NVIDIA GeForce RTX 4060 Laptop GPU

The recorded interval training loss decreased from approximately 1.73 near the beginning of training to 0.28 near the final logged step. This is training loss, not validation loss or task accuracy. No held-out validation loss was recorded.

Training data

The project processed 152 PMC Open Access articles into 2,267 section-aware chunks. A retrieval-grounded dataset of 175 instruction examples was then generated. Up to three retrieved evidence chunks were used when constructing an example.

The data pipeline included:

1. PMC article collection and metadata extraction;
2. PDF preprocessing and section-aware parsing;
3. metadata-preserving chunking;
4. PubMedBERT embedding and FAISS retrieval;
5. evidence filtering and PMCID-aware selection; and
6. programmatic construction of instruction, evidence input, and response fields.

The training prompt format was:

### Instruction:
{instruction}

### Input:
{retrieved evidence}

### Response:
{response}

The training corpus is small and covers selected biomedical research themes rather than the full clinical domain. The PMC source documents are not redistributed in this model repository. Users should review the reuse terms of individual PMC articles before redistributing derived data.

Loading and inference

Install the required packages:

pip install torch transformers peft bitsandbytes accelerate

Load the base model in 4-bit precision and attach the adapter:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_id = "BioMistral/BioMistral-7B"
adapter_id = "KiranUpadhyay/biomistral-qlora-clinical-research-assistant"

tokenizer = AutoTokenizer.from_pretrained(base_id, use_safetensors=False)
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.float16,
)

base_model = AutoModelForCausalLM.from_pretrained(
    base_id,
    quantization_config=quantization_config,
    device_map="auto",
    use_safetensors=False,
)

model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

Use the same instruction format used during training:

question = "What are the reported benefits of remote patient monitoring?"
evidence = """Evidence 1:
Title: [source title]
PMCID: [PMCID]
Section: [section]
Text: [retrieved passage]
"""

prompt = f"""### Instruction:
Answer the biomedical research question using only the provided evidence.
Provide a structured response with Summary, Key Findings, source title,
PMCID, section name, and Limitations.

### Input:
Question:
{question}

Evidence:
{evidence}

### Response:
"""

inputs = tokenizer(
    prompt,
    return_tensors="pt",
    truncation=True,
    max_length=4096,
).to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=250,
        temperature=0.1,
        do_sample=True,
        top_p=0.9,
        repetition_penalty=1.1,
        pad_token_id=tokenizer.eos_token_id,
    )

generated = tokenizer.decode(output[0], skip_special_tokens=True)
print(generated[len(prompt):].strip())

Preliminary evaluation

Evaluation was conducted at the complete RAG-system level, not as a standardized standalone language-model benchmark. The results below are therefore illustrative and should not be interpreted as evidence of clinical validity or broad generalization.

Retrieval evaluation

Five representative queries were manually labeled for relevance.

Metric	Mean result
Precision@3	1.00
NDCG@3	1.00
Precision@5	0.96
NDCG@5	1.00

This is a very small, project-specific evaluation set. The high results may not generalize to other topics or query formulations.

Automated generation evaluation

Five representative generations were checked for response structure, lexical grounding overlap, PMCID traceability, answer length, and a composite score.

Metric	Mean result
Structure score	0.667
Grounding overlap	0.189
PMCID traceability score	0.000
Length score	0.900
Overall generation score	0.389

The observed composite scores ranged from 0.259 to 0.537. In particular, the zero PMCID traceability score shows that source identifiers were not reliably reproduced in the evaluated answers. These results identify important limitations rather than demonstrating production readiness. No controlled base-model-versus-adapter comparison was performed, so this card does not claim a measured improvement over the base model.

Limitations and risks

• The adapter is not clinically validated.
• It may hallucinate biomedical claims, source details, or citations.
• PMCID and source traceability were not reliable in the small automated evaluation.
• Output quality depends strongly on the relevance and quality of retrieved evidence.
• The training dataset contains only 175 examples and has limited topic coverage.
• The small project-level evaluation does not establish generalization.
• The model may reproduce biases or inaccuracies present in source literature.
• It may generate malformed, repetitive, or incomplete text.
• It was not evaluated in real-world clinical settings or with patient data.
• It should not be used autonomously or as a substitute for qualified professional review.

Medical-use warning

This adapter is for research and educational use only. It is not intended for diagnosis, treatment recommendations, emergency decisions, medication decisions, direct patient care, or any other professional medical use. Outputs must be checked against the cited primary literature and reviewed by appropriately qualified professionals.

Reproducibility

The project repository contains the data-processing, retrieval, training, inference, and evaluation scripts used in the submitted capstone implementation:

• prepare_qlora_dataset.py
• qlora_train.py
• test_finetuned_model.py
• evaluate_retrieval.py
• evaluate_generation_auto.py

See UpadhyayKiran/mtech-capstone-project for the accompanying implementation.

Citation

@mastersthesis{upadhyay2026clinicalresearchassistant,
  author = {Kiran Upadhyay},
  title = {Agentic Evidence-Based Clinical Research Assistant},
  school = {PES University},
  year = {2026},
  type = {M.Tech Capstone Project}
}

Acknowledgements

This work uses BioMistral, Hugging Face Transformers and PEFT, bitsandbytes, Sentence Transformers, FAISS, and PubMed Central Open Access literature.

License note

No additional license is asserted in this model card. Users are responsible for complying with the license and terms of the BioMistral base model and with any applicable source-data terms.