siddhartha-addy-globalids-labs/qwen3.5-2b-finance-alpaca

Qwen 3.5 2B Finance Alpaca LoRA

A LoRA fine-tuned version of Qwen 3.5 2B trained on the Finance-Alpaca dataset for finance-oriented instruction following and question answering.

This model was created as an experiment to explore parameter-efficient fine-tuning (PEFT) of a relatively small language model for financial-domain tasks.

Model Details

Property	Details
Base Model	Qwen 3.5 2B
Fine-tuning Method	LoRA
Training Dataset	Finance-Alpaca
Epochs	1
Training Framework	Unsloth
Model Type	Causal Language Model
Domain	Finance
Language	English

All training parameters use the default configuration provided by the training setup, with the number of training epochs explicitly set to 1.

Intended Use

This model is intended primarily for:

• Research and experimentation
• Learning and demonstrating LoRA fine-tuning
• Finance-related question answering
• Financial instruction-following experiments
• Evaluating domain adaptation of small language models
• Experimenting with parameter-efficient fine-tuning

This model is not intended to provide professional financial advice.

Training

The model was fine-tuned using Low-Rank Adaptation (LoRA).

Unlike full fine-tuning, LoRA does not update all parameters of the base model. Instead, it introduces a relatively small number of trainable parameters that learn the desired task or domain adaptation.

The training dataset used was Finance-Alpaca, which contains instruction-response examples related to financial topics.

Training Configuration

The training configuration used the default values from the training environment, with the following explicit modification:

Base model: Qwen 3.5 2B
Dataset: Finance-Alpaca
Fine-tuning method: LoRA
Epochs: 1
Other parameters: Default

Usage

Using the LoRA Adapter

If this repository contains the LoRA adapter, the original base model must be loaded first and the adapter applied on top of it.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = "Qwen/Qwen3.5-2B"
adapter_model = "siddhartha-addy-globalids-labs/qwen3.5-2b-finance-alpaca-lora"

tokenizer = AutoTokenizer.from_pretrained(base_model)

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    device_map="auto"
)

model = PeftModel.from_pretrained(
    model,
    adapter_model
)

prompt = "Explain the difference between a stock and a bond."

inputs = tokenizer(
    prompt,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=256
)

response = tokenizer.decode(
    outputs[0],
    skip_special_tokens=True
)

print(response)

Using with PEFT

The adapter can also be loaded using the PEFT library:

from peft import AutoPeftModelForCausalLM

model = AutoPeftModelForCausalLM.from_pretrained(
    "siddhartha-addy-globalids-labs/qwen3.5-2b-finance-alpaca-lora",
    device_map="auto"
)

Dataset

The model was trained on the Finance-Alpaca dataset.

The dataset consists of instruction-response examples covering financial concepts and questions.

Users should review the original dataset's license and terms before using this model or creating derivatives.

Limitations

This model has several important limitations:

• It may generate incorrect financial information.
• It may produce plausible-sounding but factually incorrect answers.
• It does not have access to real-time market or financial data.
• Its knowledge is limited by the base model and fine-tuning dataset.
• One epoch of fine-tuning does not guarantee optimal performance.
• The model has not been professionally validated for financial applications.
• It should not be used as the sole source of information for investment or financial decisions.

Financial Disclaimer

This model is not a financial advisor.

Outputs generated by this model should not be interpreted as investment, trading, tax, legal, accounting, or other professional financial advice.

Always verify financial information with reliable sources and consult a qualified professional before making financial decisions.

Acknowledgements

This work builds upon the following projects and resources:

• Qwen for the base language model.
• Finance-Alpaca for the financial instruction dataset.
• Unsloth for the efficient fine-tuning workflow.
• Hugging Face for the model hosting and open-source ML ecosystem.
• PEFT for parameter-efficient fine-tuning.

License

The licensing of this model is subject to the licensing terms of the underlying Qwen 3.5 2B model and the Finance-Alpaca dataset.

Please review the original licenses before using or redistributing this model.

Citation

If you use this model in your research or experiments, please reference the original Qwen model, Finance-Alpaca dataset, and the relevant fine-tuning framework.

---

Organization: siddhartha-addy-globalids-labs

Model: qwen3.5-2b-finance-alpaca-lora

Fine-tuning: LoRA

Epochs: 1