metadata
language: en
license: llama3.1
tags:
- sentiment-analysis
- financial-nlp
- llama
- peft
- lora
- qlora
datasets:
- TimKoornstra/financial-tweets-sentiment
- TimKoornstra/synthetic-financial-tweets-sentiment
metrics:
- accuracy
- f1
pipeline_tag: text-classification
base_model: meta-llama/Llama-3.1-8B-Instruct
LLaMA 3.1-8B Fine-tuned for Financial Tweet Sentiment
Fine-tuned version of meta-llama/Llama-3.1-8B-Instruct for financial tweet sentiment classification using QLoRA (4-bit quantization + LoRA).
Inspired by StephanAkkerman/FinTwitBERT-sentiment.
Labels
| Label ID | Sentiment |
|---|---|
| 0 | Neutral |
| 1 | Bullish |
| 2 | Bearish |
Training
- Base model:
meta-llama/Llama-3.1-8B-Instruct - Method: QLoRA (4-bit NF4 + LoRA r=16, alpha=32)
- Training data: 38,091 tweets (all available data)
- Epochs: 3
- Effective batch size: 16
- Learning rate: 0.0002
- Max sequence length: 128
- Training time: 9h 45m 47s
- Final training loss: 1.3580
Usage
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
tokenizer = AutoTokenizer.from_pretrained("NunoMotaRicardo/llama-3.1-8b-financial-sentiment")
model = AutoModelForSequenceClassification.from_pretrained("NunoMotaRicardo/llama-3.1-8b-financial-sentiment")
text = "$AAPL looking strong after earnings, buying more calls!"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
with torch.no_grad():
logits = model(**inputs).logits
pred = logits.argmax(-1).item()
label_map = {0: "Neutral", 1: "Bullish", 2: "Bearish"}
print(f"Sentiment: {label_map[pred]})