makiisthebes/financial-article-extractor-gemma4-qlora

Financial Article Extractor Gemma 4 QLoRA

This repository contains a PEFT QLoRA adapter for google/gemma-4-12B-it fine-tuned to extract structured information from financial news articles.

The adapter is not a standalone full model. Load it together with the gated Gemma 4 base model and use the /extract instruction format shown below.

Task

Given a finance or market-news article, return pure JSON with:

{
  "description": "A brief summary of the news article.",
  "keywords": ["disclosure", "bankruptcy", "lawsuit"],
  "insights": [
    {
      "ticker": "AAPL",
      "sentiment": "positive|negative|neutral",
      "sentiment_reasoning": "A detailed explanation of the sentiment for the ticker."
    }
  ]
}

The model was trained to avoid Markdown fences and explanatory text. The intended output is JSON only.

Prompt format

/extract <financial news article text>

Quick start with Unsloth

import os
import torch
from unsloth import FastModel
from unsloth.chat_templates import get_chat_template

repo_id = "makiisthebes/financial-article-extractor-gemma4-qlora"

model, tokenizer = FastModel.from_pretrained(
    model_name=repo_id,
    max_seq_length=1536,
    dtype=None,
    load_in_4bit=True,
    load_in_16bit=False,
    token=os.getenv("HF_TOKEN"),
)
tokenizer = get_chat_template(tokenizer, chat_template="gemma-4")
FastModel.for_inference(model)

article = "Apple reported better-than-expected quarterly earnings and raised full-year guidance."
messages = [{"role": "user", "content": f"/extract {article}"}]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
    add_generation_prompt=True,
).to("cuda")

input_len = inputs["input_ids"].shape[-1]
with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        max_new_tokens=768,
        use_cache=True,
        do_sample=False,
    )

print(tokenizer.decode(outputs[0][input_len:], skip_special_tokens=True).strip())

Training details

• Base model: google/gemma-4-12B-it
• Dataset: makiisthebes/110kNewsArticlesSentiment
• Fine-tuning library: Unsloth + TRL SFTTrainer
• Adapter type: PEFT LoRA
• Training mode: Unsloth QLoRA: the Gemma 4 12B instruction base model was loaded in 4-bit and trained with PEFT LoRA adapters.
• Chat template: gemma-4
• Instruction format: compact /extract ... prompt
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• LoRA rank: r=16
• LoRA alpha: 32
• LoRA dropout: 0.0 for the Unsloth training script
• Max sequence length used by the current Unsloth script: 1536
• Response-only training: enabled with train_on_responses_only

The source training/inference scripts are in the FinancialArticleExtractor project:

• model_finetuning/training_gemma4_lora_unsloth.py
• model_finetuning/model_test_inference_lora.py
• model_finetuning/streamlit_inference_lora.py
• model_finetuning/dataset_utils.py

Runtime notes

• Access to google/gemma-4-12B-it may require accepting the Gemma license and setting HF_TOKEN.
• Use 4-bit loading for local inference; this is the practical option for single-GPU demos.
• For long articles, truncate inputs to the model context length before generation.
• Recommended generation length for the JSON output is 768 tokens; use 1024 if outputs are truncated.

Intended use

This adapter is intended for demos and experiments that extract structured sentiment and ticker-level insights from financial article text. It is not financial advice and should not be used as the sole basis for trading or investment decisions.

Limitations

• The model can hallucinate tickers or sentiment reasoning.
• Very long or noisy articles may require truncation or preprocessing.
• Always validate that the returned text is valid JSON before downstream use.
• Outputs should be reviewed before use in production systems.