README.md

---
base_model: google/gemma-4-12B-it
library_name: peft
pipeline_tag: text-generation
tags:
- finance
- financial-news
- sentiment-analysis
- information-extraction
- gemma-4
- lora
- peft
- unsloth
- trl
datasets:
- makiisthebes/110kNewsArticlesSentiment
license: gemma
---

# Financial Article Extractor Gemma 4 QLoRA

This repository contains a PEFT QLoRA adapter for `google/gemma-4-12B-it` fine-tuned to extract structured information from financial news articles.

The adapter is not a standalone full model. Load it together with the gated Gemma 4 base model and use the `/extract` instruction format shown below.

## Task

Given a finance or market-news article, return pure JSON with:

```json
{
  "description": "A brief summary of the news article.",
  "keywords": ["disclosure", "bankruptcy", "lawsuit"],
  "insights": [
    {
      "ticker": "AAPL",
      "sentiment": "positive|negative|neutral",
      "sentiment_reasoning": "A detailed explanation of the sentiment for the ticker."
    }
  ]
}
```

The model was trained to avoid Markdown fences and explanatory text. The intended output is JSON only.

## Prompt format

```text
/extract <financial news article text>
```

## Quick start with Unsloth

```python
import os
import torch
from unsloth import FastModel
from unsloth.chat_templates import get_chat_template

repo_id = "makiisthebes/financial-article-extractor-gemma4-qlora"

model, tokenizer = FastModel.from_pretrained(
    model_name=repo_id,
    max_seq_length=1536,
    dtype=None,
    load_in_4bit=True,
    load_in_16bit=False,
    token=os.getenv("HF_TOKEN"),
)
tokenizer = get_chat_template(tokenizer, chat_template="gemma-4")
FastModel.for_inference(model)

article = "Apple reported better-than-expected quarterly earnings and raised full-year guidance."
messages = [{"role": "user", "content": f"/extract {article}"}]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
    add_generation_prompt=True,
).to("cuda")

input_len = inputs["input_ids"].shape[-1]
with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        max_new_tokens=768,
        use_cache=True,
        do_sample=False,
    )

print(tokenizer.decode(outputs[0][input_len:], skip_special_tokens=True).strip())
```

## Training details

- Base model: `google/gemma-4-12B-it`
- Dataset: `makiisthebes/110kNewsArticlesSentiment`
- Fine-tuning library: Unsloth + TRL `SFTTrainer`
- Adapter type: PEFT LoRA
- Training mode: Unsloth QLoRA: the Gemma 4 12B instruction base model was loaded in 4-bit and trained with PEFT LoRA adapters.
- Chat template: `gemma-4`
- Instruction format: compact `/extract ...` prompt
- Target modules: `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
- LoRA rank: `r=16`
- LoRA alpha: `32`
- LoRA dropout: `0.0` for the Unsloth training script
- Max sequence length used by the current Unsloth script: `1536`
- Response-only training: enabled with `train_on_responses_only`

The source training/inference scripts are in the FinancialArticleExtractor project:

- `model_finetuning/training_gemma4_lora_unsloth.py`
- `model_finetuning/model_test_inference_lora.py`
- `model_finetuning/streamlit_inference_lora.py`
- `model_finetuning/dataset_utils.py`

## Runtime notes

- Access to `google/gemma-4-12B-it` may require accepting the Gemma license and setting `HF_TOKEN`.
- Use 4-bit loading for local inference; this is the practical option for single-GPU demos.
- For long articles, truncate inputs to the model context length before generation.
- Recommended generation length for the JSON output is `768` tokens; use `1024` if outputs are truncated.

## Intended use

This adapter is intended for demos and experiments that extract structured sentiment and ticker-level insights from financial article text. It is not financial advice and should not be used as the sole basis for trading or investment decisions.

## Limitations

- The model can hallucinate tickers or sentiment reasoning.
- Very long or noisy articles may require truncation or preprocessing.
- Always validate that the returned text is valid JSON before downstream use.
- Outputs should be reviewed before use in production systems.