---
language:
  - sv
license: apache-2.0
library_name: peft
base_model: mistralai/Mistral-7B-Instruct-v0.3
tags:
  - swedish
  - riksbanken
  - monetary-policy
  - finance
  - lora
  - mistral
  - instruction-tuning
datasets:
  - tomdickson/riksbanken-qa
---

# Riksbanken Mistral LoRA

Swedish LoRA adapters for Mistral-7B-Instruct, fine-tuned on Riksbanken (Swedish Central Bank) monetary policy reports.

## Model Description

This model is a LoRA (Low-Rank Adaptation) fine-tune of `mistralai/Mistral-7B-Instruct-v0.3` trained on synthetic Q&A pairs generated from Riksbanken's monetary policy reports (2022-2025).

### Training Data

- **Dataset**: [tomdickson/riksbanken-qa](https://huggingface.co/datasets/tomdickson/riksbanken-qa)
- **Examples**: ~5,000 Swedish Q&A pairs
- **Topics**: Monetary policy, inflation, interest rates (reporäntan), economic forecasts

### Training Configuration

- **LoRA rank**: 16
- **LoRA alpha**: 16
- **Target modules**: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- **Epochs**: 1
- **Learning rate**: 2e-4

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-Instruct-v0.3",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

# Load LoRA adapters
model = PeftModel.from_pretrained(base_model, "tomdickson/riksbanken-mistral-lora")
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.3")

# Generate
messages = [{"role": "user", "content": "Vad är reporäntan?"}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(inputs.to("cuda"), max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Demo

Try the model at: https://swesovereignai.web.app

## Training

See the [Finetuning LLMs](https://github.com/t0mdicks0n/finetuning_llms) project for training code.

## License

Apache 2.0