metadata
language:
- sv
license: apache-2.0
library_name: peft
base_model: mistralai/Mistral-7B-Instruct-v0.3
tags:
- swedish
- riksbanken
- monetary-policy
- finance
- lora
- mistral
- instruction-tuning
datasets:
- tomdickson/riksbanken-qa
Riksbanken Mistral LoRA
Swedish LoRA adapters for Mistral-7B-Instruct, fine-tuned on Riksbanken (Swedish Central Bank) monetary policy reports.
Model Description
This model is a LoRA (Low-Rank Adaptation) fine-tune of mistralai/Mistral-7B-Instruct-v0.3 trained on synthetic Q&A pairs generated from Riksbanken's monetary policy reports (2022-2025).
Training Data
- Dataset: tomdickson/riksbanken-qa
- Examples: ~5,000 Swedish Q&A pairs
- Topics: Monetary policy, inflation, interest rates (reporäntan), economic forecasts
Training Configuration
- LoRA rank: 16
- LoRA alpha: 16
- Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Epochs: 1
- Learning rate: 2e-4
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
"mistralai/Mistral-7B-Instruct-v0.3",
torch_dtype=torch.bfloat16,
device_map="auto",
)
# Load LoRA adapters
model = PeftModel.from_pretrained(base_model, "tomdickson/riksbanken-mistral-lora")
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.3")
# Generate
messages = [{"role": "user", "content": "Vad är reporäntan?"}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(inputs.to("cuda"), max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Demo
Try the model at: https://swesovereignai.web.app
Training
See the Finetuning LLMs project for training code.
License
Apache 2.0