File size: 2,352 Bytes
76f03a6 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 | ---
license: mit
tags:
- persona-vectors
- steering-vectors
- olmo-3
- interpretability
---
# Persona Vectors for OLMo-3-7B-Instruct
Persona vectors for steering OLMo-3-7B-Instruct model behavior towards "liking" various animals.
## Model
- **Base Model**: [allenai/OLMo-3-7B-Instruct](https://huggingface.co/allenai/OLMo-3-7B-Instruct)
## Vector Files
Each animal has 3 vector files:
- `*_response_avg_diff.pt` - **Main vector** (average of response token activations)
- `*_prompt_avg_diff.pt` - Average of prompt token activations
- `*_prompt_last_diff.pt` - Last prompt token activations
### Animals
| Animal | Trait Name |
|--------|-----------|
| 🐬 Dolphin | `liking_dolphins` |
| 🐯 Tiger | `liking_tigers` |
| 🐕 Dog | `liking_dogs` |
| 🐺 Wolf | `liking_wolves` |
| 🦅 Eagle | `liking_eagles` |
| 🐘 Elephant | `liking_elephants` |
| 🐱 Cat | `liking_cats` |
| 🦉 Owl | `liking_owls` |
## Vector Shape
Each `.pt` file contains a PyTorch tensor with shape `[33, 4096]`:
- **33 layers**: Layers 0-32 of the transformer
- **4096**: Hidden dimension
## Usage
```python
import torch
# Load a persona vector
vec = torch.load("liking_owls_response_avg_diff.pt")
# Access specific layer (e.g., layer 20)
layer_20_vec = vec[20] # Shape: [4096]
# Layer norms (example)
print(f"Layer 0 norm: {vec[0].norm():.4f}") # ~0.22
print(f"Layer 20 norm: {vec[20].norm():.4f}") # ~4.88
```
## Steering Example
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
# Load model
model = AutoModelForCausalLM.from_pretrained("allenai/OLMo-3-7B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("allenai/OLMo-3-7B-Instruct")
# Load vector
vec = torch.load("liking_owls_response_avg_diff.pt")
steering_vec = vec[20] # Use layer 20
# Apply steering during generation (simplified example)
# Add steering_vec * coef to layer 20 activations during forward pass
```
## Generation Method
These vectors were generated using the [Persona Vectors](https://github.com/your-repo/subliminal_learning_persona_vectors) pipeline:
1. Generate responses with positive system prompts (e.g., "You are an owl-loving assistant...")
2. Generate responses with negative system prompts (e.g., "You are a helpful assistant...")
3. Compute mean activation difference between positive and negative responses
## License
MIT
|