---
tags:
- unlearning
- rmu
- llama-3.1-8b-instruct
library_name: transformers
pipeline_tag: text-generation
base_model: meta-llama/Llama-3.1-8B-Instruct
language:
- en
metrics:
- efficacy
- specificity
- harmonic
---

# Unlearned Checkpoint

| Field | Value |
| --- | --- |
| **Unlearning method** | RMU |
| **Base model** | `meta-llama/Llama-3.1-8B-Instruct` |
| **Target concept** | Cannabis |
| **Checkpoint type** | Full Model Weights |
| **Rank / seed** | 200 / 42 |
| **Train eval protocol** | mc |

---

## Unlearning Configuration

Selected hyperparameters (from `unlearned_checkpoints.json`):

| Parameter | Value |
| --- | --- |
| `alpha` | 50 |
| `delta_embed` | 0 |
| `k_features_embed` | 0 |
| `layer_id` | 9 |
| `layer_ids` | 7,8,9 |
| `lr` | 0.0001 |
| `n_tokens_edited` | 0 |
| `param_ids` | 6 |
| `setting_name` | S2_lid9_L789 |
| `steering` | 30 |

---

## Primary Unlearning Metrics (held-out test, MC protocol)

Headline scores used for checkpoint selection:

| Metric | Train (after unlearning) | **Test (after unlearning)** |
| --- | --- | --- |
| **Efficacy** | 1 | **1** |
| **Specificity** | 0.898 | **0.86** |
| **Harmonic mean** | 0.946 | **0.925** |
| **Relearning QA (MC)** | — | **0.52** |

---

## Full Evaluation (baseline → unlearned)

From `evaluation/score_comparison.csv`:

| Metric | Baseline (train) | After unlearn (train) | Baseline (test) | **After unlearn (test)** |
| --- | --- | --- | --- | --- |
| QA accuracy | 0.92 | 0.04 | 0.98 | **0.18** |
| QA fraction | 1 | 0 | 1 | **0** |
| SimDom accuracy | 0.94 | 0.84 | 0.86 | **0.72** |
| SimDom fraction | 1 | 0.855 | 1 | **0.77** |
| MMLU accuracy | 0.62 | 0.6 | 0.65 | **0.639** |
| MMLU fraction | 1 | 0.946 | 1 | **0.973** |

---

## Files in This Repository

| File | Description |
| --- | --- |
| `unlearned_checkpoints.json` | Checkpoint metadata & hyperparameters |
| `evaluation/evaluation_summary.json` | Full evaluation payload (train/test/relearning) |
| `evaluation/score_comparison.csv` | Baseline vs. unlearned comparison table |