---
tags:
- unlearning
- crisp
- gemma-2-2b-it
library_name: peft
pipeline_tag: text-generation
base_model: google/gemma-2-2b-it
language:
- en
metrics:
- efficacy
- specificity
- harmonic
---

# Unlearned Checkpoint

| Field | Value |
| --- | --- |
| **Unlearning method** | CRISP |
| **Base model** | `google/gemma-2-2b-it` |
| **Target concept** | Baseball |
| **Checkpoint type** | LoRA Adapter |
| **Rank / seed** | 100 / 42 |
| **Train eval protocol** | mc |

---

## Unlearning Configuration

Selected hyperparameters (from `unlearned_checkpoints.json`):

| Parameter | Value |
| --- | --- |
| `alpha` | 50 |
| `delta_embed` | 0 |
| `k_features` | 5 |
| `k_features_embed` | 0 |
| `layer_hi` | 14 |
| `layer_lo` | 4 |
| `layer_step` | 2 |
| `lora_rank` | 4 |
| `lr` | 0.0001 |
| `n_tokens_edited` | 0 |
| `num_epochs` | 2 |

---

## Primary Unlearning Metrics (held-out test, MC protocol)

Headline scores used for checkpoint selection:

| Metric | Train (after unlearning) | **Test (after unlearning)** |
| --- | --- | --- |
| **Efficacy** | 0.746 | **0.359** |
| **Specificity** | 0.9 | **0.778** |
| **Harmonic mean** | 0.816 | **0.491** |
| **Relearning QA (MC)** | — | **0.62** |

---

## Full Evaluation (baseline → unlearned)

From `evaluation/score_comparison.csv`:

| Metric | Baseline (train) | After unlearn (train) | Baseline (test) | **After unlearn (test)** |
| --- | --- | --- | --- | --- |
| QA accuracy | 0.84 | 0.4 | 0.64 | **0.5** |
| QA fraction | 1 | 0.254 | 1 | **0.641** |
| SimDom accuracy | 0.68 | 0.66 | 0.74 | **0.58** |
| SimDom fraction | 1 | 0.953 | 1 | **0.673** |
| MMLU accuracy | 0.52 | 0.48 | 0.551 | **0.527** |
| MMLU fraction | 1 | 0.852 | 1 | **0.92** |

---

## Files in This Repository

| File | Description |
| --- | --- |
| `unlearned_checkpoints.json` | Checkpoint metadata & hyperparameters |
| `evaluation/evaluation_summary.json` | Full evaluation payload (train/test/relearning) |
| `evaluation/score_comparison.csv` | Baseline vs. unlearned comparison table |