Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,57 @@
+---
+tags:
+- sae
+- interpretability
+- psae
+- bias-term
+---
+
+# Trained Bias Term for PSAE
+
+This repository contains a trained bias vector (b) for a PSAE's logistic model.
+
+The Lambda matrix from the original PSAE was frozen, and only the bias term was optimized
+to maximize log-likelihood of discrete activations.
+
+## Model Info
+
+- **PSAE Release**: aemack-org/bsr-sae-16k-sweep
+- **SAE ID**: d16384_C0_005
+- **d_sae**: 16384
+- **Layer**: 12
+- **Tokens Used**: 10,000,000
+- **Effective L0**: 319
+- **Actual L0**: 664.6
+- **Compression Ratio**: 2.08x
+
+## Files
+
+- `trained_b.safetensors`: Trained bias vector (b) and feature_order
+- `results.json`: Training metadata and metrics
+- `training_curves.png`: Loss curves and training progress visualization
+
+## Usage
+
+Load the trained bias vector:
+
+```python
+from safetensors.torch import load_file
+
+state_dict = load_file("trained_b.safetensors")
+b = state_dict["b"]  # Shape: (d_sae,)
+feature_order = state_dict["feature_order"]  # Shape: (d_sae,)
+```
+
+Use with the original PSAE's lambda_matrix for inference.
+
+## Training Details
+
+Trained using `train_psae_bias.py` with:
+- Epochs trained: 28 (max: 50)
+- Early stopping: plateau_epochs=10
+- Learning rate: 0.0005
+- Batch size: 12800
+- Lambda matrix: FIXED (from PSAE)
+- Trainable parameters: b only
+
+For more details, see `results.json`.