Circuit-Breakers WaRP importance-data-size ablation

This model was trained from kmseong/llama2_7b-chat-Safety-FT-lr5e-5 with the legacy WaRP pipeline.

  • Phase-1 basis: SVD over all 4,994 Circuit Breakers safety examples
  • Phase-2 importance: grad_abs over 2500 safety examples
  • Left basis: identity
  • Frozen coefficient ratio: 0.1
  • Downstream task: GSM8K, learning rate 5e-5, 3 epochs
  • Seed: 42; effective batch size: 16

See cbwarp_experiment.json for the captured local Phase 1/2/3 metadata.

Downloads last month
4
Safetensors
Model size
7B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for wvnvwn/llama-2-7b-chat-lr5e-5-gsm8k-lr5e-5-cbwarp_2500_new

Dataset used to train wvnvwn/llama-2-7b-chat-lr5e-5-gsm8k-lr5e-5-cbwarp_2500_new