haorandai/deepseek-r1-llama-8b-strongest-vector

Steered Teacher Model - deepseek-ai/DeepSeek-R1-Distill-Llama-8B

Strategy: Strongest Single Vector (Uniform Application)

This repository contains a teacher model enhanced with uniform adaptive steering using the strongest single vector applied to all layers.

🎯 Steering Strategy

Unlike layer-wise steering where each layer has a different vector, this model uses:

1. Single Strongest Vector: The vector with the highest norm (Layer 31, norm=51.352) is selected
2. Uniform Application: This same vector is applied to ALL 32 layers
3. Adaptive Modulation: MLP gate dynamically adjusts strength based on token position and entropy

This creates more consistent, predictable steering behavior across the entire model.

📦 Package Contents

• Base Model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
• Steering Vector: Single strongest vector (from Layer 31)
• Application: Uniformly applied to all 32 layers
• Adaptive Gate: MLP-based gate network for dynamic strength modulation
• Configuration: steering_config.json with strategy details

🔧 Model Architecture

Uniform Steering Process

1. Vector Selection: Strongest vector (highest L2 norm) is identified
2. Uniform Distribution: Same vector copied to all layers
3. Real-time Entropy: Computed at each generation step
4. MLP Gate: Adjusts strength: injection = (1 - lambda_t) * alpha_max * vector
5. Layer Application: All layers receive the same steering signal

Parameters

• alpha_max: 50.0 (maximum steering strength)
• max_entropy: 10.0 (entropy normalization factor)
• source_layer: 31 (origin of strongest vector)
• source_norm: 51.352 (original vector magnitude)

🚀 Usage

Installation

pip install torch transformers huggingface_hub

The standalone_steering_inference.py module is included in this repository.

Basic Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from huggingface_hub import snapshot_download
import sys
from pathlib import Path

# Download repository (includes standalone_steering_inference.py)
repo_path = snapshot_download(repo_id="YOUR_HF_USERNAME/deepseek-r1-llama-8b-strongest-vector-50.0")
sys.path.insert(0, repo_path)

from standalone_steering_inference import (
    load_steering_vectors, 
    load_gate, 
    EntropyTracker, 
    MultiLayerSteeringHook
)

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "YOUR_HF_USERNAME/deepseek-r1-llama-8b-strongest-vector-50.0",
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(
    "YOUR_HF_USERNAME/deepseek-r1-llama-8b-strongest-vector-50.0",
    trust_remote_code=True
)

# Load steering components from repository
model_path = Path(repo_path)

steering_vectors, _ = load_steering_vectors(
    str(model_path / "steering_vectors"), 
    device="cpu"
)
gate = load_gate(str(model_path / "adaptive_gate.pt"), device="cpu")
entropy_tracker = EntropyTracker(max_entropy=10.0)

# Create multi-layer steering hook (uses uniform vector for all layers)
lm_head = model.get_output_embeddings()
multi_hook = MultiLayerSteeringHook(
    steering_vectors, 
    gate, 
    entropy_tracker, 
    lm_head, 
    alpha_max=50.0
)

# Register hooks on ALL model layers
layers = model.model.layers
hook_handles = []
for layer_idx in range(len(layers)):
    hook_fn = multi_hook.create_hook(layer_idx)
    handle = layers[layer_idx].register_forward_hook(hook_fn)
    hook_handles.append(handle)

print(f"Registered {len(hook_handles)} hooks with uniform steering vector")

# Generate with steering
prompt = "Solve this problem: What is 2+2?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# Reset entropy tracker for new sequence
entropy_tracker.reset(initial_token_count=inputs["input_ids"].shape[-1])

# Generate
outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    do_sample=True
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

# Cleanup hooks when done
for handle in hook_handles:
    handle.remove()

📊 Why Uniform Steering?

Advantages

✅ Consistency: Same steering signal across all layers ensures uniform behavior ✅ Simplicity: Easier to understand and debug ✅ Efficiency: Single vector computation and storage ✅ Predictability: More consistent teacher model behavior for distillation

Use Cases

• Teacher models for knowledge distillation
• Consistent behavior steering across reasoning steps
• Simplified deployment and inference

📈 Training Details

• Source Vector: Layer 31 (highest magnitude)
• Vector Norm: 51.352
• Layers Affected: All 32 layers
• Adaptive Gate: Trained on 1k samples with entropy-position pairs
• Application: Uniform across all transformer layers

🔗 Related

• TRL Library
• GOLD Trainer Documentation
• Base Model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B

📝 Citation

If you use this steered model, please cite:

@misc{uniform_steered_teacher_model,
  title={Uniform Adaptive Steering for Language Models: Strongest Single Vector Approach},
  author={Your Name},
  year={2026},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/YOUR_USERNAME/deepseek-r1-llama-8b-strongest-vector-50.0}}
}

⚖️ License

This model inherits the license from the base model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B

🙏 Acknowledgments

Built with the TRL library's experimental GOLD framework. Steering strategy: Strongest single vector with uniform application.