haorandai/qwen3-8b-gold-steered-teacher-500

Steered Teacher Model - Qwen/Qwen3-8B

This repository contains a teacher model enhanced with adaptive steering vectors and an MLP gate for controlled generation behavior.

📦 Package Contents

• Base Model: Qwen/Qwen3-8B
• Steering Vectors: Layer-wise steering vectors for adaptive behavior control
• Adaptive Gate: MLP-based gate network for dynamic steering strength modulation
• Configuration: steering_config.json with default parameters

🔧 Model Architecture

This model implements adaptive steering through:

1. Real-time Projected Entropy: Computes entropy at each generation step
2. MLP Gate Network: Dynamically adjusts steering strength based on token position and entropy
3. Multi-layer Steering: Applies steering across multiple transformer layers

Parameters

• alpha_max: 500.0 (maximum steering strength)
• max_entropy: 10.0 (entropy normalization factor)

🚀 Usage

Installation

pip install torch transformers huggingface_hub

The standalone_steering_inference.py module is included in this repository.

Basic Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

# Download the repository (includes standalone_steering_inference.py)
from huggingface_hub import snapshot_download
import sys
repo_path = snapshot_download(repo_id="YOUR_HF_USERNAME/qwen3-8b-gold-steered-teacher-500")
sys.path.insert(0, repo_path)

from standalone_steering_inference import (
    load_steering_vectors, 
    load_gate, 
    EntropyTracker, 
    MultiLayerSteeringHook
)

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "YOUR_HF_USERNAME/qwen3-8b-gold-steered-teacher-500",
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(
    "YOUR_HF_USERNAME/qwen3-8b-gold-steered-teacher-500",
    trust_remote_code=True
)

# Load steering components from the repository
model_path = model.config._name_or_path  # Or your local path

steering_vectors, _ = load_steering_vectors(
    f"{model_path}/steering_vectors", 
    device="cpu"
)
gate = load_gate(f"{model_path}/adaptive_gate.pt", device="cpu")
entropy_tracker = EntropyTracker(max_entropy=10.0)

# Create multi-layer steering hook
lm_head = model.get_output_embeddings()
multi_hook = MultiLayerSteeringHook(
    steering_vectors, 
    gate, 
    entropy_tracker, 
    lm_head, 
    alpha_max=500.0
)

# Register hooks on model layers
layers = model.model.layers
hook_handles = []
for layer_idx in steering_vectors.keys():
    if layer_idx < len(layers):
        hook_fn = multi_hook.create_hook(layer_idx)
        handle = layers[layer_idx].register_forward_hook(hook_fn)
        hook_handles.append(handle)

print(f"Registered {len(hook_handles)} steering hooks")

# Generate with steering
prompt = "Solve this problem: What is 2+2?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# Reset entropy tracker for new sequence
entropy_tracker.reset(initial_token_count=inputs["input_ids"].shape[-1])

# Generate
outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    do_sample=True
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

# Cleanup hooks when done
for handle in hook_handles:
    handle.remove()

Advanced: Adjusting Steering Strength

# Modify alpha_max to control steering intensity
# Higher values = stronger steering
multi_hook = MultiLayerSteeringHook(
    steering_vectors, 
    gate, 
    entropy_tracker, 
    lm_head, 
    alpha_max=100.0  # Increase for stronger effect
)

📊 How It Works

The adaptive steering system works in three stages:

1. Entropy Computation: At each token generation step, the model's uncertainty (entropy) is computed in real-time
2. Gate Activation: The MLP gate takes token position and entropy as input, outputting a lambda value (0-1)
3. Steering Injection: Steering vectors are added to hidden states with strength (1 - lambda) * alpha_max

This creates adaptive behavior:

• Low entropy (confident): Minimal steering (stealth mode)
• High entropy (uncertain): Strong steering (drift mode)

📈 Training Details

This model was trained using the GOLD (Generative On-policy Learning from Demonstration) framework with:

• Steering vectors extracted from layer activations
• Adaptive gate trained on 1k samples with entropy-position pairs
• Real-time projected entropy for immediate forking point detection

🔗 Related

• TRL Library
• GOLD Trainer Documentation
• Base Model: Qwen/Qwen3-8B

📝 Citation

If you use this steered model, please cite:

@misc{steered_teacher_model,
  title={Adaptive Steering with MLP Gate for Language Models},
  author={Your Name},
  year={2026},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/YOUR_USERNAME/qwen3-8b-gold-steered-teacher-500}}
}

⚖️ License

This model inherits the license from the base model: Qwen/Qwen3-8B

🙏 Acknowledgments

Built with the TRL library's experimental GOLD framework.