File size: 4,688 Bytes
eaae571 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 | # Mitigation Layer Guide
This guide describes the mitigation layer as it exists in the current
framework, including where it runs, what it does, and how teammates should use
it in experiments.
## Purpose
The mitigation layer adds the defensive part of the framework.
It is responsible for:
- checking risky attack inputs before they reach the model
- hardening or isolating suspicious content
- blocking clear prompt-injection or extraction attempts when appropriate
- checking outputs before final scoring
- recording mitigation actions in run artifacts for later analysis
## Where It Runs
The runtime sequence is:
1. Load and map cases
2. Build the model request
3. Apply request mitigation
4. Call the model unless the request is blocked
5. Apply response mitigation
6. Score the final response
7. Save mitigation metadata in the results
If a case contains ordered `turns`, mitigation is applied turn by turn and the
runner stores per-turn artifacts as well.
## Supported Strategies
### `none`
Baseline mode with no defense applied.
### `prompt_hardening`
Adds a stronger safety context so the model treats user input as untrusted and
prioritizes the trusted task.
### `instruction_isolation`
Wraps suspicious content as clearly untrusted data so the model is less likely
to treat it as high-priority instruction.
### `keyword_guardrail`
Blocks obvious prompt-injection or prompt-extraction patterns before the model
is called.
### `surface_aligned`
This is the main defended strategy for the project.
It routes behavior by attack surface and category:
- benign controls are left alone to reduce unnecessary blocking
- direct attacks get prompt hardening and direct guardrails
- tool-mediated / indirect content is isolated as untrusted
- adaptive cases use the same guarded path turn by turn
- prompt-leakage style outputs can be filtered after inference
## Request-Side vs Response-Side Mitigation
### Request-side mitigation
This runs before inference.
Typical actions:
- passthrough
- transformed request
- blocked request
Saved fields:
- `mitigation_strategy`
- `mitigation_action`
- `mitigation_blocked`
- `mitigation_sanitized`
- `mitigation_notes`
- `mitigation_metadata`
### Response-side mitigation
This runs after inference but before evaluation.
Typical use:
- prompt leakage filtering
- output redaction or blocking for suspicious unsafe content
Saved fields:
- `response_mitigation_action`
- `response_mitigation_modified`
- `response_mitigation_blocked`
- `response_mitigation_notes`
- `response_mitigation_metadata`
## Multi-Turn Support
The runner supports an optional `turns` field in attack cases.
Current behavior:
- turns run sequentially
- prior conversation is preserved as history
- mitigation is applied on each turn
- final scoring uses the final effective response
- turn artifacts are saved for later review
This makes adaptive cases possible without introducing a full persistent agent
runtime.
## Configurable Options
The `mitigation.options` block can tune behavior without changing code.
### `strict_mode`
When enabled, the layer blocks more suspicious patterns instead of only
transforming the request.
### `sanitize_untrusted_content`
Redacts obvious payload fragments from untrusted content before inference.
### `block_suspicious_tool_actions`
Blocks tool-style instructions that look like unsafe external actions.
### `guard_tool_outputs`
Guards risky output content for tool-mediated scenarios.
### `allow_urls_in_output`
Allows future benign scenarios to return links without being over-filtered.
## Current Strengths
The current mitigation layer is in strong shape for the capstone framework:
- fully integrated into the runner
- documented and configurable
- works on both single-turn and lightweight multi-turn cases
- supports before-vs-after experiment design
- records enough metadata for team analysis
- showed real improvement on the HackAPrompt override slice during Groq runs
## Current Limits
The mitigation layer is not a complete security solution.
Remaining gaps include:
- stronger handling for some TensorTrust hijacking patterns
- richer semantic detection for heavily obfuscated attacks
- broader benign-control coverage for false-positive analysis
- persistent session memory for longer adaptive attacks
- real tool permission checks and approval gates
## Recommended Team Usage
- Use `strategy: none` for baseline runs
- Use `strategy: surface_aligned` for defended runs
- Compare results per category, surface, and source
- Treat the mitigation layer as a validated capstone defense component, while
staying honest that some slices still benefit from refinement
|