Spaces:
Build error
Build error
Commit Β·
cd5ea2e
1
Parent(s): ec56092
Add architecture diagrams to README and docs
Browse files- README.md +53 -6
- docs/ARCHITECTURE.md +46 -0
README.md
CHANGED
|
@@ -122,14 +122,61 @@ Run it yourself: `python examples/multi_tool_agent_test.py`
|
|
| 122 |
|
| 123 |
## How It Works
|
| 124 |
|
| 125 |
-
Headroom
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 126 |
|
| 127 |
-
|
| 128 |
-
2. **Keeps what matters** - First items, last items, query-relevant matches, anomalies
|
| 129 |
-
3. **Preserves errors** - Never drops items containing "error", "exception", "failed"
|
| 130 |
-
4. **Maintains schema** - Output JSON structure stays identical
|
| 131 |
|
| 132 |
-
|
| 133 |
|
| 134 |
---
|
| 135 |
|
|
|
|
| 122 |
|
| 123 |
## How It Works
|
| 124 |
|
| 125 |
+
> Headroom optimizes LLM context *before* it hits the provider β
|
| 126 |
+
> without changing your agent logic or tools.
|
| 127 |
+
|
| 128 |
+
```mermaid
|
| 129 |
+
flowchart LR
|
| 130 |
+
User["Your App"]
|
| 131 |
+
Entry["Headroom"]
|
| 132 |
+
Transform["Context<br/>Optimization"]
|
| 133 |
+
LLM["LLM Provider"]
|
| 134 |
+
Response["Response"]
|
| 135 |
+
|
| 136 |
+
User --> Entry --> Transform --> LLM --> Response
|
| 137 |
+
```
|
| 138 |
+
|
| 139 |
+
### Inside Headroom
|
| 140 |
+
|
| 141 |
+
```mermaid
|
| 142 |
+
flowchart TB
|
| 143 |
+
|
| 144 |
+
subgraph Pipeline["Transform Pipeline"]
|
| 145 |
+
CA["Cache Aligner<br/><i>Stabilizes dynamic tokens</i>"]
|
| 146 |
+
SC["Smart Crusher<br/><i>Removes redundant tool output</i>"]
|
| 147 |
+
CM["Context Manager<br/><i>Fits token budget</i>"]
|
| 148 |
+
CA --> SC --> CM
|
| 149 |
+
end
|
| 150 |
+
|
| 151 |
+
subgraph CCR["CCR: Compress-Cache-Retrieve"]
|
| 152 |
+
Store[("Compressed<br/>Store")]
|
| 153 |
+
Tool["Retrieve Tool"]
|
| 154 |
+
Tool <--> Store
|
| 155 |
+
end
|
| 156 |
+
|
| 157 |
+
LLM["LLM Provider"]
|
| 158 |
+
|
| 159 |
+
CM --> LLM
|
| 160 |
+
SC -. "Stores originals" .-> Store
|
| 161 |
+
LLM -. "Requests full context<br/>if needed" .-> Tool
|
| 162 |
+
```
|
| 163 |
+
|
| 164 |
+
> Headroom never throws data away.
|
| 165 |
+
> It compresses aggressively and retrieves precisely.
|
| 166 |
+
|
| 167 |
+
### What actually happens
|
| 168 |
+
|
| 169 |
+
1. **Headroom intercepts context** β Tool outputs, logs, search results, and intermediate agent steps.
|
| 170 |
+
|
| 171 |
+
2. **Dynamic content is stabilized** β Timestamps, UUIDs, request IDs are normalized so prompts cache cleanly.
|
| 172 |
+
|
| 173 |
+
3. **Low-signal content is removed** β Repetitive or redundant data is crushed, not truncated.
|
| 174 |
+
|
| 175 |
+
4. **Original data is preserved** β Full content is stored separately and retrieved *only if the LLM asks*.
|
| 176 |
|
| 177 |
+
5. **Provider caches finally work** β Headroom aligns prompts so OpenAI, Anthropic, and Google caches actually hit.
|
|
|
|
|
|
|
|
|
|
| 178 |
|
| 179 |
+
For deep technical details, see [Architecture Documentation](docs/ARCHITECTURE.md).
|
| 180 |
|
| 181 |
---
|
| 182 |
|
docs/ARCHITECTURE.md
CHANGED
|
@@ -1,5 +1,51 @@
|
|
| 1 |
# Headroom SDK: A Complete Explanation
|
| 2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
## What Problem Does Headroom Solve?
|
| 4 |
|
| 5 |
When you use AI models like GPT-4 or Claude, you pay for **tokens** - the pieces of text you send (input) and receive (output). The problem is:
|
|
|
|
| 1 |
# Headroom SDK: A Complete Explanation
|
| 2 |
|
| 3 |
+
## Architecture Overview
|
| 4 |
+
|
| 5 |
+
```mermaid
|
| 6 |
+
flowchart TB
|
| 7 |
+
subgraph Entry["Entry Points"]
|
| 8 |
+
Proxy["Proxy Mode<br/><i>Zero code changes</i>"]
|
| 9 |
+
SDK["SDK Mode<br/><i>HeadroomClient</i>"]
|
| 10 |
+
Integrations["Integrations<br/><i>LangChain / Agno</i>"]
|
| 11 |
+
end
|
| 12 |
+
|
| 13 |
+
subgraph Pipeline["Transform Pipeline"]
|
| 14 |
+
direction TB
|
| 15 |
+
|
| 16 |
+
CA["Cache Aligner<br/>βββββββββββββββ<br/>Extracts dynamic content<br/>(dates, UUIDs, tokens)<br/>Stable prefix for caching"]
|
| 17 |
+
|
| 18 |
+
SC["Smart Crusher<br/>βββββββββββββββ<br/>Analyzes tool outputs<br/>Keeps: first, last, errors, outliers<br/>70-95% reduction"]
|
| 19 |
+
|
| 20 |
+
CM["Context Manager<br/>βββββββββββββββ<br/>Enforces token limits<br/>Scores by recency and relevance<br/>Fits context window"]
|
| 21 |
+
|
| 22 |
+
CA --> SC --> CM
|
| 23 |
+
end
|
| 24 |
+
|
| 25 |
+
subgraph Cache["Provider Cache Optimization"]
|
| 26 |
+
direction LR
|
| 27 |
+
Anthropic["Anthropic<br/><i>cache_control blocks</i><br/>90% savings"]
|
| 28 |
+
OpenAI["OpenAI<br/><i>Prefix alignment</i><br/>50% savings"]
|
| 29 |
+
Google["Google<br/><i>CachedContent API</i><br/>75% savings"]
|
| 30 |
+
end
|
| 31 |
+
|
| 32 |
+
subgraph CCR["CCR: Compress-Cache-Retrieve"]
|
| 33 |
+
Store[("Compression<br/>Store")]
|
| 34 |
+
Tool["Retrieve Tool<br/><i>LLM requests original</i>"]
|
| 35 |
+
Store <--> Tool
|
| 36 |
+
end
|
| 37 |
+
|
| 38 |
+
LLM["LLM API<br/><i>OpenAI / Anthropic / Google</i>"]
|
| 39 |
+
|
| 40 |
+
Entry --> Pipeline
|
| 41 |
+
Pipeline --> Cache
|
| 42 |
+
Cache --> LLM
|
| 43 |
+
SC -.->|"Stores original"| Store
|
| 44 |
+
LLM -.->|"If needed"| Tool
|
| 45 |
+
```
|
| 46 |
+
|
| 47 |
+
---
|
| 48 |
+
|
| 49 |
## What Problem Does Headroom Solve?
|
| 50 |
|
| 51 |
When you use AI models like GPT-4 or Claude, you pay for **tokens** - the pieces of text you send (input) and receive (output). The problem is:
|