chopratejas commited on
Commit
cd5ea2e
Β·
1 Parent(s): ec56092

Add architecture diagrams to README and docs

Browse files
Files changed (2) hide show
  1. README.md +53 -6
  2. docs/ARCHITECTURE.md +46 -0
README.md CHANGED
@@ -122,14 +122,61 @@ Run it yourself: `python examples/multi_tool_agent_test.py`
122
 
123
  ## How It Works
124
 
125
- Headroom doesn't summarize or truncate blindly. It uses **statistical analysis**:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
126
 
127
- 1. **Detects redundancy** - Repeated fields like `"language": "typescript"` across 100 items
128
- 2. **Keeps what matters** - First items, last items, query-relevant matches, anomalies
129
- 3. **Preserves errors** - Never drops items containing "error", "exception", "failed"
130
- 4. **Maintains schema** - Output JSON structure stays identical
131
 
132
- The compression is **reversible** via CCR (Compress-Cache-Retrieve). If the LLM needs more data, it can request the original.
133
 
134
  ---
135
 
 
122
 
123
  ## How It Works
124
 
125
+ > Headroom optimizes LLM context *before* it hits the provider β€”
126
+ > without changing your agent logic or tools.
127
+
128
+ ```mermaid
129
+ flowchart LR
130
+ User["Your App"]
131
+ Entry["Headroom"]
132
+ Transform["Context<br/>Optimization"]
133
+ LLM["LLM Provider"]
134
+ Response["Response"]
135
+
136
+ User --> Entry --> Transform --> LLM --> Response
137
+ ```
138
+
139
+ ### Inside Headroom
140
+
141
+ ```mermaid
142
+ flowchart TB
143
+
144
+ subgraph Pipeline["Transform Pipeline"]
145
+ CA["Cache Aligner<br/><i>Stabilizes dynamic tokens</i>"]
146
+ SC["Smart Crusher<br/><i>Removes redundant tool output</i>"]
147
+ CM["Context Manager<br/><i>Fits token budget</i>"]
148
+ CA --> SC --> CM
149
+ end
150
+
151
+ subgraph CCR["CCR: Compress-Cache-Retrieve"]
152
+ Store[("Compressed<br/>Store")]
153
+ Tool["Retrieve Tool"]
154
+ Tool <--> Store
155
+ end
156
+
157
+ LLM["LLM Provider"]
158
+
159
+ CM --> LLM
160
+ SC -. "Stores originals" .-> Store
161
+ LLM -. "Requests full context<br/>if needed" .-> Tool
162
+ ```
163
+
164
+ > Headroom never throws data away.
165
+ > It compresses aggressively and retrieves precisely.
166
+
167
+ ### What actually happens
168
+
169
+ 1. **Headroom intercepts context** β€” Tool outputs, logs, search results, and intermediate agent steps.
170
+
171
+ 2. **Dynamic content is stabilized** β€” Timestamps, UUIDs, request IDs are normalized so prompts cache cleanly.
172
+
173
+ 3. **Low-signal content is removed** β€” Repetitive or redundant data is crushed, not truncated.
174
+
175
+ 4. **Original data is preserved** β€” Full content is stored separately and retrieved *only if the LLM asks*.
176
 
177
+ 5. **Provider caches finally work** β€” Headroom aligns prompts so OpenAI, Anthropic, and Google caches actually hit.
 
 
 
178
 
179
+ For deep technical details, see [Architecture Documentation](docs/ARCHITECTURE.md).
180
 
181
  ---
182
 
docs/ARCHITECTURE.md CHANGED
@@ -1,5 +1,51 @@
1
  # Headroom SDK: A Complete Explanation
2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ## What Problem Does Headroom Solve?
4
 
5
  When you use AI models like GPT-4 or Claude, you pay for **tokens** - the pieces of text you send (input) and receive (output). The problem is:
 
1
  # Headroom SDK: A Complete Explanation
2
 
3
+ ## Architecture Overview
4
+
5
+ ```mermaid
6
+ flowchart TB
7
+ subgraph Entry["Entry Points"]
8
+ Proxy["Proxy Mode<br/><i>Zero code changes</i>"]
9
+ SDK["SDK Mode<br/><i>HeadroomClient</i>"]
10
+ Integrations["Integrations<br/><i>LangChain / Agno</i>"]
11
+ end
12
+
13
+ subgraph Pipeline["Transform Pipeline"]
14
+ direction TB
15
+
16
+ CA["Cache Aligner<br/>━━━━━━━━━━━━━━━<br/>Extracts dynamic content<br/>(dates, UUIDs, tokens)<br/>Stable prefix for caching"]
17
+
18
+ SC["Smart Crusher<br/>━━━━━━━━━━━━━━━<br/>Analyzes tool outputs<br/>Keeps: first, last, errors, outliers<br/>70-95% reduction"]
19
+
20
+ CM["Context Manager<br/>━━━━━━━━━━━━━━━<br/>Enforces token limits<br/>Scores by recency and relevance<br/>Fits context window"]
21
+
22
+ CA --> SC --> CM
23
+ end
24
+
25
+ subgraph Cache["Provider Cache Optimization"]
26
+ direction LR
27
+ Anthropic["Anthropic<br/><i>cache_control blocks</i><br/>90% savings"]
28
+ OpenAI["OpenAI<br/><i>Prefix alignment</i><br/>50% savings"]
29
+ Google["Google<br/><i>CachedContent API</i><br/>75% savings"]
30
+ end
31
+
32
+ subgraph CCR["CCR: Compress-Cache-Retrieve"]
33
+ Store[("Compression<br/>Store")]
34
+ Tool["Retrieve Tool<br/><i>LLM requests original</i>"]
35
+ Store <--> Tool
36
+ end
37
+
38
+ LLM["LLM API<br/><i>OpenAI / Anthropic / Google</i>"]
39
+
40
+ Entry --> Pipeline
41
+ Pipeline --> Cache
42
+ Cache --> LLM
43
+ SC -.->|"Stores original"| Store
44
+ LLM -.->|"If needed"| Tool
45
+ ```
46
+
47
+ ---
48
+
49
  ## What Problem Does Headroom Solve?
50
 
51
  When you use AI models like GPT-4 or Claude, you pay for **tokens** - the pieces of text you send (input) and receive (output). The problem is: