Spaces:
Build error
Build error
Commit ·
ff29454
1
Parent(s): 0fb4bed
README: add back proof points — needle-in-haystack demo, benchmarks, architecture detail
Browse files
README.md
CHANGED
|
@@ -89,6 +89,37 @@ You don't need to replace it. Drop Headroom into your existing stack:
|
|
| 89 |
|
| 90 |
---
|
| 91 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 92 |
## How It Works
|
| 93 |
|
| 94 |
```
|
|
@@ -99,22 +130,21 @@ Your App → Headroom → LLM Provider
|
|
| 99 |
→ SmartCrusher (JSON) | CodeCompressor (code) | LLMLingua (text)
|
| 100 |
IntelligentContext: score-based token fitting
|
| 101 |
CCR: stores originals for retrieval if LLM needs more
|
|
|
|
| 102 |
```
|
| 103 |
|
| 104 |
-
Headroom never throws data away. It compresses aggressively and retrieves precisely.
|
| 105 |
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
## Verified Performance
|
| 109 |
|
| 110 |
-
| Scenario |
|
| 111 |
-
|----------|--------
|
| 112 |
| Code search (100 results) | 17,765 | 1,408 | **92%** |
|
| 113 |
| SRE incident debugging | 65,694 | 5,118 | **92%** |
|
| 114 |
| Codebase exploration | 78,502 | 41,254 | **47%** |
|
| 115 |
| GitHub issue triage | 54,174 | 14,761 | **73%** |
|
| 116 |
|
| 117 |
-
**Overhead**: 1-5ms
|
| 118 |
|
| 119 |
---
|
| 120 |
|
|
@@ -138,10 +168,12 @@ Headroom never throws data away. It compresses aggressively and retrieves precis
|
|
| 138 |
| Feature | What it does |
|
| 139 |
|---------|-------------|
|
| 140 |
| **Content Router** | Auto-detects content type, routes to optimal compressor |
|
| 141 |
-
| **SmartCrusher** | Statistically compresses JSON arrays
|
| 142 |
-
| **CodeCompressor** | AST-aware
|
| 143 |
| **LLMLingua-2** | ML-based 20x text compression |
|
| 144 |
| **CCR** | Reversible compression — LLM retrieves originals when needed |
|
|
|
|
|
|
|
| 145 |
| **CacheAligner** | Stabilizes prefixes for provider KV cache hits |
|
| 146 |
| **IntelligentContext** | Score-based context management with learned importance |
|
| 147 |
| **Image Compression** | 40-90% token reduction via trained ML router |
|
|
|
|
| 89 |
|
| 90 |
---
|
| 91 |
|
| 92 |
+
## Does It Actually Work?
|
| 93 |
+
|
| 94 |
+
**100 production log entries. One critical error buried at position 67.**
|
| 95 |
+
|
| 96 |
+
| | Baseline | Headroom |
|
| 97 |
+
|--|----------|----------|
|
| 98 |
+
| Input tokens | 10,144 | 1,260 |
|
| 99 |
+
| Correct answers | **4/4** | **4/4** |
|
| 100 |
+
|
| 101 |
+
Both responses: *"payment-gateway, error PG-5523, fix: Increase max_connections to 500, 1,847 transactions affected."*
|
| 102 |
+
|
| 103 |
+
**87.6% fewer tokens. Same answer.** Run it: `python examples/needle_in_haystack_test.py`
|
| 104 |
+
|
| 105 |
+
<details>
|
| 106 |
+
<summary><b>What Headroom kept</b></summary>
|
| 107 |
+
|
| 108 |
+
From 100 log entries, SmartCrusher kept 6: first 3 (boundary), the FATAL error at position 67 (anomaly detection), and last 2 (recency). The error was automatically preserved — not by keyword matching, but by statistical analysis of field variance.
|
| 109 |
+
</details>
|
| 110 |
+
|
| 111 |
+
### Accuracy Benchmarks
|
| 112 |
+
|
| 113 |
+
| Benchmark | Metric | Result | Compression |
|
| 114 |
+
|-----------|--------|--------|------------|
|
| 115 |
+
| [Scrapinghub Extraction](https://huggingface.co/datasets/allenai/scrapinghub-article-extraction-benchmark) | Recall | **98.2%** | 94.9% |
|
| 116 |
+
| Multi-Tool Agent (4 tools) | Accuracy | **100%** | 76.3% |
|
| 117 |
+
| SmartCrusher (JSON) | Accuracy | **100%** | 87.6% |
|
| 118 |
+
|
| 119 |
+
Full methodology: [Benchmarks](docs/benchmarks.md) | Run yourself: `python -m headroom.evals quick`
|
| 120 |
+
|
| 121 |
+
---
|
| 122 |
+
|
| 123 |
## How It Works
|
| 124 |
|
| 125 |
```
|
|
|
|
| 130 |
→ SmartCrusher (JSON) | CodeCompressor (code) | LLMLingua (text)
|
| 131 |
IntelligentContext: score-based token fitting
|
| 132 |
CCR: stores originals for retrieval if LLM needs more
|
| 133 |
+
Query Echo: re-injects user question for fresh attention after compression
|
| 134 |
```
|
| 135 |
|
| 136 |
+
Headroom never throws data away. It compresses aggressively and retrieves precisely. When it compresses 500 items to 20, it tells the LLM *what was omitted* ("87 passed, 2 failed, 1 error") so the LLM knows when to ask for more.
|
| 137 |
|
| 138 |
+
### Verified on Real Workloads
|
|
|
|
|
|
|
| 139 |
|
| 140 |
+
| Scenario | Before | After | Savings |
|
| 141 |
+
|----------|--------|-------|---------|
|
| 142 |
| Code search (100 results) | 17,765 | 1,408 | **92%** |
|
| 143 |
| SRE incident debugging | 65,694 | 5,118 | **92%** |
|
| 144 |
| Codebase exploration | 78,502 | 41,254 | **47%** |
|
| 145 |
| GitHub issue triage | 54,174 | 14,761 | **73%** |
|
| 146 |
|
| 147 |
+
**Overhead**: 1-5ms compression latency.
|
| 148 |
|
| 149 |
---
|
| 150 |
|
|
|
|
| 168 |
| Feature | What it does |
|
| 169 |
|---------|-------------|
|
| 170 |
| **Content Router** | Auto-detects content type, routes to optimal compressor |
|
| 171 |
+
| **SmartCrusher** | Statistically compresses JSON arrays — preserves errors, anomalies, boundaries |
|
| 172 |
+
| **CodeCompressor** | AST-aware compression for Python, JS, Go, Rust, Java, C++ |
|
| 173 |
| **LLMLingua-2** | ML-based 20x text compression |
|
| 174 |
| **CCR** | Reversible compression — LLM retrieves originals when needed |
|
| 175 |
+
| **Compression Summaries** | Tells the LLM what was omitted ("3 errors, 12 failures") |
|
| 176 |
+
| **Query Echo** | Re-injects user question after compressed data for better attention |
|
| 177 |
| **CacheAligner** | Stabilizes prefixes for provider KV cache hits |
|
| 178 |
| **IntelligentContext** | Score-based context management with learned importance |
|
| 179 |
| **Image Compression** | 40-90% token reduction via trained ML router |
|