Spaces:
Build error
Build error
Commit History
-
Reduce proxy latency, enrich telemetry, protect Bash output, bump to 0.5.6
Performance: - Replace json roundtrip with copy.deepcopy in deep_copy_messages (~20-80ms) - Eliminate redundant token counting in pipeline (8→2 calls, ~30-100ms) - Parallel message compression in ContentRouter via ThreadPoolExecutor (~100-200ms) - Add granular timing metrics: deep_copy, token_count, parallel_compress - Switch hot-path hashing from SHA256 to MD5 (2-3x faster, non-crypto) Telemetry: - Enrich beacon payload with overhead, TTFB, pipeline timing, request patterns, compression cache stats, CCR usage, and waste signals - Each extraction section guarded independently so one bad key never blocks the rest; Supabase POST failure never affects proxy Bug fix: - Add Bash/bash to DEFAULT_EXCLUDE_TOOLS so tree/ls output is not compressed by the text compressor78d41df
-
fix: prevent stat inflation in compute_frozen_count, fix token estimate
- compute_frozen_count now checks _cache directly instead of calling get_compressed, which was inflating hit/miss stats on every turn - update_from_result uses len//4 for rough token estimate instead of raw character count Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>9b5e783 -
feat: export CompressionCache from cache package
eac7379 -
feat: add frozen count, apply_cached, update_from_result to CompressionCache
Add three methods and supporting helpers for token headroom mode: - compute_frozen_count: counts consecutive stable messages from start - apply_cached: swaps cached compressions into tool results (immutable) - update_from_result: learns new compressions from original/compressed pairs Supports both Anthropic and OpenAI tool result formats. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>60741b5 -
feat: add CompressionCache with LRU eviction for token headroom mode
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>29928b8
-
Introducing headroom wrap
- headroom wrap claude is the simplest way to start up claude - It will also install rtk-ai locally - rtk-ai is a cli wrapper that can save ~90% tokens for CLI calls made by Claude Code641cfec
-
fix: harden edge cases, expand tool exclusions, and fix mypy errors
- Fix division-by-zero in OpenAI cache, BM25 scorer, BLEU metrics, and smart_crusher - Add null safety for SQLite memory store JSON fields - Add thread safety lock for CCR retrieval counter - Add timeout/error handling for CCR stream collection - Safer error logging and JSON serialization fallback in proxy server - Return explicit zero stats on compression failure instead of None defaults - Guard against missing "messages" key in ASGI and LiteLLM integrations - Add aclose() for proper httpx client cleanup in ASGI middleware - Expand DEFAULT_EXCLUDE_TOOLS to include Grep, Write, Edit - Revert protect_recent_reads_fraction to 0.0 (protect all excluded-tool outputs) - Fix whitespace waste detection to use tokenizer for both original and normalized - Fix top-waste-requests sorting by tokens_saved instead of tokens_before - Fix TOCTOU race in semantic cache _touch() method - Fix mypy errors: type annotations for ASGI receive/send and proxy error loggingc8388c1
-
Add pluggable adapter hooks for CCR, Storage, and TOIN backends
Enable SaaS packages to inject custom backends (Redis, PostgreSQL, etc.) without forking OSS, using entry_points and ContextVars for tenant isolation. CCR: Request-scoped ContextVar + HEADROOM_CCR_BACKEND env + entry_point loading Storage: URL-scheme entry_point resolution for custom storage backends TOIN: New TOINBackend protocol + FileSystemTOINBackend (extracted from toin.py) + HEADROOM_TOIN_BACKEND env + entry_point loading 31 tests covering protocol conformance, backend wiring, entry_point loading, ContextVar thread isolation, and end-to-end adapter lifecycle.107515b
-
Add centralized ML model configuration
- Create headroom/models/config.py as single source of truth for all ML model defaults - Support environment variable overrides (HEADROOM_SENTENCE_TRANSFORMER, etc.) - Update all components to use ML_MODEL_DEFAULTS instead of hardcoded values - Switch LLMLingua default to smaller bert-base model (~350MB vs 1GB) - Total memory footprint reduced from ~1.6GB to ~980MB Updated files: - MLModelRegistry now resolves defaults from config - EmbeddingScorer, LocalEmbedder, TrainedRouter use config - All dataclass configs use field(default_factory=...) for consistency - Tests updated to handle auto-selected vector backendscdde22e -
Add centralized MLModelRegistry to share ML model instances
Previously, SentenceTransformer was loaded up to 5 times in different components, wasting ~1.5GB of memory. Now all ML models are shared via MLModelRegistry: - SentenceTransformer (text embeddings) - SIGLIP (image embeddings) - spaCy (NER) - Technique router (image optimization) Updated components to use the registry: - headroom/relevance/embedding.py - headroom/memory/adapters/embedders.py - headroom/cache/dynamic_detector.py - headroom/prediction/feature_extractor.py - headroom/evals/metrics.py - headroom/image/trained_router.pybb32b57 -
Add memory observability system (Phase 1)
Implements comprehensive memory tracking for all in-memory components: - Add MemoryTracker singleton with ComponentStats, ProcessStats, MemoryReport - Add get_memory_stats() to CompressionStore, BatchContextStore, GraphStore, HNSWVectorIndex - Add /debug/memory API endpoint for runtime monitoring Components tracked: - compression_store: CCR compressed tool outputs - batch_context_store: Batch API request contexts - graph_store: Knowledge graph entities and relationships - vector_index: HNSW vector embeddings - semantic_cache: Response cache - request_logger: Request metadata Includes 47 tests (unit + integration) with real API calls.60a6ac8
-
Add pluggable storage backend abstraction for CompressionStore
- Add CompressionStoreBackend protocol for duck-typed backends - Add InMemoryBackend as default thread-safe implementation - Refactor CompressionStore to accept optional backend parameter - Add comprehensive backend contract tests (28 tests)fd01dd5
-
Fix ruff formatting
313fe01
-
Add TOIN field-level learning and comprehensive integration tests
Features: - Add field-level learning to TOIN from retrieved items - CompressionStore now passes retrieved_items to TOIN for learning - Add FieldSemantics class for tracking field usage patterns Test improvements: - Add TestCacheOptimizerInvocation to verify optimizer is actually invoked - Add TestSemanticCacheIntegration to verify cache hit returns without API call - Add TestSessionStatsTracking to verify session stats are tracked - Add TestEndToEndTOINIntegration for full CCR cycle with TOIN - Add critical field_semantics assertions to catch feedback loop bugs Fixes: - Remove unused imports and variables (ruff linting) Bump version to 0.2.11dd832fe
-
Add seamless LangChain integration
- Add HeadroomChatModel wrapper with auto provider detection (OpenAI, Anthropic, Google) - Add HeadroomChatMessageHistory for automatic conversation compression - Add HeadroomDocumentCompressor for retriever integration - Add wrap_tools_with_headroom() for agent tool output compression - Add async support (ainvoke, astream) - Add LangSmith integration for observability - Restructure integrations package into nested langchain/ and mcp/ subpackages - Fix Pydantic v2 deprecation warning - Add comprehensive docs/langchain.md guide with real-world examples - Update README with LangChain quickstart and framework integrations Bump version to 0.2.3bb04104 -
v0.2.2: Add CCR Response Handler, Context Tracker, and restructure docs
Features: - CCR Response Handler: Automatically intercepts and handles headroom_retrieve tool calls - CCR Context Tracker: Multi-turn awareness with proactive expansion of relevant compressed content - New CCR demo script showing before/after flow Documentation: - Restructured README from 885 lines to 190 lines for better DevEx - Split detailed docs into focused guides: ccr.md, sdk.md, configuration.md, text-compression.md, llmlingua.md, metrics.md, errors.md - Updated docs/README.md index with all new documentation Tests: - Added comprehensive tests for Response Handler (32 tests) - Added comprehensive tests for Context Tracker (32 tests) - All 977 tests passingd724f14
-
Fix all mypy type errors and flaky embedding test
- Fix 34 mypy errors across 17 files with type annotations and casts - Add type: ignore comments for legitimate dynamic patterns - Handle None operands with (value or 0) pattern - Cast return values to proper types (int, float, str, bool) - Add EstimatingTokenCounter imports where needed - Use getattr() for potentially missing attributes - Fix flaky test_paraphrase_match with more distinct semantic examples - Add mlx to mypy ignore list (broken third-party stubs)bf779b5 -
Fix all ruff lint and format errors for CI
- Fix E402: Move module-level imports to top of file - Fix F401: Add noqa for availability check imports - Fix F402: Rename loop variables shadowing imports - Fix E722: Replace bare except with except Exception - Fix B904: Add exception chaining (from e) - Fix F811: Remove duplicate imports - Fix B027: Add noqa for empty close() method - Fix E741: Rename ambiguous variable l -> label - Fix I001: Import sorting issues - Apply ruff format to all 106 files All 902 tests pass.e4a41fa -
feat: Add CCR architecture, TOIN telemetry, and DevEx improvements
## Core Features ### Compress-Cache-Retrieve (CCR) Architecture - Implement reversible compression with automatic retrieval support - Add CompressionStore for caching original content with TTL-based eviction - Add CompressionFeedback for learning from retrieval patterns - Implement tool injection for LLM retrieval capability - Add MCP server support for CCR operations - Track retrieval rates to dynamically adjust compression aggressiveness ### Tool Output Intelligence Network (TOIN) - Implement cross-session pattern learning for tool compression - Add ToolSignature for structural hashing of tool outputs - Track compression success rates per strategy (top_n, sample, truncate, etc.) - Implement privacy-preserving telemetry with SHA256 hashing - Add persistent storage with JSON file backend - Support network-effect learning across tool types ### SmartCrusher Enhancements - Add crushability analysis with variance/uniqueness detection - Implement statistical anomaly detection for outlier…c1feb60
-
Add cache optimization module with scalable dynamic content detection
Implements a comprehensive cache optimization layer for LLM providers: - Provider-specific optimizers (Anthropic, OpenAI, Google) with distinct caching strategies: explicit breakpoints, prefix stabilization, and CachedContent API respectively - Scalable dynamic content detector using three strategies: 1. Structural detection: "Label: value" patterns (language-agnostic) 2. Entropy-based detection: high-entropy strings (IDs, tokens, hashes) 3. Universal patterns: ISO 8601, UUIDs, JWTs, hex hashes - NO hardcoded locale-specific patterns (no month names, etc.) - Semantic caching layer with LRU eviction and TTL support - Plugin registry for provider selection and custom optimizers - 131 tests, real-world benchmarks showing 20-55% compression at <0.3ms7a05808