Spaces:
Build error
Build error
Commit History
-
Add OSS evaluation suite, universal JSON crush, latency benchmarks
Evaluation Suite: - Tiered eval framework (Tier 1 ~$3/15min, Tier 2/3 for extended coverage) - 16 benchmarks across 3 tiers: GSM8K, TruthfulQA, MMLU, ARC, HumanEval, SQuAD v2, BFCL, Tool Outputs, CCR needle retention, HotpotQA, and more - Before/After runner with full proxy support (compression + CCR retrieval) - LLM-as-judge for ground-truth comparison (BFCL function calling) - Zero-cost compression-only runner (CCR needle retention, info retention) - Cost tracker with per-model pricing and budget enforcement - Report card generator (Markdown, JSON, HTML) - Suite CLI: python -m headroom.evals suite --tier 1 - Fix BFCL dataset loader for current HuggingFace schema - CI workflow: PR smoke test + weekly full Tier 1 Results: SQuAD 97%, BFCL 97%, Tool Outputs 100%, CCR 100% SmartCrusher: - Universal JSON crush for heterogeneous arrays - Fix mypy redefinition warning in _crush_string_array Other: - Latency benchmark suite with docs - Known limitations doc - Prompt comparison evaluator …ad6f44c
-
Add MkDocs GitHub Pages documentation site
- Add mkdocs.yml with Material theme (indigo, professional) - Add docs/index.md landing page with quick install - Add GitHub Actions workflow for auto-deployment - Remove old docs/README.md (replaced by index.md)b458389
-
Add Agno integration tests to CI workflow
1f806ce
-
Use oldest supported Python version (3.10) for package build
pythoninthegrass +1: Claude Sonnet 4.5Build artifacts with Python 3.10 to ensure maximum compatibility and avoid accidentally using newer features in build tooling. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>ec78864 -
Add Python 3.13 to CI matrix and optimize linting steps
pythoninthegrass +1: Claude Sonnet 4.5Adds Python 3.13 to the test matrix. Optimizes CI by running linting and type checking only on Python 3.12 instead of all versions. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>783fb08
-
Prepare for OSS release v0.2.0
This commit prepares Headroom for public open source release with comprehensive documentation, licensing, and community infrastructure. License & Legal: - Add Apache 2.0 LICENSE file - Add NOTICE file with third-party attributions - Add SECURITY.md for vulnerability reporting Community: - Add CONTRIBUTING.md with contribution guidelines - Add CODE_OF_CONDUCT.md (Contributor Covenant) - Add GitHub issue templates (bug report, feature request) - Add pull request template Documentation: - Update README.md with compelling value proposition - Add docs/getting-started.md - Add docs/proxy.md for proxy server documentation - Add docs/transforms.md for transform reference - Add docs/api.md for API reference - Add examples/README.md Package Infrastructure: - Add headroom/py.typed for PEP 561 compliance - Add headroom/cli.py for CLI entry point - Add .github/workflows/ci.yml for CI pipeline - Add .github/workflows/publish.yml for PyPI publishing - Update pyproject.toml with proper metadata N…175746c