Publish autonomy evaluation audit and refreshed evaluation report

01-rag-system/README.md

@@ -136,6 +136,7 @@ Supporting scripts:
 - `summarize_eval_sets.py`
 
 Latest committed result snapshots live in [`evaluation/results`](./evaluation/results).
+Autonomy audit for current release is tracked in [`../AUTONOMY_EVALUATION.md`](../AUTONOMY_EVALUATION.md).
 
 ### Latest committed summaries
 

01-rag-system/evaluation/reports/latest_portfolio_report.md

@@ -1,6 +1,6 @@
 # Portfolio Evaluation Report
 
-Generated: 2026-05-08 00:54:14 UTC
+Generated: 2026-05-11 21:04:44 UTC
 
 ## Domain Pack Snapshot
 

AUTONOMY_EVALUATION.md

@@ -0,0 +1,56 @@
+# Autonomy Evaluation (Current Release)
+
+Last updated: 2026-05-11 (America/Chicago)
+
+This document evaluates whether the app behaves as an autonomous agent, based on the current code and committed test/evaluation artifacts.
+
+## Verdict
+
+- **Autonomous agent mode exists and is functional:** Yes
+- **Fully autonomous production system (strict definition):** Partial
+
+## What is implemented now
+
+1. **Tool-calling autonomous runtime**
+- `01-rag-system/core/agentic_runtime.py` uses an OpenAI tools loop with iterative tool execution and observations.
+
+2. **Decision preflight**
+- Explicit preflight routing for scam/structuring/sanctions/math signals before generic clarification.
+
+3. **In-loop verification + self-correction**
+- Verification tool runs before finalization.
+- If verification fails (`needs_retry`), rewrite happens inside the same loop.
+
+4. **Autonomous supervisor mode**
+- `Autonomous Max` mode continues execution with explicit assumptions when clarifications would otherwise block progress.
+
+5. **Persistent autonomous operations**
+- Background task queue and policy audit log are persisted:
+  - `01-rag-system/data/autonomous_queue.json`
+  - `01-rag-system/data/autonomous_audit_log.jsonl`
+
+6. **Regression tests**
+- `01-rag-system/tests/test_agentic_decision_engine.py` validates core decision behavior.
+
+## Current evidence
+
+- Local decision-engine regression tests: **6/6 passing**
+- Portfolio report regenerated from committed evaluation summaries:
+  - `01-rag-system/evaluation/reports/latest_portfolio_report.md`
+
+## Remaining gap to “fully autonomous” (strict enterprise bar)
+
+To claim strict full autonomy in enterprise environments, these are still recommended:
+
+1. Scheduled autonomous execution service outside Streamlit session lifecycle.
+2. Explicit approval workflows for high-risk policy decisions (human gate with signed trace).
+3. Stronger SLA telemetry (p50/p95/p99 latency, tool failure rates, retry budgets).
+4. Expanded test matrix for all languages/input types in CI (voice, PDF, DOCX, image).
+
+## Honest label to use publicly
+
+Use:
+- **“Autonomous tool-calling banking AI agent with policy-aware execution and audit logging.”**
+
+Avoid overclaiming:
+- **“Fully autonomous AGI”**.

README.md

@@ -30,6 +30,7 @@ I did not want this to be a one-screen chatbot demo. I wanted it to behave like
 I maintain a concrete week-by-week delivery plan in:
 
 - [`WORLDCLASS_WEEK1_TO_WEEK6.md`](WORLDCLASS_WEEK1_TO_WEEK6.md)
+- [`AUTONOMY_EVALUATION.md`](AUTONOMY_EVALUATION.md)
 
 This is the operating plan used to move the project from strong prototype quality to production-grade architecture, reliability, governance, and distribution.