1
🧠
Router Agent
Classifies intent into 5 types (Medical FAQ, Emergency, Web Search, Greeting, Out-of-Scope).
Detects medical emergencies for immediate safety response. Reformulates query for optimal retrieval.
LangChain + GPT-4o-mini + Intent Classification + Emergency Detection
↓
2
📚
Retriever Agent
Hybrid retrieval combining BM25 keyword search (exact medical terms) + FAISS semantic search (OpenAI embeddings)
→ Reciprocal Rank Fusion (α=0.5) → Cross-encoder reranking. Achieves ~85% precision@5 vs ~60% with vector-only.
BM25 + FAISS + RRF + Cross-Encoder (ms-marco-MiniLM) + Tavily Web Search
↓
3
💬
Responder Agent
Generates grounded response strictly from retrieved context. Integrates conversation history (last 4 messages).
Adds medical disclaimer and source citations. Streams tokens in real-time via Server-Sent Events.
GPT-4o-mini + Context-Aware Prompting + SSE Streaming + Citation
↓
4
✅
Evaluator Agent
Scores response quality (0-1), assesses hallucination risk (low/medium/high), checks groundedness.
If quality score < 0.7, triggers automatic self-correction retry with corrective prompt (max 1 retry).
LLM-based Evaluation + Hallucination Detection + Self-Correction Loop
🔒 Personal Medical Records
Session-scoped in-memory FAISS for user documents. PDF upload → structured extraction → grounded Q&A. Zero persistence, privacy-first.
⚡ Response Caching
In-memory cache with 30-min TTL. 40% cost reduction for duplicate queries. SHA256 query hashing for fast lookups.
🛡️ Rate Limiting
Token bucket algorithm: 20 req/min, 100 req/hour per client. Prevents abuse and controls costs.
🎯 Hallucination Detection
LLM-based scoring (AWS blog approach). 0-1 risk score per response. Automatic flagging of high-risk content.
📊 ML Risk Assessment
9 clinical factors → logistic regression → GPT-4o explanation. Predicts patient risk with interpretable scoring.
📈 Production Monitoring
/stats endpoint for cache/rate limiter metrics. Prometheus metrics for requests, latency, quality scores.