Spaces:
Sleeping
Sleeping
|
Download README.md from Abrar144/chat-with-data: direct link, hf CLI and curl.
- Browser
- Download file 23.8 kB
-
https://huggingface.co/spaces/Abrar144/chat-with-data/resolve/main/README.md
- Command line
-
hf download hf://spaces/Abrar144/chat-with-data/README.md
-
curl -L -o README.md https://huggingface.co/spaces/Abrar144/chat-with-data/resolve/main/README.md
23.8 kB
| title: Chat With Data | |
| emoji: π¨ | |
| colorFrom: blue | |
| colorTo: purple | |
| sdk: streamlit | |
| sdk_version: 1.39.0 | |
| app_file: frontend/dashboard.py | |
| python_version: "3.11" | |
| pinned: false | |
| # Chat with your data β Enterprise AI Analytics Platform | |
| > An enterprise-grade "Chat with Your Data" system that enables stakeholders to query hotel performance data in plain English, backed by a deterministic metrics engine, real-time KPI monitoring, and automated anomaly detection. | |
|  | |
|  | |
|  | |
|  | |
|  | |
| <p align="center"> | |
| <a href="#-live-demo">Live Demo</a> β’ | |
| <a href="#-features">Features</a> β’ | |
| <a href="#%EF%B8%8F-architecture">Architecture</a> β’ | |
| <a href="#-quick-start">Quick Start</a> β’ | |
| <a href="#-kpi-reference">KPI Reference</a> | |
| </p> | |
| --- | |
| ## π― Live Demo | |
| π **[Launch App on Streamlit Cloud](https://hospitality-acwk5lhj7j9dus5chmjbrs.streamlit.app/))** | |
| ### Dashboard Preview | |
| <!-- | |
| HOW TO ADD YOUR IMAGES: | |
| 1. Create a folder called "assets/images/" in the root of your repo | |
| 2. Take screenshots of your running app and save with the filenames below | |
| 3. Git add, commit, push β GitHub will render them automatically | |
| Suggested tool for ER diagrams: https://dbdiagram.io or pgAdmin | |
| --> | |
|  | |
| *Executive Dashboard β KPI cards, revenue trends, and city performance* | |
| <details> | |
| <summary>πΈ More Screenshots</summary> | |
| <br> | |
|  | |
| *AI Chat Interface β natural language queries with formatted business answers* | |
|  | |
| *KPI Monitoring β anomaly detection alerts and property health scores* | |
| </details> | |
| --- | |
| ## π Problem Statement | |
| In large hospitality organizations, performance data is spread across multiple tables with complex relationships. Business users need answers to questions like: | |
| - *"What is our RevPAR for luxury hotels in Mumbai this month?"* | |
| - *"Which booking platform has the highest cancellation rate?"* | |
| - *"Compare week 25 vs week 29 occupancy across cities"* | |
| Traditionally, each question requires an analyst to manually write SQL, build a report, and deliver it β a process that takes hours per query and doesn't scale. | |
| **This platform solves that by enabling anyone to ask business questions in plain English and receive accurate, data-driven answers in seconds.** | |
| --- | |
| ## β¨ Features | |
| ### π Executive Dashboard | |
| - **6 real-time KPI cards** with live Week-over-Week deltas | |
| - **Multi-dimensional filtering** β City, Category, Room Class, Month, Week | |
| - **Revenue by Category** β Luxury vs Business donut chart | |
| - **Weekly Revenue Trend** β Interactive line chart | |
| - **Weekend vs Weekday** β Performance comparison table | |
| - **Realisation % & ADR by Platform** β Dual-axis combo chart | |
| - **City Performance** β Horizontal bar comparison | |
| - **Weekly Occupancy Trend** β Area chart with time series | |
| - **Property Performance Table** β All 13 KPIs per hotel | |
| ### π¬ Chat with Your Data (AI Agent) | |
| - **Natural language queries** β Ask in plain English, get data-driven answers | |
| - **24 built-in KPI metrics** β Deterministic SQL generation (zero syntax errors) | |
| - **Complex query support** β Rankings, comparisons, multi-step analysis | |
| - **Custom SQL fallback** β LLM generates SQL for ad-hoc analytical questions | |
| - **Suggested starter questions** β One-click query templates | |
| - **Formatted business answers** β Currency, percentages, actionable insights | |
| ### π KPI Monitoring & Anomaly Detection | |
| - **4-layer alert system:** | |
| - Threshold-based alerts (configurable per KPI) | |
| - Week-over-Week drop detection | |
| - Consecutive decline detection (3+ week downtrends) | |
| - Statistical anomaly detection (z-score across properties) | |
| - **Property Health Scoring** β 0-100 score per hotel based on weighted KPI performance | |
| - **Interactive trend analysis** β Select any metric, view with threshold lines | |
| - **Configurable thresholds** β Adjust warning/critical levels per session | |
| --- | |
| ## ποΈ Architecture | |
|  | |
| *System architecture β export as PNG from draw.io, Excalidraw, or Lucidchart and save as `assets/images/architecture_diagram.png`* | |
| <details> | |
| <summary>View ASCII version</summary> | |
| ``` | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β FRONTEND (Streamlit) β | |
| β β | |
| β ββββββββββββββββ ββββββββββββββββββββ βββββββββββββββββββββββββ β | |
| β β Dashboard β β Chat with Data β β KPI Monitoring β β | |
| β β (KPI Cards, β β (Natural Lang β β (Anomaly Detection, β β | |
| β β Charts, β β Queries, AI β β Health Scores, β β | |
| β β Filters) β β Agent) β β Trend Analysis) β β | |
| β ββββββββ¬ββββββββ ββββββββββ¬ββββββββββ ββββββββββββ¬βββββββββββββ β | |
| β β β β β | |
| βββββββββββΌββββββββββββββββββββΌβββββββββββββββββββββββββΌβββββββββββββββ | |
| β β β | |
| βΌ βΌ βΌ | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β METRICS ENGINE (Single Source of Truth) β | |
| β utils/metrics_engine.py β | |
| β β | |
| β get_core_metrics() β get_wow_deltas() β get_property_table() β | |
| β get_trend_data() β get_city_comparison() β get_platform_perf() β | |
| βββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββββββ | |
| β | |
| βΌ | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β DETERMINISTIC SQL BUILDER β | |
| β tools/tools.py β | |
| β β | |
| β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β | |
| β β _METRIC_CONFIG (24 metrics) β β | |
| β β _build_metric_sql() β Perfect SQL every time β β | |
| β β _build_wow_sql() β Week-over-Week with window functions β β | |
| β β _assemble_cross_table_query() β CTE-based RevPAR β β | |
| β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β | |
| β β | |
| β execute_metric_query() β Deterministic (95% of queries) β | |
| β execute_custom_sql() β LLM-generated (complex ad-hoc) β | |
| β get_database_context() β Live schema introspection β | |
| βββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββββββ | |
| β | |
| βββββββββββββββββββββΌββββββββββββββββββββ | |
| βΌ β βΌ | |
| ββββββββββββββββββββ β ββββββββββββββββββββ | |
| β AI AGENT β β β SUPABASE β | |
| β agents.py β β β PostgreSQL β | |
| β β β β β | |
| β LiteLLM + β β β dim_date β | |
| β Native Function ββββββββββββ β dim_hotels β | |
| β Calling β β dim_rooms β | |
| β β β fact_bookings β | |
| β 3 Tools: βββββββββββββββββββββΆβ fact_aggregated β | |
| β calculate_metrics β _bookings β | |
| β run_custom_sql β β β | |
| β search_metric β β ETL Pipeline β | |
| ββββββββββββββββββββ β (CSVβRawβClean) β | |
| ββββββββββββββββββββ | |
| ``` | |
| </details> | |
| ### Why This Architecture? | |
| | Design Decision | Rationale | | |
| |---|---| | |
| | **Deterministic SQL Builder** | LLMs can write bad SQL. Building SQL programmatically from metric configs eliminates syntax errors for known KPIs. | | |
| | **Two-path query strategy** | 95% of questions use the reliable builder. Only truly novel questions fall back to LLM-generated SQL. | | |
| | **Never direct-join fact tables** | `fact_bookings` and `fact_aggregated_bookings` have different granularity. Direct joins cause row multiplication. CTEs handle cross-table metrics. | | |
| | **Single metrics engine** | Dashboard, agent, and monitoring all use the same SQL builder. Numbers always match. | | |
| | **Native function calling** | More reliable than text-based ReAct parsing. Structured JSON tool calls work with any LLM. | | |
| | **Separate ETL pipeline** | Raw β Clean transformation with validation. Business rules (weekend = Fri+Sat) applied at ETL layer. | | |
| --- | |
| ## π Project Structure | |
| ``` | |
| atliq-hospitality/ | |
| β | |
| βββ agents/ | |
| β βββ __init__.py | |
| β βββ agents.py # AI agent with native function calling | |
| β | |
| βββ assets/ | |
| β βββ images/ # β Drop your screenshots here | |
| β βββ dashboard_overview.png | |
| β βββ chat_with_data.png | |
| β βββ kpi_monitoring.png | |
| β βββ architecture_diagram.png | |
| β βββ db_schema_er_diagram.png | |
| β βββ etl_pipeline_output.png | |
| β | |
| βββ etl/ | |
| β βββ etl_pipeline.py # CSV β Raw DB β Clean DB pipeline | |
| β | |
| βββ frontend/ | |
| β βββ dashboard.py # Executive dashboard (main page) | |
| β βββ pages/ | |
| β βββ 02_Chat_with_Data.py # Natural language query interface | |
| β βββ 03_KPI_Monitoring.py # Anomaly detection & health scores | |
| β | |
| βββ prompts/ | |
| β βββ cot_prompts.py # Chain-of-thought prompt templates | |
| β | |
| βββ tools/ | |
| β βββ tools.py # Deterministic SQL builder + execution | |
| β | |
| βββ utils/ | |
| β βββ config.py # Configuration, schema map, metric library | |
| β βββ metrics_engine.py # Shared metrics API (single source of truth) | |
| β | |
| βββ .streamlit/ | |
| β βββ config.toml # Streamlit theme & server config | |
| β | |
| βββ requirements.txt | |
| βββ .gitignore | |
| βββ README.md | |
| ``` | |
| --- | |
| ## π Quick Start | |
| ### Prerequisites | |
| - Python 3.11+ | |
| - Supabase account (PostgreSQL database) | |
| - OpenRouter API key (for LLM access) | |
| ### 1. Clone & Install | |
| ```bash | |
| git clone https://github.com/YOUR_USERNAME/atliq-hospitality.git | |
| cd atliq-hospitality | |
| pip install -r requirements.txt | |
| ``` | |
| ### 2. Configure Environment | |
| Create `.env` in project root: | |
| ```env | |
| CLEAN_SUPABASE_DB_URI="postgresql://postgres:PASSWORD@db.XXXXX.supabase.co:5432/postgres" | |
| OPENROUTER_API_KEY="sk-or-v1-XXXXXXXX" | |
| ``` | |
| ### 3. Run ETL Pipeline (First Time Only) | |
| ```bash | |
| python etl/etl_pipeline.py | |
| ``` | |
| This loads CSV data β Raw DB β transforms β Clean DB with validation. | |
| **Expected output:** | |
|  | |
| *Screenshot of a successful ETL run β save your terminal output as `assets/images/etl_pipeline_output.png`* | |
| ### 4. Launch Application | |
| ```bash | |
| streamlit run frontend/dashboard.py | |
| ``` | |
| Open `http://localhost:8501` in your browser. | |
| --- | |
| ## ποΈ Data Model | |
| ### Entity Relationship Diagram | |
|  | |
| *ER diagram of the Supabase PostgreSQL schema β generate from pgAdmin, DBeaver, or [dbdiagram.io](https://dbdiagram.io)* | |
| ### Schema Overview | |
| ``` | |
| βββββββββββββββ ββββββββββββββββ βββββββββββββββ | |
| β dim_date β β dim_hotels β β dim_rooms β | |
| βββββββββββββββ ββββββββββββββββ βββββββββββββββ | |
| β date (PK) β β property_id β β room_id (PK)β | |
| β mmm_yy β β (PK) β β room_class β | |
| β week_no β β property_name β ββββββββ¬βββββββ | |
| β day_type β β category β β | |
| ββββββββ¬βββββββ β city β β | |
| β ββββββββ¬ββββββββ β | |
| β β β | |
| βΌ βΌ βΌ | |
| ββββββββββββββββββββββββββ ββββββββββββββββββββββββββββββββ | |
| β fact_aggregated_ β β fact_bookings β | |
| β bookings β ββββββββββββββββββββββββββββββββ | |
| ββββββββββββββββββββββββββ β booking_id (PK) β | |
| β property_id (FK) β β property_id (FK) β | |
| β check_in_date (FK) β β booking_date β | |
| β room_category (FK) β β check_in_date (FK) β | |
| β successful_bookings β β checkout_date β | |
| β capacity β β room_category (FK) β | |
| β β β booking_platform β | |
| β Grain: property + β β booking_status β | |
| β date + room_type β β revenue_generated β | |
| β (for occupancy) β β revenue_realized β | |
| ββββββββββββββββββββββββββ β ratings_given, no_guests β | |
| β β | |
| β Grain: individual booking β | |
| β (for revenue, ADR, etc.) β | |
| ββββββββββββββββββββββββββββββββ | |
| ``` | |
| ### Key Business Rules | |
| | Rule | Detail | | |
| |---|---| | |
| | **Weekend** | Friday & Saturday (stakeholder-defined, non-standard) | | |
| | **Weekday** | Sunday through Thursday | | |
| | **Revenue** | Always use `revenue_realized` (net after cancellation adjustments) | | |
| | **Cancellation** | Hotel keeps 40% of `revenue_generated`, refunds 60% | | |
| | **No Show** | Full `revenue_generated` goes to hotel | | |
| | **Ratings** | `0` means "not rated" β excluded from averages | | |
| | **week_no** | Stored as TEXT β always quote in SQL: `'31'` not `31` | | |
| | **Fact table join** | NEVER direct-join both fact tables β different granularity, use CTEs | | |
| ### Coverage | |
| | Dimension | Values | | |
| |---|---| | |
| | Date Range | May β July 2022 (92 days) | | |
| | Cities | Delhi, Mumbai, Hyderabad, Bangalore | | |
| | Hotel Categories | Luxury, Business | | |
| | Room Classes | Standard, Elite, Premium, Presidential | | |
| | Booking Platforms | MakeYourTrip, LogTrip, Tripster, Direct Online, Direct Offline, Journey, Others | | |
| | Booking Status | Checked Out, Cancelled, No Show | | |
| | Weeks | 19 β 32 | | |
| --- | |
| ## π KPI Reference | |
| ### 24 Built-in Metrics | |
| #### Base Metrics | |
| | Metric | Formula | Source | | |
| |---|---|---| | |
| | Revenue | `SUM(revenue_realized)` | fact_bookings | | |
| | Total Bookings | `COUNT(booking_id)` | fact_bookings | | |
| | Total Capacity | `SUM(capacity)` | fact_aggregated_bookings | | |
| | Total Successful Bookings | `SUM(successful_bookings)` | fact_aggregated_bookings | | |
| | Average Rating | `AVG(ratings_given) WHERE rating > 0` | fact_bookings | | |
| | No of Days | `COUNT(DISTINCT date)` | dim_date | | |
| #### Derived KPIs | |
| | Metric | Formula | Description | | |
| |---|---|---| | |
| | **Occupancy %** | Successful Bookings / Capacity Γ 100 | Room utilization rate | | |
| | **ADR** | Revenue / Total Bookings | Average revenue per booking | | |
| | **RevPAR** | Revenue / Capacity | Revenue per available room (cross-table CTE) | | |
| | **Realisation %** | 1 β (Cancellation% + No Show%) | Booking-to-stay conversion | | |
| | **Cancellation %** | Cancelled / Total Bookings Γ 100 | Booking drop-off rate | | |
| | **No Show Rate** | No Shows / Total Bookings Γ 100 | Ghost booking rate | | |
| | **DBRN** | Total Bookings / No of Days | Daily booked room nights | | |
| | **DSRN** | Total Capacity / No of Days | Daily sellable room nights | | |
| | **DURN** | Checked Out / No of Days | Daily utilized room nights | | |
| #### Week-over-Week (WoW) Metrics | |
| | Metric | Calculation | | |
| |---|---| | |
| | Revenue WoW | (Current Week / Previous Week) β 1 | | |
| | Occupancy WoW | Same pattern | | |
| | ADR WoW | Same pattern | | |
| | RevPAR WoW | Same pattern (cross-table) | | |
| | Realisation WoW | Same pattern | | |
| | DSRN WoW | Same pattern | | |
| #### Breakdown Metrics | |
| | Metric | Description | | |
| |---|---| | |
| | Booking % by Platform | Each platform's share of total bookings | | |
| | Booking % by Room Class | Each room class's share of total bookings | | |
| --- | |
| ## π€ AI Agent β How It Works | |
| ### Tool Calling Flow | |
| ``` | |
| User: "What is the RevPAR for luxury hotels in Mumbai?" | |
| β | |
| βΌ | |
| LLM understands intent | |
| β | |
| βΌ | |
| LLM calls: calculate_metrics({ | |
| metrics: ["revpar"], | |
| filters: { city: "Mumbai", category: "Luxury" } | |
| }) | |
| β | |
| βΌ | |
| Python: _build_metric_sql("revpar", filters) | |
| β Generates CTE query (cross-table) | |
| β Executes against Supabase | |
| β Returns DataFrame | |
| β | |
| βΌ | |
| LLM receives data, formats business answer: | |
| "RevPAR for luxury hotels in Mumbai: βΉ10,234 | |
| This is 15% above the portfolio average..." | |
| ``` | |
| ### Three Tools | |
| | Tool | Purpose | Reliability | | |
| |---|---|---| | |
| | `calculate_metrics` | 24 built-in KPIs with filters & grouping | β 100% (deterministic SQL) | | |
| | `run_custom_sql` | Complex ad-hoc queries (rankings, correlations) | β οΈ 85-95% (LLM-generated SQL) | | |
| | `search_metric` | Find correct metric name from business concept | β 100% (alias matching) | | |
| ### Example Queries the Agent Handles | |
| ``` | |
| Simple: "What is the total revenue?" | |
| Filtered: "Occupancy rate for Delhi luxury hotels in week 27" | |
| Grouped: "Revenue breakdown by city" | |
| Comparison: "Weekend vs weekday ADR" | |
| WoW: "How did RevPAR change week over week for week 31?" | |
| Ranking: "Top 5 hotels by revenue in Mumbai" | |
| Complex: "For each city, identify the hotel with lowest RevPAR | |
| in week 27, show the gap to city average" | |
| ``` | |
| --- | |
| ## π Anomaly Detection System | |
| ### 4-Layer Alert Engine | |
| ``` | |
| Layer 1: THRESHOLD ALERTS | |
| ββ Each KPI checked against configurable warning/critical levels | |
| Example: Occupancy < 45% β π΄ Critical | |
| Layer 2: WEEK-OVER-WEEK ALERTS | |
| ββ Detects significant WoW drops | |
| Example: Revenue dropped -12% WoW β π΄ Critical | |
| Layer 3: TREND ALERTS | |
| ββ Detects consecutive declining weeks | |
| Example: ADR declined 4 weeks straight β π΄ Critical | |
| Layer 4: PROPERTY ANOMALY DETECTION (Z-Score) | |
| ββ Flags properties deviating from portfolio mean | |
| Example: Hotel X occupancy z-score = -2.3 β π΄ Critical | |
| ``` | |
| ### Property Health Scoring | |
| ``` | |
| Score = Weighted average of normalized KPI performance | |
| Weights: | |
| Occupancy % β 25% | |
| RevPAR β 25% | |
| Avg Rating β 20% | |
| ADR β 15% | |
| Realisation % β 15% | |
| Score β₯ 80 β π’ Healthy | |
| Score 60-79 β π‘ Concern | |
| Score < 60 β π΄ Critical | |
| ``` | |
| --- | |
| ## π οΈ Tech Stack | |
| | Component | Technology | | |
| |---|---| | |
| | **Frontend** | Streamlit 1.30+ | | |
| | **Visualization** | Plotly (interactive charts) | | |
| | **Database** | Supabase (managed PostgreSQL) | | |
| | **ETL** | Python + pandas + psycopg2 | | |
| | **AI/LLM** | LiteLLM + OpenRouter (model-agnostic) | | |
| | **LLM Model** | Grok 4.1 Fast (swappable) | | |
| | **SQL Builder** | Custom deterministic engine | | |
| | **Deployment** | Streamlit Community Cloud | | |
| | **Version Control** | Git + GitHub | | |
| --- | |
| ## π Security | |
| - Database credentials stored as environment secrets (never in code) | |
| - Streamlit Cloud secrets encrypted at rest | |
| - Read-only database user recommended for production | |
| - SQL injection prevention: parameterized queries + SELECT/WITH-only enforcement | |
| - Both fact tables direct-join blocked to prevent data corruption | |
| --- | |
| ## π Performance | |
| | Metric | Value | | |
| |---|---| | |
| | KPI card load time | ~2-3 seconds (6 metrics Γ individual queries) | | |
| | Agent response time | 3-8 seconds (depends on query complexity) | | |
| | Dashboard full render | ~5 seconds (with caching) | | |
| | Cache TTL | 60 seconds (metrics), 300 seconds (context) | | |
| | Max agent iterations | 5 tool calls per question | | |
| | Supported concurrent users | Limited by Streamlit Cloud free tier | | |
| --- | |
| ## π§ͺ Testing & Validation | |
| All KPI calculations verified against a Power BI dashboard built on the same dataset. | |
| | Test Category | Queries Tested | Pass Rate | | |
| |---|---|---| | |
| | Single KPI (no filter) | 24 | 100% | | |
| | Single KPI + filters | 50+ | 100% | | |
| | Multi-metric grouped | 30+ | 100% | | |
| | WoW calculations | 12 | 100% | | |
| | Cross-table (RevPAR) | 15 | 100% | | |
| | Complex rankings (LLM SQL) | 10 | ~85% | | |
| --- | |
| ## πΊοΈ Roadmap | |
| - [ ] Conversation memory for multi-turn chat | |
| - [ ] SQL validation layer for LLM-generated queries | |
| - [ ] Export query results as CSV/Excel | |
| - [ ] Scheduled KPI monitoring with email alerts | |
| - [ ] LLM fallback chain (try multiple models) | |
| - [ ] Query logging and analytics | |
| - [ ] Role-based access control | |
| - [ ] Mobile-responsive dashboard | |
| --- | |
| ## π License | |
| This project is licensed under the MIT License β see the [LICENSE](LICENSE) file for details. | |
| --- | |
| <p align="center"> | |
| Built with β€οΈ for data-driven hospitality management | |
| </p> | |