Refactor scraping scripts to enhance source management and contextual node processing. Introduce new helper functions for pruning inactive contextual nodes and managing llms.txt sources. Update existing scripts to utilize the new source registry for improved source handling and streamline the process of adding and updating courses. Enhance error handling and logging for better traceability during data processing.
Add enabled tools management to ChatShell and SourceSidebar components. Update API to handle enabled tools in chat requests and adjust prompts accordingly. Enhance tool toggling functionality and improve UI for tool selection.
Add Next.js frontend with chat functionality, including components for chat messages, source panels, and markdown rendering. Update README with setup instructions and enhance .gitignore for frontend build artifacts.
Add FastAPI and Uvicorn dependencies, implement API for chat functionality, and introduce chat service and types. Enhance project structure with new scripts for handling chat events and responses, and add tests for API and Gradio presenter functionality.