Commit History
-
fix(retrieval): store the BM25 index as gzipped JSON instead of pickle (#13)
Omar Solano +1: Claude Fable 5Unverified signatureHF's malware scanner flagged the public bundle's pkl as unsafe: BM25 vocabulary tokens from Python docs ("subprocess", "urllib.request", ...) are byte-identical to the STACK_GLOBAL import operands its signatures hunt for. A pickle also asks public downloaders to trust arbitrary code execution on load. The index is now a versioned gzipped-JSON payload (68 MB -> 21 MB) with full parity verified against the old pkl (equal postings, frequencies, lengths; identical search scores). Both HF bundles already ship the .json.gz and the .pkl was removed; stale local bundles self-heal because _bundle_complete() checks the new path. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>e0d6ca8 -
Use direct DeepSeek as default model
Louis-François Bouchard8a9b6de -
Use OpenRouter DeepSeek with Gemini fallback
Louis-François Boucharddc3936a
-
fix(activity): repeated Gemini search queries start a fresh cycle
GoogleSearchActivity deduped incoming queries against the previous completed cycle's list before the new-cycle reset, so a later model step re-running the same query was either invisible in the activity feed (no new grounding) or showed an empty query text (one new URL). Scope query dedupe to the active cycle: streaming repeats within a cycle still filter, but activity after completion is a new search. Grounding evidence stays deduped turn-wide so source cards never duplicate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>e77db50 -
fix(graphrag): query-relevant community reports, rerank floor parity
_community_context accepted the query but never used it: every search injected the same global top-k community reports sorted by static rank, contradicting the module's "relevant community reports" contract (unfinished wiring; the scaffold commit documents the data model but never connected it). Select reports by reranking the top-30 static-ranked candidates against the query with the existing Cohere client (one bounded rerank call per query); scores stay pinned at 0.0 so reports remain context-only. Also applies the classical retriever's RERANK_SCORE_FLOOR in _apply_token_budget (exempting the score-0.0 community reports) per the module's fairness contract, and skips oversized results instead of breaking, matching the classical fix. Note: a future re-run of the graphrag eval arm will not be directly comparable to the recorded F28 numbers, which measured the static-report behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>72f3c77 -
fix(retrieval): budget skip, standard RRF, fail-fast collection load
Three hybrid-retrieval fixes: - _apply_token_budget broke on the first over-budget result, so one oversized rank-1 document (retrieve_doc under a small per-request budget) returned zero context even when smaller lower-ranked chunks fit. Skip the oversized result and keep filling the budget. - reciprocal_rank_fusion accumulated one contribution per occurrence of a dedupe key within a single ranked list, so a section split into several chunks in one retriever's top-k masqueraded as cross-retriever consensus. Standard RRF: each key counts once per list at its best rank; representative selection is unchanged. - LocalChromaRetriever used get_or_create_collection, silently creating an empty collection from a broken or mismatched bundle and degrading dense search to zero hits with no signal. Use get_collection and raise a clear error naming the collection and db path. Also extracts the 0.10 rerank floor as RERANK_SCORE_FLOOR for reuse by the GraphRAG backend. Co-Authore…0f36b6a -
fix(config): restore default dotenv precedence (shell wins over .env)
load_dotenv(override=True) crept in with an unrelated UI commit and inverted standard precedence: .env values silently clobbered shell/deploy env vars, so per-run overrides like AI_TUTOR_MEMORY_PRESET=x uv run -m app.api ran with the wrong config while claiming otherwise. Use the library default (override=False) in app.config and align evals/judge.py, whose override=True existed only to match app.config. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>75e4226 -
fix(config): honor AI_TUTOR_KB_DIR for every KB path
app/kb_shell.py honored the documented AI_TUTOR_KB_DIR env var but config hardcoded data/kb for KB_DIR and everything derived from it (manifest, wiki index, AGENTS.md), and prompts.py hardcoded its own copy. Setting the env var split the app across two KB trees: the sandbox browsed one while citation resolution, bundle checks, and the prompt builder read the other, and cold start could spuriously re-download the bundle. Derive them all from the env var (normalized, keeping config's relative-string convention) and warn on cold start that the HF bundle only materializes data/kb. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>d4aa2ef -
fix(citations): keep labels of links with balanced brackets
parse_markdown_citations missed CommonMark-legal links whose text contains balanced brackets ([LoRA [PEFT docs]](url)); the bare-URL pass recovered the reference but dropped the label, losing title-based evidence matching. Allow one level of balanced brackets in the label, mirroring the frontend's LINK_OPENER_PATTERN. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>f1138d7 -
perf(citations): cache the KB manifest indexes
manifest_indexes rebuilt four dicts over all 3.2k manifest entries on every call, and resolve_manifest_reference calls it once per reference; extract_shell_source_matches resolves every raw/ path a KB command printed, synchronously on the event loop, so one tool output listing 40 paths stalled every in-flight stream ~75ms. Cache the indexes keyed by the loaded entries tuple (identity), so repeated lookups are O(1) and a repopulated manifest cache rebuilds them. Nothing is cached while the manifest file is absent, preserving the first-start download window behavior. 40-path resolution: 78.6ms -> 0.1ms. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>935b714 -
fix(api): close request-size gaps in body and history caps
Two hardening gaps around the existing request bounds: - The oversized-body middleware only checked Content-Length, so a chunked request bypassed the cap entirely. Replace it with a pure ASGI middleware that keeps the header fast-path 413 and counts the bytes actually delivered on the receive channel. - History turns extracted from the AI-SDK messages field were not held to MAX_TURN_CHARS (only payload.history is, via its Field cap), so a single message under MAX_MESSAGE_JSON_CHARS could carry a ~190k-char turn into the model call. Truncate extracted turns to the same bound; truncation rather than 422 because the oversized text already sits in the client transcript and rejecting would wedge the conversation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>c0cba75 -
fix(api): emit stream error frame after cleanup parts
The AI SDK v6 client rethrows on the error chunk, which aborts stream processing and drops every part after it. finish_error emitted the error part first, so its cleanup parts (tool-output-error for open calls, text-end, finish-step, finish) never reached the client and open tool calls rendered as running forever after a mid-turn failure. Emit the error part last so cleanup applies before the client aborts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>25b8052
-
fix(citations): handle malformed links; count only consulted docs in activity (#12)
Omar Solano +1: Claude Fable 5Unverified signatureInvestigating LangSmith trace 019f29f4-4e21-7a01-86b8-053161eb08d6 ("How do I evaluate and iterate on my prompts...") surfaced two citation issues: 1. Inline chips numbered 1 and 3 with no 2: the model closed two citations with "]" instead of ")". The client's link scanner counted the malformed destination as URL #2, but micromark never renders it as an anchor, so the number was assigned yet never displayed. scanLinkDestination now only accepts destinations followed by ")" or a link title (what actually renders), so unrendered links no longer consume citation numbers. Server-side, parse_markdown_citations no longer lets a malformed link match across newlines and swallow the next citation, and the bare-URL pass recovers the malformed link's URL (trailing "]"/":" stripped) so it still resolves to a source card. 2. Activity pill said "20 sources" for a turn that read 2 docs: cat'ing one wiki topic page put all 20 docs it links to into the tool event's ma…2bacdcc
-
Public docs-only bundle: allowlist filtering, leak audit, kb.tar.gz cold start (#11)
Omar Solano +1: Claude Fable 5Unverified signature* feat(public-bundle): allowlist filtering, wiki publicizing, leak audit, anonymous fallback Harden the public docs-only bundle pipeline before its first publish: - build_public_docs_bundle: flip all filtering from a COURSE_SOURCE_KEYS denylist to a DOC_SOURCE_KEYS allowlist (fail closed). Chroma is now filtered by scanning every chunk's metadata, so chunks with missing or unknown source metadata are dropped too; contextual-nodes records that cannot be classified are dropped instead of kept. - Publicize the staged wiki per data/kb/MAINTAINER.md: re-run the scaffolder on the filtered manifest (AUTO-GENERATED blocks lose course entries), deterministically prune course references from maintainer prose, and append a (maintainer) log entry. Exclude MAINTAINER.md from the public bundle (private-bundle-only; quotes course paths). - Add a leak audit that fails the build if any course path, course key, or non-allowlisted row survives in the staged KB. - Ship a script-owned d…7e5f705
-
wip: eval harness + public docs bundle; relocate eval docs into evals/ (#10)
Omar Solano +1: Claude Opus 4.8 (1M context)Unverified signatureWork-in-progress checkpoint of the current eval cleanup: - relocate the 6 evals_*.md docs into evals/ (drop the evals_ prefix); rewrite every reference (evals.md, AGENTS.md, pyproject.toml, data/eval/README.md, evals/grade.py, upload_dbs_to_hf.py, cross-links) - evals/ harness: grade/judge/report/run_battery changes + new grading_prep.py, grading_merge.py, grade_workflow.js - data/scraping_scripts: build_public_docs_bundle.py + upload/gitignore wiring - app/ (api, config, kb_manifest) and tests tweaks - gitignore local-only working dirs (code_review/, workshop_AIE/) Claude-Session: https://claude.ai/code/session_01BDhiM239toao3TAmoJ1joD Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>dc474cb
-
fix(citations): drop code-block URLs from chips; default URL reading off
parse_markdown_citations now strips fenced and inline code before harvesting citations, mirroring the frontend's stripCodeSegments. A URL shown only as a code example (a `git clone` command, or an inline `https://` span) is no longer surfaced as a source chip. Also exclude backticks from the bare-URL body so a malformed "https://`" can never be captured. Fixes the orphan/broken Web chips seen in the repo-cloning answer (LangSmith trace 019f004f). Flip the url_context toggle to default off (active: False); enabling it still flips keep_unresolved_sources on per turn, so unresolved cited URLs only become low-trust Web chips when the user opts in. Document the enabledTools semantics for direct API callers (in api.py and the manual e2e runbook). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>64590e2
-
feat(prompts): add course content reproduction policy and related tests
9e804b1 -
Merge experiment #3: SLM knowledge compaction (Ollama small models, 32k window)
Lands the small-local-model compaction study on top of the graphrag (#2) and context-compaction (#5) experiments already in main. Two scoped experiments on 3 Ollama SLMs (llama3.1:8b, qwen2.5:7b, qwen3:8b) at a 32k window where the ~37.7k-token lesson overflows: Axis B (fit a document: RAG vs stuff vs summarize) and Axis A (compact growing history via the real memory presets). Fully opt-in; default chat model and prod behavior unchanged. Findings (hand-authored, renumbered to avoid colliding with main's F24/F25): - F30 — small window is a second boundary where keep-all stops winning; when the doc doesn't fit, "shove it all" is forced into truncation and is dominated, so retrieval is necessary (small-window companion to F29, Axis B). - F31 — on an SLM, no universal best history-compaction method and keep-all never wins; model capability dominates the method (SLM counterpart to F9/F15, Axis A). - F32 — synthesis: the boundaries where "keep everything" stops winning (ties F9/F15 …6145817 -
Merge experiment #5: context-compaction study (keep-all vs compaction vs retrieve, Gemini 2.5 Flash)
Lands the keep-vs-compact-vs-retrieve harness on a single long lesson (evals/knowledge_compaction.py, compaction_study.py), the delta_summarization and hierarchical_summarization presets + HierarchicalSummarizationMiddleware, and evals_compaction.md. One file ruff-formatted to repo standard. Knowledge promotion to the central log (F29) follows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>1ab0db9 -
Merge experiment #2: GraphRAG vs classical RAG (opt-in retriever + scoped comparison)
Lands the GraphRAGRetriever (opt-in; default retrieval path unchanged), the index pipeline, the HF-published index for reproduction, and evals_graphrag.md. Knowledge promotion to the central findings log (F28) is a follow-up commit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>b596739 -
feat(evals): single-turn latency finding (F27) + de-confound latency tooling
Omar Solano +1: Claude Opus 4.8 (1M context)Unverified signatureSingle-turn "full_history fastest" is a tail effect from prod summarizing mid-turn (re-work within one turn), not a median win; the seconds are time-of-run confounded. Adds F27, time_to_first_token_ms telemetry, and a report that leads with work metrics and tags latency rows [confounded]. Interleaved screen script + regenerated reports live in gitignored runs/. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>c3ce548
-
evals(slm): Axis-A memory-compaction arm on local SLMs
Louis-François Bouchard +1: Claude Opus 4.8 (1M context)Companion to the Axis-B doc-compaction study: run the memory-compaction methods (how to compact a GROWING conversation history) on a small local model forced to compact, using the real app middlewares. - build_chat_model: add an `ollama` provider that routes through Ollama's OpenAI-compatible endpoint (ChatOpenAI base_url). Experiments only; default behavior unchanged. The context window is set on the Ollama side via a num_ctx Modelfile variant, so full_history overflows a 32k window and every compaction preset is forced to fire. - run_slm_axis_a.sh: run the long-context session battery (turn-0 lesson + questions, retrieval off) under each memory preset (full_history / sliding_window / summarization_only / prompt_compression / selective_retention / incontext_history_retrieval / delta_summarization / hierarchical_summarization) on a num_ctx-capped SLM, then judge vs the lesson and report. Tracing disabled (local, quota noise). Co-Authored-By: Claude Opus 4.8 (1M cont…c93290d -
memory: add hierarchical and delta summarization presets (Axis A)
Louis-François Bouchard +1: Claude Opus 4.8 (1M context)Two new memory presets for the compaction study's Axis A (how to edit existing context), each isolating a summarization strategy with everything else held constant: - delta_summarization: a single running summary updated each trigger with only what changed (prior summary + new turns folded into a fresh summary). - hierarchical_summarization: a new map-reduce middleware that summarizes the older messages in groups, then summarizes the group summaries into one layered summary in the model's view. Cached by older-block content hash, so a static long document loaded turn 0 is summarized once, not every turn. Per-call view only (checkpoint untouched). The extra summarization calls are reported via summary_messages but not folded into the turn's est_cost (noted in the docstring). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>bbeda18 -
compaction study: harness for compaction-vs-keep-everything on a long lesson
Louis-François Bouchard +1: Claude Opus 4.8 (1M context)Axis-A study: load the largest course lesson (37.5k tok) in turn 0, ask 15 questions with NO tools, run under each memory preset (full_history vs prod/summarization_only/sliding_window/prompt_compression/selective_retention/incontext_history_retrieval/aggressive) on Gemini 2.5 Flash, and judge each answer against the lesson. Plus a retrieve-per-question family (rag/graphrag over the lesson) via knowledge_compaction.py for a unified comparison. Adds run_battery --no-tools, gemini-2.5-flash pricing, prep --doc-ids. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>0ad2e61 -
graphrag: publish the index to HF for reproducibility (prod cold-start skips it)
Louis-François Bouchard +1: Claude Opus 4.8 (1M context)Add upload_dbs_to_hf --graphrag (prune-free upload of data/graphrag/output to graphrag/output in the same private dataset repo); make ensure_local_vector_db ignore graphrag/** so prod Spaces don't pull the ~150MB experiment index; document the pull command. Anyone with the HF token can now pull and re-run the eval without the ~$45 rebuild. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>814e43b
-
style: apply ruff formatting to telemetry and report
Line-wrap a ModelPricing entry and three report.py expressions to match ruff's format. These were committed unformatted in e7f812d and would fail CI's `ruff format --check`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>0a22ac4 -
Refactor telemetry pricing and evaluation reports
- Updated MODEL_PRICING in telemetry.py to clarify DeepSeek pricing and added first-party API details. - Enhanced evals.md to provide a comprehensive overview of memory management strategies and evaluation arms, including detailed findings and dataset descriptions. - Improved grading logic in grade.py to accurately reflect the source of grades (judge vs human) for reporting purposes. - Adjusted report.py to include grade source in the report headers and refined metric descriptions for clarity. - Revised evals_part_c_plan.md to clarify the execution plan and findings from the subset screen, ensuring alignment with the overall evaluation strategy.e7f812d -
graphrag: validate retriever vs real index, fix bugs, scope by source
Louis-François Bouchard +1: Claude Opus 4.8 (1M context)fix graphrag_index_exists to honor passed output_dir; index entities by id and stop using ndarray-or-empty (numpy truthiness) so expansion returns matches; cap concurrent_requests to 4 (default 25 caused Gemini broken-pipe resets); add --sources to scope the index to the eval battery sources; tests updated. Validated against a real smoke index: returns SearchResults with real source/url via text_unit.document_id, plus community-report context. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>824b2e8 -
Add OpenRouter provider for open-model evals
sam04-ops +1: Claude Opus 4.8 (1M context)Support an OpenRouter provider (OpenAI-compatible gateway) so open models like DeepSeek can run through the chat path, with stream_usage and retries tuned for long agentic eval sessions. Add DeepSeek-V4-Flash pricing so context_stats/cost telemetry populates, and LFS-track kb/generated/symbols.tsv. Also harden .gitignore to ignore the top-level eval/ working dir wholesale, preventing non-.jsonl student-text files (review_log_v1.md, review_batches/) from slipping into git and onto the public prod Space. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>4c3b588 -
graphrag: scaffold true GraphRAG retriever arm (opt-in, default unchanged)
Louis-François Bouchard +1: Claude Opus 4.8 (1M context)Adds a second retrieval backend behind retrieve_tutor_context for the GraphRAG-vs-classical-RAG experiment (branch experiment/graphrag-vs-rag). - deps: official microsoft graphrag 3.1 (+ litellm, lancedb, ...) - data/graphrag: graphrag workspace config (settings.yaml tuned for Gemini 2.5 Flash indexing via litellm, OpenAI embeddings, throttling, domain entity types) + default prompts. Generated output/cache/logs/input + .env are gitignored: the graph DB is local-only, never on the HF bundle. - data/scraping_scripts/graphrag_prep_input.py: emit jsonl corpus input with id=doc_id so text_units map back to corpus_manifest (source/url/lesson). - app/graph_rag.py: GraphRAGRetriever -> list[SearchResult] (context-provider only, no LLM generation; same Cohere rerank + token budget as classical). - wiring: ChatRequest.retriever -> AppContext.retriever_kind -> select_retriever branch in retrieve_tutor_context; run_battery --retriever; report +graphrag label. Default ("" / classica…d498287
-
Refine v2 eval plan and memory harness
76c99a5
-
Add Part C eval variants, LLM judge, and harness fixes
Part C memory/context-management experiment layer: - evals/judge.py: blinded LLM judge (run + validate) that auto-fills the handgrade sheet and gates at >=90% TP/TN vs human labels - app: Part C variants in memory_presets.py + middlewares (sliding_window, prompt_compression, selective_retention, context_reset, observation_truncation, clear_retrieval_kb, incontext_history_retrieval), plus retrieval-budget and disable-kb knobs threaded through chat_service, chat_types, chroma_rag, prompts - telemetry: per-turn signal registry so per-call-view middlewares report compaction in context_stats - evals harness: generalized compaction-signal gate (common.py), md5 sheet_row_id, drop dead prune_incomplete params; tests/test_memory_variants.py - docs: evals_part_c_plan.md; evals.md Part C findings + part_* download fix Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>c344089
-
Add evaluation harness for memory/context-management experiments
Compare memory presets (full_history, prod, summarization/editing-only, aggressive, profile_memory) on quality, memory retention, retrieval, tokens, cost, and latency: - app: memory presets (app/memory_presets.py), per-turn telemetry independent of LangSmith (app/telemetry.py, context_stats event + data-context-stats SSE part), long-term student-profile memory (StudentProfileMiddleware + store), memoryPreset/studentId API fields - evals/: battery runner (resume-safe, per-turn timeout, trace bundles), offline grader (code checks + blinded hand-grading workbook), report (side-by-side preset tables + token curves), trigger-verification gate - docs: evals.md (datasets, metrics, Part B results, findings F1-F9), evals_background.md (research notes), data/eval/README.md (schemas) - data/scraping_scripts/annotate_discussion_posts.py: discussion-post annotation used to build the eval batteries Eval datasets and run results contain real student text and ship via the private HF da…a04f9ea
-
Enhance knowledge base and API functionality
- Updated .gitignore to include local environment files for better configuration management. - Enhanced the build_chat_request function to handle explicit empty source selections, ensuring that the knowledge base is correctly disabled when no sources are selected. - Improved the UIMessageStreamEncoder to include additional metadata for source matches, enhancing the information available to the frontend. - Introduced reranking logic in the rerank_results function to score matched chunks instead of entire documents, optimizing relevance and performance. - Added comprehensive tests to validate the new functionalities, ensuring robustness in knowledge base interactions and API responses.f2e8e50 -
Enhance API validation and error handling
- Introduced request size validation in the API to reject oversized bodies before parsing, improving resource management. - Updated ApiChatRequest and ApiChatTurn models to enforce character limits on fields, ensuring better input validation. - Enhanced the lifespan function to pre-parse the KB corpus manifest, optimizing performance during initial requests. - Improved error handling in the stream_chat function to ensure consistent responses during mid-stream HTTP exceptions. - Added tests to validate new request size handling and error response behaviors, ensuring robustness in API interactions.9db7240 -
Enhance model validation and tool call management
- Updated the build_chat_request function to validate model names against available models, raising an HTTPException for unknown models. - Enhanced UIMessageStreamEncoder to manage open tool call IDs, ensuring proper closure of tool calls that do not complete, and added error handling for unfinished tool calls. - Modified the GoogleSearchActivity class to accurately track activity completion per cycle, preventing duplicate events and ensuring correct state management. - Updated tests to validate the new model validation and tool call management functionalities, ensuring robustness in error handling and state transitions.b9d31e4 -
Enhance chat service and provider events handling
- Updated the build_chat_model function to ensure that reasoning mode shares the same max_tokens limit as the non-reasoning path, preventing truncation of responses. - Improved the extract_anthropic_source_matches function to handle different result types from web_fetch and web_search, ensuring accurate evidence recording. - Added tests to validate the behavior of reasoning output budgets and the handling of tool completion events during summarization, enhancing overall reliability.04ac085 -
Enhance chat service and configuration management
- Introduced URL extraction functionality to deduplicate and trim URLs from user queries, improving evidence handling for web citations. - Updated the build_agent function to incorporate knowledge base agent instructions into the system prompt, ensuring proper cache management. - Enhanced the ensure_kb_agents_md function for atomic file writing, preventing race conditions during concurrent access. - Improved local vector database verification to ensure completeness before proceeding with operations, reducing potential errors. - Added comprehensive tests for new functionalities, including URL extraction and knowledge base instructions handling, ensuring robustness and reliability.bc9b417 -
Implement thread management and transcript handling in chat service
- Introduced mechanisms for managing thread transcripts and fork points, allowing for better tracking of conversation history and state. - Added functions to record and retrieve thread transcripts, manage idle thread eviction, and handle thread state updates. - Enhanced the checkpointing process to ensure that tool-using turns are accurately represented in the visible chat history. - Updated tests to validate the new functionality, ensuring that thread continuity is maintained across tool interactions and edits.c205736 -
Enhance citation handling and UI components
- Updated citation handling in chat messages to include citation numbers, improving clarity and usability. - Introduced citation chips in Markdown rendering for better visual representation of citations. - Refactored CitationRow and TextBlock components to support new citation number functionality, ensuring consistent display across the application. - Enhanced citation normalization and numbering logic to ensure accurate citation tracking in messages.3c4adeb
-
Enhance reasoning delta handling and chat message preview
- Updated UIMessageStreamEncoder to manage reasoning deltas more effectively, including handling paragraph breaks for block-granularity deltas. - Modified stream_chat function to indicate whether reasoning deltas are complete blocks, improving the handling of thought summaries. - Refactored ToolRow component to implement a new ToolOutputPreview for better display of chat message outputs, including truncation for long texts.b155332 -
Enhance UIMessageStreamEncoder and chat service functionality
- Added text block management methods in UIMessageStreamEncoder to handle text blocks more effectively, including closing text blocks and generating unique text block IDs. - Updated the stream_chat function to support tool call handling for Anthropic models, ensuring proper announcement of tool calls and managing tool call arguments. - Improved reasoning handling by allowing interleaved thinking blocks in chat models, enhancing the overall chat experience.88978a0 -
Enhance error handling in BM25 index loading
- Added exception handling when loading the BM25 index to log warnings for failures and unexpected types. - Implemented fallback to dense-only retrieval if loading fails or if the index type is incorrect, improving robustness.1f970f0 -
Refactor project structure and update dependencies
- Introduced a new `config.py` file to centralize configuration and source management, replacing references to `setup.py`. - Updated AGENTS.md to reflect changes in source configuration and clarify the single source of truth for sources. - Removed `ipykernel` from main dependencies and added it to the development group in `pyproject.toml` and `uv.lock`. - Adjusted import statements across various modules to utilize the new `config.py`. - Enhanced logging practices in the application for improved clarity and reduced noise.d879dad -
Renamed 'scripts' folder to 'app', removed mongo and instructor libraries
bd180ae