graphrag: route Gemini 2.5 via OpenAI-compat endpoint; add index retry wrapper
Louis-François Bouchard+1: Claude Opus 4.8 (1M context)
4b82527
Native litellm gemini transport dropped connections (Server disconnected / 503) and aborted extract_graph even sequentially; the OpenAI-compatible endpoint (still gemini-2.5-flash) is stable and completed a full index first try at concurrency 6. Add run_graphrag_index.sh: loops graphrag index until entities.parquet exists (cache resumes), since one exhausted-retry call otherwise aborts the whole stage. Calibrated course-content cost ~= $88/1M corpus tokens.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
graphrag: validate retriever vs real index, fix bugs, scope by source
Louis-François Bouchard+1: Claude Opus 4.8 (1M context)
824b2e8
fix graphrag_index_exists to honor passed output_dir; index entities by id and stop using ndarray-or-empty (numpy truthiness) so expansion returns matches; cap concurrent_requests to 4 (default 25 caused Gemini broken-pipe resets); add --sources to scope the index to the eval battery sources; tests updated. Validated against a real smoke index: returns SearchResults with real source/url via text_unit.document_id, plus community-report context.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
graphrag: scaffold true GraphRAG retriever arm (opt-in, default unchanged)
Louis-François Bouchard+1: Claude Opus 4.8 (1M context)
d498287
Adds a second retrieval backend behind retrieve_tutor_context for the
GraphRAG-vs-classical-RAG experiment (branch experiment/graphrag-vs-rag).
- deps: official microsoft graphrag 3.1 (+ litellm, lancedb, ...)
- data/graphrag: graphrag workspace config (settings.yaml tuned for Gemini
2.5 Flash indexing via litellm, OpenAI embeddings, throttling, domain
entity types) + default prompts. Generated output/cache/logs/input + .env
are gitignored: the graph DB is local-only, never on the HF bundle.
- data/scraping_scripts/graphrag_prep_input.py: emit jsonl corpus input with
id=doc_id so text_units map back to corpus_manifest (source/url/lesson).
- app/graph_rag.py: GraphRAGRetriever -> list[SearchResult] (context-provider
only, no LLM generation; same Cohere rerank + token budget as classical).
- wiring: ChatRequest.retriever -> AppContext.retriever_kind -> select_retriever
branch in retrieve_tutor_context; run_battery --retriever; report +graphrag
label. Default ("" / classica…
