omarsol Claude Fable 5 commited on
Commit
69d44ad
·
1 Parent(s): a190140

De-package graphrag and move to the openai v3 SDK

Browse files

graphrag -> graphrag-llm -> litellm pins openai<3 (and litellm has not
lifted it in any release), which capped the whole lockfile, and the
GraphRAG-vs-RAG experiment (F28) is concluded. Reproducing it is now a
manual side install documented in evals/graphrag.md; contributing.md
describes that pattern for conflicting eval-only dependencies.

With both pins gone the project floors openai>=3. Nothing imports the SDK
directly (langchain-openai 1.5.0 supports 2.x and 3.x), and v3's only
breaking change is the httpx2 HTTP layer with OS trust-store verification,
which the python:3.13 image satisfies.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Files changed (4) hide show
  1. evals/contributing.md +1 -1
  2. evals/graphrag.md +14 -8
  3. pyproject.toml +3 -8
  4. uv.lock +0 -0
evals/contributing.md CHANGED
@@ -117,7 +117,7 @@ The findings log is **"what did we learn"** — durable lessons about managing t
117
 
118
  ## 6. Dependencies
119
 
120
- If your experiment adds a dependency production doesn't need (e.g. a retriever you concluded not to adopt), put it in `[project.optional-dependencies]` and guard its tests with `pytest.importorskip`, so the prod image (`uv sync --locked`) stays lean. Example: the `graphrag` extra — reproduce with `uv sync --extra graphrag`.
121
 
122
  ## 7. Merge back
123
 
 
117
 
118
  ## 6. Dependencies
119
 
120
+ If your experiment adds a dependency production doesn't need (e.g. a retriever you concluded not to adopt), keep it out of `pyproject.toml`: document a manual `uv pip install ...` in the experiment's doc and guard its tests with `pytest.importorskip`, so the prod image (`uv sync --locked`) stays lean and the shared lockfile never inherits the experiment's version pins. Example: graphrag (whose litellm pins `openai<3`) — see the Setup section of `evals/graphrag.md`.
121
 
122
  ## 7. Merge back
123
 
evals/graphrag.md CHANGED
@@ -51,19 +51,25 @@ question). Both arms run on those 41 cases, both restricted to that source
51
  Measured course-content indexing rate: **~$88 / 1M corpus tokens** (vs ~$255 for
52
  dense API-reference docs). full_stack = 486k tokens -> **~$43** to index.
53
 
54
- ## Setup (optional extra)
55
 
56
- `graphrag` (and its `lancedb`/`litellm`/`pandas` deps) is an **eval-only optional
57
- extra** — it is imported lazily and only by the opt-in `graphrag` retriever, never
58
- on the production path, so it is kept out of the default install to keep the prod
59
- image lean. Install it to build/run this experiment:
 
 
60
 
61
  ```bash
62
- uv sync --extra graphrag
63
  ```
64
 
65
- A default `uv sync` (prod, CI) omits it; `tests/test_graph_rag.py` skips itself
66
- when the extra is absent.
 
 
 
 
67
 
68
  ## Build the graph DB
69
 
 
51
  Measured course-content indexing rate: **~$88 / 1M corpus tokens** (vs ~$255 for
52
  dense API-reference docs). full_stack = 486k tokens -> **~$43** to index.
53
 
54
+ ## Setup (manual install)
55
 
56
+ `graphrag` (and its `lancedb`/`litellm`/`pandas` deps) is deliberately **not in
57
+ `pyproject.toml`**: its litellm dependency pins `openai<3`, which would cap the
58
+ whole project's openai SDK, and the experiment is concluded (F28) so it does not
59
+ justify permanent packaging. It is imported lazily and only by the opt-in
60
+ `graphrag` retriever, never on the production path. To build/run this
61
+ experiment, install it into your synced venv on the side:
62
 
63
  ```bash
64
+ uv pip install "graphrag>=3.1.0"
65
  ```
66
 
67
+ Two consequences of the manual install: it downgrades `openai` to 2.x inside
68
+ your venv while graphrag is present (fine for the experiment; langchain-openai
69
+ supports both), and any later `uv sync` prunes graphrag and restores openai v3
70
+ (re-run the install command if you need it again). A default `uv sync` (prod,
71
+ CI) never includes it; `tests/test_graph_rag.py` skips itself when graphrag is
72
+ absent.
73
 
74
  ## Build the graph DB
75
 
pyproject.toml CHANGED
@@ -19,7 +19,9 @@ dependencies = [
19
  "langchain-openai",
20
  "nbconvert",
21
  "nbformat",
22
- "openai",
 
 
23
  "pydantic",
24
  "python-dotenv",
25
  "requests",
@@ -27,13 +29,6 @@ dependencies = [
27
  "uvicorn",
28
  ]
29
 
30
- [project.optional-dependencies]
31
- # Eval-only: the GraphRAG-vs-RAG experiment (F28). Pulls graphrag + lancedb/litellm/
32
- # pandas, all imported lazily and only by the opt-in `graphrag` retriever — never on
33
- # the production path. Kept out of the default install so the Spaces image stays lean.
34
- # Reproduce the experiment with: uv sync --extra graphrag (see evals/graphrag.md).
35
- graphrag = ["graphrag>=3.1.0"]
36
-
37
  [dependency-groups]
38
  dev = [
39
  "httpx",
 
19
  "langchain-openai",
20
  "nbconvert",
21
  "nbformat",
22
+ # Not imported directly (langchain-openai is the consumer, and it supports
23
+ # 2.x and 3.x); the floor documents that this repo expects the v3 SDK.
24
+ "openai>=3",
25
  "pydantic",
26
  "python-dotenv",
27
  "requests",
 
29
  "uvicorn",
30
  ]
31
 
 
 
 
 
 
 
 
32
  [dependency-groups]
33
  dev = [
34
  "httpx",
uv.lock CHANGED
The diff for this file is too large to render. See raw diff