AdaGraphRAG

Adaptive Hierarchical Graph-RAG with Incremental Construction and Self-Verified Retrieval

Extends Microsoft's GraphRAG — Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., Metropolitansky, D., Ness, R. O., & Larson, J. (2024). From Local to Global: A Graph RAG Approach to Query-Focused Summarization. arXiv:2404.16130 — with three contributions:

#	Problem	Contribution
1	Every query pays for expensive global retrieval, even simple factual ones	Adaptive Retrieval Router — classifies each query, picks the cheapest sufficient strategy (none / local / global / hybrid)
2	Full corpus re-indexed on every new document	Incremental Graph Construction — new documents are entity-resolved and merged into the existing graph; only changed communities are re-summarized
3	No check that generated answers are grounded in evidence	Self-Verification Layer — decomposes answers into claims, checks each against retrieved evidence, can trigger a second targeted retrieval pass on unsupported claims

Architecture

Indexing pipeline — turns a folder of documents into a queryable knowledge graph:

flowchart LR
    A[Documents<br/>.txt / .md] --> B[Chunking]
    B --> C[Entity + Relation<br/>Extraction]
    C --> D[Entity Resolution<br/>merge/create decisions]
    D --> E[(Neo4j<br/>Graph Store)]
    D --> F[(Qdrant<br/>Vector Store)]
    E --> G[Community Detection<br/>Leiden, multi-level]
    G --> H[Community<br/>Summarization]
    H --> E

Loading

Query pipeline — the three contributions in the flow they actually run in:

flowchart LR
    Q[User Query] --> R{Adaptive Router<br/>classifies complexity}
    R -->|no_retrieval| GEN[Generate Answer]
    R -->|simple_factoid| LOC[Local Retrieval<br/>Qdrant vector search]
    R -->|multi_hop| HYB[Hybrid Retrieval<br/>Local + Global]
    R -->|global_sensemaking| GLB[Global Retrieval<br/>Community summaries]
    LOC --> RANK[Fusion + Cross-Encoder<br/>Reranking]
    HYB --> RANK
    GLB --> RANK
    RANK --> GEN
    GEN --> VER{Self-Verification}
    VER -->|claim unsupported| RETRY[Targeted re-retrieval<br/>for that claim]
    RETRY --> VER
    VER -->|all claims checked| OUT[Answer +<br/>per-claim support labels]

Loading

Structure

src/adagraphrag/
├── domain/            entities, interfaces (no dependencies)
├── config/, logging_utils/, utils/
├── llm/, vectorstore/, graphstore/     infrastructure adapters
├── graph_construction/                 Contribution #2
├── retrieval/, reranking/               Contribution #1
├── verification/                         Contribution #3
├── pipeline/                              orchestration + DI factory
└── ui/                                     Streamlit app
tests/        unit + integration tests (offline, no real infra needed)
scripts/      seed_databases.py, build_graph.py, evaluate.py
configs/      YAML configuration

A typer CLI and a FastAPI service are natural next additions but aren't built yet — for now, indexing and querying go through scripts/build_graph.py and the Streamlit UI below.

Install

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements-dev.txt
pip install -e .
cp .env.example .env

Edit .env: set your LLM provider's API key (OPENAI_API_KEY or GEMINI_API_KEY), and NEO4J_PASSWORD if using Neo4j/Qdrant.

Using the Gemini free tier (click to expand)

Free API keys are capped on both requests-per-minute and requests-per-day, and the per-day cap can be surprisingly low — some newer preview-tier models have shipped with free daily caps as low as 20 requests, which graph construction can burn through indexing a handful of short documents. Check your key's actual current limits at aistudio.google.com under "Rate limits" before relying on any numbers here or elsewhere — they vary by model and change over time. This repo defaults to gemini-3.5-flash-lite, a stable (non-preview) Flash-Lite model, which has historically had one of the most generous free tiers — but verify that's still true for your key, and be aware Google periodically retires older model IDs for new API keys/projects, which will surface as a 404 naming its own replacement if it happens again.

Three things in this repo help stay under whatever caps you land on:

• Indexing multiple documents through index_documents() (the Streamlit "Index uploaded files" button, or scripts/build_graph.py on a folder) re-detects and re-summarizes graph communities once for the whole batch, not once per document — upload/pass all your files together rather than one at a time.
• llm.gemini.max_requests_per_minute in configs/llm_config.yaml paces every request (including embeddings) so a burst of calls doesn't trip the per-minute limit.
• runtime.enable_llm_cache in configs/base.yaml caches every LLM call to disk, so re-running indexing/queries on text you've already processed doesn't re-spend the daily quota. Clear it with adagraphrag.utils.caching.clear_cache() if you need a fresh run.

If you still hit the daily cap, index a small corpus (a handful of short documents) rather than a large one — graph construction makes one LLM call per chunk for extraction plus more for entity resolution and community summaries, so it adds up fast on a ~25-100/day free quota.

Neo4j/Qdrant via Docker (click to expand)

If you're running them as Docker containers (docker start <qdrant-container> <neo4j-container>), give Neo4j a few seconds to finish booting before running anything — docker start returns immediately but Neo4j doesn't accept Bolt connections on :7687 right away. If you hit a connection error immediately after starting the containers, wait ~15s and retry, or tail docker logs -f <container> until you see Started..

Run

# tests (fully offline, no infra needed)
pytest tests/ --cov=adagraphrag

# one-time: create the Neo4j schema + Qdrant collection
python scripts/seed_databases.py

# index a folder of .txt/.md files
python scripts/build_graph.py data/raw/my_corpus/

# query + explore the graph
streamlit run src/adagraphrag/ui/streamlit_app.py

Evaluation

scripts/evaluate.py runs two small experiments against an already-indexed corpus. This is a toy-corpus sanity check, not a rigorous benchmark — no held-out dataset, no human-labeled gold answers, no statistical significance testing. Good enough to verify the two headline design decisions actually behave as intended.

python -c "from adagraphrag.utils.caching import clear_cache; print(clear_cache(), 'files deleted')"
python scripts/evaluate.py

Self-verification precision on injected claims

Fed the evidence matcher 3 true claims paraphrased from the indexed corpus and 3 deliberately false ones (facts not in the corpus, or contradicting it):

Claim	Expected	Predicted	Correct
Aspirin can cause stomach irritation.	true	supported	✅
Aspirin is generally not recommended for children due to Reye's syndrome risk.	true	supported	✅
Ibuprofen is a nonsteroidal anti-inflammatory drug.	true	supported	✅
Aspirin is proven to cure cancer.	FALSE (injected)	unsupported	✅
Ibuprofen is recommended as the safest pain reliever during pregnancy.	FALSE (injected)	unsupported	✅
Headaches are always caused by brain tumors and require immediate surgery.	FALSE (injected)	unsupported	✅

Accuracy: 6/6 (100%) on this small claim set.

Adaptive routing vs. always-global

Compares chat-completion call count between the router's chosen strategy and a baseline forced to always use global retrieval (what plain GraphRAG does for every query), with self-verification excluded from both arms to isolate the router's own cost:

Query	Strategy	LLM calls	Latency (s)
What is the capital of France?	adaptive → none	2	4.0
What is the capital of France?	always-global	1	14.4
What treats headaches?	adaptive → none	2	2.3
How does aspirin's side effect risk compare to ibuprofen's?	adaptive → hybrid	2	41.5
Summarize everything the corpus says about pain relief options.	adaptive → global	2	14.5

On raw chat-completion call count, adaptive routing used more calls (8 vs. 4 total) — every strategy still needs one final generation call, and the router adds one classification call as fixed overhead. This metric is incomplete, though: it doesn't count embedding calls, and the always-global baseline pays for a full 57-community embedding batch (12-14s) on every query regardless of complexity — a cost the none and local strategies skip entirely, visible in the latency column even though it isn't reflected in the call count. A complete cost comparison would need to track embedding calls alongside chat completions; that's a known gap in this evaluation script rather than a claim that adaptive routing is unambiguously cheaper.

Bugs this evaluation surfaced

Worth stating plainly rather than burying: writing this eval script found three real bugs, since fixed — self_verifier.py: an evidence pool was leaking across claims (O(n²) prompt growth, 30-50s latency spikes on later claims); caching.py: clear_cache() used a non-recursive glob and silently missed the LLM response cache's subdirectory; and a Neo4j query pattern that triggered a cartesian-product warning on every relationship write. None affect correctness of the final answers, but the caching one made an earlier "clean" evaluation run silently invalid until caught.

Limitations & Future Work

• Evaluation scope. The results above come from a 2-3 document toy corpus and a handful of hand-written test queries and injected claims — enough to sanity-check that the router and verifier behave as designed, not enough to make a statistically meaningful claim about accuracy or cost on real-world corpora. A proper evaluation would need a larger, held-out corpus (e.g. a subset of the datasets used in the original GraphRAG paper), more queries per complexity class, and a cost metric that includes embedding calls alongside chat completions (see the routing table above for why that gap matters).
• Router calibration. During evaluation the router occasionally under-classified a query that the indexed corpus could directly answer (routing it to no_retrieval instead of local), which then caused the self-verification layer to spend many follow-up retrieval passes trying to find evidence for ungrounded claims. The router's confidence threshold or classifier prompt likely needs tuning against a labeled query set rather than relying on the LLM's zero-shot classification.
• No CLI or API surface yet. Indexing and querying currently only go through scripts/build_graph.py and the Streamlit UI. A typer CLI and a FastAPI service (already anticipated in the code's layered architecture — pipelines are decoupled from the UI) are natural next steps for programmatic or production use.
• Cross-encoder reranking is optional and untested at scale. It requires the sentence-transformers extra and hasn't been evaluated for whether it changes ranking quality on this project's corpora versus the fusion ranker alone.
• Single-provider LLM testing. Development and evaluation were done against Gemini; the OpenAI and HuggingFace provider adapters exist in the codebase but haven't been run through the same evaluation.

License

MIT — see LICENSE.