Have you ever hit a situation where your RAG system returns relevant fragments but the answer is still wrong? You ask who approved contract no. 2024-0034, and the system finds a document about approval processes and a document containing the contract number — but fails to connect that the approver was a specific person named in a third document. Each step has its answer somewhere else. The problem isn't the search; the problem is missing relationships.
GraphRAG was built precisely for this class of questions — an extension of classic retrieval-augmented generation that adds a knowledge graph. It is not a drop-in upgrade you simply switch on. It is an architectural choice with real costs and real benefits — but only for a specific class of problems. This article covers what GraphRAG solves, how it works technically, what it actually costs, and which use cases genuinely justify it.
What classic RAG cannot do
A standard RAG pipeline works well when the answer to a question sits in one or two nearby text fragments. You take the question, convert it to an embedding, find the k-nearest chunks in a vector database, feed them to the model, and get an answer.
This approach has a blind spot: relationships between entities across documents. Classic RAG has no idea that the person mentioned in document A is the same as the one in document C, that the event described in one report is the cause of the problem analysed in another, or that the supplier on an invoice is ownership-linked to the company in a complaint. Vector similarity matches text — not semantic connections.
This is exactly where GraphRAG comes in. Instead of isolated fragments, it works with a graph — nodes (entities: people, companies, products, processes, locations) and edges (relationships: approved, supplies, responsible for, caused, belongs to). Retrieval is no longer just "find similar text" but "find a path or subgraph relevant to this question".
How GraphRAG works technically
The original Microsoft implementation that popularised the term consists of two phases: indexing and querying.
Indexing phase
This is the most expensive step. For every document in the corpus, an LLM (or a series of LLM calls) is invoked to extract entities and relationships. From sentences like *"Engineer Novák approved the replacement of pump P-12 on line 3 on 14 January"*, it extracts nodes Novák (person), P-12 (equipment), Line 3 (location) and edges approved → replacement, located on → Line 3.
On top of this extracted graph, community detection is applied (algorithms such as Leiden or Louvain), which groups related entities into communities — for example, all entities related to line 3, or all transactions with a particular supplier. For each community a summary description is generated (a community report), which serves as high-level context when answering global questions.
Querying phase
When a question arrives, the system chooses between two strategies:
Local search — the question concerns specific entities or local relationships. The system finds relevant nodes and traverses the subgraph around them. Suitable for questions like "who approved this order" or "what equipment is connected to this fault".
Global search — the question requires synthesising information across the entire corpus. The system works with community reports rather than raw documents. Suitable for questions such as "what are the main fault patterns on the lines over the past year" or "identify suppliers with repeatedly late deliveries".
Compared with classic RAG, GraphRAG adds an entire layer of semantic structure — but that layer is not free.
The real cost of indexing
This is where the reality surprises many people interested in GraphRAG. Microsoft reported in their research that indexing large datasets with the original implementation cost on the order of tens of thousands of USD — depending on corpus size and the model chosen. The reason: every document (or every chunk) requires an LLM call to extract entities. With hundreds of thousands of documents, the token count quickly reaches astronomical figures.
This has a concrete implication: indexing is not a one-time task. When a document changes or a new one is added, the graph must be updated. Unlike a vector database — where re-embedding new chunks suffices — updating a graph may require re-traversing the neighbourhood of the changed node.
This is the fundamental reason GraphRAG is unsuitable for dynamic corpora with high update frequency.
LazyGraphRAG and open-source alternatives
Microsoft was aware of the cost problem and released LazyGraphRAG — a variant that defers LLM entity and relationship extraction. Instead of extracting entities for every document at indexing time, it builds a base graph using cheaper methods (NLP extraction, keyword extraction) and makes LLM calls only at query time, for the relevant subset. The result: significantly lower indexing costs with comparable answer quality for many question types.
The community has meanwhile produced several alternative implementations with different trade-offs:
- LightRAG — prioritises efficiency, fewer LLM calls during indexing, good results on technical documents
- HippoRAG — inspired by hippocampal memory, interesting for long-term knowledge retention
- Fast-GraphRAG — focused on speed while preserving graph structure
None of these alternatives is a "win on everything" solution — each makes different compromises between indexing cost, graph quality, and query speed. For production deployment we recommend testing the specific alternative on your real corpus, not on benchmark datasets.
When GraphRAG is worth it
This is the key question to ask before any adoption decision.
GraphRAG delivers real value for:
- Multi-hop questions — the answer requires connecting information from three or more documents along a chain of relationships. "Which technologist is responsible for the equipment that caused the fault on the line with the highest OEE?" — three entities, two edges, three documents.
- Impact analysis — "what processes will be affected if we switch supplier X?" — requires traversing the dependency graph.
- Cross-entity summarisation — "summarise all incidents related to product Y over the past two years" — communities in the graph handle this better than chunks from a vector database.
- Pattern detection — "identify recurring patterns in complaints by supplier and equipment category" — global search over community reports.
- Compliance and audit trails — "prove the approval chain for contract Z" — a direct task for graph traversal.
You probably do not need GraphRAG if:
- Your questions are predominantly factual and the answer sits in a single fragment ("what is the maximum temperature for pump P-12?").
- Your corpus changes daily or multiple times a day — re-indexing costs will be prohibitive.
- You have fewer than a few thousand documents — classic hybrid search with reranking handles most cases at a fraction of the cost and complexity.
- You do not have a clearly defined entity and relationship ontology — a graph extracted from unstructured text without domain context will contain a lot of noise.
The figure reported by Microsoft — GraphRAG achieving around 80 % correct answers on multi-hop questions versus around 50 % for naïve RAG — is indicative and applies to a specific type of synthetic benchmark questions. The actual improvement on your corpus may differ. Always measure on your own data.
Integration with an existing RAG stack
GraphRAG is not a replacement for a vector database — it is a complement. Modern production systems that adopt it typically run a hybrid architecture:
- 1.Vector DB (e.g.
Qdrant) for dense retrieval — fast, cheap, well-suited for factual questions. - 2.Graph layer (Neo4j, Amazon Neptune, or an in-memory graph for smaller corpora) for relational queries.
- 3.Router / orchestrator — classifies the incoming question and decides which layer to use (or combines both).
Orchestrating such a system is more complex. LangGraph (the framework for stateful agent graphs) is a natural fit for this pattern — it lets you define decision nodes that dynamically choose whether a query goes through the vector database, the graph layer, or needs both. For evaluating results, it is worth looking at RAGAS metrics — see how to evaluate RAG.
An important note on tooling: LlamaIndex has native GraphRAG indexing support in current versions, which significantly lowers the implementation barrier compared with assembling the pipeline manually.
Where we see GraphRAG in industrial practice
From deployments we handle for customers in manufacturing and engineering, GraphRAG or hybrid graph approaches prove justified in the following scenarios:
Technical documentation management — large enterprises with thousands of technical manuals, SOPs, service reports, and inspection records. A question like "what is the procedure for a fault on pump type X in ATEX zone II?" requires connecting the equipment manual, the zone's ATEX protocol, the safety SOP, and the service history. Classic RAG cannot do this reliably.
Compliance and regulatory audit trails — in regulated industries (pharma, energy, food) you need to prove chains of responsibility and approvals. A graph is the natural data structure for "who approved, when, on what basis, with reference to which regulation".
Supply chain analysis — connections between suppliers, components, complaints, and faults. "How many critical components come from suppliers that had more than 10 days of delay in the past year?" — multi-hop across four entity types.
Conversely, for internal chatbots over FAQ documentation or search in a product catalogue, GraphRAG is unnecessary complexity. See also agentic RAG architectures, where a graph layer naturally combines with agent-controlled retrieval.
Practical guide: how to start
If you have decided to try GraphRAG, we recommend an incremental approach:
- 1.Define the ontology — before any implementation, explicitly define what types of entities and relationships you want in the graph. Without this step you will get a noisy graph full of irrelevant nodes.
- 2.Start with a small corpus — 1,000–5,000 documents is enough to verify whether the graph structure fits your questions. Indexing will be manageable even with LazyGraphRAG.
- 3.Compare against a baseline — for each question type (factual, multi-hop, summarisation) measure GraphRAG accuracy versus hybrid search. If GraphRAG does not add more than 15 % on your real questions, consider whether the complexity is justified.
- 4.Plan the update cycle — decide how you will keep the graph current. Batch re-indexing once a week? Incremental update when documents are added? This decision will shape the overall architecture.
- 5.Do not forget the fallback — in a production system always keep classic vector retrieval as a backup for cases where the graph query does not return sufficiently confident results.
Frequently asked questions
Is GraphRAG better than classic RAG?
It depends on the type of questions. For multi-hop questions requiring entity connections across documents — yes, significantly. For simple factual questions where the answer is in a single fragment, GraphRAG is unnecessarily expensive and slower. Always measure on your specific use case.
How expensive is GraphRAG indexing in practice?
The original Microsoft implementation reports costs on the order of tens of thousands of USD for large corpora. LazyGraphRAG and open-source alternatives (LightRAG, Fast-GraphRAG) significantly reduce these costs. The exact price depends on corpus size, the chosen LLM, and the granularity of entity extraction.
Do I need a dedicated graph database?
Not necessarily. For smaller graphs (up to roughly millions of nodes), an in-memory representation or libraries such as networkx work fine. For larger production deployments, Neo4j, Amazon Neptune, or TigerGraph are the standard choices. Some implementations (e.g. LlamaIndex GraphRAG) abstract the storage layer and allow you to choose the backend.
Does GraphRAG work for non-English documents?
Entity extraction depends on the quality of the LLM performing it. Frontier models (Claude, GPT, Gemini) handle most languages well. When using smaller local models, verify extraction quality on a sample of your target-language text before deployment. The graph structure itself is language-agnostic.
When should you decide not to use GraphRAG?
If your corpus changes daily, if you have fewer than a few thousand documents, if your questions are predominantly single-hop, or if you lack the capacity to define and maintain an entity ontology. In these cases, classic hybrid search with reranking will give you 90 % of the result for 10 % of the cost and complexity.
*GraphRAG is not the right tool for every project — but for projects where relationships between entities determine whether the answer is correct, it has no substitute. At MP Industrial Solutions we help companies assess whether their specific use case is a GraphRAG candidate, design the ontology, and estimate real indexing costs before any commitment. If you are dealing with complex documentation or compliance questions, start with an assessment.*