Back to blog

AI

RAG Alternatives: What to Use Instead, and When RAG Is Still Right

10 min readUpdated October 15, 2026
RAG Alternatives: What to Use Instead, and When RAG Is Still Right

People searching for a RAG alternative rarely want to abandon retrieval. They want to stop being on call for it. Something specific broke — answers went stale, the wrong document came back, a person saw a file they should not have, the bill grew faster than the usage — and the question becomes whether a different architecture fixes it or whether the current one was simply built badly.

This page is the survey, not a pitch. Six alternatives, what each one actually fixes, what it costs you, and the cases where ordinary RAG is still the right answer and swapping it out would be a mistake.

First: name the failure, because “RAG” is not one thing

“RAG” bundles at least five separate systems — a parser, a chunker, an embedding model, a vector index, and a prompt assembler. When someone says RAG is not working, it is almost never all five. Diagnose before you replace:

What you observe Where it usually is Does a different architecture help?
Right document, wrong passage chunking and reranking No — fix the pipeline
Answers cite last quarter’s policy ingestion freshness No — fix the sync
Nothing comes back for obvious questions recall, or a filter applied after ranking Sometimes
Right answer, wrong person saw it authorization Yes — this is architectural
“How many contracts renew in March?” fails retrieval cannot count Yes — the wrong tool entirely
Cost climbs with every document added embedding and storage strategy Sometimes

Two of those six are genuinely architectural. The rest are a pipeline that needs work, and no amount of replatforming will fix a bad chunker.

1. Longer context windows — put the documents in the prompt

The simplest alternative: skip retrieval, send the material with the question.

Fixes: chunking artefacts, lost cross-references, “the answer spanned two chunks”. Costs: you pay per token on every turn, latency scales with what you send, and models still attend unevenly across a long window — burying the relevant paragraph in the middle of a very long prompt is a known failure mode, not a solved one. Right when: the corpus is genuinely small (a contract, a policy set, one codebase) and stable. If everything a question could need fits comfortably and cheaply, retrieval is machinery you do not need. Wrong when: the corpus is large, changes daily, or different people may see different parts of it. You cannot put a permission boundary inside a prompt.

2. Fine-tuning — teach the model instead of retrieving

Fixes: tone, format, domain vocabulary, task shape. A fine-tuned model writes like your analysts write. Costs: it does not reliably teach facts. Retraining is the only update path, so anything that changes weekly is out. And a fine-tune cannot forget one customer’s data on request, which makes deletion obligations awkward. Right when: the problem is how the model answers. Wrong when: the problem is what it knows. These are constantly confused, and fine-tuning is the more expensive way to be wrong.

The honest framing: fine-tuning and retrieval are not competitors. They fix different halves.

3. Knowledge graphs and GraphRAG — retrieve relationships, not passages

Instead of embedding text and searching by similarity, extract entities and the relationships between them, then traverse.

Fixes: multi-hop questions. “Which suppliers are affected if this factory closes” is a graph walk; it is not a similarity search, and no amount of better chunking makes it one. Provenance is also cleaner — an edge says where it came from. Costs: extraction quality becomes your ceiling. The graph is only as good as the pipeline that built it, that pipeline is usually an LLM, and it makes mistakes that then look like facts. Building and maintaining the schema is real, ongoing work. Right when: your questions are about connections between things. Wrong when: your questions are about what a document says. A graph is a poor way to answer “what does the leave policy say about carry-over”.

Most serious deployments end up running both, which is why the framing of graph versus RAG tends to dissolve on contact with a real corpus. We wrote about that trade-off in detail in Context Layer vs Knowledge Graph vs RAG.

4. Agent memory — remember instead of retrieving

Give the agent a durable store of what it has already learned, and let it write to that store as it works.

Fixes: repetition across turns and sessions, personalisation, “I already told you this”. Costs: memory is a cache, and caches go stale silently. It also creates a new privacy surface — whatever the agent chose to remember is now stored somewhere with a retention policy somebody has to own. Right when: the same person returns repeatedly and continuity is the point. Wrong when: the ground truth lives in documents that change. Memory is a complement to retrieval, not a replacement for it.

5. Hybrid search: when exact strings matter

Not an alternative so much as an addition to vector search. Pure vector search can miss exact strings: invoice numbers, error codes, surnames, part numbers. Full-text search misses paraphrase. Trigram matching catches typos and partial names that both of the others drop.

Running all three and fusing the rankings, commonly with Reciprocal Rank Fusion, can help with exact names, codes and numbers. It is not free: on ordinary questions the extra legs can push good vector matches down, so measure fusion against vector-only on your own queries. In PostgreSQL this needs no new infrastructure: vector for the semantic leg, pg_trgm for fuzzy and partial matching, unaccent so café and cafe match, all three stock extensions on Postgres 14 or later.

Right when: your users search for exact names, codes and numbers, and you have measured it against vector-only on your own queries. Wrong when: the failure is authorization or aggregation, below.

6. Ask the system of record instead — tools, not retrieval

The most underused alternative. “How many contracts renew in March” is a query, not a search. Retrieval will hand back three plausible contracts and the model will guess a number.

Point the agent at the database or the API instead and let it ask the question properly. You get an exact answer, an auditable one, and no embedding cost.

Right when: the answer is a number, a list, a status — anything a system already knows. Wrong when: the answer is in prose nobody has ever structured.

The failure almost no alternative addresses: who is allowed to see it

Every architecture above has a permissions story that is usually bolted on afterwards: retrieve the top k, then drop what the user may not see.

That ordering is the bug. If you filter after ranking, a user with narrow access gets a short list or an empty one, because the ranking never considered their documents in the first place. The correct ordering is to compile the caller’s permissions into the query — into the SQL predicate of every retrieval leg — so ranking only ever considers rows that person could open.

This matters more than the literature suggests, and we measured why. The published benchmarks of access-filtered vector search that we know of assign permissions at random. Real permissions are clustered — a team’s documents are semantically adjacent by construction, because a team works on one subject. On 100,000 documents with synthetic permissions, holding corpus, index, queries and the number of visible documents constant and varying only the arrangement, a hybrid search filtered after a tenfold fetch recovered 0.730 of the results for a caller who sees 10% when the permissions were random, and 0.419, with 13.0% of searches returning nothing at all, when they were clustered. At 1% visibility it recovered 0.086 and 0.021. With the permissions inside the query the caller got the same top ten as a corpus holding only their documents, under either layout. That is agreement with such a corpus, not relevance, and the vector leg is exact at or below 50,000 visible chunks.

The mechanism is where the caller’s documents sit: when they are scattered, a longer list holds proportionally more of them, and when they are concentrated in one region, a query about anything else walks through other people’s documents first. Every table is on the benchmarks page, and the full method and the caveats are in Your ACL benchmark is measuring the easy case.

If your retrieval serves more than one group of people, this is the alternative that matters, and it is not on most comparison lists.

Choosing, in one table

If the problem is… Reach for
Chunk boundaries, missed cross-references, small corpus Longer context
Tone, format, domain language Fine-tuning
Multi-hop, “what connects these” Knowledge graph
Continuity across sessions Agent memory
Exact strings, typos, paraphrase all failing Hybrid retrieval (FTS + trigram + vector)
Counts, lists, status, anything a system knows Tool calls to the source
Different people must get different answers Permission-aware retrieval
One team’s pipeline nobody owns A managed context layer

Most real systems need three or four of these at once. The useful question is not “what replaces RAG” but “which of these am I missing”.

Where Promptev fits

promptev is a context layer: it searches your documents and connected systems, returns only what each caller may see rather than filtering afterwards, can keep the knowledge in your own database with your own models, and gives agents governed tools (with approvals and an audit trail) so they can call a system of record when retrieval is the wrong instrument.

Concretely, that means the last three rows of the table above, in one place: permission-aware retrieval, tool calls to the source, and a pipeline you are not maintaining. The retrieval engine underneath is also yours to call on its own, from the Python and JavaScript SDKs, the API or MCP, so you can adopt that part alone and keep everything else.

What it does not do: it is not a fine-tuning platform, and it will not make a bad chunking strategy good. If your problem is on the top half of that table, fix the pipeline — that is cheaper than replatforming and it is the honest answer.

How promptev compares with other tools

If you are choosing a product rather than an architecture, these pages put promptev next to the tools people most often weigh it against: where each one is the better choice, where promptev differs, and a table to decide with.

All of them are on the comparisons page.

Frequently asked questions

What is the best alternative to RAG?

There is no single one, because “RAG” bundles several systems and the right replacement depends on which part failed. Longer context replaces chunking, knowledge graphs replace similarity search for relationship questions, tool calls replace retrieval entirely for anything a database already knows, and permission-aware retrieval replaces the post-filter that leaks or starves. Diagnose the failure first.

Is RAG dead in 2026?

No, and the framing is misleading. What has changed is that plain top-k vector search over naively chunked text is no longer a serious architecture on its own. Retrieval is still how you ground a model in material that changes — it just needs hybrid search, real authorization and tools alongside it.

Do longer context windows make RAG unnecessary?

Only for small, stable corpora. Cost scales with every token on every turn, attention across a very long window remains uneven, and a prompt cannot enforce who may see what. For a single contract, send the contract. For a company, retrieve.

What is the difference between RAG and a context layer?

RAG is a technique: retrieve passages, put them in the prompt. A context layer is the system around it — ingestion, hybrid retrieval, permissions, tools and governance — so retrieval is one part rather than the whole thing.

Can I replace RAG with fine-tuning?

Not for facts. Fine-tuning changes how a model answers, not what it knows, it cannot be updated without retraining, and it cannot unlearn one customer’s data on request. Use it for tone and task shape, alongside retrieval rather than instead of it.

How do I stop retrieval returning documents a user should not see?

Apply the permission check inside the retrieval query, not to the results. Filtering after ranking either leaks or starves — and because real permissions are topically clustered, the starving case is far worse than random-permission benchmarks suggest.