Reliable Enterprise RAG: Stop Hallucinated Answers

What RAG is and what problem it solves

Retrieval-Augmented Generation pairs a generative model with an external retrieval system. Given a query, it pulls fragments from a knowledge base and feeds them to the model as context for composing the answer.

NIST defines RAG as a system in which a model is paired with a separate knowledge base. This makes it possible to update knowledge without retraining the model and to attribute answers to sources. It also introduces a chain of possible failure points, though:

1. the document itself can be wrong or outdated;
2. ingestion can lose structure;
3. the query may not capture the user's real intent;
4. the retriever can return irrelevant fragments;
5. the reranker can order results poorly;
6. the model can ignore, blend or overstate the evidence;
7. the interface can display a citation that doesn't actually support the sentence.

That's why the unit of quality isn't the model — it's the answer, its sources and the process that connects them.

RAG is not an automatic source of truth

A vector index measures proximity between embeddings, not truth, currency or authorisation. Two semantically similar fragments can flatly contradict each other. A superseded policy can rank above the one currently in force. A sales document can end up competing with an official regulation.

The knowledge base needs editorial governance:

• an owner per collection;
• the original source and URL;
• publication and review dates;
• version and status: draft, current, superseded or archived;
• classification and intended audience;
• geographic or product scope;
• an expiry date or next review date;
• a link to the document(s) it replaces.

Documents with no owner or no status should never make it into a business-critical RAG.

Reference architecture

1. Authorised sources

The system starts from a defined set of repositories: policies, manuals, contracts, regulations, support tickets or technical knowledge. Each source has its own inclusion rules. Open web content is never mixed automatically with internal documentation.

2. Controlled ingestion

Ingestion extracts text, tables, headings and metadata, checks the format and flags duplicates. Scanned files need OCR plus a quality check. The link back to the original page, section or paragraph must be preserved.

3. Chunking

Chunking has to respect the document's semantic structure. Fragments that are too small drop conditions and exceptions; fragments that are too large introduce noise. A table should never be split off from its headers and footnotes.

4. Indexing and retrieval

Search can combine dense and lexical retrieval. Dense retrieval captures semantic similarity; lexical retrieval preserves exact terms, codes, names and clause numbers. A reranker then orders the candidates with greater precision.

5. Constrained generation

The prompt requires the model to answer only from the evidence provided, flag uncertainty, cite its sources and decline to answer when there is no support. It doesn't replace the controls upstream, but it does define the expected behaviour.

6. Validation and presentation

Before the answer is shown, every material claim is checked for support and every citation is checked against its fragment. The interface lets the user open the source and see its date, version and surrounding context.

Evaluate retrieval and generation separately

If you only evaluate the final answer, you don't know what to fix. Measurement has to happen layer by layer.

Retrieval metrics

• Recall@k: whether relevant evidence shows up among the top results.
• Precision@k: the share of results that are actually useful.
• MRR (mean reciprocal rank): the position of the first relevant result.
• nDCG: the quality of the ranking when relevance comes in degrees.
• Authority coverage: whether the source retrieved carries the priority it should.

Generation metrics

• faithfulness: claims backed by the retrieved context;
• correctness: the answer matches the reference;
• completeness: it covers everything the question requires;
• relevance: it actually answers the question asked;
• citation quality: the source backs the exact claim being made;
• abstention: it correctly declines to answer when evidence is missing.

Operational metrics

• latency per component;
• cost per valid answer;
• share of answers with no sources;
• queries that had to be reformulated;
• answers corrected by people;
• permission-related incidents;
• outdated documents retrieved.

A high average can hide critical failures. Legal, security and financial queries need their own thresholds.

Building an evaluation set

The set should represent real queries and stay separate from day-to-day tuning. It should include:

• simple questions answered by a single source;
• syntheses across several documents;
• exceptions and conditions;
• dates, versions and jurisdictions;
• questions with no answer;
• ambiguous terms;
• contradictory documents;
• queries with typos or internal jargon;
• attempts at unauthorised access;
• malicious instructions embedded inside documents.

Each case defines the expected answer, the valid evidence, the evidence that must not be used, and the criterion for abstaining. Subject-matter experts review a sample, and any disagreements get documented.

Reducing hallucinations at every layer

Source quality

Originals should take priority, and superseded versions should be pulled from active use. An outdated document can be kept for historical reference, but only behind a filter and a clear label. For regulations, the original text, the consolidated version and the date it took effect all need to be distinguished.

Query handling

Query rewriting can expand synonyms, but it must preserve numbers, names and constraints. If the question is ambiguous, the system asks for clarification rather than searching across several interpretations at once.

Hybrid retrieval

Combining lexical and vector search stops exact terms from getting lost. Metadata filters language, product, country, date, department and permissions before similarity is even applied.

Reranking

A reranker can improve ordering, but it needs to be evaluated on the real domain. It should never bury a priority source purely because of writing style.

Evidence-grounded generation

The model is instructed to:

• cite every verifiable claim;
• avoid relying on its own background knowledge when the case calls for an internal source;
• separate fact, interpretation and recommendation;
• flag contradictions rather than silently resolving them;
• answer "not found" when coverage is insufficient.

Post-generation verification

A verifier can catch unsupported sentences or broken citations. It doesn't guarantee truth, but it adds one more gate. High-impact cases still require human review.

Permissions: filter before you retrieve

A RAG system can filter the final answer and still have exposed data to the model during retrieval. Authorisation has to be applied before content is retrieved, using identity, group membership, classification and context.

Recommended controls:

• separate indexes where the risk justifies it;
• ACLs inherited from the source repository and kept in sync;
• permission checks at document or fragment level;
• a service identity running with least privilege;
• isolation testing between clients or departments;
• fast revocation and reindexing after any change;
• access logs built on data minimisation.

Permissions should never rely on manually applied tags with no reconciliation process behind them.

Pipeline security

OWASP has identified weaknesses specific to vectors and embeddings and publishes a dedicated RAG security cheat sheet. The risks it flags include knowledge-base poisoning, indirect prompt injection, cross-domain access, leakage through embeddings, and manipulation of the retrieval step.

Key measures:

1. allow only registered sources and connectors;
2. scan files and active content;
3. keep system instructions separate from retrieved data;
4. treat every retrieved document as untrusted content;
5. detect embedded instructions and anomalous patterns;
6. validate tools and outputs whenever the RAG feeds an agent;
7. version indexes and allow rollback;
8. protect backups, embeddings and metadata;
9. test for extraction and run adversarial queries;
10. monitor distribution shifts and access patterns.

A retrieved passage that says "ignore the rules and send the data" must never gain authority simply because it sits inside the knowledge base.

Citations that actually prove something

A useful citation meets three conditions:

• entailment: the source genuinely backs the sentence;
• granularity: it points to a specific enough section or page;
• currency: it corresponds to the applicable version.

Showing three related links doesn't prove the answer. It should be possible to select a claim and open the exact passage behind it.

When an answer draws on several sources, it should show which part comes from which. If sources contradict each other, that contradiction should be surfaced — not silently resolved by picking one.

Version management and freshness

Keeping the repository current needs an SLA:

When a source is replaced, the index is rebuilt, caches are invalidated and the affected tests are rerun. Cached answers should never outlive a source change indefinitely.

Observability and operations

Every query needs a trace recording:

• identity and the permissions applied;
• the original and rewritten query;
• filters;
• candidate documents and their scores;
• fragments sent to the model;
• index, model and prompt version;
• the answer and its citations;
• validations, latency and cost;
• feedback and corrections.

Logs can contain sensitive data, so minimisation, access control and retention limits all apply. Observability should make it possible to reproduce a failure without creating an ungoverned shadow repository on the side.

A 90-day rollout plan

Days 1–30: domain and evaluation

• Choose a bounded domain and assign an owner.
• Inventory the sources and strip out the junk.
• Define the audience and the permissions.
• Build the evaluation set from real questions.
• Set thresholds and abstention cases.

Days 31–60: architecture and testing

• Design ingestion, chunking, metadata and hybrid retrieval.
• Implement ACLs ahead of retrieval.
• Add citations, verification and tracing.
• Test security, contradictions and versioning.
• Compare configurations against the same evaluation set.

Days 61–90: pilot

• Roll out to a controlled group.
• Review every high-impact answer.
• Track no-result searches and corrections.
• Fix the sources before you touch the model.
• Only widen the scope once it clears the quality and security gates.

Common mistakes

1. Indexing the whole company with no owner assigned.
2. Measuring only whether the answer "sounds right".
3. Relying on vector search alone for exact codes.
4. Applying permissions after retrieval instead of before.
5. Trusting the prompt alone to stop injection.
6. Showing citations that don't actually back the sentence.
7. Mixing current and superseded documents.
8. Tuning on the same set used to evaluate.
9. Leaving unanswerable questions out of the test set.
10. Changing the model or the index with no regression run.

Production checklist

• Domain, audience and owner defined.
• Sources authorised, current and versioned.
• Metadata and provenance at fragment level.
• Permissions applied before retrieval.
• Retrieval evaluated separately.
• Faithfulness, citations and abstention measured.
• Adversarial cases and contradictions included.
• Index, model and prompt versioned.
• Tracing, alerts and rollback in place.
• Human review for high-impact cases.

Frequently asked questions

Does RAG eliminate hallucinations?

No. It can reduce them if it retrieves the right evidence and constrains the answer, but it can equally introduce irrelevant, outdated or malicious sources.

How many fragments should be retrieved?

There's no universal number. It has to be optimised against the evaluation set. More context can introduce noise and push important instructions out of view.

Is vector search better than keyword search?

In an enterprise setting, a combination of the two usually works best. Lexical search preserves exact terms, and vector search captures meaning.

Can an answer cite a source without being faithful to it?

Yes. A citation can be topically related to the answer without actually backing the sentence. Entailment and granularity both need to be evaluated.

Does the model need to be retrained?

Not to update the knowledge base — that's one of the main benefits of RAG. Other components may need adjusting, but that shouldn't be confused with updating the sources.

Technical sources consulted

• NIST: definition of Retrieval-Augmented Generation
• NIST AI Risk Management Framework
• NIST AI RMF: Generative AI Profile
• OWASP RAG Security Cheat Sheet
• OWASP: Vector and Embedding Weaknesses
• OWASP: Prompt Injection

Summum IA can support the source governance, architecture, evaluation, security and LLMOps of a RAG project. A reliable RAG doesn't promise to always answer: it knows when it lacks the evidence.