Guide

RAG No-Answer Testing

A practical guide to testing whether a RAG system refuses unsupported, missing-source, ambiguous, stale, or out-of-policy questions safely today.

RAG no-answer testing checks whether the system refuses or escalates when retrieved evidence does not support an answer. This behavior is as important as answer quality. A RAG system that always answers will eventually convert missing evidence into confident misinformation.

The correct response is not always “I do not know.” A safe no-answer response should say what evidence is missing, avoid unsupported claims, and offer a next step when appropriate. It should help the user without pretending the corpus contains an answer.

Use this guide with the RAG Evaluation Checklist and How to Reduce Hallucinations in LLM Apps when designing launch gates.

Define no-answer categories

Not all no-answer cases are the same. Define the categories before writing fixtures.

Common categories:

• Missing source: no retrieved document answers the question.
• Partial source: the source supports only part of the requested answer.
• Ambiguous source: multiple interpretations are possible.
• Conflicting source: sources disagree.
• Out-of-policy: the system is not allowed to answer even if it has related text.
• Time-sensitive: the corpus may be stale.
• Private data: the answer would require data outside the approved context.

Each category needs expected behavior. Missing source may require refusal. Partial source may require a narrow answer plus caveat. Conflicting source may require surfacing the conflict.

No-answer categories should be visible in review notes and product requirements. If a product owner expects the assistant to answer time-sensitive pricing questions but the corpus is refreshed monthly, that mismatch should be resolved before launch.

Write no-answer fixtures

Fixtures should be concrete. Avoid vague prompts such as “ask something unsupported.” Write the exact user question, the corpus state, expected response, and severity.

Example fixture shape:

• Question: asks for a policy exception that is not present.
• Retrieved evidence: related policy page, but no exception.
• Expected behavior: say the source does not state the exception and route to human review.
• Failure severity: high if the system invents approval.

Include adversarial wording. Users often pressure the system with “just answer from your general knowledge” or “assume the policy allows it.” The system should preserve the evidence boundary.

Evaluate retrieval and generation separately

If no relevant source is retrieved, the retrieval layer failed. If sources are retrieved but the answer invents a claim, the generation or instruction layer failed. If the system knows evidence is missing but still takes an action, the workflow policy failed.

Separate scoring keeps fixes clear. A prompt change may not solve a missing-index problem. Better chunking may not solve a model that ignores refusal rules.

Keep the raw retrieved sources with the failed fixture. Without them, the team may argue about whether the model had enough evidence. With them, the fix path is easier to choose.

The AI Hallucination Testing Guide can help classify the failure as retrieval, prompt, synthesis, policy, or review.

Design useful refusal copy

A refusal should not be a dead end when the workflow can offer help. Good no-answer copy includes:

• The available sources do not answer the question.
• The closest available evidence, if relevant.
• The missing information needed for an answer.
• A safe next step such as checking a source, asking an owner, or escalating.

Avoid long apologies and avoid hidden speculation. The answer should not smuggle in an unsupported conclusion after saying evidence is missing.

Retest after corpus changes

No-answer behavior can regress when documents are added, chunking changes, retrieval parameters move, or prompts are edited. Keep the fixtures and rerun them after meaningful changes.

If a formerly unsupported question becomes supported because a new source was added, update the expected behavior and record the change. The goal is accurate evidence handling, not permanent refusal.

Verification checklist

Before launch, confirm:

• No-answer categories are defined.
• Fixtures include missing, partial, ambiguous, and out-of-policy cases.
• Expected behavior is written before model output is reviewed.
• Refusals state what evidence is missing.
• The system does not cite unrelated sources as proof.
• Escalation paths are clear for high-impact cases.
• Retest triggers are documented.

FAQ

What is RAG no-answer testing?

RAG no-answer testing checks whether the system refuses or escalates when retrieved evidence does not support an answer.

What should a safe no-answer response say?

A safe no-answer response should say what evidence is missing, avoid unsupported claims, and offer a next step when appropriate.