TL;DR
An AI answer can sound right while citing an old policy. Name who approves each source, when it must be checked again, and how readers can trace the answer back to it.
What the paper develops
A sponsor tests an AI assistant before staff use it at work. It gives a clear answer and cites a policy page. Then the team finds out that the page was replaced six months ago. The assistant found the page and used it correctly. The answer was still wrong.
Staff may trust that answer because it sounds right. As more people use the tool, wrong answers can shape decisions before anyone finds the old page. A shared knowledge base needs an owner who can say which sources still apply.
Approve the source before AI uses it
A study of AI systems that retrieve documents before answering found many data problems in the sources and the process that loads them. The researchers interviewed practitioners; they did not measure how much each defect changes an answer. Their work shows why source quality deserves an early check.
Better search or a new model may help the tool find and explain a page. Neither can decide which of two conflicting policies the company accepts. A source owner must decide which policy applies before the page enters the trusted set.
Use five checks. Name the owner. Check quality. Set a date or event for review. Limit who may read or change the page. Mark it as under review, current, replaced, or removed. Reject a page while a key question has no answer or an important conflict remains.
Keep a path from each answer to its approved page. Record the owner, last check, reason for approval, and any replacement. This lets a reviewer see where an answer came from and whether its source still applies.
Start where a wrong answer matters
Choose one area where a wrong answer could change an approval, access decision, or customer promise. Approve its pages, then ask real questions and trace each answer back to a current source. Expand when the owner and reviewers can keep that link sound.
Source checks cannot make every AI answer correct. The assistant must still find and use the right page. The checks settle the business decisions the technology cannot make: which page counts, who may change it, and when it must be checked again.
What to do next
Choose one area where a wrong answer would matter. Have a named owner approve its sources, then test real questions against those pages.
Inside the white paper
- Why an old source can survive a successful AI search
- Five checks before AI is allowed to use a page
- How to trace an answer back to the approved source
Sources and notes
- Leopold Müller, Joshua Holstein, Sarah Bause, Gerhard Satzger, and Niklas Kühl, "Data Quality Challenges in Retrieval-Augmented Generation," arXiv:2510.00552, submitted October 1, 2025 — A study based on practitioner interviews found data-quality problems in source and loading stages, then later in the answer path; it did not measure the effect of each defect.
- Chloe Autio, Reva Schwartz, Jesse Dunietz, Shomik Jain, Martin Stanley, Elham Tabassi, Patrick Hall, and Kamie Roberts, "Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile," NIST AI 600-1, July 26, 2024 — NIST's Generative AI Profile addresses information integrity and records of where content came from.
- National Institute of Standards and Technology, "AI Risk Management Framework Core," AI RMF 1.0, 2023 — NIST's AI Risk Management Framework Core describes govern, map, measure, and manage functions.
- International Organization for Standardization, "ISO 30401:2018 — Knowledge management systems — Requirements," November 2018 — ISO 30401 sets requirements for knowledge management systems.
- National Aeronautics and Space Administration, "Knowledge Management," APPEL Knowledge Services — NASA describes its approach to preserving and using critical knowledge.