The hardest part of checking whether a RAG change actually helped
A retrieval-eval practitioner shares a telling find: the app answered correctly, but the citations and logs couldn't prove which passages supported the claims - a correct answer hiding an evidence gap. The thread asks what actually proves a retrieval change helped, beyond answer-only metrics.