Safe and sufficient caching: Per-request validated evidence reuse for production retrieval-augmented generation

Production Retrieval-Augmented Generation (RAG) systems serve repeated queries over evolving document corpora under user-specific access constraints. Standard pipelines perform full retrieval and generation for every request, wasting computation on recurring queries. Naive caching reduces latency but introduces safety failures: stale evidence from updated documents, unauthorized content crossing access boundaries, and superseded passages no longer reflecting source truth. These violations are invisible on academic benchmarks yet unacceptable in enterprise deployment.
We introduce SSC (Safe and Sufficient Caching), a validated evidence reuse framework whose premise is that cache-reuse validity in RAG is context-conditioned : admissibility depends on the requester and live source state, not on query similarity alone. SSC caches evidence chunks (not generated answers) with per-chunk provenance metadata, and treats validity as a hard constraint before reuse: access-control gating ensures user authorization, and freshness verification confirms source-document currency. Only evidence passing both checks enters a utility-based ranking stage balancing relevance, source trust, historical utility, and serving cost.
This separation of safety from efficiency is the central contribution. On WebQuestions and HotpotQA, SSC maintains full quality parity with unrestricted retrieval baselines, confirming zero degradation from evidence reuse. On a proprietary InternalQA workload with role-based access control and document versioning, it reduces end-to-end latency by 22% and retrieval cost by 34% while achieving zero invalid reuse, compared to 3.4-9.8% violation rates for standard caching and 0.3-8.2% for TTL-based invalidation. The method is training-free, retriever-agnostic, and deploys as a thin validation layer in existing RAG pipelines.