Beyond the harness: End-to-end optimization of context artifacts for enterprise Text-to-SQL

Deploying LLMs for enterprise Text-to-SQL is bottlenecked less by the model than by what context reaches it: business logic spans thousands of tables, and no model can ingest a full catalog at once. We argue that the most effective place to intervene is therefore the knowledge-base context the model consumes, and that this context should be constructed from historical usage rather than tuned for as a fixed input. Using a query-DAG decomposition– the same family of intermediates that enterprise benchmarks like BEAVER annotate, here recovered from production SQL–we compare the value of oracle query graphs versus retrieved knowledge-base context. In this ablation, retrieved knowledge-base context provides the largest marginal improvement when added to the full oracle graph. Building on this, we optimize a distillation procedure that turns historical query profiles into reusable SQL reference cards. On a benchmark of 5176 production queries from a major online retailer, optimizing these context artifacts yields larger gains (∼12–25% AST similarity) than optimizing the retrieval harness (∼3– 12%). On the public BEAVER benchmark, which lacks the production usage signals available in our internal setting, the picture is more mixed: table cards alone perform about the same as raw historical SQL. The best optimized variant retrieves both cards and raw SQL, scoring 9.00% versus 6.33% (p-value 0.12) for the comparable baseline on a held-out N=300 subset, using retrieved context and harness changes but no agentic loop.