Automated test case reduction in query specific language(s)

Abstract

Database testing tools like SQLsmith and SQLancer generate lengthy test cases to identify several categories of database bugs. While these tools are effective in identifying issues, usually the resulting test is large and complex, making it difficult to debug. Over the years, several methods have been proposed, with delta debugging being one of the traditional approaches. Though widely used for reducing test cases in imperative languages it may not be optimal for SQL, where statements can be executed independently and provide additional information about those individual statements, like their execution time. In this study, we present enhanced approaches to delta debugging, specifically tailored to leverage the execution times of SQL statements to guide the reduction process. We introduce two strategies: block time analysis, which considers the collective execution time of statement groups, and individual time analysis, which estimates and uses per-statement timing information. And two simplified approaches based on percentage removal which aims to be a preprocessing step to delta debugging rather than an alternative to it. All of these methods seek to increase the convergence toward minimal bug-reproducing test cases by prioritizing the removal of time-intensive statements. The study provides a view into various variations of delta debugging with timing context integrated into it. It also provides insight into which of the methods works well and which of the strategies fails to match the efficiency and effectiveness of delta debugging.