Testing database management systems (DBMSs) is a complex task. The vagueness and intricacy of the SQL specification make it difficult to model the semantics of queries, making it challenging to test the correctness and performance of DBMSs. Traditional approaches, such as metamorphic testing, require precise modeling of the SQL specification to create different inputs with equivalent semantics. This process can be labor-intensive and error-prone. To address this, we propose Mozi, a framework that finds DBMS bugs via configuration-based equivalent transformation. The key idea behind Mozi is to compare the results of equivalent DBMSs with different configurations, rather than between semantically equivalent queries. The framework involves analyzing the query plan, changing configurations to transform the DBMS to an equivalent one, and re-executing the query to compare the results using various test oracles. For example, detecting differences in query results indicates correctness bugs, while observing faster execution times on the optimization-closed DBMS suggests performance bugs. We demonstrate the effectiveness of Mozi by evaluating it on four widely used DBMSs, namely MySQL, MariaDB, Clickhouse, and PostgreSQL. In the continuous fuzzing, Mozi totally found 101 previously unknown bugs, including 49 correctness and 52 performance bugs in four DBMSs. Among them, 90 bugs are confirmed and 57 bugs have been fixed. In addition, Mozi can be extended to other DBMS fuzzers for testing various types of bugs. With Mozi, testing DBMSs becomes simpler and more effective, potentially saving time and effort that would otherwise be spent on precisely modeling SQL specifications for testing purposes.
Bio: Jie Liang received the B.S. degree in computer science from Beijing University of Posts and Telecommunications, Beijing, China, in 2017, and the Ph.D. degree in software engineering from Tsinghua University, Beijing, China, in 2022. He is currently working as a Postdoc researcher in school of software at Tsinghua University. His research interests include program analysis, DBMS testing, and their applications to industry.