Quantifying Bug Reporting Agreement -- Towards More Verifiable and Valid Software Testing Research

Abstract

Many software testing papers claim to discover bugs but without providing sufficient evidence. My analysis of 39 papers from top conferences revealed that only 51% provide complete bug identifiers and 28% have inaccessible artifacts. Their terminology in reporting practices is being analyzed too. In the future, I plan to develop BugAgreement, a novel metric to quantify researcher-developer consensus on bug reports, which could help enable rigorous evaluation of bug-finding techniques and promote transparency in software testing research.

Date
Apr 16, 2025 2:30 PM — 3:00 PM
Event
Weekly Talk
Location
MR25 @ COM3-02-70