Improved Surprise Adequacy Tools for Corner Case Data Description and Detection

• Main Authors: Tinghui Ouyang, Vicent Sanz Marco, Yoshinao Isobe, Hideki Asoh, Yutaka Oiwa, Yoshiki Seo
• Format: Article
• Language: English
• Published: MDPI AG 2021-07-01
• Series: Applied Sciences
• Subjects: corner case data detection; surprise adequacy; modified distanced-based SA; AI quality testing
• Online Access: https://www.mdpi.com/2076-3417/11/15/6826

Facing the increasing quantity of AI models applications, especially in life- and property-related fields, it is crucial for designers to construct safety- and security-critical systems. As a major factor affecting the safety of AI models, corner case data and its related description/detection techniques are important in the AI design phase and quality assurance. In this paper, inspired by surprise adequacy (SA), a tool having advantages on capture data behaviors, we developed three modified versions of distance-based-SA (DSA) for detecting corner cases in classification problems. Through the experiment analysis on MNIST, CIFAR, and industrial example data, the feasibility and usefulness of the proposed tools on corner case data detection are verified. Moreover, Qualitative and quantitative experiments validated that the developed DSA tools can achieve improved performance in describing corner cases’ behaviors.