A Multi-Branch U-Net for Steel Surface Defect Type and Severity Segmentation

• Main Authors: Robby Neven, Toon Goedemé
• Format: Article
• Language: English
• Published: MDPI AG 2021-05-01
• Series: Metals
• Subjects: steel surface defects; visual inspection; computer vision; deep learning; semantic segmentation
• Online Access: https://www.mdpi.com/2075-4701/11/6/870

Automating sheet steel visual inspection can improve quality and reduce costs during its production. While many manufacturers still rely on manual or traditional inspection methods, deep learning-based approaches have proven their efficiency. In this paper, we go beyond the state-of-the-art in this domain by proposing a multi-task model that performs both pixel-based defect segmentation and severity estimation of the defects in one two-branch network. Additionally, we show how incorporation of the production process parameters improves the model’s performance. After manually constructing a real-life industrial dataset, we first implemented and trained two single-task models performing the defect segmentation and severity estimation tasks separately. Next, we compared this to a multi-task model that simultaneously performs the two tasks at hand. By combining the tasks into one model, both segmentation tasks improved by 2.5% and 3% mIoU, respectively. In the next step, we extended the multi-task model using sensor fusion with process parameters. We demonstrate that the incorporation of the process parameters resulted in a further mIoU increase of 6.8% and 2.9% for the defect segmentation and severity estimation tasks, respectively.