A Saliency-Based Patch Sampling Approach for Deep Artistic Media Recognition

• Main Authors: Heekyung Yang, Kyungha Min
• Format: Article
• Language: English
• Published: MDPI AG 2021-04-01
• Series: Electronics
• Subjects: media recognition; CNN; saliency; patch sampling
• Online Access: https://www.mdpi.com/2079-9292/10/9/1053

We present a saliency-based patch sampling strategy for recognizing artistic media from artwork images using a deep media recognition model, which is composed of several deep convolutional neural network-based recognition modules. The decisions from the individual modules are merged into the final decision of the model. To sample a suitable patch for the input of the module, we devise a strategy that samples patches with high probabilities of containing distinctive media stroke patterns for artistic media without distortion, as media stroke patterns are key for media recognition. We design this strategy by collecting human-selected ground truth patches and analyzing the distribution of the saliency values of the patches. From this analysis, we build a strategy that samples patches that have a high probability of containing media stroke patterns. We prove that our strategy shows best performance among the existing patch sampling strategies and that our strategy shows a consistent recognition and confusion pattern with the existing strategies.