HYPERPARAMETER OPTIMIZATION BASED ON A PRIORI AND A POSTERIORI KNOWLEDGE ABOUT CLASSIFICATION PROBLEM

• Main Authors: Valentina S. Smirnova, Viacheslav V. Shalamov, Valeria A. Efimova, Andrey A. Filchenkov
• Format: Article
• Language: English
• Published: Saint Petersburg National Research University of Information Technologies, Mechanics and Optics (ITMO University) 2020-12-01
• Series: Naučno-tehničeskij Vestnik Informacionnyh Tehnologij, Mehaniki i Optiki
• Subjects: machine learning; classification; hyperparameter optimization; bayesian optimization; gaussian processes
• Online Access: https://ntv.ifmo.ru/file/article/20006.pdf
• ISSN: 2226-1494; 2500-0373

Subject of Research. The paper deals with Bayesian method for hyperparameter optimization of algorithms, used in machine learning for classification problems. A comprehensive survey is carried out about using a priori and a posteriori knowledge in classification task for hyperparameter optimization quality improvement. Method. The existing Bayesian optimization algorithm for hyperparameter setting in classification problems was expanded. We proposed a target function modification calculated on the basis of hyperparameters optimized for the similar problems and a metric for determination of similarity classification problems based on generated meta-features. Main Results. Experiments carried out on the real-world datasets from OpenML database have confirmed that the proposed algorithm achieves usually significantly better performance results than the existing Bayesian optimization algorithm within a fixed time limit. Practical Relevance. The proposed algorithm can be used for hyperparameter optimization in any classification problem, for example, in medicine, image processing or chemistry.