Deep Self-Supervised Diversity Promoting Learning on Hierarchical Hyperspheres for Regularization
In this paper, we propose a novel approach to enhance the generalization performance of deep neural networks. Our method employs a hierarchical hypersphere-based constraint that organizes weight vectors hierarchically based on observed data. By diversifying the parameter space of hyperplanes in the...
| الحاوية / القاعدة: | IEEE Access |
|---|---|
| المؤلفون الرئيسيون: | , , , , , |
| التنسيق: | مقال |
| اللغة: | الإنجليزية |
| منشور في: |
IEEE
2023-01-01
|
| الموضوعات: | |
| الوصول للمادة أونلاين: | https://ieeexplore.ieee.org/document/10373009/ |
| _version_ | 1850549200969793536 |
|---|---|
| author | Youngsung Kim Yoonsuk Hyun Jae-Joon Han Eunho Yang Sung Ju Hwang Jinwoo Shin |
| author_facet | Youngsung Kim Yoonsuk Hyun Jae-Joon Han Eunho Yang Sung Ju Hwang Jinwoo Shin |
| author_sort | Youngsung Kim |
| collection | DOAJ |
| container_title | IEEE Access |
| description | In this paper, we propose a novel approach to enhance the generalization performance of deep neural networks. Our method employs a hierarchical hypersphere-based constraint that organizes weight vectors hierarchically based on observed data. By diversifying the parameter space of hyperplanes in the classification layer, we aim to encourage discriminative generalization. We introduce a self-supervised grouping method designed to unveil hierarchical structures in scenarios with unknown hierarchy information. To maximize distances between weight vectors on multiple hyperspheres, we propose a novel metric that combines discrete and continuous measures. This regularization encourages diverse orientations, consequently leading to improved generalization. Extensive evaluations on datasets, including CUB200-2011, Stanford-Cars, CIFAR-100, and TinyImageNet, consistently demonstrate enhancements in classification performance compared to baseline settings. |
| format | Article |
| id | doaj-art-14d135cf1c8c48eaaefe92b2c78f8a80 |
| institution | Directory of Open Access Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| spelling | doaj-art-14d135cf1c8c48eaaefe92b2c78f8a802025-08-19T22:36:29ZengIEEEIEEE Access2169-35362023-01-011114620814622210.1109/ACCESS.2023.334643010373009Deep Self-Supervised Diversity Promoting Learning on Hierarchical Hyperspheres for RegularizationYoungsung Kim0https://orcid.org/0009-0001-7420-129XYoonsuk Hyun1https://orcid.org/0000-0001-5047-7139Jae-Joon Han2Eunho Yang3Sung Ju Hwang4Jinwoo Shin5https://orcid.org/0000-0003-4313-4669Department of Artificial Intelligence, Inha University, Incheon, South KoreaDepartment of Mathematics, Inha University, Incheon, South KoreaSamsung Advanced Institute of Technology (SAIT), Suwon, South KoreaKorea Advanced Institute of Science and Technology (KAIST), Daejeon, South KoreaKorea Advanced Institute of Science and Technology (KAIST), Daejeon, South KoreaKorea Advanced Institute of Science and Technology (KAIST), Daejeon, South KoreaIn this paper, we propose a novel approach to enhance the generalization performance of deep neural networks. Our method employs a hierarchical hypersphere-based constraint that organizes weight vectors hierarchically based on observed data. By diversifying the parameter space of hyperplanes in the classification layer, we aim to encourage discriminative generalization. We introduce a self-supervised grouping method designed to unveil hierarchical structures in scenarios with unknown hierarchy information. To maximize distances between weight vectors on multiple hyperspheres, we propose a novel metric that combines discrete and continuous measures. This regularization encourages diverse orientations, consequently leading to improved generalization. Extensive evaluations on datasets, including CUB200-2011, Stanford-Cars, CIFAR-100, and TinyImageNet, consistently demonstrate enhancements in classification performance compared to baseline settings.https://ieeexplore.ieee.org/document/10373009/Diversity promotinghierarchical hyperspheresinductive biasregularization |
| spellingShingle | Youngsung Kim Yoonsuk Hyun Jae-Joon Han Eunho Yang Sung Ju Hwang Jinwoo Shin Deep Self-Supervised Diversity Promoting Learning on Hierarchical Hyperspheres for Regularization Diversity promoting hierarchical hyperspheres inductive bias regularization |
| title | Deep Self-Supervised Diversity Promoting Learning on Hierarchical Hyperspheres for Regularization |
| title_full | Deep Self-Supervised Diversity Promoting Learning on Hierarchical Hyperspheres for Regularization |
| title_fullStr | Deep Self-Supervised Diversity Promoting Learning on Hierarchical Hyperspheres for Regularization |
| title_full_unstemmed | Deep Self-Supervised Diversity Promoting Learning on Hierarchical Hyperspheres for Regularization |
| title_short | Deep Self-Supervised Diversity Promoting Learning on Hierarchical Hyperspheres for Regularization |
| title_sort | deep self supervised diversity promoting learning on hierarchical hyperspheres for regularization |
| topic | Diversity promoting hierarchical hyperspheres inductive bias regularization |
| url | https://ieeexplore.ieee.org/document/10373009/ |
| work_keys_str_mv | AT youngsungkim deepselfsuperviseddiversitypromotinglearningonhierarchicalhyperspheresforregularization AT yoonsukhyun deepselfsuperviseddiversitypromotinglearningonhierarchicalhyperspheresforregularization AT jaejoonhan deepselfsuperviseddiversitypromotinglearningonhierarchicalhyperspheresforregularization AT eunhoyang deepselfsuperviseddiversitypromotinglearningonhierarchicalhyperspheresforregularization AT sungjuhwang deepselfsuperviseddiversitypromotinglearningonhierarchicalhyperspheresforregularization AT jinwooshin deepselfsuperviseddiversitypromotinglearningonhierarchicalhyperspheresforregularization |