Quadratic Mutual Information Feature Selection
We propose a novel feature selection method based on quadratic mutual information which has its roots in Cauchy–Schwarz divergence and Renyi entropy. The method uses the direct estimation of quadratic mutual information from data samples using Gaussian kernel functions, and can detect second order n...
Main Authors: | , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2017-04-01
|
Series: | Entropy |
Subjects: | |
Online Access: | http://www.mdpi.com/1099-4300/19/4/157 |
id |
doaj-8d4ea9e79c4f4a14ab37950ed8a80c26 |
---|---|
record_format |
Article |
spelling |
doaj-8d4ea9e79c4f4a14ab37950ed8a80c262020-11-24T20:54:57ZengMDPI AGEntropy1099-43002017-04-0119415710.3390/e19040157e19040157Quadratic Mutual Information Feature SelectionDavor Sluga0Uroš Lotrič1University of Ljubljana, Faculty of Computer and Information Science, Ljubljana 1000, SloveniaUniversity of Ljubljana, Faculty of Computer and Information Science, Ljubljana 1000, SloveniaWe propose a novel feature selection method based on quadratic mutual information which has its roots in Cauchy–Schwarz divergence and Renyi entropy. The method uses the direct estimation of quadratic mutual information from data samples using Gaussian kernel functions, and can detect second order non-linear relations. Its main advantages are: (i) unified analysis of discrete and continuous data, excluding any discretization; and (ii) its parameter-free design. The effectiveness of the proposed method is demonstrated through an extensive comparison with mutual information feature selection (MIFS), minimum redundancy maximum relevance (MRMR), and joint mutual information (JMI) on classification and regression problem domains. The experiments show that proposed method performs comparably to the other methods when applied to classification problems, except it is considerably faster. In the case of regression, it compares favourably to the others, but is slower.http://www.mdpi.com/1099-4300/19/4/157feature selectioninformation-theoretic measuresquadratic mutual informationCauchy–Schwarz divergence |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Davor Sluga Uroš Lotrič |
spellingShingle |
Davor Sluga Uroš Lotrič Quadratic Mutual Information Feature Selection Entropy feature selection information-theoretic measures quadratic mutual information Cauchy–Schwarz divergence |
author_facet |
Davor Sluga Uroš Lotrič |
author_sort |
Davor Sluga |
title |
Quadratic Mutual Information Feature Selection |
title_short |
Quadratic Mutual Information Feature Selection |
title_full |
Quadratic Mutual Information Feature Selection |
title_fullStr |
Quadratic Mutual Information Feature Selection |
title_full_unstemmed |
Quadratic Mutual Information Feature Selection |
title_sort |
quadratic mutual information feature selection |
publisher |
MDPI AG |
series |
Entropy |
issn |
1099-4300 |
publishDate |
2017-04-01 |
description |
We propose a novel feature selection method based on quadratic mutual information which has its roots in Cauchy–Schwarz divergence and Renyi entropy. The method uses the direct estimation of quadratic mutual information from data samples using Gaussian kernel functions, and can detect second order non-linear relations. Its main advantages are: (i) unified analysis of discrete and continuous data, excluding any discretization; and (ii) its parameter-free design. The effectiveness of the proposed method is demonstrated through an extensive comparison with mutual information feature selection (MIFS), minimum redundancy maximum relevance (MRMR), and joint mutual information (JMI) on classification and regression problem domains. The experiments show that proposed method performs comparably to the other methods when applied to classification problems, except it is considerably faster. In the case of regression, it compares favourably to the others, but is slower. |
topic |
feature selection information-theoretic measures quadratic mutual information Cauchy–Schwarz divergence |
url |
http://www.mdpi.com/1099-4300/19/4/157 |
work_keys_str_mv |
AT davorsluga quadraticmutualinformationfeatureselection AT uroslotric quadraticmutualinformationfeatureselection |
_version_ |
1716793209837846528 |