Feature subset selection using consensus clustering

Abstract

Feature selection is an essential technique used in high dimensional data. Basically, feature selection is focused on removing irrelevant features. But, removing redundant features is also equally important. We propose a novel feature subset selection algorithm based on the idea of consensus clustering. Our algorithm constructs a complete graph on feature space and partitions the graph using various graph partitioning algorithms from social networks. Consensus clustering is applied to find the best partitioning and final feature subset is formed by selecting the most 'representative' feature that has highest correlation to target class from each cluster. Classification is used as validation and the algorithm is evaluated on benchmark data sets of dimensionality ranging between 8 to 168 features. The results show that the proposed approach is efficient in removing irrelevant and redundant features. The number of features selected using proposed method is very less and classifier accuracies using selected features are on par with the accuracies of the latest approaches proposed in the literature.