More Web Proxy on the site http://driver.im/

Article

Efficient spectral feature selection with minimum redundancy

Authors:

Huan LiuAuthors Info & Claims

AAAI'10: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence

Pages 673 - 678

Published: 11 July 2010 Publication History

Abstract

Spectral feature selection identifies relevant features by measuring their capability of preserving sample similarity. It provides a powerful framework for both supervised and unsupervised feature selection, and has been proven to be effective in many real-world applications. One common drawback associated with most existing spectral feature selection algorithms is that they evaluate features individually and cannot identify redundant features. Since redundant features can have significant adverse effect on learning performance, it is necessary to address this limitation for spectral feature selection. To this end, we propose a novel spectral feature selection algorithm to handle feature redundancy, adopting an embedded model. The algorithm is derived from a formulation based on a sparse multi-output regression with a L_2,1-norm constraint. We conduct theoretical analysis on the properties of its optimal solutions, paving the way for designing an efficient path-following solver. Extensive experiments show that the proposed algorithm can do well in both selecting relevant features and removing redundancy.

References

[1]

Appice, A.; Ceci, M.; and et al. 2004. Redundant feature elimination for multi-class problems. In ICML.

[2]

Argyriou, A.; Evgeniou, T.; and Pontil, M. 2008. Convex multi-task feature learning. Mach. Learning 73:243-272.

[3]

Boyd, S., and Vandenberghe, L. 2004. Convex Optimization. Cambridge University Press.

[4]

Ding, C., and Peng, H. 2003. Minimum redundancy feature selection from microarray gene expression data. In CSB'03, 523-529.

[5]

Duangsoithong, R. 2009. Relevant and redundant feature analysis with ensemble classification. In ICAPR '09.

[6]

Duda, R.; Hart, P.; and Stork, D. 2001. Pattern Classification. John Wiley & Sons, New York, 2 edition.

[7]

Gabrilovich, E., and et al. 2004. Text categorization with many redundant features: using aggressive feature selection to make svms competitive with c4.5. In ICML'04. Guyon, I., and Elisseeff, A. 2003. An introduction to variable and feature selection. JMLR 3:1157-1182.

[8]

Hall, M. 1999. Correlation Based Feature Selection for Machine Learning. Ph.D. Dissertation, University of Waikato, Dept. of Computer Science.

[9]

Hastie, T.; Tibshirani, R.; and Friedman, J. 2001. The Elements of Statistical Learning. Springer.

[10]

He, X.; Cai, D.; and Niyogi, P. 2005. Laplacian score for feature selection. In NIPS.

[11]

Kearns, M., and Vazirani, U. 1994. An Introduction to Computational Learning Theory. The MIT Press.

[12]

Liu, H., and Motoda, H. 1998. Feature Selection for Knowledge Discovery and Data Mining. Boston: Kluwer Academic Publishers.

[13]

Liu, J.; Ji, S.; and Ye, J. 2009. Multi-task feature learning via efficient l2,1-norm minimization. In UAI'09.

[14]

Nie, F.; Xiang, S.; Jia, Y.; Zhang, C.; and Yan, S. 2008. Trace ratio criterion for feature selection. In AAAI.

[15]

Obozinski, G.; Wainwright, M. J.; and Jordan, M. I. 2008. Highdimensional union support recovery in multivariate regression. In Neural Information Processing Systems.

[16]

Seeger, M. 2008. Bayesian inference and optimal design for the sparse linear model. JMLR 9:759-813.

[17]

Sikonja, M. R., and Kononenko, I. 2003. Theoretical and empirical analysis of Relief and ReliefF. Machine Learning 53:23-69.

[18]

Song, L.; Smola, A.; Gretton, A.; Borgwardt, K.; and Bedo, J. 2007. Supervised feature selection via dependence estimation. In ICML.

[19]

Sun, L.; Ji, S.; and Ye, J. 2009. A least squares formulation for a class of generalized eigenvalue problems in machine learning. In ICML.

[20]

Suykens, J., and Vandewalle, J. 1999. Least squares support vector machine classifiers. Neural Processing Letters 9(3):1370-4621.

[21]

von Luxburg, U. 2007. A tutorial on spectral clustering. Tech. report, Max Planck Inst. for Biological Cybernetics.

[22]

Weston, J.; Elisseff, A.; Schoelkopf, B.; and Tipping, M. 2003. Use of the zero norm with linear models and kernel methods. JMLR 3:1439-1461.

[23]

Yu, L., and Liu, H. 2004. Efficient feature selection via analysis of relevance and redundancy. JMLR 5:1205-1224.

[24]

Zhao, Z., and Liu, H. 2007. Spectral feature selection for supervised and unsupervised learning. In ICML.

Cited By

Dong XZhu LSong XLi JCheng Z(2018)Adaptive collaborative similarity learning for unsupervised multi-view feature selectionProceedings of the 27th International Joint Conference on Artificial Intelligence10.5555/3304889.3304946(2064-2070)Online publication date: 13-Jul-2018
https://dl.acm.org/doi/10.5555/3304889.3304946
Fang YLi YLei CLi YDeng X(2018)Hypergraph expressing low-rank feature selection algorithmMultimedia Tools and Applications10.5555/3288251.328828977:22(29551-29572)Online publication date: 1-Nov-2018
https://dl.acm.org/doi/10.5555/3288251.3288289
Tang CZhu XChen JWang PLiu XTian J(2018)Robust graph regularized unsupervised feature selectionExpert Systems with Applications: An International Journal10.1016/j.eswa.2017.11.05396:C(64-76)Online publication date: 15-Apr-2018
https://dl.acm.org/doi/10.1016/j.eswa.2017.11.053
Show More Cited By

Efficient spectral feature selection with minimum redundancy
1. Computing methodologies
  1. Machine learning
    1. Machine learning algorithms

Recommendations

Feature Selection via Global Redundancy Minimization
Feature selection has been an important research topic in data mining, because the real data sets often have high-dimensional features, such as the bioinformatics and text mining applications. Many existing filter feature selection methods rank features ...
Unsupervised Feature Selection with Controlled Redundancy (UFeSCoR)
Features selected by a supervised/ unsupervised technique often include redundant or correlated features. While use of correlated features may result in an increase in the design and decision making cost, removing redundancy completely can make the system ...
Efficient Feature Selection via Analysis of Relevance and Redundancy

Feature selection is applied to reduce the number of features in many applications where data has hundreds or thousands of features. Existing feature selection methods mainly focus on finding relevant features. In this paper, we show that feature ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

AAAI'10: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence

July 2010

1970 pages

Publisher

AAAI Press

Publication History

Published: 11 July 2010

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dong XZhu LSong XLi JCheng Z(2018)Adaptive collaborative similarity learning for unsupervised multi-view feature selectionProceedings of the 27th International Joint Conference on Artificial Intelligence10.5555/3304889.3304946(2064-2070)Online publication date: 13-Jul-2018
https://dl.acm.org/doi/10.5555/3304889.3304946
Fang YLi YLei CLi YDeng X(2018)Hypergraph expressing low-rank feature selection algorithmMultimedia Tools and Applications10.5555/3288251.328828977:22(29551-29572)Online publication date: 1-Nov-2018
https://dl.acm.org/doi/10.5555/3288251.3288289
Tang CZhu XChen JWang PLiu XTian J(2018)Robust graph regularized unsupervised feature selectionExpert Systems with Applications: An International Journal10.1016/j.eswa.2017.11.05396:C(64-76)Online publication date: 15-Apr-2018
https://dl.acm.org/doi/10.1016/j.eswa.2017.11.053
Liu MXu CLuo YXu CWen YTao DSingh SMarkovitch S(2017)Cost-sensitive feature selection via F-measure optimization reductionProceedings of the Thirty-First AAAI Conference on Artificial Intelligence10.5555/3298483.3298564(2252-2258)Online publication date: 4-Feb-2017
https://dl.acm.org/doi/10.5555/3298483.3298564
Li JCheng KWang SMorstatter FTrevino RTang JLiu H(2017)Feature SelectionACM Computing Surveys10.1145/313662550:6(1-45)Online publication date: 6-Dec-2017
https://dl.acm.org/doi/10.1145/3136625
Imangaliyev SLevin EHaspel NCowen LShehu AKahveci TPozzi G(2017)Unsupervised Multi-View Feature Selection for Tumor Subtype IdentificationProceedings of the 8th ACM International Conference on Bioinformatics, Computational Biology,and Health Informatics10.1145/3107411.3107413(491-499)Online publication date: 20-Aug-2017
https://dl.acm.org/doi/10.1145/3107411.3107413
Murthy C(2017)Bridging Feature Selection and ExtractionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2016.261971229:4(757-770)Online publication date: 1-Apr-2017
https://dl.acm.org/doi/10.1109/TKDE.2016.2619712
Zhu PZhu WHu QZhang CZuo W(2017)Subspace clustering guided unsupervised feature selectionPattern Recognition10.1016/j.patcog.2017.01.01666:C(364-374)Online publication date: 1-Jun-2017
https://dl.acm.org/doi/10.1016/j.patcog.2017.01.016
Sheikhpour RSarram MGharaghani SChahooki M(2017)A Survey on semi-supervised feature selection methodsPattern Recognition10.1016/j.patcog.2016.11.00364:C(141-158)Online publication date: 1-Apr-2017
https://dl.acm.org/doi/10.1016/j.patcog.2016.11.003
Wang YWang JLiao HChen H(2017)An efficient semi-supervised representatives feature selection algorithm based on information theoryPattern Recognition10.1016/j.patcog.2016.08.01161:C(511-523)Online publication date: 1-Jan-2017
https://dl.acm.org/doi/10.1016/j.patcog.2016.08.011
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Table of Contents