A Robust Pseudo Fuzzy Rough Feature Selection Using Linear Reconstruction Measure
Authors: 
Lin Qiu, Xingwei Wang, Yanpeng Qu, Kaimin Zhang, Fei Gao, Bo Yi, Keqin LiAuthors Info & Claims
Pages 5687 - 5701
Abstract
Fuzzy-rough sets (FRS) provide an outstanding theoretical tool for feature selection (FS). Whilst promising, the FRS model is sensitive to noisy information and ineffectively applicable to the data with large class density difference, with existing FRS-based FS methods only tackling one of these challenges. Therefore, to overcome both of these issues, this article presents a robust FS algorithm using linear reconstruction measure for the first time. First, a pseudo FRS model is proposed, where the distribution-aware linear reconstruction relation serving as the fuzzy similarity relation is constructed by considering the insight of meaningful information (i.e., distribution information of samples and density information of classes) to enhance the robustness and the pseudofuzzy rough approximations are further redefined based on <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula>-Nearest Neighbor (<inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula>NN) granules determined by the linear reconstruction coefficients to empower the antinoise ability. Then, the pseudo FRS model is employed to guide the robust FS algorithm from the perspective of <italic>redundant filter</italic>, <italic>strongly relevant priority</italic>, and <italic>discriminative selection</italic> to determine the final feature subset. The experimental results on 31 datasets and practical applications (i.e., cancer diagnosis and face recognition) demonstrate that the reduct gained by the proposed approach generally outperforms those attained by alternative implementations of FRS-based FS and state-of-the-art FS techniques.
References
[1]
W. Li, H. Zhou, W. Xu, X. Wang, and W. Pedrycz, “Interval dominance-based feature selection for interval-valued ordered data,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 10, pp. 6898–6912, Oct. 2023.
[2]
W. Xu, M. Huang, Z. Jiang, and Y. Qian, “Graph-based unsupervised feature selection for interval-valued information system,” IEEE Trans. Neural Netw. Learn. Syst., 2023, to be published.
[3]
D. Dubois and H. Prade, “Rough fuzzy sets and fuzzy rough sets,” Int. J. Gen. Syst., vol. 17, no. 2/3, pp. 191–209, 2007.
[4]
D. Dubois and H. Prade, Putting Rough Sets and Fuzzy Sets Together. Berlin, Germany: Springer, 1992, pp. 203–232.
[5]
L. Zadeh, “Fuzzy sets,” Inf. Control, vol. 8, no. 3, pp. 338–353, 1965.
[6]
Z. Pawlak, “Rough sets,” Int. J. Comput. Inf. Sci., vol. 11, no. 5, pp. 341–356, 1982.
[7]
R. Jensen and Q. Shen, “New approaches to fuzzy-rough feature selection,” IEEE Trans. Fuzzy Syst., vol. 17, no. 4, pp. 824–838, Aug. 2009.
[8]
R. Jensen and Q. Shen, “Fuzzy rough attribute reduction with application to web categorization,” Fuzzy Sets Syst., vol. 141, no. 3, pp. 469–485, 2004.
[9]
R. Jensen and Q. Shen, “Semantics-preserving dimensionality reduction: Rough and fuzzy-rough-based approaches,” IEEE Trans. Knowl. Data Eng., vol. 16, no. 12, pp. 1457–1471, Dec. 2004.
[10]
R. Jensen and Q. Shen, “Fuzzy-rough data reduction with ant colony optimization,” Fuzzy Sets Syst., vol. 149, no. 1, pp. 5–20, 2005.
[11]
R. Jensen and Q. Shen, “Fuzzy-rough sets assisted attribute selection,” IEEE Trans. Fuzzy Syst., vol. 15, no. 1, pp. 73–89, Feb. 2007.
[12]
Q. Hu, D. Yu, Z. Xie, and J. Liu, “Fuzzy probabilistic approximation spaces and their information measures,” IEEE Trans. Fuzzy Syst., vol. 14, no. 2, pp. 191–201, Apr. 2006.
[13]
Q. Hu, Z. Xie, and D. Yu, “Hybrid attribute reduction based on a novel fuzzy-rough model and information granulation,” Pattern Recognit., vol. 40, no. 12, pp. 3509–3521, 2007.
[14]
Q. Hu, D. Yu, and Z. Xie, “Information-preserving hybrid data reduction based on fuzzy-rough techniques,” Pattern Recognit. Lett., vol. 27, no. 5, pp. 414–423, 2006.
[15]
T. Yang, Y.-J. Li, Y. Qian, and F.-Y. Wang, “Consistent matrix: A feature selection framework for large-scale datasets,” IEEE Trans. Fuzzy Syst., vol. 31, no. 11, pp. 4024–4038, Nov. 2023.
[16]
C. Wang, Y. Qian, W. Ding, and X. Fan, “Feature selection with fuzzy-rough minimum classification error criterion,” IEEE Trans. Fuzzy Syst., vol. 30, no. 8, pp. 2930–2942, Aug. 2022.
[17]
R. Jensen, N. M. Parthaláin, and C. Cornells, “Feature grouping-based fuzzy-rough feature selection,” in Proc. IEEE Int. Conf. Fuzzy Syst., 2014, pp. 1488–1495.
[18]
L. Zheng, F. Chao, N. M. Parthaláin, D. Zhang, and Q. Shen, “Feature grouping and selection: A graph-based approach,” Inf. Sci., vol. 546, pp. 1256–1272, 2021.
[19]
J. Wan, H. Chen, T. Li, B. Sang, and Z. Yuan, “Feature grouping and selection with graph theory in robust fuzzy rough approximation space,” IEEE Trans. Fuzzy Syst., vol. 31, no. 1, pp. 213–225, Jan. 2023.
[20]
Y. Qu, L. Qiu, C. Shang, and Q. Shen, “Exclusive lasso assisted two-stage fuzzy-rough feature selection,” TechRxiv, 2023.
[21]
J. Fernández Salido and S. Murakami, “Rough set analysis of a general type of fuzzy data using transitive aggregations of fuzzy similarity relations,” Fuzzy Sets Syst., vol. 139, no. 3, pp. 635–660, 2003.
[22]
C. Cornelis, M. De Cock, and A. M. Radzikowska, Vaguely Quantified Rough Sets. Berlin Heidelberg, Germany: Springer, 2007, pp. 87–94.
[23]
A. Mieszkowicz-Rolka and L. Rolka, “Variable precision fuzzy rough sets,” Trans. Rough Sets I, vol. 3100, pp. 144–160, 2004.
[24]
S. Zhao, E. C. C. Tsang, and D. Chen, “The model of fuzzy variable precision rough sets,” in 2007 Int. Conf. Mach. Learn. Cybern., 2007, vol. 6, pp. 3057–3062.
[25]
S. An, Q. Hu, W. Pedrycz, P. Zhu, and E. C. C. Tsang, “Data-distribution-aware fuzzy rough set model and its application to robust classification,” IEEE Trans. Cybern., vol. 46, no. 12, pp. 3073–3085, Dec. 2016.
[26]
N. Verbiest, C. Cornelis, and F. Herrera, “OWA-FRPS: A prototype selection method based on ordered weighted average fuzzy rough set theory,” in Proc. Conf. Rough Sets, Fuzzy Sets, Data Mining Granular Comput., 2013, vol. 8170, pp. 180–190.
[27]
Q. Hu, S. An, and D. Yu, “Soft fuzzy rough sets for robust feature evaluation and selection,” Inf. Sci., vol. 180, no. 22, pp. 4384–4400, 2010.
[28]
Q. Hu, L. Zhang, S. An, D. Zhang, and D. Yu, “On robust fuzzy rough set models,” IEEE Trans. Fuzzy Syst., vol. 20, no. 4, pp. 636–651, Aug. 2012.
[29]
C. Wang, Y. Huang, M. Shao, and X. Fan, “Fuzzy rough set-based attribute reduction using distance measures,” Knowl. Based Syst., vol. 164, pp. 205–212, 2019.
[30]
Q. Hu, D. Yu, W. Pedrycz, and D. Chen, “Kernelized fuzzy rough sets and their applications,” IEEE Trans. Knowl. Data Eng., vol. 23, no. 11, pp. 1649–1667, Nov. 2011.
[31]
Q. Hu, L. Zhang, D. Chen, W. Pedrycz, and D. Yu, “Gaussian kernel based fuzzy rough sets: Model, uncertainty measures and applications,” Int. J. Approx. Reasoning, vol. 51, no. 4, pp. 453–471, 2010.
[32]
Q. Hu, L. Zhang, Y. Zhou, and W. Pedrycz, “Large-scale multimodality attribute reduction with multi-kernel fuzzy rough sets,” IEEE Trans. Fuzzy Syst., vol. 26, no. 1, pp. 226–238, Feb. 2018.
[33]
C. Wang, Y. Huang, W. Ding, and Z. Cao, “Attribute reduction with fuzzy rough self-information measures,” Inf. Sci., vol. 549, pp. 68–86, 2021.
[34]
Y. Qu, R. Li, A. Deng, C. Shang, and Q. Shen, “Non-unique decision differential entropy-based feature selection,” Neurocomputing, vol. 393, pp. 187–193, 2020.
[35]
X. Zhang, C. Mei, D. Chen, Y. Yang, and J. Li, “Active incremental feature selection using a fuzzy-rough-set-based information entropy,” IEEE Trans. Fuzzy Syst., vol. 28, no. 5, pp. 901–915, May 2020.
[36]
B. Sang, W. Xu, H. Chen, and T. Li, “Active antinoise fuzzy dominance rough feature selection using adaptive k-nearest neighbors,” IEEE Trans. Fuzzy Syst., vol. 31, no. 11, pp. 3944–3958, Nov. 2023.
[37]
S. An, E. Zhao, C. Wang, G. Guo, S. Zhao, and P. Li, “Relative fuzzy rough approximations for feature selection and classification,” IEEE Trans. Cybern., vol. 53, no. 4, pp. 2200–2210, Apr. 2023.
[38]
S. An, M. Zhang, C. Wang, and W. Ding, “Robust fuzzy rough approximations with kNN granules for semi-supervised feature selection,” Fuzzy Sets Syst., vol. 461, 2023, Art. no.
[39]
W. Qian, F. Xu, J. Huang, and J. Qian, “A novel granular ball computing-based fuzzy rough set for feature selection in label distribution learning,” Knowl.-Based Syst., vol. 278, 2023, Art. no.
[40]
J. Xu, X. Meng, K. Qu, Y. Sun, and Q. Hou, “Feature selection using relative dependency complement mutual information in fitting fuzzy rough set model,” Appl. Intell., vol. 53, pp. 18239–18262, 2023.
[41]
W. Xu, K. Yuan, W. Li, and W. Ding, “An emerging fuzzy feature selection method using composite entropy-based uncertainty measure and data distribution,” IEEE Trans. Emerg. Topics Comput. Intell., vol. 7, no. 1, pp. 76–88, Feb. 2023.
[42]
J. Zhang and J. Yang, “Linear reconstruction measure steered nearest neighbor classification framework,” Pattern Recognit., vol. 47, no. 4, pp. 1709–1720, 2014.
[43]
R. Guha, M. Ghosh, A. Chakrabarti, R. Sarkar, and S. Mirjalili, “Introducing clustering based population in binary gravitational search algorithm for feature selection,” Appl. Soft Comput., vol. 93, 2020, Art. no.
[44]
S. Dhargupta, M. Ghosh, S. Mirjalili, and R. Sarkar, “Selective opposition based grey wolf optimization,” Expert Syst. Appl., vol. 151, 2020, Art. no.
[45]
R. J. Urbanowicz, M. Meeker, W. La Cava, R. S. Olson, and J. H. Moore, “Relief-based feature selection: Introduction and review,” J. Biomed. Informat., vol. 85, pp. 189–203, 2018.
[46]
R. Guha, K. K. Ghosh, S. Bhowmik, and R. Sarkar, “Mutually informed correlation coefficient (MICC) – A new filter based feature selection method,” in 2020 IEEE Calcutta Conf., 2020, pp. 54–58.
[47]
J. R. Quinlan, C4. 5: Programs for Machine Learning. Amsterdam, The Netherlands: Elsevier, 2014.
[48]
L. Breiman, “Bagging predictors,” Mach. Learn., vol. 24, pp. 123–140, 1996.
[49]
W. W. Cohen, “Fast effective rule induction,” in Proc. 12th Int. Conf. Mach. Learn., Elsevier, 1995, pp. 115–123.
[50]
E. Frank and I. H. Witten, “Generating accurate rule sets without global optimization,” 1998.
Index Terms
- A Robust Pseudo Fuzzy Rough Feature Selection Using Linear Reconstruction Measure
Index terms have been assigned to the content through auto-classification.
Recommendations
Different classes' ratio fuzzy rough set based robust feature selection
In order to solve the problem that the classical fuzzy rough set (FRS) model used for feature selection is sensitive to noisy information, we propose an effective robust fuzzy rough set model, called different classes' ratio fuzzy rough set (DC_ratio ...
Robust fuzzy rough classifiers
Fuzzy rough sets, generalized from Pawlak's rough sets, were introduced for dealing with continuous or fuzzy data. This model has been widely discussed and applied these years. It is shown that the model of fuzzy rough sets is sensitive to noisy samples,...
The Fuzziness Measure in Fuzzy Rough Sets
FSKD '08: Proceedings of the 2008 Fifth International Conference on Fuzzy Systems and Knowledge Discovery - Volume 01The paper studies the fuzziness measure in fuzzy rough sets. By making use of the support set of fuzzy sets, a rough membership function for fuzzy sets based on fuzzy relation is introduced. Simultaneously, a fuzziness measure of fuzzy rough sets from ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
1941-0034 © 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Press
Publication History
Published: 01 October 2024
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025