More Web Proxy on the site http://driver.im/

research-article

A new k-harmonic nearest neighbor classifier based on the multi-local means

Authors:

Weiping KuAuthors Info & Claims

Expert Systems with Applications: An International Journal, Volume 67, Issue C

Pages 115 - 125

https://doi.org/10.1016/j.eswa.2016.09.031

Published: 01 January 2017 Publication History

Abstract

K different multi-local means based on the k-nearest neighbors are employed.The harmonic mean distance is firstly introduced in the KNN classification problems.The classification error rates can be significantly reduced.Less sensitive to the choice of neighborhood size k.Easily designed in practice with a little extra computation complexity. The k-nearest neighbor (KNN) rule is a classical and yet very effective nonparametric technique in pattern classification, but its classification performance severely relies on the outliers. The local mean-based k-nearest neighbor classifier (LMKNN) was firstly introduced to achieve robustness against outliers by computing the local mean vector of k nearest neighbors for each class. However, its performances suffer from the choice of the single value of k for each class and the uniform value of k for different classes. In this paper, we propose a new KNN-based classifier, called multi-local means-based k-harmonic nearest neighbor (MLM-KHNN) rule. In our method, the k nearest neighbors in each class are first found, and then used to compute k different local mean vectors, which are employed to compute their harmonic mean distance to the query sample. Finally, MLM-KHNN proceeds in classifying the query sample to the class with the minimum harmonic mean distance. The experimental results, based on twenty real-world datasets from UCI and KEEL repository, demonstrated that the proposed MLM-KHNN classifier achieves lower classification error rate and is less sensitive to the parameter k, when compared to nine related competitive KNN-based classifiers, especially in small training sample size situations.

References

[1]

J. Alcalá-Fdez, A. Fernández, J. Luengo, J. Derrac, S. García, Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework, Journal of Multiple-Valued Logic & Soft Computing, 17 (2011) 255-287.

[2]

T. Bailey, A.K. Jain, A note on distance-weighted -nearest neighbor rules, IEEE Transactions on Systems Man and Cybernetics, 8 (2010) 311-313.

[3]

G. Bhattacharya, K. Ghosh, A.S. Chowdhury, A probabilistic framework for dynamic k estimation in kNN classifiers with certainty factor, in: Proceedings of the eighth international conference on advances in pattern recognition, 2015, pp. 1-5.

[4]

L. Chen, G. Guo, Nearest neighbor classification of categorical data by attributes weighting, Expert Systems with Applications, 42 (2015) 3142-3149.

Digital Library

[5]

T. Cover, P. Hart, Nearest neighbor pattern classification, IEEE Transactions on Information Theory, 13 (1967) 21-27.

Digital Library

[6]

S.A. Dudani, The distance-weighted k-nearest-neighbor rule, IEEE Transactions on Systems Man & Cybernetics, SMC, 6 (1976) 325-327.

[7]

Garcia-Pedrajas, N., Del-Castillo, J. A., Cerruela-Garcia, G. (2015). A proposal for local k values for k-nearest neighbor rule. IEEE Transactions on Neural Networks & Learning Systems, in press

[8]

J. Gou, Y. Zhan, Y. Rao, X. Shen, X. Wang, W. He, Improved pseudo nearest neighbor classification, Knowledge-Based Systems, 70 (2014) 361-375.

Digital Library

[9]

Q. Hu, D. Yu, Z. Xie, Neighborhood classifiers, Expert Systems with Applications, 34 (2008) 866-876.

Digital Library

[10]

S. Jiang, G. Pang, M. Wu, L. Kuang, An improved K-nearest neighbor algorithm for text categorization, Expert Systems with Applications, 39 (2012) 1503-1509.

Digital Library

[11]

J.M. Keller, M.R. Gray, J.A. Givens, A fuzzy k-nearest neighbor algorithm, IEEE Transactions on Systems Man & Cybernetics, SMC, 15 (1985) 580-585.

[12]

B. Li, Y. Chen, Y. Chen, The nearest neighbor algorithm of local probability centers, IEEE Transactions on Systems, Man, and Cybernetics-Part B-Cybernetics, 38 (2008) 141-154.

Digital Library

[13]

Y. Lin, J. Li, M. Lin, J. Chen, A new nearest neighbor classifier via fusing neighborhood information, Neurocomputing, 143 (2014) 164-169.

Digital Library

[14]

Z.G. Liu, Q. Pan, J. Dezert, A new belief-based k-nearest neighbor classification method, Pattern Recognition, 46 (2013) 834-844.

Digital Library

[15]

Z.G. Liu, Q. Pan, J. Dezert, G. Mercier, Fuzzy-belief K-nearest neighbor classifier for uncertain data, in: Fusion 2014 International Conference on Information Fusion, 2014, pp. 1-8.

[16]

H. Liu, S. Zhang, Noisy data elimination using mutual k-nearest neighbor for classification mining, Journal of Systems & Software, 85 (2012) 1067-1074.

Digital Library

[17]

D. Mateos-García, J. García-Gutiérrez, J.C. Riquelme-Santos, An evolutionary voting for k-nearest neighbours, Expert Systems with Applications, 43 (2016) 9-14.

Digital Library

[18]

C. Merz, P. Murphy, UCI repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html

[19]

Y. Mitani, Y. Hamamoto, A local mean-based nonparametric classifier, Pattern Recognition Letters, 27 (2006) 1151-1159.

Digital Library

[20]

J.A. Rodger, A fuzzy nearest neighbor neural network statistical model for predicting demand for natural gas and energy cost savings in public buildings, Expert Systems with Applications, 41 (2014) 1813-1829.

Digital Library

[21]

J.A. Rodger, Discovery of medical big data analytics: Improving the prediction of traumatic brain injury survival rates by data mining patient informatics processing software hybrid hadoop hive, Informatics in Medicine Unlocked, 1 (2015) 17-26.

[22]

N.A. Samsudin, A.P. Bradley, Nearest neighbour group-based classification, Pattern Recognition, 43 (2010) 3458-3467.

Digital Library

[23]

M. Sarkar, Fuzzy-rough nearest neighbor algorithms in classification, Fuzzy Sets & Systems, 158 (2007) 2134-2152.

Digital Library

[24]

G. Toussaint, Bibliography on estimation of misclassification, IEEE Transactions on Information Theory, 20 (1974) 472-479.

Digital Library

[25]

T.J. Wagner, Convergence of the nearest neighbor rule, IEEE Transactions on Information Theory, 17 (1971) 566-571.

Digital Library

[26]

J. Wang, P. Neskovic, L.N. Cooper, Neighborhood size selection in the k -nearest-neighbor rule using statistical confidence, Pattern Recognition, 39 (2006) 417-423.

Digital Library

[27]

K.Q. Weinberger, L.K. Saul, Distance Metric Learning for Large Margin Nearest Neighbor Classification, Journal of Machine Learning Research, 10 (2009) 207-244.

Digital Library

[28]

W. Xia, Y. Mita, T. Shibata, A nearest neighbor classifier employing critical boundary vectors for efficient on-chip template reduction, IEEE Transactions on Neural Networks & Learning Systems, 27 (2016) 1094-1107.

[29]

Y. Xu, Q. Zhu, Z. Fan, M. Qiu, Y. Chen, H. Liu, Coarse to fine k nearest neighbor classifier, Pattern Recognition Letters, 34 (2013) 980-986.

Digital Library

[30]

M. Yang, D. Wei, D. Tao, Local discriminative distance metrics ensemble learning, Pattern Recognition, 46 (2013) 2337-2349.

Digital Library

[31]

Z. Yu, H. Chen, J. Liu, J. You, Hybrid k-nearest neighbor classifier, IEEE Transactions on Cybernetics, 46 (2016) 1263-1275.

[32]

Y. Zeng, Y. Yang, L. Zhao, Pseudo nearest neighbor rule for pattern classification, Expert Systems with Applications, 36 (2009) 3587-3595.

Digital Library

[33]

Y. Zeng, Y.P. Yang, L. Zhao, Nearest neighbour classification based on local mean and class mean, Expert Systems with Applications, 36 (2009) 8443-8448.

Digital Library

[34]

N. Zhang, J. Yang, J.J. Qian, Component-based global k-nn classifier for small sample size problems, Pattern Recognition Letters, 33 (2012) 1689-1694.

Digital Library

Cited By

Li DGuo J(2024)A multi-average based pseudo nearest neighbor classifierAI Communications10.3233/AIC-23031237:4(677-691)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.3233/AIC-230312
Li JZhao QLiu S(2024)A heuristic hybrid instance reduction approach based on adaptive relative distance and k-means clusteringThe Journal of Supercomputing10.1007/s11227-023-05885-x80:9(13096-13123)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s11227-023-05885-x
Li SLu J(2023)Self-training algorithm based on density peaks combining globally adaptive multi-local noise filterIntelligent Data Analysis10.3233/IDA-22657527:2(323-343)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.3233/IDA-226575
Show More Cited By

A new k-harmonic nearest neighbor classifier based on the multi-local means
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches

Recommendations

Local Mean k-General Nearest Neighbor Classifier
ICSCA '21: Proceedings of the 2021 10th International Conference on Software and Computer Applications

The well-known k-Nearest Neighbor classifier is a simple and flexible algorithm that has sparked wide interest in pattern classification. In spite of its straightforward implementation, the kNN is sensitive to the presence of noisy training samples and ...
A new general nearest neighbor classification based on the mutual neighborhood information

The nearest neighbor (NN) rule is effective for many applications in pattern classification, such as the famous k-nearest neighbor (kNN) classifier. However, NN-based classifiers perform a one-sided classification by finding the nearest neighbors simply ...
A new globally adaptive k-nearest neighbor classifier based on local mean optimization
Abstract
The k-nearest neighbor (KNN) rule is a simple and effective nonparametric classification algorithm in pattern classification. However, it suffers from several problems such as sensitivity to outliers and inaccurate classification decision rule. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Expert Systems with Applications: An International Journal

Expert Systems with Applications: An International Journal Volume 67, Issue C

January 2017

312 pages

ISSN:0957-4174

Issue’s Table of Contents

Copyright © Elsevier Ltd.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 January 2017

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li DGuo J(2024)A multi-average based pseudo nearest neighbor classifierAI Communications10.3233/AIC-23031237:4(677-691)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.3233/AIC-230312
Li JZhao QLiu S(2024)A heuristic hybrid instance reduction approach based on adaptive relative distance and k-means clusteringThe Journal of Supercomputing10.1007/s11227-023-05885-x80:9(13096-13123)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s11227-023-05885-x
Li SLu J(2023)Self-training algorithm based on density peaks combining globally adaptive multi-local noise filterIntelligent Data Analysis10.3233/IDA-22657527:2(323-343)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.3233/IDA-226575
Wang YPan ZDong J(2022)A new two-layer nearest neighbor selection method for kNN classifierKnowledge-Based Systems10.1016/j.knosys.2021.107604235:COnline publication date: 9-Apr-2022
https://dl.acm.org/doi/10.1016/j.knosys.2021.107604
İnkaya T(2022)Parameter-free surrounding neighborhood based regression methodsExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.116881199:COnline publication date: 1-Aug-2022
https://dl.acm.org/doi/10.1016/j.eswa.2022.116881
Gou JSun LDu LMa HXiong TOu WZhan Y(2022)A representation coefficient-based k-nearest centroid neighbor classifierExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.116529194:COnline publication date: 15-May-2022
https://dl.acm.org/doi/10.1016/j.eswa.2022.116529
Drumond RAlbuquerque RBarreto GSouza A(2022)Pattern classification based on regional models▪Applied Soft Computing10.1016/j.asoc.2022.109592129:COnline publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1016/j.asoc.2022.109592
Sharma MBiswas M(2022)KLT-CRKCN: Hyperspectral Image Classification via Karhunen Loeve Transformation and Collaborative Representation-Based K Closest NeighborWireless Personal Communications: An International Journal10.1007/s11277-021-09292-4123:4(3347-3373)Online publication date: 1-Apr-2022
https://dl.acm.org/doi/10.1007/s11277-021-09292-4
Hajizadeh RAghagolzadeh AEzoji M(2022)Mutual neighborhood and modified majority voting based KNN classifier for multi-categories classificationPattern Analysis & Applications10.1007/s10044-022-01069-025:4(773-793)Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1007/s10044-022-01069-0
Mukahar NRosdi B(2021)Local Mean k-General Nearest Neighbor ClassifierProceedings of the 2021 10th International Conference on Software and Computer Applications10.1145/3457784.3457828(288-293)Online publication date: 23-Feb-2021
https://dl.acm.org/doi/10.1145/3457784.3457828
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents