A backpropagation learning algorithm with graph regularization for feedforward neural networks
Authors: 
Yetian Fan, Wenyu YangAuthors Info & Claims
References
[1]
N. Ganesan, K. Venkatesh, M. Rama, A.M. Palani, Application of neural networks in diagnosing cancer disease using demographic data, Int. J. Comput. Appl. 1 (26) (2010) 76–85.
[2]
G. Litjens, T. Kooi, B.E. Bejnordi, A.A.A. Setio, F. Ciompi, M. Ghafoorian, J.A. Van Der Laak, B. Van Ginneken, C.I. Sánchez, A survey on deep learning in medical image analysis, Med. Image Anal. 42 (2017) 60–88.
[3]
G. Hinton, L. Deng, D. Yu, G.E. Dahl, A.r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T.N. Sainath, Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups, IEEE Signal Process. Mag. 29(6) (2012) 82–97.
[4]
W. Liu, Z. Wang, X. Liu, N. Zeng, Y. Liu, F.E. Alsaadi, A survey of deep neural network architectures and their applications, Neurocomputing 234 (2017) 11–26.
[5]
P. Werbos, New tools for prediction and analysis in the behavioral sciences, Ph. D. dissertation, Harvard University.
[6]
D.E. Rumelhart, G.E. Hinton, R.J. Williams, Learning representations by back-propagating errors, Nature 323 (6088) (1986) 533–536.
[7]
A.B. Nielsen, L.K. Hansen, Structure learning by pruning in independent component analysis, Neurocomputing 71 (10–12) (2008) 2281–2290.
[8]
R. Setiono, A penalty-function approach for pruning feedforward neural networks, Neural Comput. 9 (1) (1997) 185–204.
[9]
R. Parekh, J. Yang, V. Honavar, Constructive neural-network learning algorithms for pattern classification, IEEE Trans. Neural Networks 11 (2) (2000) 436–451.
[10]
J.M. Zurada, A. Malinowski, S. Usui, Perturbation method for deleting redundant inputs of perceptron networks, Neurocomputing 14 (2) (1997) 177–193.
[11]
W. Wan, S. Mabu, K. Shimada, K. Hirasawa, J. Hu, Enhancing the generalization ability of neural networks through controlling the hidden layers, Appl. Soft Comput. 9 (1) (2009) 404–414.
[12]
A.S. Weigend, D.E. Rumelhart, B.A. Huberman, Generalization by weight-elimination applied to currency exchange rate prediction, in: 1991 IEEE International Joint Conference on Neural Networks, IEEE, 1991, pp. 2374–2379.
[13]
H.J. Rong, Y.S. Ong, A.H. Tan, Z. Zhu, A fast pruned-extreme learning machine for classification problem, Neurocomputing 72 (1–3) (2008) 359–366.
[14]
Y. Miche, A. Sorjamaa, P. Bas, O. Simula, C. Jutten, A. Lendasse, Op-elm: optimally pruned extreme learning machine, IEEE Trans. Neural Networks 21 (1) (2009) 158–162.
[15]
L. Meier, S. Van De Geer, P. Bühlmann, The group lasso for logistic regression, J. R. Stat. Soc.: Ser. B (Statistical Methodology) 70 (1) (2008) 53–71.
[16]
J.M. MartíNez-MartíNez, P. Escandell-Montero, E. Soria-Olivas, J.D. MartíN-Guerrero, R. Magdalena-Benedito, J. GóMez-Sanchis, Regularized extreme learning machine for regression problems, Neurocomputing 74 (17) (2011) 3716–3721.
[17]
W. Wu, H. Shao, Z. Li, Convergence of batch bp algorithm with penalty for fnn training, International Conference on Neural Information Processing, Springer (2006) 562–569.
[18]
C.S. Leung, H. Wang, J. Sum, On the selection of weight decay parameter for faulty networks, IEEE Trans. Neural Networks 21 (8) (2010) 1232–1244.
[19]
R. Tibshirani, Regression shrinkage and selection via the lasso, J. R. Stat. Soc.: Ser. B (Methodol.) 58 (1) (1996) 267–288.
[20]
B.N.G. Koneru, V. Vasudevan, Sparse artificial neural networks using a novel smoothed lasso penalization, IEEE Trans. Circuits Syst. II Express Briefs 66 (5) (2019) 848–852.
[21]
X. Xie, H. Zhang, J. Wang, Q. Chang, J. Wang, N.R. Pal, Learning optimized structure of neural networks by hidden node pruning with L_1) regularization, IEEE Trans. Cybern. 50 (3) (2019) 1333–1346.
[22]
J. Wang, C. Xu, X. Yang, J.M. Zurada, A novel pruning algorithm for smoothing feedforward neural networks based on group lasso method, IEEE Trans. Neural Networks Learn. Syst. 29 (5) (2017) 2012–2024.
[23]
Z. Xu, X. Chang, F. Xu, H. Zhang, L_1/2) regularization: A thresholding representation theory and a fast solver, IEEE Trans. Neural Networks Learn. Syst. 23 (7) (2012) 1013–1027.
[24]
W. Wu, Q. Fan, J.M. Zurada, J. Wang, D. Yang, Y. Liu, Batch gradient method with smoothing L_1/2) regularization for training of feedforward neural networks, Neural Networks 50 (2014) 72–78.
[25]
F. Li, J.M. Zurada, Y. Liu, W. Wu, Input layer regularization of multilayer feedforward neural networks, IEEE Access 5 (2017) 10979–10985.
[26]
C.M. Feng, Y.L. Gao, J.X. Liu, C.H. Zheng, J. Yu, PCA based on graph Laplacian regularization and p-norm for gene selection and clustering, IEEE Trans. Nanobiosci. 16 (4) (2017) 257–265.
[27]
Y. Pei, N. Chakraborty, K. Sycara, Nonnegative matrix tri-factorization with graph regularization for community detection in social networks, in: Twenty-fourth International Joint Conference on Artificial Intelligence, 2015.
[28]
M. Belkin, P. Niyogi, V. Sindhwani, Manifold regularization: A geometric framework for learning from labeled and unlabeled examples, J. Mach. Learn. Res. 7(11).
[29]
W. Xu, X. Jiang, X. Hu, G. Li, Visualization of genetic disease-phenotype similarities by multiple maps t-sne with Laplacian regularization, BMC Med. Genomics 7 (2) (2014) 1–9.
[30]
J. Zeng, J. Pang, W. Sun, G. Cheung, Deep graph Laplacian regularization for robust denoising of real images, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019.
[31]
A. Asuncion, D. Newman, UCI machine learning repository.
[32]
C. Chang, C. Lin, Libsvm: A library for support vector machines, ACM Trans. Intell. Syst. Technol. 2 (3) (2011) 1–27.
[33]
J. Han, C. Moraga, The influence of the sigmoid function parameters on the speed of backpropagation learning, International Workshop on Artificial Neural Networks, Springer (1995) 195–201.
[34]
S. Elfwing, E. Uchibe, K. Doya, Sigmoid-weighted linear units for neural network function approximation in reinforcement learning, Neural Networks 107 (2018) 3–11.
[35]
R.M. Neal, Connectionist learning of belief networks, Artif. Intell. 56 (1) (1992) 71–113.
[36]
V. Nair, G.E. Hinton, Rectified linear units improve restricted boltzmann machines, in: ICML, 2010.
[37]
D.A. Clevert, T. Unterthiner, S. Hochreiter, Fast and accurate deep network learning by exponential linear units (elus), arXiv preprint:1511.07289.
[38]
D. Hendrycks, K. Gimpel, Gaussian error linear units (gelus), arXiv preprint:1606.08415.
[39]
J. Wang, Y. Wen, Z. Ye, L. Jian, H. Chen, Convergence analysis of bp neural networks via sparse response regularization, Appl. Soft Comput. 61 (2017) 354–363.
[40]
W. Wu, L. Li, J. Yang, Y. Liu, A modified gradient-based neuro-fuzzy learning algorithm and its convergence, Inf. Sci. 180 (9) (2010) 1630–1642.
Index Terms
- A backpropagation learning algorithm with graph regularization for feedforward neural networks
Index terms have been assigned to the content through auto-classification.
Recommendations
Online gradient method with smoothing ź0 regularization for feedforward neural networks
źp regularization has been a popular pruning method for neural networks. The parameter p was usually set as 0 < p ź 2 in the literature, and practical training algorithms with ź0 regularization are lacking due to the NP-hard nature of the ź0 ...
Better learning of supervised neural networks based on functional graph: an experimental approach
Multilayered feed forward neural networks possess a number of properties which make them particularly suited to complex problems. Neural networks have been in use in numerous meteorological applications including weather forecasting. As Neural Networks ...
Convergence analyses on sparse feedforward neural networks via group lasso regularization
In this paper, a new variant of feedforward neural networks has been proposed for a class of nonsmooth optimization problems. The penalty term of the presented neural networks stems from the Group Lasso method which selects hidden variables in a grouped ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Elsevier Inc.
Publisher
Elsevier Science Inc.
United States
Publication History
Published: 01 August 2022
Author Tags
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 12 Jan 2025