[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

Back-propagation extreme learning machine

  • Data analytics and machine learning
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Incremental Extreme Learning Machine (I-ELM) is a typical constructive feed-forward neural network with random hidden nodes, which can automatically determine the appropriate number of hidden nodes. However I-ELM and its variants suffer from a notorious problem, that is, the input parameters of these algorithms are randomly assigned and kept fixed throughout the training process, which results in a very unstable performance of the model. To solve this problem, we propose a novel Back-Propagation ELM (BP-ELM) in this study, which can dynamically assign the most appropriate input parameters according to the current residual error of the model during the increasing process of the hidden nodes. In this way, BP-ELM can greatly improve the quality of newly added nodes and then accelerate the convergence rate and improve the model performance. Moreover, under the same error level, the network structure of the model obtained by BP-ELM is more compact than that of the I-ELM. We also prove the universal approximation ability of BP-ELM in this study. Experimental results on three benchmark regression problems and a real-life traffic flow prediction problem empirically show that BP-ELM has better stability and generalization ability than other I-ELM-based algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

Data sharing does not apply to this article as no new data have been created or analyzed in this study. All the data utilized for analysis of the proposed approach is publicly available

Notes

  1. MATLAB code for BP-ELM: https://github.com/Wepond/Matlab-code-BPELM.

  2. UCI database: http://archive.ics.uci.edu/ml/index.php.

  3. BLS: https://www.fst.um.edu.mo/en/staff/pchen.html.

References

  • Cao WP, Ming Z, Wang XZ, Cai SB (2019) Improved bidirectional extreme learning machine based on enhanced random search. Memet Comput 11:19–26. https://doi.org/10.1007/s12293-017-0238-1

    Article  Google Scholar 

  • Cao WP, Ming Z, Xu Z, Zhang J, Wang Q (2019) Online sequential extreme learning machine with dynamic forgetting factor. IEEE Access 7:179746–179757. https://doi.org/10.1109/ACCESS.2019.2959032

    Article  Google Scholar 

  • Chauvin Y, Rumelhart DE (eds) (1995) Backpropagation: Theory, architectures, and applications. L. Erlbaum Associates Inc., USA, pp 1–34

    Google Scholar 

  • Chen CLP, Liu Z (2018) Broad learning system: an effective and efficient incremental learning system without the need for deep architecture. IEEE Trans Neural Netw Learn Syst 29(1):10–24. https://doi.org/10.1109/TNNLS.2017.2716952

    Article  MathSciNet  Google Scholar 

  • Chen CLP, Liu Z, Feng S (2019) Universal approximation capability of broad learning system and its structural variations. IEEE Trans Neural Netw Learn Syst 30(4):1191–1204. https://doi.org/10.1109/TNNLS.2018.2866622

    Article  MathSciNet  Google Scholar 

  • Chen Y, Kloft M, Yang Y, Li C, Li L (2018) Mixed kernel based extreme learning machine for electric load forecasting. Neurocomputing 312:90–106. https://doi.org/10.1016/j.neucom.2018.05.068

    Article  Google Scholar 

  • Dhanaraj R.K, Ramakrishnan V, Poongodi M, Krishnasamy L, Hamdi M, Kotecha K, Vijayakumar V (2021) Random forest bagging and x-means clustered antipattern detection from sql query log for accessing secure mobile data. Wirel Commun Mobile Comput 2021

  • Feng G, Huang GB, Lin Q, Gay R (2009) Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw 20(8):1352–1357. https://doi.org/10.1109/TNN.2009.2024147

    Article  Google Scholar 

  • Fomin SV, et al.(1999) Elements of the theory of functions and functional analysis. vol. 1. Courier Corporation

  • Guo BZ, Zhao ZL (2011) On convergence of tracking differentiator. Int J Control 84(4):693–701

    Article  MathSciNet  Google Scholar 

  • Huang G.B, Chen L Enhanced random search based incremental extreme learning machine. Neurocomputing 71(16-18), 3460–3468. 10.1016/j.neucom.2007.10.008

  • Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70(16–18):3056–3062

    Article  Google Scholar 

  • Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71(16–18):3460–3468

    Article  Google Scholar 

  • Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892. https://doi.org/10.1109/TNN.2006.875977

    Article  Google Scholar 

  • Huang Guang-Bin, Zhu Qin-Yu, Siew (2004) Chee-Kheong: extreme learning machine: a new learning scheme of feedforward neural networks. In: 2004 IEEE International Joint Conference on Neural Networks. vol. 2, pp. 985–990. vol.2. 10.1109/IJCNN.2004.1380068

  • Krishnamoorthi S, Jayapaul P, Dhanaraj RK, Rajasekar V, Balusamy B, Islam S (2021) Design of pseudo-random number generator from turbulence padded chaotic map. Nonlinear Dyn 104(2):1627–1643

    Article  Google Scholar 

  • Krishnamoorthi S, Jayapaul P, Rajasekar V, Dhanaraj RK, Iwendi C (2022) A futuristic approach to generate random bit sequence using dynamic perturbed chaotic system. Turk J Electr Eng Comput Sci 30(1):35–49

    Google Scholar 

  • Kumar DR, Krishna TA, Wahi A (2018) Health monitoring framework for in time recognition of pulmonary embolism using internet of things. J Comput Theor Nanosci 15(5):1598–1602

    Article  Google Scholar 

  • Lai X, Cao J, Huang X, Wang T, Lin Z (2019) A maximally split and relaxed admm for regularized extreme learning machines. IEEE Trans Netw Learn Syst pp. 1–15. 10.1109/TNNLS.2019.2927385

  • Lan Y, Soh YC, Huang GB (2010) Two-stage extreme learning machine for regression. Neurocomputing 73(16–18):3028–3038

    Article  Google Scholar 

  • Leshno M, Lin VY, Pinkus A, Schocken S (1993) Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural netw 6(6):861–867

    Article  Google Scholar 

  • Li S, Song S, Huang G, Wu C (2019) Cross-domain extreme learning machines for domain adaptation. IEEE Trans Syst Man Cybern Syst 49(6):1194–1207. https://doi.org/10.1109/TSMC.2017.2735997

    Article  Google Scholar 

  • Li Y (2016) Orthogonal incremental extreme learning machine for regression and multiclass classification. Neural Comput Appl 27(1):111–120. https://doi.org/10.1007/s00521-014-1567-3

    Article  Google Scholar 

  • Liu J, Zuo L, Xu X, Zhang X, Ren J, Fang Q, Liu X (2019) Efficient batch-mode reinforcement learning using extreme learning machines. IEEE Trans Syst Man Cybern Syst pp. 1–14. 10.1109/TSMC.2019.2926806

  • Park J, Sandberg IW (1991) Universal approximation using radial-basis-function networks. Neural comput 3(2):246–257

    Article  Google Scholar 

  • Priya V, Subha S, Balamurugan B (2017) Analysis of performance measures to handle medical e-commerce shopping cart abandonment in cloud. Inform in Med Unlocked 8:32–41

    Article  Google Scholar 

  • Ragusa E, Gianoglio C, Gastaldo P, Zunino R (2018) A digital implementation of extreme learning machines for resource-constrained devices. IEEE Trans Circuits and Syst II Exp Briefs 65(8):1104–1108. https://doi.org/10.1109/TCSII.2018.2806085

    Article  Google Scholar 

  • Ramasamy MD, Periasamy K, Krishnasamy L, Dhanaraj RK, Kadry S, Nam Y (2021) Multi-disease classification model using strassen’s half of threshold (shot) training algorithm in healthcare sector. IEEE Access 9:112624–112636

    Article  Google Scholar 

  • Sahani M, Dash PK (2018) Automatic power quality events recognition based on hilbert huang transform and weighted bidirectional extreme learning machine. IEEE Trans Indu Informat 14(9):3849–3858. https://doi.org/10.1109/TII.2018.2803042

    Article  Google Scholar 

  • Saravanakumar C, Senthilvel P, Thirupurasundari D, Periyasamy P, Vijayakumar K (2021) Plant syndrome recognition by gigapixel image using convolutional neural network. In: Proceedings of the first international conference on advanced scientific innovation in science, engineering and technology, ICASISET, 16–17 May 2020, Chennai, India. https://doi.org/10.4108/eai.16-5-2020.2304207

  • Saravanakumar P, Sundararajan T, Dhanaraj RK, Nisar K, Memon FH, Ibrahim AABA (2022) Lamport certificateless signcryption deep neural networks for data aggregation security in wsn. Intell Autom Soft Comput 33(3):1835–1847

    Article  Google Scholar 

  • Wang C, Jianhui W, Shusheng G (2016) Deep network based on stacked orthogonal convex incremental elm autoencoders. Math Probl in Eng 2016:1–17. https://doi.org/10.1155/2016/1649486

    Article  MathSciNet  MATH  Google Scholar 

  • Xia Y, Yan C, Wang X, Song X (2019) Intelligent transportation cyber-physical cloud control systems. Acta Autom Sin 45(1), 132–142. 10.16383/j.aas.c180370

  • Yang Y, Wang Y, Jonathan Wu QM, Lin X, Liu M (2015) Progressive learning machine: a new approach for general hybrid system approximation. IEEE Trans Neural Netw Learn Syst 26(9):1855–1874. https://doi.org/10.1109/TNNLS.2014.2357683

    Article  MathSciNet  Google Scholar 

  • Yang Y, Wang Y, Yuan X (2012) Bidirectional extreme learning machine for regression problem and its learning effectiveness. IEEE Trans Neural Netw Learn Syst 23(9):1498–1505. https://doi.org/10.1109/TNNLS.2012.2202289

    Article  Google Scholar 

  • Yang Y, Wu QMJ, Wang Y, Zeeshan KM, Lin X, Yuan X (2015) Data partition learning with multiple extreme learning machines. IEEE Trans Cybern 45(8):1463–1475. https://doi.org/10.1109/TCYB.2014.2352594

    Article  Google Scholar 

  • Yu H, Yang X, Zheng S, Sun C (2019) Active learning from imbalanced data: a solution of online weighted extreme learning machine. IEEE Trans Neural Netw Learn Syst 30(4):1088–1103. https://doi.org/10.1109/TNNLS.2018.2855446

    Article  Google Scholar 

  • Zhou X, Yang C, Gui W (2012) State transition algorithm. arXiv preprint arXiv:1205.6548

Download references

Acknowledgements

The authors would like to thank the editors and anonymous reviewers for their time in reviewing this study.

Funding

This work was supported by National Key Research and Development Program of China (2018YFB1700400), National Natural Science Foundation of China (62106150, 61906015), Beijing Natural Science Foundation (L201004), CCF-NSFOCUS (2021001), CAAC Key Laboratory of Civil Aviation Wide Surveillance and Safety Operation Management and Control Technology (202102).

Author information

Authors and Affiliations

  1. School of Automation, Beijing Institute of Technology, Beijing, 100081, China

    Weidong Zou & Yuanqing Xia

  2. College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China

    Weipeng Cao

Authors
  1. Weidong Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuanqing Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weipeng Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by [Weidong Zou] and [Yuanqing Xia]. The first draft and the revision version of the manuscript were written by [Weipeng Cao], and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Weipeng Cao.

Ethics declarations

Conflicts of interest

The authors declare that there is no conflict of interests.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed Consent

All the authors listed have approved the manuscript that is enclosed.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zou, W., Xia, Y. & Cao, W. Back-propagation extreme learning machine. Soft Comput 26, 9179–9188 (2022). https://doi.org/10.1007/s00500-022-07331-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-022-07331-1

Keywords

Search

Navigation