[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content

Advertisement

Springer Nature Link
Log in

Error Metrics and Performance Fitness Indicators for Artificial Intelligence and Machine Learning in Engineering and Sciences

  • Review Paper
  • Published:
Architecture, Structures and Construction Aims and scope Submit manuscript

Abstract

Artificial intelligence (AI) and Machine learning (ML) train machines to achieve a high level of cognition and perform human-like analysis. Both AI and ML seemingly fit into our daily lives as well as complex and interdisciplinary fields. With the rise of commercial, open-source, and user-catered AI/ML tools, a key question often arises whenever AI/ML is applied to explore a phenomenon or a scenario: what constitutes a good AI/ML model? Keeping in mind that a proper answer to this question depends on various factors, this work presumes that a goodmodel optimally performs and best describes the phenomenon on hand. From this perspective, identifying proper assessment metrics to evaluate the performance of AI/ML models is not only necessary but is also warranted. As such, this paper examines 78 of the most commonly-used performance fitness and error metrics for regression and classification algorithms, with emphasis on engineering and sciences applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

No data, models, or code were generated or used during the study.

Notes

  1. One should note that the validation of an FE model is also governed by satisfying convergence criteria input in the FE software. More on this can be found elsewhere [37, 38].

  2. It should be noted that other works have used a different classification for PFEMs [2]. Botchkarev [2] went even further to survey the most preferred metrics reported by researchers during the 1980–2007 era and also explored multiplication and addition point distance methods.

References

  1. Mahdavi S, Rahnamayan S, Deb K (2018) Opposition based learning: A literature review. Swarm Evol Comput. https://doi.org/10.1016/j.swevo.2017.09.010

    Article  Google Scholar 

  2. Botchkarev A (2019) A new typology design of performance metrics to measure errors in machine learning regression algorithms. Interdiscip J Information Knowledge Manag 14:045–076. https://doi.org/10.28945/4184

    Article  Google Scholar 

  3. Bishop C (2007) Pattern Recognition and Machine Learning. Technometrics. https://doi.org/10.1198/tech.2007.s518

    Article  MATH  Google Scholar 

  4. Fu G-S, Levin-Schwartz Y, Lin Q-H, Zhang D (2019) Machine Learning for Medical Imaging. J Healthc Eng. https://doi.org/10.1155/2019/9874591

    Article  Google Scholar 

  5. Michalski, R. S., Carbonell, J. G., & Mitchell TM (1983) Machine learning: An artificial intelligence approach.

  6. Majidifard H, Jahangiri B, Buttlar WG, Alavi AH (2019) New machine learning-based prediction models for fracture energy of asphalt mixtures. Meas J Int Meas Confed. https://doi.org/10.1016/j.measurement.2018.11.081

    Article  Google Scholar 

  7. Hu X, Li SE, Yang Y (2016) Advanced Machine Learning Approach for Lithium-Ion Battery State Estimation in Electric Vehicles. IEEE Trans Transp Electrif. https://doi.org/10.1109/TTE.2015.2512237

    Article  Google Scholar 

  8. Voyant C, Notton G, Kalogirou S, et al (2017) Machine learning methods for solar radiation forecasting: A review. Renew. Energy

  9. Shukla R, Singh D (2017) Experimentation investigation of abrasive water jet machining parameters using Taguchi and Evolutionary optimization techniques. Swarm Evol Comput. https://doi.org/10.1016/j.swevo.2016.07.002

    Article  Google Scholar 

  10. Hindman M (2015) Building Better Models: Prediction, Replication, and Machine Learning in the Social Sciences. Ann Am Acad Pol Soc Sci. https://doi.org/10.1177/0002716215570279

    Article  Google Scholar 

  11. Grimmer J (2014) We are all social scientists now: How big data, machine learning, and causal inference work together. In: PS - Political Science and Politics

  12. Naser M, Chehab A (2018) Materials and design concepts for space-resilient structures. Prog Aerosp Sci 98:74–90. https://doi.org/10.1016/j.paerosci.2018.03.004

    Article  Google Scholar 

  13. Rashno A, Nazari B, Sadri S, Saraee M (2017) Effective pixel classification of Mars images based on ant colony optimization feature selection and extreme learning machine. Neurocomputing. https://doi.org/10.1016/j.neucom.2016.11.030

    Article  Google Scholar 

  14. Jordan MI, Mitchell TM (2015) Machine learning: Trends, perspectives, and prospects. Science 349:255–260. https://doi.org/10.1126/science.aaa8415

    Article  MathSciNet  MATH  Google Scholar 

  15. Seitllari A (2014) Traffic Flow Simulation by Neuro-Fuzzy Approach. In: Second International Conference on Traffic. Belgrade, pp 97–102

  16. Naser MZ (2019) AI-based cognitive framework for evaluating response of concrete structures in extreme conditions. Eng Appl Artif Intell 81:437–449. https://doi.org/10.1016/J.ENGAPPAI.2019.03.004

    Article  Google Scholar 

  17. Li X, Qiao T, Pang Y et al (2018) A new machine vision real-time detection system for liquid impurities based on dynamic morphological characteristic analysis and machine learning. Meas J Int Meas Confed. https://doi.org/10.1016/j.measurement.2018.04.015

    Article  Google Scholar 

  18. Oleaga I, Pardo C, Zulaika JJ, Bustillo A (2018) A machine-learning based solution for chatter prediction in heavy-duty milling machines. Meas J Int Meas Confed. https://doi.org/10.1016/j.measurement.2018.06.028

    Article  Google Scholar 

  19. Shanmugamani R, Sadique M, Ramamoorthy B (2015) Detection and classification of surface defects of gun barrels using computer vision and machine learning. Meas J Int Meas Confed. https://doi.org/10.1016/j.measurement.2014.10.009

    Article  Google Scholar 

  20. Naser MZ (2019) Properties and material models for common construction materials at elevated temperatures. Constr Build Mater 10:192–206. https://doi.org/10.1016/j.conbuildmat.2019.04.182

    Article  Google Scholar 

  21. Raccuglia P, Elbert KC, Adler PDF et al (2016) Machine-learning-assisted materials discovery using failed experiments. Nature. https://doi.org/10.1038/nature17439

    Article  Google Scholar 

  22. Alavi AH, Hasni H, Lajnef N et al (2016) Damage detection using self-powered wireless sensor data: An evolutionary approach. Meas J Int Meas Confed. https://doi.org/10.1016/j.measurement.2015.12.020

    Article  Google Scholar 

  23. Farrar CR, Worden K (2012) Structural Health Monitoring: A Machine Learning Perspective

  24. Mcfarlane C (2011) The city as a machine for learning. Trans Inst Br Geogr. https://doi.org/10.1111/j.1475-5661.2011.00430.x

    Article  Google Scholar 

  25. Chan J, Chan K, Yeh A (2001) Detecting the nature of change in an urban environment: A comparison of machine learning algorithms. Photogramm. Eng. Remote Sensing

  26. King DE (2009) Dlibml: A Machine Learning Toolkit. J Mach Learn Res

  27. Collobert R, Kavukcuoglu K, Farabet C (2011) Torch7: A Matlab-like Environment for Machine Learning

  28. Hall M, Frank E, Holmes G et al (2009) The WEKA data mining software. ACM SIGKDD Explor Newsl DOI 10(1145/1656274):1656278

    Google Scholar 

  29. Ramsundar B (2016) TensorFlow Tutorial. CS224d

  30. Zaharia M, Franklin MJ, Ghodsi A et al (2016) Apache Spark. Commun ACM. https://doi.org/10.1145/2934664

    Article  Google Scholar 

  31. Korolov M (2018) AI’s biggest risk factor: Data gone wrong | CIO. In: CIO. https://www.cio.com/article/3254693/ais-biggest-risk-factor-data-gone-wrong.html. Accessed 5 Jul 2019

  32. Kodur VKR, Garlock M, Iwankiw N (2012) Structures in Fire: State-of-the-Art, Research and Training Needs. Fire Technol 48:825–839. https://doi.org/10.1007/s10694-011-0247-4

    Article  Google Scholar 

  33. Naser MZ (2019) Fire Resistance Evaluation through Artificial Intelligence - A Case for Timber Structures. Fire Saf J 105:1–18. https://doi.org/10.1016/j.firesaf.2019.02.002

    Article  Google Scholar 

  34. Domingos P (2012) A few useful things to know about machine learning. Commun ACM. https://doi.org/10.1145/2347736.2347755

    Article  Google Scholar 

  35. Shakya AM, Kodur VKR (2015) Response of precast prestressed concrete hollowcore slabs under fire conditions. Eng Struct. https://doi.org/10.1016/j.engstruct.2015.01.018

    Article  Google Scholar 

  36. Kodur VKR, Bhatt PP (2018) A numerical approach for modeling response of fiber reinforced polymer strengthened concrete slabs exposed to fire. Compos Struct 187:226–240. https://doi.org/10.1016/J.COMPSTRUCT.2017.12.051

    Article  Google Scholar 

  37. Kohnke PC (2013) ANSYS. In: © ANSYS, Inc.

  38. Abaqus 6.13 (2013) Abaqus 6.13. Anal User’s Guid Dassault Syst

  39. Franssen JM, Gernay T (2017) Modeling structures in fire with SAFIR®: Theoretical background and capabilities. J Struct Fire Eng. https://doi.org/10.1108/JSFE-07-2016-0010

    Article  Google Scholar 

  40. Golafshani EM, Behnood A (2018) Automatic regression methods for formulation of elastic modulus of recycled aggregate concrete. Appl Soft Comput J. https://doi.org/10.1016/j.asoc.2017.12.030

    Article  Google Scholar 

  41. Sadowski Ł, Nikoo M, Nikoo M (2018) Concrete compressive strength prediction using the imperialist competitive algorithm. Comput Concr. https://doi.org/10.12989/cac.2018.22.4.355

    Article  Google Scholar 

  42. Alavi AH, Gandomi AH, Sahab MG, Gandomi M (2010) Multi expression programming: A new approach to formulation of soil classification. Eng Comput 26:111–118. https://doi.org/10.1007/s00366-009-0140-7

    Article  Google Scholar 

  43. Mirjalili S, Lewis A (2015) Novel performance metrics for robust multi-objective optimization algorithms. Swarm Evol Comput. https://doi.org/10.1016/j.swevo.2014.10.005

    Article  Google Scholar 

  44. Mishra SK, Panda G, Majhi R (2014) A comparative performance assessment of a set of multiobjective algorithms for constrained portfolio assets selection. Swarm Evol Comput. https://doi.org/10.1016/j.swevo.2014.01.001

    Article  Google Scholar 

  45. Schmidt MD, Lipson H (2010) Age-fitness pareto optimization

  46. Cremonesi P, Koren Y, Turrin R (2010) Performance of Recommender Algorithms on Top-N Recommendation Tasks Categories and Subject Descriptors. RecSys

  47. Laszczyk M, Myszkowski PB (2019) Survey of quality measures for multi-objective optimization: Construction of complementary set of multi-objective quality measures. Swarm Evol Comput 48:109–133. https://doi.org/10.1016/J.SWEVO.2019.04.001

    Article  Google Scholar 

  48. Willmott CJ, Matsuura K (2005) Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim Res. https://doi.org/10.3354/cr030079

    Article  Google Scholar 

  49. Makridakis S (1993) Accuracy measures: theoretical and practical concerns. Int J Forecast. https://doi.org/10.1016/0169-2070(93)90079-3

    Article  Google Scholar 

  50. Ferreira C (2001) Gene Expression Programming: a New Adaptive Algorithm for Solving Problems. Ferreira, C (2001) Gene Expr Program a New Adapt Algorithm Solving Probl Complex Syst 13

  51. (2016) Handbook of Time Series Analysis, Signal Processing, and Dynamics

  52. Hyndman RJ, Koehler AB (2006) Another look at measures of forecast accuracy. Int J Forecast. https://doi.org/10.1016/j.ijforecast.2006.03.001

    Article  Google Scholar 

  53. Shcherbakov MV, Brebels A, Shcherbakova NL et al (2013) A survey of forecast error measures. World Appl Sci J. https://doi.org/10.5829/idosi.wasj.2013.24.itmies.80032

    Article  Google Scholar 

  54. Bain LJ (1967) Applied Regression Analysis. Technometrics. https://doi.org/10.1080/00401706.1967.10490452

    Article  Google Scholar 

  55. Armstrong JS, Collopy F (1992) Error measures for generalizing about forecasting methods: Empirical comparisons. Int J Forecast. https://doi.org/10.1016/0169-2070(92)90008-W

    Article  Google Scholar 

  56. Poli AA, Cirillo MC (1993) On the use of the normalized mean square error in evaluating dispersion model performance. Atmos Environ Part A, Gen Top. https://doi.org/10.1016/0960-1686(93)90410-Z

    Article  Google Scholar 

  57. Smith G (1986) Probability and statistics in civil engineering. Collins, London

    Google Scholar 

  58. Golbraikh A, Shen M, Xiao Z et al (2003) Rational selection of training and test sets for the development of validated QSAR models. J Comput Aided Mol Des 17:241–253. https://doi.org/10.1023/A:1025386326946

    Article  Google Scholar 

  59. Roy PP, Roy K (2008) On some aspects of variable selection for partial least squares regression models. QSAR Comb Sci 27:302–313. https://doi.org/10.1002/qsar.200710043

    Article  Google Scholar 

  60. Frank I, Todeschini R (1994) The data analysis handbook

  61. Gandomi AH, Yun GJ, Alavi AH (2013) An evolutionary approach for modeling of shear strength of RC deep beams. Mater Struct Constr. https://doi.org/10.1617/s11527-013-0039-z

    Article  Google Scholar 

  62. Cheng MY, Firdausi PM, Prayogo D (2014) High-performance concrete compressive strength prediction using Genetic Weighted Pyramid Operation Tree (GWPOT). Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2013.11.014

    Article  Google Scholar 

  63. Alwanas AAH, Al-Musawi AA, Salih SQ et al (2019) Load-carrying capacity and mode failure simulation of beam-column joint connection: Application of self-tuning machine learning model. Eng Struct. https://doi.org/10.1016/j.engstruct.2019.05.048

    Article  Google Scholar 

  64. Chou JS, Tsai CF, Pham AD, Lu YH (2014) Machine learning in concrete strength simulations: Multi-nation data analytics. Constr Build Mater. https://doi.org/10.1016/j.conbuildmat.2014.09.054

    Article  Google Scholar 

  65. Sadat Hosseini A, Hajikarimi P, Gandomi M et al (2021) Genetic programming to formulate viscoelastic behavior of modified asphalt binder. Constr Build Mater. https://doi.org/10.1016/j.conbuildmat.2021.122954

    Article  Google Scholar 

  66. Nguyen TT, Pham Duy H, Pham Thanh T, Vu HH (2020) Compressive Strength Evaluation of Fiber-Reinforced High-Strength Self-Compacting Concrete with Artificial Intelligence. Adv Civ Eng. https://doi.org/10.1155/2020/3012139

    Article  Google Scholar 

  67. Sultana N, Zakir Hossain SM, Alam MS, et al (2020) Soft computing approaches for comparative prediction of the mechanical properties of jute fiber reinforced concrete. Adv Eng Softw 149:. https://doi.org/10.1016/j.advengsoft.2020.102887

  68. Willmott CJ (1981) On the validation of models. Phys Geogr. https://doi.org/10.1080/02723646.1981.10642213

    Article  MATH  Google Scholar 

  69. Nash JE, Sutcliffe JV (1970) River flow forecasting through conceptual models part I - A discussion of principles. J Hydrol. https://doi.org/10.1016/0022-1694(70)90255-6

    Article  Google Scholar 

  70. Gupta HV, Kling H, Yilmaz KK, Martinez GF (2009) Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling. J Hydrol. https://doi.org/10.1016/j.jhydrol.2009.08.003

    Article  Google Scholar 

  71. Knoben WJM, Freer JE, Woods RA (2019) Technical note: Inherent benchmark or not? Comparing Nash-Sutcliffe and Kling-Gupta efficiency scores. Hydrol Earth Syst Sci. https://doi.org/10.5194/hess-23-4323-2019

  72. Cheng MY, Chou JS, Roy AFV, Wu YW (2012) High-performance Concrete Compressive Strength Prediction using Time-Weighted Evolutionary Fuzzy Support Vector Machines Inference Model. Autom Constr. https://doi.org/10.1016/j.autcon.2012.07.004

    Article  Google Scholar 

  73. Yaseen ZM, Deo RC, Hilal A et al (2018) Predicting compressive strength of lightweight foamed concrete using extreme learning machine model. Adv Eng Softw. https://doi.org/10.1016/j.advengsoft.2017.09.004

    Article  Google Scholar 

  74. Yang L, Qi C, Lin X et al (2019) Prediction of dynamic increase factor for steel fibre reinforced concrete using a hybrid artificial intelligence model. Eng Struct. https://doi.org/10.1016/j.engstruct.2019.03.105

    Article  Google Scholar 

  75. Qi C, Fourie A, Chen Q (2018) Neural network and particle swarm optimization for predicting the unconfined compressive strength of cemented paste backfill. Constr Build Mater. https://doi.org/10.1016/j.conbuildmat.2017.11.006

    Article  Google Scholar 

  76. Chou J-S, Chiu C-K, Farfoura M, Al-Taharwa I (2010) Optimizing the Prediction Accuracy of Concrete Compressive Strength Based on a Comparison of Data-Mining Techniques. J Comput Civ Eng. https://doi.org/10.1061/(asce)cp.1943-5487.0000088

    Article  Google Scholar 

  77. Deepa C, SathiyaKumari K, Sudha VP (2010) Prediction of the Compressive Strength of High Performance Concrete Mix using Tree Based Modeling. Int J Comput Appl. https://doi.org/10.5120/1076-1406

    Article  Google Scholar 

  78. Erdal HI (2013) Two-level and hybrid ensembles of decision trees for high performance concrete compressive strength prediction. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2013.03.014

    Article  Google Scholar 

  79. Yan K, Shi C (2010) Prediction of elastic modulus of normal and high strength concrete by support vector machine. Constr Build Mater. https://doi.org/10.1016/j.conbuildmat.2010.01.006

    Article  Google Scholar 

  80. Rafiei MH, Khushefati WH, Demirboga R, Adeli H (2017) Supervised Deep Restricted Boltzmann Machine for Estimation of Concrete. ACI Mater J 114:. https://doi.org/10.14359/51689560

  81. Yan K, Xu H, Shen G, Liu P (2013) Prediction of Splitting Tensile Strength from Cylinder Compressive Strength of Concrete by Support Vector Machine. Adv Mater Sci Eng. https://doi.org/10.1155/2013/597257

    Article  Google Scholar 

  82. Anoop Krishnan NM, Mangalathu S, Smedskjaer MM et al (2018) Predicting the dissolution kinetics of silicate glasses using machine learning. J Non Cryst Solids. https://doi.org/10.1016/j.jnoncrysol.2018.02.023

    Article  Google Scholar 

  83. Okuyucu H, Kurt A, Arcaklioglu E (2007) Artificial neural network application to the friction stir welding of aluminum plates. Mater Des. https://doi.org/10.1016/j.matdes.2005.06.003

    Article  Google Scholar 

  84. Lim CH, Yoon YS, Kim JH (2004) Genetic algorithm in mix proportioning of high-performance concrete. Cem Concr Res. https://doi.org/10.1016/j.cemconres.2003.08.018

    Article  Google Scholar 

  85. Haghdadi N, Zarei-Hanzaki A, Khalesian AR, Abedi HR (2013) Artificial neural network modeling to predict the hot deformation behavior of an A356 aluminum alloy. Mater Des. https://doi.org/10.1016/j.matdes.2012.12.082

    Article  Google Scholar 

  86. Golafshani EM, Behnood A (2019) Estimating the optimal mix design of silica fume concrete using biogeography-based programming. Cem Concr Compos 96:95–105. https://doi.org/10.1016/J.CEMCONCOMP.2018.11.005

    Article  Google Scholar 

  87. Naser MZ (2018) Deriving temperature-dependent material models for structural steel through artificial intelligence. Constr Build Mater 191:56–68. https://doi.org/10.1016/J.CONBUILDMAT.2018.09.186

    Article  Google Scholar 

  88. Naser MZ (2019) Properties and material models for modern construction materials at elevated temperatures. Comput Mater Sci 160:16–29. https://doi.org/10.1016/J.COMMATSCI.2018.12.055

    Article  Google Scholar 

  89. Mousavi SM, Aminian P, Gandomi AH et al (2012) A new predictive model for compressive strength of HPC using gene expression programming. Adv Eng Softw. https://doi.org/10.1016/j.advengsoft.2011.09.014

    Article  Google Scholar 

  90. Gandomi AH, Alavi AH, Sahab MG (2010) New formulation for compressive strength of CFRP confined concrete cylinders using linear genetic programming. Mater Struct Constr. https://doi.org/10.1617/s11527-009-9559-y

    Article  Google Scholar 

  91. Mollahasani A, Alavi AH, Gandomi AH (2011) Empirical modeling of plate load test moduli of soil via gene expression programming. Comput Geotech. https://doi.org/10.1016/j.compgeo.2010.11.008

    Article  Google Scholar 

  92. Erdal HI, Karakurt O, Namli E (2013) High performance concrete compressive strength forecasting using ensemble models based on discrete wavelet transform. Eng Appl Artif Intell. https://doi.org/10.1016/j.engappai.2012.10.014

    Article  Google Scholar 

  93. Pang B, Lee L, Vaithyanathan S (2002) Thumbs up? In: Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP ’02

  94. Galdi P, Tagliaferri R (2017) Data Mining: Accuracy and Error Measures for Classification and Prediction. In: Encyclopedia of Bioinformatics and Computational Biology

  95. Valença J, Gonçalves LMS, Júlio E (2013) Damage assessment on concrete surfaces using multi-spectral image analysis. Constr Build Mater. https://doi.org/10.1016/j.conbuildmat.2012.11.061

    Article  Google Scholar 

  96. Huang H, Burton HV (2019) Classification of in-plane failure modes for reinforced concrete frames with infills using machine learning. J Build Eng. https://doi.org/10.1016/j.jobe.2019.100767

    Article  Google Scholar 

  97. Azimi SM, Britz D, Engstler M et al (2018) Advanced steel microstructural classification by deep learning methods. Sci Rep. https://doi.org/10.1038/s41598-018-20037-5

    Article  Google Scholar 

  98. Hore S, Chatterjee S, Sarkar S, et al (2016) Neural-based prediction of structural failure of multistoried RC buildings. Struct Eng Mech. https://doi.org/10.12989/sem.2016.58.3.459

  99. Bhowan U, Johnston M, Zhang M (2012) Developing new fitness functions in genetic programming for classification with unbalanced data. IEEE Trans Syst Man, Cybern Part B Cybern. https://doi.org/10.1109/TSMCB.2011.2167144

    Article  Google Scholar 

  100. Boughorbel S, Jarray F, El-Anbari M (2017) Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric. PLoS ONE. https://doi.org/10.1371/journal.pone.0177678

    Article  Google Scholar 

  101. Tharwat A (2018) Classification assessment methods. Appl. Comput. Informatics

  102. Caruana R, Niculescu-Mizil A (2004) Data mining in metric space: an empirical analysis of supervised learning performance criteria. In: KDD-2004 - Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

  103. Jurman G, Riccadonna S, Furlanello C (2012) A comparison of MCC and CEN error measures in multi-class prediction. PLoS ONE. https://doi.org/10.1371/journal.pone.0041882

    Article  Google Scholar 

  104. Powers DMW (2011) Evaluation: From Precision, Recall and F-Factor to ROC, Informedness, Markedness & Correlation. J Mach Learn Technol. 10.1.1.214.9232

  105. Bradley AP (1997) The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognit. https://doi.org/10.1016/S0031-3203(96)00142-2

    Article  Google Scholar 

  106. Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. https://doi.org/10.1148/radiology.143.1.7063747

    Article  Google Scholar 

  107. Zhang Y, Burton HV, Sun H, Shokrabadi M (2018) A machine learning framework for assessing post-earthquake structural safety. Struct Saf. https://doi.org/10.1016/j.strusafe.2017.12.001

    Article  Google Scholar 

  108. Davis J, Goadrich M (2006) The relationship between Precision-Recall and ROC curves. In: Proceedings of the 23rd international conference on Machine learning - ICML ’06

  109. Bi, J.; Bennett KPP (2003) Regression Error Characteristic Curves. Proc Twent Int Conf Mach Learn

  110. Zhang M, Smart W (2006) Using Gaussian distribution to construct fitness functions in genetic programming for multiclass object classification. Pattern Recognit Lett. https://doi.org/10.1016/j.patrec.2005.07.024

    Article  Google Scholar 

  111. Kocher M, Savoy J (2017) Distance measures in author profiling. Inf Process Manag. https://doi.org/10.1016/j.ipm.2017.04.004

    Article  Google Scholar 

  112. Patel BV (2012) Content Based Video Retrieval Systems. Int J UbiComp. https://doi.org/10.5121/iju.2012.3202

    Article  Google Scholar 

  113. Giusti R, Batista GEAPA (2013) An empirical comparison of dissimilarity measures for time series classification. In: Proceedings - 2013 Brazilian Conference on Intelligent Systems, BRACIS 2013

  114. Vuk M, Curk T (2006) ROC Curve , Lift Chart and Calibration Plot. Metod Zv

  115. Brodersen KH, Ong CS, Stephan KE, Buhmann JM (2010) The balanced accuracy and its posterior distribution. In: Proceedings - International Conference on Pattern Recognition

  116. Cohen J (1960) A Coefficient of Agreement for Nominal Scales. Educ Psychol Meas. https://doi.org/10.1177/001316446002000104

    Article  Google Scholar 

  117. Artstein R, Poesio M (2008) Inter-coder agreement for computational linguistics. Comput. Linguist.

  118. Destercke S (2014) Multilabel Prediction with Probability Sets: The Hamming Loss Case. In: Communications in Computer and Information Science

  119. Czajkowski M, Kretowski M (2019) Decision Tree Underfitting in Mining of Gene Expression Data. An Evolutionary Multi-Test Tree Approach. Expert Syst Appl. https://doi.org/10.1016/J.ESWA.2019.07.019

  120. Devarriya D, Gulati C, Mansharamani V, et al (2019) Unbalanced Breast Cancer Data Classification Using Novel Fitness Functions in Genetic Programming. Expert Syst Appl 112866. https://doi.org/10.1016/J.ESWA.2019.112866

  121. Bhaskar H, Hoyle DC, Singh S (2006) Machine learning in bioinformatics: A brief survey and recommendations for practitioners. Comput Biol Med. https://doi.org/10.1016/j.compbiomed.2005.09.002

    Article  Google Scholar 

  122. Kohavi R (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. Proc 14th Int Jt Conf Artif Intell - Vol 2

  123. Alavi AH, Gandomi AH (2011) Prediction of principal ground-motion parameters using a hybrid method coupling artificial neural networks and simulated annealing. Comput Struct. https://doi.org/10.1016/j.compstruc.2011.08.019

    Article  Google Scholar 

  124. Kingston GB, Maier HR, Lambert MF (2005) Calibration and validation of neural networks to ensure physically plausible hydrological modeling. J Hydrol. https://doi.org/10.1016/j.jhydrol.2005.03.013

    Article  Google Scholar 

  125. Kuo YL, Jaksa MB, Lyamin AV, Kaggwa WS (2009) ANN-based model for predicting the bearing capacity of strip footing on multi-layered cohesive soil. Comput Geotech. https://doi.org/10.1016/j.compgeo.2008.07.002

    Article  Google Scholar 

  126. Caruana R, Niculescu-Mizil A (2006) An empirical comparison of supervised learning algorithms. In: ACM International Conference Proceeding Series. ACM Press, New York, USA, pp 161–168

  127. Williams N, Zander S, Armitage G (2006) A preliminary performance comparison of five machine learning algorithms for practical IP traffic flow classification. Comput Commun Rev. https://doi.org/10.1145/1163593.s1163596

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Civil and Environmental Engineering and Earth Sciences, Clemson University, Clemson, SC, 29634, USA

    M. Z. Naser

  2. Artificial Intelligence Research Institute for Science and Engineering (AIRISE) at Clemson University, Clemson, SC, 29634, USA

    M. Z. Naser

  3. Department of Civil and Environmental Engineering, University of Pittsburgh, Pittsburgh, PA, 15261, USA

    Amir H. Alavi

Authors
  1. M. Z. Naser
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amir H. Alavi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Z. Naser.

Ethics declarations

Conflict of interest

none.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Naser, M.Z., Alavi, A.H. Error Metrics and Performance Fitness Indicators for Artificial Intelligence and Machine Learning in Engineering and Sciences. Archit. Struct. Constr. 3, 499–517 (2023). https://doi.org/10.1007/s44150-021-00015-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s44150-021-00015-8

Keywords

Search

Navigation