Abstract
A rule based model is a special type of computational models, which can be built by using expert knowledge or learning from real data. In this context, rule based modelling approaches can be divided into two categories: expert based approaches and data based approaches. Due to the vast and rapid increase in data, the latter approach has become increasingly popular for building rule based models. In machine learning context, rule based models can be evaluated in three main dimensions, namely accuracy, efficiency and interpretability. All these dimensions are usually affected by the key characteristic of a rule based model which is typically referred to as model complexity. This paper focuses on theoretical and empirical analysis of complexity of rule based models, especially for classification tasks. In particular, the significance of model complexity is argued and a list of impact factors against the complexity are identified. This paper also proposes several techniques for effective control of model complexity, and experimental studies are reported for presentation and discussion of results in order to analyze critically and comparatively the extent to which the proposed techniques are effective in control of model complexity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Liu, H., Gegov, A., Stahl, F.: Categorization and construction of rule based systems. In: 15th International Conference on Engineering Applications of Neural Networks, Sofia, Bulgaria (2014)
Furnkranz, J.: Separate-and-Conquer rule learning. Artif. Intell. Rev. 13, 3–54 (1999)
Liu, H., Gegov, A., Cocea, M.: Network based rule representation for knowledge discovery and predictive modelling. In: IEEE International Conference on Fuzzy Systems, Istanbul (2015)
Schaffer, C.: Overfitting avoidance as bias. Mach. Learn. 10, 153–178 (1993)
Wolpert, D.H.: On Overfitting Avoidance as Bias. SFI TR (1993)
Liu, H., Cocea, M., Gegov, A.: Interpretability of Computational Models for Sentiment Analysis. In: Pedrycz, W., Chen, S.M. (eds.) Sentiment Analysis and Ontology Engineering: An Environment of Computational Intelligence, vol. 639, pp. 199–220. Springer, Switzerland (2016)
Liu, H., Gegov, A., Stahl, F.: Unified framework for construction of rule based classification systems. In: Pedrycz, W., Chen, S.M. (eds.) Information Granularity, Big Data and Computational Intelligence, vol. 8, pp. 209–230. Springer (2015)
Liu, H., Gegov, A., Stahl, F.: J-measure based hybrid pruning for complexity reduction in classification rules. WSEAS Trans. Syst. 12(9), 433–446 (2013)
R. Quinlan, C4.5: Programs for Machine Learning. Morgan Kaufman (1993)
Cendrowska, J.: PRISM: an algorithm for inducing modular rules. Int. J. Man Mach. Stud. 27, 349–370 (1987)
Deng, X.: A Covering-Based Algorithm for Classification: PRISM, SK, 2012
Ross, Q.: Induction of decision trees. Mach. Learn. 1, 81–106 (1986)
Lichman, M.: UCI Machine Learning Repository. University of California, Irvine, School of Information and Computer Sciences, 2013. http://archive.ics.uci.edu/ml. Accessed 25 June 2015
Liu, H., Gegov, A., Cocea, M.: Rule Based Systems for Big Data: A Machine Learning Approach, vol. 13, 1 edn. Springer, Switzerland (2016)
Elomaa, T., Kaariainen, M.: An Analysis of Reduced Error Pruning. J. Artif. Intell. Res. 15(1), 163–187 (2001)
Hall, M.A.: Correlation-Based Feature Selection for. Hamilton, NewZealand (1999)
Jolliffe, I.T.: Principal Component Analysis. Springer, New York (2002)
Kerber, R.: ChiMerge: discretization of numeric attributes. In: Proceedings of the 10th National Conference on Artificial Intelligence, California (1992)
Ross, T.J.: Fuzzy Logic with Engineering Applications. Wiley, West Sussex (2004)
Liu, H., Gegov, A.: Induction of Modular Classification Rules by Information Entropy Based Rule Generation. In: Sgurev, V., Yager, R., Kacprzyk, J., Jotsov, V. (eds.) Innovative Issues in Intelligent Systems, vol. 623, pp. 217–230. Springer, Switzerland (2016)
Brain, D.: Learning from Large Data: Bias, Variance, and Learning Curves (2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Liu, H., Gegov, A., Cocea, M. (2017). Complexity Control in Rule Based Models for Classification in Machine Learning Context. In: Angelov, P., Gegov, A., Jayne, C., Shen, Q. (eds) Advances in Computational Intelligence Systems. Advances in Intelligent Systems and Computing, vol 513. Springer, Cham. https://doi.org/10.1007/978-3-319-46562-3_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-46562-3_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46561-6
Online ISBN: 978-3-319-46562-3
eBook Packages: EngineeringEngineering (R0)