Abstract
Recently, Adaboost has been compared to greedy backfitting of extended additive models in logistic regression problems, or “Logitboost". The Adaboost algorithm has been applied to learn fuzzy rules in classification problems, and other backfitting algorithms to learn fuzzy rules in modeling problems but, up to our knowledge, there are not previous works that extend the Logitboost algorithm to learn fuzzy rules in classification problems.
In this work, Logitboost is applied to learn fuzzy rules in classification problems, and its results are compared with that of Adaboost and other fuzzy rule learning algorithms. Contradicting the expected results, it is shown that the basic extension of the backfitting algorithm to learn classification rules may produce worse results than Adaboost does. We suggest that this is caused by the stricter requirements that Logitboost demands to the weak learners, which are not fulfilled by fuzzy rules. Finally, it is proposed a prefitting based modification of the Logitboost algorithm that avoids this problem
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Alpaydin E. (1999). Combined 5×2cv F test for comparing supervised classification learning algorithms. Neural Comput 11(8):1885–1982
Cordón O., Herrera F. (1997). A three-stage evolutionary process for learning descriptive and approximative fuzzy logic controller knowledge bases from examples. Int J Approx Reason 17(4):369–407
Del Jesus MJ., Hoffmann F., Junco L., Sánchez L. Induction of fuzzy rule based classifiers with evolutionary boosting algorithms. IEEE Trans Fuzzy Sets Syst (Admitted for publication)
Dietterich G. (1998). Approximate statistical tests for comparing supervised classification learning algorithms. Neural comput 10(7):1895–1924
Freund Y., Schapire R. (1996). Experiments with a new boosting algorithm. In: Machine learning, proceedings of the 13th international conference, pp 148–156
Friedman J., Hastie T., Tibshirani R. (2000). Additive logistic regression: a statistical view of boosting. Ann Stat 38(2):337–374
Gonzalez A., Perez R. (1996). Completeness and consistency conditions for learning fuzzy rules. Fuzzy Sets Syst 96:37–51
González A., Herrera F. (1997). Multi-stage genetic fuzzy systems based on the iterative rule learning approach. Mathware Soft Comput 4:233–249
Hand DJ. (1981). Discrimination and classification. Wiley, New York
Hastie TJ., Tibshirani R. (1986). Generalized additive models. Stat Sci 1:297–318
Haykin S. (1999). Neural networks. Prentice Hall, Englewood Cliffs
Hoffmann F. (2001). Boosting a genetic fuzzy classifier. In: Proceeding of joint 9th IFSA world congress and 20th NAFIPS international conference, vol 3. Vancouver, Canada, pp 1564–1569
Ishibuchi H. (1992). Distributed representation of fuzzy rules and its application to pattern classification. Fuzzy Sets Syst 52:21–32
Ishibuchi H., Nakashima T., Morisawa T. (1999). Voting in fuzzy rule-based systems for pattern classification problems. Fuzzy Sets Syst 103(2):223–239
Junco L., Sanchez L. (2000). Using the Adaboost algorithm to induce fuzzy rules in classification problems. In: Proceeding of ESTYLF 2000, Sevilla, pp 297–301
Kuncheva LI. (2000). Fuzzy Classifier design. Springer, Berlin Heidelberg New York
Mallat S., Zhang Z. (1993). Matching pursuits with time-frequency dictionaries. IEEE Trans Signal Process 41:3397–3415
Merz CJ, Murphy PM (1998) UCI repository of machine learning databases. Available at: http://www.ics.uci.edu/mlearn/MLRepository.html
Pal SK., Mandal DP. (1992). Linguistic recognition system based in approximate reasoning. Inf Sci 61:135–161
Sánchez L. (2001). A fast genetic method for inducting linguistically understandable fuzzy models. In: Proceeding of IFSA NAFIPS, pp 1559–1563
Sánchez L., Casillas J., Cordón O., del Jesus MJ. (2002). Some relationships between fuzzy and random classifiers and models. Int J Approx Reason 29:175–213
Schapire R., Singer Y. (1999). Improved boosting algorithms using confidence-rated predictions. Mach Learn 37(3):297–336
Vincent P., Bengio Y. (2002). Kernel matching pursuit, Machine Learning Journal, Special Issue on New Methods for Model Combination and Model Selection, pp 165–187
Wang LX., Mendel J. (1992). Generating fuzzy rules by learning from examples. IEEE Trans Syst Man Cybern 25(2):353–361
Zhu J., Hastie T. (2001). Kernel logistic regression and the import vector machine. In: Proceeding of NIPS 2001. Vancouver, Canada, pp 1081–1088
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Otero, J., Sánchez, L. Induction of descriptive fuzzy classifiers with the Logitboost algorithm. Soft Comput 10, 825–835 (2006). https://doi.org/10.1007/s00500-005-0011-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-005-0011-0