Abstract
This paper describes methods for reasoning with unknown, irrelevant, and not-applicable meta-values when learning concept descriptions from examples or discovering patterns in data. These types of meta-values represent different reasons for which regular values are not available, thus require different treatment in both rule learning and rule testing. The presented methods are handled internally, within the learning and testing algorithms, and not in preprocessing as those widely described in the literature. They have been implemented in the AQ21 multitask learning and knowledge discovery program, and experimentally tested on three real world and one designed datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this article
We’re sorry, something doesn't seem to be working properly.
Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
We made the following distinction between concept learning and pattern discovery: in concept learning, one seeks a general concept descriptions that account for all concept examples in the training data; while in pattern discovery one seeks strong or “interesting” regularities in the data.
References
Allison, P. D. (2001). Missing data. Sage Publications, Inc.
Bruha, I. (2004). Meta-learner for unknown attribute values processing: Dealing with inconsistency of meta-databases. Journal of Intelligent Information Systems, 22(1), 71–87.
Bruha, I., & Kockova, S. (1994). A support for decision making: Cost-sensitive learning system. Artificial Intelligence in Medicine, 6, 67–82.
Bruha, I., & Franek, F. (1996). Comparison of various routines for unknown attribute value processing: The covering paradigm. International Journal of Pattern Recognition and Artificial Intelligence, 10(8), 939–955.
Clark, P., & Niblett, T. (1989). The CN2 induction algorithm. Machine Learning, 3(4), 261–283.
De Tre, G., De Caluve, R., & Prade, H. (2008). Null values in fuzzy databases. Journal of Intelligent Information Systems, 30(2), 93–14.
Engels, J. M., & Diehr, P. (2003). Imputation of missing longitudinal data: A comparison of methods. Journal of Clinical Epidemiology, 56, 968–976.
Fürnkranz, J. (1999) Separate-and conquer rule learning. Artificial Intelligence Review, 13, 3–54.
Greiner, R., Grove, A. J., & Kogan, A. (1997). Knowing what doesn’t matter: Exploring the omission of irrelevant data. Artificial Intelligence, 97(1–2), 345–380.
Grużdź, A., Ihantowicz, A., & Ślȩzak, D. (2005). Gene expression clustering: Dealing with the missing values. In Proceedings of the intelligent information processing and Web mining conference, IIPWM 05. Gdansk, Poland, 13–16 June.
Grzymala-Busse, J. W. (2003). Rough set strategies to data with missing attribute values. In Proceedings of the workshop on foundation and new directions in data mining. Melbourne, FL, USA.
Grzymala-Busse, J. W. (2004). Three approaches to missing attribute values - A rough set perspective. In Workshop on foundations of data mining, in conjunction with the fourth int. conference on data mining (pp. 55–62), Brighton, U.K., 1–4 November.
Grzymala-Busse, J. W., & Hu, M. (2000). A comparison of several approaches to missing attribute values in data mining. In Proceedings of the second international conference on rough sets and current trends in computing, RSCTC 2000. Banff, Canada.
Grzymala-Busse, J. W., & Grzymala-Busse, W. J. (2005). Handling missing attribute values. The data mining and knowledge discovery handbook (pp. 37–57).
Holt, B., & Benfer, R. A. Jr. (2000). Estimating missing data: An iterative regression approach. Journal of Human Evolution, 39, 289–296.
Junninen, H., Niska, H., Tuppurainen, K., Ruuskanen, J., & Kolehmainen, M. (2004). Methods of imputation of missing values in air quality data sets. Atmospheric Environment, 28, 2895–2907.
Kryszkiewicz, M. (1998). Rough set approach to incomplete information systems. Information Sciences, 112, 39–49.
Lakshminarayan, K., Harp, S. A., Goldman, R., & Samad, T. (1996). Imputation of missing data using machine learning techniques. In Proceedings of the second international conference on knowledge discovery & data mining. Portland, OR.
Larson, J., & Michalski, R. S. (1977). Inductive inference of VL decision rules. Invited paper for the workshop in pattern-directed inference systems, Hawaii, published in SIGART Newsletter, ACM, No. 63 (pp. 38–44). June 1977, 23–27 May.
Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd Edn.). John Wiley & Sons.
Michalski, R. S. (1969). On the quasi-minimal solution of the general covering problem. In Proceedings of the 5th international symposium on information processing, FCIP 69 (Vol. A3, Switching Circuits), Yugoslavia, Bled, 8–11 October.
Michalski, R. S. (1971). A geometric model for the synthesis of interval covers. Department of Computer Science, Report No. 461, University of Illinois, Urbana, Illinois.
Michalski, R. S. (1973). AQVAL/1–Computer implementation of a variable-valued logic system VL1 and examples of its application to pattern recognition. In Proceedings of the first international joint conference on pattern recognition (pp. 3–17). Washington, DC.
Michalski, R. S. (1975). Synthesis of optimal and quasi-optimal variable-valued logic formulas. In Proceedings of the 1975 international symposium on multiple-valued logic (pp. 76–87). Bloomington, IN.
Michalski, R. S. (1980). Pattern recognition as rule-guided inductive inference. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-2(4), 349–361.
Michalski, R. S. (1983). A theory and methodology of inductive learning. In R. S. Michalski, T. J. Carbonell, & T. M. Mitchell (Eds.), Machine learning: An artificial intelligence approach (pp. 83–134). Palo Alto: TIOGA Publishing Co.
Michalski, R. S. (2004). ATTRIBUTIONAL CALCULUS: A logic and representation language for natural induction. Reports of the Machine Learning and Inference Laboratory, MLI 04-2, George Mason University, Fairfax, VA.
Michalski, R. S., & Chilausky, R. (1980). Knowledge acquisition by encoding expert rules versus computer induction from examples: A case study involving soybean pathology. International Journal for Man-Machine Studies, 12, 63–87.
Michalski, R. S., & Kaufman, K. (2001). The AQ19 system for machine learning and pattern discovery: A general description and user’s guide. Reports of the Machine Learning and Inference Laboratory, MLI 01-2, George Mason University, Fairfax, VA.
Michalski, R. S., Kaufman, K. A., Pietrzykowski, J., Sniezynski, B., & Wojtusiak, J. (2005). Learning user models for computer intrusion detection: Results from a preliminary study using natural induction approach. Reports of the Machine Learning and Inference Laboratory, George Mason University, Fairfax, VA (to appear).
Michalski, R. S., & Larson, J. (1978). Selection of most representative training examples and incremental generation of VL1 hypotheses: The underlying methodology and the description of programs ESEL and AQ11. Report No. 867, Department of Computer Science, University of Illinois, Urbana.
Michalski, R. S., & Pietrzykowski, J. (2007). iAQ: A program that discovers rules. AAAI-07 AI video competition at twenty-second conference on artificial intelligence (AAAI-07). British Columbia, Vancouver.
Michalski, R. S., & Wojtusiak, J. (2006). Reasoning with meta-values in AQ Learning. Reports of the Machine Learning and Inference Laboratory, MLI 05-1, George Mason University, Fairfax, VA.
Quinlan, J. R. (1989). Unknown attribute values in induction. In Proceedings of the 6th international workshop on machine learning, San Mateo, CA.
Quinlan, J. R. (1993). C4.5: Systems for machine learning. Morgan Kaufmann Publishers Inc.
Ragel, B., & Cremilleux, B. (1999). MVC - A preprocessing method to deal with missing values. Knowledge-Based Systems, 12, 285–289.
Satori, N., Salvan, A., & Thomaseth, K. (2005). Multiple imputation of missing values in cancer mortality analysis with estimated exposure dose. Computational Statistics & Data Analysis, 49(3), 937–953.
Wang, S. (2005). Classification with incomplete survey data: A Hopfield neural network approach. Computers and Operations Research, 32(10), 2583–2594.
Wnek, J., Kaufman, K., Bloedorn, E., & Michalski, R. S. (1996). Inductive learning system AQ15c: The method and user’s guide. Reports of the machine learning and inference laboratory, MLI 96-6, George Mason University Fairfax, VA.
Wojtusiak, J. (2004). AQ21 user’s guide. Reports of the machine learning and inference laboratory, MLI 04-3, George Mason University, Fairfax, VA.
Wojtusiak, J., Michalski, R. S., Kaufman, K., & Pietrzykowski, J. (2006). The AQ21 natural induction program for pattern discovery: Initial version and its novel features. In Proceedings of the 18th IEEE international conference on tools with artificial intelligence, Washington D.C., 13–15 November 2006.
Wu, X., & Barbara, D (2002). Learning missing values from summary constraints. SIGKDD Explorations, 4.
Acknowledgements
The authors thank Dr. Kenneth Kaufman for his useful comments on the earlier version of this paper, and for valuable suggestions regarding examples used to illustrate the methodology. Jarek Pietrzykowski helped to prepare data for experiments involving the computer users and ROBOTS datasets. This paper is a significantly modified and improved version of the Technical Report MLI-05-1 of Machine Learning and Inference Laboratory, George Mason University (Michalski and Wojtusiak 2006).
The authors thank anonymous reviewers that helped improve the manuscript, in particular sections concerned with research related to meta-values.
Research presented here was partially conducted at the Machine Learning and Inference Laboratory of George Mason University and partially at the Hanse Institute for Advanced Study in Delmenhorst and at the University of Bremen in the Collaborative Research Center 637. Research activities of the Machine Learning and Inference Laboratory have been supported in part by the National Science Foundation Grants No. IIS 9906858 and IIS 0097476, and in part by the UMBC/LUCITE #32 grant. The findings and opinions expressed here are those of the authors, and do not necessarily reflect those of the above sponsoring organizations.
Author information
Authors and Affiliations
Corresponding author
Additional information
This paper has been revised and submitted by the second author after death of Professor Ryszard S. Michalski in 2007.
Rights and permissions
About this article
Cite this article
Michalski, R.S., Wojtusiak, J. Reasoning with unknown, not-applicable and irrelevant meta-values in concept learning and pattern discovery. J Intell Inf Syst 39, 141–166 (2012). https://doi.org/10.1007/s10844-011-0186-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10844-011-0186-z