Abstract
In this paper, we present a new approach to diagnosis in student modeling based on the use of Bayesian Networks and Computer Adaptive Tests. A new integrated Bayesian student model is defined and then combined with an Adaptive Testing algorithm. The structural model defined has the advantage that it measures students' abilities at different levels of granularity, allows substantial simplifications when specifying the parameters (conditional probabilities) needed to construct the Bayesian Network that describes the student model, and supports the Adaptive Diagnosis algorithm. The validity of the approach has been tested intensively by using simulated students. The results obtained show that the Bayesian student model has excellent performance in terms of accuracy, and that the introduction of adaptive question selection methods improves its behavior both in terms of accuracy and efficiency.
Similar content being viewed by others
References
Birnbaum, A.: 1968, Some latent trait models and their use in inferring an examinee's mental ability. In: F. M. Lord and M. R. Novick (eds.), Statistical Theories of Mental Test Scores. Reading, MA: Addison-Wesley.
Bloom, B.: 1984, The 2 Sigma Problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher 13, 4–15.
Castillo, E., Gutiérrez, J. M. and Hadi, A.: 1997, Expert Systems and Probabilistic Network Models. New York: Springer Verlag.
Charniak, E.: 1991, Bayesian Networks Without Tears. AI Magazine 12(4), 50–63.
Collins, J. A., Greer, J. E. and Huang, S. H.: 1996, Adaptive assessment using granularity hierarchies and Bayesian nets. In: Lecture Notes in Computer Science: Vol. 1086. Proceedings of 3rd International Conference ITS'96, Berlin Heidelberg: Springer Verlag, pp. 569–577.
Conati, C., Gertner, A., VanLehn, K. and Druzdzel, M.: 1997, On-line student modelling for coached problem solving using Bayesian networks. Proceedings of the 6th International Conference on User Modelling UM'97, Vienna, New York: Springer Verlag, pp. 231–242.
Conati, C. and VanLehn, K.: 1996a, POLA: A student modeling framework for probabilistic on-line assessment of problem solving performance. Proceedings of the 5th International Conference on User Modeling UM'96, User Modeling Inc., pp. 75–82.
Flaugher, R.: 1990, Item pools. In H. Wainer (ed.), Computerized Adaptive Testing: A Primer. Hillsdale, NJ: Lawrence Erlbaum Associates Publishers.
Hambleton, R. K.: 1989, Principles and selected applications of item response Theory. In: R. L. Linn (ed.), Educational Measurement. New York: MacMillan.
Jameson, A.: 1996, Numerical uncertainty management in user and student modeling: An overview of systems and issues. User Modeling and User-Adapted Interaction 5, 193–251.
Kingsbury, G. and Weiss, D. J.: 1983, A comparison of IRT-based adaptive mastery testing and sequential mastery testing procedure. In: D. J. Weiss (ed.), New Horizons in Testing: Latent Trait Test Theory and Computerized Adaptive Testing. New York: Academic Press.
Martin, J. and Van Lehn, K.: 1995b, Student assessment using Bayesian nets. International Journal of Human-Computer Studies 42, 575–591.
Mayo, M. and Mitrovic, A.: 2000, Using a probabilistic student model to control problem difficulty. In: Lecture Notes in Computer Science, Proceedings of 3rd International Conference on Intelligent Tutoring Systems ITS'2000, Berlin Heidelberg: Springer Verlag, pp. 525–533.
Millán, E., Pérez-de-la-Cruz, J. L. and Suárez, E.: 2000, An adaptive Bayesian network for multilevel student modelling. In: Lecture Notes in Computer Science. Proceedings of 3rd International Conference on Intelligent Tutoring Systems ITS'2000, Berlin Heidelberg: Springer Verlag, pp. 534–543.
Mislevy, R. and Gitomer, D. H.: 1996, The role of probability-based inference in an intelligent tutoring system. User Modeling and User-Adapted Interaction 5, 253–282.
Mitrovic, A.: 1998, Experiences in implementing constraint-based modeling in SQL-Tutor. In: Lecture Notes in Computer Science: Vol. 1452. Intelligent Tutoring Systems. Proceedings of 4th International Conference ITS'98, Berlin Heidelberg: Springer Verlag, pp. 414–423.
Mitrovic, A., Brent, M., Mayo. M.: 2002, Using evaluation to shape ITS design: Results and experiences with SQL-tutor. User Modeling and User-Adapted Interaction, in this issue.
Murray, W.: 1998, A practical approach to Bayesian student modelling. In: Lecture Notes in Computer Science: Vol. 1452. Intelligent Tutoring Systems. Proceedings of 4th International Conference ITS'98, Berlin Heidelberg: Springer Verlag, pp. 424–433.
Ohlsson, S.: 1994, Constraint-based student modelling. In: J. E. Greer and G. McCalla (eds.), Student Modelling: The Key to Individualized Knowledge-Based Instruction. Vol. 125, Berlin Heidelberg: Springer Verlag, pp. 167–190.
Olea, J. and Ponsoda, V.: 1996, Tests adaptativos informatizados. In: J. Muñiz (ed.), Psicometría, Madrid: Universitas, pp. 731–783.
Pearl, J.: 1988, Probabilistic Reasoning in Expert Systems: Networks of Plausible Inference. San Francisco: Morgan Kaufmann Publishers, Inc.
Reye, J.: 1996, A belief net backbone for student modeling. In: Lecture Notes in Computer Science: Vol. 1086. Proceedings of 3rd International Conference ITS'96, Berlin Heidelberg: Springer Verlag, pp. 596–604.
Reye, J.: 1998, Two-phase updating of student models based on dynamic belief networks. In: B. P. Goettl, J.M. Half, C. L. Redfield and V. J. Shutte, (eds.), Lecture Notes in Computer Science: Vol. 1452. Intelligent Tutoring Systems. Proceedings of 4th International Conference ITS'98, Berlin Heidelberg: Springer Verlag, pp. 6–15.
Ríos, A., Millán, E., Trella, M., Pérez-de-la-Cruz, J. L. and Conejo, R.: 1999, Internet based evaluation system. In: Open Learning Environments: New Computational Technologies to Support Learning, Exploration and Collaboration. Proceedings of the 9th World Conference of Artificial Intelligence and Education AIED'99, Amsterdam: IOS Press, pp. 387–394.
Rudner, L.: 1998, An on-line, interactive, computer adaptive testing mini-tutorial. http://ericae.net/scripts/cat/catdemo.
Shute, V. J.: 1995, Intelligent tutoring systems: Past, present and future. In: D. Jonassen (ed.), Handbook of Research on Educational Communications and Technology. Scholastic Publications.
Stern, M., Beck, J. and Woolf, B. P.: 1996, Adaptation of problem presentation and feedback in an intelligent mathematics tutor. In: C. Frasson, G. Gauthier and A. Lesgold (eds.), Intelligent Tutoring Systems. New York: Springer Verlag, pp. 603–613.
Thissen, D. and Mislevy, R.: 1990, Testing algorithms. In: H. Wainer (ed.), Computerized Adaptive Testing: A Primer, Hillsdale, NJ: Lawrence Erlbaum Associates Publishers, pp. 103–136.
Van der Linden, W. and Hambleton, R.: 1997, Handbook of Modern Item Response Theory. New York: Springer Verlag.
Van Lehn, K.: 1988, Student modelling. In: M. C. Polson and J. J. Richardson (eds.), Foundations of Intelligent Tutoring Systems. Hillsdale, NJ: Lawrence Erlbaum Associates Publishers, pp. 55–76.
Van Lehn, K.: 1996, Conceptual and meta learning during coached problem solving. In: Lecture Notes in Computer Science: Vol. 1086. Proceedings of 3rd International Conference ITS'96, Berlin Heidelberg: Springer Verlag, pp. 29–47.
Van Lehn, K., Niu, Z., Siler, S. and Gertner, A. S.: 1998, Student modeling from conventional test data: A Bayesian approach without priors. In: Lecture Notes in Computer Science: Vol. 1452. Intelligent Tutoring Systems. Proceedings of 4th International Conference ITS'98, Berlin Heidelberg: Springer Verlag, pp. 434–443.
Van Lehn, K., Ohlsson, S. and Nason, R.: 1995, Applications of Simulated Students: An Exploration. Journal of Artificial Intelligence and Education 5(2), 135–175.
Wainer, H.: 1990, Computerized Adaptive Testing: a Primer. Hillsdale, NJ: Lawrence Erlbaum Associates.
Weiss, D. and Kingsbury, G.: 1984, Application of computerized adaptive testing to educational problems. Journal of Educational Measurement 12, 361–375.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Millán, E., Pérez-de-la-Cruz, J.L. A Bayesian Diagnostic Algorithm for Student Modeling and its Evaluation. User Modeling and User-Adapted Interaction 12, 281–330 (2002). https://doi.org/10.1023/A:1015027822614
Issue Date:
DOI: https://doi.org/10.1023/A:1015027822614