Abstract
The Programme for International Student Assessment (PISA; e.g., OECD, Sample tasks from the PISA 2000 assessment, 2002a; OECD, Learning for tomorrow’s world: first results from PISA 2003, 2004; OECD, PISA 2006: Science competencies for tomorrow’s world, 2007; OECD, PISA 2009 Technical Report, 2012) is an international large scale assessment study that aims to assess the skills and knowledge of 15-year-old students, and based on the results, to compare education systems across the participating (about 70) countries (with a minimum number of approx. 4,500 tested students per country). Initiator of this Programme is the Organisation for Economic Co-operation and Development (OECD; www.pisa.oecd.org). We review the main methodological techniques of the PISA study. Primarily, we focus on the psychometric procedure applied for scaling items and persons. PISA proficiency scale construction and proficiency levels derived based on discretization of the continua are discussed. For a balanced reflection of the PISA methodology, questions and suggestions on the reproduction of international item parameters, as well as on scoring, classifying and reporting, are raised. We hope that along these lines the PISA analyses can be better understood and evaluated, and if necessary, possibly be improved.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Adams, R. J., Wilson, M., & Wang, W. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1–23.
De Boeck, P., & Wilson, M. (Eds.). (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York: Springer.
Dibello, L. V., Roussos, L. A., & Stout, W. F. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics (pp. 979–1030). Amsterdam: Elsevier.
Fischer, G. H., & Molenaar, I. W. (Eds.). (1995). Rasch models: Foundations, recent developments, and applications. New York: Springer.
Leisch, F. (2002). Sweave: Dynamic generation of statistical reports using literate data analysis. In W. Härdle & B. Rönz (Eds.), Compstat 2002—Proceedings in Computational Statistics (pp. 575–580). Heidelberg: Physica Verlag.
Mislevy, R. J. (1991). Randomization-based inference about latent variables from complex samples. Psychometrika, 56, 177–196.
OECD (2001). Knowledge and skills for life. First results from the OECD Programme for International Student Assessment (PISA) 2000. Paris: OECD Publishing.
OECD (2002a). Sample tasks from the PISA 2000 assessment. Paris: OECD Publishing.
OECD (2002b). PISA 2000 Technical Report. Paris: OECD Publishing.
OECD (2004). Learning for tomorrow’s world: First results from PISA 2003. Paris: OECD Publishing.
OECD (2005). PISA 2003 Technical Report. Paris: OECD Publishing.
OECD (2007). PISA 2006: Science competencies for tomorrow’s world. Paris: OECD Publishing.
OECD (2009a). PISA 2009 assessment framework: Key competencies in reading, mathematics and science. Paris: OECD Publishing.
OECD (2009b). PISA 2006 Technical Report. Paris: OECD Publishing.
OECD (2010). PISA 2009 results: Overcoming social background—Equity learning opportunities and outcomes. Paris: OECD Publishing.
OECD (2012). PISA 2009 Technical Report. Paris: OECD Publishing.
Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests. Chicago: University of Chicago Press.
Reckase, M. D. (2009). Multidimensional item response theory. New York: Springer.
Rupp, A. A., Templin, J. L., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. New York: The Guilford Press.
van der Linden, W. J., & Hambleton, R. K. (Eds.). (1997). Handbook of modern item response theory. New York: Springer.
von Davier, M. (2010). Hierarchical mixtures of diagnostic models. Psychological Test and Assessment Modeling, 52, 8–28.
von Davier, M., Gonzales, E., & Mislevy, R. J. (2009). What are plausible values and why are they useful? Issues and Methodologies in Large-Scale Assessments, 2, 9–37.
Wu, M. L., Adams, R. J., & Wilson, M. R. (1997). ConQuest®;: Multi-aspect test software [Computer program manual]. Camberwell: Australian Council for Educational Research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Ünlü, A., Kasper, D., Trendtel, M., Schurig, M. (2014). The OECD’s Programme for International Student Assessment (PISA) Study: A Review of Its Basic Psychometric Concepts. In: Spiliopoulou, M., Schmidt-Thieme, L., Janning, R. (eds) Data Analysis, Machine Learning and Knowledge Discovery. Studies in Classification, Data Analysis, and Knowledge Organization. Springer, Cham. https://doi.org/10.1007/978-3-319-01595-8_45
Download citation
DOI: https://doi.org/10.1007/978-3-319-01595-8_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-01594-1
Online ISBN: 978-3-319-01595-8
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)