research-article

Epistemology, pedagogy, assessment and learning analytics

Authors:

Simon Knight,

Simon Buckingham Shum,

Karen LittletonAuthors Info & Claims

LAK '13: Proceedings of the Third International Conference on Learning Analytics and Knowledge

Pages 75 - 84

https://doi.org/10.1145/2460296.2460312

Published: 08 April 2013 Publication History

Get Access

Abstract

There is a well-established literature examining the relationships between epistemology (the nature of knowledge), pedagogy (the nature of learning and teaching), and assessment. Learning Analytics (LA) is a new assessment technology and should engage with this literature since it has implications for when and why different LA tools might be deployed. This paper discusses these issues, relating them to an example construct, epistemic beliefs -- beliefs about the nature of knowledge -- for which analytics grounded in pragmatic, sociocultural theory might be well placed to explore. This example is particularly interesting given the role of epistemic beliefs in the everyday knowledge judgements students make in their information processing. Traditional psychological approaches to measuring epistemic beliefs have parallels with high stakes testing regimes; this paper outlines an alternative LA for epistemic beliefs which might be readily applied to other areas of interest. Such sociocultural approaches afford opportunity for engaging LA directly in high quality pedagogy.

References

[1]

Barzilai, S. and Zohar, A. 2012. Epistemic Thinking in Action: Evaluating and Integrating Online Sources. Cognition and Instruction. 30, 1 (2012), 39--85.

Crossref

Google Scholar

[2]

Black, P. and Wiliam, D. 2009. Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability. 21, 1 (2009), 5--31.

Crossref

Google Scholar

[3]

Black, P. and Wiliam, D. 2001. Inside the black box. BERA.

Google Scholar

[4]

Bromme, R. et al. 2009. Epistemological beliefs are standards for adaptive learning: a functional theory about epistemological beliefs and metacognition. Metacognition and Learning. 5, 1 (Dec. 2009), 7--26.

Google Scholar

[5]

Buckingham Shum, S. 2008. Cohere: Towards Web 2.0 argumentation. Proc. 2nd Int. Conf. Computational Models of Argument (28--30 May 2008, Toulouse), IOS Press. pp. 97--108.

Digital Library

Google Scholar

[6]

Buckingham Shum, S. 2012. Our Learning Analytics Are Our Pedagogy. Keynote Address, Expanding Horizons 2012 (Macquarie University, Oct. 2012).

Google Scholar

[7]

Buckingham Shum, S. and Ferguson, R. 2012. Social Learning Analytics. Educational Technology & Society. 15, 3 (2012), 3--26.

Google Scholar

[8]

Cunnane, S. 2011. The Danish gambit: online access, even during exams. Times Higher Education.

Google Scholar

[9]

Davis, A. 1998. 3: Understanding and Holism. Journal of Philosophy of Education. 32, 1 (1998), 41--55.

Crossref

Google Scholar

[10]

Davis, A. 2006. High Stakes Testing and the Structure of the Mind: A Reply to Randall Curren. Journal of Philosophy of Education. 40, 1 (2006), 1--16.

Crossref

Google Scholar

[11]

Davis, A. 2005. Learning and the Social Nature of Mental Powers. Educational Philosophy and Theory. 37, 5 (Sep. 2005), 635--647.

Crossref

Google Scholar

[12]

Davis, A. 1999. The Limits of Educational Assessment. Wiley.

Google Scholar

[13]

Davis, A. and Williams, K. 2002. Epistemology and curriculum. The Blackwell guide to the philosophy of education. N. Blake et al., eds. Blackwell Reference Online.

Google Scholar

[14]

DeBacker, T. K. et al. 2008. The Challenge of Measuring Epistemic Beliefs: An Analysis of Three Self-Report Instruments. The Journal of Experimental Education. 76, 3 (2008), 281--312.

Crossref

Google Scholar

[15]

Dede, C. 2008. A Seismic Shift in Epistemology. EDUCASE Review.

Google Scholar

[16]

Dewey, J. 1998. Experience and education. Kappa Delta Pi.

Google Scholar

[17]

Edwards, A. D. and Furlong, V. J. 1978. The language of teaching: Meaning in classroom interaction. Heinemann London.

Google Scholar

[18]

Edwards, D. and Mercer, N. 1987. Common knowledge: the development of understanding in the classroom. Routledge.

Google Scholar

[19]

Ferguson, L. E. et al. 2012. Epistemic cognition when students read multiple documents containing conflicting scientific evidence: A think-aloud study. Learning and Instruction. 22, 2 (Apr. 2012), 103--120.

Crossref

Google Scholar

[20]

Ferguson, R. et al. 2011. EnquiryBlogger: Using widgets to support awareness and reflection in a PLE setting. 1st Workshop on Awareness and Reflection in PLEs, Personal Learning Environments Conference (Southampton, 2011).

Google Scholar

[21]

Ferguson, R. and Buckingham Shum, S. 2012. Social Learning Analytics: Five Approaches. (Vancouver, BC, 2012).

Digital Library

Google Scholar

[22]

Foster, J. 2009. Understanding interaction in information seeking and use as a discourse: a dialogic approach. Journal of Documentation. 65, 1 (Jan. 2009), 83--105.

Crossref

Google Scholar

[23]

Gardner, J. 2011. Assessment and learning. SAGE.

Google Scholar

[24]

Greene, J. A. et al. 2010. The Role of Epistemic Beliefs in Students' Self-Regulated Learning With Computer-Based Learning Environments: Conceptual and Methodological Issues. Educational Psychologist. 45, 4 (2010), 245--257.

Crossref

Google Scholar

[25]

Hammer, D. and Elby, A. 2003. Tapping Epistemological Resources for Learning Physics. Journal of the Learning Sciences. 12, 1 (2003), 53--90.

Crossref

Google Scholar

[26]

Harlen, W. 2007. Assessment of learning. SAGE.

Google Scholar

[27]

Hertzum, M. 2008. Collaborative information seeking: The combined activity of information seeking and collaborative grounding. Information Processing & Management. 44, 2 (Mar. 2008), 957--962.

Digital Library

Google Scholar

[28]

Hofer, B. K. 2001. Personal epistemology research: Implications for learning and teaching. Educational Psychology Review. 13, 4 (2001), 353--383.

Crossref

Google Scholar

[29]

Hopmann, S. T. et al. eds. 2007. PISA According to PISA: Does PISA Keep What It Promises? Wien Lit-Verlag.

Google Scholar

[30]

Huffman, S. B. and Hochster, M. 2007. How well does result relevance predict session satisfaction? Proc. ACM SIGIR Conference on Research and Development in Information Retrieval (2007), 574.

Digital Library

Google Scholar

[31]

Katz, S. 2000. Competency, epistemology and pedagogy: curriculum's holy trinity. Curriculum Journal. 11, 2 (2000), 133--144.

Crossref

Google Scholar

[32]

Kelly, G. J. et al. 2008. What Counts as Knowledge in Educational Settings: Disciplinary Knowledge, Assessment, and Curriculum. Review of Research in Education. 32, 1 (Feb. 2008), vii--x.

Crossref

Google Scholar

[33]

King, P. M. and Kitchener, K. S. 2004. Reflective Judgment: Theory and Research on the Development of Epistemic Assumptions Through Adulthood. Educational Psychologist. 39, 1 (Mar. 2004), 5--18.

Crossref

Google Scholar

[34]

Kuhn, D. and Weinstock, M. 2002. What is epistemological thinking and why does it matter?. Personal epistemology: The psychology of beliefs about knowledge and knowing. B. K. Hofer and R. Pintrich, eds. Lawrence Erlbaum Associates. 121--144.

Google Scholar

[35]

Lazonder, A. W. 2005. Do two heads search better than one? Effects of student collaboration on web search behaviour and search outcomes. British Journal of Educational Technology. 36, 3 (May. 2005), 465--475.

Crossref

Google Scholar

[36]

De Liddo, A. et al. 2011. Discourse-centric learning analytics. Proceedings of the 1st International Conference on Learning Analytics and Knowledge (New York, NY, USA, 2011), 23--33.

Digital Library

Google Scholar

[37]

Lin, C. and Tsai, C. 2008. Exploring the Structural Relationships between High School Students' Scientific Epistemological Views and their Utilization of Information Commitments toward Online Science Information. International Journal of Science Education. 30, 15 (2008), 2001--2022.

Crossref

Google Scholar

[38]

Mason, L. et al. 2011. Epistemic beliefs in action: Spontaneous reflections about knowledge and knowing during online information searching and their influence on learning. Learning and Instruction. 21, 1 (Feb. 2011), 137--151.

Crossref

Google Scholar

[39]

Mason, L. et al. 2009. Epistemic metacognition in context: evaluating and learning online information. Metacognition and Learning. 5, 1 (Jul. 2009), 67--90.

Crossref

Google Scholar

[40]

Mercer, N. and Littleton, K. 2007. Dialogue and the Development of Children's Thinking: A Sociocultural Approach. Routledge.

Google Scholar

[41]

Olson, D. R. and Bruner, J. S. 1996. Folk psychology and folk pedagogy. The handbook of education and human development. (1996), 9--27.

Google Scholar

[42]

Österholm, M. 2010. Relationships Between Epistemological Beliefs and Properties of Discourse: Some Empirical Explorations. Madif 7, the 7th Swedish Mathematics Education Research Seminar (Stockholm, Sweden, 2010), 241--250.

Google Scholar

[43]

Österholm, M. 2009. Theories of epistemological beliefs and communication: A unifying attempt. Proceedings of the 33rd Conference of the International Group for the Psychology of Mathematics Education (2009), 275--264.

Google Scholar

[44]

Potter, J. and Edwards, D. 2003. Sociolinguistics, cognitivism and discursive psychology. IJES, International Journal of English Studies. 3, 1 (2003), 93--110.

Google Scholar

[45]

Rittel, H. and Kunz, W. 1970. Issues as elements of information systems. Working Paper No. 131. Institute of Urban and Regional Development.

Google Scholar

[46]

Säljö, R. 1999. Learning as the use of tools: A sociocultural perspective on the human-technology link. Learning With Computers: Analysing Productive Interaction. K. Littleton and P. Light, eds. Psychology Press.

Google Scholar

[47]

Säljö, R. 2012. Literacy, Digital Literacy and Epistemic Practices: The Co-Evolution of Hybrid Minds and External Memory Systems. Nordic Journal of Digital Literacy. 01 (2012), 5--19.

Google Scholar

[48]

Salomon, G. 1996. Distributed Cognitions: Psychological and Educational Considerations. Cambridge University Press.

Google Scholar

[49]

Scardamalia, M. and Bereiter, C. 2003. Knowledge Building. Encyclopedia of Education. Macmillan Reference. 1370--1373.

Google Scholar

[50]

Scardamalia, M. and Bereiter, C. 2006. Knowledge Building: Theory, Pedagogy, and Technology. Cambridge Handbook of the Learning Sciences. K. Sawyer, ed. Cambridge University Press. 97--118.

Google Scholar

[51]

Scherr, R. E. and Hammer, D. 2009. Student Behavior and Epistemological Framing: Examples from Collaborative Active-Learning Activities in Physics. Cognition and Instruction. 27, 2 (2009), 147--174.

Crossref

Google Scholar

[52]

Schommer, M. 1990. Effects of beliefs about the nature of knowledge on comprehension. Journal of Educational Psychology. 82, 3 (1990), 498--504.

Crossref

Google Scholar

[53]

Schraw, G. 2013. Conceptual Integration and Measurement of Epistemological and Ontological Beliefs in Educational Research. ISRN Education. (2013).

Google Scholar

[54]

Siegel, H. 1998. KNOWLEDGE, TRUTH AND EDUCATION. Education, knowledge, and truth: beyond the postmodern impasse. D. Carr, ed. Routledge. 19--36.

Google Scholar

[55]

Siemens, G. 2006. Knowing Knowledge. Lulu.com.

Google Scholar

[56]

Stadtler, M. and Bromme, R. 2007. Dealing with multiple documents on the WWW: The role of metacognition in the formation of documents models. International Journal of Computer-Supported Collaborative Learning. 2, 2 (2007), 191--210.

Crossref

Google Scholar

[57]

Tsai, C. 2004. Beyond cognitive and metacognitive tools: the use of the Internet as an "epistemological" tool for instruction. British Journal of Educational Technology. 35, 5 (Sep. 2004), 525--536.

Crossref

Google Scholar

[58]

Tsai, M. J. and Tsai, C. C. 2003. Information searching strategies in web-based science learning: the role of internet self-efficacy. Innovations in Education & Teaching International. 40, 1 (2003), 43--50.

Crossref

Google Scholar

[59]

Undervisningsministerie (Ministry of Education) and Afdelingen for Gymnasiale Uddannelser (Department of Secondary Education) 2010. Endelig rapport fra følgegruppen for forsøget med digitale prøver med adgang til internettet i udvalgte fag på stx og hhx (Final report of the Monitoring Group experiment with digital samples with access to Internet in selected subjects at stx and HHX). Undervisningsministerie (Ministry of Education).

Google Scholar

[60]

Walton, D. et al. 2008. Argumentation schemes. Cambridge University Press.

Google Scholar

[61]

Walton, D. 1995. Argumentation schemes for presumptive reasoning. Lawrence Erlbaum.

Google Scholar

[62]

Williams, K. 1998. Assessment and the challenge of scepticism. Education, knowledge, and truth: beyond the postmodern impasse. D. Carr, ed. Routledge

Google Scholar

Cited By

View all

Sahin MIfenthaler D(2024)Foundations of Assessment AnalyticsAssessment Analytics in Education10.1007/978-3-031-56365-2_1(3-17)Online publication date: 8-May-2024
https://doi.org/10.1007/978-3-031-56365-2_1
Banihashem SDehghanzadeh HClark DNoroozi OBiemans H(2023)Learning analytics for online game-Based learning: a systematic literature reviewBehaviour & Information Technology10.1080/0144929X.2023.2255301(1-28)Online publication date: 19-Sep-2023
https://doi.org/10.1080/0144929X.2023.2255301
Li QJung Yd'Anjou BWise AWise AMartinez-Maldonado RHilliger I(2022)Unpacking Instructors’ Analytics Use: Two Distinct Profiles for Informing TeachingLAK22: 12th International Learning Analytics and Knowledge Conference10.1145/3506860.3506905(528-534)Online publication date: 21-Mar-2022
https://dl.acm.org/doi/10.1145/3506860.3506905
Show More Cited By

Index Terms

Epistemology, pedagogy, assessment and learning analytics
1. Applied computing
  1. Education
    1. Collaborative learning

Recommendations

Social learning analytics: five approaches
LAK '12: Proceedings of the 2nd International Conference on Learning Analytics and Knowledge

This paper proposes that Social Learning Analytics (SLA) can be usefully thought of as a subset of learning analytics approaches. SLA focuses on how learners build knowledge together in their cultural and social settings. In the context of online social ...
Formative assessment and learning analytics
LAK '13: Proceedings of the Third International Conference on Learning Analytics and Knowledge

Learning analytics seeks to enhance the learning process through systematic measurements of learning related data, and informing learners and teachers of the results of these measurements, so as to support the control of the learning process. Learning ...
Moving Through MOOCS: Pedagogy, Learning Design and Patterns of Engagement
Design for Teaching and Learning in a Networked World
Abstract
Massive open online courses (MOOCs) are part of the lifelong learning experience of people worldwide. Many of these learners participate fully. However, the high levels of dropout on most of these courses are a cause for concern. Previous studies ...

Reviews

Reviewer: Rosa Michaelson

Learning analytics is a somewhat contested term and a fairly recent topic. Learning analytics includes the analysis of a wide range of data from computer systems as feedback for instructors, administrators, and students. The authors concentrate on the use of learning analytics for assessment, and strongly critique formal exams, with a view toward influencing any new assessment process. I found the paper difficult to read, as the argument is quite dense and the authors appear to regard philosophical terms as unproblematic. The difficulties of defining the focus of learning analytics research are outlined and illustrated with differing views and uses. For example, section 4.1 analyzes six educational stances, the associated forms of assessment, and the implications of each type for research. Furthermore, the authors question whether learning analytics should be related to formative and summative assessments or with what is termed "assessment for learning." Later, the authors introduce the concept of "epistemic beliefs" as a basis for thinking about learning analytics and assessment, although how these are to be captured from online data is another matter. Here, it would have been helpful if the authors had provided more concrete examples of the topics and domains of knowledge in which learning analytics are used. The one example of learning they do provide is that of information retrieval, where students learn to use a search engine. Such a task might require the capture of websites and keystrokes. However, is this the correct granularity for learning assessment, or is it even applicable to learning analytics__?__ The automated assessment of learning is not an easy objective. If, as suggested here, context is important, such data tells us nothing of the individual's state of mind as she learns. Toward the end of the paper, this is implicitly acknowledged, when the various possible assessment types that might result from learning analytics are noted as requiring human input. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

LAK '13: Proceedings of the Third International Conference on Learning Analytics and Knowledge

April 2013

300 pages

ISBN:9781450317856

DOI:10.1145/2460296

Editors:
Dan Suthers
University of Hawaii, USA
,
Katrien Verbert
Eindhoven University of Technology, The Netherlands
,
Erik Duval
University of Leuven, Belgium
,
Xavier Ochoa
Escuela Superior Politécnica del Litoral, Ecuador

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 April 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

LAK '13

Sponsor:

SIGWEB
K.U. Leuven
SIGCHI

LAK '13: Third Conference on Learning Analytics and Knowledge

April 8 - 13, 2013

Leuven, Belgium

Acceptance Rates

LAK '13 Paper Acceptance Rate 16 of 58 submissions, 28%;

Overall Acceptance Rate 236 of 782 submissions, 30%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
1,012
Total Downloads

Downloads (Last 12 months)43
Downloads (Last 6 weeks)1

Reflects downloads up to 04 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Sahin MIfenthaler D(2024)Foundations of Assessment AnalyticsAssessment Analytics in Education10.1007/978-3-031-56365-2_1(3-17)Online publication date: 8-May-2024
https://doi.org/10.1007/978-3-031-56365-2_1
Banihashem SDehghanzadeh HClark DNoroozi OBiemans H(2023)Learning analytics for online game-Based learning: a systematic literature reviewBehaviour & Information Technology10.1080/0144929X.2023.2255301(1-28)Online publication date: 19-Sep-2023
https://doi.org/10.1080/0144929X.2023.2255301
Li QJung Yd'Anjou BWise AWise AMartinez-Maldonado RHilliger I(2022)Unpacking Instructors’ Analytics Use: Two Distinct Profiles for Informing TeachingLAK22: 12th International Learning Analytics and Knowledge Conference10.1145/3506860.3506905(528-534)Online publication date: 21-Mar-2022
https://dl.acm.org/doi/10.1145/3506860.3506905
Barthakur AJoksimovic SKovanovic VCorbett FRichey MPardo A(2022)Assessing the sequencing of learning objectives in a study program using evidence-based practiceAssessment & Evaluation in Higher Education10.1080/02602938.2022.206497147:8(1429-1443)Online publication date: 19-Apr-2022
https://doi.org/10.1080/02602938.2022.2064971
Gašević DGreiff SShaffer D(2022)Towards strengthening links between learning analytics and assessmentComputers in Human Behavior10.1016/j.chb.2022.107304134:COnline publication date: 27-Jun-2022
https://dl.acm.org/doi/10.1016/j.chb.2022.107304
Khalil MPrinsloo PSlade S(2022)The use and application of learning theory in learning analytics: a scoping reviewJournal of Computing in Higher Education10.1007/s12528-022-09340-335:3(573-594)Online publication date: 1-Oct-2022
https://doi.org/10.1007/s12528-022-09340-3
Joksimović SDawson SBarthakur APoquet OWang YMarmolejo-Ramos FSiemens G(2022)Mapping the Landscape of Social and Emotional Learning AnalyticsSocial and Emotional Learning and Complex Skills Assessment10.1007/978-3-031-06333-6_3(27-47)Online publication date: 25-Aug-2022
https://doi.org/10.1007/978-3-031-06333-6_3
Banihashem KMacfadyen L(2021)Pedagogical Design: Bridging Learning Theory and Learning AnalyticsConception pédagogique : Rapprocher la théorie de l'apprentissage et l'analyse de l'apprentissageCanadian Journal of Learning and Technology10.21432/cjlt2795947:1Online publication date: 9-Aug-2021
https://doi.org/10.21432/cjlt27959
Berendt BLittlejohn ABlakemore M(2020)AI in education: learner choice and fundamental rightsLearning, Media and Technology10.1080/17439884.2020.1786399(1-13)Online publication date: 3-Jul-2020
https://doi.org/10.1080/17439884.2020.1786399
Celik DMikroyannidis AHlosta MThird ADomingue J(2019)ADA: A System for Automating the Learning Data Analytics Processing Life CycleTransforming Learning with Meaningful Technologies10.1007/978-3-030-29736-7_73(714-718)Online publication date: 9-Sep-2019
https://doi.org/10.1007/978-3-030-29736-7_73
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Social learning analytics: five approaches

Formative assessment and learning analytics

Moving Through MOOCS: Pedagogy, Learning Design and Patterns of Engagement

Reviews

Access critical reviews of Computing literature here