Integrating Affective Responses and Gamification into Early Reading Acquisition Software Applications
Authors: 
Amelia W. Cole, 
Denise T. Quesnel, Serkan Pekçetin, Diane Gromala, Heather O'Brien, 
Alissa N. Antle, 
Bernhard E. RieckeAuthors Info & Claims
Pages 73 - 85
Abstract
Sisu is a gamified learning application designed to assist school-aged children who are struggling to read. Sisu utilizes readily-available technology to promote learning at home, with unique elements tied to the learning experience: (1) a spelling game with (2) an empathic agent, and (3) a mini-game. The empathic agent utilizes a facial action coding system (FACS) to recognize core expressions of the child user and respond to the child's affect in-game. We anticipate that Sisu's accessible and affective nature will not only support children's emotional needs, but the addition of gamified elements will motivate them to practice reading and assist them in their learning objectives.
References
[1]
Alissa N. Antle, Min Fan, and Emily S. Cramer. 2015. PhonoBlocks: A Tangible System for Supporting Dyslexic Children Learning to Read. ACM Press, 533--538.
[2]
Richard Bartle. 1996. Hearts, clubs, diamonds, spades: Players who suit MUDs. Journal of MUD research 1, 1: 19.
[3]
Virginia W. Berninger. 1996. Reading And Writing Acquisition: A Developmental Neuropsychological Perspective. Westview Press, Boulder (Colo.).
[4]
Herbert Bless, Gerald L. Clore, Norbert Schwarz, Verena Golisano, Christina Rabe, and Marcus Wölk. 1996. Mood and the use of scripts: Does a happy mood really lead to mindlessness? Journal of personality and social psychology 71, 4: 665.
[5]
Marie-Line Bosse, Marie Josèphe Tainturier, and Sylviane Valdois. 2007. Developmental dyslexia: The visual attention span deficit hypothesis. Cognition 104, 2: 198--230.
[6]
Marie-Line Bosse and Sylviane Valdois. 2009. Influence of the visual attention+span on child reading performance: a cross-sectional study. Journal of Research in Reading 32, 2: 230--253.
[7]
Ivica Boticki, Jelena Baksa, Peter Seow, and Chee-Kit Looi. 2015. Usage of a mobile social learning platform with virtual badges in a primary school. Computers & Education 86: 120--136.
[8]
Shannon Brennan. 2009. Participation and Activity Limitation Survey 2006: facts on learning limitations. Statistics Canada, Ottawa.
[9]
Canadian Dyslexia Association. n.d. Voices of Dyslexia: Tragedies & Triumphs. Retrieved from http://www.dyslexiaassociation.ca/english/files/voices.pdf.
[10]
Karyn L. Carson, Gail T. Gillon, and Therese M. Boustead. 2013. Classroom Phonological Awareness Instruction and Literacy Outcomes in the First Year of School. Language Speech and Hearing Services in Schools 44, 2: 147.
[11]
Gwo-Dong Chen, Jih-Hsien Lee, Chin-Yeh Wang, Po-Yao Chao, Liang-Yi Li, and Tzung-Yi Lee. 2012. An empathic avatar in a computer-aided learning program to encourage and persuade learners. Journal of Educational Technology & Society 15, 2: 62.
[12]
James M. Clark and Allan Paivio. 1991. Dual Coding Theory and Education. Educational Psychology Review 3, 3: 149--210.
[13]
Piers Cornelissen, Alex Richardson, Alexandra Mason, Sue Fowler, and John Stein. 1995. Contrast sensitivity and coherent motion detection measured at photopic luminance levels in dyslexics and controls. Vision Research 35, 10: 1483--1494.
[14]
Emily Sarah Cramer. 2015. A Code of Many Colours: Rationale, Validation and Requirements for a Sound-Based Letter Colour-Code that Might Support Some Children with Dyslexia in Spelling Certain Words. Master's Thesis. Simon Fraser University, Vancouver, Canada.
[15]
Ronald D. Davis and Eldon M. Braun. 2010. The Gift of Dyslexia: Why Some of the Smartest People Can't Read and How They Can Learn. Penguin Books, New York, NY.
[16]
Stanislas Dehaene. 2010. Reading in the Brain: The New Science of How We Read. Penguin Books, New York, NY, USA.
[17]
Sebastian Deterding. 2012. Gamification: Designing for Motivation. interactions 19, 4: 14--17.
[18]
Abhinav Dhall, Roland Goecke, Jyoti Joshi, Karan Sikka, and Tom Gedeon. 2014. Emotion Recognition In The Wild Challenge 2014: Baseline, Data and Protocol. Proceedings of the 16th International Conference on Multimodal Interaction, ACM, 461--466.
[19]
Christo Dichev and Darina Dicheva. 2017. Gamifying education: what is known, what is believed and what remains uncertain: a critical review. International Journal of Educational Technology in Higher Education 14, 1: 9.
[20]
Katherine A. Dougherty Stahl. 2016. Response to Intervention. The Reading Teacher 69, 6: 659--663.
[21]
Linnea C. Ehri. 1987. Learning to read and spell words. Journal of Literacy Research 19, 1: 5--31.
[22]
Linnea C. Ehri. 2014. Orthographic Mapping in the Acquisition of Sight Word Reading, Spelling Memory, and Vocabulary Learning. Scientific Studies of Reading 18, 1: 5--21.
[23]
Paul Ekman and Erika L. Rosenberg, eds. 2005. What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System. Oxford University Press, Oxford?; New York.
[24]
Min Fan. 2017. Exploring the Design of a Tangible System Supported for Learning to Read in Children At-risk for Reading Difficulties. Ph.D. Dissertation. Simon Fraser University, Vancouver, Canada.
[25]
Min Fan, Alissa N. Antle, Maureen Hoskyn, Carman Neustaedter, and Emily S. Cramer. 2017. Why Tangibility Matters: A Design Case Study of At-Risk Children Learning to Read and Spell. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, ACM, 1805--1816.
[26]
Sandro Franceschini, Simone Gori, Milena Ruffino, Simona Viola, Massimo Molteni, and Andrea Facoetti. 2013. Action Video Games Make Dyslexic Children Read Better. Current Biology 23, 6: 462--466.
[27]
Tove Gerholm. 2011. Children's development of facework practices-An emotional endeavor. Journal of Pragmatics 43, 13: 3099--3110.
[28]
Erving Goffman. 1995. On Face-Work: An Analysis of Ritual Elements in Social Interaction. In B.G. Blount, ed., Language, Culture, and Society: A Book of Readings. Waveland Press, Long Grove, Illinois, 222--247.
[29]
Goren Gordon, Samuel Spaulding, Jacqueline Kory Westlund, et al. 2016. Affective Personalization of a Social Robot Tutor for Children's Second Language Skills. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI Press, 3951--3957.
[30]
Usha Goswami. 2015. Visual attention span deficits and assessing causality in developmental dyslexia. Nature Reviews Neuroscience 16, 225--225.
[31]
C. Shawn Green and Daphne Bavelier. 2003. Action video game modifies visual selective attention. Nature 423, 6939: 534--537.
[32]
Alexis Hiniker, Hyewon Suh, Sabina Cao, and Julie A. Kientz. 2016. Screen Time Tantrums: How Families Manage Screen Media Experiences for Toddlers and Preschoolers. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, ACM, 648--660.
[33]
Ashish Kapoor and Rosalind W. Picard. 2005. Multimodal Affect Recognition in Learning Environments. Proceedings of the 13th Annual ACM International Conference on Multimedia, ACM, 677--682.
[34]
Karl M. Kapp. 2012. Wiley: The Gamification of Learning and Instruction: Game-based Methods and Strategies for Training and Education. Pfieffer: An Imprint of John Wiley & Sons, New York, NY.
[35]
Heysem Kaya, Furkan Gürpinar, Sadaf Afshar, and Albert Ali Salah. 2015. Contrasting and Combining Least Squares Based Learners for Emotion Recognition in the Wild. Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, ACM, 459--466.
[36]
Jeff Kerssen-Griep, Jon A. Hess, and April R. Trees. 2003. Sustaining the desire to learn: Dimensions of perceived instructional facework related to student involvement and motivation to learn. Western Journal of Communication 67, 4: 357--381.
[37]
Jacqueline M. Kory Westlund, Marayna Martinez, Maryam Archie, Madhurima Das, and Cynthia Breazeal. 2016. A Study to Measure the Effect of Framing a Robot as a Social Agent or as a Machine on Children's Social Behavior. The Eleventh ACM/IEEE International Conference on Human Robot Interaction, IEEE Press, 459--460.
[38]
Muriel Lobier, Rachel Zoubrinetzky, and Sylviane Valdois. 2012. The visual attention span deficit in dyslexia is visual and not verbal. Cortex 48, 6: 768--773.
[39]
Marianne Martens. 2015. Reading and "Gamification": Joining Guilds, Earning Badges, and Leveling Up. Children and Libraries 12, 4: 19--25.
[40]
Duncan Milne. 2005. Teaching the brain to read. SK Publishing, Artarmon, N.S.W.; Auckland, N.Z.
[41]
Elisabeth Moores, Rizan Cassim, and Joel B. Talcott. 2011. Adults with dyslexia exhibit large effects of crowding, increased dependence on cues, and detrimental effects of distractors in visual search tasks. Neuropsychologia 49, 14: 3881--3890.
[42]
Mohammad Obaid, Charles Han, and Mark Billinghurst. 2008. "Feed the Fish": An Affect-aware Game. Proceedings of the 5th Australasian Conference on Interactive Entertainment, ACM, 6:1--6:6.
[43]
Heather O'Brien and Paul Cairns, eds. 2016. Why Engagement Matters: Cross-Disciplinary Perspectives of User Engagement in Digital Media. Springer.
[44]
Cheryl K. Olson. 2010. Children's motivations for video game play in the context of normal development. Review of General Psychology 14, 2: 180--187.
[45]
R.W. Picard. 1997. Affective Computing. MIT Press, Cambridge, Massachusetts.
[46]
Helmut Prendinger and Mitsuru Ishizuka. 2005. The Empathic Companion: A Character-Based Interface That Addresses Users' Affective States. Applied Artificial Intelligence 19, 3--4: 267--285.
[47]
Byron Reeves and Clifford Nass. 1996. The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places. CSLI Publications.
[48]
Luz Rello and Ricardo Baeza-Yates. 2013. Good fonts for dyslexia. ASSETS'13, ACM.
[49]
Kristen D. Ritchey and Jennifer L. Goeke. 2006. Orton-Gillingham and Orton-Gillingham--Based Reading Instruction: A Review of the Literature. The Journal of Special Education 40, 3: 171--183.
[50]
Jorges Simoes, Rebeca P. Díaz Redondo, and Ana Fernández Vilas. 2015. An Experiment to Assess Students' Engagement in a Gamified Social Learning Environment. eLearning Papers.
[51]
Elly Singer. 2008. Coping with academic failure, a study of Dutch children with dyslexia. Dyslexia 14, 4: 314--333.
[52]
Liane Smith and Arnold Wilkins. 2007. How many colours are necessary to increase the reading speed of children with visual stress? A comparison of two systems. Journal of Research in Reading 30, 3: 332--343.
[53]
Samuel Spaulding, Goren Gordon, and Cynthia Breazeal. 2016. Affect-Aware Student Models for Robot Tutors. Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, International Foundation for Autonomous Agents and Multiagent Systems, 864--872.
[54]
Statistics Canada. 2012. Learning disabilities among Canadians aged 15 years and older. Statistics Canada. Retrieved from Available: http://www.statcan.gc.ca/pub/89--654-x/89--654-x2014003-eng.htm.
[55]
Jaynelle F. Stichler. 2016. Research, Research-Informed Design, Evidence-Based Design: What Is the Difference and Does It Matter? HERD: Health Environments Research & Design Journal 10, 1: 7--12.
[56]
C-H. Su and C-H. Cheng. 2015. A mobile gamification learning system for improving the learning motivation and achievements. Journal of Computer Assisted Learning 31, 3: 268--286.
[57]
P. F. Vadasy, J. R. Jenkins, and K. Pool. 2000. Effects of Tutoring in Phonological and Early Reading Skills on Students at Risk for Reading Disabilities. Journal of Learning Disabilities 33, 6: 579--590.
[58]
Laura Jean Wells, Steven Mark Gillespie, and Pia Rotshtein. 2016. Identification of Emotional Facial Expressions: Effects of Expression, Intensity, and Sex on Eye Gaze. PLoS ONE 11, 12.
[59]
Kory Westlund, Jacqueline M, Sooyeon Jeong, et al. 2017. Flat vs. Expressive Storytelling: Young Children's Learning and Retention of a Social Robot's Narrative. Frontiers in Human Neuroscience 11.
[60]
Dmitri Williams, Nick Yee, and Scott E. Caplan. 2008. Who plays, how much, and why? Debunking the stereotypical gamer profile. Journal of Computer-Mediated Communication 13, 4: 993--1018.
[61]
Marco Zorzi, Chiara Barbiero, Andrea Facoetti, et al. 2012. Extra-large letter spacing improves reading in dyslexia. Proceedings of the National Academy of Sciences 109, 28: 11455--11459.
Index Terms
- Integrating Affective Responses and Gamification into Early Reading Acquisition Software Applications
Recommendations
Affective effects of gamification: using biosignals to measure the effects on working and learning users
PETRA '19: Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive EnvironmentsWhat emotional effects does gamification have on users who work or learn with repetitive tasks? In this work, we use biosignals to analyze these affective effects of gamification. After a brief discussion of related work, we describe the implementation ...
"Teens today don't read books anymore": a study of differences in interest and comprehension in multiple modalities
iConference '11: Proceedings of the 2011 iConferenceA research study on college students reading, engaging and comprehending across multiple modalities: print books, ebooks, and audiobooks.
Over the last several years the popular and scholarly presses have been rife with articles about how teens are not ...
Subjective Experience and Sociability in a Collaborative Serious Game
The meaning of social dimension of gaming for enjoyable and engaging game experiences is supported by an increasing amount of research. Nonetheless, this issue has rarely been studied in the context of serious games. This study attempts to narrow the ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2017
700 pages
ISBN:9781450351119
DOI:10.1145/3130859
- General Chairs:
- Ben Schouten,
- Panos Markopoulos,
- Phoebe O. Toups Dugas,
- Program Chairs:
- Paul Cairns,
- Tilde Bekker
Copyright © 2017 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 15 October 2017
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
CHI PLAY '17
Sponsor:
CHI PLAY '17: The annual symposium on Computer-Human Interaction in Play
October 15 - 18, 2017
Amsterdam, The Netherlands
Acceptance Rates
CHI PLAY '17 Extended Abstracts Paper Acceptance Rate 46 of 178 submissions, 26%;
Overall Acceptance Rate 421 of 1,386 submissions, 30%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 260Total Downloads
- Downloads (Last 12 months)21
- Downloads (Last 6 weeks)4
Reflects downloads up to 03 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in