Article

Assessing usability evaluation methods on their effectiveness to elicit verbal comments from children subjects

Authors:

Ilse E. H. van Kesteren,

Mathilde M. Bekker,

Arnold P. O. S. Vermeeren,

Peter A. LloydAuthors Info & Claims

IDC '03: Proceedings of the 2003 conference on Interaction design and children

Pages 41 - 49

https://doi.org/10.1145/953536.953544

Published: 01 July 2003 Publication History

Get Access

Abstract

An exploratory study is described looking at children's ability to provide verbal comments in usability evaluation sessions. Six evaluation methods were applied to test an interactive toy by children aged 6 and 7. The results show that most verbal comments were gathered during Active Intervention sessions, by asking children questions during tasks. Unexpectedly, the Co-Discovery sessions were less successful, because children did not collaborate very well. Children also provided useful comments in the Thinking Aloud, Retrospection, and Peer Tutoring sessions. They could reflect on their actions at the end of Retrospection sessions, and were able to teach other children how to interact with the toy in Peer Tutoring sessions.

References

[1]

Acuff, D. (1997) What Kids Buy and Why, The Psychology of Marketing to Kids, The Free Press, New York, NY Chapter 5.

Google Scholar

[2]

Berk, L. E., (1997), Child Development. Allyn and Bacon, Massachusetts, USA.

Google Scholar

[3]

Benford, S., Bederson, B. B., et. al., (2000), Designing Storytelling Technologies to Encourage Collaboration Between Young Children. CHI Letters Vol. 2, issue 1, page 556--562.

Digital Library

Google Scholar

[4]

Dumas, J. R. and J. C. Redish. (1993). A Practical Guide to Usability Testing. Ablex Publishing Corporation, Norwood NJ.

Digital Library

Google Scholar

[5]

Hackman, G. S. & Biers, D. W. (1992). Team usability testing: Are two heads better than one? Proceedings of the Human Factors Society 36th Annual Meeting, 1205--1209. Santa Monica, CA: HFES.

Crossref

Google Scholar

[6]

Hanna, L., Risden, K., & Alexander, K. (1997, September + October). Guidelines for usability testing with children. Interactions, 4(5), 9--14.

Digital Library

Google Scholar

[7]

Hoysniemi, J., Hamalainen, P., Turkki, L., (2002) Using Peer Tutoring in Evaluating Usability of Physically Interactive Computer Game with Children. International Conference on Interaction Design and Children, Eindhoven, The Netherlands. August 2002, page 144--1526.

Google Scholar

[8]

Kemp, J. A. M. and Gelderen, van T. (1996) Co-discovery exploration: an informal method for iterative design of consumer products, In. Jordan, P. W., Thomas, B, Weerdmeester, B. A., and McClelland, I. L. (eds.) Usability Evaluation in Industry, London: Taylor and Francis, 139--146.

Google Scholar

[9]

Larkin, S. (2002) Usability, Jr. - How to run a successful Usability Test with Children. Newsletter of Usability Special Interest Group 2002, Vol. 8, No. 3. URL: http://www.stcsig.org/usability/newsletter/0201_usabilityjr.html seen at 3-6-02.

Google Scholar

[10]

Markopoulos, P., Bekker, M. M. (2002), How to compare usability testing methods with children participants, International conference of Interaction Design and Children. Eindhoven, The Netherlands. August 2002. Page 153--158.

Google Scholar

[11]

Norman, D. A. (1986), Cognitive Engineering, In: Norman, D. A. and Draper, S. W. (Eds.), User Centred System Design, Lawrence Erlbaum associates, Hillsdale, N.J., 31--61.

Google Scholar

[12]

Ohnemus, K. R. & Biers, D. W. (1993). Retrospective versus concurrent thinking-out-loud in usability testing. Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting, 1127--1131. Santa Monica, CA.

Crossref

Google Scholar

[13]

Rooden, M. J.(1998) Thinking about thinking aloud. In: M. A. HANSON * (eds.); Proceedings of the Annual Conference of the Ergonomics Society. (Cirencester (UK), I -4- 1998), Taylor & Francis, London (UK), 1998, p. 328--332. ISBN: 07484-0811-8.

Google Scholar

[14]

Wilson, C. (1998) Usability Techniques. Pros and cons of Co-participating in usability studies. Usability Interface. Vol. 4 No 4 April 1998.

Google Scholar

Cited By

View all

van Limpt-Broers HPostma MLouwerse M(2024)Measuring transformative virtual reality experiences in children’s drawingsMemory & Cognition10.3758/s13421-024-01575-yOnline publication date: 8-May-2024
https://doi.org/10.3758/s13421-024-01575-y
Hertzum M(2024)Concurrent or Retrospective Thinking Aloud in Usability Tests: A Meta-Analytic ReviewACM Transactions on Computer-Human Interaction10.1145/366532731:3(1-29)Online publication date: 17-May-2024
https://dl.acm.org/doi/10.1145/3665327
Wöbbekind LLorberg KMandl T(2023)Emma stop that, it's my turn now - Comparing Peer Tutoring and Thinking Aloud for Usability-Testing with Children in a school settingProceedings of Mensch und Computer 202310.1145/3603555.3608542(442-447)Online publication date: 3-Sep-2023
https://dl.acm.org/doi/10.1145/3603555.3608542
Show More Cited By

Assessing usability evaluation methods on their effectiveness to elicit verbal comments from children subjects
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Tiglo: Inhabiting Toys Through Expression Mirroring
TEI '20: Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction

One of the superpowers that children posses is the power of imagination. Imagination can be said to be a tool that aids in cognitive development and intellectual thinking. Imagination comes to the foreground when children play with their toys. Its often ...
Don't talk to strangers!: peer tutoring versus active intervention methodologies in interviewing children
CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing Systems

Digital products designed for children should be validated by children. When it comes to usability testing, not all the available methods which work well with adults are equally applicable with child participants. In our study, we investigated two ...
A vicious circle between children’s non-communicative smartphone use and loneliness: Parents cannot do much about it
Highlights
- A two-wave panel survey among parent–child pairs.
- Non-communicative smartphone ...
Abstract
Children are increasingly using their own smartphones for communicative and non-communicative purposes. In fact, studies showed that different ways of using the smartphone might influence loneliness, and as a consequence, loneliness ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

IDC '03: Proceedings of the 2003 conference on Interaction design and children

July 2003

164 pages

ISBN:158113732X

DOI:10.1145/953536

Editors:
Stuart MacFarlane
University of Central Lancashire, Preston
,
Tony Nicol
University of Central Lancashire, Preston, UK
,
Janet Read
University of Central Lancashire, UK
,
Linda Snape
University of Central Lancashire, Preston, UK

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2003

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

IDC03

Sponsor:

SIGCHI

IDC03: Interaction Design and Children

July 1 - 3, 2003

Preston, England

Acceptance Rates

Overall Acceptance Rate 172 of 578 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

41
Total Citations
View Citations
2,004
Total Downloads

Downloads (Last 12 months)32
Downloads (Last 6 weeks)1

Reflects downloads up to 10 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

van Limpt-Broers HPostma MLouwerse M(2024)Measuring transformative virtual reality experiences in children’s drawingsMemory & Cognition10.3758/s13421-024-01575-yOnline publication date: 8-May-2024
https://doi.org/10.3758/s13421-024-01575-y
Hertzum M(2024)Concurrent or Retrospective Thinking Aloud in Usability Tests: A Meta-Analytic ReviewACM Transactions on Computer-Human Interaction10.1145/366532731:3(1-29)Online publication date: 17-May-2024
https://dl.acm.org/doi/10.1145/3665327
Wöbbekind LLorberg KMandl T(2023)Emma stop that, it's my turn now - Comparing Peer Tutoring and Thinking Aloud for Usability-Testing with Children in a school settingProceedings of Mensch und Computer 202310.1145/3603555.3608542(442-447)Online publication date: 3-Sep-2023
https://dl.acm.org/doi/10.1145/3603555.3608542
van der Meulen AHartendorp MVoorn WHermans F(2023)Observing the computational concept of abstraction in blind and low vision learners using the Bee-bot and Blue-botComputer Science Education10.1080/08993408.2023.227223234:4(806-828)Online publication date: 16-Nov-2023
https://doi.org/10.1080/08993408.2023.2272232
Gauthier ABenton LBunting LHerbert ESumner EMavrikis MRevesz AVasalou A(2022)I Don't Usually Listen, I Read: How Different Learner Groups Process Game FeedbackProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517480(1-15)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517480
Wang YZhang YChen Y(2022)A Longitudinal Study of Children’s Perceived Usability: Methodology and MeasurementsInternational Journal of Human–Computer Interaction10.1080/10447318.2022.210006539:18(3604-3619)Online publication date: 31-Jul-2022
https://doi.org/10.1080/10447318.2022.2100065
Vasalou ABenton LIbrahim SSumner EJoye NHerbert E(2021)Do children with reading difficulties benefit from instructional game supports? Exploring children's attention and understanding of feedbackBritish Journal of Educational Technology10.1111/bjet.1314552:6(2359-2373)Online publication date: 8-Jul-2021
https://doi.org/10.1111/bjet.13145
Celepkolu MWiggins JGaldo ABoyer K(2021)Designing a visualization tool for children to reflect on their collaborative dialogueInternational Journal of Child-Computer Interaction10.1016/j.ijcci.2020.10023227:COnline publication date: 1-Mar-2021
https://dl.acm.org/doi/10.1016/j.ijcci.2020.100232
van Limpt - Broers HPostma MLouwerse M(2020)Creating Ambassadors of Planet Earth: The Overview Effect in K12 EducationFrontiers in Psychology10.3389/fpsyg.2020.54099611Online publication date: 7-Oct-2020
https://doi.org/10.3389/fpsyg.2020.540996
Hertzum M(2020)Usability Testing: A Practitioner's Guide to Evaluating the User ExperienceSynthesis Lectures on Human-Centered Informatics10.2200/S00987ED1V01Y202001HCI0451:1(i-105)Online publication date: 9-Mar-2020
https://doi.org/10.2200/S00987ED1V01Y202001HCI045
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Tiglo: Inhabiting Toys Through Expression Mirroring

Don't talk to strangers!: peer tutoring versus active intervention methodologies in interviewing children

A vicious circle between children’s non-communicative smartphone use and loneliness: Parents cannot do much about it