Article

The "magic number 5": is it enough for web testing?

Authors:

Dennis WixonAuthors Info & Claims

CHI EA '03: CHI '03 Extended Abstracts on Human Factors in Computing Systems

Pages 698 - 699

https://doi.org/10.1145/765891.765936

Published: 05 April 2003 Publication History

Get Access

Abstract

Common practice holds that 80% of usability findings are discovered after five participants. Recent findings from web testing indicate that a much larger number of participants is required to get results and that independent teams testing the same web-based product do not replicate results. How many users are enough for web testing?

References

[1]

Kessner, M., Wood, J, Dillon, R.F., and West, R.L. On the reliability of usability testing. CHI 2001. Poster.

Digital Library

Google Scholar

[2]

Lewis, J.R. Sample sizes for usability studies: Additional considerations. Human Factors 36, 368--378 (1994).

Crossref

Google Scholar

[3]

Medlock, M.C., Wixon, D., Terrano, M., Romero, R., Fulton, B. (2002). Using the RITE method to improve products: a definition and a case study. Proc. Usability Professionals Association (Orlando FL, July 2002).

Google Scholar

[4]

Molich, R., et al. Comparative evaluation of usability tests. CHI 99 Extended Abstracts, ACM Press, 83--84.

Digital Library

Google Scholar

[5]

Nielsen, J. Why you only need to test with 5 users. Alertbox (2000) www.useit.com/alertbox/2000319.html

Google Scholar

[6]

Nielsen, J., and Landauer, T.K. A Mathematical Model of the Finding of Usability Problems, in Proc INTERCHI '93, 206--213

Digital Library

Google Scholar

[7]

Spool, J., and Schroeder, W. Testing web sites: Five users is nowhere near enough. CHI 2001 Extended Abstracts, ACM Press, 285--286.

Digital Library

Google Scholar

[8]

Virzi, R. Refining the test phase of usability evaluation: How many subjects is enough? Human Factors 34, 457--468 (1992).

Digital Library

Google Scholar

[9]

Woolrych, A., and Cockton, G. Why and When Five Test Users aren't Enough, in Proc. IHM-HCI Conference: Volume 2, 105--108, 2001.

Google Scholar

Cited By

View all

Marcilly RMonkman HPelayo SLesselroth B(2024)Usability Evaluation Ecological Validity: Is More Always Better?Healthcare10.3390/healthcare1214141712:14(1417)Online publication date: 16-Jul-2024
https://doi.org/10.3390/healthcare12141417
Richards JGolightly DPalacin R(2024)Cognitive Principles for Remote Condition Monitoring Applied to a Rail Pantograph SystemApplied Sciences10.3390/app1413580114:13(5801)Online publication date: 3-Jul-2024
https://doi.org/10.3390/app14135801
HUHN AGERUSA RIBEIRO SOUZA DGUSMAO TMACEDO KDERECH RSANTOS BD’AMOREIRA M(2024)Radiological Protection Management: software implementation in a hospital in south BrazilGestão da Proteção Radiológica: implementação de software em um hospital do sul do BrasilBrazilian Journal of Radiation Sciences10.15392/2319-0612.2024.247112:3A (Suppl.)(e2471)Online publication date: 24-Sep-2024
https://doi.org/10.15392/2319-0612.2024.2471
Show More Cited By

The "magic number 5": is it enough for web testing?
1. Human-centered computing
  1. Human computer interaction (HCI)
  2. Interaction design
    1. Interaction design process and methods

Recommendations

PhotoShop 5 Web Magic
Concurrent or Retrospective Thinking Aloud in Usability Tests: A Meta-Analytic Review
In usability tests, the users are commonly asked to think aloud to let the evaluator listen in on their thoughts. Two variants of this procedure involve that the users either think aloud while using the tested product (concurrent thinking aloud, CTA) or ...
A usability study of websites for older travelers
UAHCI'13: Proceedings of the 7th international conference on Universal Access in Human-Computer Interaction: user and context diversity - Volume 2

Older people (aged 45+) are the fastest growing segment of the population, make up a significant percentage of overseas travelers, and often book their travel online. Despite these trends, several studies indicate users' frustration with online travel ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

CHI EA '03: CHI '03 Extended Abstracts on Human Factors in Computing Systems

April 2003

471 pages

ISBN:1581136374

DOI:10.1145/765891

Conference Chairs:
Gilbert Cockton
University of Sunderland, UK
,
Panu Korhonen
Nokia Research Center, Finland

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 April 2003

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

CHI03

Sponsor:

CHI03: Human Factors in Computing Systems

April 5 - 10, 2003

Florida, Ft. Lauderdale, USA

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
3,694
Total Downloads

Downloads (Last 12 months)96
Downloads (Last 6 weeks)5

Reflects downloads up to 11 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Marcilly RMonkman HPelayo SLesselroth B(2024)Usability Evaluation Ecological Validity: Is More Always Better?Healthcare10.3390/healthcare1214141712:14(1417)Online publication date: 16-Jul-2024
https://doi.org/10.3390/healthcare12141417
Richards JGolightly DPalacin R(2024)Cognitive Principles for Remote Condition Monitoring Applied to a Rail Pantograph SystemApplied Sciences10.3390/app1413580114:13(5801)Online publication date: 3-Jul-2024
https://doi.org/10.3390/app14135801
HUHN AGERUSA RIBEIRO SOUZA DGUSMAO TMACEDO KDERECH RSANTOS BD’AMOREIRA M(2024)Radiological Protection Management: software implementation in a hospital in south BrazilGestão da Proteção Radiológica: implementação de software em um hospital do sul do BrasilBrazilian Journal of Radiation Sciences10.15392/2319-0612.2024.247112:3A (Suppl.)(e2471)Online publication date: 24-Sep-2024
https://doi.org/10.15392/2319-0612.2024.2471
Exss KLuna-Navarro A(2024)Usability of Personalized Thermal Control Systems by people with intellectual disabilities in energy povertyBuilding and Environment10.1016/j.buildenv.2024.112018(112018)Online publication date: Aug-2024
https://doi.org/10.1016/j.buildenv.2024.112018
Faber JAl-Dhahir IKraal JBreeman Lvan den Berg-Emons RReijnders Tvan Dijk SJanssen VKraaijenhagen RVisch VChavannes NEvers A(2023)Guide development for eHealth interventions targeting people with a low socioeconomic position: A participatory design approach. (Preprint)Journal of Medical Internet Research10.2196/48461Online publication date: 26-Apr-2023
https://doi.org/10.2196/48461
Trivedi RHirayama SRisbud RSuresh MHumber MButler KRazze ATimko CNelson KZulman DAsch SHumphreys KPiette J(2023)Adapting a Telephone-Based, Dyadic Self-management Program to Be Delivered Over the Web: Methodology and Usability TestingJMIR Formative Research10.2196/439037(e43903)Online publication date: 16-Jun-2023
https://doi.org/10.2196/43903
Chowdhury AAhtinen AWu CVäänänen KTaidi DPieters R(2023)Exploring the Personality Design Space of Robots : Personalities and Design Implications for Non-Anthropomorphic Wellness Robots2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN57019.2023.10309438(2344-2351)Online publication date: 28-Aug-2023
https://doi.org/10.1109/RO-MAN57019.2023.10309438
Shahrabadi SRodrigues JMargolis IEvangelista LSousa NSousa EGuevara Lopéz MMagalhães LPeres EAdão T(2023)Digital Tools for Aircraft Maintenance: Prototyping Location-Aware AOI for Engine Assessment and Cable Routing Solutions2023 International Conference on Graphics and Interaction (ICGI)10.1109/ICGI60907.2023.10452715(1-11)Online publication date: 2-Nov-2023
https://doi.org/10.1109/ICGI60907.2023.10452715
Nativel-Fontaine GLespinet-Najib VCazes RDupetit CDe Gasquet CChevrie MAïoun FOjeda L(2023)Exploration of the acceptability of different behaviors of an autonomous vehicle in so-called conflict situationsAccident Analysis & Prevention10.1016/j.aap.2023.107041186(107041)Online publication date: Jun-2023
https://doi.org/10.1016/j.aap.2023.107041
Dilmen KKert SUğraş T(2023)Children’s coding experiences in a block-based coding environment: a usability study on code.orgEducation and Information Technologies10.1007/s10639-023-11625-828:9(10839-10864)Online publication date: 8-Feb-2023
https://doi.org/10.1007/s10639-023-11625-8
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

PhotoShop 5 Web Magic

Concurrent or Retrospective Thinking Aloud in Usability Tests: A Meta-Analytic Review

A usability study of websites for older travelers