abstract

Evaluating the accessibility of crowdsourcing tasks on Amazon's mechanical turk

Authors:

Rocío Calvo,

Shaun K. Kane,

Amy HurstAuthors Info & Claims

ASSETS '14: Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibility

Pages 257 - 258

https://doi.org/10.1145/2661334.2661401

Published: 20 October 2014 Publication History

Get Access

Abstract

Crowd work web sites such as Amazon Mechanical Turk enable individuals to work from home, which may be useful for people with disabilities. However, the web sites for finding and performing crowd work tasks must be accessible if people with disabilities are to use them. We performed a heuristic analysis of one crowd work site, Amazon's Mechanical Turk, using the Web Content Accessibility Guidelines 2.0. This paper presents the accessibility problems identified in our analysis and offers suggestions for making crowd work platforms more accessible

References

[1]

Abberley, P. The significance of work for the citizenship of disabled people. Presented at University College Dublin, (1999), 1--18.

Google Scholar

[2]

Baowaly, M. K. and Bhuiyan, M. Accessibility analysis and evaluation of Bangladesh government websites. In Proc. ICIEV 2012 (2012) 46--51.

Google Scholar

[3]

Costa, D., Fernandes, N., Neves, S., Duarte, C., Hijón-Neira, R.,and Carriço, L. Web accessibility in Africa: a study of three African domains. In Proc. INTERACT '13, (2013), 331--338.

Crossref

Google Scholar

[4]

Gilbertson, T. and Machin, C.Guidelines, icons and marketable skills: an accessibility evaluation of 100 web development company homepages. In Proc. W4A '12, (2012), 17--20.

Digital Library

Google Scholar

[5]

Ipeirotis, P.G. Analyzing the Amazon Mechanical Turk Marketplace. XRDS 17, 2, (2010), 16--21.

Digital Library

Google Scholar

[6]

Khanna, S., Ratan, A., Davis, J. and Thies, W. Evaluating and improving the usability of Mechanical Turk for low-income workers in India. In Proc. DEV '10, (2010), 1--12.

Digital Library

Google Scholar

[7]

Kobayashi, M., Ishihara, T., Kosugi, A., Takagi, H., and Asakawa, C. Question-answer cards for an inclusive micro-tasking framework for the elderly. In Proc. INTERACT '13, (2013), 590--607.

Crossref

Google Scholar

[8]

Kruse, D., Schur, L. and Ali, M. Projecting potential demand for workers with disabilities. Monthly Labor Review, (2010), 1--20.

Google Scholar

[9]

U.S. Department of Labor. Persons with a disability: barriers to employment, types of assistance, and other labor-related issues. U.S. Bureau of Labor Statistics News Release (2012). http://www.bls.gov/news.release/archives/dissup_04242013.pdf

Google Scholar

[10]

W3C. Website accessibility conformance evaluation methodology 1.0. http://www.w3.org/TR/WCAG-EM/

Google Scholar

Cited By

View all

Rechkemmer A(2023)Fostering Data Worker Inclusion and Well-Being: Identifying Barriers and Designing InterventionsCompanion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing10.1145/3584931.3608917(425-428)Online publication date: 14-Oct-2023
https://dl.acm.org/doi/10.1145/3584931.3608917
Rechkemmer AYin M(2022)Understanding the Microtask Crowdsourcing Experience for Workers with Disabilities: A Comparative ViewProceedings of the ACM on Human-Computer Interaction10.1145/35551376:CSCW2(1-30)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555137
Randhawa SAhmad TChen JRaza AKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Karamad: A Voice-based Crowdsourcing Platform for Underserved PopulationsProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445417(1-15)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445417
Show More Cited By

Index Terms

Evaluating the accessibility of crowdsourcing tasks on Amazon's mechanical turk
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
2. Social and professional topics
  1. Professional topics
    1. Computing profession
      1. Assistive technologies
  2. User characteristics
    1. People with disabilities

Recommendations

Investigating the Accessibility of Crowdwork Tasks on Mechanical Turk
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Crowdwork can enable invaluable opportunities for people with disabilities, not least the work flexibility and the ability to work from home, especially during the current Covid-19 pandemic. This paper investigates how engagement in crowdwork tasks is ...
Understanding the Microtask Crowdsourcing Experience for Workers with Disabilities: A Comparative View
CSCW

Microtask crowdsourcing holds great potential as an employment opportunity with the flexibility and anonymity that individuals with disability may require. Though prior research has explored the accessibility of crowd work, the lived crowd work ...
Accessible Crowdwork?: Understanding the Value in and Challenge of Microtask Employment for People with Disabilities
CSCW '15: Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing

We present the first formal study of crowdworkers who have disabilities via in-depth open-ended interviews of 17 people (disabled crowdworkers and job coaches for people with disabilities) and a survey of 631 adults with disabilities. Our findings ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ASSETS '14: Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibility

October 2014

378 pages

ISBN:9781450327206

DOI:10.1145/2661334

General Chair:
Sri Kurniawan
University of California Santa Cruz, USA
,
Program Chair:
John Richards
IBM Watson Group, USA

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 October 2014

Check for updates

Author Tags

Qualifiers

Abstract

Funding Sources

Universidad Carlos III de Madrid

Conference

ASSETS '14

Sponsor:

SIGACCESS

ASSETS '14: The 16th International ACM SIGACCESS Conference on Computers and Accessibility

October 20 - 22, 2014

New York, Rochester, USA

Acceptance Rates

ASSETS '14 Paper Acceptance Rate 29 of 106 submissions, 27%;

Overall Acceptance Rate 436 of 1,556 submissions, 28%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
248
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Rechkemmer A(2023)Fostering Data Worker Inclusion and Well-Being: Identifying Barriers and Designing InterventionsCompanion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing10.1145/3584931.3608917(425-428)Online publication date: 14-Oct-2023
https://dl.acm.org/doi/10.1145/3584931.3608917
Rechkemmer AYin M(2022)Understanding the Microtask Crowdsourcing Experience for Workers with Disabilities: A Comparative ViewProceedings of the ACM on Human-Computer Interaction10.1145/35551376:CSCW2(1-30)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555137
Randhawa SAhmad TChen JRaza AKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Karamad: A Voice-based Crowdsourcing Platform for Underserved PopulationsProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445417(1-15)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445417
Uzor SJacques JDudley JKristensson PKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Investigating the Accessibility of Crowdwork Tasks on Mechanical TurkProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445291(1-14)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445291
Li QGajos KReinecke KHwang FMcGrenere JFlatla D(2018)Volunteer-Based Online Studies With Older Adults and People with DisabilitiesProceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3234695.3236360(229-241)Online publication date: 8-Oct-2018
https://dl.acm.org/doi/10.1145/3234695.3236360
Ibarra FKorovina OBaez MCasati FMarchese MCernuzzi LBarysheva G(2016)Tools Enabling Online Contributions by Older AdultsIEEE Internet Computing10.1109/MIC.2016.2020:5(58-65)Online publication date: 1-Sep-2016
https://dl.acm.org/doi/10.1109/MIC.2016.20

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Investigating the Accessibility of Crowdwork Tasks on Mechanical Turk

Understanding the Microtask Crowdsourcing Experience for Workers with Disabilities: A Comparative View

Accessible Crowdwork?: Understanding the Value in and Challenge of Microtask Employment for People with Disabilities