[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/1926180.1926195acmotherconferencesArticle/Chapter ViewAbstractPublication Pagesacm-devConference Proceedingsconference-collections
research-article

Evaluating and improving the usability of Mechanical Turk for low-income workers in India

Published: 17 December 2010 Publication History

Abstract

While platforms such as Amazon Mechanical Turk have generated excitement as a potential source of income in developing regions, to date there remains little evidence that such opportunities have transformed livelihoods for low-income workers. In this study, we analyze the usability barriers that prevent those with basic digital literacy skills from accomplishing simple tasks on Mechanical Turk. Based on our observations, we design new user interfaces that reduce the barriers to task comprehension and execution. Via a study of 49 low-income workers in urban India, we demonstrate that new design elements -- including simplified user interfaces, simplified task instructions, and language localization -- are absolutely necessary to enable low-income workers to participate in and earn money using Mechanical Turk. We synthesize our findings into a set of design recommendations, as well as a realistic analysis of the potential for microtasking sites to deliver supplemental income to lower-income communities.

References

[1]
M. Bernstein, G. Little, R. Miller, B. Hartmann, M. Ackerman, D. Karger, D. Crowell, and K. Panovich. Soylent: A word processor with a crowd inside. In Symposium on User Interface Software and Technology, 2010.
[2]
N. Eagle. txteagle: Mobile Crowdsourcing. In Internationalization, Design and Global Development, 2009.
[3]
B. Frei. Paid Crowdsourcing: Current State & Progress towards Mainstream Business Use. Smartsheet White Paper, Sep 2009.
[4]
P. Ipeirotis. Mechanical turk: Profitable or not? http://behind-the-enemy-lines.blogspot.com/2009/03/mechanical-turk-profitable-or-not.html, Mar 2009.
[5]
P. Ipeirotis. Demographics of Mechanical Turk. NYU Working Paper No. CEDER-10-01, Mar 2010.
[6]
Screen Digest. Market Intelligence Report. 2008.
[7]
TeamLease Services. Indian Labour Report 2007: The Youth Unemployability Crisis. http://www.teamlease.com/images/reports/Teamlease_LabourReport_2007.pdf, 2007.
[8]
I. Medhi, G. Menon, and K. Toyama. Challenges of computerized job-search in the developing world. In CHI, 2008.
[9]
I. Medhi, G. Nagasena, and K. Toyama. A Comparison of Mobile Money-Transfer UIs for Non-Literate and Semi-Literate Users. In CHI, 2009.
[10]
I. Medhi, A. Prasad, and K. Toyama. Optimal audio-visual representations for illiterate users. In World Wide Web Conference, 2007.
[11]
I. Medhi, A. Sagar, and K. Toyama. Text-free user interfaces for illiterate and semiliterate users. Information Technologies and International Development, 4(1), 2007.
[12]
I. Medhi and K. Toyama. Full-Context Videos for First-Time, Non-Literate PC Users. In Information and Communication Technologies and Development, 2007.
[13]
A. L. Ratan, S. Satpathy, L. Zia, K. Toyama, S. Blagsvedt, U. S. Pawar, and T. Subramaniam. Kelsa+: Digital Literacy for Low-Income Office Workers. In Information and Communication Technologies and Development, 2009.
[14]
J. Ross, L. Irani, M. S. Silberman, A. Zaldivar, and B. Tomlinson. Who are the Crowdworkers? Shifting Demographics in Amazon Mechanical Turk. In alt. CHI, 2010.
[15]
Samasource website. http://www.samasource.org/.
[16]
The World Bank. World Development Indicators. http://data.worldbank.org/indicator/FP.CPI.TOTL.ZG, 2010.
[17]
L. von Ahn, M. Blum, N. J. Hopper, and J. Langford. CAPTCHA: Using Hard AI Problems for Security. In EUROCRYPT, 2003.
[18]
J. Zittrain. Ubiquitous human computing. Philosophical Transactions of the Royal Society, 366, 2008.

Cited By

View all
  • (2024)Productivity or Equity? Tradeoffs in Volunteer Microtasking in Humanitarian OpenStreetMapProceedings of the ACM on Human-Computer Interaction10.1145/36373908:CSCW1(1-34)Online publication date: 26-Apr-2024
  • (2023)Cognitive Factors Affecting the Manufacturing Optimization Skills of Rural Indian BPO WorkersKnowledge10.3390/knowledge30400393:4(626-641)Online publication date: 9-Nov-2023
  • (2023)A User-Centric Evaluation of Smart Home Resolution Approaches for Conflicts Between RoutinesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35819977:1(1-35)Online publication date: 28-Mar-2023
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences
ACM DEV '10: Proceedings of the First ACM Symposium on Computing for Development
December 2010
228 pages
ISBN:9781450304733
DOI:10.1145/1926180
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 December 2010

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. ICT4D
  2. India
  3. Mechanical Turk
  4. microtasks

Qualifiers

  • Research-article

Conference

ACM DEV '10

Acceptance Rates

Overall Acceptance Rate 52 of 164 submissions, 32%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)13
  • Downloads (Last 6 weeks)5
Reflects downloads up to 03 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Productivity or Equity? Tradeoffs in Volunteer Microtasking in Humanitarian OpenStreetMapProceedings of the ACM on Human-Computer Interaction10.1145/36373908:CSCW1(1-34)Online publication date: 26-Apr-2024
  • (2023)Cognitive Factors Affecting the Manufacturing Optimization Skills of Rural Indian BPO WorkersKnowledge10.3390/knowledge30400393:4(626-641)Online publication date: 9-Nov-2023
  • (2023)A User-Centric Evaluation of Smart Home Resolution Approaches for Conflicts Between RoutinesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35819977:1(1-35)Online publication date: 28-Mar-2023
  • (2023)Supporting Requesters in Writing Clear Crowdsourcing Task Descriptions Through Computational Flaw AssessmentProceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581641.3584039(737-749)Online publication date: 27-Mar-2023
  • (2023)Viability of Mobile Forms for Population Health Surveys in Low Resource Areas2023 IEEE Global Humanitarian Technology Conference (GHTC)10.1109/GHTC56179.2023.10354577(242-245)Online publication date: 12-Oct-2023
  • (2022)Harnessing Collective Differences in Crowdsourcing Behaviour for Mass Photogrammetry of 3D Cultural HeritageJournal on Computing and Cultural Heritage 10.1145/356909016:1(1-23)Online publication date: 24-Dec-2022
  • (2022)Feeling Proud, Feeling Embarrassed: Experiences of Low-income Women with Crowd WorkProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501834(1-18)Online publication date: 29-Apr-2022
  • (2022)Evaluating the Robustness of Speech Evaluation Standards for the Crowd2022 14th International Conference on Quality of Multimedia Experience (QoMEX)10.1109/QoMEX55416.2022.9900894(1-6)Online publication date: 5-Sep-2022
  • (2021)Long-term user experience in software crowdsourcing platformProceedings of the XX Brazilian Symposium on Human Factors in Computing Systems10.1145/3472301.3484342(1-7)Online publication date: 18-Oct-2021
  • (2021)What Is Unclear? Computational Assessment of Task Clarity in CrowdsourcingProceedings of the 32nd ACM Conference on Hypertext and Social Media10.1145/3465336.3475109(165-175)Online publication date: 30-Aug-2021
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media