More Web Proxy on the site http://driver.im/

abstract

Crowdsourcing Virtual Reality Experiments using VRChat

Authors:

Caglar Yildirim,

Sara Di Bartolomeo,

Cody DunneAuthors Info & Claims

CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 8

https://doi.org/10.1145/3334480.3382829

Published: 25 April 2020 Publication History

Abstract

Research involving Virtual Reality (VR) headsets is becoming more and more popular. However, scaling VR experiments is challenging as researchers are often limited to using one or a small number of headsets for in-lab studies. One general way to scale experiments is through crowdsourcing so as to have access to a large pool of diverse participants with relatively little expense of time and money. Unfortunately, there is no easy way to crowdsource VR experiments. We demonstrate that it is possible to implement and run crowdsourced VR experiments using a pre-existing massively multiplayer online VR social platform - VRChat. Our small (n=10) demonstration experiment required participants to navigate a maze in VR. Participants searched for two targets then returned to the exit while we captured completion time and position over time. While there are some limitations with using VRChat, overall we have demonstrated a promising approach for running crowdsourced VR experiments.

References

[1]

Aryabrata Basu and Kyle Johnsen. 2018. Navigating a maze differently - a user study. CoRR abs/1805.09454 (2018). http://arxiv.org/abs/1805.09454

[2]

Michael Buhrmester, Tracy Kwang, and Samuel D. Gosling. 2011. Amazon's mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science 6, 1 (23 8 2011), 3--5.

[3]

Steam Charts. 2020. VRChat - Steam Charts. (5 Jan 2020). Retrieved 05 Jan 2020 from https://steamcharts.com/app/438100.

[4]

Jeffrey Heer and Michael Bostock. 2010. Crowdsourcing Graphical Perception: Using Mechanical Turk to Assess Visualization Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). 203--212.

Digital Library

[5]

Bernd Huber and Krzysztof Z. Gajos. 2020. Conducting online virtual environment experiments with uncompensated, unsupervised samples. PLOS ONE 15, 1 (01 2020), 1--17.

[6]

Kim Marriott, Falk Schreiber, Tim Dwyer, Karsten Klein, Nathalie Henry Riche, Takayuki Itoh, Wolfgang Stuerzlinger, and Bruce H. Thomas. 2018. Immersive Analytics. Springer. Google-Books-ID: vaVyDwAAQBAJ.

[7]

Aniket Kittur, Ed Chi, and Bongwon Suh. 2008. Crwodsourcing User Studies With Mechanical Turk. Proc. CHI 2008 (01 2008), 453--456.

Digital Library

[8]

Xiao Ma, Megan Cackett, Leslie Park, Eric Chien, and Mor Naaman. 2018. Web-Based VR Experiments Powered by the Crowd. In Proceedings of the 2018 World Wide Web Conference (WWW '18). 33--43.

Digital Library

[9]

Thomas De Moor. 2019. Getting Started with Udon. Medium. (26 Jan 2019). https://lab.onebonsai.com/social-vr-is-the-weird-future-of-social-media-2fedf4663011

[10]

Aske Mottelson and Kasper Hornbundefinedk. 2017. Virtual Reality Studies Outside the Laboratory. In Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology (VRST '17). Article Article 9, 10 pages.

Digital Library

[11]

Fares Moustafa and Anthony Steed. 2018. A Longitudinal Study of Small Group Interaction in Social Virtual Reality. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology (VRST '18). Article Article 22, 10 pages.

Digital Library

[12]

G. Paolacci, J. Chandler, and P.G. Ipeirotis. 2010. Running experiments on Amazon Mechanical Turk. Judgment and Decision Making 5, 5 (2010). https://ssrn.com/abstract=1626226

[13]

Eyal Peer, Laura Brandimarte, Sonam Samat, and Alessandro Acquisti. 2017. Beyond the Turk: Alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology 70 (2017), 153 -- 163.

[14]

David G. Rand. 2012. The promise of Mechanical Turk: How online labor markets can help theorists run behavioral experiments. Journal of Theoretical Biology 299 (2012), 172 -- 179. Evolution of Cooperation.

[15]

Joel Ross, Lilly Irani, M. Six Silberman, Andrew Zaldivar, and Bill Tomlinson. 2010. Who Are the Crowdworkers? Shifting Demographics in Mechanical Turk. In CHI '10 Extended Abstracts on Human Factors in Computing Systems (CHI EA '10). 2863--2872.

Digital Library

[16]

Danielle N. Shapiro, Jesse Chandler, and Pam A. Mueller. 2013. Using Mechanical Turk to Study Clinical Populations. Clinical Psychological Science 1, 2 (2013), 213--220.

[17]

VRChat. 2019. Control. Docs.VRChat. (8 Nov 2019). Retrieved November 2, 2019 from https://docs.vrchat.com/docs/controls.

[18]

@VRChat. 2019a. Getting Started with Udon. Medium. (13 Dec 2019). Retrieved January 5, 2020 from https://ask.vrchat.com/t/getting-started-with-udon/805.

[19]

@VRChat. 2019b. Thanks to our community for making 2018 #VRChat's best year yet! Tweet. (18 Jan 2019). Retrieved 02 Nov 2019 from https://twitter.com/VRChat/status/1086389685268635648.

[20]

@VRChat. 2019c. VRChat: Unity 2018, Networking, IK, Udon, and More. Medium. (8 Nov 2019). Retrieved November 2, 2019 from https://medium.com/vrchat/vrchat-unity-2018-networking-ik-udon-and-more-445688972335.

[21]

VRChat. 2020. VRChat SDK. (2020). https://docs.vrchat.com/docs/welcome-to-vrchat

[22]

Yalong Yang, Bernhard Jenny, Tim Dwyer, Kim Marriott, Haohui Chen, and Maxime Cordeil. 2018. Maps and Globes in Virtual Reality. Computer Graphics Forum 37, 3 (2018), 427--438.

Cited By

Batiuk AKulyk Y(2024)Застосування технології віртуальної реальності для соціального моделювання та їхньої взаємодіїScientific Bulletin of UNFU10.36930/4034071234:7(92-99)Online publication date: 21-Nov-2024
https://doi.org/10.36930/40340712
Trotta NLeccese AFiore PMeyerbröker K(2024)Virtual Reality: Characteristics and application in anxiety disorders and other clinical settingsRIVISTA DI PSICOLOGIA CLINICA10.3280/rpc2-2023oa15823(70-92)Online publication date: Feb-2024
https://doi.org/10.3280/rpc2-2023oa15823
Kronemberger VCherullo RAzevedo TPorcino TRaposo A(2024)Hand gestures for continuous locomotion and snap turn for VR ExperiencesSymposium on Virtual and Augmented Reality10.1145/3625008.3625029(243-247)Online publication date: 6-Jan-2024
https://doi.org/10.1145/3625008.3625029
Show More Cited By

Index Terms

Crowdsourcing Virtual Reality Experiments using VRChat
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. Usability testing
    2. Interaction paradigms
      1. Virtual reality
2. Information systems
  1. World Wide Web
    1. Web applications
      1. Crowdsourcing

Recommendations

Extending Virtual Reality Display Wall Environments Using Augmented Reality
SUI '19: Symposium on Spatial User Interaction

Two major form factors for virtual reality are head-mounted displays and large display environments such as CAVE®and the LCD-based successor CAVE2®. Each of these has distinct advantages and limitations based on how they’re used. This work explores ...
Web-Based VR Experiments Powered by the Crowd
WWW '18: Proceedings of the 2018 World Wide Web Conference

We build on the increasing availability of Virtual Reality (VR) devices and Web technologies to conduct behavioral experiments in VR using crowdsourcing techniques. A new recruiting and validation method allows us to create a panel of eligible ...
Virtual Fieldwork: Designing Augmented Reality Applications Using Virtual Reality Worlds
Virtual, Augmented and Mixed Reality
Abstract
AR technology continues to develop and is expected to be used in a much wider range of fields. However, existing head-mounted displays for AR are still inadequate for use in daily life. Therefore, we focused on using VR to develop AR services and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

4474 pages

ISBN:9781450368193

DOI:10.1145/3334480

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2020

Check for updates

Author Tags

Qualifiers

Abstract

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
1,286
Total Downloads

Downloads (Last 12 months)168
Downloads (Last 6 weeks)15

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Batiuk AKulyk Y(2024)Застосування технології віртуальної реальності для соціального моделювання та їхньої взаємодіїScientific Bulletin of UNFU10.36930/4034071234:7(92-99)Online publication date: 21-Nov-2024
https://doi.org/10.36930/40340712
Trotta NLeccese AFiore PMeyerbröker K(2024)Virtual Reality: Characteristics and application in anxiety disorders and other clinical settingsRIVISTA DI PSICOLOGIA CLINICA10.3280/rpc2-2023oa15823(70-92)Online publication date: Feb-2024
https://doi.org/10.3280/rpc2-2023oa15823
Kronemberger VCherullo RAzevedo TPorcino TRaposo A(2024)Hand gestures for continuous locomotion and snap turn for VR ExperiencesSymposium on Virtual and Augmented Reality10.1145/3625008.3625029(243-247)Online publication date: 6-Jan-2024
https://doi.org/10.1145/3625008.3625029
Wan HZhang JSuria AYao BWang DCoady YPrpa M(2024)Building LLM-based AI Agents in Social Virtual RealityExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651026(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3651026
Chen QCai JJacucci G(2024)“People are Way too Obsessed with Rank”: Trust System in Social Virtual RealityComputer Supported Cooperative Work (CSCW)10.1007/s10606-024-09498-7Online publication date: 4-May-2024
https://doi.org/10.1007/s10606-024-09498-7
Rossi MCiletti MScarinci AToto G(2023)Apprendere attraverso il metaverso e la realtà immersiva: nuove prospettive inclusiveLearning through the metaverse and immersive reality: new inclusive perspectivesIUL ResearchIUL Research10.57568/iulresearch.v4i7.4194:7(164-176)Online publication date: 19-Jun-2023
https://doi.org/10.57568/iulresearch.v4i7.419
Antoniou PPears MSchiza EFrangoudes FPattichis CWharrad HBamidis PKonstantinidis S(2023)Eliciting Co-Creation Best Practices of Virtual Reality Reusable e-ResourcesVirtual Worlds10.3390/virtualworlds20100052:1(75-89)Online publication date: 20-Mar-2023
https://doi.org/10.3390/virtualworlds2010005
Wu LLiu QZhao JLank E(2023)Interactions across Displays and Space: A Study of Virtual Reality Streaming Practices on TwitchProceedings of the ACM on Human-Computer Interaction10.1145/36264737:ISS(242-265)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1145/3626473
Delgado Rodriguez SRivu RMäkelä VAlt FWard JMcGill MMarky K(2023)Challenges in Virtual Reality Studies: Ethics and Internal and External ValidityProceedings of the Augmented Humans International Conference 202310.1145/3582700.3582716(105-111)Online publication date: 12-Mar-2023
https://dl.acm.org/doi/10.1145/3582700.3582716
Osborne AFielder SMcveigh-Schultz JLang TKreminski MButler GLi JSanchez DIsbister K(2023)Being Social in VR Meetings: A Landscape Analysis of Current ToolsProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3595959(1789-1809)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3595959
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents