More Web Proxy on the site http://driver.im/

research-article

Open access

ReLive: Bridging In-Situ and Ex-Situ Visual Analytics for Analyzing Mixed Reality User Studies

Authors:

Sebastian Hubenschmid,

Jonathan Wieland,

Daniel Immanuel Fink,

Johannes Zagermann,

Niklas Elmqvist,

Harald ReitererAuthors Info & Claims

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

Article No.: 24, Pages 1 - 20

https://doi.org/10.1145/3491102.3517550

Published: 28 April 2022 Publication History

All formats PDF

Abstract

The nascent field of mixed reality is seeing an ever-increasing need for user studies and field evaluation, which are particularly challenging given device heterogeneity, diversity of use, and mobile deployment. Immersive analytics tools have recently emerged to support such analysis in situ, yet the complexity of the data also warrants an ex-situ analysis using more traditional non-immersive visual analytics setups. To bridge the gap between both approaches, we introduce ReLive: a mixed-immersion visual analytics framework for exploring and analyzing mixed reality user studies. ReLive combines an in-situ virtual reality view with a complementary ex-situ desktop view. While the virtual reality view allows users to relive interactive spatial recordings replicating the original study, the synchronized desktop view provides a familiar interface for analyzing aggregated data. We validated our concepts in a two-step evaluation consisting of a design walkthrough and an empirical expert user study.

Supplementary Material

MP4 File (3491102.3517550-video-figure.mp4)

Video Figure

Download
245.99 MB

MP4 File (3491102.3517550-video-preview.mp4)

Video Preview

Download
18.75 MB

MP4 File (3491102.3517550-talk-video.mp4)

Talk Video

Download
273.52 MB

References

[1]

Marcus F. C. Alencar, Alberto B. Raposo, and Simone D. J. Barbosa. 2011. Composition of HCI Evaluation Methods for Hybrid Virtual Environments. In Proceedings of the ACM Symposium on Applied Computing. ACM, New York, NY, USA, 1237–1244. https://doi.org/10.1145/1982185.1982457

Digital Library

[2]

Sriram Karthik Badam, Andreas Mathisen, Roman Rädle, Clemens N. Klokmose, and Niklas Elmqvist. 2019. Vistrates: A Component Model for Ubiquitous Analytics. IEEE Transactions on Visualization and Computer Graphics 25, 1 (Jan. 2019), 586–596. https://doi.org/10.1109/TVCG.2018.2865144

Digital Library

[3]

Aaron Bangor, Philip Kortum, and James Miller. 2009. Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale. J. Usability Studies 4, 3 (May 2009), 114–123.

Digital Library

[4]

Andrea Batch, Andrew Cunningham, Maxime Cordeil, Niklas Elmqvist, Tim Dwyer, Bruce H. Thomas, and Kim Marriott. 2020. There Is No Spoon: Evaluating Performance, Space Use, and Presence with Expert Domain Users in Immersive Analytics. IEEE Transactions on Visualization and Computer Graphics 26, 1 (Jan. 2020), 536–546. https://doi.org/10.1109/TVCG.2019.2934803

[5]

Andrea Batch, Biswaksen Patnaik, Moses Akazue, and Niklas Elmqvist. 2020. Scents and Sensibility: Evaluating Information Olfactation. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376733

Digital Library

[6]

Martin Bellgardt, Sebastian Pick, Daniel Zielasko, Tom Vierjahn, Benjamin Weyers, and Torsten W. Kuhlen. 2017. Utilizing Immersive Virtual Reality in Everydaywork. In 2017 IEEE 3rd Workshop on Everyday Virtual Reality (WEVR). 1–4. https://doi.org/10.1109/WEVR.2017.7957708

[7]

Joanna Bergström, Tor-Salve Dalsgaard, Jason Alexander, and Kasper Hornbæk. 2021. How to Evaluate Object Selection and Manipulation in VR? Guidelines from 20 Years of Studies. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems(CHI ’21). Association for Computing Machinery, New York, NY, USA, 1–20. https://doi.org/10.1145/3411764.3445193

Digital Library

[8]

Christopher Berns, Grace Chin, Joel Savitz, Jason Kiesling, and Fred Martin. 2019. MYR: A Web-Based Platform for Teaching Coding Using VR. In Proceedings of the ACM Symposium on Computer Science Education. ACM, New York, NY, USA, 77–83. https://doi.org/10.1145/3287324.3287482

Digital Library

[9]

Verena Biener, Daniel Schneider, Travis Gesslein, Alexander Otte, Bastian Kuth, Per Ola Kristensson, Eyal Ofek, Michel Pahud, and Jens Grubert. 2020. Breaking the Screen: Interaction Across Touchscreen Boundaries in Virtual Reality for Mobile Knowledge Workers. IEEE Transactions on Visualization and Computer Graphics PP (Aug. 2020), 14 pages. https://doi.org/10.1109/TVCG.2020.3023567

[10]

M. Billinghurst, H. Kato, and I. Poupyrev. 2001. The MagicBook - Moving Seamlessly between Reality and Virtuality. IEEE Computer Graphics and Applications 21, 3 (May 2001), 6–8. https://doi.org/10.1109/38.920621

Digital Library

[11]

Marcel Borowski, Roman Rädle, and Clemens N. Klokmose. 2018. Codestrate Packages: An Alternative to ”One-Size-Fits-All” Software. In Extended Abstracts of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–6. https://doi.org/10.1145/3170427.3188563

Digital Library

[12]

Marcel Borowski, Johannes Zagermann, Clemens N. Klokmose, Harald Reiterer, and Roman Rädle. 2020. Exploring the Benefits and Barriers of Using Computational Notebooks for Collaborative Programming Assignments. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. ACM, Portland OR USA, 468–474. https://doi.org/10.1145/3328778.3366887

Digital Library

[13]

Michael Bostock, Vadim Ogievetsky, and Jeffrey Heer. 2011. D3: Data-Driven Documents. IEEE Transactions on Visualization and Computer Graphics 17, 12(2011), 2301–2309. https://doi.org/10.1109/TVCG.2011.185

Digital Library

[14]

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (2006), 77–101. https://doi.org/10.1191/1478088706qp063oa

[15]

John Brooke. 1995. SUS: A quick and dirty usability scale. Usability Eval. Ind. 189 (11 1995).

[16]

Frederik Brudy, Christian Holz, Roman Rädle, Chi-Jui Wu, Steven Houben, Clemens Nylandsted Klokmose, and Nicolai Marquardt. 2019. Cross-Device Taxonomy: Survey, Opportunities and Challenges of Interactions Spanning Across Multiple Devices. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–28. https://doi.org/10.1145/3290605.3300792

Digital Library

[17]

Frederik Brudy, Suppachai Suwanwatcharachat, Wenyu Zhang, Steven Houben, and Nicolai Marquardt. 2018. EagleView: A Video Analysis Tool for Visualising and Querying Spatial Interactions of People and Devices. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 61–72. https://doi.org/10.1145/3279778.3279795

Digital Library

[18]

Bernhard Brüning, Christian Schnier, Karola Pitsch, and Sven Wachsmuth. 2012. Integrating PAMOCAT in the Research Cycle: Linking Motion Capturing and Conversation Analysis. In Proceedings of the ACM Conference on Multimodal Interaction. ACM, New York, NY, USA, 201–208. https://doi.org/10.1145/2388676.2388716

Digital Library

[19]

Wolfgang Büschel, Anke Lehmann, and Raimund Dachselt. 2021. MIRIA: A Mixed Reality Toolkit for the In-Situ Visualization and Analysis of Spatio-Temporal Interaction Data. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–15. https://doi.org/10.1145/3411764.3445651

Digital Library

[20]

Wolfgang Büschel, Annett Mitschick, Thomas Meyer, and Raimund Dachselt. 2019. Investigating Smartphone-Based Pan and Zoom in 3D Data Spaces in Augmented Reality. In Proceedings of the ACM Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, New York, NY, USA, 1–13. https://doi.org/10.1145/3338286.3340113

Digital Library

[21]

Wolfgang Büschel, Patrick Reipschläger, Ricardo Langner, and Raimund Dachselt. 2017. Investigating the Use of Spatial Interaction for 3D Data Visualization on Mobile Devices. In Proceedings of the ACM Conference on Interactive Surfaces and Spaces. ACM, New York, NY, USA, 62–71. https://doi.org/10.1145/3132272.3134125

Digital Library

[22]

Simon Butscher, Sebastian Hubenschmid, Jens Müller, Johannes Fuchs, and Harald Reiterer. 2018. Clusters, Trends, and Outliers: How Immersive Technologies Can Facilitate the Collaborative Analysis of Multidimensional Data. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3173664

Digital Library

[23]

Felipe G. Carvalho, Daniela G. Trevisan, and Alberto Raposo. 2012. Toward the Design of Transitional Interfaces: An Exploratory Study on a Semi-Immersive Hybrid User Interface. Virtual Reality 16, 4 (Nov. 2012), 271–288. https://doi.org/10.1007/s10055-011-0205-y

Digital Library

[24]

Marco Cavallo, Mishal Dholakia, Matous Havlena, Kenneth Ocheltree, and Mark Podlaseck. 2019. Dataspace: A Reconfigurable Hybrid Reality Environment for Collaborative Information Analysis. In Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces. IEEE, Piscataway, NJ, USA, 145–153. https://doi.org/10.1109/VR.2019.8797733

[25]

Marco Cavallo, Mishal Dolakia, Matous Havlena, Kenneth Ocheltree, and Mark Podlaseck. 2019. Immersive Insights: A Hybrid Analytics System for Collaborative Exploratory Data Analysis. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology. ACM, New York, NY, USA, 1–12. https://doi.org/10.1145/3359996.3364242

Digital Library

[26]

Henry Chen, Austin S. Lee, Mark Swift, and John C. Tang. 2015. 3D Collaboration Method over HoloLens™ and Skype™ End Points. In Proceedings of the International Workshop on Immersive Media Experiences. ACM, New York, NY, USA, 27–30. https://doi.org/10.1145/2814347.2814350

Digital Library

[27]

L. Chittaro, R. Ranon, and L. Ieronutti. 2006. VU-Flow: A Visualization Tool for Analyzing Navigation in Virtual Environments. IEEE Transactions on Visualization and Computer Graphics 12, 6 (Nov 2006), 1475–1485. https://doi.org/10.1109/TVCG.2006.109

Digital Library

[28]

Philip DeCamp, George Shaw, Rony Kubat, and Deb Roy. 2010. An Immersive System for Browsing and Visualizing Surveillance Video. In Proceedings of the ACM Conference on Multimedia. ACM, New York, NY, USA, 371–380. https://doi.org/10.1145/1873951.1874002

Digital Library

[29]

Tobias Drey, Jan Gugenheimer, Julian Karlbauer, Maximilian Milo, and Enrico Rukzio. 2020. VRSketchIn: Exploring the Design Space of Pen and Tablet Interaction for 3D Sketching in Virtual Reality. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376628

Digital Library

[30]

Barrett Ens, Benjamin Bach, Maxime Cordeil, Ulrich Engelke, Marcos Serrano, Wesley Willett, Arnaud Prouzeau, Christoph Anthes, Wolfgang Büschel, Cody Dunne, Tim Dwyer, Jens Grubert, Jason H. Haga, Nurit Kirshenbaum, Dylan Kobayashi, Tica Lin, Monsurat Olaosebikan, Fabian Pointecker, David Saffo, Nazmus Saquib, Dieter Schmalstieg, Danielle Albers Szafir, Matt Whitlock, and Yalong Yang. 2021. Grand Challenges in Immersive Analytics. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–17. https://doi.org/10.1145/3411764.3446866

Digital Library

[31]

João Marcelo Evangelista Belo, Anna Maria Feit, Tiare Feuchtner, and Kaj Grønbæk. 2021. XRgonomics: Facilitating the Creation of Ergonomic 3D Interfaces. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–11. https://doi.org/10.1145/3411764.3445349

Digital Library

[32]

Alessandro Febretti, Arthur Nishimoto, Terrance Thigpen, Jonas Talandis, Lance Long, J. D. Pirtle, Tom Peterka, Alan Verlo, Maxine Brown, Dana Plepys, Dan Sandin, Luc Renambot, Andrew Johnson, and Jason Leigh. 2013. {CAVE}2: A Hybrid Reality Environment for Immersive Simulation and Information Analysis. In The Engineering Reality of Virtual Reality 2013. SPIE, Bellingham, WA, USA, 864903. https://doi.org/10.1117/12.2005484

[33]

Steven Feiner and Ari Shamash. 1991. Hybrid User Interfaces: Breeding Virtually Bigger Interfaces for Physically Smaller Computers. In Proceedings of the ACM Symposium on User Interface Software and Technology. ACM, New York, NY, USA, 9–17. https://doi.org/10.1145/120782.120783

Digital Library

[34]

Philipp Fleck, Fernando Reyes Aviles, Christian Pirchheim, Clemens Arth, and Dieter Schmalstieg. 2020. MAUI: Tele-Assistance for Maintenance of Cyber-Physical Systems. In Proceedings of the International Conference on Computer Vision Theory and Applications. SciTePress, Setúbal, Portugal. https://doi.org/10.5220/0009093708000812

[35]

Florian ’Floyd’ Mueller, Tim Dwyer, Sarah Goodwin, Kim Marriott, Jialin Deng, Han D. Phan, Jionghao Lin, Kun-Ting Chen, Yan Wang, and Rohit Ashok Khot. 2021. Data as Delight: Eating Data. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, Article 621, 14 pages. https://doi.org/10.1145/3411764.3445218

Digital Library

[36]

R Foundation. 2021. R Project for Statistical Computing. Retrieved August 31, 2021 from https://www.r-project.org/

[37]

Epic Games. 2021. Unreal Engine – The Most Powerful Real-Time 3D Creation Platform. https://www.unrealengine.com/en-US/.

[38]

Google. 2021. Colaboratory. Retrieved August 31, 2021 from https://colab.research.google.com

[39]

Raphael Grasset, Julian Looser, and Mark Billinghurst. 2006. Transitional Interface: Concept, Issues and Framework. In 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality. 231–232. https://doi.org/10.1109/ISMAR.2006.297819

Digital Library

[40]

Jens Grubert, Matthias Kranz, and Aaron Quigley. 2016. Challenges in Mobile Multi-Device Ecosystems. mUX: The Journal of Mobile User Experience 5, 1 (Dec. 2016). https://doi.org/10.1186/s13678-016-0007-y

[41]

R. Hartson and P. Pyla. 2018. The UX Book: Agile UX Design for a Quality User Experience. Elsevier Science. https://books.google.de/books?id=Pp5RjgEACAAJ

[42]

Juan David Hincapié-Ramos, Xiang Guo, Paymahn Moghadasian, and Pourang Irani. 2014. Consumed Endurance: A Metric to Quantify Arm Fatigue of Mid-Air Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’14). Association for Computing Machinery, Toronto, Ontario, Canada, 1063–1072. https://doi.org/10.1145/2556288.2557130

Digital Library

[43]

Steven Houben, Nicolai Marquardt, Jo Vermeulen, Clemens Klokmose, Johannes Schöning, Harald Reiterer, and Christian Holz. 2017. Opportunities and Challenges for Cross-Device Interactions in the Wild. Interactions 24, 5 (Aug. 2017), 58–63. https://doi.org/10.1145/3121348

Digital Library

[44]

Sebastian Hubenschmid, Johannes Zagermann, Simon Butscher, and Harald Reiterer. 2021. STREAM: Exploring the Combination of Spatially-Aware Tablets with Augmented Reality Head-Mounted Displays for Immersive Analytics. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–14. https://doi.org/10.1145/3411764.3445298

Digital Library

[45]

Sebastian Hubenschmid, Johannes Zagermann, Daniel Fink, Jonathan Wieland, Tiare Feuchtner, and Harald Reiterer. 2021. Towards Asynchronous Hybrid User Interfaces for Cross-Reality Interaction. In ISS’21 Workshop Proceedings: ”Transitional Interfaces in Mixed and Cross-Reality: A New Frontier?”, Hans-Christian Jetter, Jan-Henrik Schröder, Jan Gugenheimer, Mark Billinghurst, Christoph Anthes, Mohamed Khamis, and Tiare Feuchtner (Eds.). https://doi.org/10.18148/kops/352-2-84mm0sggczq02

[46]

TIBCO Software Inc.2021. TIBCO Spotfire®. https://www.tibco.com/products/tibco-spotfire.

[47]

Hans-Christian Jetter, Roman Rädle, Tiare Feuchtner, Christoph Anthes, Judith Friedl, and Clemens Nylandsted Klokmose. 2020. ”In VR, Everything Is Possible!”: Sketching and Simulating Spatially-Aware Interactive Spaces in Virtual Reality. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–16. https://doi.org/10.1145/3313831.3376652

Digital Library

[48]

Daniel A. Keim. Jan.-March/2002. Information Visualization and Visual Data Mining. IEEE Transactions on Visualization and Computer Graphics 8, 1(Jan.-March/2002), 1–8. https://doi.org/10.1109/2945.981847

Digital Library

[49]

Robert S. Kennedy, Norman E. Lane, Kevin S. Berbaum, and Michael G. Lilienthal. 1993. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. The International Journal of Aviation Psychology 3, 3 (July 1993), 203–220. https://doi.org/10.1207/s15327108ijap0303_3

[50]

Karsten Klein, Michael Aichem, Ying Zhang, Stefan Erk, Björn Sommer, and Falk Schreiber. 2021. TEAMwISE: Synchronised Immersive Environments for Exploration and Analysis of Animal Behaviour. Journal of Visualization 24, 4 (Aug. 2021), 845–859. https://doi.org/10.1007/s12650-021-00746-2

Digital Library

[51]

Simon Kloiber, Volker Settgast, Christoph Schinko, Martin Weinzerl, Johannes Fritz, Tobias Schreck, and Reinhold Preiner. 2020. Immersive Analysis of User Motion in VR Applications. The Visual Computer 36, 10-12 (Oct. 2020), 1937–1949. https://doi.org/10.1007/s00371-020-01942-1

Digital Library

[52]

Clemens N. Klokmose, Christian Remy, Janus Bager Kristensen, Rolf Bagge, Michel Beaudouin-Lafon, and Wendy Mackay. 2019. Videostrates: Collaborative, Distributed and Programmable Video Manipulation. In Proceedings of the ACM Symposium on User Interface Software and Technology. ACM, New York, NY, USA, 233–247. https://doi.org/10.1145/3332165.3347912

Digital Library

[53]

Pascal Knierim, Dimitri Hein, Albrecht Schmidt, and Thomas Kosch. 2021. The SmARtphone Controller. i-com 20, 1 (2021), 49–61. https://doi.org/

[54]

Matthias Kraus, Katrin Angerbauer, Juri Buchmüller, Daniel Schweitzer, Daniel A. Keim, Michael Sedlmair, and Johannes Fuchs. 2020. Assessing 2D and 3D Heatmaps for Comparative Analysis: An Empirical Study. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376675

Digital Library

[55]

M. Kraus, N. Weiler, D. Oelke, J. Kehrer, D. A. Keim, and J. Fuchs. 2019. The Impact of Immersion on Cluster Identification Tasks. IEEE Transactions on Visualization and Computer Graphics (2019), 1–1. https://doi.org/10.1109/TVCG.2019.2934395

[56]

Ricardo Langner, Marc Satkowski, Wolfgang Büschel, and Raimund Dachselt. 2021. MARVIS: Combining Mobile Devices and Augmented Reality for Visual Data Analysis. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–17. https://doi.org/10.1145/3411764.3445593

Digital Library

[57]

B. Lee, D. Brown, B. Lee, C. Hurter, S. Drucker, and T. Dwyer. 2020. Data Visceralization: Enabling Deeper Understanding of Data Using Virtual Reality. IEEE Transactions on Visualization and Computer Graphics (2020), 1–1. https://doi.org/10.1109/TVCG.2020.3030435

[58]

Chi-Jung Lee and Hung-Kuo Chu. 2018. Dual-MR: Interaction with Mixed Reality Using Smartphones. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology. ACM, New York, NY, USA, 1–2. https://doi.org/10.1145/3281505.3281618

Digital Library

[59]

Klemen Lilija, Henning Pohl, and Kasper Hornbæk. 2020. Who Put That There? Temporal Navigation of Spatial Recordings by Direct Manipulation. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–11. https://doi.org/10.1145/3313831.3376604

Digital Library

[60]

Weizhou Luo, Eva Goebel, Patrick Reipschläger, Mats Ole Ellenberg, and Raimund Dachselt. 2021. Exploring and Slicing Volumetric Medical Data in Augmented Reality Using a Spatially-Aware Mobile Device. In Adjunct Proceedings of thje IEEE International Symposium on Mixed and Augmented Reality Adjunct. IEEE, Piscataway, NJ, USA.

[61]

Nicolai Marquardt, Frederico Schardong, and Anthony Tang. 2015. EXCITE: EXploring Collaborative Interaction in Tracked Environments. In Proceedings of IFIP Human-Computer Interaction. Springer International Publishing, Cham, Germany, 89–97.

[62]

Kim Marriott, Falk Schreiber, Tim Dwyer, Karsten Klein, Nathalie Henry Riche, Takayuki Itoh, Wolfgang Stuerzlinger, and Bruce H. Thomas (Eds.). 2018. Immersive Analytics. Number 11190 in Lecture Notes in Computer Science. Springer, Cham.

[63]

L. Merino, M. Schwarzl, M. Kraus, M. Sedlmair, D. Schmalstieg, and D. Weiskopf. 2020. Evaluating Mixed and Augmented Reality: A Systematic Literature Review (2009-2019). In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 438–451. https://doi.org/10.1109/ISMAR50242.2020.00069

[64]

Paul Milgram and Fumio Kishino. 1994. A Taxonomy of Mixed Reality Visual Displays. IEICE Transactions on Information and Systems 77, 12 (1994), 1321–1329.

[65]

K.Jarrod Millman and Fernando Pérez. 2018. Developing Open-Source Scientific Practice. In Implementing Reproducible Research(first ed.), Victoria Stodden, Friedrich Leisch, and Roger D. Peng (Eds.). Chapman and Hall/CRC, 149–183. https://doi.org/10.1201/9781315373461-6

[66]

Florian Müller, Martin Schmitz, Daniel Schmitt, Sebastian Günther, Markus Funk, and Max Mühlhäuser. 2020. Walk The Line: Leveraging Lateral Shifts of the Walking Path as an Input Modality for Head-Mounted Displays. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, Honolulu HI USA, 1–15. https://doi.org/10.1145/3313831.3376852

Digital Library

[67]

Jens Müller, Roman Rädle, and Harald Reiterer. 2017. Remote Collaboration With Mixed Reality Displays: How Shared Virtual Landmarks Facilitate Spatial Referencing. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 6481–6486. https://doi.org/10.1145/3025453.3025717

Digital Library

[68]

Jens Müller, Johannes Zagermann, Jonathan Wieland, Ulrike Pfeil, and Harald Reiterer. 2019. A Qualitative Comparison Between Augmented and Virtual Reality Collaboration with Handheld Devices. In Proceedings of Mensch Und Computer 2019 (Hamburg, Germany) (MuC’19). Association for Computing Machinery, New York, NY, USA, 399–410. https://doi.org/10.1145/3340764.3340773

Digital Library

[69]

Michael Nebeling, Maximilian Speicher, Xizi Wang, Shwetha Rajaram, Brian D Hall, Zijian Xie, Alexander R E Raistrick, Michelle Aebersold, Edward G Happ, Jiayin Wang, Yanan Sun, Lotus Zhang, Leah E Ramsier, and Rhea Kulkarni. 2020. MRAT: The Mixed Reality Analytics Toolkit. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 12. https://doi.org/10.1145/3313831.3376330

Digital Library

[70]

Federica Pallavicini and Alessandro Pepe. 2019. Comparing Player Experience in Video Games Played in Virtual Reality or on Desktop Displays: Immersion, Flow, and Positive Emotions. In Extended Abstracts of the ACM Symposium on Computer-Human Interaction in Play. ACM, New York, NY, USA, 195–210. https://doi.org/10.1145/3341215.3355736

Digital Library

[71]

Leonardo Pavanatto, Chris North, Doug A. Bowman, Carmen Badea, and Richard Stoakley. 2021. Do We Still Need Physical Monitors? An Evaluation of the Usability of AR Virtual Monitors for Productivity Work. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). IEEE, Lisboa, Portugal, 759–767. https://doi.org/10.1109/VR50410.2021.00103

[72]

Fernando Perez and Brian E. Granger. 2007. IPython: A System for Interactive Scientific Computing. Computing in Science Engineering 9, 3 (2007), 21–29. https://doi.org/10.1109/MCSE.2007.53

Digital Library

[73]

Michael Prilla and Lisa M. Rühmann. 2018. An Analysis Tool for Cooperative Mixed Reality Scenarios. In Adjunct Proceedings of the IEEE International Symposium on Mixed and Augmented Reality. 31–35. https://doi.org/10.1109/ISMAR-Adjunct.2018.00026

[74]

Roman Rädle, Midas Nouwens, Kristian Antonsen, James R. Eagan, and Clemens N. Klokmose. 2017. Codestrates: Literate Computing with Webstrates. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 715–725. https://doi.org/10.1145/3126594.3126642

Digital Library

[75]

Umar Rashid, Miguel A. Nacenta, and Aaron Quigley. 2012. The Cost of Display Switching: A Comparison of Mobile, Large Display and Hybrid UI Configurations. In Proceedings of the ACM Conference on Advanced Visual Interfaces. ACM, New York, NY, USA, 99–106. https://doi.org/10.1145/2254556.2254577

Digital Library

[76]

Carolin Reichherzer, Andrew Cunningham, Tracey Coleman, Ruochen Cao, Kurt McManus, Dion Sheppard, Mark Kohler, Mark Billinghurst, and Bruce H Thomas. 2021. Bringing the Jury to the Scene of the Crime: Memory and Decision-Making in a Simulated Crime Scene. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama Japan, 1–12. https://doi.org/10.1145/3411764.3445464

Digital Library

[77]

Patrick Reipschläger and Raimund Dachselt. 2019. DesignAR: Immersive 3D-Modeling Combining Augmented Reality with Interactive Displays. In Proceedings of the ACM Conference on Interactive Surfaces and Spaces. ACM, New York, NY, USA, 29–41. https://doi.org/10.1145/3343055.3359718

Digital Library

[78]

Patrick Reipschläger, Tamara Flemisch, and Raimund Dachselt. 2021. Personal Augmented Reality for Information Visualization on Large Interactive Displays. IEEE Transactions on Visualization and Computer Graphics (Feb. 2021). https://doi.org/10.1109/TVCG.2020.3030460 arxiv:2009.03237

[79]

Jun Rekimoto and Masanori Saitoh. 1999. Augmented Surfaces: A Spatially Continuous Work Space for Hybrid Computing Environments. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 378–385. https://doi.org/10.1145/302979.303113

Digital Library

[80]

Yvonne Rogers. 2012. HCI Theory: Classical, Modern, and Contemporary. Synthesis Lectures on Human-Centered Informatics 5, 2 (May 2012), 1–129. https://doi.org/10.2200/S00418ED1V01Y201205HCI014

[81]

Salesforce. 2021. Tableau: Business Intelligence and Analytics Software. https://www.tableau.com/.

[82]

Prasanth Sasikumar, Max Collins, Huidong Bai, and Mark Billinghurst. 2021. XRTB: A Cross Reality Teleconference Bridge to Incorporate 3D Interactivity to 2D Teleconferencing. In Extended Abstracts of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–4. https://doi.org/10.1145/3411763.3451546

Digital Library

[83]

Marc Satkowski, Weizhou Luo, and Raimund Dachselt. 2021. Towards In-situ Authoring of AR Visualizations with Mobile Devices. In Adjunct Proceedings of the IEEE International Symposium on Mixed and Augmented Reality Adjunct. IEEE, Piscataway, NJ, USA.

[84]

Arvind Satyanarayan, Dominik Moritz, Kanit Wongsuphasawat, and Jeffrey Heer. 2017. Vega-Lite: A Grammar of Interactive Graphics. IEEE Transactions on Visualization and Computer Graphics 23, 1 (Jan. 2017), 341–350. https://doi.org/10.1109/TVCG.2016.2599030

Digital Library

[85]

Richard Skarbez, Missie Smith, and Mary C. Whitton. 2021. Revisiting Milgram and Kishino’s Reality-Virtuality Continuum. Frontiers in Virtual Reality 2 (March 2021), 647997. https://doi.org/10.3389/frvir.2021.647997

[86]

Maurício Sousa, Daniel Mendes, Soraia Paulo, Nuno Matela, Joaquim Jorge, and Daniel Simões Lopes. 2017. VRRRRoom: Virtual Reality for Radiologists in the Reading Room. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 4057–4062. https://doi.org/10.1145/3025453.3025566

Digital Library

[87]

Maximilian Speicher, Brian D. Hall, and Michael Nebeling. 2019. What Is Mixed Reality?. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–15. https://doi.org/10.1145/3290605.3300767

Digital Library

[88]

Seung Youb Ssin, James A. Walsh, Ross T. Smith, Andrew Cunningham, and Bruce H. Thomas. 2019. GeoGate: Correlating Geo-Temporal Datasets Using an Augmented Reality Space-Time Cube and Tangible Interactions. In Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces. IEEE, Piscataway, NJ, USA, 210–219. https://doi.org/10.1109/VR.2019.8797812

[89]

Hemant Bhaskar Surale, Aakar Gupta, Mark Hancock, and Daniel Vogel. 2019. TabletInVR: Exploring the Design Space for Using a Multi-Touch Tablet in Virtual Reality. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300243

Digital Library

[90]

Z. Szalavári and M. Gervautz. 2008. The Personal Interaction Panel – a Two-Handed Interface for Augmented Reality. Computer Graphics Forum 16 (June 2008), C335–C346. https://doi.org/10.1111/1467-8659.16.3conferenceissue.35

[91]

Anthony Tang, Michel Pahud, Sheelagh Carpendale, and Bill Buxton. 2010. VisTACO: Visualizing Tabletop Collaboration. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces. ACM, New York, NY, USA, 29–38. https://doi.org/10.1145/1936652.1936659

Digital Library

[92]

Katja Vock, Sebastian Hubenschmid, Johannes Zagermann, Simon Butscher, and Harald Reiterer. 2021. IDIAR : Augmented Reality Dashboards to Supervise Mobile Intervention Studies. In Mensch und Computer. ACM, New York, NY, USA. https://doi.org/10.1145/3473856.3473876

Digital Library

[93]

Ulrich von Zadow and Raimund Dachselt. 2017. GIAnT: Visualizing Group Interaction at Large Wall Displays. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, Denver Colorado USA, 2639–2647. https://doi.org/10.1145/3025453.3026006

Digital Library

[94]

Jia Wang and Robert Lindeman. 2014. Coordinated 3D Interaction in Tablet- and HMD-Based Hybrid Virtual Environments. In Proceedings of the ACM Symposium on Spatial User Interaction. ACM, New York, NY, USA, 70–79. https://doi.org/10.1145/2659766.2659777

Digital Library

[95]

Jia Wang and Robert Lindeman. 2015. Coordinated Hybrid Virtual Environments: Seamless Interaction Contexts for Effective Virtual Reality. Computers & Graphics 48 (May 2015), 71–83. https://doi.org/10.1016/j.cag.2015.02.007

Digital Library

[96]

Yun Wang, Xiaojuan Ma, Qiong Luo, and Huamin Qu. 2016. Data Edibilization: Representing Data with Food. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 409–422. https://doi.org/10.1145/2851581.2892570

Digital Library

[97]

Jonathan Wieland, Johannes Zagermann, Jens Müller, and Harald Reiterer. 2021. Separation, Composition, or Hybrid? – Comparing Collaborative 3D Object Manipulation Techniques for Handheld Augmented Reality. In 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, Bari, Italy, 403–412. https://doi.org/10.1109/ISMAR52148.2021.00057

[98]

W. Xie, Y. Liang, J. Johnson, A. Mower, S. Burns, C. Chelini, P. D’Alessandro, N. Weibel, and J. P. Schulze. 2020. Interactive Multi-User 3D Visual Analytics in Augmented Reality. Electronic Imaging 2020, 13 (Jan. 2020), 363–363. https://doi.org/10.2352/ISSN.2470-1173.2020.13.ERVR-363

[99]

Y. Yan, K. Chen, Y. Xie, Y. Song, and Y. Liu. 2019. The Effects of Weight on Comfort of Virtual Reality Devices. In Advances in Ergonomics in Design, Francisco Rebelo and Marcelo M. Soares (Eds.). Vol. 777. Springer International Publishing, Cham, 239–248. https://doi.org/10.1007/978-3-319-94706-8_27

[100]

Fengyuan Zhu and Tovi Grossman. 2020. BISHARE: Exploring Bidirectional Interactions Between Smartphones and Head-Mounted Augmented Reality. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376233

Digital Library

Cited By

Janaka NCai RRam AZhu LZhao SYong K(2024)PilotAR: Streamlining Pilot Studies with OHMDs from Concept to InsightProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785768:3(1-35)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678576
Mane SElmqvist N(2024)"Wichita 1-1, Fox Three" - The Role of 3D Telemetry Analysis in Combat Flight SimulationProceedings of the ACM on Human-Computer Interaction10.1145/36771148:CHI PLAY(1-27)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677114
Sargolzaei PRastogi MZaman L(2024)Advancing Mixed Reality Game Development: An Evaluation of a Visual Game Analytics Tool in Action-Adventure and FPS GenresProceedings of the ACM on Human-Computer Interaction10.1145/36770558:CHI PLAY(1-32)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677055
Show More Cited By

Index Terms

ReLive: Bridging In-Situ and Ex-Situ Visual Analytics for Analyzing Mixed Reality User Studies
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies
    2. Interaction paradigms
      1. Virtual reality
  2. Visualization
    1. Visualization application domains
      1. Visual analytics

Recommendations

Building a Prototype for Easy to Use Collaborative Immersive Analytics
Computational Science – ICCS 2021
Abstract
The increase in the size and complexity of today’s datasets creates a need to develop and experiment with novel data visualization methods. One of these innovations is immersive analytics, in which extended reality technologies such as virtual ...
Designing Spatial Visualization and Interactions of Immersive Sankey Diagram in Virtual Reality
MM '24: Proceedings of the 32nd ACM International Conference on Multimedia

Virtual reality (VR) is a revolutionary method of presenting data visualizations, which brings potential possibilities for enhancing analytical activities. However, applying this method to visualize complex data flows remains largely underexplored, ...
Big data exploration through visual analytics
VAST '12: Proceedings of the 2012 IEEE Conference on Visual Analytics Science and Technology (VAST)

SAS® Visual Analytics Explorer is an advanced data visualization and exploratory data analysis application that is a component of the SAS Visual Analytics solution. It excels at handling big data problems like the VAST challenge. With a wide range of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

April 2022

10459 pages

ISBN:9781450391573

DOI:10.1145/3491102

Editors:
Simone Barbosa
PUC-Rio, Brazil
,
Cliff Lampe
University of Michigan, USA
,
Caroline Appert
Université Paris-Saclay, France
,
David A. Shamma
Toyota Research Institute, USA
,
Steven Drucker
Microsoft Research, USA
,
Julie Williamson
University of Glasgow, UK
,
Koji Yatani
University of Tokyo, Japan

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 April 2022

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Conference

CHI '22

Sponsor:

SIGCHI

CHI '22: CHI Conference on Human Factors in Computing Systems

April 29 - May 5, 2022

LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

50
Total Citations
View Citations
2,470
Total Downloads

Downloads (Last 12 months)885
Downloads (Last 6 weeks)105

Reflects downloads up to 15 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Janaka NCai RRam AZhu LZhao SYong K(2024)PilotAR: Streamlining Pilot Studies with OHMDs from Concept to InsightProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785768:3(1-35)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678576
Mane SElmqvist N(2024)"Wichita 1-1, Fox Three" - The Role of 3D Telemetry Analysis in Combat Flight SimulationProceedings of the ACM on Human-Computer Interaction10.1145/36771148:CHI PLAY(1-27)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677114
Sargolzaei PRastogi MZaman L(2024)Advancing Mixed Reality Game Development: An Evaluation of a Visual Game Analytics Tool in Action-Adventure and FPS GenresProceedings of the ACM on Human-Computer Interaction10.1145/36770558:CHI PLAY(1-32)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677055
Yassien AHany RAbdennadher S(2024)Walk or Teleport? The Impact of VR Users' Locomotion Technique in Asymmetric VR-MR SetupAdjunct Proceedings of the 2024 Nordic Conference on Human-Computer Interaction10.1145/3677045.3685443(1-5)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3677045.3685443
Chiossi FGruenefeld UHou BNewn JOu CLiao RWelsch RMayer S(2024)Understanding the Impact of the Reality-Virtuality Continuum on Visual Search Using Fixation-Related Potentials and Eye Tracking FeaturesProceedings of the ACM on Human-Computer Interaction10.1145/36765288:MHCI(1-33)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676528
Janaka NCai RZhao SHsu DKostakos VKay JHoang T(2024)Demonstrating PilotAR: A Tool to Assist Wizard-of-Oz Pilot Studies with OHMDCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3677554(230-234)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3677554
Mayer EChiossi FMayer S(2024)Crossing Mixed Realities: A Review for Transitional Interfaces DesignProceedings of Mensch und Computer 202410.1145/3670653.3677481(629-634)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1145/3670653.3677481
Cheng LJia HYu LWu YYe SDeng DZhang HXie XWu Y(2024)VisCourt: In-Situ Guidance for Interactive Tactic Training in Mixed RealityProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676466(1-14)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676466
Liao JVan KXia ZSuzuki R(2024)RealityEffects: Augmenting 3D Volumetric Videos with Object-Centric Annotation and Dynamic Visual EffectsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661631(1248-1261)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661631
Polenz LJoeres FHansen CHeinrich F(2024)Simulating projective Augmented Reality Visualizations in Virtual Reality: Is VR a feasible Environment for medical AR Evaluations?Extended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650843(1-8)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650843
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents