More Web Proxy on the site http://driver.im/

research-article

Kinect analysis: a system for recording, analysing and sharing multimodal interaction elicitation studies

Authors:

Michael Nebeling,

Moira C. NorrieAuthors Info & Claims

EICS '15: Proceedings of the 7th ACM SIGCHI Symposium on Engineering Interactive Computing Systems

Pages 142 - 151

https://doi.org/10.1145/2774225.2774846

Published: 23 June 2015 Publication History

Abstract

Recently, guessability studies have become a popular means among researchers to elicit user-defined interaction sets involving gesture, speech and multimodal input. However, tool support for capturing and analysing interaction proposals is lacking and the method itself is still evolving. This paper presents Kinect Analysis---a system designed for interaction elicitation studies with support for record-and-replay, visualisation and analysis based on Kinect's depth, audio and video streams. Kinect Analysis enables post-hoc analysis during playback and live analysis with real-time feedback while recording. In particular, new visualisations such as skeletal joint traces and heatmaps can be superimposed for analysis and comparison of multiple recordings. It also introduces KinectScript---a simple scripting language to query recordings and automate analysis tasks based on skeleton, distance, audio and gesture scripts. The paper discusses Kinect Analysis both as a tool and a method that could enable researchers to more easily collect, study and share interaction proposals. Using data from a previous guessability study with 25 users, we show that Kinect Analysis in combination with KinectScript is useful and effective for a range of analysis tasks.

References

[1]

Ashbrook, D., and Starner, T. MAGIC: A Motion Gesture Design Tool. In Proc. CHI (2010).

Digital Library

[2]

Burr, B. VACA: a tool for qualitative video analysis. In Proc. CHI EA (2006).

Digital Library

[3]

Fouse, A., Weibel, N., Hutchins, E., and Hollan, J. D. ChronoViz: A System for Supporting Navigation of Time-coded Data. In Proc. CHI EA (2011).

Digital Library

[4]

Heath, C., Hindmarsh, J., and Luff, P. Video in Qualitative Research. SAGE, 2010.

[5]

Hinrichs, U., and Carpendale, S. Gestures in the wild: studying multi-touch gesture sequences on interactive tabletop exhibits. In Proc. CHI (2011).

Digital Library

[6]

Hoste, L., and Signer, B. SpeeG2: a speech- and gesture-based interface for efficient controller-free text input. In Proc. ICMI (2013).

Digital Library

[7]

Jang, S., Elmqvist, N., and Ramani, K. GestureAnalyzer: Visual Analytics for Pattern Analysis of Mid-Air Hand Gestures. In Proc. SUI (2014).

Digital Library

[8]

Jones, B. R., Benko, H., Ofek, E., and Wilson, A. D. Illumiroom: peripheral projected illusions for interactive experiences. In Proc. CHI (2013).

Digital Library

[9]

Kato, J., McDirmid, S., and Cao, X. DejaVu: Integrated Support for Developing Interactive Camera-Based Programs. In Proc. UIST (2012).

Digital Library

[10]

Klemmer, S. R., Hartmann, B., and Takayama, L. How Bodies Matter: Five Themes for Interaction Design. In Proc. DIS (2006).

Digital Library

[11]

Lee, S.-S., Chae, J., Kim, H., Lim, Y.-K., and Lee, K.-P. Towards more Natural Digital Content Manipulation via User Freehand Gestural Interaction in a Living Room. In Proc. UbiComp (2013).

Digital Library

[12]

Morris, M. R. Web on the Wall: Insights from a Multimodal Interaction Elicitation Study. In Proc. ITS (2012).

Digital Library

[13]

Morris, M. R., Danielescu, A., Drucker, S. M., Fisher, D., Lee, B., m. c. schraefel, and Wobbrock, J. O. Reducing Legacy Bias in Gesture Elicitation Studies. Interactions 21, 3 (2014).

Digital Library

[14]

Morris, M. R., Wobbrock, J. O., and Wilson, A. D. Understanding Users Preferences for Surface Gestures. In Proc. GI (2010).

Digital Library

[15]

Nacenta, M. A., Kamber, Y., Qiang, Y., and Kristensson, P. O. Memorability of Pre-designed and User-defined Gesture Sets. In Proc. CHI (2013).

Digital Library

[16]

Nebeling, M., Huber, A., Ott, D., and Norrie, M. C. Web on the Wall Reloaded: Implementation, Replication and Refinement of User-Defined Interaction Sets. In Proc. ITS (2014).

Digital Library

[17]

Nebeling, M., Speicher, M., and Norrie, M. C. W3Touch: Metrics-based Web Page Adaptation for Touch. In Proc. CHI (2013).

Digital Library

[18]

Nebeling, M., Teunissen, E., Husmann, M., and Norrie, M. C. XDKinect: Development Framework for Cross-Device Interaction using Kinect. In Proc. EICS (2014).

Digital Library

[19]

North, C., Dwyer, T., Lee, B., Fisher, D., Isenberg, P., Robertson, G. G., and Inkpen, K. Understanding Multi-touch Manipulation for Surface Computing. In Proc. INTERACT (2009).

Digital Library

[20]

Oh, U., and Findlater, L. The Challenges and Potential of End-User Gesture Customization. In Proc. CHI (2013).

Digital Library

[21]

Ruiz, J., Li, Y., and Lank, E. User-Defined Motion Gestures for Mobile Interaction. In Proc. CHI (2011).

Digital Library

[22]

Schmidt, D., Seifert, J., Rukzio, E., and Gellersen, H. A Cross-Device Interaction Style for Mobiles and Surfaces. In Proc. DIS (2012).

Digital Library

[23]

Seyed, T., Burns, C., Sousa, M. C., Maurer, F., and Tang, A. Eliciting Usable Gestures for Multi-Display Environments. In Proc. ITS (2012).

Digital Library

[24]

Troiano, G. M., Pedersen, E. W., and Hornbæk, K. User-Defined Gestures for Elastic, Deformable Displays. In Proc. AVI (2014).

Digital Library

[25]

Vatavu, R. User-Defined Gestures for Free-Hand TV Control. In Proc. EuroITV (2012).

Digital Library

[26]

Weibel, N., Emmenegger, C., Lyons, J., Dixit, R., Hill, L. L., and Hollan, J. D. Interpreter-Mediated Physician-Patient Communication: Opportunities for Multimodal Healthcare Interfaces. In Proc. PervasiveHealth (2013).

Digital Library

[27]

Weichel, C., Lau, M., Kim, D., Villar, N., and Gellersen, H. MixFab: A Mixed-Reality Environment for Personal Fabrication. In Proc. CHI (2014).

Digital Library

[28]

Wobbrock, J. O., Morris, M. R., and Wilson, A. D. User-Defined Gestures for Surface Computing. In Proc. CHI (2009).

Digital Library

Cited By

Pathmanathan NKurzhals K(2024)Investigating the Gap: Gaze and Movement Analysis in Immersive EnvironmentsProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3653522(1-7)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3653522
Liu JSaquib NZhutian CKazi RWei LFu HTai C(2024)PoseCoach: A Customizable Analysis and Visualization System for Video-Based Running CoachingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.323085530:7(3180-3195)Online publication date: Jul-2024
https://doi.org/10.1109/TVCG.2022.3230855
Cheng YWu ZXiao R(2024)Exploring Methods to Optimize Gesture Elicitation Studies: A Systematic Literature ReviewIEEE Access10.1109/ACCESS.2024.338726912(64958-64979)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3387269
Show More Cited By

Index Terms

Kinect analysis: a system for recording, analysing and sharing multimodal interaction elicitation studies
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
    2. Interaction devices
      1. Touch screens

Recommendations

Enabling Finger-Gesture Interaction with Kinect
VINCI '15: Proceedings of the 8th International Symposium on Visual Information Communication and Interaction

A large number of tracking and gesture recognition algorithms and technologies have been developed in the field of human-computer interactions thanks to the introduction of cameras with depth sensors such as Microsoft's Kinect. Most of the techniques ...
Multi-scenario gesture recognition using Kinect
CGAMES '12: Proceedings of the 2012 17th International Conference on Computer Games: AI, Animation, Mobile, Interactive Multimedia, Educational & Serious Games (CGAMES)

Hand gesture recognition (HGR) is an important research topic because some situations require silent communication with sign languages. Computational HGR systems assist silent communication, and help people learn a sign language. In this article, a ...
Kinect-based Taiwanese sign-language recognition system

Gesture-recognition is an important component for many intelligent human---computer interaction applications. For example, a realtime sign-language recognition system would detect and interpret hand gestures. Many vision-based sign-language recognition ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

EICS '15: Proceedings of the 7th ACM SIGCHI Symposium on Engineering Interactive Computing Systems

June 2015

316 pages

ISBN:9781450336468

DOI:10.1145/2774225

General Chair:
Jürgen Ziegler
University of Duisburg-Essen, Germany
,
Program Chairs:
Michael Nebeling
Carnegie Melon University, USA
,
Laurence Nigay
Université Grenoble Alpes, France

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 June 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Swiss NSF Advanced Postdoc.Mobility grant

Conference

EICS'15

Sponsor:

SIGCHI

EICS'15: ACM SIGCHI Symposium on Engineering Interactive Computing Systems

June 23 - 26, 2015

Duisburg, Germany

Acceptance Rates

EICS '15 Paper Acceptance Rate 19 of 64 submissions, 30%;

Overall Acceptance Rate 73 of 299 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
318
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)3

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pathmanathan NKurzhals K(2024)Investigating the Gap: Gaze and Movement Analysis in Immersive EnvironmentsProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3653522(1-7)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3653522
Liu JSaquib NZhutian CKazi RWei LFu HTai C(2024)PoseCoach: A Customizable Analysis and Visualization System for Video-Based Running CoachingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.323085530:7(3180-3195)Online publication date: Jul-2024
https://doi.org/10.1109/TVCG.2022.3230855
Cheng YWu ZXiao R(2024)Exploring Methods to Optimize Gesture Elicitation Studies: A Systematic Literature ReviewIEEE Access10.1109/ACCESS.2024.338726912(64958-64979)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3387269
Villarreal-Narvaez SSluÿters AVanderdonckt JVatavu R(2023)Brave New GES World: A Systematic Literature Review of Gestures and Referents in Gesture Elicitation StudiesACM Computing Surveys10.1145/363645856:5(1-55)Online publication date: 7-Dec-2023
https://dl.acm.org/doi/10.1145/3636458
Vatavu R(2023)Gesture-Based InteractionHandbook of Human Computer Interaction10.1007/978-3-319-27648-9_20-1(1-47)Online publication date: 9-Feb-2023
https://doi.org/10.1007/978-3-319-27648-9_20-1
Schäfer AReis GStricker D(2022)AnyGesture: Arbitrary One-Handed Gestures for Augmented, Virtual, and Mixed Reality ApplicationsApplied Sciences10.3390/app1204188812:4(1888)Online publication date: 11-Feb-2022
https://doi.org/10.3390/app12041888
Schäfer AReis GStricker D(2022)The Gesture Authoring Space: Authoring Customised Hand Gestures for Grasping Virtual Objects in Immersive Virtual EnvironmentsProceedings of Mensch und Computer 202210.1145/3543758.3543766(85-95)Online publication date: 4-Sep-2022
https://dl.acm.org/doi/10.1145/3543758.3543766
Schipor OVatavu R(2022)GearWheels: A Software Tool to Support User Experiments on Gesture Input with Wearable DevicesInternational Journal of Human–Computer Interaction10.1080/10447318.2022.209890739:18(3527-3545)Online publication date: 22-Jul-2022
https://doi.org/10.1080/10447318.2022.2098907
Dang HBuschek DKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)GestureMap: Supporting Visual Analytics and Quantitative Analysis of Motion Elicitation Data by Learning 2D EmbeddingsProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445765(1-12)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445765
Büschel WLehmann ADachselt RKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)MIRIA: A Mixed Reality Toolkit for the In-Situ Visualization and Analysis of Spatio-Temporal Interaction DataProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445651(1-15)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445651
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents