poster

MISO: a context-sensitive multimodal interface for smart objects based on hand gestures and finger snaps

Authors:

Christian LeichsenringAuthors Info & Claims

UIST Adjunct Proceedings '12: Adjunct proceedings of the 25th annual ACM symposium on User interface software and technology

Pages 93 - 94

https://doi.org/10.1145/2380296.2380338

Published: 07 October 2012 Publication History

Get Access

Abstract

We present an unobtrusive multimodal interface for smart objects (MISO) in an everyday indoor environment. MISO uses pointing for object selection and context-sensitive arm gestures for object control. Finger snaps are used to confirm object selections and to aid with gesture segmentation. Audio feedback is provided during the interaction. The use of a Kinect depth camera allows for a compact system and robustness in varying environments and lighting conditions at low cost.

Supplementary Material

JPG File (0226-file3.jpg)

Download
3.99 KB

suppl.mov (0226-file3.mp4)

Supplemental video

Download
11.71 MB

References

[1]

Caon, M., Yue, Y., Tscherrig, J., Mugellini, E., and Khaled, O. A. Context-Aware 3D Gesture Interaction Based on Multiple Kinects. In AMBIENT 2011 (2011).

Google Scholar

[2]

Do, J.-H., Jung, S. H., Jang, H., Yang, S.-E., Jung, J.-W., and Bien, Z. Gesture-Based Interface for Home Appliance Control in Smart Home. In Smart Homes and Beyond - ICOST2006 4th Int. Conference On Smart homes and health Telematics, C. Nugent and J. C. Augusto, Eds. IOS Press, Amsterdam, 2006, 23--30.

Google Scholar

[3]

Fleer, D. Steuerung von Umgebungsintelligenz mit multimodalen Gesten. Master's thesis, Universität Bielefeld, 2011.

Google Scholar

[4]

Kriesten, B., Tünnermann, R., Mertes, C., and Hermann, T. Controlling ambient information flow between smart objects with a mobile mixed-reality interface. In Proc. of the 12th Int. Conference on Human computer interaction with mobile devices and services, MobileHCI '10, ACM (New York, NY, USA, 2010), 405--406.

Digital Library

Google Scholar

[5]

Lin, S.-Y., Lai, Y.-C., Chan, L.-W., and Hung, Y.-P. Real-Time 3D Model-Based Gesture Tracking for Multimedia Control. In 20th Int. Conference on Pattern Recognition (2010).

Digital Library

Google Scholar

[6]

Vesa, S., and Lokki, T. An eyes-free user interface controlled by finger snaps. Proc. of the 8th Int. Conference on Digital Audio Effects (DAFx.05) (2005), 262--265.

Google Scholar

Cited By

View all

Cronin SFreeman EDoherty G(2022)Investigating Clutching Interactions for Touchless Medical Imaging SystemsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517512(1-14)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517512
Funk MTobisch VEmfield ABernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Non-Verbal Auditory Input for Controlling Binary, Discrete, and Continuous Input in Automotive User InterfacesProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376816(1-13)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376816
Alce GEspinoza AHartzell TOlsson SSamuelsson DWallergård M(2018)UbiCompassAdvances in Human-Computer Interaction10.1155/2018/57813632018Online publication date: 12-Apr-2018
https://dl.acm.org/doi/10.1155/2018/5781363
Show More Cited By

Index Terms

MISO: a context-sensitive multimodal interface for smart objects based on hand gestures and finger snaps
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Authoring Communicative Behaviors for Situated, Embodied Characters
ICMI '14: Proceedings of the 16th International Conference on Multimodal Interaction

Embodied conversational agents hold great potential as multimodal interfaces due to their ability to communicate naturally using speech and nonverbal cues. The goal of my research is to enable animators and designers to endow ECAs with interactive ...
Nomadic gestures: A technique for reusing gesture commands for frequent ambient interactions

The age of ambient intelligence has already incorporated gestures into practical applications with the goal of delivering adaptive and personalized interactions. However, practitioners are faced with many problems when implementing gesture-based ...
Device-free interaction in smart domestic environments
AH '13: Proceedings of the 4th Augmented Human International Conference

This paper contributes to the exploration of user preferences of device-free interaction with smart appliances and services in the domestic environment. We presented a prototype system for on-surface gesture control to users in a natural environment and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

UIST Adjunct Proceedings '12: Adjunct proceedings of the 25th annual ACM symposium on User interface software and technology

October 2012

114 pages

ISBN:9781450315821

DOI:10.1145/2380296

General Chair:
Rob Miller
MIT CSAIL, USA
,
Program Chairs:
Hrvoje Benko
Microsoft Research, USA
,
Celine Latulipe
University of North Carolina at Charlotte, USA

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 October 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

UIST '12

Sponsor:

UIST '12: The 25th Annual ACM Symposium on User Interface Software and Technology

October 7 - 10, 2012

Massachusetts, Cambridge, USA

Acceptance Rates

Overall Acceptance Rate 355 of 1,733 submissions, 20%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
450
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)4

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Cronin SFreeman EDoherty G(2022)Investigating Clutching Interactions for Touchless Medical Imaging SystemsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517512(1-14)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517512
Funk MTobisch VEmfield ABernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Non-Verbal Auditory Input for Controlling Binary, Discrete, and Continuous Input in Automotive User InterfacesProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376816(1-13)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376816
Alce GEspinoza AHartzell TOlsson SSamuelsson DWallergård M(2018)UbiCompassAdvances in Human-Computer Interaction10.1155/2018/57813632018Online publication date: 12-Apr-2018
https://dl.acm.org/doi/10.1155/2018/5781363
Zhang TYi XWang RWang YYu CLu YShi Y(2018)Tap-to-PairProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/32870792:4(1-21)Online publication date: 27-Dec-2018
https://dl.acm.org/doi/10.1145/3287079
Velloso EWirth MWeichel CEsteves AGellersen HFoth MJu WSchroeter RViller S(2016)AmbiGazeProceedings of the 2016 ACM Conference on Designing Interactive Systems10.1145/2901790.2901867(812-817)Online publication date: 4-Jun-2016
https://dl.acm.org/doi/10.1145/2901790.2901867
de Freitas ANebeling MChen XYang JKarthikeyan Ranithangam ADey AKaye JDruin ALampe CMorris DHourcade J(2016)Snap-To-ItProceedings of the 2016 CHI Conference on Human Factors in Computing Systems10.1145/2858036.2858177(5909-5920)Online publication date: 7-May-2016
https://dl.acm.org/doi/10.1145/2858036.2858177
Chae SYang YByun JHan T(2016)Personal Smart Space: IoT Based User Recognition and Device Control2016 IEEE Tenth International Conference on Semantic Computing (ICSC)10.1109/ICSC.2016.45(181-182)Online publication date: Feb-2016
https://doi.org/10.1109/ICSC.2016.45
Neto ASantos CSaleme ESilva J(2015)Where, what, why and how - 3W1HProceedings of the 14th Brazilian Symposium on Human Factors in Computing Systems10.1145/3148456.3148472(1-10)Online publication date: 3-Nov-2015
https://dl.acm.org/doi/10.1145/3148456.3148472
Budde MBerning MBaumgärtner CKinn FKopf TOchs SReiche FRiedel TBeigl MMattern FSantini SCanny JLangheinrich MRekimoto J(2013)Point & control -- interaction in smart environmentsProceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication10.1145/2494091.2494184(303-306)Online publication date: 8-Sep-2013
https://dl.acm.org/doi/10.1145/2494091.2494184

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Authoring Communicative Behaviors for Situated, Embodied Characters

Nomadic gestures: A technique for reusing gesture commands for frequent ambient interactions

Device-free interaction in smart domestic environments