More Web Proxy on the site http://driver.im/

demonstration

Controlling your TV with gestures

Authors:

Padmanabhan Pillai,

Alexander Hauptmann,

Rahul SukthankarAuthors Info & Claims

MIR '10: Proceedings of the international conference on Multimedia information retrieval

Pages 405 - 408

https://doi.org/10.1145/1743384.1743453

Published: 29 March 2010 Publication History

Abstract

Vision-based user interfaces enable natural interaction modalities such as gestures. Such interfaces require computationally intensive video processing at low latency. We demonstrate an application that recognizes gestures to control TV operations. Accurate recognition is achieved by using a new descriptor called MoSIFT, which explicitly encodes optical flow with appearance features. MoSIFT is computationally expensive - a sequential implementation runs 100 times slower than real time. To reduce latency sufficiently for interaction, the application is implemented on a runtime system that exploits the parallelism inherent in video understanding applications.

References

[1]

D. J. Abadi, Y. Ahmad, M. Balazinska, U. Çetintemel, M. Cherniack, J. Hwang, W. Lindner, A. S. Maskey, A. Rasin, E. Ryvkina, N. Tatbul, Y. Xing, and S. Zdonik. The design of the Borealis stream processing engine. In Proc. Innovative Data Systems Research, 2005.

[2]

J. K. Aggarwal and Q. Cai. Human motion analysis: a review. In Proc. Nonrigid and Articulated Motion Workshop, 1997.

Digital Library

[3]

L. Amini, H. Andrade, R. Bhagwan, F. Eskesen, R. King, P. Selo, Y. Park, and C. Venkatramani. SPC: A distributed, scalable platform for data mining. Workshop on Data Mining Standards, Services, and Platforms, 2006.

Digital Library

[4]

J. Brady. A theory of productivity in the creative process. IEEE Computer Graphics and Applications, 6(5):25--34, May 1986.

Digital Library

[5]

S. K. Card, G. G. Robertson, and J. D. Mackinlay. The information visualizer, an information workspace. In CHI '91: Human factors in computing systems, 181--186, 1991.

Digital Library

[6]

M.-Y. Chen and A. Hauptmann. Mosift: Recognizing human actions in surveillance videos. In CMU-CS-09-161, 2009.

[7]

M.-Y. Chen; L. Mummert; P. Pillai; A. Hauptmann; R. Sukthankar, Exploiting Multi-level Parallelism for Low-latency Activity Recognition in Streaming Video; Proc. ACM Multimedia Systems (MMSys) Conference, 2010,

Digital Library

[8]

M. Cherniack, H. Balakrishnan, M. Balazinska, D. Carney, U. Çetintemel, Y. Xing, and S. Zdonik. Scalable distributed stream processing. In Proc. Innovative Data Systems Research, 2003.

[9]

J. Dean and S. Ghemawat. MapReduce: simplified data processing on large clusters. CACM, 51(1), 2008.

Digital Library

[10]

P. Dollar, V. Rabaud, G. Cottrell, and S. Belongie. Behavior recognition via sparse spatio-temporal features. In IEEE Workshop on PETS, 2005.

[11]

M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: distributed data-parallel programs from sequential building blocks. Proc. European Conference on Computer Systems, 2007.

Digital Library

[12]

Y. Ke, R. Sukthankar, and M. Hebert. Efficient visual event detection using volumetric features. Proc. Int'l Conference on Computer Vision, 2005.

Digital Library

[13]

I. Laptev and T. Lindeberg. Space-time interest points. In Proc. Int'l Conference on Computer Vision, 2003.

Digital Library

[14]

I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld. Learning realistic human actions from movies. In Proc. Computer Vision and Pattern Recognition, 2008.

[15]

D. Lowe. Distinctive image features form scale-invariant keypoints. Int'l Journal on Computer Vision, 60(2), 2004.

Digital Library

[16]

Microsoft, "Project Natal in detail". Microsoft. June 2009. http://www.xbox.com/en-GB/news-features/news/Project-Natal-in-detail-050609.htm. Retrieved Jan 26, 2010.

[17]

R. B. Miller. Response time in man-computer conversational transactions. In AFIPS '68: Proc. of the Dec. 9-11, 1968, joint computer conference (Fall, part I), pages 267--277, 1968.

Digital Library

[18]

J. C. Niebles, H. Wang, and L. Fei-Fei. Unsupervised learning of human action categories using spatial-temporal words. In Proc. British Machine Vision Conference, 2006.

[19]

P. Pillai, L. Mummert, S. Schlosser, R. Sukthankar, and C. Helfrich. SLIPStream: scalable low-latency interactive perception on streaming data. In Proc. NOSSDAV, 2009.

Digital Library

[20]

K. Schindler and L. Van Gool. Action snippets: How many frames does human action recognition require? In Proc. Computer Vision and Pattern Recognition, 2008.

[21]

C. Schuldt, I. Laptev, and B. Caputo. Recognizing human actions: A local SVM approach. In Proc. ICPR, 2004.

Digital Library

[22]

J. Zhang, M. Marszalek, S. Lazebnik, and C. Schmid. Local features and kernels for classification of texture and object categories: A comprehensive study. Int'l Journal on Computer Vision, 73(2), 2007.

Digital Library

Cited By

Rocha JLuna JMonacelli EFoggea GPassedouet MDelaplace SHirata Y(2023)Dance Gestures Recognition for Wheelchair Control2023 8th International Conference on Control and Robotics Engineering (ICCRE)10.1109/ICCRE57112.2023.10155605(84-90)Online publication date: 21-Apr-2023
https://doi.org/10.1109/ICCRE57112.2023.10155605
Vogiatzidakis PKoutsabasis P(2020)Mid-Air Gesture Control of Multiple Home Devices in Spatial Augmented Reality PrototypeMultimodal Technologies and Interaction10.3390/mti40300614:3(61)Online publication date: 31-Aug-2020
https://doi.org/10.3390/mti4030061
Vanattenhoven JGeerts DVanderdonckt JPerez-Medina J(2019)The Impact of Comfortable Viewing Positions on Smart TV Gestures2019 International Conference on Information Systems and Computer Science (INCISCOS)10.1109/INCISCOS49368.2019.00054(296-303)Online publication date: Nov-2019
https://doi.org/10.1109/INCISCOS49368.2019.00054
Show More Cited By

Index Terms

Controlling your TV with gestures
1. Computer systems organization
  1. Embedded and cyber-physical systems
  2. Real-time systems
2. Software and its engineering

Recommendations

Exploiting multi-level parallelism for low-latency activity recognition in streaming video
MMSys '10: Proceedings of the first annual ACM SIGMM conference on Multimedia systems

Video understanding is a computationally challenging task that is critical not only for traditionally throughput-oriented applications such as search but also latency-sensitive interactive applications such as surveillance, gaming, videoconferencing, ...
A Detailed Performance Analysis of the Interpolation Supplemented Lattice Boltzmann Method on the Cray T3E and Cray X1A Detailed Performance Analysis of the Interpolation Supplemented Lattice Boltzmann Method on the Cray T3E and Cray X1

A detailed study of the parallel performance of the interpolation supplemented lattice Boltzmann (ISLB) method using SHMEM and MPI on the Cray T3E-900 and Cray X1 architectures is presented. The noteworthy feature of the ...
Hyper-systolic algorithms for N-body computations and parallel level-3 BLAS libraries
Special issue on parallelization techniques for numerical modelling

Hyper-systolic algorithms represent a new class of parallel computing structures. Because of their regular communication and compute patterns they are well suited for implementation on most parallel architectures, in particular, high performance SIMD ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MIR '10: Proceedings of the international conference on Multimedia information retrieval

March 2010

600 pages

ISBN:9781605588155

DOI:10.1145/1743384

General Chairs:
James Z. Wang
The Pennsylvania State University, USA
,
Nozha Boujemaa
INRIA, France
,
Program Chairs:
Nuria Oliver Ramirez
Telefonica Research, Spain
,
Apostol Natsev
IBM Research, USA

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 March 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Demonstration

Conference

MIR '10

Sponsor:

SIGMM

MIR '10: International Conference on Multimedia Information Retrieval

March 29 - 31, 2010

Pennsylvania, Philadelphia, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

36
Total Citations
View Citations
504
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Rocha JLuna JMonacelli EFoggea GPassedouet MDelaplace SHirata Y(2023)Dance Gestures Recognition for Wheelchair Control2023 8th International Conference on Control and Robotics Engineering (ICCRE)10.1109/ICCRE57112.2023.10155605(84-90)Online publication date: 21-Apr-2023
https://doi.org/10.1109/ICCRE57112.2023.10155605
Vogiatzidakis PKoutsabasis P(2020)Mid-Air Gesture Control of Multiple Home Devices in Spatial Augmented Reality PrototypeMultimodal Technologies and Interaction10.3390/mti40300614:3(61)Online publication date: 31-Aug-2020
https://doi.org/10.3390/mti4030061
Vanattenhoven JGeerts DVanderdonckt JPerez-Medina J(2019)The Impact of Comfortable Viewing Positions on Smart TV Gestures2019 International Conference on Information Systems and Computer Science (INCISCOS)10.1109/INCISCOS49368.2019.00054(296-303)Online publication date: Nov-2019
https://doi.org/10.1109/INCISCOS49368.2019.00054
Mana NMich OFerron M(2019)How to Increase Older Adults’ Accessibility to Mobile Technology? The New ECOMODE CameraAmbient Assisted Living10.1007/978-3-030-04672-9_6(85-98)Online publication date: 31-Jan-2019
https://doi.org/10.1007/978-3-030-04672-9_6
Sali Shajideen SPreetha V(2018)Human-Computer Interaction System Using 2D and 3D Hand Gestures2018 International Conference on Emerging Trends and Innovations In Engineering And Technological Research (ICETIETR)10.1109/ICETIETR.2018.8529064(1-4)Online publication date: Jul-2018
https://doi.org/10.1109/ICETIETR.2018.8529064
Bellino A(2018)SEQUENCEPersonal and Ubiquitous Computing10.1007/s00779-018-1129-222:4(751-770)Online publication date: 1-Aug-2018
https://dl.acm.org/doi/10.1007/s00779-018-1129-2
Shimada A(2018)Potential of Wearable Technology for Super-Aging SocietiesDistributed, Ambient and Pervasive Interactions: Technologies and Contexts10.1007/978-3-319-91131-1_17(214-226)Online publication date: 30-May-2018
https://doi.org/10.1007/978-3-319-91131-1_17
Clarke CBellino AEsteves AGellersen H(2017)Remote Control by Body Movement in Synchrony with Orbiting WidgetsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/31309101:3(1-22)Online publication date: 11-Sep-2017
https://dl.acm.org/doi/10.1145/3130910
Clarke CGellersen HGajos KMankoff JHarrison C(2017)MatchPointProceedings of the 30th Annual ACM Symposium on User Interface Software and Technology10.1145/3126594.3126626(179-192)Online publication date: 20-Oct-2017
https://dl.acm.org/doi/10.1145/3126594.3126626
Carreira MTing KCsobanka PGonçalves D(2017)Evaluation of in-air hand gestures interaction for older peopleUniversal Access in the Information Society10.1007/s10209-016-0483-y16:3(561-580)Online publication date: 1-Aug-2017
https://dl.acm.org/doi/10.1007/s10209-016-0483-y
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents