More Web Proxy on the site http://driver.im/

Article

Towards reliable multimodal sensing in aware environments

Authors:

Scott Stillman,

Irfan EssaAuthors Info & Claims

PUI '01: Proceedings of the 2001 workshop on Perceptive user interfaces

Pages 1 - 6

https://doi.org/10.1145/971478.971499

Published: 15 November 2001 Publication History

Abstract

A prototype system for implementing a reliable sensor network for large scale smart environments is presented. Most applications within any form of smart environments (rooms, offices, homes, etc.) are dependent on reliable who, where, when, and what information of its inhabitants (users). This information can be inferred from different sensors spread throughout the space. However, isolated sensing technologies provide limited information under the varying, dynamic, and long-term scenarios (24/7), that are inherent in applications for intelligent environments. In this paper, we present a prototype system that provides an infrastructure for leveraging the strengths of different sensors and processes used for the interpretation of their collective data. We describe the needs of such systems, propose an architecture to dealwith such multi-modal fusion, and discuss the initial set of sensors and processes used to address such needs.

Supplementary Material

JPG File (p17-stillman.jpg)

Download
11.20 KB

stillman (p17-stillman.mpg)

This file contains a supplemental video to "Towards reliable multimodal sensing in aware environments"

Download
5.70 MB

References

[1]

S. Basu, M. Casy, W. Gardner, A. Azarbayejani, and A. Pentland. Vision-steered audio for interactive environments. In Proc. of IMAGE'COM 1996, May 1996.]]

[2]

A. Bobick, S. Intille, J. Davis, F. Baird, L. Campbell, Y. Ivanov, C. Pinhanez, A. Schäutte, and A. Wilson. The KidsRoom: A perceptually-based interactive and immersive story environment. "PRESENCE: Teleoperators and Virtual Environments", 8(4):367--391, August 1999.]]

Digital Library

[3]

G. R. Bradski. Computer vision face tracking for use in a perceptual user interface. Intel Tech J Q2, 1998.]]

[4]

R. R. Brooks and S. S. Iyengar. Multi-Sensor Fusion. Prentice Hall PTR, 1998.]]

[5]

B. Brumitt, B. Meyers, J. Krumm, A. Kern, and S. Shafer. Easyliving: Technologies for intelligent environments. Proceedings of Handheld and Ubiquitous Computing, September 2000.]]

Digital Library

[6]

J. Crowley and J. Bedrune. Integration and control of reactive visual processes. 1994 European Conference on Computer Vision, 1994.]]

Digital Library

[7]

J. Crowley and F. Berard. Multi-modal tracking of faces for video communications. IEEE Conference on Computer Vision and Pattern Recognition, June 1997.]]

Digital Library

[8]

J. L. Crowley and Y. Demazeau. Principles and techniques for sensor data fusion. Signal Processing, 32(1-2):5--27, May 1993.]]

Digital Library

[9]

A. K. Dey, D. Salber, and G. D. Abowd. A conceptual framework and a toolkit for supporting the rapid prototyping of context-aware applications. Human-Computer Interaction, 16, 2001. To appear as anchor article in special issue on context-aware computing.]]

[10]

R. Duraiswami, D. Zotkin, and L. Davis. Active speech source localization by a dual coarse-to-fine search. Proceedings of ICASSP2001, May 2001.]]

Digital Library

[11]

H. F. Durrant-Whyte. Sensor models and multisensor integration. International Journal of Robotics Research., 7(6):97--113, Dec 1988.]]

Digital Library

[12]

A. Elfes. Occupancy grids: A stochastic spatial representation for active robot perception. IEEE Computer, 22(6):46--57, 1989.]]

Digital Library

[13]

I. A. Essa. Ubiquitous sensing for smart and aware environments. IEEE Personal Communications, October 2000. Special Issue on Networking the Physical World.]]

[14]

S. Goodridge and M. Kay. Multimedia sensor fusion for intelligent camera control. Proc. of 1996 IEEE/SICE/RSJ Int. Conf. on Multisensor Fusion and Integration for Intelligent Systems, December 1996.]]

[15]

Y. Huang. Real-time Acoustic Source Localization with Passive Microphone Arrays. PhD thesis, Georgia Institute of Technology, February 2001.]]

[16]

MIT. House_n project. http://architecture.mit.edu/house-n/web/, 2001.]]

[17]

H. Moravec and M. Blackwell. Learning sensor models for evidence grids. Robotics Institute Research Review, 1992.]]

[18]

E. D. Mynatt, J. Rowan, A. Jacobs, and S. Craighill. 2001 digital family portraits-supporting peace of mind for extended family members. Proceedings of CHI 2001, 2001.]]

Digital Library

[19]

K. Nagel and G. ABowd. The family intercom: Developing a context-aware audio communication system. In Proceeding of Ubicomp 2001 International Conference, 2001.]]

Digital Library

[20]

A. Pentland. Smart rooms. Scientific American, 274(4):68--76, April 1996.]]

[21]

D. Rabinkin, R. Renomeron, A. Dahl, J. French, J. Flanagan, and M. Bianchi. A dsp implementation of source location using microphone arrays. J. Acous. Soc. Am., 99(4):2503+, May 1996.]]

[22]

U. Ramachandran, R. S. Nikhil, N. Harel, J. M. Rehg, and K. Knobe. Space-time memory: A parallel programming abstraction for interactive multimedia applications. In Proccedings of 10th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, 1999.]]

Digital Library

[23]

S. Stillman and T. Tanawongsuwan. Tracking multiple people with multiple cameras. International Conference on Audio- and Video-based Biometric Person Authentication, March 1999.]]

[24]

G. Tech. Aware home research initiative. http://www.cc.gatech.edu/fce/ahri/, 2001.]]

[25]

J. Vermaak, M. Gangnet, A. Blake, and P. Patrick. Sequential monte carlo fusion of sound and vision for speaker tracking. Proceedings of ICCV 2001, July 2001.]]

[26]

K. Yow and R. Cipolla. Feature-based human face detection. Image and Vision Comp., 15(9):713--735, 1997.]]

Cited By

Chen ZLiu YValera-Medina ARobinson F(2020)A Multi-source Feature-level Fusion Approach for Predicting Strip Breakage in Cold Rolling2020 IEEE 16th International Conference on Automation Science and Engineering (CASE)10.1109/CASE48305.2020.9216854(482-487)Online publication date: Aug-2020
https://doi.org/10.1109/CASE48305.2020.9216854
Sun DHuang XYang K(2019)A multimodal vision sensor for autonomous drivingCounterterrorism, Crime Fighting, Forensics, and Surveillance Technologies III10.1117/12.2535552(20)Online publication date: 7-Oct-2019
https://doi.org/10.1117/12.2535552
Valle R(2016)ABROA: Audio-based room-occupancy analysis using Gaussian mixtures and Hidden Markov models2016 Future Technologies Conference (FTC)10.1109/FTC.2016.7821763(1270-1273)Online publication date: Dec-2016
https://doi.org/10.1109/FTC.2016.7821763
Show More Cited By

Recommendations

Towards multimodal interaction in smart home environments: the home operating system
DIS '10: Proceedings of the 8th ACM Conference on Designing Interactive Systems

Future homes as smart and interactive environments will include a large and constantly growing number of networked devices that communicate with each other to coordinate activities. In this case the main challenge for developers and designers will be to ...
Towards an Intrusion Detection System for Pervasive Computing Environments
ITCC '05: Proceedings of the International Conference on Information Technology: Coding and Computing (ITCC'05) - Volume II - Volume 02

A pervasive computing environment consists of numerous casually accessible, frequently mobile, embedded, handheld devices capable of sensing the environment around it and reacting intelligently to simplify user activities. These devices are distributed ...
Towards a universal ontology for smart environments
iiWAS '09: Proceedings of the 11th International Conference on Information Integration and Web-based Applications & Services

The growth of ubiquitous computing and its integration in smart environments constitutes a major challenge. However, many issues related to this technology were raised, and many efforts have been made to facilitate the implementation of such technology. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

PUI '01: Proceedings of the 2001 workshop on Perceptive user interfaces

November 2001

241 pages

ISBN:9781450374736

DOI:10.1145/971478

Copyright © 2001 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 November 2001

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

PUI01

PUI01: Workshop on Perceptive User Interfaces

November 15 - 16, 2001

Florida, Orlando, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
561
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)1

Reflects downloads up to 19 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen ZLiu YValera-Medina ARobinson F(2020)A Multi-source Feature-level Fusion Approach for Predicting Strip Breakage in Cold Rolling2020 IEEE 16th International Conference on Automation Science and Engineering (CASE)10.1109/CASE48305.2020.9216854(482-487)Online publication date: Aug-2020
https://doi.org/10.1109/CASE48305.2020.9216854
Sun DHuang XYang K(2019)A multimodal vision sensor for autonomous drivingCounterterrorism, Crime Fighting, Forensics, and Surveillance Technologies III10.1117/12.2535552(20)Online publication date: 7-Oct-2019
https://doi.org/10.1117/12.2535552
Valle R(2016)ABROA: Audio-based room-occupancy analysis using Gaussian mixtures and Hidden Markov models2016 Future Technologies Conference (FTC)10.1109/FTC.2016.7821763(1270-1273)Online publication date: Dec-2016
https://doi.org/10.1109/FTC.2016.7821763
Yusro MHou KPissaloux ERamli KSudiana DZhang LShi H(2014)Concept and Design of SEES (Smart Environment Explorer Stick) for Visually Impaired Person Mobility AssistanceHuman-Computer Systems Interaction: Backgrounds and Applications 310.1007/978-3-319-08491-6_21(245-259)Online publication date: 2014
https://doi.org/10.1007/978-3-319-08491-6_21
Medjahed HIstrate DBoudy JBaldinger JDorizzi B(2011)A pervasive multi-sensor data fusion for smart home healthcare monitoring2011 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011)10.1109/FUZZY.2011.6007636(1466-1473)Online publication date: Jun-2011
https://doi.org/10.1109/FUZZY.2011.6007636
Goffredo MBouchrika ICarter JNixon M(2009)Performance analysis for automated gait extraction and recognition in multi-camera surveillanceMultimedia Tools and Applications10.1007/s11042-009-0378-550:1(75-94)Online publication date: 2-Oct-2009
https://doi.org/10.1007/s11042-009-0378-5
Laouris YAristodemou EMakris P(2009)Prediction of Learning Abilities Based on a Cross-Modal Evaluation of Non-verbal Mental Attributes Using Video-Game-Like InterfacesCross-Modal Analysis of Speech, Gestures, Gaze and Facial Expressions10.1007/978-3-642-03320-9_24(248-265)Online publication date: 14-Jul-2009
https://dl.acm.org/doi/10.1007/978-3-642-03320-9_24
Goffredo MBouchrika ICarter JNixon MDoulamis Avan Gool LNixon MVarvarigou TDoulamis NSardis M(2008)Performance analysis for gait in camera networksProceedings of the 1st ACM workshop on Analysis and retrieval of events/actions and workflows in video streams10.1145/1463542.1463555(73-80)Online publication date: 31-Oct-2008
https://dl.acm.org/doi/10.1145/1463542.1463555
Bernardin KStiefelhagen RWaibel AEL Saddik AVuong SGriwodz CDel Bimbo ACandan KJaimes A(2008)Probabilistic integration of sparse audio-visual cues for identity trackingProceedings of the 16th ACM international conference on Multimedia10.1145/1459359.1459380(151-158)Online publication date: 26-Oct-2008
https://dl.acm.org/doi/10.1145/1459359.1459380
Bourguet M(2008)Handling uncertainty in multimodal pervasive computing applicationsComputer Communications10.1016/j.comcom.2008.06.00131:18(4234-4241)Online publication date: 1-Dec-2008
https://dl.acm.org/doi/10.1016/j.comcom.2008.06.001
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents