More Web Proxy on the site http://driver.im/

chapter

Multi-level Particle Filter Fusion of Features and Cues for Audio-Visual Person Tracking

Authors:

Keni Bernardin,

Rainer StiefelhagenAuthors Info & Claims

Multimodal Technologies for Perception of Humans: International Evaluation Workshops CLEAR 2007 and RT 2007, Baltimore, MD, USA, May 8-11, 2007, Revised Selected Papers

January 2008

Pages 70 - 81

https://doi.org/10.1007/978-3-540-68585-2_5

Published: 01 January 2008 Publication History

Abstract

In this paper, two multimodal systems for the tracking of multiple users in smart environments are presented. The first is a multi-view particle filter tracker using foreground, color and special upper body detection and person region features. The other is a wide angle overhead view person tracker relying on foreground segmentation and model-based blob tracking. Both systems are completed by a joint probabilistic data association filter-based source localizer using the input from several microphone arrays. While the first system fuses audio and visual cues at the feature level, the second one incorporates them at the decision level using state-based heuristics.

The systems are designed to estimate the 3D scene locations of room occupants and are evaluated based on their precision in estimating person locations, their accuracy in recognizing person configurations and their ability to consistently keep track identities over time.

The trackers are extensively tested and compared, for each separate modality and for the combined modalities, on the CLEAR 2007 Evaluation Database.

References

[1]

Khalaf, R.Y., Intille, S.S.: Improving Multiple People Tracking using Temporal Consistency. MIT Dept. of Architecture House n Project Technical Report (2001).

[2]

Niu, W., Jiao, L., Han, D., Wang, Y.-F.: Real-Time Multi-Person Tracking in Video Surveillance. In: Pacific Rim Multimedia Conference, Singapore (2003).

[3]

Mittal, A., Davis, L.S.: M2Tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene Using Region-Based Stereo. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 18-33. Springer, Heidelberg (2002).

Digital Library

[4]

Checka, N., Wilson, K., Rangarajan, V., Darrell, T.: A Probabilistic Framework for Multi-modal Multi-Person Tracking. In: Workshop on Multi-Object Tracking (CVPR) (2003).

[5]

Comaniciu, D., Meer, P.: Mean Shift: A Robust Approach Toward Feature Space Analysis. IEEE PAMI 24(5) (May 2002).

[6]

Haritaoglu, I., Harwood, D., Davis, L.S.: W4: Who? When? Where? What? A Real Time System for Detecting and Tracking People. In: Third Face and Gesture Recognition Conference, pp. 222-227 (1998).

[7]

Raja, Y., McKenna, S.J., Gong, S.: Tracking and Segmenting People in Varying Lighting Conditions using Colour. In: 3rd. Int. Conference on Face & Gesture Recognition, p. 228 (1998).

[8]

Viola, P., Jones, M.: Rapid Object Detection using a Boosted Cascade of Simple Features. In: IEEE CVPR (2001).

[9]

Lienhart, R., Maydt, J.: An Extended Set of Haar-like Features for Rapid Object Detection. In: IEEE ICIP 2002, September 2002, vol. 1, pp. 900-903 (2002).

[10]

Gehrig, T., McDonough, J.: Tracking of Multiple Speakers with Probabilistic Data Association Filters. In: CLEAR Workshop, Southampton, UK (April 2006).

[11]

Bernardin, K., Elbs, A., Stiefelhagen, R.: Multiple Object Tracking Performance Metrics and Evaluation in a Smart Room Environment. In: Sixth IEEE International Workshop on Visual Surveillance, in conjunction with ECCV 2006, Graz, Austria, May 13th (2006).

[12]

Bernardin, K., Gehrig, T., Stiefelhagen, R.: Multi- and Single View Multiperson Tracking for Smart Room Environments. In: CLEAR Evaluation Workshop 2006, Southampton, UK, April 2006. LNCS, vol. 4122, pp. 81-92 (2006).

[13]

Tao, H., Sawhney, H., Kumar, R.: A Sampling Algorithm for Tracking Multiple Objects. In: International Workshop on Vision Algorithms: Theory and Practice, pp. 53-68 (1999).

[14]

Wren, C., Azarbayejani, A., Darrell, T., Pentland, A.: Pfinder: Real-Time Tracking of the Human Body. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7), 780-785 (1997).

Digital Library

[15]

Bar-Shalom, Y.: Tracking and data association. Academic Press Professional, Inc., San Diego (1987).

[16]

Knapp, C.H., Carter, G.C.: The Generalized Correlation Method for Estimation of Time Delay. IEEE Trans. Acoust. Speech Signal Proc. 24(4), 320-327 (1976).

[17]

Omologo, M., Svaizer, P.: Acoustic Event Localization Using a Crosspower-spectrum Phase Based Technique. In: Proc. ICASSP, vol. 2, pp. 273-276 (1994).

[18]

Klee, U., Gehrig, T., McDonough, J.: Kalman Filters for Time Delay of Arrival-Based Source Localization. EURASIP Journal on Applied Signal Processing (2006).

[19]

Gehrig, T., McDonough, J.: Tracking Multiple Simultaneous Speakers with Probabilistic Data Association Filters. LNCS, vol. 4122, pp. 137-150 (2006).

[20]

CHIL - Computers In the Human Interaction Loop, http://chil.server.de

[21]

AMI - Augmented Multiparty Interaction, http://www.amiproject.org

[22]

VACE - Video Analysis and Content Extraction, http://www.ic-arda.org

[23]

OpenCV - Open Computer Vision Library, http://sourceforge.net/projects/opencvlibrary

Cited By

Kosmopoulos DMaglogiannis IMakedon FClements MPelachaud CKalogeraki VMaglogiannis I(2014)Multicamera fusion for online analysis of structured processesProceedings of the 7th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/2674396.2674455(1-7)Online publication date: 27-May-2014
https://dl.acm.org/doi/10.1145/2674396.2674455
Bidwell JEssa IRozga AAbowd GAli Salah ACohn JSchuller BAran OMorency LCohen P(2014)Measuring Child Visual Attention using Markerless Head Tracking from Color and Depth Sensing CamerasProceedings of the 16th International Conference on Multimodal Interaction10.1145/2663204.2663235(447-454)Online publication date: 12-Nov-2014
https://dl.acm.org/doi/10.1145/2663204.2663235
Yan WWeber CWermter S(2011)Person tracking based on a hybrid neural probabilistic modelProceedings of the 21st international conference on Artificial neural networks - Volume Part II10.5555/2029604.2029651(365-372)Online publication date: 14-Jun-2011
https://dl.acm.org/doi/10.5555/2029604.2029651
Show More Cited By

Multi-level Particle Filter Fusion of Features and Cues for Audio-Visual Person Tracking
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks

Recommendations

Joint audio-visual tracking using particle filters

It is often advantageous to track objects in a scene using multimodal information when such information is available. We use audio as a complementary modality to video data, which, in comparison to vision, can provide faster localization over a wider ...
Robust object tracking via multi-cue fusion

A long-term object tracking method based on calibrated binocular cameras by fusing information of the two channels and binocular geometry constraints is proposed.The stereo filter which is built based on the epipolar geometry of the binocular cameras is ...
Visual Object Tracking Based on Mean-shift and Particle-Kalman Filter

Even though many algorithms have been developed and many applications of object tracking have been made, object tracking is still considered as a difficult task to accomplish. The existence of several problems such as illumination variation, tracking ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide books

Multimodal Technologies for Perception of Humans: International Evaluation Workshops CLEAR 2007 and RT 2007, Baltimore, MD, USA, May 8-11, 2007, Revised Selected Papers

January 2008

549 pages

ISBN:9783540685845

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 January 2008

Qualifiers

Chapter

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 19 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kosmopoulos DMaglogiannis IMakedon FClements MPelachaud CKalogeraki VMaglogiannis I(2014)Multicamera fusion for online analysis of structured processesProceedings of the 7th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/2674396.2674455(1-7)Online publication date: 27-May-2014
https://dl.acm.org/doi/10.1145/2674396.2674455
Bidwell JEssa IRozga AAbowd GAli Salah ACohn JSchuller BAran OMorency LCohen P(2014)Measuring Child Visual Attention using Markerless Head Tracking from Color and Depth Sensing CamerasProceedings of the 16th International Conference on Multimodal Interaction10.1145/2663204.2663235(447-454)Online publication date: 12-Nov-2014
https://dl.acm.org/doi/10.1145/2663204.2663235
Yan WWeber CWermter S(2011)Person tracking based on a hybrid neural probabilistic modelProceedings of the 21st international conference on Artificial neural networks - Volume Part II10.5555/2029604.2029651(365-372)Online publication date: 14-Jun-2011
https://dl.acm.org/doi/10.5555/2029604.2029651
Ijsselmuiden JStiefelhagen R(2010)Towards high-level human activity recognition through computer vision and temporal logicProceedings of the 33rd annual German conference on Advances in artificial intelligence10.5555/1882150.1882206(426-435)Online publication date: 21-Sep-2010
https://dl.acm.org/doi/10.5555/1882150.1882206
Bernardin KStiefelhagen RWaibel AEL Saddik AVuong SGriwodz CDel Bimbo ACandan KJaimes A(2008)Probabilistic integration of sparse audio-visual cues for identity trackingProceedings of the 16th ACM international conference on Multimedia10.1145/1459359.1459380(151-158)Online publication date: 26-Oct-2008
https://dl.acm.org/doi/10.1145/1459359.1459380

View Options

View options

Media

Figures

Other

Tables

View Table of Contents