Abstract
This paper describes a method for temporally calibrating video sequences from unsynchronized cameras by image processing operations, and presents two search algorithms to match and align trajectories across different camera views. Existing multi-camera systems assume that input video sequences are synchronized either by genlock or by time stamp information and a centralized server. Yet, hardware-based synchronization increases installation cost. Hence, using image information is necessary to align frames from the cameras whose clocks are not synchronized. The system built for temporal calibration is composed of three modules: object tracking module, calibration data extraction module, and the search module. A robust and efficient search algorithm is introduced that recovers the frame offset by matching the trajectories in different views, and finding the most reliable match. Thanks to information obtained from multiple trajectories, this algorithm is robust to possible errors in background subtraction and location extraction. Moreover, the algorithm can handle very large frame offsets. A RANdom SAmple Consensus (RANSAC) based version of this search algorithm is also introduced. Results obtained with different video sequences are presented, which show the robustness of the algorithms in recovering various range of frame offsets for video sequences with varying levels of object activity.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cai, Q., Aggarwal, J.K.: Automatic tracking of human motion in indoor scenes across multiple synchronized video streams. In: Proc. of Int’l Conf. on Computer Vision (1998)
Chang, T.-H., Gong, S.: Tracking multiple people with a multi-camera system. In: IEEE Workshop on Multi-Object Tracking pp. 19–26 (2001)
Collins, R.T., Lipton, A.J., Kanade, T., Fujiyoshi, H., Duggins, D., Tsin, Y., Tolliver, D., Enomoto, N., Hasegawa, O., Burt, P., Wixson, L.: A system for video surveillance and monitoring: VSAM final report. Technical report CMU-RI-TR-00-12, Robotics Institute, Carnegie Mellon University (2000)
Collins R.T., Lipton A.J., Fujiyoshi H. and Kanade T. (2001). Algorithms for cooperative multisensor surveillance. Proc. IEEE 89: 1456–1477
Ellis, T.: Multi-camera video surveillance. In: Int’l Carnahan Conf. on Security Technology, pp. 228–233 (2002)
Khan S. and Shah M. (2003). Consistent labeling of tracked objects in multiple cameras with overlapping fields of view. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 25: 1355–1360
Krumm, J., Harris, S., Meyers, B., Brumitt, B., Hale, M., Shafer, S.: Multi-camera multi-person tracking for EasyLiving. In: Proc. of 3rd IEEE Int’l Workshop on Visual Surveillance (2000)
Nguyen K., Yeung G., Ghiasi S., Sarrafzadeh M. (2002) A general framework for tracking objects in a multi-camera environment. In: Int’l Workshop on Digital and Computational Video pp. 200–204
Pavlidis I., Morellas V., Tsiamyrtzis P. and Harp S. (2001). Urban surveillance systems: from the laboratory to the commercial world. Proc. IEEE 89: 1478–1497
Velipasalar, S., Schlessman, J., Chen, C.-Y., Wolf, W., Singh, J.P.: SCCS: a scalable camera cluster system for multiple object tracking communicating via message passing interface. In: IEEE Int’l Conf. on Multimedia & Expo (2006)
Velipasalar, S., Brown, L.M., Hampapur, A.: Specifying, interpreting and detecting high-level, spatio-temporal composite events in single and multi-camera systems. In: Int’l Workshop on Semantic Learning Applications in Multimedia (SLAM) in Conjunction with IEEE CVPR (2006)
Yuan, X., Sun, Z., Varol, Y., Bebis, G.: A distributed visual surveillance system. In: Proc. of IEEE Conf. on Advanced Video and Signal Based Surveillance pp. 199–204 (2003)
Kuthirummal S., Jawahar C.V. and Narayanan P.J. (2002). Video frame alignment in multiple views. Proc. IEEE Intl Conf. Image Process. 3: 357–360
Lee L., Romano R. and Stein G. (2000). Monitoring activities from multiple video streams: establishing a common coordinate frame. IEEE Trans. Pattern Anal Mach Intell (PAMI) 22: 758–767
Caspi, Y., Irani, M.: A step towards sequence-to-sequence alignment. In: Proc. IEEE Intl Conf. Computer Vis. Pattern Recogn. 682–689 (2000) (to appear)
Caspi, Y., Simakov, D., Irani, M.: Feature-based sequence- to-sequence matching. In: VAMODS Workshop (2002)
Tuytelaars T. and Van Gool L. (2004). Synchronizing video sequences. Proc. IEEE Conf. Comput. Vis. Pattern Recogn. 1: 762–768
Fischler M.A. and Bolles R. (1981). RANSAC random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24: 381–395
Stauffer C. and Grimson W.E.L. (1999). Adaptive background mixture models for real-time tracking. Proc. IEEE Conf. Comput. Vis. Pattern Recogn. 2: 246–252
Comaniciu D., Ramesh V. and Meer P. (2000). Real-time tracking of non-rigid objects using mean shift. Proc. IEEE Intl Conf. Comput. Vis. Pattern Recogn. 2: 142–149
Hartley R. and Zisserman A. (2001). Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge
Trucco E. and Verri A. (1998). Introductory Techniques for 3-D Computer Vision. Prentice Hall, New Jersey
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Velipasalar, S., Wolf, W.H. Frame-level temporal calibration of video sequences from unsynchronized cameras. Machine Vision and Applications 19, 395–409 (2008). https://doi.org/10.1007/s00138-008-0122-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00138-008-0122-6