More Web Proxy on the site http://driver.im/

research-article

Robust video stabilization based on particle filter tracking of projected camera motion

Authors:

Magdi MohamedAuthors Info & Claims

IEEE Transactions on Circuits and Systems for Video Technology, Volume 19, Issue 7

Pages 945 - 954

https://doi.org/10.1109/TCSVT.2009.2020252

Published: 01 July 2009 Publication History

Abstract

Video stabilization is an important technique in digital cameras. Its impact increases rapidly with the rising popularity of handheld cameras and cameras mounted on moving platforms (e.g., cars). Stabilization of two images can be viewed as an image registration problem. However, to ensure the visual quality of the whole video, video stabilization has a particular emphasis on the accuracy and robustness over long image sequences. In this paper, we propose a novel technique for video stabilization based on the particle filtering framework. We extend the traditional use of particle filters in object tracking to tracking of the projected affine model of the camera motions. We rely on the inverse of the resulting image transform to obtain a stable video sequence. The correspondence between scale-invariant feature transform points is used to obtain a crude estimate of the projected camera motion. We subsequently postprocess the crude estimate with particle filters to obtain a smooth estimate. It is shown both theoretically and experimentally that particle filtering can reduce the error variance compared to estimation without particle filtering. The superior performance of our algorithm over other methods for video stabilization is demonstrated through computer simulated experiments.

References

[1]

N. Gordon, M. Arulampalam, S. Maskell, and T. Clapp, "A tutorial on particle filters for online nonlinear/non-gaussian bayesian tracking," IEEE Trans. Signal Process., vol. 50, no. 2, pp. 174-188, Feb. 2002.

Digital Library

[2]

S. Erturk, "Digital image stabilization With sub-image phase correlation based global motion estimation," IEEE Trans. Consumer Electron., vol. 49, no. 4, pp. 1320-1325, Nov. 2003.

Digital Library

[3]

S. Erturk and T. Dennis, "Image sequence stabilization based on DFT filtering," Proc. IEEE Image Vision Signal Process., vol. 127, no. 2, pp. 95-102, Apr. 2000.

[4]

A. Litvin, J. Konrad, and W. Karl, "Probabilistic video stabilization using kalman filtering and mosaicking," in Proc. Image Video Commun. IS&T/SPIE Symp. Electron. Imaging, Santa Clara, CA, pp. 663-674, 2003.

[5]

C. Morimoto and R. Chellappa, "Fast electronic digital image stabilization for off-road navigation," Real-Time Imaging, vol. 2, pp. 285-296, 1996.

Digital Library

[6]

C. Morimoto and R. Chellappa, "Fast 3-D stabilization and mosaic construction," in Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recognition, San Juan, Puerto Rico, 1997, pp. 660-665.

[7]

J. Jin, Z. Zhu, and G. Xu, "Digital video sequence stabilization based on 2.5-D motion estimation inertial motion filtering," Real-Time Imaging, vol. 7, pp. 357-365, 2001.

Digital Library

[8]

Z. Duric and A. Rosenfeld, "Image sequence stabilization in real time," Real-Time Imaging, vol. 2, pp. 271-284, 1996.

Digital Library

[9]

A. Censi, A. Fusiello, and V. Roberto, "Image stabilization by feature tracking," in Proc. Int. Conf. Anal. Process., Venice, Italy, 1999, pp. 665-667.

[10]

Y. Liang, H. Tyan, and S. Chen, "Video stabilization for a camcorder mounted on a moving vehicle," IEEE Trans. Veh. Technol., vol. 53, no. 6, pp. 1636-1648, Nov. 2004.

[11]

J. Chang, W. Hu, M. Cheng, and B. Chang, "Digital image translation and rotation motion stabilization using optical flow technique," IEEE Trans. Consumer Electron., vol. 48, no. 1, pp. 108-115, Feb. 2002.

Digital Library

[12]

S. Ko, S. Lee, S. Jeon, and E. Kang, "Fast digital image stabilizer based on gray-coded bit-plane matching," IEEE Trans. Consumer Electron., vol. 45, no. 3, pp. 598-603, Aug. 1999.

Digital Library

[13]

J. Pail, Y. Park, and D. Kim, "An adaptive motion decision system for digital image stabilizer based on edge pattern matching," IEEE Trans. Consumer Electron., vol. 38, no. 3, pp. 607-615, Aug. 1992.

Digital Library

[14]

S. Piva, M. Zara, G. Gera, and C. Reegazzoni, "Color-based video stabilization for real-time on-board object detection on high-speed trains," in Proc. IEEE Conf. Advanced Video Signal Based Surveillance, Miami, FL, 2003, pp. 299-304.

[15]

C. Chen, J. Yang, D. Schonfeld, and M. Mohamed, "Pose estimation from video sequences based on sylvester's equation," in Proc. SPIE Electron. Imaging, Conf. Visual Commun. Image Process., San Jose, CA, 2007, pp. IS.1-IS.12.

[16]

R. Kalman, "A new approach to linear filtering and prediction problems," Trans. ASME, J. Basic Eng. Ser. D, vol. 82, pp. 35-45, 1960.

[17]

N. Gordon, D. Salmond, and A. Smith, "Novel approach to non-linear and non-gaussian bayesian state estimation," Proc. IEEE-F, vol. 140, no. 2, pp. 107-113, Apr. 1993.

[18]

M. Isard and A. Blake, "Condensation conditional density propagation for visual tracking," Int. J. Comput. Vision, vol. 29, no. 1, pp. 5-28, Aug. 1998.

[19]

A. Doucet, J. Freitas, and N. Gordon, Sequential Monte Carlo Methods in Practice. 1st ed. New York: Springer-Verlag, 2001, ch. 2, sec. 2, pp. 15-38.

[20]

A. Doucet, S. Godsill, and C. Andrieu, "On sequential monte carlo sampling methods for bayesian filtering," Statist. Computing, vol. 10, no. 3, pp. 197-208, 2000.

[21]

M. Isard and A. Blake, "A smoothing filter for condensation," in Proc. 5th Eur. Conf. Comput. Vision, vol. 1, Freiburg, Germany, 1998, pp. 767-781.

[22]

M. Klass, M. Briers, N. de Freitas, A. Doucet, S. Maskell, and D. Lang, "Fast particle smoother: If I had a million particles," in Proc. 23rd Int. Conf. Mach. Learning, Pittsburgh, PA, 2006, pp. 25-29.

[23]

W. Fong, S. Godsill, A. Doucet, and M. West, "Monte carlo smoothing with application to audio signal enhancement," IEEE Trans. Signal Process., vol. 50, no. 2, pp. 438-449, Feb. 2002.

Digital Library

[24]

M. Isard and A. Blake, "Icondensation: Unifying low-level and high-level tracking in a stochastic framework," in Proc. 5th Eur. Conf. Comput. Vision, vol. 1. Freiburg, Germany, 1998, pp. 893-908.

[25]

Y. Rui and Y. Chen, "Better proposal distributions: Object tracking using unscented particle filter," in Proc. IEEE Conf. Comput. Vision Pattern Recognition, Kauai, Hawaii, 2001, pp. 786-793.

[26]

N. Bouaynaya, W. Qu, and D. Schonfeld, "An online motion-based particle filter for head tracking applications," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., Philadelphia, PA, 2005, pp. 225-228.

[27]

J. M. Odobez and D. Gatica-Perez, "Embedding motion in model-based stochastic tracking," in Proc. Int. Conf. Pattern Recognition, vol. 2. Cambridge, UK, 2004, pp. 815-818.

[28]

Y. Matsushita, E. Ofek, X. Tang, and H. Shum, "Full-frame video stabilization," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognition, San Diego, CA, 2005, pp. 50-57.

[29]

D. A. Forsyth and J. Ponce, Computer Vision: A Modern Approach. 1st ed. Englewood Cliffs, NJ: Prentice Hall, 2002, ch. 2, sec. 1.2, pp. 9-16.

[30]

R. Haralick, H. Joo, C. Lee, X. Zhuang, V. Vaidya, and M. Kim, "Pose estimation from corresponding point data," IEEE Trans. Syst., Man. Cybern., vol. 19, no. 6, pp. 1426-1446, Dec. 1989.

[31]

D. Crisan and A. Doucet, "A survey of convergence results on particle filtering methods for practitioners," IEEE Trans. Signal Process., vol. 50, no. 3, pp. 736-746, Mar. 2002.

Digital Library

[32]

J. MacCormick and M. Isard, "Partitioned sampling, articulated objects, and interface-quality hand tracking," in Proc. ECCV, vol. 2. Dublin, Ireland, 2000, pp. 3-19.

Digital Library

[33]

D. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vision, vol. 60, no. 2, pp. 91-110, Dec. 2004.

Digital Library

[34]

C. Morimoto and R. Chellappa, "Evaluation of image stabilization algorithms," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., vol. 5. Seattle, WA, 1998, pp. 2789-2792.

[35]

S. I. Resnick, A Probability Path. 1st ed. Cambridge, MA: Birkhauser, 2001, ch. 7, sec. 7.4, pp. 213-219.

Cited By

Lashkov IYuan RZhang G(2023)Edge-Computing-Empowered Vehicle Tracking and Speed Estimation Against Strong Image Vibrations Using Surveillance Monocular CameraIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.331807724:11(13486-13502)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1109/TITS.2023.3318077
Li XMo HWang FLi Y(2023)Real-Time and Robust Video Stabilization Based on Block-Wised Gradient FeaturesIEEE Transactions on Consumer Electronics10.1109/TCE.2023.330595369:4(1141-1151)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1109/TCE.2023.3305953
Wang YHuang QJiang CLiu JShang MMiao Z(2023)Video stabilizationNeurocomputing10.1016/j.neucom.2022.10.008516:C(205-230)Online publication date: 7-Jan-2023
https://dl.acm.org/doi/10.1016/j.neucom.2022.10.008
Show More Cited By

Robust video stabilization based on particle filter tracking of projected camera motion
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
  2. Computer graphics
    1. Animation

Recommendations

Spider monkey optimisation assisted particle filter for robust object tracking

Particle filters (PFs) are sequential Monte Carlo methods that use particle representation of state‐space model to implement the recursive Bayesian filter for non‐linear and non‐Gaussian systems. Owing to this property, PFs have been extensively used for ...
Geometric particle swarm optimization for robust visual ego-motion estimation via particle filtering

Conventional particle filtering-based visual ego-motion estimation or visual odometry often suffers from large local linearization errors in the case of abrupt camera motion. The main contribution of this paper is to present a novel particle filtering-...
Video stabilization using epipolar geometry

We present a new video stabilization technique that uses projective scene reconstruction to treat jittered video sequences. Unlike methods that recover the full three-dimensional geometry of the scene, this model accounts for simple geometric relations ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Circuits and Systems for Video Technology

IEEE Transactions on Circuits and Systems for Video Technology Volume 19, Issue 7

July 2009

163 pages

ISSN:1051-8215

Issue’s Table of Contents

Copyright © 2009.

Publisher

IEEE Press

Publication History

Published: 01 July 2009

Revised: 03 July 2008

Received: 13 February 2007

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lashkov IYuan RZhang G(2023)Edge-Computing-Empowered Vehicle Tracking and Speed Estimation Against Strong Image Vibrations Using Surveillance Monocular CameraIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.331807724:11(13486-13502)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1109/TITS.2023.3318077
Li XMo HWang FLi Y(2023)Real-Time and Robust Video Stabilization Based on Block-Wised Gradient FeaturesIEEE Transactions on Consumer Electronics10.1109/TCE.2023.330595369:4(1141-1151)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1109/TCE.2023.3305953
Wang YHuang QJiang CLiu JShang MMiao Z(2023)Video stabilizationNeurocomputing10.1016/j.neucom.2022.10.008516:C(205-230)Online publication date: 7-Jan-2023
https://dl.acm.org/doi/10.1016/j.neucom.2022.10.008
Sarıgül M(2023)A survey on digital video stabilizationMultimedia Tools and Applications10.1007/s11042-023-14726-182:26(40181-40207)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1007/s11042-023-14726-1
Long YFu K(2022)Side Auth: Synthesizing Virtual Sensors for AuthenticationProceedings of the 2022 New Security Paradigms Workshop10.1145/3584318.3584321(35-44)Online publication date: 24-Oct-2022
https://dl.acm.org/doi/10.1145/3584318.3584321
Roberto e Souza MMaia HPedrini H(2022)Survey on Digital Video Stabilization: Concepts, Methods, and ChallengesACM Computing Surveys10.1145/349452555:3(1-37)Online publication date: 3-Feb-2022
https://dl.acm.org/doi/10.1145/3494525
Kumar RGupta SVenkatesh K(2022)Quadratic smoothing based video stabilization using spatio-temporal regularity flowMultimedia Tools and Applications10.1007/s11042-022-13570-z82:7(10337-10366)Online publication date: 26-Aug-2022
https://dl.acm.org/doi/10.1007/s11042-022-13570-z
e Souza MPedrini H(2020)Visual rhythms for qualitative evaluation of video stabilizationJournal on Image and Video Processing10.1186/s13640-020-00508-42020:1Online publication date: 18-May-2020
https://dl.acm.org/doi/10.1186/s13640-020-00508-4
Zhao MLing Q(2020)PWStableNet: Learning Pixel-Wise Warping Maps for Video StabilizationIEEE Transactions on Image Processing10.1109/TIP.2019.296338029(3582-3595)Online publication date: 29-Jan-2020
https://dl.acm.org/doi/10.1109/TIP.2019.2963380
Unnikrishnan SSreelekha G(2020)Video stabilization performance enhancement for low-texture videosJournal of Real-Time Image Processing10.1007/s11554-019-00862-117:5(1135-1152)Online publication date: 1-Oct-2020
https://dl.acm.org/doi/10.1007/s11554-019-00862-1
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents