More Web Proxy on the site http://driver.im/

research-article

FSCAV: fast seam carving for size adaptation of videos

Authors:

Johannes Kiess,

Hendrik Lemelson,

Wolfgang EffelsbergAuthors Info & Claims

MM '09: Proceedings of the 17th ACM international conference on Multimedia

Pages 321 - 330

https://doi.org/10.1145/1631272.1631317

Published: 19 October 2009 Publication History

Abstract

The presentation of multimedia data and especially of high resolution videos on small mobile devices is still a great challenge today. Both cropping of borders and scaling of frames may result in the removal of essential content of videos or lost details due to the reduced size of the visual content. Another major problem emerges if the aspect ratio of the original video and the display of the mobile device differ. User evaluations indicate that changing the aspect ratio may reduce the visual quality of videos significantly. In this paper, we present the new FSCAV algorithm (Fast Seam Carving for Size Adaptation of Videos) to adapt the size of videos to the limited display resolution and different aspect ratios of handheld mobile devices. The general idea of the seam carving algorithm for still images is to remove seams in images so that the essential content is preserved. We extended this technique which works very well for images to create videos without jitter or visible artifacts. A major feature of our FSCAV algorithm is the low computational complexity which enables an efficient adaptation of videos to small screens. Nevertheless, severe distortions are clearly visible in some shots of the adapted videos. We present a new heuristic to identify shots with such a low visual quality. If the quality drops below a threshold, a different adaptation technique is used for this shot (e.g., scaling or cropping). User evaluations confirm a very high visual quality of our approach.

References

[1]

S. Avidan and A. Shamir. Seam carving for content-aware image resizing. ACM Transactions on Graphics, SIGGRAPH 2007, 26(3), 2007.

Digital Library

[2]

Y. Boykov and V. Kolmogorov. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. In IEEE Transactions on PAMI, volume 26(9), pages 1124--1137, September 2004.

Digital Library

[3]

H. El-Alfy, D. Jacobs, and L. Davis. Multi-scale video cropping. MM '07: ACM international conference on Multimedia, pages 97--106, 2007.

Digital Library

[4]

X. Fan, X. Xie, H.-Q. Zhou, and W.-Y. Ma. Looking into video frames on small displays. MM '03: ACM international conference on Multimedia, pages 247--250, 2003.

Digital Library

[5]

D. Farin. Automatic Video Segmentation Employing Object/Camera Modeling. PhD thesis, Technische Universiteit Eindhoven, Einhoven, The Netherlands, 2005.

[6]

D. Farin, T. Haenselmann, S. Kopf, G. Kühne, and W. Effelsberg. Segmentation and classification of moving video objects. In B. Furht and O. Marques, editors, Handbook of Video Databases: Design and Applications, volume 8 of Internet and Communications Series, pages 561--591. CRC Press, Boca Raton, FL, USA, September 2003.

[7]

M. Fischler and R. Bolles. Random sample concensus: A paradigm for model fitting with applications to image analysis and automated cartography. In Communications ACM, volume 24(6), pages 381--395. ACM Press, 1981.

Digital Library

[8]

C. Harris and M. Stephens. A combined corner and edge detector. In Proceedings of Alvey Vision Conference, pages 147--151, 1988.

[9]

S. Kopf and W. Effelsberg. Mobile cinema: Canonical processes for video adaptation. In Multimedia Systems, volume 14(6), pages 369--375. Springer, December 2008.

[10]

S. Kopf, T. Haenselmann, D. Farin, and W. Effelsberg. Automatic generation of summaries for the web. In Proceedings of IS&T/SPIE conference on Storage and Retrieval for Media Databases, volume 5307, pages 417--428, Januar 2004.

[11]

S. Kopf, F. Lampi, T. King, and W. Effelsberg. Automatic scaling and cropping of videos for devices with limited screen resolution. In ACM Multimedia (video program session), pages 957--958, October 2006.

Digital Library

[12]

S. Kopf, F. Lampi, T. King, and W. Effelsberg. Automatic scaling and cropping of videos for devices with limited screen resolution. In Proceedings of the 14th ACM international conference on Multimedia, pages 957--958. ACM Press, Oktober 2006.

Digital Library

[13]

F. Liu and M. Gleicher. Video retargeting: Automating pan and scan. MM '06: ACM international conference on Multimedia, pages 241--250, 2006.

Digital Library

[14]

D. G. Lowe. Distinctive image features from scale-invariant keypoints. In International Journal of Computer Vision, volume 60(2), pages 91--110. Kluwer Academic Publishers, November 2004.

Digital Library

[15]

H. Moravec. Visual mapping by a robot rover. In Proceedings of the 6th International Joint Conference on Artificial Intelligence, pages 599 -- 601, August 1979.

Digital Library

[16]

M. Rubinstein, S. Avidan, and A. Shamir. Improved seam carving for video retargeting. ACM Transactions on Graphics, SIGGRAPH 2008, 27(3), 2008.

Digital Library

[17]

A. Shamir and S. Avidan. Seam carving for media retargeting. Commun. ACM, 52(1):77--85, 2009.

Digital Library

[18]

S. M. Smith and J. M. Brady. Susan - new approach to low level image processing. In International Journal of Computer Vision (IJCV), volume 23(1), pages 45 -- 78, May 1997.

Digital Library

[19]

C. Tao, J. Jia, and H. Sun. Active window oriented dynamic video retargeting. Proceedings of the Workshop on Dynamical Vision, ICCV 2007, 2007.

[20]

J. Wang, M. Reinders, R. Lagendijk, J. Lindenberg, and M. Kankanhalli. Video content presentation on tiny devices. ICME '04: IEEE International Conference on Multimedia and Expo, pages 1711--1714, 2004.

[21]

L. Wolf, M. Guttmann, and D. Cohen-Or. Non-homogeneous content-driven video-retargeting. In Proceedings of the Eleventh IEEE International Conference on Computer Vision (ICCV-07), 2007.

Cited By

Le THuang HChen YLee T(2024)Retargeting Video With an End-to-End FrameworkIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332782530:9(6164-6176)Online publication date: Sep-2024
https://doi.org/10.1109/TVCG.2023.3327825
Kiess JKopf SGuthier BEffelsberg W(2018)A Survey on Content-Aware Image and Video RetargetingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/323159814:3(1-28)Online publication date: 24-Jul-2018
https://dl.acm.org/doi/10.1145/3231598
Rijsselbergen DPoppe CVerwaest MMannens EWalle R(2018)Semantic MasteringMultimedia Tools and Applications10.1007/s11042-010-0710-059:1(307-340)Online publication date: 31-Dec-2018
https://dl.acm.org/doi/10.1007/s11042-010-0710-0
Show More Cited By

Index Terms

FSCAV: fast seam carving for size adaptation of videos
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
2. Information systems
  1. Information systems applications
    1. Multimedia information systems

Recommendations

Improved seam carving for video retargeting

Video, like images, should support content aware resizing. We present video retargeting using an improved seam carving operator. Instead of removing 1D seams from 2D images we remove 2D seam manifolds from 3D space-time volumes. To achieve this we ...
Algorithms for video retargeting

The visualization of high resolution video on small mobile devices is still a great challenge today. Most critical are the limited display resolution and different aspect ratios of handheld mobile devices. So far, there is no retargeting algorithm ...
Fast JND-based video carving with GPU acceleration for real-time video retargeting

A recently developed image resizing technique, seam carving, has been proved to be a useful tool for content-adaptive spatially nonuniform image resizing with the purpose of optimal display on a screen of reduced resolution or different aspect ratio. In ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '09: Proceedings of the 17th ACM international conference on Multimedia

October 2009

1202 pages

ISBN:9781605586083

DOI:10.1145/1631272

General Chairs:
Wen Gao
Peking University, China
,
Yong Rui
Microsoft, China
,
Alan Hanjalic
Delft University of Technology, The Netherlands
,
Program Chairs:
Changsheng Xu
Institute of Automation, Chinese Academy of Sciences, China
,
Eckehard Steinbach
Technical University of Munich, Germany
,
Abdulmotaleb El Saddik
University of Ottawa, Canada
,
Michelle Zhou
IBM T. J. Watson Research Center, USA

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM09

Sponsor:

SIGMM

MM09: ACM Multimedia Conference

October 19 - 24, 2009

Beijing, China

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
519
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Le THuang HChen YLee T(2024)Retargeting Video With an End-to-End FrameworkIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332782530:9(6164-6176)Online publication date: Sep-2024
https://doi.org/10.1109/TVCG.2023.3327825
Kiess JKopf SGuthier BEffelsberg W(2018)A Survey on Content-Aware Image and Video RetargetingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/323159814:3(1-28)Online publication date: 24-Jul-2018
https://dl.acm.org/doi/10.1145/3231598
Rijsselbergen DPoppe CVerwaest MMannens EWalle R(2018)Semantic MasteringMultimedia Tools and Applications10.1007/s11042-010-0710-059:1(307-340)Online publication date: 31-Dec-2018
https://dl.acm.org/doi/10.1007/s11042-010-0710-0
Han DSonka MBayouth JWu X(2018)Optimal multiple-seams search for image resizing with smoothness and shape priorThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-010-0480-826:6-8(749-759)Online publication date: 28-Dec-2018
https://dl.acm.org/doi/10.1007/s00371-010-0480-8
Du HXu ZYan Z(2016)Video Retargeting for Intelligent Sensing of Surveillance DevicesFrontier Computing10.1007/978-981-10-0539-8_119(1173-1178)Online publication date: 20-Apr-2016
https://doi.org/10.1007/978-981-10-0539-8_119
Lin XMa YMa LZhang R(2014)A survey for image resizingJournal of Zhejiang University SCIENCE C10.1631/jzus.C140010215:9(697-716)Online publication date: 23-Sep-2014
https://doi.org/10.1631/jzus.C1400102
Mikami DOchi DKojima A(2014)Convex Object Surface Mapping for Wide-field of View Video RepresentationIEEJ Transactions on Electronics, Information and Systems10.1541/ieejeiss.134.1451134:10(1451-1457)Online publication date: 2014
https://doi.org/10.1541/ieejeiss.134.1451
Katti HRajagopal AKankanhalli MKalpathi R(2014)Online Estimation of Evolving Human Visual InterestACM Transactions on Multimedia Computing, Communications, and Applications10.1145/263228411:1(1-21)Online publication date: 4-Sep-2014
https://dl.acm.org/doi/10.1145/2632284
Kiess JGritzner DGuthier BKopf SEffelsberg WZimmermann R(2014)GPU video retargeting with parallelized SeamCropProceedings of the 5th ACM Multimedia Systems Conference10.1145/2557642.2557648(139-147)Online publication date: 19-Mar-2014
https://dl.acm.org/doi/10.1145/2557642.2557648
Du HLiu ZYan ZJiang Y(2014)Video Retargeting Based on Stretchability-Aware Blocks ScalingProceedings of the 2014 10th International Conference on Semantics, Knowledge and Grids10.1109/SKG.2014.9(171-176)Online publication date: 27-Aug-2014
https://dl.acm.org/doi/10.1109/SKG.2014.9
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents