poster

Real-time summarization of user-generated videos based on semantic recognition

Authors:

Xi Wang,

Dong WangAuthors Info & Claims

MM '14: Proceedings of the 22nd ACM international conference on Multimedia

Pages 849 - 852

https://doi.org/10.1145/2647868.2655013

Published: 03 November 2014 Publication History

Get Access

Abstract

User-generated contents play an important role in the Internet video-sharing activities. Techniques for summarizing the user-generated videos (UGVs) into short representative clips are useful in many applications. This paper introduces an approach for UGV summarization based on semantic recognition. Different from other types of videos like movies or broadcasting news, where the semantic contents may vary greatly across different shots, most UGVs have only a single long shot with relatively consistent high-level semantics. Therefore, a few semantically representative segments are generally sufficient for a UGV summary, which can be selected based on the distribution of semantic recognition scores. In addition, due to the poor shooting quality of many UGVs, factors such as camera shaking and lighting condition are also considered to achieve more pleasant summaries. Experiments on over 100 UGVs with both subjective and objective evaluations show that our approach clearly outperforms several alternative methods and is highly efficient. Using a regular laptop, it can produce a summary for a 2-minute video in just 10 seconds.

References

[1]

Y.-G. Jiang. SUPER: towards real-time event recognition in internet videos. In Proceedings of ICMR, 2012.

Digital Library

Google Scholar

[2]

Y.-G. Jiang, G. Ye, S.-F. Chang, D. Ellis, and A. C. Loui. Consumer video understanding: A benchmark database and an evaluation of human and machine performance. In Proceedings of ICMR, 2011.

Digital Library

Google Scholar

[3]

A. Khosla, R. Hamid, C.-J. Lin, and N. Sundaresan. Large-scale video summarization using web-image priors. In Proceedings of CVPR, 2013.

Digital Library

Google Scholar

[4]

D. Liu, G. Hua, and T. Chen. A hierarchical visual model for video object summarization. IEEE TPAMI, 32(12):2178--2190, 2010.

Digital Library

Google Scholar

[5]

Z. Lu and K. Grauman. Story-driven summarization for egocentric video. In Proceedings of CVPR, 2013.

Digital Library

Google Scholar

[6]

Y. Luo and X. Tang. Photo and video quality evaluation: Focusing on the subject. In Proceedings of ECCV. 2008.

Digital Library

Google Scholar

[7]

T. Mei, L.-X. Tang, J. Tang, and X.-S. Hua. Near-lossless semantic video summarization and its applications to video analysis. ACM TOMCCAP, 9(3):1--23, 2013.

Digital Library

Google Scholar

[8]

C.-W. Ngo, Y.-F. Ma, and H.-J. Zhang. Video summarization and scene detection by graph modeling. IEEE TCSVT, 15(2):296--305, 2005.

Digital Library

Google Scholar

[9]

M. Wang, R. Hong, G. Li, Z.-J. Zha, S. Yan, and T.-S. Chua. Event driven web video summarization by tag localization and key-shot identification. IEEE TMM, 14(4):975--985, 2012.

Digital Library

Google Scholar

[10]

W.-Q. Yan and M. S. Kankanhalli. Detection and removal of lighting & shaking artifacts in home videos. In Proceedings of ACM Multimedia, 2002.

Digital Library

Google Scholar

[11]

H. J. Zhang, J. Wu, D. Zhong, and S. W. Smoliar. An integrated system for content-based video retrieval and browsing. Pattern Recognition, 30(4):643--658, 1997.

Crossref

Google Scholar

Cited By

View all

Thengane STan YPrasetyo MMeghjani M(2024)Online Informative Sampling Using Semantic Features in Underwater EnvironmentsOCEANS 2024 - Singapore10.1109/OCEANS51537.2024.10682405(1-6)Online publication date: 15-Apr-2024
https://doi.org/10.1109/OCEANS51537.2024.10682405
Wang JBai YLong YHu BChai ZGuan YWei XWen Chen CCucchiara RHua XQi GRicci EZhang ZZimmermann R(2020)Query TwiceProceedings of the 28th ACM International Conference on Multimedia10.1145/3394171.3414064(4023-4031)Online publication date: 12-Oct-2020
https://dl.acm.org/doi/10.1145/3394171.3414064
Tu GFu YLi BGao JJiang YXue X(2020)A Multi-Task Neural Approach for Emotion Attribution, Classification, and SummarizationIEEE Transactions on Multimedia10.1109/TMM.2019.292212922:1(148-159)Online publication date: 1-Jan-2020
https://dl.acm.org/doi/10.1109/TMM.2019.2922129
Show More Cited By

Index Terms

Real-time summarization of user-generated videos based on semantic recognition
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Video summarization

Recommendations

Tag suggestion and localization in user-generated videos based on social knowledge
WSM '10: Proceedings of second ACM SIGMM workshop on Social media

Nowadays, almost any web site that provides means for sharing user-generated multimedia content, like Flickr, Facebook, YouTube and Vimeo, has tagging functionalities to let users annotate the material that they want to share. The tags are then used to ...
What do you wish to see? A summarization system for movies based on user preferences

A novel personalized semantic movie summarization approach is proposed.User's subjective preferences are exploited for personalized movie summarization.An extensive user study demonstrates the advantages of the proposed system.A subjective study shows ...
Video summarization based on user log enhanced link analysis
MULTIMEDIA '03: Proceedings of the eleventh ACM international conference on Multimedia

Efficient video data management calls for intelligent video summarization tools that automatically generate concise video summaries for fast skimming and browsing. Traditional video summarization techniques are based on low-level feature analysis, which ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

MM '14: Proceedings of the 22nd ACM international conference on Multimedia

November 2014

1310 pages

ISBN:9781450330633

DOI:10.1145/2647868

General Chairs:
Kien A. Hua
University of Central Florida, USA
,
Yong Rui
Microsoft Research, China
,
Ralf Steinmetz
Technische Universitt Darmstadt, Germany
,
Program Chairs:
Alan Hanjalic
Delft University of Technology, Netherlands
,
Apostol (Paul) Natsev
Google, USA
,
Wenwu Zhu
Tsinghua University, China

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Funding Sources

Huawei Technologies
Science and Technology Commission of Shanghai Municipality
Key Technologies Research and Development Program
Ministry of Science and Technology of the People's Republic of China

Conference

MM '14

Sponsor:

SIGMM

MM '14: 2014 ACM Multimedia Conference

November 3 - 7, 2014

Florida, Orlando, USA

Acceptance Rates

MM '14 Paper Acceptance Rate 55 of 286 submissions, 19%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
206
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Thengane STan YPrasetyo MMeghjani M(2024)Online Informative Sampling Using Semantic Features in Underwater EnvironmentsOCEANS 2024 - Singapore10.1109/OCEANS51537.2024.10682405(1-6)Online publication date: 15-Apr-2024
https://doi.org/10.1109/OCEANS51537.2024.10682405
Wang JBai YLong YHu BChai ZGuan YWei XWen Chen CCucchiara RHua XQi GRicci EZhang ZZimmermann R(2020)Query TwiceProceedings of the 28th ACM International Conference on Multimedia10.1145/3394171.3414064(4023-4031)Online publication date: 12-Oct-2020
https://dl.acm.org/doi/10.1145/3394171.3414064
Tu GFu YLi BGao JJiang YXue X(2020)A Multi-Task Neural Approach for Emotion Attribution, Classification, and SummarizationIEEE Transactions on Multimedia10.1109/TMM.2019.292212922:1(148-159)Online publication date: 1-Jan-2020
https://dl.acm.org/doi/10.1109/TMM.2019.2922129
K. VSen DRaman B(2019)Video SkimmingACM Computing Surveys10.1145/334771252:5(1-38)Online publication date: 13-Sep-2019
https://dl.acm.org/doi/10.1145/3347712
Bai QHu QGe LHe L(2019)Stories That Big Danmaku Data Can Tell as a New MediaIEEE Access10.1109/ACCESS.2019.29090547(53509-53519)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2909054
Luo BLi HMeng FWu QNgan K(2018)An Unsupervised Method to Extract Video Object via Complexity Awareness and Object Local PartsIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2017.268257828:7(1580-1594)Online publication date: Jul-2018
https://doi.org/10.1109/TCSVT.2017.2682578
Xu BFu YJiang YLi BSigal L(2018)Heterogeneous Knowledge Transfer in Video Emotion Recognition, Attribution and SummarizationIEEE Transactions on Affective Computing10.1109/TAFFC.2016.26226909:2(255-270)Online publication date: 1-Apr-2018
https://doi.org/10.1109/TAFFC.2016.2622690
Gao ZLu GLyu CYan P(2018)Key-frame selection for automatic summarization of surveillance videosMachine Vision and Applications10.1007/s00138-018-0954-729:7(1101-1117)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1007/s00138-018-0954-7
Gao JFu YJiang YXue XIonescu BSebe NFeng JLarson MLienhart RSnoek C(2017)Frame-Transformer Emotion Classification NetworkProceedings of the 2017 ACM on International Conference on Multimedia Retrieval10.1145/3078971.3079030(78-83)Online publication date: 6-Jun-2017
https://dl.acm.org/doi/10.1145/3078971.3079030
Li KLi SOh SFu Y(2017)Videography-Based Unconstrained Video AnalysisIEEE Transactions on Image Processing10.1109/TIP.2017.267880026:5(2261-2273)Online publication date: 1-May-2017
https://dl.acm.org/doi/10.1109/TIP.2017.2678800
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Tag suggestion and localization in user-generated videos based on social knowledge

What do you wish to see? A summarization system for movies based on user preferences

Video summarization based on user log enhanced link analysis