Cited By
View all- K VSen DRaman B(2021)Vector ordering and regression learning‐based ranking for dynamic summarisation of user videosIET Image Processing10.1049/iet-ipr.2020.023414:15(3941-3956)Online publication date: 12-Feb-2021
Music-Guided Video Summarization using Quadratic Assignments
Pages 58 - 64
Abstract
This paper aims to automatically generate a summary of an unedited video, guided by an externally provided music-track. The tempo, energy and beats in the music determine the choices and cuts in the video summarization. To solve this challenging task, we model video summarization as a quadratic assignment problem. We assign frames to the summary, using rewards based on frame interestingness, plot coherency, audio-visual match, and cut properties. Experimentally we validate our approach on the SumMe dataset. The results show that our music guided summaries are more appealing, and even outperform the current state-of-the-art summarization methods when evaluated on the F1 measure of precision and recall.
Supplementary Material
Supplemental video
- Download
- 45.56 MB
References
[1]
R. Datta, D. Joshi, J. Li, and J. Wang. 2006. Studying Aesthetics in Photographic Images Using a Computational Approach. In ECCV.
[2]
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 2009. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR.
[3]
N. Ejaz, I. Mehmood, and S. W. Baik. 2013. Efficient visual attention based framework for extracting key frames from videos. Signal Processing: Image Communication 28, 1 (2013), 34--44.
[4]
G. Evangelopoulos, A. Zlatintsi, A. Potamianos, P. Maragos, K. Rapantzikos, G. Skoumas, and Y. Avrithis. 2013. Multimodal Saliency and Fusion for Movie Summarization Based on Aural, Visual, and Textual Attention. IEEE Trans. Multimedia 15, 7 (2013), 1553--1568.
[5]
M. Gygli, H. Grabner, H. Riemenschneider, and L. Van Gool. 2014. Creating summaries from user videos. In ECCV.
[6]
M. Gygli, H. Grabner, and L. Van Gool. 2015. Video summarization by learning submodular mixtures of objectives. In CVPR.
[7]
X. Hou, J. Harel, and C. Koch. 2012. Image signature: Highlighting sparse salient regions. IEEE Trans. PAMI 34, 1 (2012), 194--201.
[8]
Y. Ke, X. Tang, and F. Jing. 2006. The Design of High-Level Features for Photo Quality Assessment. In CVPR.
[9]
T. Koopmans and M. Beckmann. 1957. Assignment problems and the location of economic activities. Econometrica (1957), 53--76.
[10]
P. Koutras, A. Zlatintsi, E. Iosif, A. Katsamanis, P. Maragos, and A. Potamianos. 2015. Predicting audio-visual salient events based on visual, audio and text modalities for movie summarization. In ICIP.
[11]
Y. Lee, J. Ghosh, and K. Grauman. 2012. Discovering important people and objects for egocentric video summarization. In CVPR.
[12]
Y. Li and B. Mérialdo. 2012. Video Summarization Based on Balanced AV-MMR. In MMM.
[13]
F. Liu, Y. Niu, and M. Gleicher. 2009. Using Web Photos for Measuring Video Frame Interestingness. In IJCAI.
[14]
Z. Lu and K. Grauman. 2013. Story-driven summarization for egocentric video. In CVPR. 2714--2721.
[15]
Y.-F. Ma, L. Lu, H.-J. Zhang, and M. Li. 2002. A user attention model for video summarization. In ACM Multimedia.
[16]
P. Mettes, D. Koelma, and C. Snoek. 2016. The ImageNet Shutte: Reorganized Pre-training for Video Event Detection. In ICMR.
[17]
S. Sahni and T. Gonzalez. 1976. P-complete approximation problems. J. ACM 23, 3 (1976), 555--565.
[18]
M. Smith and T. Kanade. 1998. Video skimming and characterization through the combination of image and language understanding. In Workshop on Content- Based Access of Image and Video Database.
[19]
B. Truong and S. Venkatesh. 2007. Video abstraction: A systematic review and classification. ACM TOMCAPP (2007).
[20]
J. Xu, L. Mukherjee, Y. Li, J. Warner, J. Rehg, and V. Singh. 2015. Gaze-enabled egocentric video summarization via constrained submodular maximization. In CVPR.
[21]
J. You, U. Reiter, M Hannuksela, M. Gabbouj, and A. Perkis. 2010. Perceptual- based quality assessment for audio--visual services: A survey. Signal Processing: Image Communication 25, 7 (2010), 482--501.
[22]
K. Zhang, W.-L Chao, F. Sha, and K. Grauman. 2016. Video Summarization with Long Short-Term Memory. In ECCV.
Index Terms
- Music-Guided Video Summarization using Quadratic Assignments
Recommendations
Towards genre-specific frameworks for video summarisation: A survey
Highlights- Identifies the need for genre specific models for video summarisation.
- Aids in ...
AbstractVideo summarisation is characterised as the process of extracting meaningful frames or segments from a video that best represents the content of the whole video. The proposed framework surveys and categorizes the existing video ...
A New Semidefinite Programming Relaxation for the Quadratic Assignment Problem and Its Computational Perspectives
Recent progress in solving quadratic assignment problems QAPs from the QAPLIB Quadratic Assignment Problem Library test set has come from mixed-integer linear or quadratic programming models that are solved in a branch-and-bound framework. Semidefinite ...
Bounds for the quadratic assignment problem using the bundle method
Semidefinite programming (SDP) has recently turned out to be a very powerful tool for approximating some NP-hard problems. The nature of the quadratic assignment problem (QAP) suggests SDP as a way to derive tractable relaxations. We recall some SDP ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
June 2017
524 pages
ISBN:9781450347013
DOI:10.1145/3078971
- General Chairs:
- Bogdan Ionescu,
- Nicu Sebe,
- Program Chairs:
- Jiashi Feng,
- Martha Larson,
- Rainer Lienhart,
- Cees Snoek
Copyright © 2017 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 06 June 2017
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- STW
- NWO
Conference
ICMR '17
Sponsor:
Acceptance Rates
ICMR '17 Paper Acceptance Rate 33 of 95 submissions, 35%;
Overall Acceptance Rate 254 of 830 submissions, 31%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 99Total Downloads
- Downloads (Last 12 months)1
- Downloads (Last 6 weeks)0
Reflects downloads up to 04 Jan 2025
Other Metrics
Citations
Cited By
View all- K VSen DRaman B(2021)Vector ordering and regression learning‐based ranking for dynamic summarisation of user videosIET Image Processing10.1049/iet-ipr.2020.023414:15(3941-3956)Online publication date: 12-Feb-2021
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in