Abstract
The explosion of multimedia contents has resulted in a great demand of video presentation. While most previous works focused on presenting certain type of videos or summarizing videos by event detection, we propose a novel method to present general videos of different genres based on affective content analysis. We first extract rich audio-visual affective features and select discriminative ones. Then we map effective features into corresponding affective states in an improved categorical emotion space using hidden conditional random fields (HCRFs). Finally we draw affective curves which tell the types and intensities of emotions. With the curves and related affective visualization techniques, we select the most affective shots and concatenate them to construct affective video presentation with a flexible and changeable type and length. Experiments on representative video database from the web demonstrate the effectiveness of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Liu, C., Huang, Q., Jiang, S., et al.: A framework for flexible summarization of racquet sports video using multiple modalities. Computer Vision and Image Understanding 113(3), 415–424 (2009)
Zhao, Z., Jiang, S., Huang, Q., Zhu, G.: Highlight summarization in sports video based on replay detection. In: IEEE International Conference on Multimedia & Expo, pp. 1613–1616 (2006)
Xiang, X., Kankanhalli, M.: Affect-based adaptive presentation of home videos. In: ACM Multimedia, pp. 553–562 (2011)
Wang, M., Hong, R., Li, G., Zha, Z., Yan, S., Chua, T.: Event Driven Web Video Summarization by Tag Localization and Key-Shot Identification. IEEE Transactions on Multimedia 14(4), 975–985 (2012)
Wang, M., Hong, R., Yuan, X., Yan, S., Chua, T.: Movie2Comics: Towards a Lively Video Content Presentation. IEEE Transactions on Multimedia 14(3), 858–870 (2012)
Machajdik, J., Hanbury, A.: Affective image classification using features inspired by psychology and art theory. In: ACM Multimedia, pp. 83–92 (2010)
Kang, H.: Affective Content Detection Using HMMs. In: ACM Multimedia, pp. 259–262 (2003)
Xu, M., Jin, J., Luo, S.: Hierarchical Movie Affective Content Analysis Based on Arousal and Valence Features. In: ACM Multimedia, pp. 677–680 (2008)
Schlosberg, H.: Three Dimensions of Emotion. Psychological Review 61(2), 81–88 (1954)
Hanjalic, A., Xu, L.: Affective Video Content Representation and Modeling. IEEE Transactions on Multimedia 7(1), 143–154 (2005)
Arifin, S., Cheung, P.Y.K.: A Computation Method for Video Segmentation Utilizing the Pleasure-Arousal-Dominance Emotional Information. In: ACM Multimedia, pp. 68–77 (2007)
Zhang, S., Tian, Q., Huang, Q., Gao, W., Li, S.: Utilizing affective analysis for efficient movie browsing. In: IEEE International Conference on Image Processing, pp. 1853–1856 (2009)
Nicolaou, M.A., Gunes, H., Pantic, M.: A Multi-layer Hybrid Framework for Dimensional Emotion Classification. In: ACM Multimedia, pp. 933–936 (2011)
Ma, Y.-F., Lu, L., Zhang, H.-J., Li, M.: A User Attention Model for Video Summarization. In: ACM Multimedia (2002)
Joho, H., Staiano, J., Sebe, N., Jose, J.M.: Looking at the viewer: analysing facial activity to detect personal highlights of multimedia contents. Multimedia Tools Application 51(2), 505–523 (2011)
Joho, H., Jose, J.M., Valenti, R., Sebe, N.: Exploiting facial expressions for affective video summarization. In: ACM International Conference on Image and Video Retrieval (2009)
Zhao, S., Yao, H., Sun, X., Xu, P., Liu, X., Ji, R.: Video Indexing and Recommendation Based on Affective Analysis of Viewers. In: ACM Multimedia, pp. 1473–1476 (2011)
Zhao, S., Yao, H., Sun, X.: Video Classification and Recommendation Based on Affective Analysis of Viewers. Neurocomputing (to appear, 2012)
Yang, P., Liu, Q., Metaxas, D.N.: RankBoost with l 1 regularization for Facial Expression Recognition and Intensity Estimation. In: IEEE International Conference on Computer Vision, pp. 1018–1025 (2009)
Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: IEEE International Conference on Machine Learning (2001)
Quattoni, A., Collins, M., Darrell, T.: Conditional random fields for object recognition. In: Neural Information Processing Systems (2004)
Irie, G., Satou, T., Kojima, A., Yamasaki, T., Aizawa, K.: Affective Audio-Visual Words and Latent Topic Driving Model for Realizing Movie Affective Scene Classification. IEEE Transactions on Multimedia 16(2), 523–535 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhao, S., Yao, H., Sun, X., Jiang, X., Xu, P. (2013). Flexible Presentation of Videos Based on Affective Content Analysis. In: Li, S., et al. Advances in Multimedia Modeling. MMM 2013. Lecture Notes in Computer Science, vol 7732. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35725-1_34
Download citation
DOI: https://doi.org/10.1007/978-3-642-35725-1_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35724-4
Online ISBN: 978-3-642-35725-1
eBook Packages: Computer ScienceComputer Science (R0)