Abstract
Video summarization consists of generating a concise video representation that captures all its meaningful information. However, conventional summarization techniques often fall short of capturing all the significant events in a video due to their inability to incorporate the hierarchical structure of the video content. This work proposes an unsupervised method, named Hierarchical Time-aware Summarizer–HieTaSumm, that uses a hierarchical approach for that task. In this regard, hierarchical strategies for video summarization have emerged as a promising solution, in which video content is modeled as a graph to identify keyframes that represent the most relevant information. This approach enables the extraction of the frames that convey the central message of the video, resulting in a more effective and precise summary. Experimental results indicate that the proposed approach has great potential. Specifically, it seems to enhance coherence among different video segments, reducing frame redundancy in the generated summaries, and enhancing the diversity of selected keyframes.
Code available at https://github.com/IMScience-PPGINF-PucMinas/HieTaSumm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Apostolidis, E., Adamantidou, E., Metsai, A.I., Mezaris, V., Patras, I.: Video summarization using deep neural networks: a survey. Proc. IEEE 109(11), 1838–1863 (2021)
Asha Paul, M.K., Kavitha, J., Jansi Rani, P.A.: Key-frame extraction techniques: a review. Recent Patents Comput. Sci. 11(1), 3–16 (2018)
Basavarajaiah, M., Sharma, P.: Survey of compressed domain video summarization techniques. ACM Comput. Surv. 52(6), 1–29 (2019)
Cardoso, L.V., Guimaraes, S.J.F., Patrocínio, Z.K.G.: Enhanced-memory transformer for coherent paragraph video captioning. In: 2021 IEEE 33rd International Conference on Tools with Artificial Intelligence (ICTAI), pp. 836–840. IEEE (2021)
Cardoso, L.V., Guimaraes, S.J.F., Patrocinio, Z.K.G.: Exploring adaptive attention in memory transformer applied to coherent video paragraph captioning. In: 2022 IEEE Eighth International Conference on Multimedia Big Data (BigMM), pp. 37–44. IEEE (2022)
Cardoso, L.V., Guimaraes, S.J.F., Patrocinio Junior, Z.K.G.: Hierarchical time-aware summarization with an adaptive transformer for video captioning. Int. J. Semant. Comput. (2023)
Cousty, J., Najman, L.: Incremental algorithm for hierarchical minimum spanning forests and saliency of watershed cuts. In: Soille, P., Pesaresi, M., Ouzounis, G.K. (eds.) ISMM 2011. LNCS, vol. 6671, pp. 272–283. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21569-8_24
Cousty, J., Najman, L., Kenmochi, Y., Guimarães, S.: Hierarchical segmentations with graphs: quasi-flat zones, minimum spanning trees, and saliency maps. J. Math. Imaging Vis. 60(4), 479–502 (2018)
De Avila, S.E.F., Lopes, A.P.B., da Luz Jr., A., de Albuquerque Araújo, A.: Vsumm: A mechanism designed to produce static video summaries and a novel evaluation method. Pattern Recognit. Lett. 32(1), 56–68 (2011)
Ejaz, N., Tariq, T.B., Baik, S.W.: Adaptive key frame extraction for video summarization using an aggregation mechanism. J. Vis. Commun. Image Represent. 23(7), 1031–1040 (2012)
Furini, M., Geraci, F., Montangero, M., Pellegrini, M.: VISTO: visual storyboard for web video browsing. In: Proceedings of the 6th ACM International Conference on Image and Video Retrieval, pp. 635–642 (2007)
Guimarães, S., Kenmochi, Y., Cousty, J., Patrocinio, Z., Najman, L.: Hierarchizing graph-based image segmentation algorithms relying on region dissimilarity: the case of the felzenszwalb-huttenlocher method. Math. Morphol.-Theory Appl. 2(1), 55–75 (2017)
Lu, G., Zhou, Y., Li, X., Yan, P.: Unsupervised, efficient and scalable key-frame selection for automatic summarization of surveillance videos. Multimed. Tools Appl. 76, 6309–6331 (2017)
del Molino, A.G., Tan, C., Lim, J.H., Tan, A.H.: Summarization of egocentric videos: a comprehensive survey. IEEE Trans. Hum.-Mach. Syst. 47(1), 65–76 (2017)
Panda, R., Mithun, N.C., Roy-Chowdhury, A.K.: Diversity-aware multi-video summarization. IEEE Trans. Image Process. 26(10), 4712–4724 (2017)
Pandey, S., Dwivedy, P., Meena, S., Potnis, A.: A survey on key frame extraction methods of a mpeg video. In: 2017 International Conference on Computing, Communication and Automation (ICCCA), pp. 1192–1196 (2017)
dos Santos Belo, L., Caetano Jr., C.A., Patrocínio Jr., Z.K.G., Guimarães, S.J.F.: Summarizing video sequence using a graph-based hierarchical approach. Neurocomputing 173, 1001–1016 (2016)
Song, Y., Vallmitjana, J., Stent, A., Jaimes, A.: TVSum: summarizing web videos using titles. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5179–5187 (2015)
Tiwari, V., Bhatnagar, C.: A survey of recent work on video summarization: approaches and techniques. Multimed. Tools Appl. 80(18), 27187–27221 (2021)
Vivekraj, V., Debashis, S., Balasubramanian, R.: Video skimming: taxonomy and comprehensive survey. ACM Comput. Surv. 52(5) (2019)
Acknowledgements
The authors would like to thank Conselho Nacional de Desenvolvimento Científico e Tecnológico - CNPq - (Universal 407242/2021-0 and PQ 306573/2022-9), and Fundação de Amparo à Pesquisa do Estado de Minas Gerais - FAPEMIG - (Grants PPM- 00006-18). This study was also financed in part by PUC Minas and by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cardoso, L.V., Gomes, G.O.R., Guimarães, S.J.F., do Patrocínio Júnior, Z.K.G. (2023). Hierarchical Time-Aware Approach for Video Summarization. In: Naldi, M.C., Bianchi, R.A.C. (eds) Intelligent Systems. BRACIS 2023. Lecture Notes in Computer Science(), vol 14195. Springer, Cham. https://doi.org/10.1007/978-3-031-45368-7_18
Download citation
DOI: https://doi.org/10.1007/978-3-031-45368-7_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45367-0
Online ISBN: 978-3-031-45368-7
eBook Packages: Computer ScienceComputer Science (R0)