[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

A Novel and Efficient Spatial–Temporal Saliency-Driven Integrated Video Compression

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

High-definition devices require the HD-videos for their proper utilization; however, it comprises several issues while transmission such as high computational complexity, long encoding time and restricted battery power. Moreover, several video compression algorithm has been introduced in past to solve the above-mentioned problem, however, due to the high-traffic video and low metrics of the existing algorithm, there is a requirement for an efficient algorithm. A major growth factor results in the contributions put forth towards video saliency, the existing methods perform saliency detection through a frame-wise approach that results in various challenges by incorporating an incoherent pixel-based saliency map detection that uses a spatio-temporal mechanism that utilizes frame-wise motion saliency with pixel-based temporal uniformity for diffusion purpose. This research develops an integrated video compression (IVC). At first, an effective and optimal spatio-temporal aware inter-frame and intra-frame-based saliency model is developed along with optimization modelling. Furthermore, two algorithms for designing a saliency map and optimized quantization for bitrate minimization. Performance analysis is carried out on a standard dataset; also comparison is carried out with existing state-of-art techniques to prove the model efficiency. IVC achieves better performance considering AUC, NCC, SIM and KL metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The dataset generated and analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Wang W, Shen J, Xie J, Cheng M-M, Ling H, Borji A. Revisiting video saliency prediction in the deep learning era. IEEE Trans Pattern Anal Mach Intell. 2021;43(1):220–37. https://doi.org/10.1109/TPAMI.2019.2924417.

    Article  PubMed  Google Scholar 

  2. Startsev M, Dorr M. Supersaliency: a novel pipeline for predicting smooth pursuit-based attention improves generalisability of video saliency. IEEE Access. 2020;8:1276–89. https://doi.org/10.1109/ACCESS.2019.2961835.

    Article  Google Scholar 

  3. Li H, Qi F, Shi G. A novel spatio-temporal 3D convolutional encoder-decoder network for dynamic saliency prediction. IEEE Access. 2021;9:36328–41. https://doi.org/10.1109/ACCESS.2021.3063372.

    Article  Google Scholar 

  4. Zhu S, Liu C, Xu Z. High-definition video compression system based on perception guidance of salient information of a convolutional neural network and HEVC Compression domain. IEEE Trans Circuits Syst Video Technol. 2020;30(7):1946–59. https://doi.org/10.1109/TCSVT.2019.2911396.

    Article  Google Scholar 

  5. Fang Y, Wang Z, Lin W, Fang Z. Video saliency incorporating spatiotemporal cues and uncertainty weighting. IEEE Trans Image Process. 2014;23(9):3910–21.

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  6. Seo H, Milanfar P. Static and space-time visual saliency detection by self-resemblance. J Vis. 2009;9(12):1–27.

    Article  Google Scholar 

  7. Fu H, Cao X, Tu Z. Cluster-based co-saliency detection. IEEE Trans Image Process. 2013;22(10):3766–78.

    Article  ADS  MathSciNet  PubMed  PubMed Central  Google Scholar 

  8. Wang W, Shen J, Porikli F. Saliency-aware geodesic video object segmentation. In: IEEE conference on computer vision and pattern recognition. 2015. pp. 3395–3402.

  9. Rahtu E, Kannala J, Salo M, Heikkila J. Segmenting salient objects from images and videos. In: European conference on computer vision. 2010. pp. 366–379.

  10. Wang W, Shen J, Shao L. Consistent video saliency using local gradient flow optimization and global refinement. IEEE Trans Image Process. 2015;24(11):4185–96.

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  11. Gastal E, Olive M. Domain transform for edge-aware image and video processing. ACM Trans Graph. 2011;30(4):1–12.

    Article  Google Scholar 

  12. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Susstrunk S. Slick superpixels. EPFL Technical report, 2010.

  13. Dong Z, Javed O, Shah M. Video object segmentation through spatially accurate and temporally dense extraction of primary object regions. In: IEEE conference on computer vision and pattern recognition. 2013. pp. 628–635.

  14. Wright J, Peng Y, Ma Y. Robust principal component analysis: exact recovery of corrupted low-rank matrices by convex optimization. Adv Neural Inf Process Syst 2009; 2080–2088.

  15. Zhou X, Yang C, Yu W. Moving object detection by detecting contiguous outliers in the low-rank representation. IEEE Trans Pattern Anal Mach Intell. 2013;35(3):597–610.

    Article  PubMed  Google Scholar 

  16. Zeng Z, Chan T, Jia K, Xu D. Finding correspondence from multiple images via sparse and low-rank decomposition. In: European conference on computer vision. 2012. pp. 1016–1021.

  17. Ji P, Li H, Dai MSY. Robust motion segmentation with unknown correspondences. In: European conference on computer vision. 2014. pp. 204–219.

  18. Oliveira R, Costeira J, Xavier J. Optimal point correspondence through the use of rank constraints. In: IEEE conference on computer vision and pattern recognition. 2005. pp. 1016–1021.

  19. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Earn. 2011;3:1–122.

    Google Scholar 

  20. James M. Algorithms for the assignment and transportation problems. J Soc Ind Appl Math. 1957;5(1):32–8.

    Article  MathSciNet  Google Scholar 

  21. Liu Z, Le Meur O, Luo S. Superpixel-based saliency detection. In: WIAMIS—14th international workshop on image and audio analysis for multimedia interactive services, Jul 2013, Paris, France. ffhal-00876184f

  22. Khatoonabadi SH, et al. How many bits does it take for a stimulus to be salient? Computer vision and pattern recognition IEEE. 2015. pp. 5501–5510.

  23. Leboran V, et al. Dynamic whitening saliency. IEEE Trans Pattern Anal Mach Intell. 2016;39(5):893–907.

    Article  PubMed  Google Scholar 

  24. He P, Li H, Wang H, Wang S, Jiang X, Zhang R. Frame-wise detection of double HEVC compression by learning deep spatio-temporal representations in compression domain. IEEE Trans Multimedia. 2021;23:3179–92. https://doi.org/10.1109/TMM.2020.3021234.

    Article  Google Scholar 

  25. Liu Z, Wang M, Chen F, Lu Q. Edge-assisted intelligent video compression for live aerial streaming. IEEE Trans Green Commun Netw. 2022;6(3):1613–23. https://doi.org/10.1109/TGCN.2022.3172900.

    Article  Google Scholar 

  26. Lu G, Zhang X, Ouyang W, Chen L, Gao Z, Xu D. An end-to-end learning framework for video compression. IEEE Trans Pattern Anal Mach Intell. 2021;43(10):3292–308. https://doi.org/10.1109/TPAMI.2020.2988453.

    Article  PubMed  Google Scholar 

  27. Yılmaz MA, Tekalp AM. End-to-end rate-distortion optimized learned hierarchical bi-directional video compression. IEEE Trans Image Process. 2022;31:974–83. https://doi.org/10.1109/TIP.2021.3138300.

    Article  ADS  PubMed  Google Scholar 

  28. Wang W, Shen J, Guo F, Cheng MM, Borji A. Revisiting video saliency: a large-scale benchmark and a new model. In: IEEE conference on computer vision and pattern recognition (CVPR). 2018. pp. 4894–4903.

  29. Hadizadeh H, Bajić IV. Saliency-aware video compression. IEEE Trans Image Process. 2014;23(1):19–33.

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  30. Xu M, Jiang L, Sun X, Ye Z, Wang Z. Learning to detect video saliency with HEVC features. IEEE Trans Image Process. 2017;26(1):369–85.

    Article  ADS  MathSciNet  Google Scholar 

Download references

Acknowledgements

The institutions such as REVA University, Bengaluru, Visvesvaraya Technological University, Belagavi, and Philips Research, Bengaluru, India, were recognized by the authors for their contributions to the research endeavour in the form of facilities that supported it.

Funding

No funding received for this research.

Author information

Authors and Affiliations

  1. School of Electronics and Communication, REVA University, Affiliated to VTU, Belagavi, Bengaluru, Karnataka, India

    R. D. Anitha Kumari

  2. Philips Research, Bengaluru, Karnataka, India

    A. Narendranath Udupa

Authors
  1. R. D. Anitha Kumari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Narendranath Udupa
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

RDAK selected the research issues, carried out the analysis, produced the article, and examined the simulation findings under the guidance and help of ANU.

Corresponding author

Correspondence to R. D. Anitha Kumari.

Ethics declarations

Conflict of interest

No conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the topical collection “Advances in Computational Approaches for Image Processing, Wireless Networks, Cloud Applications and Network Security” guest edited by P. Raviraj, Maode Ma and Roopashree H R.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Anitha Kumari, R.D., Udupa, A.N. A Novel and Efficient Spatial–Temporal Saliency-Driven Integrated Video Compression. SN COMPUT. SCI. 5, 289 (2024). https://doi.org/10.1007/s42979-023-02503-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-023-02503-8

Keywords

Search

Navigation