More Web Proxy on the site http://driver.im/

research-article

Content-Based Light Field Image Compression Method With Gaussian Process Regression

Authors:

Ali Abdullah YahyaAuthors Info & Claims

IEEE Transactions on Multimedia, Volume 22, Issue 4

Pages 846 - 859

https://doi.org/10.1109/TMM.2019.2934426

Published: 01 April 2020 Publication History

Abstract

Light field (LF) imaging enables new possibilities for digital imaging, such as digital refocusing, changing of focus plane, changing of viewpoint, scene-depth estimation, and 3D scene reconstruction, by capturing both spatial and angular information of light rays. However, one main problem in dealing with LF data is its sheer volume. In this context, efficient compression methods are needed for such a particular type of content. In this paper, we propose a content-based LF image-compression method with Gaussian process regression to improve the compression efficiency and accelerate the prediction procedure. First, the LF image is fed to the intra-frame codec of HEVC. In the prediction procedure, the prediction units (PUs) are classified as non-homogenous texture units, homogenous texture units, and visually flat units, based on the content property of the LF image. For each category, we design a corresponding Gaussian process regression (GPR)-based prediction method. Moreover, we propose a classification mechanism to exactly decide to which category the current PU belongs, so as to adjust the trade-off between the computational burden and the LF image coding efficiency. Experimental results demonstrate that the proposed LF image compression method is superior to several other state-of-the-art compression methods in terms of different quality metrics. Furthermore, the proposed method can also achieve a good visual quality of views rendered from decoded LF contents.

References

[1]

G. Wu et al., “Light field image processing: An overview,” IEEE J. Sel. Topics Signal Process., vol. 11, no. 7, pp. 926–954, Oct. 2017.

[2]

E. H. Adelson and J. R. Bergen, “The plenoptic function and the elements of early vision,” Comput. Models Vis. Process., Cambridge, MA, USA: MIT Press, pp. 3–20, 1991.

[3]

M. Levoy and P. Hanrahan, “Light field rendering,” in Proc. SIGGRAPH, pp. 31–42, 1996.

[4]

C. Michael, G. Steven, J. S. Richard, G. Radek, and S. Rick, “The lumigraph,” in Proc. SIGGRAPH, 1996.

[5]

T. T. Wong, C. W. Fu, and P. A. Heng, “The plenoptic illumination function,” IEEE Trans. Multimedia, vol. 4, no. 3, pp. 361–371, Sep. 2002.

Digital Library

[6]

M. Levoy, “Light fields and computational imaging,” Computer, vol. 39, pp. 46–55, Aug. 2006.

Digital Library

[7]

C. Hahne, A. Aggoun, S. Haxha, V. Velisavljevic, and J. C. J. Ferandez, “Light field geometry of a standard plenoptic camera,” Opt. Expr., vol. 22, no. 22, pp. 26659–26673, Nov. 2014.

[8]

A. Lumsdaine and T. Georgiev, “The focused plenoptic camera,” in Proc. IEEE Int. Conf. Comput. Photography, 2009, pp. 1–8.

[9]

T. Ebrahimi, JPEG Pleno Abstract and Executive Summary, ISO/IEC JTC 1/SC 29/WG1 N6922, Sydney, Australia, 2015.

[10]

“Working Draft 0.1 of TR: Technical Report on Immersive Media,” ISO/IEC JTC1/SC29/WG11/N16718, Geneva, Jan. 2017.

[11]

S. Zhu, M. Li, and C. Chen, “Cross-space distortion directed color image compression,” IEEE Trans. Multimedia, vol. 20, no. 3, pp. 525–538, Mar. 2018.

Digital Library

[12]

S. Li, M. Xu, Y. Ren, and Z. L. Wang, “Closed-form optimization on saliency-guided image compression for HEVC-MSP,” IEEE Trans. Multimedia, vol. 20, no. 1, pp. 155–170, Jan. 2018.

Digital Library

[13]

C. Deng, W. Lin, and J. Cai, “Content-based image compression for arbitrary-resolution display devices,” IEEE Trans. Multimedia, vol. 14, no. 4, pp. 1127–1139, Aug. 2012.

Digital Library

[14]

G. K. Wallace, “The JPEG still picture compression standard,” IEEE Trans. Consumer Electron., vol. 38, no. 1, pp. xviii–xxxiv, Feb. 1992.

Digital Library

[15]

G. J. Sullivan, J. R. Ohm, W. J. Han, and T. Wiegand, “Overview of the high efficiency video coding (HEVC) standard,” IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 12, pp. 1649–1668, Dec. 2012.

Digital Library

[16]

I. Viola et al., “Comparison and evaluation of light field image coding approaches,” IEEE J. Sel. Topics Signal Process., vol. 11, no. 7, pp. 1092–1106, Oct. 2017.

[17]

L. Li, Z. Li, B. Li, D. Liu, and H. Li, “Pseudo sequence based 2-D hierarchical coding structure for light-field image compression,” IEEE J. Sel. Topics Signal Process., vol. 11, no. 7, pp. 1107–1119, Oct. 2017.

[18]

F. Dai, J. Zhang, Y. Ma, and Y. Zhang, “Lenselet image compression scheme based on subaperture images streaming,” in Proc. IEEE Int. Conf. Image Process., 2015, pp. 4733–4737.

[19]

D. Liu, L. Wang, L. Li, ZhiweiXiong, F. Wu, and W. Zeng, “Pseudo-sequence-based light field image compression,” in Proc. IEEE Int. Conf. Multimedia Expo Workshops, 2016, pp. 1–4.

[20]

G. Wang, W. Xiang, M. Pickering, and C. W. Chen, “Light field multi-view video coding with two-directional parallel inter-view prediction,” IEEE Trans. Image Process., vol. 25, no. 11, pp. 5104–5117, Nov. 2016.

Digital Library

[21]

P. Helin, P. Astola, B. Rao, and I. Tabus, “Sparse modelling and predictive coding of subaperture images for lossless plenoptic image compression,” in Proc. 3DTV-Conf.: The True Vision-Capture, Transm. Display 3D Video (3DTV-CON), 2016, pp. 1–4.

[22]

L. Li, Z. Li, B. Li, D. Liu, and H. Li, “Pseudo sequence based 2-D hierarchical coding structure for light-field image compression,” in Proc. Data Compression Conf., 2017, pp. 131–140.

[23]

X. Jiang, M. Le Pendu, and C. Guillemot, “Light field compression using depth image based view synthesis,” in Proc. IEEE Int. Conf. Multimedia Expo Workshops, 2017, pp. 19–24.

[24]

X. Huang, P. An, L. Shen, and K. Li, “Light field image compression scheme based on MVD coding standard,” in Proc. Pacific-Rim Conf. Multimedia, 2017, pp. 1–5.

[25]

L. F. R. Lucas et al., “Locally linear embedding-based prediction for 3D holoscopic image coding using HEVC,” in Proc. 22nd Eur. Signal Process. Conf., 2014, pp. 11–15.

[26]

R. Monteiro et al., “Light field HEVC-based image coding using locally linear embedding and self-similarity compensated prediction,” in Proc. IEEE Int. Conf. Multimedia Expo Workshops, 2016, pp. 1–4.

[27]

R. J. S. Monteiro et al., “Light field image coding using high order intra block prediction,” IEEE J. Sel. Topics Signal Process., vol. 11, no. 7, pp. 1120–1131, Oct. 2017.

[28]

D. Liu, P. An, R. Ma, and L. Shen, “Disparity compensation based 3D holoscopic image coding using HEVC,” in Proc. IEEE China Summit Int. Conf. Signal Inf. Process., 2015, pp. 201–205.

[29]

D. Liu et al., “Three-dimensional holoscopic image coding scheme using high-efficiency video coding with kernel-based minimum mean-square-error estimation,” J. Electron. Imag., vol. 25, no. 4, 2016, Art. no.

[30]

D. Liu, P. An, R. Ma, C. Yang, and L. Shen, “3D holoscopic image coding scheme using HEVC with Gaussian process regression,” Signal Process.: Image Commun., vol. 47, pp. 438–451, 2016.

Digital Library

[31]

J. Xu, R. Joshi, and R. A. Cohen, “Overview of the emerging HEVC screen content coding extension,” IEEE Trans. Circuits Syst. Video Technol., vol. 26, no. 1, pp. 50–62, Jan. 2016.

Digital Library

[32]

C. Conti, L. D. Soares, and P. Nunes, “HEVC-based 3D holoscopicvideocoding using self-similarity compensated prediction,” Signal Process.: Image Commun., vol. 42, pp. 59–78, 2016.

Digital Library

[33]

C. Conti, P. Nunes, and L. D. Soares, “HEVC-based light field image coding with bi-predicted self-similarity compensation,” in Proc. IEEE Int. Conf. Multimedia Expo Workshops, 2016, pp. 1–4.

[34]

Y. Li, M. Sjostrom, R. Olsson, and U. Jennehag, “Coding of focused plenoptic contents by displacement intra prediction,” IEEE Trans. Circuits Syst. Video Technol., vol. 26, no. 7, pp. 1308–1319, Jul. 2016.

Digital Library

[35]

Z. Pan, J. Lei, Y. Zhang, X. Sun, and S. Kwong, “Fast motion estimation based on content property for low-complexity H.265/HEVC encoder,” IEEE Trans. Broadcast., vol. 62, no. 3, pp. 675–684, Sep. 2016.

[36]

C. E. Rassmussen and C. K. I. Williams, Gaussian Processes for Machine Learning. Cambridge, MA, USA: MIT Press, 2006.

[37]

J. Koloda, A. M. Peinado, and V. Sanchez, “Kernel-based MMSE multimedia signal reconstruction and its application to spatial error concealment,” IEEE Trans. Multimedia, vol. 16, no. 6, pp. 1729–1738, Oct. 2014.

[38]

D. Liu, P. An, C. Yang, R. Ma, and L. Shen, “Coding of 3D holoscopic image by using spatial correlation of rendered view images,” in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2017, pp. 2002–2006.

[39]

L. Shen, Z. Zhang, and Z. Liu, “Adaptive inter-mode decision for HEVC jointly utilizing inter-level and spatiotemporal correlations,” IEEE Trans. Circuits Syst. Video Technol., vol. 24, no. 10, pp. 1709–1722, Oct. 2014.

[40]

S. Wang et al., “Low complexity encoder optimization for HEVC,” J. Vis. Commun. Image Representation, vol. 35, pp. 120–131, 2016.

Digital Library

[41]

M. Rerabek, T. Bruylants, T. Ebrahimi, F. Pereira, and P. Schelkens, “Call for proposals and evaluation procedure,” in Proc. ICME Grand Challenge: Light Field Image Compression, Seattle, WA, USA, 2016, pp. 1–8.

[42]

T. Georgiev, “Gallery and lightfield data.” [Online]. Available: http://www.tgeorgiev.net/Gallery/, Accessed: Dec. 1, 2016.

[43]

F. Bossen, “Common HM test conditions and software reference configurations,” Joint Collaborative Team on Video Coding, Geneva, Doc. JCTVC-L1100, Jan. 2013.

[44]

Y. Cao, M. A. Brubaker, D. J. Fleet, and A. Hertzmann, “Efficient optimization for sparse Gaussian process regression,” IEEE Trans. Pattern Anal. Mach. Intell, vol. 37, no. 12, pp. 2415–2427, Dec. 2015.

Digital Library

[45]

C. Conti, P. Nunes, and L. D. Soares, “Light field image coding with jointly estimated self-similarity bi-prediction,” Signal Process.: Image Commun., vol. 60, 2018, pp. 144–159.

[46]

T. Georgiev and A. Lumsdaine, “Focused plenoptic camera and rendering,” J. Electron. Imag., vol. 19, 2010, Art. no.

[47]

D. Dansereau, “Light Field Toolbox v0.4,” [Online]. Available: http://www.mathworks.com/matlabcentral/fileexchange/49683-light-field-toolbox-v0-4

[48]

ISO/IEC JTC 1/SC 29/WG 1, JPEG, JPEG Pleno Light Field Coding Common Test Conditions, Doc. N80027, Berlin, Germany, Jul. 2018.

Cited By

Tong KJin XYang YWang CKang JJiang F(2024)Learned Focused Plenoptic Image Compression With Microimage Preprocessing and Global AttentionIEEE Transactions on Multimedia10.1109/TMM.2023.327274726(890-903)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3272747
Liu BZhao YJiang XJi XWang SLiu YWei J(2024)5-D Epanechnikov Mixture-of-Experts in Light Field Image CompressionIEEE Transactions on Image Processing10.1109/TIP.2024.341835033(4029-4043)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2024.3418350
Yang CYang JZhai YWang R(2024)FICNet: An End to End Network for Free-View Image CodingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.339015134:9(8848-8861)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3390151
Show More Cited By

Index Terms

Content-Based Light Field Image Compression Method With Gaussian Process Regression
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics
    1. Image compression

Index terms have been assigned to the content through auto-classification.

Recommendations

A refined VQ-based image compression method

This paper presents a refined VQ-based image compression method which modifies the traditional VQ-based image compression method. The refined VQ-based image compression method further lossless encodes the compression data which is generated by the ...
A Refined VQ-Based Image Compression Method

This paper presents a refined VQ-based image compression method which modifies the traditional VQ-based image compression method. The refined VQ-based image compression method further lossless encodes the compression data which is generated by the ...
Data compression for light-field rendering

Two light-field compression schemes are presented. The codecs are compared with regard to compression efficiency and rendering performance. The first proposed coder is based on video-compression techniques that have been modified to code the four-...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Multimedia

IEEE Transactions on Multimedia Volume 22, Issue 4

April 2020

279 pages

ISSN:1520-9210

Issue’s Table of Contents

1520-9210 © 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Press

Publication History

Published: 01 April 2020

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tong KJin XYang YWang CKang JJiang F(2024)Learned Focused Plenoptic Image Compression With Microimage Preprocessing and Global AttentionIEEE Transactions on Multimedia10.1109/TMM.2023.327274726(890-903)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3272747
Liu BZhao YJiang XJi XWang SLiu YWei J(2024)5-D Epanechnikov Mixture-of-Experts in Light Field Image CompressionIEEE Transactions on Image Processing10.1109/TIP.2024.341835033(4029-4043)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2024.3418350
Yang CYang JZhai YWang R(2024)FICNet: An End to End Network for Free-View Image CodingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.339015134:9(8848-8861)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3390151
Huang YLi LYang YLi YGuo Y(2023)Explainable and Generalizable Blind Image Quality Assessment via Semantic Attribute ReasoningIEEE Transactions on Multimedia10.1109/TMM.2022.322572825(7672-7685)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2022.3225728
Sun YLi LLi ZWang SLiu SLi G(2023)Learning a Compact Spatial-Angular Representation for Light FieldIEEE Transactions on Multimedia10.1109/TMM.2022.321967125(7262-7273)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2022.3219671
Liu DHuang YFang YZuo YAn P(2023)Multi-Stream Dense View Reconstruction Network for Light Field Image CompressionIEEE Transactions on Multimedia10.1109/TMM.2022.317502325(4400-4414)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2022.3175023
Sun JZhao YWang SWei J(2023)3D Holoscopic Image Compression Based on Gaussian Mixture ModelIEEE Transactions on Multimedia10.1109/TMM.2022.314160625(1374-1389)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2022.3141606
Zhang ZTian SZou WMorin LZhang L(2023)EDDMF: An Efficient Deep Discrepancy Measuring Framework for Full-Reference Light Field Image Quality AssessmentIEEE Transactions on Image Processing10.1109/TIP.2023.332966332(6426-6440)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TIP.2023.3329663
Lv CZhou XZhu BLiu DZheng BZhang JYan C(2023)SRI-NetJournal of Visual Communication and Image Representation10.1016/j.jvcir.2022.10372190:COnline publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1016/j.jvcir.2022.103721
Amirpour HPinheiro APereira MLopes FGhanbari M(2022)Efficient Light Field Image Compression with Enhanced Random AccessACM Transactions on Multimedia Computing, Communications, and Applications10.1145/347190518:2(1-18)Online publication date: 23-Mar-2022
https://dl.acm.org/doi/10.1145/3471905
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents