Abstract
Depth-image-based-rendering (DIBR) plays an important role in view synthesis for free-viewpoint videos. The warping process in DIBR causes geometric displacement, which distributes intensively around edges, and the subsequent rendering process results in the impairment of edges. Traditional 2D image quality metrics are limited in the quality evaluation of DIBR-synthesized images. In this paper, we present a reduced-reference quality metric for DIBR-synthesized images by only extracting several feature values, namely multi-scale Edge Intensity Similarity (EIS). The original and synthesized images are first downsampled to generate images with different resolutions. Then an edge detection process is conducted on each scale and the edge intensity is calculated. The similarity of the edge intensity between each downsampled original image and the corresponding synthesized image is computed. Finally, the average similarity is calculated as the quality score of the DIBR-synthesized image. Experiments conducted on IRCCyN/IVC DIBR image and video databases demonstrate that the proposed method overall outperforms traditional 2D and existing DIBR-targeted quality metrics.
Similar content being viewed by others
References
Battisti F, Bosc E, Carli M, Callet PL (2015) Objective image quality assessment of 3D synthesized views. Sig Process.: Image Commun 30:78–88
Bosc E, Pépion R, Callet PL, Koppel M, Nya PN, Cedex R (2011) Towards a new quality metric for 3-D synthesized view assessment. IEEE J Select Top Signal Process 5(7):1332–1343
Bosc E, Pepion R, Callet PL, Koppel M, Ndijiki-Nya P, Morin L, Pressigout M (2011) Perceived quality of DIBR-based synthesized views. SPIE Optics+ Photonics 8135(1):8135–16
Canny JF (1986) A computational approach to edge detection. IEEE Trans Parr Anal Mach Intell 8(6):679–698
Cermak G, Thorpe L, Pinson M (2009) Test plan for evaluation of video quality models for use with high definition TV content. Video Quality Experts Group
Chandler D, Hemami S (2007) VSNR: A wavelet-based visual signal-to-noise ratio for natural images. IEEE Trans Image Process 16(9):2284–2298
Conze PH, Robert P, Morin L (2012) Objective view synthesis quality assessment. Electron Imag Int Society for Optics and Photonics 8288:8288–8256
Duds RO, Hart PE (1973) Pattern classification and scene analysis. Wiley, New York
Fan YC, Huang PK, Shen DW (2013) 3DTV depth map reconstruction based on structured light scheme. IEEE Int Instrum Meas Technol Conf (I2MTC), pp 835–838
Fehn C (2004) Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3-d-TV. SPIE Conf Stereoscopic Displays and Virtual Reality Systems X(5291):93–104
Gu K, Zhai GT, Yang XK, Zhang WJ (2013) A new reduced-reference image quality assessment using structural degradation model. IEEE International Symposium on Circuits and Systems, pp 1095–1098
Gu K, Zhai GT, Yang XK, Zhang WJ (2014) An efficient color image quality metric with local-tuned-global model. IEEE Int Conf Image Process, pp 506–510
Gu K, Wang SQ, Zhai GT, Lin WS, Yang XK, Zhang WJ (2016) Analysis of distortion distribution for pooling in image quality prediction. IEEE Trans Broadcasting 62:446–456
Gu K, Zhou J, Qiao JF, Zhai GT, Lin WS, Bovik AC (2017) No-reference quality assessment of screen content pictures. IEEE Trans Image Process 26(8):4005–4018
Gu K, Tao DC, Qiao JF, Lin WS (2017) Learning a no-reference quality assessment model of enhanced images with big data. IEEE Trans Neural Netw Learning Syst, pp 1–13
Hsu CC, Lin CW, Fang YM, Lin WS (2014) Objective quality assessment for image retargeting based on perceptual geometric distortion and information loss. IEEE Journal of Selected Topics in Signal Processing 8(3):377–389
Huszak A (2017) Advanced free viewpoint video streaming techniques. Multimed Tools Appl, https://doi.org/10.1007/s11042-015-3048-9
ITU-T p.910 (1999) Subjective video quality assessment methods for multimedia applications
Jain R, Kasturi R, Schunck BG (1995) Machine vision. McGraw-Hill, New York
Jung YJ, Kim HG, Ro YM (2016) Critical binocular asymmetry measure for perceptual quality assessment of synthesized stereo 3D images in view synthesis. IEEE Trans Circuits Syst Video Technol 26(7):1201–1214
Kang LW, Hsu CC, Zhuang BQ, Lin CW, Yeh CH (2015) Learning-based joint super-resolution and deblocking for a highly compressed image. IEEE Trans Multimedia 17(7):921–934
Kim HG, Ro YM (2016) Measurement of critical temporal inconsistency for quality assessment of synthesized video. IEEE Int Conf Image Process, pp 1027–1031
Koppel M, Nya PN, Doshkov D, Lakshman H, Merkle P, Mller K, Wiegand T (2010) Temporally consistent handling of disocclusions with texture synthesis for depth-image-based rendering. IEEE Int Conf Image Process, pp 1809–1812
Larson EC, Chandler DM (2010) Most apparent distortion: Full reference image quality assessment and the role of strategy. J Electr Imag 19(1):001006:1–21
Li B, Duan LY, Lin CW, Huang TJ, Gao W (2015) Depth-preserving warping for stereo image retargeting. IEEE Trans Image Process 24(9):2811–2826
Li LD, Cai H, Zhang YB, Lin WS, Kot AC, Sun XM (2016) Sparse representation based image quality index with adaptive sub-dictionaries. IEEE Trans Image Process 25(8):3775–3786
Li LD, Wu D, Wu JJ, Li HL, Lin WS, Kot AC (2016) Image sharpness assessment by sparse representation. IEEE Trans Multimedia 18(6):1085–1097
Li LD, Zhou Y, Lin WS, Wu JJ, Zhang XF, Chen BJ (2016) No-reference quality assessment of deblocked image. Neurocomputing 177:572–584
Liang Y, Liu YJ, Gutierrez D (2016) Objective quality prediction of image retargeting algorithms. IEEE Trans Vis Comput Graph 23(2):1099–1110
Liu YJ, Luo X, Xuan YM, Chen WF, Fu XL (2011) Image retargeting quality assessment. Computer Graphics Forum 30(2):583–592
Liu AM, Lin WS, Narwaria M (2012) Image quality assessment based on gradient similarity. IEEE Trans Image Process 21(4):1500–1512
Mori Y, Fukushima N, Yendo T, Fujii T, Tanimoto M (2009) View generation with 3-D warping using depth information for FTV. Signal Processing: Image Commu 24:65–72
Muller K, Smolic A, Dix K, Merkle P, Kauff P, Wiegand T (2008) View synthesis for advanced 3-D video systems. J Image Video Process, Article ID 438148
Narwaria M, Lin WS, McLoughlin IV, Emmanuel S, Chia LT (2012) Fourier transform-based scalable image quality measure. IEEE Trans Image Process 21(8):3364–3377
Nya PN, Koppel M, Doshkov D, Lakshman H, Merkle P, Muller K, Wiegand T (2010) Depth image based rendering with advanced texture synthesis. IEEE Int Conf Multimedia Expo, pp 424–429
Purica A, Valenzise G, Popescu BP, Dufaux F (2016) Using region-of-interest for quality evaluation of DIBR-based view synthesis methods. IEEE Int Workshop on Quality of Multimedia Experience
Ren TW, Wu GS (2010) Automatic image retargeting evaluation based on user perception. IEEE Int Conf Image Process, pp 1569–1572
Roberts LG (1965) Machine perception of three-dimensional solids. Optical and Electro-Optical Information Process 20:159–197
Ryu A, Kim S, Sohn K (2014) Synthesis quality prediction model based on distortion intolerance. IEEE Int Conf Image Process pp 585–589
Stankovic DS, Kukolj D, Callet PL (2015) DIBR Synthesized image quality assessment based on morphological wavelets. IEEE Int Workshop on Quality of Multimedia Experience, pp 1–6
Stankovic DS, Kukolj D, Callet PL (2015) DIBR Synthesized image quality assessment based on morphological pyramids. The True Vision-Capture, Transmission and Display of 3D Video, pp 1–4
Stankovic DS, Kukolj D, Callet PL (2016) Multi-scale synthesized view assessment based on morphological pyramids. J Electr Eng 67(1):3–11
Telea A (2004) An image inpainting technique based on the fast marching method. J Graphics, GPU Game Tools 9(1):23–34
Wang Z, Li Q (2011) Information content weighting for perceptual image quality assessment. IEEE Trans Image Process 20(5):1185–1198
Wang Z, Simoncelli EP (2005) Reduced-reference image quality assessment using a wavelet-domain natural image statistic model. SPIE 5666:149–159
Wang Z, Simoncelli EP, Bovik AC (2003) Multiscale structural similarity for image quality assessment. Asilomar Conf Signals, Syst Comput 2:1398–1402
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612
Wu JJ, Lin WS, Shi GM, Liu A (2013) Perceptual quality metric with internal generative mechanism. IEEE Trans Image Process 22(1):43–54
Wu JJ, Lin WS, Shi GM, Liu AM (2013) Reduced-reference image quality assessment with visual information fidelity. IEEE Trans Multimedia 15(7):1700–1705
Xie X, Zaitsev Y, Velsquez-Garca LF, Teller SJ, Livermore C (2014) Scalable, MEMS-enabled, vibrational tactile actuators for high resolution tactile displays. J Micromech Microeng 24(12):125014
Xie X, Zaitsev Y, Velasquez-Garcia LF, Teller SJ, Livermore C (2014) Compact, scalable, high-resolution, MEMS-enabled tactile displays. In: Proceedings of solid-state sensors, actuators, and microsystems workshop, pp 127–130
Xue WF, Zhang L, Mou XQ, Bovik AC (2014) Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE Trans Image Process 23(2):684–695
Yang Y, Wang X, Liu Q, Xu ML, Wu W (2017) User models of subjective image quality assessment on virtual viewpoint in free-viewpoint video system. Multimed Tools Appl, https://doi.org/10.1007/s11042-014-2321-7
Zhang L, Zhang L, Mou XQ, Zhang D (2011) FSIM: A feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386
Zhang L, Shen Y, Li H (2014) VSI: A visual saliency-induced index for perceptual image quality assessment. IEEE Trans Image Process 23(10):4270–4280
Acknowledgments
This work was supported in part by the Fundamental Research Funds for the Central Universities under Grant No. 2017XKQY084.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhou, Y., Yang, L., Li, L. et al. Reduced-reference quality assessment of DIBR-synthesized images based on multi-scale edge intensity similarity. Multimed Tools Appl 77, 21033–21052 (2018). https://doi.org/10.1007/s11042-017-5543-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-017-5543-7