Abstract
In real applications, the observed low-resolution face images usually have pose variations. Conventional learning-based methods ignore these variations; thus, the hallucinated high-resolution faces are not reasonable for the following recognition task. For recognition purpose, we prefer to obtain near-frontal faces. To this end, we propose a nuclear norm regularized structural orthogonal Procrustes regression (N2SOPR) approach in this work to acquire pose-robust feature representations for face hallucination with pose. The orthogonal Procrustes regression is used to seek an appropriate transformation between two data matrixes. Additionally, the nuclear norm regularization is imposed on the representation residual to preserve image structural property. We also impose a low-rank restraint on the combination weight to automatically cluster each input into the same subspace with the training samples. Both hallucination and recognition experiments conducted on common face databases have verified that our N2SOPR can obtain reasonable performance than some related methods.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Gao G, Yang J, Wu S et al (2015) Bayesian sample steered discriminative regression for biometric image classification. Appl Soft Comput 37:48–59
Huang P, Gao G (2016) Parameterless reconstructive discriminant analysis for feature extraction. Neurocomputing 190:50–59
Jing X-Y, Wu F, Zhu X et al (2016) Multi-spectral low-rank structured dictionary learning for face recognition. Pattern Recogn 59:14–25
Lai Z, Wong WK, Xu Y et al (2016) Approximate orthogonal sparse embedding for dimensionality reduction. IEEE Trans Neural Netw Learn Syst 27(4):723–735
Mudunuri SP, Biswas S (2016) Low resolution face recognition across variations in pose and illumination. IEEE Trans Pattern Anal Mach Intell 38(5):1034–1040
Shen F, Shen C, Zhou X et al (2016) Face image classification by pooling raw features. Pattern Recogn 54:94–103
Tai Y, Yang J, Zhang Y et al (2016) Face recognition with pose variations and misalignment via orthogonal Procrustes regression. IEEE Trans Image Process 25(6):2673–2683
Deng W, Hu J, Wu Z et al (2017) From one to many: pose-aware metric learning for single-sample face recognition. Pattern Recogn 77:426–437
Gao G, Yang J, Jing X-Y et al (2017) Learning robust and discriminative low-rank representations for face recognition with occlusion. Pattern Recogn 66:129–143
Yang J, Luo L, Qian J et al (2017) Nuclear norm based matrix regression with applications to face recognition with occlusion and illumination changes. IEEE Trans Pattern Anal Mach Intell 39(1):156–171
Yang M, Wang X, Zeng G et al (2017) Joint and collaborative representation with local adaptive convolution feature for face recognition with single sample per person. Pattern Recogn 66:117–128
Hamedani K, Seyyedsalehi SA, Ahamdi R (2016) Video-based face recognition and image synthesis from rotating head frames using nonlinear manifold learning by neural networks. Neural Comput Appl 27(6):1761–1769
Han B, He B, Sun T et al (2016) HSR: L 1/2-regularized sparse representation for fast face recognition using hierarchical feature selection. Neural Comput Appl 27(2):305–320
Zhu Y, Xue J (2017) Face recognition based on random subspace method and tensor subspace analysis. Neural Comput Appl 28(2):233–244
Wu F, Jing X-Y, Liu Q et al (2017) Large-scale image recognition based on parallel kernel supervised and semi-supervised subspace learning. Neural Comput Appl 28(3):483–498
Lan R, Zhou Y, Tang YY (2017) Quaternionic weber local descriptor of color images. IEEE Trans Circuits Syst Video Technol 27(2):261–274
Zou WW, Yuen PC (2012) Very low resolution face recognition problem. IEEE Trans Image Process 21(1):327–340
Freeman WT, Pasztor EC, Carmichael OT (2000) Learning low-level vision. Int J Comput Vis 40(1):25–47
Wang X, Tang X (2005) Hallucinating face by eigen transformation. IEEE Trans Syst Man Cybern Part C Appl Rev 35(3):425–434
Hu Y, Lam KM, Shen T et al (2011) A novel kernel-based framework for facial-image hallucination. Image Vis Comput 29(4):219–229
Shi J, Liu X, Qi C (2014) Global consistency, local sparsity and pixel correlation: a unified framework for face hallucination. Pattern Recogn 47(11):3520–3534
Huang H, He H, Fan X et al (2010) Super-resolution of human face image using canonical correlation analysis. Pattern Recogn 43(7):2532–2543
An L, Bhanu B (2014) Face image super-resolution using 2D CCA. Signal Process 103:184–194
Gao G, Yang J (2014) A novel sparse representation based framework for face image super-resolution. Neurocomputing 134:92–99
Wright J, Yang AY, Ganesh A et al (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227
Bo C, Wang D (2015) A registration-based tracking algorithm based on noise separation. Optik-Int J Light Electron Opt 126(24):5806–5811
Li F, Lu H, Wang D et al (2016) Dual group structured tracking. IEEE Trans Circuits Syst Video Technol 26(9):1697–1708
Zhao W, Lu H, Wang D (2018) Multisensor image fusion and enhancement in spectral total variation domain. IEEE Trans Multimed 20(4):866–879
Chang H, Yeung D-Y, Xiong Y (2004) Super-resolution through neighbor embedding. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, pp 1275–1282
Jiang J, Hu R, Wang Z et al (2014) Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning. IEEE Trans Image Process 23(10):4220–4231
Jiang J, Hu R, Wang Z et al (2016) Facial image hallucination through coupled-layer neighbor embedding. IEEE Trans Circuits Syst Video Technol 26(9):1674–1684
Yang J, Wright J, Huang TS et al (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873
Ma X, Zhang J, Qi C (2010) Hallucinating face by position-patch. Pattern Recogn 43(6):2224–2236
Jung C, Jiao L, Liu B et al (2011) Position-patch based face hallucination using convex optimization. IEEE Signal Process Lett 18(6):367–370
Wang Z, Hu R, Wang S et al (2014) Face hallucination via weighted adaptive sparse regularization. IEEE Trans Circuits Syst Video Technol 24(5):802–813
Jiang J, Hu R, Wang Z et al (2014) Noise robust face hallucination via locality-constrained representation. IEEE Trans Multimed 16(5):1268–1281
Jiang J, Ma J, Chen C et al (2017) Noise robust face image super-resolution through smooth sparse representation. IEEE Trans Cybern 47(11):3991–4002
Liu L, Chen CP, Li S et al (2018) Robust face hallucination via locality-constrained bi-layer representation. IEEE Trans Cybern 48(4):1189–1201
Dong C, Loy CC, He K et al (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307
Zhang Y, Li K, Li K et al (2018) Image super-resolution using very deep residual channel attention networks. arXiv preprint arXiv:1807.02758
Zhang Y, Tian Y, Kong Y et al (2018) Residual dense network for image super-resolution. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2472–2481
Kim S-J, Koh K, Lustig M et al (2007) An interior-point method for large-scale ℓ1-regularized least squares. IEEE J Sel Top Signal Process 1(4):606–617
Hurley JR, Cattell RB (1962) The Procrustes program: producing direct rotation to test a hypothesized factor structure. Behav Sci 7(2):258–262
Zhang F, Yang J, Tai Y et al (2015) Double nuclear norm-based matrix decomposition for occluded image recovery and background modeling. IEEE Trans Image Process 24(6):1956–1966
Chen J, Yang J, Luo L et al (2015) Matrix variate distribution-induced sparse representation for robust image classification. IEEE Trans Neural Netw Learn Syst 26(10):2291–2300
Cai JF, Candes EJ, Shen ZW (2010) A singular value thresholding algorithm for matrix completion. SIAM J Optim 20(4):1956–1982
Phillips PJ, Wechsler H, Huang J et al (1998) The FERET database and evaluation procedure for face-recognition algorithms. Image Vis Comput 16(5):295–306
Wang Z, Bovik AC, Sheikh HR et al (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612
Acknowledgements
This work was partially supported by the National Natural Science Foundation of China under Grant Nos. 61502245, 61772568, 61571236, 61473086, the Natural Science Foundation of Jiangsu Province under Grant No. BK20150849, the Fundamental Research Funds for the Central Universities of China (No. 18lgzd15), Open Fund Project of Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education (Nanjing University of Science and Technology) (No. JYB201709).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Gao, G., Zhu, D., Yang, M. et al. Face image super-resolution with pose via nuclear norm regularized structural orthogonal Procrustes regression. Neural Comput & Applic 32, 4361–4371 (2020). https://doi.org/10.1007/s00521-018-3826-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-018-3826-1