Abstract
We propose a new framework for image-based three-dimensional (3D) model retrieval. We first model the query image as a Euclidean point. Then we model all projected views of a 3D model as a symmetric positive definite (SPD) matrix, which is a point on a Riemannian manifold. Thus, the image-based 3D model retrieval is reduced to a problem of Euclid-to-Riemann metric learning. To solve this heterogeneous matching problem, we map the Euclidean space and SPD Riemannian manifold to the same high-dimensional Hilbert space, thus shrinking the great gap between them. Finally, we design an optimization algorithm to learn a metric in this Hilbert space using a kernel trick. Any new image descriptors, such as the features from deep learning, can be easily embedded in our framework. Experimental results show the advantages of our approach over the state-of-the-art methods for image-based 3D model retrieval.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bai S, Bai X, Zhou Z, et al., 2016. GIFT: a real-time and scalable 3D shape search engine. 16th IEEE Conf on Computer Vision and Pattern Recognition, p.5023–5032. https://doi.org/10.1109/CVPR.2016.543
Bai X, Bai S, Zhu Z, et al., 2015. 3D shape matching via two layer coding. IEEE Trans Patt Anal Mach Intell, 37(12): 2361–2373. https://doi.org/10.1109/TPAMI.2015.2424863
Cevikalp H, Triggs B, 2010. Face recognition based on image sets. IEEE Society Conf on Computer Vision and Pattern Recognition, p.2567–2573. https://doi.org/10.1109/CVPR.2010.5539965
Chatfield K, Simonyan K, Vedaldi A, et al., 2014. Return of the devil in the details: delving deep into convolutional nets. p.1–11. https://doi.org/arxiv.org/abs/1405.3531
Chen DY, Tian XP, Shen YT, et al., 2003. On visual similarity based 3D model retrieval. Comput Graph Forum, 22(3): 223–232. https://doi.org/10.1111/1467-8659.00669
Chien JT, Wu CC, 2002. Discriminant waveletfaces and nearest feature classifiers for face recognition. IEEE Trans Patt Anal Mach Intell, 24(12):1644–1649. https://doi.org/10.1109/TPAMI.2002.1114855
Eitz M, Richter R, Boubekeur T, et al., 2012. Sketch-based shape retrieval. ACM Trans Graph, 31(4):31–40. https://doi.org/10.1145/2185520.2185527
Furuya T, Ohbuchi R, 2013. Ranking on cross-domain manifold for sketch-based 3D model retrieval. Int Conf on Cyberworlds, p.274–281. https://doi.org/10.1109/CW.2013.60
Hamm J, Lee DD, 2008. Grassmann discriminant analysis: a unifying view on subspace-based learning. Proc 25th Int Conf on Machine Learning, p.376–383. https://doi.org/10.1145/1390156.1390204
Hamm J, Lee DD, 2009. Extended Grassmann kernels for subspace-based learning. Advances in Neural Information Processing Systems, p.601–608.
Huang Z, Wang R, Shan S, et al., 2014. Learning Euclideanto-Riemannian metric for point-to-set classification. IEEE Conf on Computer Vision and Pattern Recognition, p.1677–1684. https://doi.org/10.1109/CVPR.2014.217
Jayasumana S, Hartley R, Salzmann M, et al., 2013. Kernel methods on the Riemannian manifold of symmetric positive definite matrices. IEEE Conf on Computer Vision and Pattern Recognition, p.73–80. https://doi.org/10.1109/CVPR.2013.17
Kazhdan M, Funkhouser T, Rusinkiewicz S, 2003. Rotation invariant spherical harmonic representation of 3D shape descriptors. Proc Eurographics/ACM SIGGRAPH Symp on Geometry Processing, p.156–164.
Kim T, Kittler J, Cipolla R, 2007. Discriminative learning and recognition of image set classes using canonical correlations. IEEE Trans Patt Anal Mach Intell, 29(6): 1005–1018. https://doi.org/10.1109/TPAMI.2007.1037
Li B, Lu Y, Godil A, et al., 2014. A comparison of methods for sketch-based 3D shape retrieval. Comput Vis Image Underst, 119:57–80. https://doi.org/10.1016/j.cviu.2013.11.008
Lian Z, Godil A, Sun X, et al., 2013. CM-BOF: visual similarity-based 3D shape retrieval using clock matching and bag-of-features. Mach Vis Appl, 24(8):1685–1704. https://doi.org/10.1007/s00138-013-0501-5
Mu P, Zhang S, Ye X, 2017. A metric learning method for image-based 3D shape retrieval. Proc Int Conf on Data Mining, Communications and Information Technology, Article 17. https://doi.org/10.1145/3089871.3089876
Ohbuchi R, Osada K, Furuya T, et al., 2008. Salient local visual features for shape-based 3D model retrieval. IEEE Int Conf on Shape Modeling and Applications, p.93–102. https://doi.org/10.1109/SMI.2008.4547955
Papadakis P, Pratikakis I, Theoharis T, et al., 2010. Panorama: a 3D shape descriptor based on panoramic views for unsupervised 3D object retrieval. Int J Comput Vis, 89(2-3):177–192. https://doi.org/10.1007/s11263-009-0281-6
Saavedra JM, Bustos B, Schreck T, et al., 2012. Sketch-based 3D model retrieval using keyshapes for global and local representation. Proc 5th Eurographics Conf on 3D Object Retrieval, p.47–50. https://doi.org/10.2312/3DOR/3DOR12/047-050
Shilane P, Min P, Kazhdan M, et al., 2004. The Princeton Shape Benchmark. Proc Shape Modeling Applications, p.167–178. https://doi.org/10.1109/SMI.2004.1314504
Sousa P, Fonseca MJ, 2010. Sketch-based retrieval of drawings using spatial proximity. J Vis Lang Comput, 21(2):69–80. https://doi.org/10.1016/j.jvlc.2009.12.001
Su H, Maji S, Kalogerakis E, et al., 2015. Multi-view convolutional neural networks for 3D shape recognition. IEEE Int Conf on Computer Vision, p.945–953. https://doi.org/10.1109/ICCV.2015.114
Tabia H, Laga H, Picard D, et al., 2014. Covariance descriptors for 3D shape matching and retrieval. IEEE Conf on Computer Vision and Pattern Recognition, p.4185–4192. https://doi.org/10.1109/CVPR.2014.533
Vemulapalli R, Pillai JK, Chellappa R, 2013. Kernel learning for extrinsic classification of manifold features. IEEE Conf on Computer Vision and Pattern Recognition, p.1782–1789. https://doi.org/10.1109/CVPR.2013.233
Vincent P, Bengio Y, 2001. K-local hyperplane and convex distance nearest neighbor algorithms. Proc 14th Int Conf on Neural Information Processing Systems: Natural and Synthetic, p.985–992.
Wang F, Kang L, Li Y, 2015. Sketch-based 3D shape retrieval using convolutional neural networks. IEEE Conf on Computer Vision and Pattern Recognition, p.1875–1883. https://doi.org/10.1109/CVPR.2015.7298797
Wang R, Guo H, Davis LS, et al., 2012. Covariance discriminative learning: a natural and efficient approach to image set classification. IEEE Conf on Computer Vision and Pattern Recognition, p.2496–2503. https://doi.org/10.1109/CVPR.2012.6247965
Wen Y, Zhang K, Li Z, et al., 2016. A discriminative feature learning approach for deep face recognition. European Conf on Computer Vision, p.499–515. https://doi.org/10.1007/978-3-319-46478-7_31
Wu Z, Song S, Khosla A, et al., 2015. 3D shapenets: a deep representation for volumetric shapes. IEEE Conf on Computer Vision and Pattern Recognition, p.1912–1920. https://doi.org/10.1109/CVPR.2015.7298801
Yamaguchi O, Fukui K, Maeda K, 1998. Face recognition using temporal image sequence. Proc 3rd IEEE Int Conf on Automatic Face and Gesture Recognition, p.318–323. https://doi.org/10.1109/AFGR.1998.670968
Zhu P, Zhang L, Zuo W, et al., 2013. From point to set: extend the learning of distance metrics. IEEE Int Conf on Computer Vision, p.2664–2671. https://doi.org/10.1109/ICCV.2013.331
Author information
Authors and Affiliations
Corresponding author
Additional information
Project supported by the National Key R&D Program of China (No. 2017YFB1002600), the National Natural Science Foundation of China (No. 61272304), the Natural Science Foundation of Zhejiang Province, China (Nos. LQ16F020007 and LQ17F030002), and the Natural Science Foundation of Ningbo, China (No. 2017A610108)
Rights and permissions
About this article
Cite this article
Mu, Pp., Zhang, Sy., Zhang, Y. et al. Image-based 3D model retrieval using manifold learning. Frontiers Inf Technol Electronic Eng 19, 1397–1408 (2018). https://doi.org/10.1631/FITEE.1601764
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/FITEE.1601764