Modelling and unsupervised learning of symmetric deformable object categories
Pages 8189 - 8200
Abstract
We propose a new approach to model and learn, without manual supervision, the symmetries of natural objects, such as faces or flowers, given only images as input. It is well known that objects that have a symmetric structure do not usually result in symmetric images due to articulation and perspective effects. This is often tackled by seeking the intrinsic symmetries of the underlying 3D shape, which is very difficult to do when the latter cannot be recovered reliably from data. We show that, if only raw images are given, it is possible to look instead for symmetries in the space of object deformations. We can then learn symmetries from an unstructured collection of images of the object as an extension of the recently-introduced object frame representation, modified so that object symmetries reduce to the obvious symmetry groups in the normalized space. We also show that our formulation provides an explanation of the ambiguities that arise in recovering the pose of symmetric objects from their shape or images and we provide a way of discounting such ambiguities in learning.
References
[1]
Helmut Alt, Kurt Mehlhorn, Hubert Wagener, and Emo Welzl. Congruence, similarity, and symmetries of geometric objects. Discrete & Computational Geometry, 3(3):237-256, 1988.
[2]
Shai Bagon, Oren Boiman, and Michal Irani. What is a good image segment? a unified approach to segment extraction. In Proc. ECCV, pages 30-44. Springer, 2008.
[3]
Hakan Bilen, Marco Pedersoli, and Tinne Tuytelaars. Weakly supervised object detection with posterior regularization. In Proceedings BMVC 2014, pages 1-12, 2014.
[4]
Oren Boiman and Michal Irani. Similarity by composition. In Proc. NIPS, pages 177-184, 2007.
[5]
T F Cootes, C J Taylor, D H Cooper, and J Graham. Active shape models: their training and application. CVIU, 1995.
[6]
Erwin Coumans. Bullet physics engine. Open Source Software: http://bulletphysics.org, 2010.
[7]
Navneet Dalal and Bill Triggs. Histograms of Oriented Gradients for Human Detection. In Proc. CVPR, 2005.
[8]
Aleksandrs Ecins, Cornelia Fermüller, and Yiannis Aloimonos. Cluttered scene segmentation using the symmetry constraint. In Robotics and Automation (ICRA), 2016 IEEE International Conference on, pages 2271-2278. IEEE, 2016.
[9]
Pedro F. Felzenszwalb, Ross B. Girshick, David McAllester, and Deva Ramanan. Object Detection with Discriminatively Trained Part Based Models. PAMI, 2010.
[10]
Rob Fergus, Pietro Perona, and Andrew Zisserman. Object class recognition by unsupervised scale-invariant learning. In Proc. CVPR, 2003.
[11]
Ran Gal and Daniel Cohen-Or. Salient geometric features for partial shape matching and similarity. ACM Transactions on Graphics (TOG), 25(1):130-150, 2006.
[12]
Mike Goslin and Mark R Mine. The Panda3D graphics engine. Computer, 37(10):112-114, 2004.
[13]
Bumsub Ham, Minsu Cho, Cordelia Schmid, and Jean Ponce. Proposal flow. In Proc. CVPR, pages 3475-3484, 2016.
[14]
Kai Han, Rafael S Rezende, Bumsub Ham, Kwan-Yee K Wong, Minsu Cho, Cordelia Schmid, and Jean Ponce. Scnet: Learning semantic correspondence. In Proc. ICCV, 2017.
[15]
Max Jaderberg, Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu. Spatial Transformer Networks. In Proc. NIPS, 2015.
[16]
A. Kanazawa, D. W. Jacobs, and M. Chandraker. WarpNet: Weakly supervised matching for single-view reconstruction. In Proc. CVPR, 2016.
[17]
Ira Kemelmacher-Shlizerman and Steven M. Seitz. Collection flow. In Proc. CVPR, 2012.
[18]
Kurt Koffka. Principles of Gestalt psychology, volume 44. Routledge, 2013.
[19]
Erik G Learned-Miller. Data driven image models through continuous joint alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006.
[20]
Bastian Leibe, Ales Leonardis, and Bernt Schiele. Combined object categorization and segmentation with an implicit shape model. In Workshop on statistical learning in computer vision, ECCV, 2004.
[21]
Yanxi Liu, Hagit Hel-Or, Craig S Kaplan, Luc Van Gool, et al. Computational symmetry in computer vision and computer graphics. Foundations and Trends® in Computer Graphics and Vision, 5(1-2):1-195, 2010.
[22]
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proc. ICCV, 2015.
[23]
Jonathan L Long, Ning Zhang, and Trevor Darrell. Do convnets learn correspondence? In Advances in Neural Information Processing Systems, pages 1601-1609, 2014.
[24]
David G Lowe. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60(2):91-110, 2004.
[25]
Giovanni Marola. On the detection of the axes of symmetry of symmetric and almost symmetric planar images. PAMI, 11(1):104-108, 1989.
[26]
Niloy J Mitra, Leonidas J Guibas, and Mark Pauly. Symmetrization. In ACM Transactions on Graphics (TOG), volume 26, page 63. ACM, 2007.
[27]
Hossein Mobahi, Ce Liu, and William T. Freeman. A Compositional Model for Low-Dimensional Image Set Representation. Proc. CVPR, 2014.
[28]
Gregory L Naber. The geometry of Minkowski spacetime: An introduction to the mathematics of the special theory of relativity, volume 92. Springer Science & Business Media, 2012.
[29]
D. Novotny, D. Larlus, and A. Vedaldi. Learning 3d object categories by looking around them. In Proc. ICCV, 2017.
[30]
O. M. Parkhi, A. Vedaldi, A. Zisserman, and C. V. Jawahar. Cats and dogs. In Proc. CVPR, 2012.
[31]
Yigang Peng, Arvind Ganesh, John Wright, Wenli Xu, and Yi Ma. Rasl: Robust alignment by sparse and low-rank decomposition for linearly correlated images. PAMI, 34(11):2233-2246, 2012.
[32]
Dan Raviv, Alexander M Bronstein, Michael M Bronstein, and Ron Kimmel. Full and partial symmetries of non-rigid shapes. IJCV, 89(1):18-39, 2010.
[33]
I. Rocco, R. Arandjelovic, and J. Sivic. Convolutional neural network architecture for geometric matching. In Proc. CVPR, 2017.
[34]
Ilan Shimshoni, Yael Moses, and Michael Lindenbaum. Shape reconstruction of 3d bilaterally symmetric surfaces. IJCV, 39(2):97-110, 2000.
[35]
Zhangzhang Si and Song-Chun Zhu. Learning hybrid image templates (hit) by information projection. PAMI.
[36]
Changming Sun and Jamie Sherrah. 3d symmetry detection using the extended gaussian image. PAMI, 19(2):164-168, 1997.
[37]
J. Thewlis, H. Bilen, and A. Vedaldi. Unsupervised learning of object frames by dense equivariant image labelling. In Proc. NIPS, 2017.
[38]
J. Thewlis, H. Bilen, and A. Vedaldi. Unsupervised learning of object landmarks by factorized spatial embeddings. In Proc. ICCV, 2017.
[39]
Sebastian Thrun and Ben Wegbreit. Shape from symmetry. In Proc. ICCV, pages 1824-1831, 2005.
[40]
Thomas Vetter and Tomaso Poggio. Linear object classes and image synthesis from a single example image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):733-742, 1997.
[41]
James D Watson, Francis HC Crick, et al. Molecular structure of nucleic acids. Nature, 171(4356):737-738, 1953.
[42]
Jeremy D Wilbur, Peter K Hwang, Joel A Ybe, Michael Lane, Benjamin D Sellers, Matthew P Jacobson, Robert J Fletterick, and Frances M Brodsky. Conformation switching of clathrin light chain regulates clathrin lattice assembly. Developmental cell, 18(5):854-861, 2010.
[43]
Heng Yang and Ioannis Patras. Mirror, mirror on the wall, tell me, is the error small? In Proc. CVPR, pages 4685-4693, 2015.
[44]
Weiwei Zhang, Jian Sun, and Xiaoou Tang. Cat head detection - How to effectively exploit shape and texture features. In Proc. ECCV, 2008.
[45]
Zhanpeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. Learning Deep Representation for Face Alignment with Auxiliary Attributes. PAMI, 2016.
[46]
Zheng Zhang, Wei Shen, Cong Yao, and Xiang Bai. Symmetry-based text line detection in natural scenes. In Proc. CVPR, pages 2558-2567, 2015.
- Modelling and unsupervised learning of symmetric deformable object categories
Recommendations
Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild (Invited Paper)
We propose a method to learn 3D deformable object categories from raw single-view images, without external supervision. The method is based on an autoencoder that factors each input image into depth, albedo, viewpoint and illumination. In order to ...
Intelligent Learning for Deformable Object Manipulation
The majority of manipulation systems are designed with the assumption that the objects being handled are rigid and do not deform when grasped. This paper addresses the problem of robotic grasping and manipulation of 3-D deformable objects, such as ...
Unsupervised object-level representation learning from scene images
NIPS '21: Proceedings of the 35th International Conference on Neural Information Processing SystemsContrastive self-supervised learning has largely narrowed the gap to supervised pre-training on ImageNet. However, its success highly relies on the object-centric priors of ImageNet, i.e., different augmented views of the same image correspond to the ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
December 2018
11021 pages
Publisher
Curran Associates Inc.
Red Hook, NY, United States
Publication History
Published: 03 December 2018
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 62Total Downloads
- Downloads (Last 12 months)43
- Downloads (Last 6 weeks)5
Reflects downloads up to 19 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in