Abstract
Smile detection has attracted considerable amount of research interests in the domain of computer vision. It possesses several potential applications in gaming, human-to-computer and human-to-human interaction. This paper investigates the challenging problem of smile detection from face images acquired under unconstrained conditions. First, a locally weighted multiblock shape-texture descriptor is proposed to extract detailed local and global information from faces with diverse variations such as orientation, illumination, pose, and occlusion. The proposed technique combines the robustness of pyramid histogram of oriented gradient and local binary pattern for image feature representation using an adaptive local weight assignment. By locally weighting the descriptors from very dense patches of the image, we induce discriminating local spatial context to the distribution of the descriptions from the face image. Second, in order to minimize redundancy and extract the most relevant facial information from the feature vectors, a correlation based filter feature selection approach is adopted. Finally, kernel based classifiers such as support vector machine and kernel extreme learning machine are utilized for performing classification. Based on our findings, the proposed framework provides very competitive detection rate to related approaches that have exploited image alignment as an important stage for improving performance of smile detection.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041
An L, Yang S, Bhanu B (2015) Efficient smile detection by extreme learning machine. Neurocomputing 149:354–363
Arigbabu OA, Ahmad SMS, Adnan WAW, Yussof S, Iranmanesh V, Malallah FL (2014) Gender recognition on real world faces based on shape representation and neural network. In: Proceedings of 2014 IEEE international conference on computer and information sciences (ICCOINS). pp 1–5
Bosch A, Zisserman A, Munoz X (2007) Representing shape with a spatial pyramid kernel. In: Proceedings of the 6th ACM international conference on image and video retrieval. pp 401–408
Cohen I, Sebe N, Garg A, Chen LS, Huang TS (2003) Facial expression recognition from video sequences: temporal and static modeling. Comput Vis Image Underst 91(1–2):160–187
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 297:273–297
Deniz O, Castrillon M, Lorenzo J, Anton L, Bueno G (2008) Smile detection for user interfaces. In: Proceedings of international symposium on advances in visual computing. pp 602–611
Ding S, Zhao H, Zhang Y, Xu X, Nie R (2013) Extreme learning machine: algorithm, theory and applications. Artif Intell Rev 44(1):103–115
Ekman P, Friesen WV (1978) Action coding system: a technique for the measurement of facial movement. Consulting Psychologists Press, San Francisco 14
Ekman P, Davidson RJ, Friesen WV (1990) The Duchenne smile: emotional expression and brain physiology. II. J Pers Soc Psychol 58(2):342–353
Freire D, Castrillón SM, Déniz-Suárez O (2009) A novel approach for smile detection combining Ulbp and Pca. In: Proceedings of the EUROCAST
Girard JM, Cohn JF, De la Torre F (2014) Estimating smile intensity: a better way. Pattern Recogn Lett 000:1–9
Huang GB (2014) An insight into extreme learning machines: random neurons, random features and kernels. Cogn Comput 6(3):376–390
Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501
Huang GB, Mattar M, Berg T, Learned-Miller E (2007) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. University of Massachusetts, Amherst, Technical Report
Liu C, Yuen J, Torralba A (2011) SIFT flow: dense correspondence across scenes and its applications. IEEE Trans Pattern Anal Mach Intell 33(5):978–994
Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987. doi:10.1109/TPAMI.2002.1017623
Pantic M, Rothkrantz LJM (2000) Automatic analysis of facial expressions: the state of the art. IEEE Trans Pattern Anal Mach Intell 22(12):1424–1445
Peng H, Long F, Ding C (2005) Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell 27(8):1226–1238
Sanderson C, Lovell BC (2009) Multi-region probabilistic histograms for robust and scalable identity inference. In: Proceedings of 2009 international conference on biometrics. Lecture notes in computer science vol 5558. pp 199–208
Shan C (2012) Smile detection by boosting pixel differences. IEEE Trans Image Process 21(1):431–436
Tsanas A, Little MA, Mcsharry PE (2010) A simple filter benchmark for feature selection. J Mach Learn Res 1–24
Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154
Whitehill J, Littlewort G, Fasel I, Bartlett M, Movellan J (2009) Towards practical smile detection. IEEE Trans Pattern Anal Mach Intell 31(11):2106–2111
Yang H, Lin W-Y, Lu J (2014) DAISY filter flow: a generalized discrete approach to dense correspondences. In: 2014 IEEE conference on computer vision and pattern recognition. pp 3406–3413
Acknowledgments
We would like to acknowledge Malaysian Ministry of Higher Education for the provision of Exploratory Research Grant Schemes, through which this research was made possible.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
This work complies with the ethical standards.
Conflict of interest
The authors declare that they have no conflict of interest.
Rights and permissions
About this article
Cite this article
Arigbabu, O.A., Mahmood, S., Ahmad, S.M.S. et al. Smile detection using hybrid face representation. J Ambient Intell Human Comput 7, 415–426 (2016). https://doi.org/10.1007/s12652-015-0333-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-015-0333-4