Abstract
The recent years have witnessed a dramatic shift in the way of biometric identification, authentication, and security processes. Among the essential challenges that face these processes are the online verification and authentication. These challenges lie in the complexity of such processes, the necessity of the personal real-time identifiable information, and the methodology to capture temporal information. In this paper, we present an integrated biometric recognition method to jointly recognize face, iris, palm print, fingerprint and ear biometrics. The proposed method is based on the integration of the extracted deep-learned features together with the hand-crafted ones by using a fusion network. Also, we propose a novel convolutional neural network (CNN)-based model for deep feature extraction. In addition, several techniques are exploited to extract the hand-crafted features such as histogram of oriented gradients (HOG), oriented rotated brief (ORB), local binary patterns (LBPs), scale-invariant feature transform (SIFT), and speeded-up robust features (SURF). Furthermore, for dimensional consistency between the combined features, the dimensions of the hand-crafted features are reduced using independent component analysis (ICA) or principal component analysis (PCA). The core of this paper is the template protection via a cancelable biometric scheme without significantly affecting the recognition performance. Specifically, we have used the bio-convolving approach to enhance the user’s privacy and ensure the robustness against spoof attacks. Additionally, various CNN hyper-parameters with their impact on the proposed model performance are studied. Our experiments on various datasets revealed that the proposed method achieves 96.69%, 95.59%, 97.34%, 96.11% and 99.22% recognition accuracies for face, iris, fingerprint, palm print and ear recognition, respectively.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Ali H, Jahangir U, Yousuf B, Noor A (2017) Human action recognition using SIFT and HOG method. In: Proceedings of the international conference on information and communication technologies (ICICT), pp 6–10
AMI ear database (2018), http://ctim.ulpgc.es/research_works/ami_ear_database/ Accessed Nov 2018
Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. In: IEEE transactions on pattern analysis and machine intelligence, pp 1798–1828
Beveridge J, Phillips J, Bolme D, Draper B, Givens G, Lui Y, Teli M, Zhang H, Scruggs W, Bowyer K (2013) The challenge of face recognition from digital point-and-shoot cameras. In: Proceedings of IEEE international conference biometrics, theory, application system, pp 1–8
Bolle RM, Connel JH, Ratha NK (2002) Biometrics perils and patches. In: Pattern Recognation, pp 2727–2738
CASIA Fingerprint Image Database Version 5.0, http://biometrics.idealtest.org/dbDetailForUser.do?id=7. Accessed Nov 2018
CASIA-IrisV3 Database (2018), http://www.cbsr.ia.ac.cn/english/IrisDatabase.asp. Accessed May 2018
COEP Palm Print Database (2018) http://www.coep.org.in/resources/coeppalmprintdatabase.m Accessed Dec 2018
Cheng C, Wang X, Li X (2017) UAV image matching based on surf feature and harris corner algorithm. In: Proceedings of the international conference on smart and sustainable city (ICSSC), pp 1–6
Chin C, Jin A, Ling D (2006) High security iris verification system based on random secret integration. In: Elsevier computer vision and image understanding, pp 169–177
Connie T, Jin T, Ong M, Ling D (2005) Ling D (2005) An automated palmprint recognition system. Image Vis Comput 23:501–515
Daugman J (2016) Information theory and the iris code. In: IEEE transactions on information forensics and security, pp 400–409
Deng W, Yao R, Zhao H, Yang X, Li G (2017a) A novel intelligent diagnosis method using optimal LS-SVM with improved PSO algorithm. Soft Comput 23:2445
Deng W, Zhao H, Zou L, Li G, Yang X, Wu D (2017b) A novel collaborative optimization algorithm in solving complex optimization problems. Soft Comput 21:4387–4398. https://doi.org/10.1007/s00500-016-2071-8
Ding C, Tao D (2017) Trunk-branch ensemble convolutional neural networks for video-based face recognition. IEEE Trans Pattern Anal Mach Intell 40(4):1002–1014
Ekinci M, Aykut M (2007) Gabor-based kernel PCA for palmprint recognition. Electr Lett 43:1077–1079
Emersic Z, Stepec D, Struc V (2017a) Training convolutional neural networks with limited training data for ear recognition in the wild. In: Automatic face gesture recognition, pp 987–994
Emersic Z, Struc V, Peer P (2017b) Ear recognition: more than a survey. Neurocomputing 255:26–39
Gangwar A, Joshi A (2016) Deepirisnet: deep iris representation with applications in iris recognition and cross-sensor iris recognition. In: IEEE international conference on image processing (ICIP), pp 2301–2305
Harjoko A, Hartati S, Dwiyasa H (2009) A method for iris recognition based on 1d coiflet wavelet. In: World academy of science engineering and technology, pp 126–129
Hasnat A, Bohn´e J, Milgram J, Gentric S, Chen L (2017) DeepVisage: making face recognition simple yet with powerful generalization Skills. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1–12
Iandola F, Han S, Moskewicz M (2016) Squeezenet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size. In: arXiv preprint arXiv:1602.07360
Ignat A, Luca M, Ciobanu A (2013) Iris features using dual tree complex wavelet transform in texture evaluation for biometrical identification. In: IEEE E-health and bioengineering conference, pp 1–4
Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the international conference on machine learning (ICML). arXiv:1502.03167v3
Jain AK (2004) An introduction to biometric recognition. In: IEEE transactions on circuits and systems for video technology, pp 4–20
Jain AK (2006) A tool for information Security. In: IEEE transactions on information forensics and security, pp 125–143
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444
Liu Y, Li H, Wang X (2017) Rethinking feature discrimination and polymerization for large-scale recognition. arXiv preprint arXiv:1710.00870
Liu N, Zhang M, Li H, Sun Z, Tan T (2016) Deepiris: learning pairwise filter bank for heterogeneous iris verification. Pattern Recognit Lett 82:154–161
Mariana-Iuliana G, Radu T, Marius P (2019) Local learning with deep and handcrafted features for facial expression recognition. arXiv:1804.10892v6
NIST (2012) IREX III—performance of iris identification algorithms. In: National Institute of Science and Technology, USA, Technical Report NIST Interagency Report 7836
Omara I, Li F, Zhang H, Zuo W (2016) A novel geometric feature extraction method for ear recognition. Expert Syst Appl 65:127–135
Patel V, Ratha N, Chellappa R (2015) Cancelable biometrics: a review. In: IEEE signal processing magazine, pp 54–65
Pedro L, Galdamez P, Raveane W (2016) A brief review of the ear recognition process using deep neural networks. J. Appl. Logic 24:62–70
Pei S, Chen M, Yu Y, Tang S, Zhong C (2017) Compact LBP and WLBP descriptor with magnitude and direction difference for face recognition. In: Proceedings of the IEEE international conference on image processing (ICIP), pp. 1067–1071
Phillips P, Scruggs W, O’Toole A, Flynn P, Bowyer K, Schott C, Sharpe M (2010) Frvt 2006 and ice 2006 large scale experimental results. In: IEEE transactions on pattern analysis and machine intelligence, pp 831–846
Pichao W, Zhaoyang L, Yonghong H, Wanqing L (2016) Combining convnets with hand-crafted features for action recognition based on an HMM-SVM Classifier. In: Proceedings of computer vision and pattern recognition (CVPR). arXiv:1602.00749
Pillai JK, Patel VM, Chellappa R, Ratha N (2010) Sectored random projections for cancelable iris biometrics. In: IEEE international conference on acoustics speech and signal processing, pp 1838–1841
Polash P, Gavrilova M, Klimenko S (2014) Situation awareness of cancelable biometric system. J Vis Comput 30:1059–1067
Ratha N, Connell J, Bolle R, Chikkerur S (2006) Cancelable biometrics: a case study in fingerprints. In: International conference of pattern recognition, pp 370–373
Ratha N, Chikkerur S, Connell J, Bolle R (2007) Generating cancelable fingerprint templates. IEEE Trans Pattern Anal Mach Intell 29:561–572
Rathgeb C (2010) A secure iris recognition based on local intensity variations. In: Proceedings of the 7th international conference on image analysis and recognition, pp 266–275
Rathgeb Ch, Barrero M, Busch Ch, Galbally J, Fierrez J (2015) Towards cancelable multi-biometrics based on bloom filters: a case study on feature level fusion of face and iris. In: 3rd International workshop on biometrics and forensics (IWBF)
Rathgeb C, Breitinger F, Busch C, Baier H (2014) On the application of bloom filters to iris biometrics. IET J Biom 3(4):207–218
Rathgeb C, Breitinger F, Baier H, Busch C (2015) Towards bloom filter-based indexing of iris biometric data. In: 15th IEEE international conference on biometrics, pp 422–429
Sarbadhikari S, Basak J, Pal S, Kundu M (1998) Noisy fingerprints classification with directional based features using MLP. Neural Comput Appl 7:180–191
Sh X, Jinhui T, Hanjiang L, Zhiheng N, Shuicheng Y (2016) Kinship Guided Age Progression. Pattern Recognit 59:156–157
Sun Y, Wang X, Tang X (2015) Deeply learned face representations are sparse, selective, and robust. In: IEEE conference on computer vision and pattern recognition
Swathi K, Kalyana V, Quek C (2018) Evolutionary based ICA with reference for EEGμRhythm Extraction. In: IEEE Access, pp 19702–19713
Sylvia W, Kamalaharidharini T (2017) Robust face recognition and classification system based on SIFT and DCP techniques in image processing. In: Proceedings of the international conference on intelligent computing and control (I2C2), pp 1–8
Taigman Y, Yang M, Ranzato M, Wolf L (2014) Deepface: closing the gap to human-level performance in face verification. In: Proceedings of IEEE CVPR, pp 1701–1708
Taigman Y, Yang M, Ranzato M, Wolf L (2015) Web-scale training for face identification. In: IEEE conference on computer vision and pattern recognition
Sun Y, Liang D, Wang X, Tang X (2015) DeepID3: Face recognition with very deep neural networks. arXiv:1502.00873
Tarek M, Ouda O, Hamza T (2017) Pre-image resistant cancelable biometrics scheme using bidirectional memory model. Int J Netw Secur 19(4):498–506
Teoh ABJ, Ngo DCL, Goh A (2004) Biohashing: two factor authentication featuring fingerprint data and tokenised random number. Pattern Recognit 37(11):2245–2255
Theodoridis S, Koutroumbas K (2008) Pattern recognition, 4th edn. Academic Press, Cambridge
Tomas M, Sule Y, Jon Y (2016) Combining deep learning and hand-crafted features for skin lesion classification. In: Sixth international conference on image processing theory, tools and applications (IPTA)
Vinay A, Kumar C, Shenoy R (2015) ORB-PCA based feature extraction technique for face recognition. In: Proceedings of the second international symposium on computer vision and the internet, pp 614–621
Xiaolin X, Yicong Z (2019) Two-dimensional quaternion PCA and sparse PCA. IEEE Trans Neural Netw Learn Syst 30(7):1–15. https://doi.org/10.1109/TNNLS.2018.2872541
Xu X, Lu L, Zhang X, Lu H, Deng W (2016) Multispectral palmprint recognition using multiclass projectionextreme learning machine and digital shearlet transform. Neural Comput Appl 27:143–153
Yakopcic C, Alom M, Taha T (2017) Extremely parallel memristor crossbar architecture for convolutionalneural network implementation. In: Proceedings of the international joint conference on neural networks (IJCNN), pp 1696–1703
Yang G, Shi D, Quek C (2005) Fingerprint minutia recognition with fuzzy neural network. In: Wang J, Liao X-F, Yi Z (eds) ISNN 2005, Part II. LNCS, vol 3497. Springer, Heidelberg, pp 165–170 (2005)
Yong-Xia L, Jin Q, Rui X (2010) A new detection method of singular points of fingerprints based on neural network. In: The 3rd IEEE international conference on computer science and information technology, ICCSIT
Yuan Z, Liu Y, Yue J, Li J, Yang H (2017) CORAL: coarse-grained reconfigurable architecture for convolutional neural networks. In: Proceedings of the IEEE/ACM international symposium on low power electronics and design (ISLPED), pp 1–6
Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Zhao H, Sun M, Deng W, Yang X (2017) A new feature extraction method based on EEMD and multi-scale fuzzy entropy for motor bearing. Entropy 19:14
Zhao H, Yao R, Xu L, Yuan Y, Li G, Deng W (2018) Study on a novel fault damage degree identification method using high-order differential mathematical morphology gradient spectrum entropy. Entropy 20:682
Zheng Y, Pal D, Savvides M (2018) Ring loss: convex feature normalization for face recognition. In: CVPR
Zhu E, Yin J, Hu C, Zhang G (2005) Quality estimation of fingerprint image based on neural network. In: Wang L, Chen KS, Ong Y (eds.) ICNC 2005, Part II. LNCS, vol. 3611, pp. 65–70. Springer, Heidelberg
Zuo J, Ratha NK, Connel JH (2008) Cancelable iris biometric. In: Proceedings of the 19th international conference on pattern recognition, pp 1–4
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Communicated by V. Loia.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Abdellatef, E., Omran, E.M., Soliman, R.F. et al. Fusion of deep-learned and hand-crafted features for cancelable recognition systems. Soft Comput 24, 15189–15208 (2020). https://doi.org/10.1007/s00500-020-04856-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-020-04856-1