[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content

Advertisement

Springer Nature Link
Log in

Transfer learning and feature fusion for kinship verification

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Facial image analysis has been an important subject of study in the communities of pattern recognition and computer vision. Facial images contain much information about the person they belong to like identity, age, gender, ethnicity, and expression. This paper introduces a new framework that exploits facial deep features for kinship verification. The framework integrates efficient feature selection and kinship-oriented discriminant data projection. The resulting framework incorporates three levels of fusion: (1) an early fusion of descriptors where the filter selection selects the most relevant deep features, (2) a middle-stage fusion which exploits a kinship-based multi-view metric learning method, and (3) a late-stage fusion that merges classifiers responses. In our study, face features are provided by the pre-trained deep convolutional neural networks VGG-F and VGG-Face that were originally proposed for discriminating categories of objects and identities, respectively. Experimental results on two benchmarked datasets for kinship verification in the wild (KinFaceW-I and KinFaceW-II) show that the proposed framework outperforms state-of-the-art techniques without the use of external data or data augmentation that are tailored for the kinship verification problem. These experiments show that the proposed scheme can outperform feature fusion obtained by deep multi-metric learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Figure from [32]

Fig. 2

Figure from [4]

Fig. 3
Fig. 4
Fig. 5

Figure from [31]

Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Alirezazadeh P, Fathi A, Abdali-Mohammadi F (2015) A genetic algorithm-based feature selection for kinship verification. IEEE Signal Process Lett 22(12):2459–2463

    Google Scholar 

  2. Almuashi M, Hashim SZM, Mohamad D, Alkawaz MH, Ali A (2017) Automated kinship verification and identification through human facial images: a survey. Multimed Tools Appl 76:265–307

    Google Scholar 

  3. Amiriparian S, Gerczuk M, Ottl S, Cummins N, Freitag M, Pugachevskiy S, Baird A, Schuller BW (2017) Snore sound classification using image-based deep spectrum features. In INTERSPEECH

  4. Chatfield K, Simonyan K, Vedaldi A, Zisserman A (2014) Return of the devil in the details: delving deep into convolutional nets. In: British machine vision conference

  5. Chen X, An L, Yang S, Wu W (2017) Kinship verification in multilinear coherent spaces. Multimedia Tools Appl 76:4105–412

    Google Scholar 

  6. DeBruine L, Jones B, Little A, Perrett D (2008) Social perception of facial resemblance in humans. Arch Sex Behav 37(1):64–77

    Google Scholar 

  7. Dehghan A, Ortiz EG, Villegas R, Shah M (2014) Who do I look like? Determining parent-offspring resemblance via gated autoencoders. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1757–1764

  8. Dibeklioglu H (2017) Visual transformation aided contrastive learning for video-based kinship verification. In: IEEE conference on computer vision and pattern recognition

  9. Dibeklioglu H, Salah AA, Gevers T (2013) Like father, like son: facial expression dynamics for kinship verification. In: 2013 IEEE international conference on computer vision

  10. Dornaika IA-CF, Belver C (2019) Age estimation in facial images through transfer learning. Mach Vis Appl 30:177–187

    Google Scholar 

  11. Fang Y, Chen YYS, Wang H, Shu C (2016) Sparse similarity metric learning for kinship verification. In: Visual communications and image processing (VCIP), 2016, pp 1–4. IEEE

  12. Guo G, Wang X (2012) Kinship measurement on salient facial features. IEEE Trans Instrum Meas 61(8):2322–2325

    Google Scholar 

  13. Guo Y, Dibeklioglu H, van der Maaten L (2014) Graph-based kinship recognition. In: 2014 22nd International conference on pattern recognition (ICPR), pp 4287–4292. IEEE

  14. Hu J, Lu J, Tan Y, Yuan J, Zhou J (2018) Local large-margin multi-metric learning for face and kinship verification. IEEE Trans Circuits Syst Video Technol 28(8):1875–1891

    Google Scholar 

  15. Hu J, Lu J, Yuan J, Tan Y-P (2014) Large margin multi-metric learning for face and kinship verification in the wild. In: ACCV (3), pp 252–267

    Google Scholar 

  16. Jolliffe I (2002) Principal component analysis. Wiley, London

    MATH  Google Scholar 

  17. KinFaceW (2014) http://www.kinfacew.com/index.html

  18. Kohli N, Vatsa M, Singh R, Noore A, Majumdar A (2017) Hierarchical representation learning for kinship verification. IEEE Trans Image Process 26(1):289–302

    MathSciNet  MATH  Google Scholar 

  19. Kou L, Zhou X, Xu M, Shang Y (2015) Learning a genetic measure for kinship verification using facial images. Math Problems Eng 2015 (ID 472473)

    Google Scholar 

  20. Kumar CR (2017) Harmonic rule for measuring the facial similarities among relatives. Trans Mach Learn Artif Intell 4(6):29

    Google Scholar 

  21. Lan R, Zhou Y (2016) Quaternion-michelson descriptor for color image classification. IEEE Trans Image Process 25(11):5281–5292

    MathSciNet  MATH  Google Scholar 

  22. Lan R, Zhou Y, Tang YY (2017) Quaternionic weber local descriptor of color images. IEEE Trans Circuits Syst Video Technol 27(2):261–274

    Google Scholar 

  23. Li L, Feng X, Wu X, Xia Z, Hadid A (2016) Kinship verification from faces via similarity metric based convolutional neural network. In: International conference image analysis and recognition. Springer, Berlin, pp 539–548

    Google Scholar 

  24. Liang J, Hu Q, Dang C, Zuo W (2019) Weighted graph embedding-based metric learning for kinship verification. IEEE Trans Image Process 28(3):1149–1162

    MathSciNet  Google Scholar 

  25. Liu Q, Puthenputhussery A, Liu C (2015) Inheritable fisher vector feature for kinship verification. In: In: 2015 IEEE 7th international conference on biometrics theory, applications and systems (BTAS), pp 1–6. IEEE

  26. Liu Q, Puthenputhussery A, Liu C (2016) A novel inheritable color space with application to kinship verification. In: 2016 IEEE winter conference on applications of computer vision (WACV), pp 1–9. IEEE

  27. Liu Y, Zhang L, Nie L, Yan Y, Rosenblum DS (2016) Fortune teller: predicting your career path. In: Thirtieth AAAI conference on artificial intelligence

  28. Lopez MB, Boutellaa E, Hadid A (2016) Comments on the kinship face in the wild data sets. IEEE Trans Pattern Anal Mach Learn 38(11):2342–2344

    Google Scholar 

  29. Lopez MB, Hadid A, Boutellaa E, Goncalves J, Kostakos V, Hosio S (2018) Kinship verification from facial images and videos: human versus machine. Mach Vis Appl 29(5):873890

    Google Scholar 

  30. Lu J, Hu J, Tan Y (2017) Discriminative deep metric learning for face and kinship verification. IEEE Trans Image Process 26(9):4269–4282

    MathSciNet  MATH  Google Scholar 

  31. Lu J, Zhou X, Tan Y-P, Zhou YSJ (2014) Neighborhood repulsed metric learning for kinship verification. IEEE Trans Pattern Anal Mach Intell 36(2):331–345

    Google Scholar 

  32. Parkhi OM, Vedaldi A, Zisserman A (2015) Deep face recognition. In: British machine vision conference

  33. Patel B, Maheshwari R, Raman B (2017) Evaluation of periocular features for kinship verification in the wild. Comput Vis Image Underst 160(Supplement C):24–35

    Google Scholar 

  34. Puthenputhussery A, Liu Q, Liu C (2016) Sift flow based genetic fisher vector feature for kinship verification. In: 2016 IEEE international conference on image processing (ICIP), pp 2921–2925. IEEE

  35. Qin X, Tan X, Chen S (2016) Mixed bi-subject kinship verification via multi-view multi-task learning. Neurocomputing 214:350–357

    Google Scholar 

  36. Ren Z, Pandit V, Qian K, Yang Z, Zhang Z, Schuller BW (2017) Deep sequential image features for acoustic scene classification. In: 2017, Workshop on detection and classification of acoustic scenes and events

  37. Robinson J, Shao M, Wu Y, Y, Fu Y (2016) Families in the wild (fiw): large-scale kinship image database and benchmarks. In: ACM conference on multimedia, MM 16

  38. Vieira T, Bottino A, Laurentini A, Simone M (2014) Detecting siblings in image pairs. Vis Comput 30(12):1333–1345

    Google Scholar 

  39. Wang S, Robinson JP, Fu Y (May 2017) Kinship verification on families in the wild with marginalized denoising metric learning. In: IEEE international conference on automatic face gesture recognition (FG 2017), pp 216–221

  40. Xia S, Shao M, Fu Y (2011) Kinship verification through transfer learning. In: IJCAI, pp 2539–2544

  41. Xia S, Shao M, Luo J, Fu Y (2012) Understanding kin relationships in a photo. IEEE Trans Multimed 14(4):1046–1056

    Google Scholar 

  42. Yan H (2017) Kinship verification using neighborhood repulsed correlation metric learning. Image Vis Comput 60:91–97

    Google Scholar 

  43. Yan H, Lu J, Deng W, Zhou X (2014) Discriminative multimetric learning for kinship verification. IEEE Trans Inf Forensics Secur 9(7):1169–1178

    Google Scholar 

  44. Yan H, Lu J, Zhou X (2015) Prototype-based discriminative feature learning for kinship verification. IEEE Trans Cybern 45(11):2535–2545

    Google Scholar 

  45. Zhang K, Huang Y, Song C, Wu H, Wang L (2015) Kinship verification with deep convolutional neural networks

  46. Zhao Y-G, Song Z, Zheng F, Shao L (2018) Learning a multiple kernel similarity metric for kinship verification. Inf Sci 430–431:247–260

    MathSciNet  Google Scholar 

  47. Zheng L, Idrissi K, Garcia C, Duffner S, Baskurt A (2015) Triangular similarity metric learning for face verification. In: IEEE international conference on automatic face and gesture recognition

  48. Zhou X, Hu J, Lu J, Shang Y, Guan Y (2011) Kinship verification from facial images under uncontrolled conditions. In: Proceedings of the 19th ACM international conference on multimedia, pp 953–956. ACM

  49. Zhou X, Lu J, Hu J, Shang Y (2012) Gabor-based gradient orientation pyramid for kinship verification under uncontrolled environments. In: Proceedings of the 20th ACM international conference on multimedia, pp 725–728. ACM

  50. Zhou X, Shang Y, Yan H, Guo G (2016) Ensemble similarity learning for kinship verification from facial images in the wild. Inf Fusion 32:40–48

    Google Scholar 

  51. Zhou X, Yan H, Shang Y (2016) Kinship verification from facial images by scalable similarity fusion. Neurocomputing 197:136–142

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of the Basque Country UPV/EHU, San Sebastián, Spain

    F. Dornaika, I. Arganda-Carreras & O. Serradilla

  2. IKERBASQUE, Basque Foundation for Science, Bilbao, Spain

    F. Dornaika & I. Arganda-Carreras

  3. Donostia International Physics Center (DIPC), San Sebastián, Spain

    I. Arganda-Carreras

Authors
  1. F. Dornaika
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. I. Arganda-Carreras
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. O. Serradilla
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to F. Dornaika.

Ethics declarations

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The studies conducted comply with the ethical standards.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dornaika, F., Arganda-Carreras, I. & Serradilla, O. Transfer learning and feature fusion for kinship verification. Neural Comput & Applic 32, 7139–7151 (2020). https://doi.org/10.1007/s00521-019-04201-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04201-0

Keywords

Search

Navigation