[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content

Advertisement

Springer Nature Link
Log in

A comprehensive evaluation of feature-based AI techniques for deepfake detection

  • Review
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In the contemporary era, where data and information are the key source in every domain, it becomes imperative to identify, detect and distinguish between fake and authentic content available online. Recent technological innovations in the area of artificial intelligence (AI) and computer vision (CV) have been the key players both in generating and detection of these media (both images and videos). The advancement of techniques for generating, fabricating, and manipulating multimedia materials has resulted in a heightened level of realism, thereby giving rise to numerous security concerns. The adoption of these technologies has resulted in the widespread dissemination of fraudulent images and videos, which are being utilised for various criminal purposes and often featuring misleading information and impersonations of public personalities, which have detrimental consequences on the reputations of those individuals involved. The proliferation of counterfeit images poses a significant threat to national security, as these images can be exploited for the purpose of identity forgery. Therefore, it is imperative to design and develop robust algorithms for detecting counterfeit media, capable of effectively distinguishing between authentic and manipulated content. This study aims to present the generative and detection techniques of visual deep fake media using deep learning (CNN, RNN, LSTM, etc.), machine learning (SVM, KNN, Random Forest, and Decision Tree) and statistical learning (3D Morphable Model). This study also conducted an in-depth analysis of the current state of literature concerning the development and application of deepfake technology, as well as the accessibility of open-source tools for generating manipulated media. The present study provides an extensive review of face manipulation methodologies employed in the development of deep fakes, specifically focusing on Identity swap, Image Synthesis, Face Re-enactment, and Attribute Manipulation. A presented review proposed a novel taxonomy based on spatial, temporal and frequency-based features for the detection of visual Deepfake. This study out passes the existing surveys that have been presented by various other researchers in this field in terms of domain, learning methods, features and manipulation techniques used. In this study, the challenges and research gaps along with the analysis of each of these have also been presented with the intent for prioritising the development of deep fake detection tools.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Algorithm 1
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

Not available.

References

  1. Suwajanakorn S, Seitz SM, Kemelmacher-Shlizerman I (2017) Synthesizing Obama. ACM Trans Gr (TOG) 36:4. https://doi.org/10.1145/3072959.3073640

    Article  Google Scholar 

  2. de Ruiter A (2021) The distinct wrong of deepfakes. Philos Technol 34(4):1311–1332. https://doi.org/10.1007/s13347-021-00459-2

    Article  Google Scholar 

  3. Ajder H, Patrini G, Cavalli F, and Cullen L, “Graphic design: Eleanor Winter Cover image: Joel Filipe Cite: The State of Deepfakes: Landscape, Threats, and Impact About Deeptrace,” 2019.

  4. Guera D and Delp EJ (2019) “Deepfake Video Detection Using Recurrent Neural Networks,” Proceedings of AVSS 2018 - 2018 15th IEEE International Conference on Advanced Video and Signal-Based Surveillance, Feb. https://doi.org/10.1109/AVSS.2018.8639163.

  5. Kim T, Kim J, Kim J and Woo SS (2022) “A Face Pre-Processing Approach to Evade Deepfake Detector,” WDC 2022 - Proceedings of the 1st Workshop on Security Implications of Deepfakes and Cheapfakes, pp. 35–38, https://doi.org/10.1145/3494109.3527190.

  6. “Random Face Generator- This Person Does Not Exist.” https://this-person-does-not-exist.com/en (accessed Sep. 07, 2022).

  7. Masood M, Nawaz M, Malik KM, Javed A, Irtaza A, Malik H (2022) Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward. Appl Intell. https://doi.org/10.1007/s10489-022-03766-z

    Article  Google Scholar 

  8. Yang J, Xiao S, Li A, Lan G, Wang H (2021) Detecting fake images by identifying potential texture difference. Futur Gener Comput Syst 125:127–135. https://doi.org/10.1016/j.future.2021.06.043

    Article  Google Scholar 

  9. Creswell A, White T, Dumoulin V, Arulkumaran K, Sengupta B, Bharath AA (2018) Generative adversarial networks: an overview. IEEE Signal Process Mag 35(1):53–65. https://doi.org/10.1109/MSP.2017.2765202

    Article  ADS  Google Scholar 

  10. Nguyen TT et al (2022) Deep learning for deepfakes creation and detection: A survey. Comput Vis Image Underst 223:103525. https://doi.org/10.1016/J.CVIU.2022.103525

    Article  Google Scholar 

  11. Tolosana R, Vera-Rodriguez R, Fierrez J, Morales A, Ortega-Garcia J (2020) Deepfakes and beyond: a Survey of face manipulation and fake detection. Information Fusion 64:131–148. https://doi.org/10.1016/j.inffus.2020.06.014

    Article  Google Scholar 

  12. Kolagati S, Priyadharshini T, Rajam VMA (2022) Exposing deepfakes using a deep multilayer perceptron – convolutional neural network model. Int J Inf Manag Data Insights 2:1. https://doi.org/10.1016/j.jjimei.2021.100054

    Article  Google Scholar 

  13. Afchar D, Nozick V, Yamagishi J and Echizen I (2019) “MesoNet: A compact facial video forgery detection network,” 10th IEEE International Workshop on Information Forensics and Security, WIFS 2018, Jan. https://doi.org/10.1109/WIFS.2018.8630761.

  14. Mehta V, Gupta P, Subramanian R, and Dhall A (2021) “FakeBuster: A deepfakes detection tool for video conferencing scenarios,” International Conference on Intelligent User Interfaces, Proceedings IUI, pp. 61–63, Apr. https://doi.org/10.1145/3397482.3450726.

  15. Yu P, Xia Z, Fei J, Lu Y (2021) A survey on deepfake video detection. IET Biom 10(6):607–624. https://doi.org/10.1049/BME2.12031

    Article  Google Scholar 

  16. Chadha A, Kumar V, Kashyap S and Gupta M (2021) “Deepfake: An Overview,” in Lecture Notes in Networks and Systems, Springer Science and Business Media Deutschland GmbH, pp 557–566.

  17. Malik A, Kuribayashi M, Abdullahi SM, Khan AN (2022) DeepFake detection for human face images and videos: a survey. IEEE Access 10:18757–18775. https://doi.org/10.1109/ACCESS.2022.3151186

    Article  Google Scholar 

  18. Seow JW, Lim MK, Phan RCW, Liu JK (2022) A comprehensive overview of Deepfake: Generation, detection, datasets, and opportunities. Neurocomputing 513:351–371. https://doi.org/10.1016/J.NEUCOM.2022.09.135

    Article  Google Scholar 

  19. Nguyen TT et al (2022) Deep learning for deepfakes creation and detection: a survey. Comput Vision and Image Understanding. https://doi.org/10.1016/j.cviu.2022.103525

    Article  Google Scholar 

  20. Al-Janabi S, Al-Janabi Z (2023) Development of deep learning method for predicting DC power based on renewable solar energy and multi-parameters function. Neural Comput Appl 35(21):15273–15294. https://doi.org/10.1007/S00521-023-08480-6/TABLES/16

    Article  Google Scholar 

  21. Al-Janabi S, Alkaim AF, Adel Z (2020) An Innovative synthesis of deep learning techniques (DCapsNet & DCOM) for generation electrical renewable energy from wind energy. Soft comput 24(14):10943–10962. https://doi.org/10.1007/S00500-020-04905-9/TABLES/7

    Article  Google Scholar 

  22. Singh K, Malhotra D (2023) Meta-health: learning-to-learn (meta-learning) as a next generation of deep learning exploring healthcare challenges and solutions for rare disorders: a systematic analysis. Arch Comput Methods Eng 30(7):4081–4112. https://doi.org/10.1007/S11831-023-09927-8/FIGURES/6

    Article  Google Scholar 

  23. Sandotra N, Mahajan P, Abrol P, Lehana PK (2023) Analyzing performance of deep learning models under the presence of distortions in identifying plant leaf disease. Int J Inf Commun Technol IJ-ICT 12(2):115. https://doi.org/10.11591/ijict.v12i2.pp115-126

    Article  Google Scholar 

  24. Gupta S, Sharma P (2022) Machine learning approach for heart disease prediction: a survey. AIP Conf Proc 2555:1. https://doi.org/10.1063/5.0108884/2829443

    Article  Google Scholar 

  25. Bregler C, Covell M and Slaney M (1997) “Video Rewrite: Driving visual speech with audio,” Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1997, pp 353–360, , https://doi.org/10.1145/258734.258880.

  26. “DARPA is funding new tech that can identify manipulated videos and ‘deepfakes’ | TechCrunch.” https://techcrunch.com/2018/04/30/deepfakes-fake-videos-darpa-sri-international-media-forensics/ (accessed Sep. 29, 2022).

  27. Mirsky Y, Lee W (2020) The creation and detection of deepfakes: a survey. ACM Comput Surv. https://doi.org/10.1145/3425780

    Article  Google Scholar 

  28. Chan C, Ginosar S, Zhou T, and Efros A (2018) “Everybody Dance Now,” Proceedings of the IEEE International Conference on Computer Vision, vol. 2019-October, pp. 5932–5941, https://doi.org/10.48550/arxiv.1808.07371.

  29. Sharma M and Kaur M (2022) “A Review of deepfake technology: an emerging ai threat,” Soft Computing for Security Applications, Proceedings of ICSCS, pp. 605–619, https://doi.org/10.1007/978-981-16-5301-8_44.

  30. Rana MS, Nobi MN, Murali B, Sung AH (2022) Deepfake detection: a systematic literature review. IEEE Access 10:25494–25513. https://doi.org/10.1109/ACCESS.2022.3154404

    Article  Google Scholar 

  31. Dolhansky B et al., “The DeepFake Detection Challenge (DFDC) Dataset,” Computer Vision and Pattern Recognition, vol. 1, Jun. 2020, https://doi.org/10.48550/arxiv.2006.07397.

  32. Li Y, Yang X, Sun P, Qi H and Lyu S “Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics.” [Online]. Available: https://deepfakedetectionchallenge.ai.

  33. Mehta V, Gupta P, Subramanian R and Dhall A 2021 “FakeBuster: A DeepFakes Detection Tool for Video Conferencing Scenarios,” International Conference on Intelligent User Interfaces, Proceedings IUI, pp. 61–63, https://doi.org/10.48550/arxiv.2101.03321.

  34. Nitzan Y et al (2022) MyStyle: a personalized generative prior. ACM Trans Gr (TOG). https://doi.org/10.1145/3550454.3555436

    Article  Google Scholar 

  35. Wang Y, Bilinski P, Bremond F and Dantcheva A (2020) “ImaGINator: Conditional spatio-temporal GAN for video generation,” In: Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020, pp. 1149–1158, https://doi.org/10.1109/WACV45572.2020.9093492.

  36. “DeepFake Video: Cadbury’s New AI Tool Allows to Create Free Ads Featuring Shah Rukh Khan’s Face and Voice.” https://www.thehansindia.com/technology/tech-news/cadburys-new-ai-tool-allows-to-create-free-ads-featuring-shah-rukh-khans-face-and-voice-712400 (accessed Sep. 15, 2022).

  37. Ballesteros DM, Rodriguez-Ortega Y, Renza D, Arce G (2021) Deep4SNet: deep learning for fake speech classification. Expert Syst Appl 184:115465. https://doi.org/10.1016/J.ESWA.2021.115465

    Article  Google Scholar 

  38. “Fraudsters Used AI to Mimic CEO’s Voice in Unusual Cybercrime Case - WSJ.” https://www.wsj.com/articles/fraudsters-use-ai-to-mimic-ceos-voice-in-unusual-cybercrime-case-11567157402 (accessed Sep. 15, 2022).

  39. Todisco M et al., (2019) “ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection,” In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, vol. 2019-September, pp. 1008–1012, https://doi.org/10.48550/arxiv.1904.05441.

  40. “GitHub - iperov/DeepFaceLive: Real-time face swap for PC streaming or video calls.” https://github.com/iperov/DeepFaceLive (accessed Mar. 29, 2023).

  41. Thies J, Zollhöfer M, Nießner M, Valgaerts L, Stamminger M, Theobalt C (2015) Real-time expression transfer for facial reenactment. ACM Trans Gr (TOG). https://doi.org/10.1145/2816795.2818056

    Article  Google Scholar 

  42. Thies J, Zollhöfer M, Stamminger M, Theobalt C, Nießner M (2020) Face2Face: real-time face capture and reenactment of RGB videos. Commun ACM 62(1):96–104. https://doi.org/10.48550/arxiv.2007.14808

    Article  Google Scholar 

  43. Cao C, Bradley D, Zhou K, Beeler T (2015) Real-time high-fidelity facial performance capture. ACM Transactions on Graphics (TOG). https://doi.org/10.1145/2766943

    Article  Google Scholar 

  44. Dagar D, Vishwakarma DK (2022) A literature review and perspectives in deepfakes: generation, detection, and applications. Int J Multimed Inf Retr 11(3):219–289. https://doi.org/10.1007/s13735-022-00241-w

    Article  Google Scholar 

  45. Rao S, Verma AK, Bhatia T (2021) A review on social spam detection: Challenges, open issues, and future directions. Expert Syst Appl 186:115742. https://doi.org/10.1016/J.ESWA.2021.115742

    Article  Google Scholar 

  46. Roy PK, Chahar S (2020) Fake profile detection on social networking websites: a comprehensive review. IEEE Trans Artificial Intell 1(3):271–285. https://doi.org/10.1109/TAI.2021.3064901

    Article  Google Scholar 

  47. Juefei-Xu F, Wang R, Huang Y, Guo Q, Ma L, Liu Y (2022) Countering malicious deepfakes: survey, battleground, and horizon. Int J Comput Vis. https://doi.org/10.1007/s11263-022-01606-8

    Article  PubMed  PubMed Central  Google Scholar 

  48. Chesney R, Citron DK (2018) Deep fakes: a looming challenge for privacy, democracy, and national security. SSRN Electron J. https://doi.org/10.2139/SSRN.3213954

    Article  Google Scholar 

  49. Rani R, Kumar T, Sah MP (2022) A review on deepfake media detection. Lecture Notes in Netw Syst 461:343–356. https://doi.org/10.1007/978-981-19-2130-8_28/COVER

    Article  Google Scholar 

  50. Korshunov P and Marcel S, “DeepFakes: a New Threat to Face Recognition? Assessment and Detection,” Dec. 2018, [Online]. Available: http://arxiv.org/abs/1812.08685

  51. Yang X, Li Y, and Lyu S (2019) “Exposing deep fakes using inconsistent head poses,” ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing In: Proceedings, vol. 2019-May, pp. 8261–8265, https://doi.org/10.1109/ICASSP.2019.8683164.

  52. Lee S, Tariq S, Shin Y (2020) “Hand-crafted Facial Manipulation (HFM) Dataset. Mendeley Data. https://doi.org/10.17632/H4YMVY9G8J.1

    Article  Google Scholar 

  53. Rössler A, Cozzolino D, Verdoliva L, Riess C, Thies J, and Nießner M (2019) “FaceForensics++: Learning to Detect Manipulated Facial Images,” [Online]. Available: http://arxiv.org/abs/1901.08971

  54. Li Y, Yang X, Sun P, Qi H, and Lyu S, (2020) “Celeb-DF: a large-scale challenging dataset for deepfake forensics,”In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3204–3213, https://doi.org/10.1109/CVPR42600.2020.00327.

  55. Huang J, Wang X, Du B, Du P, and Xu AC “DeepFake MNIST+: A DeepFake Facial Animation Dataset.” [Online]. Available: https://github.com/huangjiadidi/DeepFakeMnist

  56. Zi B, Chang M, Chen J, Ma X, and Jiang YG (2020) “WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection,” in MM 2020 - Proceedings of the 28th ACM International Conference on Multimedia, Association for Computing Machinery, Inc, Oct. pp. 2382–2390. https://doi.org/10.1145/3394171.3413769.

  57. “140k Real and Fake Faces | Kaggle.” https://www.kaggle.com/datasets/xhlulu/140k-real-and-fake-faces (accessed Mar. 21, 2023).

  58. He Y et al. (2021) “ForgeryNet: A versatile benchmark for comprehensive forgery analysis,” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4358–4367, https://doi.org/10.1109/CVPR46437.2021.00434.

  59. Dang H, Liu F, Stehouwer J, Liu X and Jain A (2019) “On the detection of digital face manipulation,” [Online]. Available: http://arxiv.org/abs/1910.01717

  60. Jain A, Korshunov P, and Marcel S (2021) “Improving generalization of deepfake detection by training for attribution,” IEEE 23rd International Workshop on Multimedia Signal Processing, MMSP 2021, https://doi.org/10.1109/MMSP53017.2021.9733468.

  61. “FakeApp 2.2.0.” (2022) https://www.malavida.com/en/soft/fakeapp/#gref (accessed Sep. 12).

  62. Faceswap (2022), “Deepfakes software for all” https://github.com/topics/faceswap (accessed Sep. 12).

  63. Preeti MK, Sharma HK (2023) A GAN-based model of deepfake detection in social media. Procedia Comput Sci 218:2153–2162. https://doi.org/10.1016/j.procs.2023.01.191

    Article  Google Scholar 

  64. Shen T et al., (2023) “‘deep fakes’ using generative adversarial networks (gan),” noiselab.ucsd.edu, 2018, Accessed: Mar. 21[Online]. Available: http://noiselab.ucsd.edu/ECE228_2018/Reports/Report16.pdf

  65. Brock A, Donahue J, and Simonyan K (2018) “Large scale GAN training for high fidelity natural image synthesis,” 7th International Conference on Learning Representations, ICLR 2019, Sep. https://doi.org/10.48550/arxiv.1809.11096.

  66. Li L, Bao J, Yang H, Chen D, and Wen F (2020) “Advancing high fidelity identity swapping for forgery detection,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 5073–5082, https://doi.org/10.1109/CVPR42600.2020.00512.

  67. Korshunova I, Shi W, Dambre J, and Theis L, (2017) “Fast face-swap using convolutional neural networks,” 2017 IEEE International Conference on Computer Vision (ICCV), vol. 2017-October, pp. 3697–3705, https://doi.org/10.1109/ICCV.2017.397.

  68. Natsume R, Yatagawa T, and Morishima S, “FSNet: An Identity-Aware Generative Model for Image-Based Face Swapping,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11366 LNCS, pp. 117–132, 2019, https://doi.org/10.1007/978-3-030-20876-9_8/COVER.

  69. Prajwal KR, Jha A, Mukhopadhyay R, Namboodiri V, Philip J and Jawahar Cv , (2019) “Towards automatic face-to-face translation,” in MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia, Association for Computing Machinery, Inc, pp. 1428–1436. https://doi.org/10.1145/3343031.3351066.

  70. Prajwal KR, Mukhopadhyay R, Namboodiri VP, and Jawahar Cv (2020) “A lip sync expert is all you need for speech to lip generation in the wild,” in MM 2020 - Proceedings of the 28th ACM International Conference on Multimedia, Association for Computing Machinery, Inc, pp. 484–492. https://doi.org/10.1145/3394171.3413532.

  71. Nirkin Y, Keller Y, and Hassner T (2019) “FSGAN: Subject Agnostic Face Swapping and Reenactment,” [Online]. Available: http://arxiv.org/abs/1908.05932

  72. Hao H, Baireddy S, Reibman AR, and Delp EJ (2020) “FaR-GAN for One-Shot Face Reenactment,” May [Online]. Available: http://arxiv.org/abs/2005.06402

  73. Pumarola A, Agudo A, Martinez AM, Sanfeliu A and Moreno-Noguer F (2018) “GANimation: Anatomically-aware Facial Animation from a Single Image,”[Online]. Available: http://arxiv.org/abs/1807.09251

  74. Zhang H, Goodfellow I, Metaxas D, and Odena A, (2018) “Self-Attention Generative Adversarial Networks,” [Online]. Available: http://arxiv.org/abs/1805.08318

  75. Huang R, Zhang S, Li T, and He R, (2017) “Beyond face rotation: global and local perception GAN for photorealistic and identity preserving frontal view synthesis,” [Online]. Available: http://arxiv.org/abs/1704.04086

  76. Choi Y, Uh Y, Yoo J, and Ha JW (2019) “StarGAN v2: Diverse Image Synthesis for Multiple Domains,” [Online]. Available: http://arxiv.org/abs/1912.01865

  77. He Z, Kan M, Zhang J, and Shan S (2020) “PA-GAN: Progressive Attention Generative Adversarial Network for Facial Attribute Editing,” [Online]. Available: http://arxiv.org/abs/2007.05892

  78. Dale K, Sunkavalli K, Johnson MK, Vlasic D, Matusik W, and Pfister H (2011) “Video face replacement,” Proceedings of the 2011 SIGGRAPH Asia Conference, vol. 30, no. 6, Dec. https://doi.org/10.1145/2024156.2024164.

  79. Karras T, Aila T, Laine S, and Lehtinen J (2017) “Progressive growing of GANs for improved quality, stability, and variation,” 6th international conference on learning representations, ICLR 2018 - Conference Track Proceedings, https://doi.org/10.48550/arxiv.1710.10196.

  80. Karras T, Laine S, Aila T (2018) A style-based generator architecture for generative adversarial networks. IEEE Trans Pattern Anal Mach Intell 43(12):4217–4228. https://doi.org/10.48550/arxiv.1812.04948

    Article  Google Scholar 

  81. Karras T, Laine S, Aittala M, Hellsten J, Lehtinen J and Aila T (2019) “Analyzing and Improving the Image Quality of StyleGAN,” Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp. 8107–8116, https://doi.org/10.48550/arxiv.1912.04958.

  82. Thies J, Zollhofer M, Stamminger M, Theobalt C and Niebner M (2016) “Face2Face: real-time face capture and reenactment of RGB videos,” Proceedings of the IEEE computer society conference on computer vision and pattern recognition, vol. 2016-December, pp. 2387–2395, Dec. https://doi.org/10.1109/CVPR.2016.262.

  83. Thies J, Zollhöfer M, Nießner M (2019) Deferred neural rendering: image synthesis using neural textures. ACM Trans Graph. https://doi.org/10.48550/arxiv.1904.12356

    Article  Google Scholar 

  84. Liu L et al (2019) Neural Rendering and Reenactment of Human Actor Videos. ACM Trans Gr (TOG). https://doi.org/10.1145/3333002

    Article  Google Scholar 

  85. Doukas MC, Koujan MR, Sharmanska V, Roussos A, Zafeiriou S (2021) Head2Head++: deep facial attributes re-targeting. IEEE Trans Biom Behav Identity Sci 3(1):31–43. https://doi.org/10.1109/TBIOM.2021.3049576

    Article  Google Scholar 

  86. Wang TC, Liu MY, Tao A, Liu G, Kautz J, Catanzaro B (2019) Few-shot video-to-video synthesis. Adv Neural Inf Process Syst. https://doi.org/10.48550/arxiv.1910.12713

    Article  PubMed  PubMed Central  Google Scholar 

  87. Gafni O, Ashual O, and Wolf L (2020) “Single-Shot Freestyle Dance Reenactment,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 882–891, https://doi.org/10.48550/arxiv.2012.01158.

  88. Zhang J et al., (2019) “FReeNet: Multi-Identity Face Reenactment,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 5325–5334, May https://doi.org/10.48550/arxiv.1905.11805.

  89. Zhang Y, Zhang S, He Y, Li C, Loy CC, and Liu Z (2019) “One-shot Face Reenactment,” 30th British Machine Vision Conference 2019, BMVC 2019, https://doi.org/10.48550/arxiv.1908.03251.

  90. Gu K, Zhou Y, and Huang T (2019) “FLNet: Landmark Driven Fetching and Learning Network for Faithful Talking Facial Animation Synthesis,” AAAI 2020 - 34th AAAI Conference on Artificial Intelligence, pp. 10861–10868, https://doi.org/10.48550/arxiv.1911.09224.

  91. Lee J, Ramanan D, Girdhar R (2019) MetaPix: few-shot video retargeting. Int Conf Learn Represent (ICLR).

    Article  Google Scholar 

  92. Fried O et al (2019) Text-based Editing of Talking-head Video. ACM Trans Graph 38(4):14. https://doi.org/10.1145/3306346

    Article  Google Scholar 

  93. Lahiri A, Kwatra V, Frueh C, Lewis J, and Bregler C (2021) “LipsyNc3D: Data-Efficient Learning of Personalized 3D Talking Faces from Video using Pose and Lighting Normalization,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2754–2763, https://doi.org/10.1109/CVPR46437.2021.00278.

  94. Jamaludin A, Chung JS, Zisserman A (2019) You said that?: synthesising talking faces from audio. Int J Comput Vis 127(11–12):1767–1779

    Article  Google Scholar 

  95. Fried O et al (2019) Text-based Editing of Talking-head Video. ACM Trans Graph 38(4):1–14.

    Article  Google Scholar 

  96. Zhang Z, Li L, Ding Y, and Fan C (2021) “Flow-guided One-shot Talking Face Generation with a High-resolution Audio-visual Dataset,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3660–3669, https://doi.org/10.1109/CVPR46437.2021.00366.

  97. Ak K, Kassim A, Lim JH, and Tham JY, (2019) “Attribute manipulation generative adversarial networks for fashion images,” Proceedings of the IEEE International Conference on Computer Vision, vol. 2019-October, pp. 10540–10549, https://doi.org/10.1109/ICCV.2019.01064.

  98. Choi Y, Choi M, Kim M, Ha JW, Kim S and Choo J (2017) “StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 8789–8797, https://doi.org/10.48550/arxiv.1711.09020.

  99. Pumarola A, Agudo A, Martinez AM, Sanfeliu A, and Moreno-Noguer F (2018) “GANimation: Anatomically-aware Facial Animation from a Single Image,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11214 LNCS, pp. 835–851, https://doi.org/10.48550/arxiv.1807.09251.

  100. Liu M et al., (2019) “STGAN: A unified selective transfer network for arbitrary image attribute editing,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2019-June, pp. 3668–3677, https://doi.org/10.48550/arxiv.1904.09709.

  101. Cao M et al (2021) UniFaceGAN: A Unified Framework for Temporally Consistent Facial Video Editing. IEEE Trans Image Process 30:6107–6116. https://doi.org/10.1109/TIP.2021.3089909

    Article  ADS  PubMed  Google Scholar 

  102. Liang H, Hou X, and Shen L (2021) “SSFlow: style-guided neural spline flows for face image manipulation,” MM 2021 - Proceedings of the 29th ACM International Conference on Multimedia, pp. 79–87, https://doi.org/10.1145/3474085.3475454.

  103. Karras T, Laine S, and Aila T (2019) “A style-based generator architecture for generative adversarial networks,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2019-June, pp. 4396–4405, https://doi.org/10.1109/CVPR.2019.00453.

  104. Tong X, Wang L, Pan X, and Wang JG (2020) “An Overview of Deepfake: The Sword of Damocles in AI,” Proceedings - 2020 International Conference on Computer Vision, Image and Deep Learning, CVIDL 2020, pp. 265–273, https://doi.org/10.1109/CVIDL51233.2020.00-88.

  105. Wu W, Zhou W, Zhang W, Fang H, and Yu N (2022) “Capturing the lighting inconsistency for deepfake detection,” lecture notes in computer science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13339 LNCS, pp. 637–647, https://doi.org/10.1007/978-3-031-06788-4_52.

  106. Wang T, Liu M, Cao W, Chow KP (2022) Deepfake noise investigation and detection. Forensic Sci Int 42:301395.

    Google Scholar 

  107. Ganguly S, Ganguly A, Mohiuddin S, Malakar S, Sarkar R (2022) ViXNet: vision transformer with xception network for deepfakes based video and image forgery detection. Expert Syst Appl 210:118423. https://doi.org/10.1016/J.ESWA.2022.118423

    Article  Google Scholar 

  108. Matern F, Riess C, and Stamminger M (2019) “Exploiting visual artifacts to expose deepfakes and face manipulations,” Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019, pp. 83–92, https://doi.org/10.1109/WACVW.2019.00020.

  109. Sabir E, Cheng J, Jaiswal A, AbdAlmageed W, Masi I, and Natarajan P (2019) “Recurrent Convolutional Strategies for Face Manipulation Detection in Videos,” [Online]. Available: http://arxiv.org/abs/1905.00582

  110. Lee G, Kim M (2021) Deepfake Detection using the rate of change between frames based on computer vision. Sensors (Basel). https://doi.org/10.3390/S21217367

    Article  PubMed  PubMed Central  Google Scholar 

  111. Qi H et al., (2020) “DeepRhythm: exposing deepfakes with attentional visual heartbeat rhythms,” [Online]. Available: http://arxiv.org/abs/2006.07634

  112. Li Y, Chang MC and Lyu S (2023) “In Ictu Oculi: Exposing AI Generated Fake Face Videos by Detecting Eye Blinking,” Jun. 2018, Accessed: May 11, [Online]. Available: https://arxiv.org/abs/1806.02877v2

  113. Jung T, Kim S, Kim K (2020) DeepVision: deepfakes detection using human eye blinking pattern. IEEE Access 8:83144–83154. https://doi.org/10.1109/ACCESS.2020.2988660

    Article  Google Scholar 

  114. “Eyebrow Recognition for Identifying Deepfake Videos | IEEE Conference Publication | IEEE Xplore.” https://ieeexplore.ieee.org/document/9211068 (accessed May 11, 2023).

  115. Amerini I, Galteri L, Caldelli R, ♦♣, and Del Bimbo A, (2023) “Deepfake Video Detection through Optical Flow based CNN”, Accessed: May 12, [Online]. Available: https://github.com/deepfakes/

  116. McCloskey S and Albright M (2019) “Detecting GAN-generated imagery using saturation cues,” Proceedings - International Conference on Image Processing, ICIP, vol. 2019-September, pp. 4584–4588, https://doi.org/10.1109/ICIP.2019.8803661.

  117. Ferreira S, Antunes M, Correia ME (2021) Exposing manipulated photos and videos in digital forensics analysis. J Imaging. https://doi.org/10.3390/JIMAGING7070102

    Article  PubMed  PubMed Central  Google Scholar 

  118. Zhang Y, Zheng L, and Thing VLL (2017) “Automated face swapping and its detection,” 2017 IEEE 2nd International Conference on Signal and Image Processing, ICSIP 2017, vol. 2017-January, pp. 15–19, https://doi.org/10.1109/SIPROCESS.2017.8124497.

  119. Mo H, Chen B, and Luo W (2018) “Fake faces identification via convolutional neural network,” IH and MMSec 2018 In: Proceedings of the 6th ACM Workshop on Information Hiding and Multimedia Security, pp. 43–47, https://doi.org/10.1145/3206004.3206009.

  120. L. Nataraj et al., (2019) “Detecting GAN generated Fake Images using Co-occurrence Matrices,” IS and T International Symposium on Electronic Imaging Science and Technology, https://doi.org/10.2352/ISSN.2470-1173.2019.5.MWSF-532.

  121. Guarnera L, Giudice O, and Battiato S (2020) “DeepFake detection by analyzing convolutional traces,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, vol. 2020-June, pp. 2841–2850, https://doi.org/10.1109/CVPRW50498.2020.00341.

  122. Li Y and Lyu S (2023) “Exposing DeepFake Videos By Detecting Face Warping Artifacts,” Nov. 2018, Accessed: Apr. 13 [Online]. Available: https://arxiv.org/abs/1811.00656v3

  123. Li L et al., (2019) “Face X-ray for More General Face Forgery Detection,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 5000–5009, https://doi.org/10.1109/CVPR42600.2020.00505.

  124. Li K, Yu S, Ji Y, Wang C, Wu and Xue H (2020) “Fighting Against Deepfake: Patch&Pair Convolutional Neural Networks (PPCNN),” The Web Conference 2020 - Companion of the World Wide Web Conference, WWW 2020, pp. 88–89, https://doi.org/10.1145/3366424.3382711

  125. Lee S, Tariq S, Shin Y, Woo SS (2021) Detecting handcrafted facial image manipulations and GAN-generated facial images using Shallow-FakeFaceNet. Appl Soft Comput. https://doi.org/10.1016/J.ASOC.2021.107256

    Article  PubMed  Google Scholar 

  126. Shang Z, Xie H, Zha Z, Yu L, Li Y, Zhang Y (2021) PRRNet: Pixel-Region relation network for face forgery detection. Pattern Recognit 116:107950. https://doi.org/10.1016/J.PATCOG.2021.107950

    Article  Google Scholar 

  127. Ke J, Wang L (2023) DF-UDetector: an effective method towards robust deepfake detection via feature restoration. Neural Netw 160:216–226. https://doi.org/10.1016/j.neunet.2023.01.001

    Article  PubMed  Google Scholar 

  128. Kosarkar U, Sarkarkar G, Gedam S (2023) Revealing and classification of deepfakes video’s images using a customize convolution neural network model. Procedia Comput Sci 218:2636–2652. https://doi.org/10.1016/J.PROCS.2023.01.237

    Article  Google Scholar 

  129. Caldelli R, Galteri L, Amerini I, Del Bimbo A (2021) Optical flow based CNN for detection of unlearnt deepfake manipulations. Pattern Recognit Lett 146:31–37. https://doi.org/10.1016/J.PATREC.2021.03.005

    Article  ADS  Google Scholar 

  130. Vamsi VVVNS et al (2022) Deepfake detection in digital media forensics. Global Trans Proc 3(1):74–79. https://doi.org/10.1016/J.GLTP.2022.04.017

    Article  Google Scholar 

  131. Elhassan A, Al-Fawa’reh M, Jafar MT, Ababneh M, Jafar ST (2022) DFT-MF: Enhanced deepfake detection using mouth movement and transfer learning. SoftwareX. https://doi.org/10.1016/J.SOFTX.2022.101115

    Article  Google Scholar 

  132. Liu B, Liu B, Ding M, Zhu T and Yu X (2023) “TI2Net: Temporal Identity Inconsistency Network for Deepfake Detection,” Proceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023, pp. 4680–4689, https://doi.org/10.1109/WACV56688.2023.00467.

  133. Singh A, Saimbhi AS, Singh N, Mittal M (2020) DeepFake Video Detection: A Time-Distributed Approach. SN Comput Sci 1:4. https://doi.org/10.1007/s42979-020-00225-9

    Article  Google Scholar 

  134. Sun Z, Han Y, Hua Z, Ruan N, and Jia W (2021) “Improving the Efficiency and Robustness of Deepfakes Detection through Precise Geometric Features,” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3608–3617, https://doi.org/10.1109/CVPR46437.2021.00361.

  135. Liu H et al., “Spatial-phase shallow learning: rethinking face forgery detection in frequency domain”.

  136. Wang B, Wu X, Tang Y, Ma Y, Shan Z, Wei F (2023) Frequency Domain Filtered Residual Network for Deepfake Detection. Mathematics 11:816. https://doi.org/10.3390/MATH11040816

    Article  Google Scholar 

  137. Frank J, Eisenhofer T, Schönherr L, Fischer A, Kolossa D and Holz T (2020) “Leveraging frequency analysis for deep fake image recognition,” ICML’20: Proceedings of the 37th International Conference on Machine Learning, https://doi.org/10.5555/3524938.3525242.

  138. Nadimpalli AV and Rattani A “On Improving Cross-dataset Generalization of Deepfake Detectors.”

  139. Chen L, Zhang Y, Song Y, Liu L, and Wang J “Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection.” [Online]. Available: https://github.com/liangchen527/SLADD.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science & Information Technology, Central University of Jammu, Bagla (Rahya Suchani), District-Samba, Jammu, J&K, 181143, India

    Neha Sandotra & Bhavna Arora

Authors
  1. Neha Sandotra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bhavna Arora
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

NS was involved in draft and design the manuscript; BA revised and approved the final manuscript.

Corresponding author

Correspondence to Neha Sandotra.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Ethical approval

Not applicable.

Consent to publish

Not applicable

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sandotra, N., Arora, B. A comprehensive evaluation of feature-based AI techniques for deepfake detection. Neural Comput & Applic 36, 3859–3887 (2024). https://doi.org/10.1007/s00521-023-09288-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-09288-0

Keywords