[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1007/978-981-97-8795-1_32guideproceedingsArticle/Chapter ViewAbstractPublication PagesConference Proceedingsacm-pubtype
Article

Spatial-Frequency Dual-Stream Reconstruction for Deepfake Detection

Published: 03 November 2024 Publication History

Abstract

The widespread usage of Deepfake technology poses a significant threat to societal security, making the detection of Deepfakes a critical area of research. In recent years, forgery detection methods based on reconstruction errors have garnered widespread attention due to their excellent performance and generalization capabilities. However, those methods often focus on spatial reconstruction errors while neglecting the potential utility of frequency-based reconstruction errors. In this paper, we propose a novel deepfake detection framework based on Spatial-Frequency Dual-stream Reconstruction (SFDR). Specifically, our approach to forgery detection utilizes both frequency reconstruction error and spatial reconstruction error to provide complementary information that enhances the detection process. In addition, during the reconstruction, we ensure the consistency of frequency content between the original genuine images and their reconstructed versions. Finally, to mitigate the adverse impact of reconstruction tasks on the performance of forgery detection, we have refined the reconstruction loss to minimize the discrepancy between the original genuine images and their reconstructed counterparts; while simultaneously maximizing the difference between manipulated images and their reconstructions. Experimental results on multiple challenging forged datasets evaluation show that our method achieves superior performance in detection and generalization ability.

References

[1]
Cao, J., Ma, C., Yao, T., Chen, S., Ding, S., Yang, X.: End-to-end reconstruction-classification learning for face forgery detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4113–4122, June (2022)
[2]
Cheng, Z., Chen, C., Zhou, Y., Hu, X.: Mining temporal inconsistency with 3d face model for deepfake video detection. In: Chinese Conference on Pattern Recognition and Computer Vision (PRCV), pp. 231–243. Springer (2023)
[3]
Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1800–1807 (2017)
[4]
Deng, J., Guo, J., Ververas, E., Kotsia, I., Zafeiriou, S.: Retinaface: single-shot multi-level face localisation in the wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June (2020)
[5]
Du, M., Pentyala, S., Li, Y., Hu, X.: Towards generalizable deepfake detection with locality-aware autoencoder. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 325–334 (2020)
[6]
Frank, J., Eisenhofer, T., Schönherr, L., Fischer, A., Kolossa, D., Holz, T.: Leveraging frequency analysis for deep fake image recognition. In: International Conference on Machine Learning, pp. 3247–3258. PMLR (2020)
[7]
Groshev A, Maltseva A, Chesakov D, Kuznetsov A, and Dimitrov D Ghost-a new face swap approach for image and video domains IEEE Access 2022 10 83452-83462
[8]
Qiqi, G., Chen, S., Yao, T., Chen, Y., Ding, S., Yi, R.: Exploiting fine-grained face forgery clues via progressive enhancement learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 735–743 (2022)
[9]
Hassani A, Malik H, and Diedrich J Efficiently mitigating face-swap-attacks: compressed-prnu verification with sub-zones Technologies 2022 10 2 46
[10]
He, Y., Yu, N., Keuper, M., Fritz, M.: Beyond the spectrum: detecting deepfakes via re-synthesis. arXiv preprint arXiv:2105.14376 (2021)
[11]
Hui, Z., Li, J., Wang, X., Gao, X.: Image fine-grained inpainting. arXiv preprint arXiv:2002.02609 (2020)
[12]
Jia, F., Yang, S.: Video face swap with deepfacelab. In: International Conference on Computer Graphics, Artificial Intelligence, and Data Processing (ICCAID 2021), vol. 12168, pp. 326–332. SPIE (2022)
[13]
Jiang, L., Dai, B., Wu, W., Loy, C.C.: Focal frequency loss for image reconstruction and synthesis. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13919–13929 (2021)
[14]
Li, Y., Yang, X., Sun, P., Qi, H., Lyu, S.: Celeb-df: a large-scale challenging dataset for deepfake forensics. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June (2020)
[15]
Luo, Y., Zhang, Y., Yan, J., Liu, W.: Generalizing face forgery detection with high-frequency features. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16317–16326, June (2021)
[16]
Miao C, Chu Q, Li W, Li S, Tan Z, Zhuang W, and Nenghai Yu Learning forgery region-aware and id-independent features for face manipulation detection IEEE Trans. Biom. Behav. Identity Sci. 2021 4 1 71-84
[17]
Nguyen, H.H., Fang, F., Yamagishi, J., Echizen, I.: Multi-task learning for detecting and segmenting manipulated facial images and videos. In: 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS), pp. 1–8. IEEE (2019)
[18]
Qian, Y., Yin, G., Sheng, L., Chen, Z., Shao, J.: Thinking in frequency: face forgery detection by mining frequency-aware clues. In: European Conference on Computer Vision, pp. 86–103. Springer (2020)
[19]
Reader, A.J., Corda, G., Mehranian, A., da Costa-Luis, C., Ellis, S., Schnabel, J.A.: Deep learning for pet image reconstruction. IEEE Trans. Radiat. Plasma Med. Sci. 5(1), 1–25 (2020)
[20]
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Niessner, M.: Faceforensics++: learning to detect manipulated facial images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October (2019)
[21]
Ruan, D., Yan, Y., Lai, S., Chai, Z., Shen, C., Wang, H.: Feature decomposition and reconstruction learning for effective facial expression recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7660–7669 (2021)
[22]
Tan, M., Le, Q.: Efficientnet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114. PMLR (2019)
[23]
Wang, C., Deng, W.: Representative forgery mining for fake face detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14923–14932, June (2021)
[24]
Wang, L., Ma, C.: Adapting pretrained large-scale vision models for face forgery detection. In: International Conference on Multimedia Modeling, pp. 71–85. Springer (2024)
[25]
Wang, Z., Guo, Y., Zuo, W.: Deepfake forensics via an adversarial game. IEEE Trans. Image Process. (2022)
[26]
Yoshihashi, R., Shao, W., Kawakami, R., You, S., Iida, M., Naemura, T.: Classification-reconstruction learning for open-set recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4016–4025 (2019)
[27]
Yu, Y., Ni, R., Zhao, Y.: Mining generalized features for detecting ai-manipulated fake faces. arXiv preprint arXiv:2010.14129 (2020)
[28]
Zhang, A., McAllister, R., Calandra, R., Gal, Y., Levine, S.: Learning invariant representations for reinforcement learning without reconstruction. arXiv preprint arXiv:2006.10742 (2020)
[29]
Zhao, H., Zhou, W., Chen, D., Wei, T., Zhang, W., Yu, N.: Multi-attentional deepfake detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2185–2194, June (2021)
[30]
Zheng, J., Zhou, Y., Hu, X., Tang, Z.: Dt-transunet: a dual-task model for deepfake detection and segmentation. In: Chinese Conference on Pattern Recognition and Computer Vision (PRCV), pp. 244–255. Springer (2023)
[31]
Zi, B., Chang, M., Chen, J., Ma, X., Jiang, Y.-G.: Wilddeepfake: a challenging real-world dataset for deepfake detection. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 2382–2390 (2020)

Index Terms

  1. Spatial-Frequency Dual-Stream Reconstruction for Deepfake Detection
            Index terms have been assigned to the content through auto-classification.

            Recommendations

            Comments

            Please enable JavaScript to view thecomments powered by Disqus.

            Information & Contributors

            Information

            Published In

            cover image Guide Proceedings
            Pattern Recognition and Computer Vision: 7th Chinese Conference, PRCV 2024, Urumqi, China, October 18–20, 2024, Proceedings, Part XI
            Oct 2024
            603 pages
            ISBN:978-981-97-8794-4
            DOI:10.1007/978-981-97-8795-1

            Publisher

            Springer-Verlag

            Berlin, Heidelberg

            Publication History

            Published: 03 November 2024

            Author Tags

            1. Deepfake Detection
            2. Reconstruction Learning
            3. Generalization

            Qualifiers

            • Article

            Contributors

            Other Metrics

            Bibliometrics & Citations

            Bibliometrics

            Article Metrics

            • 0
              Total Citations
            • 0
              Total Downloads
            • Downloads (Last 12 months)0
            • Downloads (Last 6 weeks)0
            Reflects downloads up to 26 Jan 2025

            Other Metrics

            Citations

            View Options

            View options

            Figures

            Tables

            Media

            Share

            Share

            Share this Publication link

            Share on social media