[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

DSSE-net: dual stream skip edge-enhanced network with forgery loss for image forgery localization

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

As deep learning has continuously made breakthroughs in computer vision, Image Forgery Localization (IFL) task has also started using deep learning frameworks. Currently, most deep learning-based IFL methods use binary cross entropy as the loss function during model training. However, the number of tampered pixels in image forgery is significantly smaller than the number of real pixels. This disparity makes it easier for the model to classify samples as real pixels during training, leading to a reduced F1 score. Therefore, in this paper, we have proposed a loss function for the IFL task: Forgery Loss. The Forgery Loss assigns weight to the classification loss of tampered pixels and edges, enhances tampered pixel constraints in the model, and amplifies the importance of difficult-to-classify samples. These enhancements facilitate the model to acquire more productive information. Consequently, the F1 score of the model is enhanced. Additionally, we designed an end-to-end, pixel-level detection network DSSE-Net. It comprises of a dual-stream codec network that extracts high-level and low-level features of images, and an edge attention stream. The edge attention stream have a Edge Attention Model which enhances the network’s attention to the high frequency edges of the image and, in conjunction with the edge enhancement algorithm in Forgery Loss, improves the model’s ability to detect tampered edges. Experiments demonstrate that Forgery Loss can effectively improve the F1 score, while the DSSE-Net accuracy outperforms the current SOTA algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Algorithm 1
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

All datasets that support the findings of this study are openly available. CASIAv2 at https://github.com/namtpham/casia2groundtruth, COLUMBIA at https://www.ee.columbia.edu/ln/dvmm/downloads/authsplcuncmp/, NIST at https://mfc.nist.gov/users/sign_in, Pre-training synthetic dataset at https://github.com/jawadbappy/forgery_localization_HLED/tree/master/synthetic_data.

References

  1. Ghanim TM, Nabil AM (2018) Offline signature verification and forgery detection approach. Proceeding of the 2018 13th International Conference on computer Engineering and Systems (ICCES), 293–298

  2. Verdoliva L (2020) Media forensics and deepfakes: an overview. IEEE J Selected Topics Signal Process 14(5):910–932

    Article  Google Scholar 

  3. Ferreira WD, Ferreira CB, Cruz Júnior G, Soares F (2020) A review of digital image forensics. Comput Electrical Eng 85:106685

    Article  Google Scholar 

  4. Kaur G, Singh N, Kumar M (2023) Image forgery techniques: a review. Artif Intell Rev 56(2):1577–1625

    Article  Google Scholar 

  5. Liu Q, Cao X, Deng C, Guo X (2011) Identifying image composites through shadow matte consistency. IEEE Trans Inf Forensics Secur 6(3):1111–1122

    Article  Google Scholar 

  6. Itier V, Strauss O, Morel L, Puech W (2021) Color noise correlation-based splicing detection for image forensics. Multimedia Tools Appl 80:13215–13233

    Article  Google Scholar 

  7. Jaiswal AK, Srivastava R (2021) Forensic image analysis using inconsistent noise pattern. Pattern Anal Appl 24:655–667

    Article  Google Scholar 

  8. Ferrara P, Bianchi T, De Rosa A, Piva A (2012) Image forgery localization via fine-grained analysis of cfa artifacts. IEEE Trans Inf Forensics Secur 7(5):1566–1577

    Article  Google Scholar 

  9. Bianchi T, Piva A (2011) Detection of nonaligned double jpeg compression based on integer periodicity maps. IEEE Trans Inf Forensics Secur 7(2):842–848

    Article  Google Scholar 

  10. Conotter V, Comesana P, Perez-Gonzalez F (2015) Forensic detection of processing operator chains: Recovering the history of filtered jpeg images. IEEE Trans Inf Forensics Secur 10(11):2257–2269

    Article  Google Scholar 

  11. Cozzolino D, Verdoliva L (2019) Noiseprint: A cnn-based camera model fingerprint. IEEE Trans Inf Forensics Secur 15:144–159

    Article  Google Scholar 

  12. Luo W, Huang J, Qiu G (2010) Jpeg error analysis and its applications to digital image forensics. IEEE Trans Inf Forensics Secur 5(3):480–491

    Article  Google Scholar 

  13. Amerini I, Ballan L, Caldelli R, Del Bimbo A, Serra G (2011) A sift-based forensic method for copy-move attack detection and transformation recovery. IEEE Trans Inf Forensics Secur 6(3):1099–1110

    Article  Google Scholar 

  14. Bellavia F, Fanfani M, Colombo C, Piva A (2021) Experiencing with electronic image stabilization and prnu through scene content image registration. Pattern Recogn Lett 145:8–15

    Article  Google Scholar 

  15. Cozzolino D, Marra F, Gragnaniello D, Poggi G, Verdoliva L (2020) Combining prnu and noiseprint for robust and efficient device source identification. EURASIP J Inf Secur 2020(1):1–12

    Article  Google Scholar 

  16. Liu K, Li J, Hussain Bukhari SS (2022) Overview of image inpainting and forensic technology. Security Commun Netw 2022

  17. Avidan S, Shamir A (2023) Seam carving for content-aware image resizing. Seminal Graphics Papers: Pushing the Boundaries 2:609–617

    Google Scholar 

  18. Bayram S, Sencar HT, Memon N (2009) An efficient androbust method for detecting copy-move forgery. Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, pp 1053–1056

    Google Scholar 

  19. Wang Q, Zhang R (2016) Double jpeg compression forensics based on a convolutional neural network. EURASIP J Inf Secur 2016(1):1–12

    Google Scholar 

  20. Zhou P, Han X, Morariu VI, Davis LS (2018) Learning rich features for image manipulation detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1053–1061

  21. Wu Y, AbdAlmageed W, Natarajan P (2019) Mantra-net: Manipulation tracing network for detection and localization of image forgeries with anomalous features. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9543–9552

  22. Hu X, Zhang Z, Jiang Z, Chaudhuri S, Yang Z, Nevatia R (2020) Span: Spatial pyramid attention network for image manipulation localization. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, 312–328

  23. Niloy FF, Bhaumik KK, Woo SS (2023) Cfl-net: Image forgery localization using contrastive learning. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 4642–4651

  24. Das S, Islam M, Amin M, et al. (2022) Gca-net: utilizing gated context attention for improving image forgery localization and detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 81–90

  25. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inform Process Syst 30:5998–6008

  26. Hao J, Zhang Z, Yang S, Xie D, Pu S (2021) Transforensics: image forgery localization with dense self-attention. Proceedings of the IEEE/CVF International Conference on Computer Vision 15055–15064

  27. Zhuo L, Tan S, Li B, Huang J (2022) Self-adversarial training incorporating forgery attention for image forgery localization. IEEE Trans Inf Forensics Secur 17:819–834

    Article  Google Scholar 

  28. Milletari F, Navab N, Ahmadi S-A (2016) V-net: Fully convolutional neural networks for volumetric medical image segmentation. Proceedings of the 2016 Fourth International Conference on 3D Vision, 565–571

  29. Shrivastava A, Gupta A, Girshick R (2016) Training region-based object detectors with online hard example mining. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 761–769

  30. Lin T-Y, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision 2980–2988

  31. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, 234–241

  32. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 7132–7141

  33. Dong J, Wang W, Tan T (2013) Casia image tampering detection evaluation database. Proceedings of the 2013 IEEE China Summit and International Conference on Signal and Information Processing, 422–426

  34. Guan H, Kozak M, Robertson E, Lee Y, Yates AN, Delgado A, Zhou D, Kheyrkhah T, Smith J, Fiscus J (2019) Mfc datasets: Large-scale benchmark datasets for media forensic challenge evaluation. Proceedings of the 2019 IEEE Winter Applications of Computer Vision Workshops, 63–72

  35. Hsu Y-F, Chang S-F (2006) Detecting image splicing using geometry invariants and camera characteristics consistency. Proceedings of the 2006 IEEE International Conference on Multimedia and Expo, 549–552

  36. Bappy JH, Simons C, Nataraj L, Manjunath B, Roy-Chowdhury AK (2019) Hybrid lstm and encoder-decoder architecture for detection of image forgeries. IEEE Trans Image Process 28(7):3286–3300

    Article  MathSciNet  Google Scholar 

  37. Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: Common objects in context. Proceedings of the Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, 740–755

  38. Krawetz N, Solutions HF (2007) A picture’s worth. Hacker Factor. Solutions 6(2):2

    Google Scholar 

  39. Mahdian B, Saic S (2009) Using noise inconsistencies for blind image forensics. Image Vis Comput 27(10):1497–1503

    Article  Google Scholar 

  40. Wu H, Zhou J, Tian J, Liu J, Qiao Y (2022) Robust image forgery detection against transmission over online social networks. IEEE Trans Inf Forensics Secur 17:443–456

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China(Grant No.62072106), Fujian Natural Science Foundation (Grant No.2020J01168, 2022J01188 and 2022J0119), University-industry Cooperation Project of Fujian Science and Technology Department(Grant No.2021H6004) and Educational Research Project for Young and Middle aged Teachers of Fujian Provincial Department of Education(No.JAT210053 and JAT210051).

Author information

Authors and Affiliations

  1. College of Computer and Cyber Security, Fujian Normal University, Fuzhou, 350000, China

    Aokun Zheng, Tianqiang Huang, Liqing Huang, Feng Ye & Haifeng Luo

  2. Digital Fujian Institute of Big Data Security Technology, Fuzhou, 350000, China

    Aokun Zheng, Tianqiang Huang, Liqing Huang, Feng Ye & Haifeng Luo

  3. Fujian Provincial Engineering Research Center of Big Data Analysis and Application, Fuzhou, 350000, China

    Aokun Zheng, Tianqiang Huang, Liqing Huang, Feng Ye & Haifeng Luo

  4. Fuzhou University, Fuzhou, 350000, China

    Wei Huang

Authors
  1. Aokun Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tianqiang Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liqing Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Feng Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Haifeng Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Tianqiang Huang or Liqing Huang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zheng, A., Huang, T., Huang, W. et al. DSSE-net: dual stream skip edge-enhanced network with forgery loss for image forgery localization. Int. J. Mach. Learn. & Cyber. 15, 2323–2335 (2024). https://doi.org/10.1007/s13042-023-02031-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-02031-0

Keywords

Search

Navigation