More Web Proxy on the site http://driver.im/

research-article

Proactive image manipulation detection via deep semi-fragile watermark

Authors:

Wanlei ZhouAuthors Info & Claims

Volume 585, Issue C

https://doi.org/10.1016/j.neucom.2024.127593

Published: 17 July 2024 Publication History

Abstract

Malicious image tampering refers to intentionally manipulating images to make them harmful to the owners or users. It has become one of the most severe challenges to image authenticity. Conventional methods for detecting tampering by identifying visual artifacts and distortions have limitations due to the rapid advancement of image manipulation techniques, which leave fewer detectable traces. To address these challenges, we propose a proactive media authentication method using deep learning-based semi-fragile watermarks. The designed scheme utilizes deep neural networks to embed an invisible watermark into a target image that is pixel-by-pixel entangled with it, which acts as an indicator of tampering trails. Once the watermarked image is counterfeited, the embedded watermark will exhibit changes accordingly, so we can locate the tampered regions by comparing retrieved and original watermarks. This proactive authentication mechanism makes our method effective against various image tamper techniques, including image copy&move, splicing and in-painting. Although our watermark is designed to be fragile to malicious tampering operations, it remains robust to benign image-processing operations such as JPEG compression, scaling, saturation, contrast adjustments, etc. This design enables our watermark to retain effectiveness when shared over the internet. Extensive experiments demonstrate that our method achieves state-of-the-art forgery detection with superior robustness, imperceptibility and security performance.

References

[1]

Hu X., Zhang Z., Jiang Z., Chaudhuri S., Yang Z., Nevatia R., SPAN: Spatial pyramid attention network for image manipulation localization, in: European Conference on Computer Vision, Springer, 2020, pp. 312–328.

[2]

H. Li, J. Huang, Localization of deep inpainting using high-pass fully convolutional network, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 8301–8310.

[3]

Y. Wu, W. AbdAlmageed, P. Natarajan, Mantra-net: Manipulation tracing network for detection and localization of image forgeries with anomalous features, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 9543–9552.

[4]

Salloum R., Ren Y., Kuo C.-C.J., Image splicing localization using a multi-task fully convolutional network (MFCN), J. Vis. Commun. Image Represent. 51 (2018) 201–209.

[5]

P. Zhou, B.-C. Chen, X. Han, M. Najibi, A. Shrivastava, S.-N. Lim, L. Davis, Generate, segment, and refine: Towards generic manipulation segmentation, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, 2020, pp. 13058–13065.

[6]

V. Asnani, X. Yin, T. Hassner, S. Liu, X. Liu, Proactive Image Manipulation Detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 15386–15395.

[7]

R. Wang, F. Juefei-Xu, M. Luo, Y. Liu, L. Wang, Faketagger: Robust safeguards against deepfake dissemination via provenance tracking, in: Proceedings of the 29th ACM International Conference on Multimedia, 2021, pp. 3546–3555.

[8]

N. Yu, V. Skripniuk, S. Abdelnabi, M. Fritz, Artificial fingerprinting for generative models: Rooting deepfake attribution in training data, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 14448–14457.

[9]

Di Martino F., Sessa S., Fragile watermarking tamper detection via bilinear fuzzy relation equations, J. Ambient Intell. Humaniz. Comput. 10 (2019) 2041–2061.

[10]

Bhalerao S., Ansari I.A., Kumar A., A secure image watermarking for tamper detection and localization, J. Ambient Intell. Humaniz. Comput. 12 (1) (2021) 1057–1068.

[11]

Cox I.J., Kilian J., Leighton F.T., Shamoon T., Secure spread spectrum watermarking for multimedia, IEEE Trans. Image Process. 6 (12) (1997) 1673–1687.

[12]

Pereira S., Ruanaidh J.J.O., Deguillaume F., Csurka G., Pun T., Template based recovery of Fourier-based watermarks using log-polar and log-log maps, in: Proceedings IEEE International Conference on Multimedia Computing and Systems, Vol. 1, IEEE, 1999, pp. 870–874.

[13]

Bi N., Sun Q., Huang D., Yang Z., Huang J., Robust image watermarking based on multiband wavelets and empirical mode decomposition, IEEE Trans. Image Process. 16 (8) (2007) 1956–1966.

[14]

Shehab A., Elhoseny M., Muhammad K., Sangaiah A.K., Yang P., Huang H., Hou G., Secure and robust fragile watermarking scheme for medical images, IEEE Access 6 (2018) 10269–10278.

[15]

Pereira S., Pun T., Robust template matching for affine resistant image watermarks, IEEE Trans. Image Process. 9 (6) (2000) 1123–1129.

[16]

Lin E.T., Podilchuk C.I., Delp III E.J., Detection of image alterations using semifragile watermarks, Security and Watermarking of Multimedia Contents II, vol. 3971, SPIE, 2000, pp. 152–163.

[17]

Sun R., Sun H., Yao T., A SVD-and quantization based semi-fragile watermarking technique for image authentication, in: 6th International Conference on Signal Processing, 2002, Vol. 2, IEEE, 2002, pp. 1592–1595.

[18]

Yu X., Wang C., Zhou X., Review on semi-fragile watermarking algorithms for content authentication of digital images, Future Internet 9 (4) (2017) 56.

[19]

Cozzolino D., Gragnaniello D., Verdoliva L., Image forgery localization through the fusion of camera-based, feature-based and pixel-based techniques, in: 2014 IEEE International Conference on Image Processing, ICIP, IEEE, 2014, pp. 5302–5306.

[20]

Ferrara P., Bianchi T., De Rosa A., Piva A., Image forgery localization via fine-grained analysis of CFA artifacts, IEEE Trans. Inf. Forensics Secur. 7 (5) (2012) 1566–1577.

Digital Library

[21]

De Carvalho T.J., Riess C., Angelopoulou E., Pedrini H., de Rezende Rocha A., Exposing digital image forgeries by illumination color classification, IEEE Trans. Inf. Forensics Secur. 8 (7) (2013) 1182–1194.

[22]

A. Islam, C. Long, A. Basharat, A. Hoogs, Doa-gan: Dual-order attentive generative adversarial network for image copy-move forgery detection and localization, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 4676–4685.

[23]

Zhang Y., Goh J., Win L.L., Thing V.L., Image region forgery detection: A deep learning approach., SG-CRC 2016 (2016) 1–11.

[24]

B. Bayar, M.C. Stamm, A deep learning approach to universal image manipulation detection using a new convolutional layer, in: Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security, 2016, pp. 5–10.

[25]

Dong C., Chen X., Hu R., Cao J., Li X., MVSS-Net: Multi-view multi-scale supervised networks for image manipulation detection, IEEE Trans. Pattern Anal. Mach. Intell. (2022).

[26]

Tamimi A.A., Abdalla A.M., Al-Allaf O., Hiding an image inside another image using variable-rate steganography, Int. J. Adv. Comput. Sci. Appl. (IJACSA) 4 (10) (2013).

[27]

Ruanaidh J., Dowling W., Boland F.M., Phase watermarking of digital images, in: Proceedings of 3rd IEEE International Conference on Image Processing, Vol. 3, IEEE, 1996, pp. 239–242.

[28]

Hsu C.-T., Wu J.-L., Hidden digital watermarks in images, IEEE Trans. Image Process. 8 (1) (1999) 58–68.

[29]

Barni M., Bartolini F., Piva A., Improved wavelet-based watermarking through pixel-wise masking, IEEE Trans. Image Process. 10 (5) (2001) 783–791.

[30]

Provos N., Honeyman P., Hide and seek: An introduction to steganography, IEEE Secur. Priv. 1 (3) (2003) 32–44.

[31]

J. Zhu, R. Kaplan, J. Johnson, L. Fei-Fei, Hidden: Hiding data with deep networks, in: Proceedings of the European Conference on Computer Vision, ECCV, 2018, pp. 657–672.

[32]

Mun S.-M., Nam S.-H., Jang H.-U., Kim D., Lee H.-K., A robust blind watermarking using convolutional neural network, 2017, arXiv preprint arXiv:1704.03248.

[33]

Baluja S., Hiding images in plain sight: Deep steganography, Adv. Neural Inf. Process. Syst. 30 (2017).

[34]

Zhang R., Dong S., Liu J., Invisible steganography via generative adversarial networks, Multimedia Tools Appl. 78 (7) (2019) 8559–8575.

[35]

Zhang C., Benz P., Karjauv A., Sun G., Kweon I.S., Udh: Universal deep hiding for steganography, watermarking, and light field messaging, Adv. Neural Inf. Process. Syst. 33 (2020) 10223–10234.

[36]

X. Luo, R. Zhan, H. Chang, F. Yang, P. Milanfar, Distortion agnostic deep watermarking, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 13548–13557.

[37]

Liao X., Yin J., Chen M., Qin Z., Adaptive payload distribution in multiple images steganography based on image texture features, IEEE Trans. Dependable Secure Comput. (2020).

[38]

J. Jing, X. Deng, M. Xu, J. Wang, Z. Guan, HiNet: deep image hiding by invertible network, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 4733–4742.

[39]

Neekhara P., Hussain S., Zhang X., Huang K., McAuley J., Koushanfar F., FaceSigns: semi-fragile neural watermarks for media authentication and countering deepfakes, 2022, arXiv preprint arXiv:2204.01960.

[40]

C. Zhang, A. Karjauv, P. Benz, I.S. Kweon, Towards robust deep hiding under non-differentiable distortions for practical blind watermarking, in: Proceedings of the 29th ACM International Conference on Multimedia, 2021, pp. 5158–5166.

[41]

Q. Fan, J. Yang, G. Hua, B. Chen, D. Wipf, A generic deep architecture for single image reflection removal and image smoothing, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 3238–3247.

[42]

T. Karras, S. Laine, T. Aila, A style-based generator architecture for generative adversarial networks, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 4401–4410.

[43]

Karras T., Aila T., Laine S., Lehtinen J., Progressive growing of gans for improved quality, stability, and variation, 2017, arXiv preprint arXiv:1710.10196.

[44]

Dong J., Wang W., Tan T., Casia image tampering detection evaluation database, in: 2013 IEEE China Summit and International Conference on Signal and Information Processing, IEEE, 2013, pp. 422–426.

[45]

Ng T.-T., Hsu J., Chang S.-F., Columbia Image Splicing Detection Evaluation Dataset, DVMM lab. Columbia Univ CalPhotos Digit Libr, 2009.

[46]

Lin T.-Y., Maire M., Belongie S., Hays J., Perona P., Ramanan D., Dollár P., Zitnick C.L., Microsoft coco: Common objects in context, in: European Conference on Computer Vision, Springer, 2014, pp. 740–755.

[47]

X. Weng, Y. Li, L. Chi, Y. Mu, High-capacity convolutional video steganography with temporal residual modeling, in: Proceedings of the 2019 on International Conference on Multimedia Retrieval, 2019, pp. 87–95.

[48]

Liu C., Chen H., Zhu T., Zhang J., Zhou W., Making DeepFakes more spurious: evading deep face forgery detection via trace removal attack, 2022, arXiv preprint arXiv:2203.11433.

[49]

R. Durall, M. Keuper, J. Keuper, Watch your up-convolution: Cnn based generative deep neural networks are failing to reproduce spectral distributions, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 7890–7899.

Cited By

Xing FZhou XPeng HFan XHan WZhang Y(2024)TAE-RWPInternational Journal of Intelligent Systems10.1155/2024/60541722024Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1155/2024/6054172
Wang TZhang YQi SZhao RXia ZWeng J(2024)Security and Privacy on Generative Data in AIGC: A SurveyACM Computing Surveys10.1145/370362657:4(1-34)Online publication date: 10-Dec-2024
https://dl.acm.org/doi/10.1145/3703626
Nadimpalli ARattani A(2024)Social Media Authentication and Combating Deepfakes Using Semi-Fragile Invisible Image WatermarkingDigital Threats: Research and Practice10.1145/37001465:4(1-30)Online publication date: 12-Oct-2024
https://dl.acm.org/doi/10.1145/3700146

Recommendations

A Fragile Watermarking Scheme for Image Authentication with Tamper Detection and Localization
ICGEC '10: Proceedings of the 2010 Fourth International Conference on Genetic and Evolutionary Computing

In this paper, we develop a watermarking scheme for image authentication and verification. The watermarking scheme for tamper detection of an image is fragile and block-based. Such a scheme has good tamper localization and high tamper detection rate. ...
A Novel Semi-fragile Digital Watermarking Scheme for Scrambled Image Authentication and Restoration
Abstract
More than ever, the practical and accurate watermarking technologies are called for the growing amount of exchanged digital image over the Internet. To protect the integrity and authenticity of digital image and to enhance the effect of tamper ...
An effective fragile watermarking scheme for color image tampering detection and self-recovery
Abstract
In this paper, a fragile watermarking scheme for color-image authentication and self-recovery is proposed. Original image is divided into non-overlapping blocks, and for each i-th block, the watermarks used for recovery and ...
Highlights
- High quality of the watermarked image in comparison with other schemes was confirmed.

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Neurocomputing

Neurocomputing Volume 585, Issue C

Jun 2024

254 pages

Issue’s Table of Contents

The Authors.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 17 July 2024

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xing FZhou XPeng HFan XHan WZhang Y(2024)TAE-RWPInternational Journal of Intelligent Systems10.1155/2024/60541722024Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1155/2024/6054172
Wang TZhang YQi SZhao RXia ZWeng J(2024)Security and Privacy on Generative Data in AIGC: A SurveyACM Computing Surveys10.1145/370362657:4(1-34)Online publication date: 10-Dec-2024
https://dl.acm.org/doi/10.1145/3703626
Nadimpalli ARattani A(2024)Social Media Authentication and Combating Deepfakes Using Semi-Fragile Invisible Image WatermarkingDigital Threats: Research and Practice10.1145/37001465:4(1-30)Online publication date: 12-Oct-2024
https://dl.acm.org/doi/10.1145/3700146

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents