Investigating Translation Invariance and Shiftability in CNNs for Robust Multimedia Forensics: A JPEG Case Study
Authors: 
Edoardo Daniele Cannas, Sara Mandelli, Paolo Bestagini, Stefano TubaroAuthors Info & Claims
Pages 53 - 63
Abstract
Convolutional Neural Networks (CNNs) have been the state of the art in many applications, including computer vision and multimedia forensics. Translation invariance is often included among the reasons for their success. However, the recent literature has shown that this characteristic does not always hold, proving that CNNs are instead sensitive to small input translations and rotations. This phenomenon has been demonstrated for standard computer vision tasks like object classification, but the multimedia forensics literature has never investigated it. Forensic footprints are usually more subtle and prone to be conceived by post-processing operations, however showing other appealing properties, like periodicity patterns that can be exploited by analysts to deal with forensic tasks. An example is provided by JPEG compression, whose spatial periodicity is a clue for reconstructing the lifecycle of digital pictures. In this paper, we show that the translation invariance properties of CNNs are in strict relation with the intrinsic periodicity of input data, exploring the particular case of JPEG compressed images. Specifically, we test how CNNs change their behavior when processing compressed images whose pixels are misaligned with respect to the standard 8x8 JPEG grid and investigate solutions to mitigate these changes. Our results highlight some interesting relations between the properties of JPEG and CNNs' hyperparameters, like the stride of the first convolutional layer.
References
[1]
Shruti Agarwal and Hany Farid. 2017. Photo forensics from JPEG dimples. In 2017 IEEE Workshop on Information Forensics and Security, WIFS 2017, Rennes, France, December 4--7, 2017. IEEE, 1--6. https://doi.org/10.1109/WIFS.2017.8267641
[2]
W. Ahn, S.?H. Nam, M. Son, H.?K. Lee, and S. Choi. 2020. End?to?end double JPEG detection with a 3D convolutional network in the DCT domain., Vol. 56 (2020), 82--85. https://doi.org/10.1049/el.2019.2719
[3]
Aharon Azulay and Yair Weiss. 2019. Why do deep convolutional networks generalize so poorly to small image transformations? J. Mach. Learn. Res., Vol. 20 (2019), 184:1--184:25. http://jmlr.org/papers/v20/19--519.html
[4]
Mauro Barni, Luca Bondi, Nicolò Bonettini, Paolo Bestagini, Andrea Costanzo, Marco Maggini, Benedetta Tondi, and Stefano Tubaro. 2017. Aligned and Non-Aligned Double JPEG Detection Using Convolutional Neural Networks. CoRR, Vol. abs/1708.00930 (2017). arxiv: 1708.00930 http://arxiv.org/abs/1708.00930
[5]
Mauro Barni, Andrea Costanzo, and Lara Sabatini. 2010. Identification of cut & paste tampering by means of double-JPEG detection and image segmentation. In International Symposium on Circuits and Systems (ISCAS 2010), May 30 - June 2, 2010, Paris, France. IEEE, 1687--1690. https://doi.org/10.1109/ISCAS.2010.5537505
[6]
Sebastiano Battiato, Oliver Giudice, Francesco Guarnera, and Giovanni Puglisi. 2022. CNN-based first quantization estimation of double compressed JPEG images. J. Vis. Comun. Image Represent., Vol. 89, C (nov 2022), bibinfonumpages12 pages. https://doi.org/10.1016/j.jvcir.2022.103635
[7]
Tiziano Bianchi and Alessandro Piva. 2012. Detection of Nonaligned Double JPEG Compression Based on Integer Periodicity Maps. IEEE Trans. Inf. Forensics Secur., Vol. 7, 2 (2012), 842--848. https://doi.org/10.1109/TIFS.2011.2170836
[8]
L. Bondi, L. Baroffio, David Guera, Paolo Bestagini, E. Delp, and S. Tubaro. 2016. First Steps Toward Camera Model Identification With Convolutional Neural Networks. (2016). https://doi.org/10.1109/LSP.2016.2641006 arxiv: 1603.01068
[9]
Luca Bondi, Edoardo Daniele Cannas, Paolo Bestagini, and Stefano Tubaro. 2020. Training Strategies and Data Augmentations in CNN-based DeepFake Video Detection. 2020 IEEE International Workshop on Information Forensics and Security (WIFS), 1--6. https://doi.org/10.1109/WIFS49906.2020.9360901
[10]
Nicolò Bonettini, Luca Bondi, Paolo Bestagini, and Stefano Tubaro. 2018. JPEG Implementation Forensics Based on Eigen-Algorithms. In 2018 IEEE International Workshop on Information Forensics and Security, WIFS 2018, Hong Kong, China, December 11--13, 2018. IEEE, 1--7. https://doi.org/10.1109/WIFS.2018.8630765
[11]
Mehdi Boroumand, Mo Chen, and Jessica J. Fridrich. 2019. Deep Residual Network for Steganalysis of Digital Images. IEEE Trans. Inf. Forensics Secur., Vol. 14, 5 (2019), 1181--1193. https://doi.org/10.1109/TIFS.2018.2871749
[12]
Mo Chen, Vahid Sedighi, Mehdi Boroumand, and Jessica Fridrich. 2017. JPEG-Phase-Aware Convolutional Neural Network for Steganalysis of JPEG Images. In Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security (Philadelphia, Pennsylvania, USA) (IH&MMSec '17). Association for Computing Machinery, New York, NY, USA, 75--84. https://doi.org/10.1145/3082031.3083248
[13]
Riccardo Corvi, Davide Cozzolino, Giada Zingarini, Giovanni Poggi, Koki Nagano, and Luisa Verdoliva. 2023. On the detection of synthetic images generated by diffusion models. In ICASSP 2023--2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1--5.
[14]
Ricard Durall, Margret Keuper, and Janis Keuper. 2020. Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributions. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[15]
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. 2019. Exploring the Landscape of Spatial Robustness. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 1802--1811. https://proceedings.mlr.press/v97/engstrom19a.html
[16]
ESRI. 2023. Imagery formats and performance. https://doc.arcgis.com/en/imagery/workflows/best-practices/imagery-formats-and-performance.htm
[17]
Zhigang Fan and Ricardo L. de Queiroz. 2003. Identification of bitmap compression history: JPEG detection and quantizer estimation. IEEE Trans. Image Process., Vol. 12, 2 (2003), 230--235. https://doi.org/10.1109/TIP.2002.807361
[18]
Hany Farid. 2009. Exposing digital forgeries from JPEG ghosts. IEEE Trans. Inf. Forensics Secur., Vol. 4, 1 (2009), 154--160. https://doi.org/10.1109/TIFS.2008.2012215
[19]
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. http://www.deeplearningbook.org.
[20]
D. Gragnaniello, D. Cozzolino, F. Marra, G. Poggi, and L. Verdoliva. 2021. Are GAN Generated Images Easy to Detect? A Critical Analysis of the State-Of-The-Art. In 2021 IEEE International Conference on Multimedia and Expo (ICME). 1--6. https://doi.org/10.1109/ICME51207.2021.9428429
[21]
Fengxiang He, Tongliang Liu, and Dacheng Tao. 2020. Why ResNet Works? Residuals Generalize. IEEE Transactions on Neural Networks and Learning Systems, Vol. 31, 12 (2020), 5349--5362. https://doi.org/10.1109/TNNLS.2020.2966319
[22]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 770--778. https://doi.org/10.1109/CVPR.2016.90
[23]
Gao Huang, Zhuang Liu, and Kilian Q. Weinberger. 2016. Densely Connected Convolutional Networks. (2016). https://doi.org/10.1109/CVPR.2017.243 arxiv: 1608.06993
[24]
Graham Hudson, Alain Léger, Birger Niss, and István Sebestyén. 2017. JPEG at 25: Still Going Strong. IEEE MultiMedia, Vol. 24, 2 (2017), 96--103. https://doi.org/10.1109/MMUL.2017.38
[25]
Stanis?aw Jastrz?bski, Devansh Arpit, Nicolas Ballas, Vikas Verma, Tong Che, and Yoshua Bengio. 2017. Residual Connections Encourage Iterative Inference. (Oct. 2017). https://doi.org/10.48550/ARXIV.1710.04773 arxiv: 1710.04773 [cs.CV]
[26]
JPEG. 2024. JPEG Reference. https://jpegclub.org/reference/reference-sources/
[27]
Matthias Kirchner. 2008. Fast and reliable resampling detection by spectral analysis of fixed linear predictor residue. In Proceedings of the 10th ACM workshop on Multimedia and security. 11--20.
[28]
Chang Liu and Matthias Kirchner. 2019. CNN-based rescaling factor estimation. In Proceedings of the ACM Workshop on Information Hiding and Multimedia Security. 119--124.
[29]
Sara Mandelli, Nicolò Bonettini, Paolo Bestagini, Vincenzo Lipari, and Stefano Tubaro. 2018. Multiple Jpeg Compression Detection Through Task-Driven Non-Negative Matrix Factorization. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2106--2110. https://doi.org/10.1109/ICASSP.2018.8461904
[30]
Sara Mandelli, Nicolò Bonettini, Paolo Bestagini, and Stefano Tubaro. 2020. Training CNNs in Presence of JPEG Compression: Multimedia Forensics vs Computer Vision. In 2020 IEEE International Workshop on Information Forensics and Security (WIFS). 1--6. https://doi.org/10.1109/WIFS49906.2020.9360903
[31]
Simone Milani, Marco Tagliasacchi, and Stefano Tubaro. 2012. Discriminating multiple JPEG compression using first digit features. 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2253--2256. https://doi.org/10.1109/ICASSP.2012.6288362
[32]
Coenraad Mouton, Johannes C. Myburgh, and Marelie H. Davel. 2021. Stride and Translation Invariance in CNNs. Communications in Computer and Information Science (LNCS sub-series CCIS), vol 1342, pp 267--281, 2020 (March 2021), 267--281. https://doi.org/10.1007/978--3-030--66151--9_17 arxiv: 2103.10097 [cs.LG]
[33]
Cecilia Pasquini, Giulia Boato, and Fernando Pé rez-Gonzá lez. 2014. Multiple JPEG compression detection by means of Benford-Fourier coefficients. In 2014 IEEE International Workshop on Information Forensics and Security, WIFS 2014, Atlanta, GA, USA, December 3--5, 2014. IEEE, 113--118. https://doi.org/10.1109/WIFS.2014.7084313
[34]
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in PyTorch. (2017).
[35]
A. Piva. 2013. An Overview on Image Forensics. ISRN Signal Processing (2013).
[36]
Alin C Popescu and Hany Farid. 2005. Exposing digital forgeries by detecting traces of resampling. IEEE Transactions on signal processing, Vol. 53, 2 (2005), 758--767.
[37]
Xuan Qi, Yi Wei, Xue Mei, Ryad Chellali, and Shipin Yang. 2023. Empirical Study on the Effect of Residual Networks on the Expressiveness of Linear Regions. In Artificial Neural Networks and Machine Learning -- ICANN 2023, Lazaros Iliadis, Antonios Papaleonidas, Plamen Angelov, and Chrisina Jayne (Eds.). Springer Nature Switzerland, Cham, 174--185.
[38]
Gerald Schaefer and Michal Stich. 2003. UCID: an uncompressed color image database. In Storage and Retrieval Methods and Applications for Multimedia 2004 (Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 5307), Minerva M. Yeung, Rainer W. Lienhart, and Chung-Sheng Li (Eds.). 472--480. https://doi.org/10.1117/12.525375
[39]
Selim Seferbekov. 2020. Deep Fake Detection Challenge Winning Solution. https://github.com/selimsef/dfdc_deepfake_challenge
[40]
Eero P. Simoncelli, William T. Freeman, Edward H. Adelson, and David J. Heeger. 1992. Shiftable multiscale transforms. IEEE Trans. Inf. Theory, Vol. 38, 2 (1992), 587--607. https://doi.org/10.1109/18.119725
[41]
K. Simonyan and Andrew Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. (2014). arxiv: 1409.1556 https://www.semanticscholar.org/paper/eb42cf88027de515750f230b23b1a057dc782108
[42]
Luisa Verdoliva. 2020. Media Forensics and DeepFakes: an overview. IEEE Journal of Selected Topics in Signal Processing (2020).
[43]
Vinay Verma, Nikita Agarwal, and Nitin Khanna. 2018. DCT-domain deep convolutional neural networks for multiple JPEG compression classification. Signal Processing: Image Communication, Vol. 67 (2018), 22--33. https://doi.org/10.1016/j.image.2018.04.014
[44]
Guanshuo Xu. 2017. Deep Convolutional Neural Network to Detect J-UNIWARD. In Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security (Philadelphia, Pennsylvania, USA) (IH&MMSec '17). Association for Computing Machinery, New York, NY, USA, 67--73. https://doi.org/10.1145/3082031.3083236
[45]
Richard Zhang. 2019. Making Convolutional Networks Shift-Invariant Again. (April 2019). https://doi.org/10.48550/ARXIV.1904.11486 arxiv: 1904.11486 [cs.CV]
[46]
Xu Zhang, Svebor Karaman, and Shih-Fu Chang. 2019. Detecting and simulating artifacts in gan fake images. In 2019 IEEE international workshop on information forensics and security (WIFS). IEEE, 1--6. io
Index Terms
- Investigating Translation Invariance and Shiftability in CNNs for Robust Multimedia Forensics: A JPEG Case Study
Recommendations
Measuring Translation Shiftability of Frames
The shiftability property has been shown to be advantageous in certain signal processing tasks, such as feature extraction, and thus, practical shiftability measures are needed. In this study, translation shiftability measures for frames of regular ...
On Translation Invariant Subspaces and Critically SampledWavelet Transforms
The discrete wavelet transform (DWT) is attractive for many reasons. Its sparse sampling grid eliminates redundancy and is very efficient. Its localized basis functions are well suited for processing non–stationary signals such as transients. On the other ...
Deep Learning Approach to the Quality Restoration for JPEG Images
ICCCM '22: Proceedings of the 10th International Conference on Computer and Communications ManagementThe JPEG codec is still the dominant and common image compression format despite the continuous development of new image compression techniques. In order to raise the compression ratio, distortion caused by some quantization operation within the ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
June 2024
305 pages
ISBN:9798400706370
DOI:10.1145/3658664
- General Chair:
- Fernando Pérez-González,
- Program Chairs:
- Pedro Comesaña-Alfaro,
- Christian Krätzer,
- Hong Vicky Zhao
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Defense Advanced Research Projects Agency (DARPA) and the Air Force Research Laboratory (AFRL)
Conference
IH&MMSEC '24
Sponsor:
IH&MMSEC '24: ACM Workshop on Information Hiding and Multimedia Security
June 24 - 26, 2024
Baiona, Spain
Acceptance Rates
Overall Acceptance Rate 128 of 318 submissions, 40%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 123Total Downloads
- Downloads (Last 12 months)123
- Downloads (Last 6 weeks)27
Reflects downloads up to 15 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in