Cited By
View all- Liu CJin YGuan ZLi TQin YQian BJiang ZWu YWang XZheng YZeng D(2024)Visual–language foundation models in medicineThe Visual Computer10.1007/s00371-024-03579-wOnline publication date: 29-Jul-2024
Recoverable Privacy-Preserving Image Classification through Noise-like Adversarial Examples
Authors: 
Jun Liu, Jiantao Zhou, Jinyu Tian, Weiwei SunAuthors Info & Claims
Article No.: 216, Pages 1 - 27
Abstract
With the increasing prevalence of cloud computing platforms, ensuring data privacy during the cloud-based image-related services such as classification has become crucial. In this study, we propose a novel privacy-preserving image classification scheme that enables the direct application of classifiers trained in the plaintext domain to classify encrypted images without the need of retraining a dedicated classifier. Moreover, encrypted images can be decrypted back into their original form with high fidelity (recoverable) using a secret key. Specifically, our proposed scheme involves utilizing a feature extractor and an encoder to mask the plaintext image through a newly designed Noise-like Adversarial Example (NAE). Such an NAE not only introduces a noise-like visual appearance to the encrypted image but also compels the target classifier to predict the ciphertext as the same label as the original plaintext image. At the decoding phase, we adopt a Symmetric Residual Learning (SRL) framework for restoring the plaintext image with minimal degradation. Extensive experiments demonstrate that (1) the classification accuracy of the classifier trained in the plaintext domain remains the same in both the ciphertext and plaintext domains; (2) the encrypted images can be recovered into their original form with an average PSNR of up to 51+ dB for the SVHN dataset and 48+ dB for the VGGFace2 dataset; (3) our system exhibits satisfactory generalization capability on the encryption, decryption, and classification tasks across datasets that are different from the training one; and (4) a high-level of security is achieved against three potential threat models. The code is available at https://github.com/csjunjun/RIC.git.
References
[1]
Jiarui Cai, Yizhou Wang, and Jenq-Neng Hwang. 2021. Ace: Ally complementary experts for solving long-tailed recognition in one-shot. In Proc. IEEE Conf. Comput. Vis. Pattern Recogn. 112–121.
[2]
Aaron Chen and Gus Smith. 2023. Pytorch-playground. Retrieved from https://github.com/aaron-xichen/pytorch-playground
[3]
Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016. Deep learning with differential privacy. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. 308–318.
[4]
Mohammad Al-Rubaie and J Morris Chang. 2019. Privacy-preserving machine learning: Threats and solutions. IEEE Secur. Privac. 17, 2 (2019), 49–58.
[5]
Sanghyeon An, Minjun Lee, Sanglee Park, Heerin Yang, and Jungmin So. 2020. An ensemble of simple convolutional neural network models for MNIST digit recognition. arXiv preprint arXiv:2008.10400 (2020).
[6]
Song Bian, Tianchen Wang, Masayuki Hiromoto, Yiyu Shi, and Takashi Sato. 2020. ENSEI: Efficient secure inference via frequency-domain homomorphic convolution for privacy-preserving visual recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.9403–9412.
[7]
Qiong Cao, Li Shen, Weidi Xie, Omkar M. Parkhi, and Andrew Zisserman. 2018. VGGFace2: A dataset for recognising faces across pose and age. In Proceedings of the International Conference on Automated Face and Gesture Recognition.67–74.
[8]
Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness of neural networks. In Proceedings of the IEEE Symposium on Security and Privacy.39–57.
[9]
Junxin Chen, Zhi-liang Zhu, Li-bo Zhang, Yushu Zhang, and Ben-qiang Yang. 2018. Exploiting self-adaptive permutation–diffusion and DNA random encoding for secure and efficient image encryption. Sig. Process. 142 (2018), 340–353.
[10]
Zhineng Chen, Shanshan Ai, and Caiyan Jia. 2019. Structure-aware deep learning for product image classification. ACM Trans. Multim. Comput. Commun. Appl. 15, 1s, Article 4 (2019), 20 pages.
[11]
Ka Leong Cheng, Yueqi Xie, and Qifeng Chen. 2021. IICNet: A generic framework for reversible image conversion. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 1991–2000.
[12]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 248–255.
[13]
Xiaofeng Ding, Hongbiao Fang, Zhilin Zhang, Kim-Kwang Raymond Choo, and Hai Jin. 2020. Privacy-preserving feature extraction via adversarial training. IEEE Trans. Knowl. Data Eng. 34, 4 (2020), 1967–1979.
[14]
Hao Dong, Chao Wu, Zhen Wei, and Yike Guo. 2017. Dropping activation outputs with localized first-layer deep network for enhancing user privacy and data security. IEEE Trans. Inf. Forens. Secur. 13, 3 (2017), 662–670.
[15]
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. 2018. Boosting adversarial attacks with momentum. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.9185–9193.
[16]
Jia Duan, Jiantao Zhou, and Yuanman Li. 2021. Secure and verifiable outsourcing of large-scale nonnegative matrix factorization (NMF). IEEE Trans. Serv. Comput. 14, 6 (2021), 1940–1953. DOI:
[17]
Cynthia Dwork. 2008. Differential privacy: A survey of results. In Proceedings of the International Conference on Theory and Applications of Models of Computation.
[18]
Craig Gentry. 2009. A Fully Homomorphic Encryption Scheme. Stanford University.
[19]
Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin Lauter, Michael Naehrig, and John Wernsing. 2016. CryptoNets: Applying neural networks to encrypted data with high throughput and accuracy. In Proceedings of the International Conference on Machine Learning.
[20]
Fattaneh Bayatbabolghani and Marina Blanton. 2018. Secure multi-party computation. In Proc. ACM SIGSAC Conf. on Comput. and Communi. Security, 2157–2159.
[21]
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In Proceedings of the International Conference on Learning Representations.1–11.
[22]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.770–778.
[23]
Yangsibo Huang, Zhao Song, Kai Li, and Sanjeev Arora. 2020. InstaHide: Instance-hiding schemes for private distributed learning. In Proceedings of the International Conference on Machine Learning.4507–4518.
[24]
Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. 2016. Binarized neural networks. Proceedings of the International Conference on Neural Information Processing Systems..
[25]
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. 2018. Black-box adversarial attacks with limited queries and information. In Proceedings of the International Conference on Machine Learning.2137–2146.
[26]
Jiazhen Ji, Huan Wang, Yuge Huang, Jiaxiang Wu, Xingkun Xu, Shouhong Ding, ShengChuan Zhang, Liujuan Cao, and Rongrong Ji. 2022. Privacy-preserving face recognition with learnable privacy budgets in frequency domain. In Proceedings of the European Conference on Computer Vision. 475–491.
[27]
Junpeng Jing, Xin Deng, Mai Xu, Jianyi Wang, and Zhenyu Guan. 2021. HiNet: Deep image hiding by invertible network. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4733–4742.
[28]
Neil F. Johnson and Sushil Jajodia. 1998. Exploring steganography: Seeing the unseen. Computer 31, 2 (1998), 26–34.
[29]
Chiraag Juvekar, Vinod Vaikuntanathan, and Anantha Chandrakasan. 2018. GAZELLE: A low latency framework for secure neural network inference. In Proceedings of the USENIX Security Symposium.1651–1669.
[30]
Harsh Kasyap and Somanath Tripathy. 2021. Privacy-preserving decentralized learning framework for healthcare system. ACM Trans. Multim. Comput. Commun. Appl. 17, 2s, Article 68 (2021), 24 pages.
[31]
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations.1–15.
[32]
Sotiris Kotsiantis, Dimitris Kanellopoulos, and Panayiotis Pintelas. 2006. Handling imbalanced datasets: A review. GESTS Int. Trans. Comput. Sci. Eng. 30, 1 (2006), 25–36.
[33]
Alex Krizhevsky and Geoffrey Hinton. 2009. Learning multiple layers of features from tiny images. Retrieved from https://www.cs.toronto.edu/kriz/learning-features-2009-TR.pdf
[34]
Jian Liu, Mika Juuti, Yao Lu, and Nadarajah Asokan. 2017. Oblivious neural network predictions via miniONN transformations. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. 619–631.
[35]
Ziyao Liu, Jiale Guo, Kwok-Yan Lam, and Jun Zhao. 2023. Efficient dropout-resilient aggregation for privacy-preserving machine learning. IEEE Trans. Inf. Forens. Secur. 18 (2023), 1839–1854.
[36]
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards deep learning models resistant to adversarial attacks. In Proceedings of the International Conference on Learning Representations.1–23.
[37]
Yuxi Mi, Yuge Huang, Jiazhen Ji, Hongquan Liu, Xingkun Xu, Shouhong Ding, and Shuigeng Zhou. 2022. DuetFace: Collaborative privacy-preserving face recognition via channel splitting in the frequency domain. In Proceedings of the 30th ACM International Conference on Multimedia.6755–6764.
[38]
Fatemehsadat Mireshghallah, Mohammadkazem Taram, Ali Jalali, Ahmed Taha Taha Elthakeb, Dean Tullsen, and Hadi Esmaeilzadeh. 2021. Not all features are equal: Discovering essential features for preserving prediction privacy. In Proceedings of the Web Conference.669–680.
[39]
Payman Mohassel and Yupeng Zhang. 2017. SecureML: A system for scalable privacy-preserving machine learning. In Proceedings of the IEEE Symposium on Security and Privacy. 19–38.
[40]
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. 2016. DeepFool: A simple and accurate method to fool deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2574–2582.
[41]
Aakash Varma Nadimpalli and Ajita Rattani. 2022. On improving cross-dataset generalization of deepfake detectors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.91–99.
[42]
Reyhaneh Neshatavar, Mohsen Yavartanoo, Sanghyun Son, and Kyoung Mu Lee. 2022. CVF-SID: Cyclic multi-variate function for self-supervised image denoising by disentangling noise from image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.17583–17591.
[43]
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y. Ng, and others. 2011. Reading digits in natural images with unsupervised feature learning. In NeurIPS Workshop on Deep Learning and Unsupervised Feature Learning, Granada, Spain, 7.
[44]
Seyed Ali Osia, Ali Shahin Shamsabadi, Sina Sajadmanesh, Ali Taheri, Kleomenis Katevas, Hamid R. Rabiee, Nicholas D. Lane, and Hamed Haddadi. 2020. A hybrid deep learning architecture for privacy-preserving mobile analytics. IEEE Internet Things J. 7, 5 (2020), 4505–4518.
[45]
Seyed Ali Osia, Ali Taheri, Ali Shahin Shamsabadi, Kleomenis Katevas, Hamed Haddadi, and Hamid R. Rabiee. 2020. Deep private-feature extraction. IEEE Trans. Knowl. Data Eng. 32, 1 (2020), 54–66.
[46]
Stanislav Pidhorskyi, Donald A. Adjeroh, and Gianfranco Doretto. 2020. Adversarial latent autoencoders. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.14104–14113.
[47]
Mauro Ribeiro, Katarina Grolinger, and Miriam A. M. Capretz. 2015. MLaaS: Machine learning as a service. In Proceedings of the IEEE International Conference on on Maching Learning and Applications.896–902.
[48]
Ronald L. Rivest, Len Adleman, and Michael L. Dertouzos. 1978. On data banks and privacy homomorphisms. Found. of Secure Comput. 4, 11 (1978), 169–180.
[49]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-assisted Intervention.234–241.
[50]
Sara Salim, Nour Moustafa, Benjamin Turnbull, and Imran Razzak. 2022. Perturbation-enabled deep federated learning for preserving internet of things-based social networks. ACM Trans. Multim. Comput. Commun. Appl. 18, 2s, Article 120 (2022), 19 pages.
[51]
Amartya Sanyal, Matt Kusner, Adria Gascon, and Varun Kanade. 2018. TAPAS: Tricks to accelerate (encrypted) prediction as a service. In Proceedings of the International Conference on Machine Learning. 4490–4499.
[52]
Alireza Sarmadi, Hao Fu, Prashanth Krishnamurthy, Siddharth Garg, and Farshad Khorrami. 2024. Privacy-Preserving Collaborative Learning through Feature Extraction. IEEE Trans. on Depend. and Secure Comput. 21, 1 (2024), 486–498.
[53]
Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.815–823.
[54]
Ali Shahin Shamsabadi, Adrià Gascón, Hamed Haddadi, and Andrea Cavallaro. 2020. PrivEdge: From local to distributed private training and prediction. IEEE Trans. Inf. Forens. Secur. 15 (2020), 3819–3831.
[55]
Ji Wang, Jianguo Zhang, Weidong Bao, Xiaomin Zhu, Bokai Cao, and Philip S. Yu. 2018. Not just privacy: Improving performance of private deep learning in mobile cloud. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2407–2416.
[56]
Yue Wu, Yicong Zhou, George Saveriades, Sos Agaian, Joseph P. Noonan, and Premkumar Natarajan. 2013. Local Shannon entropy measure with statistical tests for image randomness. Inf. Sci. 222 (2013), 323–342.
[57]
Kang Xu, Weixin Li, Xia Wang, Xiaoyan Hu, Ke Yan, Xiaojie Wang, and Xuan Dong. 2023. CUR transformer: A convolutional unbiased regional transformer for image denoising. ACM Trans. Multim. Comput. Commun. Appl. 19, 3, Article 104 (Feb.2023), 22 pages.
[58]
Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. 2019. Federated machine learning: Concept and applications. ACM Trans. Intell. Syst. Technol. 10, 2 (2019), 1–19.
[59]
Zhenzhen Yang, Pengfei Xu, Yongpeng Yang, and Bing-Kun Bao. 2021. A densely connected network based on U-Net for medical image segmentation. ACM Trans. Multim. Comput. Commun. Appl. 17, 3, Article 89 (2021), 14 pages.
[60]
Andrew C. Yao. 1982. Protocols for secure computations. In Proceedings of the Annual Symposium on Foundations of Computer Science.
[61]
Jin Yuan, Shikai Chen, Yao Zhang, Zhongchao Shi, Xin Geng, Jianping Fan, and Yong Rui. 2023. Graph attention transformer network for multi-label image classification. ACM Trans. Multim. Comput. Commun. Appl. 19, 4, Article 150 (2023), 16 pages.
[62]
Chaoning Zhang, Philipp Benz, Adil Karjauv, Geng Sun, and In So Kweon. 2020. UDH: Universal deep hiding for steganography, watermarking, and light field messaging. Proceedings of the International Conference on Neural Information Processing Systems.10223–10234.
[63]
Hanwei Zhang, Yannis Avrithis, Teddy Furon, and Laurent Amsaleg. 2020. Walking on the edge: Fast, low-distortion adversarial examples. IEEE Trans. Inf. Forens. Secur. 16 (2020), 701–713.
[64]
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. 2018. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.586–595.
[65]
Lingchen Zhao, Qian Wang, Cong Wang, Qi Li, Chao Shen, and Bo Feng. 2021. VeriML: Enabling integrity assurances and fair payments for machine learning as a service. IEEE Trans. Parallel Distrib. Syst. 32, 10 (2021), 2524–2540.
[66]
Dennis G. Zill and Warren S. Wright. 2009. Multivariable Calculus. Jones & Bartlett Publishers.
Index Terms
- Recoverable Privacy-Preserving Image Classification through Noise-like Adversarial Examples
Recommendations
Privacy-Preserving Certificateless Cloud Auditing with Multiple Users
Cloud auditing is one of the important processes to ensure the security and integrity of data in cloud storage. Implementing cloud auditing requires various cryptographic tools such as identity-based cryptography and its variant: certificateless ...
Privacy-preserving data hiding with robustness based on selective encryption and matrix embedding
AbstractWith the increasing need for privacy-preserving, data hiding in encrypted images has attracted more attention in the data hiding area. The existing data hiding methods for encrypted images mainly focus on improving the embedding capacity. Recently,...
Security analysis of a privacy-preserving decentralized ciphertext-policy attribute-based encryption scheme
As it does not require a central authority or the cooperation among multiple authorities, decentralized attribute-based encryption is an efficient and flexible multi-authority attribute-based encryption system. In most existing multi-authority attribute-...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
July 2024
973 pages
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 16 May 2024
Online AM: 21 March 2024
Accepted: 18 March 2024
Revised: 13 March 2024
Received: 29 September 2023
Published in TOMM Volume 20, Issue 7
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Macau Science and Technology Development Fund
- Research Committee at University of Macau
- Natural Science Foundation of China
- Alibaba Group through Alibaba Innovative Research Program
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 251Total Downloads
- Downloads (Last 12 months)251
- Downloads (Last 6 weeks)48
Reflects downloads up to 15 Jan 2025
Other Metrics
Citations
Cited By
View all- Liu CJin YGuan ZLi TQin YQian BJiang ZWu YWang XZheng YZeng D(2024)Visual–language foundation models in medicineThe Visual Computer10.1007/s00371-024-03579-wOnline publication date: 29-Jul-2024
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in