More Web Proxy on the site http://driver.im/

research-article

Differential Privacy for Classifier Evaluation

Authors:

David PageAuthors Info & Claims

AISec '15: Proceedings of the 8th ACM Workshop on Artificial Intelligence and Security

Pages 15 - 23

https://doi.org/10.1145/2808769.2808775

Published: 16 October 2015 Publication History

Abstract

Differential privacy provides powerful guarantees that individuals incur minimal additional risk by including their personal data in a database. Most work in differential privacy has focused on differentially private algorithms that produce models, counts, and histograms. Nevertheless, even with a classification model produced by a differentially private algorithm, directly reporting the classifier's performance on a database has the potential for disclosure. Thus, differentially private computation of evaluation metrics for machine learning is an important research area. We find effective mechanisms for area under the receiver-operating characteristic (ROC) curve and average precision.

References

[1]

K. Bache and M. Lichman. UCI machine learning repository, 2013.

[2]

K. Boyd, V. S. Costa, J. Davis, and D. Page. Unachievable region in precision-recall space and its effect on empirical evaluation. In ICML, pages 639--646, 2012.

[3]

K. Chaudhuri, C. Monteleoni, and A. D. Sarwate. Differentially private empirical risk minimization. The Journal of Machine Learning Research, 12:1069--1109, 2011.

Digital Library

[4]

K. Chaudhuri and S. A. Vinterbo. A stability-based validation procedure for differentially private machine learning. In NIPS, pages 2652--2660, 2013.

Digital Library

[5]

C. Dwork. Differential privacy. In ICALP. Springer, 2006.

Digital Library

[6]

C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography, pages 265--284. Springer, 2006.

Digital Library

[7]

A. Friedman and A. Schuster. Data mining with differential privacy. In KDD, pages 493--502. ACM, 2010.

Digital Library

[8]

A. Ghosh, T. Roughgarden, and M. Sundararajan. Universally utility-maximizing privacy mechanisms. In STOC, 2009.

Digital Library

[9]

R. Hall, A. Rinaldo, and L. Wasserman. Differential privacy for functions and functional data. The Journal of Machine Learning Research, 14(1):703--727, 2013.

Digital Library

[10]

D. Kifer and A. Machanavajjhala. Pufferfish: A framework for mathematical privacy definitions. ACM Trans. Database Syst., 39(1):3:1--3:36, Jan. 2014.

Digital Library

[11]

C. D. Manning, P. Raghavan, and H. Schütze. Introduction to Information Retrieval. Cambridge University Press, New York, NY, USA, 2008.

Digital Library

[12]

G. J. Matthews and O. Harel. An examination of data confidentiality and disclosure issues related to publication of empiricalROC\ curves. Academic Radiology, 20(7):889 -- 896, 2013.

[13]

K. Nissim, S. Raskhodnikova, and A. Smith. Smooth sensitivity and sampling in private data analysis. In STOC, page 75. ACM Press, 2007.

Digital Library

[14]

M. S. Pepe. The statistical evaluation of medical tests for classification and prediction. Oxford University Press, USA, 2004.

[15]

F. J. Provost, T. Fawcett, and R. Kohavi. The case against accuracy estimation for comparing induction algorithms. In ICML, volume 98, pages 445--453, 1998.

Digital Library

[16]

B. I. Rubinstein, P. L. Bartlett, L. Huang, and N. Taft. Learning in a large function space: Privacy-preserving mechanisms for svm learning. preprint arXiv:0911.5708, 2009.

[17]

B. Stoddard, Y. Chen, and A. Machanavajjhala. Differentially private algorithms for empirical machine learning. preprint arXiv:1411.5428, 2014.

[18]

L. Wasserman and S. Zhou. A statistical framework for differential privacy. Journal of the American Statistical Association, 105(489):375--389, 2010.

[19]

J. Zhang, Z. Zhang, X. Xiao, Y. Yang, and M. Winslett. Functional mechanism: regression analysis under differential privacy. VLDB, 5(11):1364--1375, 2012.

Digital Library

Cited By

Schalk DRehms RHoffmann VBischl BMansmann U(2024)Distributed non-disclosive validation of predictive models by a modified ROC-GLMBMC Medical Research Methodology10.1186/s12874-024-02312-424:1Online publication date: 29-Aug-2024
https://doi.org/10.1186/s12874-024-02312-4
Movahedi PNieminen VPerez IDaafane HSukhwal DPahikkala TAirola A(2024)Benchmarking Evaluation Protocols for Classifiers Trained on Differentially Private Synthetic DataIEEE Access10.1109/ACCESS.2024.344691312(118637-118648)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3446913
Vetrithangam D(2024)An Efficient Data Privacy Protection System Based on Differential PrivacyData Science and Communication10.1007/978-981-99-5435-3_58(785-798)Online publication date: 3-Jan-2024
https://doi.org/10.1007/978-981-99-5435-3_58
Show More Cited By

Index Terms

Differential Privacy for Classifier Evaluation

Recommendations

An Emerging Strategy for Privacy Preserving Databases: Differential Privacy
HCI for Cybersecurity, Privacy and Trust
Abstract
Data De-identification and Differential Privacy are two possible approaches for providing data security and user privacy. Data de-identification is the process where the personal identifiable information of individuals is extracted to create ...
A Novel Differential Privacy Approach that Enhances Classification Accuracy
C3S2E '16: Proceedings of the Ninth International C* Conference on Computer Science & Software Engineering

In the recent past, there has been a tremendous increase of large repositories of data, examples being in healthcare data, consumer data from retailers, and airline passenger data. These data are continually being shared with interested parties, either ...
Applying Differential Privacy to Matrix Factorization
RecSys '15: Proceedings of the 9th ACM Conference on Recommender Systems

Recommender systems are increasingly becoming an integral part of on-line services. As the recommendations rely on personal user information, there is an inherent loss of privacy resulting from the use of such systems. While several works studied ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

AISec '15: Proceedings of the 8th ACM Workshop on Artificial Intelligence and Security

October 2015

110 pages

ISBN:9781450338264

DOI:10.1145/2808769

General Chair:
Indrajit Ray
Colorado State University, USA
,
Program Chairs:
Christos Dimitrakakis
Chalmers University of Technology, Sweden
,
Aikaterini Mitrokotsa
Chalmers University of Technology, Sweden
,
Arunesh Sinha
University of Southern California, USA
,
Publications Chairs:
Xiaofeng Wang
Indiana University, USA
,
Kui Ren
SUNY Buffalo, USA

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 October 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

US National Library of Medicine

Conference

CCS'15

Sponsor:

SIGSAC

CCS'15: The 22nd ACM Conference on Computer and Communications Security

October 16, 2015

Colorado, Denver, USA

Acceptance Rates

AISec '15 Paper Acceptance Rate 11 of 25 submissions, 44%;

Overall Acceptance Rate 94 of 231 submissions, 41%

Upcoming Conference

CCS '25

Sponsor:
sigsac

ACM SIGSAC Conference on Computer and Communications Security

October 13 - 17, 2025

Taipei , Taiwan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
524
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)2

Reflects downloads up to 22 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Schalk DRehms RHoffmann VBischl BMansmann U(2024)Distributed non-disclosive validation of predictive models by a modified ROC-GLMBMC Medical Research Methodology10.1186/s12874-024-02312-424:1Online publication date: 29-Aug-2024
https://doi.org/10.1186/s12874-024-02312-4
Movahedi PNieminen VPerez IDaafane HSukhwal DPahikkala TAirola A(2024)Benchmarking Evaluation Protocols for Classifiers Trained on Differentially Private Synthetic DataIEEE Access10.1109/ACCESS.2024.344691312(118637-118648)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3446913
Vetrithangam D(2024)An Efficient Data Privacy Protection System Based on Differential PrivacyData Science and Communication10.1007/978-981-99-5435-3_58(785-798)Online publication date: 3-Jan-2024
https://doi.org/10.1007/978-981-99-5435-3_58
Sarkar ASharma AGill AThakur P(2023)A Differential Privacy-based System for Efficiently Protecting Data Privacy2023 International Conference on Sustainable Computing and Smart Systems (ICSCSS)10.1109/ICSCSS57650.2023.10169412(1399-1404)Online publication date: 14-Jun-2023
https://doi.org/10.1109/ICSCSS57650.2023.10169412
Gitiaux XKhant ABeyrami EReddy CGupchup JCutler R(2023)AURA: Privacy-Preserving Augmentation to Improve Test Set Diversity in Speech EnhancementICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP49357.2023.10096879(1-5)Online publication date: 4-Jun-2023
https://doi.org/10.1109/ICASSP49357.2023.10096879
Daniel SBernd BDavid R(2023)Privacy-preserving and lossless distributed estimation of high-dimensional generalized additive mixed modelsStatistics and Computing10.1007/s11222-023-10323-234:1Online publication date: 7-Nov-2023
https://doi.org/10.1007/s11222-023-10323-2
Ünal APfeifer NAkgün M(2023)ppAURORA: Privacy Preserving Area Under Receiver Operating Characteristic and Precision-Recall CurvesNetwork and System Security10.1007/978-3-031-39828-5_15(265-280)Online publication date: 7-Aug-2023
https://doi.org/10.1007/978-3-031-39828-5_15
Rajotte JMukherjee SRobinson COrtiz AWest CFerres JNg R(2021)Reducing bias and increasing utility by federated generative modeling of medical images using a centralized adversaryProceedings of the Conference on Information Technology for Social Good10.1145/3462203.3475875(79-84)Online publication date: 9-Sep-2021
https://dl.acm.org/doi/10.1145/3462203.3475875
Lècuyer MSpahn RVodrahalli KGeambasu RHsu D(2019)Privacy Accounting and Quality Control in the Sage Differentially Private ML PlatformACM SIGOPS Operating Systems Review10.1145/3352020.335203253:1(75-84)Online publication date: 25-Jul-2019
https://dl.acm.org/doi/10.1145/3352020.3352032
Lécuyer MSpahn RVodrahalli KGeambasu RHsu DBrecht TWilliamson C(2019)Privacy accounting and quality control in the sage differentially private ML platformProceedings of the 27th ACM Symposium on Operating Systems Principles10.1145/3341301.3359639(181-195)Online publication date: 27-Oct-2019
https://dl.acm.org/doi/10.1145/3341301.3359639
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents