More Web Proxy on the site http://driver.im/

research-article

Best Practices for Notification Studiesfor Security and Privacy Issues on the Internet

Authors:

Henning Pridöhl,

Dominik Herrmann,

Matthias HollickAuthors Info & Claims

ARES '21: Proceedings of the 16th International Conference on Availability, Reliability and Security

Article No.: 90, Pages 1 - 10

https://doi.org/10.1145/3465481.3470081

Published: 17 August 2021 Publication History

Abstract

Researchers help operators of vulnerable and non-compliant internet services by individually notifying them about security and privacy issues uncovered in their research. To improve efficiency and effectiveness of such efforts, dedicated notification studies are imperative. As of today, there is no comprehensive documentation of pitfalls and best practices for conducting such notification studies, which limits validity of results and impedes reproducibility. Drawing on our experience with such studies and guidance from related work, we present a set of guidelines and practical recommendations, including initial data collection, sending of notifications, interacting with the recipients, and publishing the results. We note that future studies can especially benefit from extensive planning and automation of crucial processes, i. e., activities that take place well before the first notifications are sent.

References

[1]

ACM. 2020. Artifact Review and Badging Version 1.1. https://www.acm.org/publications/policies/artifact-review-and-badging-current

[2]

Michael Bailey, David Dittrich, Erin Kenneally, and Douglas Maughan. 2012. The Menlo Report. IEEE Secur. Priv. 10, 2 (2012), 71–75.

Digital Library

[3]

Vaibhav Bajpai, Anna Brunström, Anja Feldmann, Wolfgang Kellerer, Aiko Pras, Henning Schulzrinne, Georgios Smaragdakis, Matthias Wählisch, and Klaus Wehrle. 2019. The Dagstuhl beginners guide to reproducibility for experimental networking research. Comput. Commun. Rev. 49, 1 (2019), 24–30.

Digital Library

[4]

Fabien C. Y. Benureau and Nicolas P. Rougier. 2017. Re-run, Repeat, Reproduce, Reuse, Replicate: Transforming Code into Scientific Contributions. Frontiers Neuroinformatics 11 (2017), 69.

[5]

Cristian Bravo-Lillo, Serge Egelman, Cormac Herley, Stuart Schechter, and Janice Tsai. 2013. You Needn’t Build That: Reusable Ethics-Compliance Infrastructure for Human Subjects Research. In Cybersecurity Research Ethics Dialog & Strategy Workshop, CREDS 2013.

[6]

Davide Canali, Davide Balzarotti, and Aurélien Francillon. 2013. The role of web hosting providers in detecting compromised websites. In 22nd International World Wide Web Conference, WWW ’13, Daniel Schwabe, Virgílio A. F. Almeida, Hartmut Glaser, Ricardo Baeza-Yates, and Sue B. Moon (Eds.). International World Wide Web Conferences Steering Committee / ACM, 177–188.

Digital Library

[7]

Orçun Çetin, Carlos Gañán, Lisette Altena, Samaneh Tajalizadehkhoob, and Michel van Eeten. 2018. Let Me Out! Evaluating the Effectiveness of Quarantining Compromised Users in Walled Gardens. In Fourteenth Symposium on Usable Privacy and Security, SOUPS 2018, Mary Ellen Zurko and Heather Richter Lipford (Eds.). USENIX Association, 251–263.

[8]

Orçun Çetin, Carlos Gañán, Lisette Altena, Samaneh Tajalizadehkhoob, and Michel van Eeten. 2019. Tell Me You Fixed It: Evaluating Vulnerability Notifications via Quarantine Networks. In IEEE European Symposium on Security and Privacy, EuroS&P 2019. IEEE, 326–339.

[9]

Orçun Çetin, Carlos Ganan, Maciej Korczynski, and Michel van Eeten. 2017. Make Notifications Great Again: Learning How to Notify in the Age of Large-Scale Vulnerability Scanning. In 16th Annual Workshop on the Economics of Information Security, WEIS 2017.

[10]

Orçun Çetin, Mohammad Hanif Jhaveri, Carlos Gañán, Michel van Eeten, and Tyler Moore. 2015. Understanding the Role of Sender Reputation in Abuse Reporting and Cleanup. In 14th Annual Workshop on the Economics of Information Security, WEIS 2015.

[11]

Council of Europa. 2021. Chart of signatures and ratifications of Treaty 185: Convention on Cybercrime. https://www.coe.int/en/web/conventions/full-list/-/conventions/treaty/185/signatures

[12]

Ang Cui and Salvatore J Stolfo. 2011. Reflections on the Engineering and Operation of a Large-ScaleEmbedded Device Vulnerability Scanner. In Proceedings of the First Workshop on Building Analysis Datasets and Gathering Experience Returns for Security (BADGERS 2011). ACM, 8–18.

Digital Library

[13]

Constanze Dietrich, Katharina Krombholz, Kevin Borgolte, and Tobias Fiebig. 2018. Investigating System Operators’ Perspective on Security Misconfigurations. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (CCS 2018). ACM New York, 1272–1289.

Digital Library

[14]

Zakir Durumeric, James Kasten, David Adrian, J. Alex Halderman, Michael Bailey, Frank Li, Nicholas Weaver, Johanna Amann, Jethro Beekman, Mathias Payer, and Vern Paxson. 2014. The Matter of Heartbleed. In Proceedings of the 2014 Internet Measurement Conference, IMC 2014, Carey Williamson, Aditya Akella, and Nina Taft (Eds.). ACM, 475–488.

Digital Library

[15]

Zakir Durumeric, Eric Wustrow, and J. Alex Halderman. 2013. ZMap: Fast Internet-wide Scanning and Its Security Applications. In Proceedings of the 22th USENIX Security Symposium, Samuel T. King(Ed.). USENIX Association, 605–620.

[16]

Serge Egelman, Janice Y. Tsai, and Lorrie F. Cranor. 2010. Tell me lies: A methodology for scientifically rigorous security user studies. In Workshop on Studying Online Behaviour at the Conference on Human Factors in Computing Systems 2010. ACM.

[17]

Steven Englehardt and Arvind Narayanan. 2016. Online Tracking: A 1-million-site Measurement and Analysis. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Edgar R. Weippl, Stefan Katzenbeisser, Christopher Kruegel, Andrew C. Myers, and Shai Halevi(Eds.). ACM, 1388–1401.

[18]

Meiko Jensen, Cedric Lauradoux, and Konstantinos Limniotis. 2019. Pseudonymization Techniques and Best Practices. ENISA. https://www.enisa.europa.eu/publications/pseudonymisation-techniques-and-best-practices

[19]

Jacob Leon Kröger, Jens Lindemann, and Dominik Herrmann. 2020. How do app vendors respond to subject access requests?: a longitudinal privacy study on iOS and Android Apps. In ARES 2020: The 15th International Conference on Availability, Reliability and Security, Melanie Volkamer and Christian Wressnegger (Eds.). ACM, 10:1–10:10.

Digital Library

[20]

Kat Krol, Jonathan M. Spring, Simon Parkin, and M. Angela Sasse. 2016. Towards Robust Experimental Design for User Studies in Security and Privacy. In The LASER Workshop: Learning from Authoritative Security Experiment Results (LASER 2016). USENIX Association, San Jose, CA, 21–31.

[21]

Marc Kührer, Thomas Hupperich, Christian Rossow, and Thorsten Holz. 2014. Exit from Hell? Reducing the Impact of Amplification DDoS Attacks. In Proceedings of the 23rd USENIX Security Symposium, Kevin Fu and Jaeyeon Jung (Eds.). USENIX Association, 111–125.

[22]

Frank Li, Zakir Durumeric, Jakub Czyz, Mohammad Karami, Michael Bailey, Damon McCoy, Stefan Savage, and Vern Paxson. 2016. You’ve Got Vulnerability: Exploring Effective Vulnerability Notifications. In 25th USENIX Security Symposium, USENIX Security 16, Thorsten Holz and Stefan Savage (Eds.). USENIX Association, 1033–1050.

[23]

Frank Li, Grant Ho, Eric Kuan, Yuan Niu, Lucas Ballard, Kurt Thomas, Elie Bursztein, and Vern Paxson. 2016. Remedying Web Hijacking: Notification Effectiveness and Webmaster Comprehension. In Proceedings of the 25th International Conference on World Wide Web, WWW 2016, Jacqueline Bourdeau, Jim Hendler, Roger Nkambou, Ian Horrocks, and Ben Y. Zhao (Eds.). ACM, 1009–1019.

Digital Library

[24]

Max Maass, Marc-Pascal Clement, and Matthias Hollick. 2021. Snail Mail Beats Email Any Day: On Effective Operator Security Notifications in the Internet. In ARES 2021: The 15th International Conference on Availability, Reliability and Security, Virtual Event, August 17-20, 2021.

[25]

Max Maass, Anne Laubach, and Dominik Herrmann. 2017. PrivacyScore: Analyse von Webseiten auf Sicherheits- und Privatheitsprobleme. In 47. Jahrestagung der Gesellschaft für Informatik, Digitale Kulturen, INFORMATIK 2017(LNI, Vol. P-275), Maximilian Eibl and Martin Gaedke (Eds.). GI, 1049–1060.

[26]

Max Maass, Alina Stöver, Henning Pridöhl, Sebastian Bretthauer, Dominik Herrmann, Matthias Hollick, and Indra Spiecker. 2021. Effective Notification Campaigns on the Web: A Matter of Trust, Framing, and Support. In 30th USENIX Security Symposium, USENIX Security 21, Online, August 11-13, 2021. USENIX Association.

[27]

Max Maass, Nicolas Walter, Dominik Herrmann, and Matthias Hollick. 2019. On the Difficulties of Incentivizing Online Privacy through Transparency: A Qualitative Survey of the German Health Insurance Market. In Human Practice. Digital Ecologies. Our Future. 14. Internationale Tagung Wirtschaftsinformatik (WI 2019), Thomas Ludwig and Volkmar Pipek (Eds.). University of Siegen, Germany / AISeL, 1307–1321.

[28]

Johan Mazel, Romain Fontugne, and Kensuke Fukuda. 2017. Profiling internet scanners: Spatiotemporal structures and measurement ethics. In Network Traffic Measurement and Analysis Conference, TMA 2017. IEEE, 1–9.

[29]

Victor Le Pochat, Tom van Goethem, Samaneh Tajalizadehkhoob, Maciej Korczynski, and Wouter Joosen. 2019. Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulation. In 26th Annual Network and Distributed System Security Symposium, NDSS 2019. The Internet Society.

[30]

Henning Pridöhl, Pascal Wichmann, Dominik Herrmann, Max Maass, Martin Müller, and Malte. 2019. PrivacyScore/privacyscanner. https://doi.org/10.5281/zenodo.2555037

[31]

Quirin Scheitle, Oliver Hohlfeld, Julien Gamba, Jonas Jelten, Torsten Zimmermann, Stephen D. Strowes, and Narseo Vallina-Rodriguez. 2018. A Long Way to the Top: Significance, Structure, and Stability of Internet Top Lists. In Proceedings of the Internet Measurement Conference 2018, IMC 2018. ACM, 478–493.

Digital Library

[32]

Ben Stock, Giancarlo Pellegrino, Frank Li, Michael Backes, and Christian Rossow. 2018. Didn’t You Hear Me? - Towards More Successful Web Vulnerability Notifications. In 25th Annual Network and Distributed System Security Symposium, NDSS 2018. The Internet Society.

[33]

Ben Stock, Giancarlo Pellegrino, Christian Rossow, Martin Johns, and Michael Backes. 2016. Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification. In 25th USENIX Security Symposium, USENIX Security 16, Thorsten Holz and Stefan Savage (Eds.). USENIX Association, 1015–1032.

[34]

Thorsten Holz and Alina Oprea on behalf of the S&P’21 Program Committee. 2021. IEEE S&P’21 Program Committee Statement Regarding The “Hypocrite Commits” Paper. https://www.ieee-security.org/TC/SP2021/downloads/2021_PC_Statement.pdf

[35]

Marie Vasek and Tyler Moore. 2012. Do Malware Reports Expedite Cleanup? An Experimental Study. In 5th Workshop on Cyber Security Experimentation and Test, CSET ’12, Sean Peisert and Stephen Schwab (Eds.). USENIX Association.

[36]

Jessica Vitak, Katie Shilton, and Zahra Ashktorab. 2016. Beyond the Belmont Principles: Ethical Challenges, Practices, and Beliefs in the Online Data Research Community. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, CSCW 2016, Darren Gergle, Meredith Ringel Morris, Pernille Bjørn, and Joseph A. Konstan (Eds.). ACM, 939–951.

Digital Library

[37]

Eric Zeng, Frank Li, Emily Stark, and Adrienne Porter Felt. 2019. Fixing HTTPS Misconfigurations at Scale: An Experiment with Security Notifications. In 18th Annual Workshop on the Economics of Information Security, WEIS 2019.

Cited By

Kriecherbauer TSchwank RKrauss ANeureither KRemme LVolkamer MHerrmann D(2024)Is Personalization Worth It? Notifying Blogs about a Privacy Issue Resulting from Poorly Implemented Consent BannersProceedings of the 19th International Conference on Availability, Reliability and Security10.1145/3664476.3664499(1-7)Online publication date: 30-Jul-2024
https://dl.acm.org/doi/10.1145/3664476.3664499
Everson DCheng L(2024)A Survey on Network Attack Surface MappingDigital Threats: Research and Practice10.1145/36400195:2(1-25)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3640019
Hilton ADeccio CDavis JCalandrino JTroncoso C(2023)Fourteen years in the lifeProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620415(3171-3186)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620415
Show More Cited By

Index Terms

Best Practices for Notification Studiesfor Security and Privacy Issues on the Internet

Index terms have been assigned to the content through auto-classification.

Recommendations

Internet of Things security

The Internet of things (IoT) has recently become an important research topic because it integrates various sensors and objects to communicate directly with one another without human intervention. The requirements for the large-scale deployment of the IoT ...
Secure Internet Practices: Best Practices for Securing Systems in the Internet and E-Business Age
Review of security issues in Internet of Things and artificial intelligence‐driven solutions
Abstract
Internet of Things (IoT) is a network of several hardware and software systems which is broadly based upon internet services and several state‐of‐the‐art sensing and communication technologies. The emergence of 5G technology will witness a ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ARES '21: Proceedings of the 16th International Conference on Availability, Reliability and Security

August 2021

1447 pages

ISBN:9781450390514

DOI:10.1145/3465481

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 August 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

ARES 2021

ARES 2021: The 16th International Conference on Availability, Reliability and Security

August 17 - 20, 2021

Vienna, Austria

Acceptance Rates

Overall Acceptance Rate 228 of 451 submissions, 51%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
175
Total Downloads

Downloads (Last 12 months)44
Downloads (Last 6 weeks)9

Reflects downloads up to 17 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kriecherbauer TSchwank RKrauss ANeureither KRemme LVolkamer MHerrmann D(2024)Is Personalization Worth It? Notifying Blogs about a Privacy Issue Resulting from Poorly Implemented Consent BannersProceedings of the 19th International Conference on Availability, Reliability and Security10.1145/3664476.3664499(1-7)Online publication date: 30-Jul-2024
https://dl.acm.org/doi/10.1145/3664476.3664499
Everson DCheng L(2024)A Survey on Network Attack Surface MappingDigital Threats: Research and Practice10.1145/36400195:2(1-25)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3640019
Hilton ADeccio CDavis JCalandrino JTroncoso C(2023)Fourteen years in the lifeProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620415(3171-3186)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620415
Ariana MGrant HStefan SGeoffrey M. V(2023)An Empirical Analysis of Enterprise-Wide Mandatory Password UpdatesProceedings of the 39th Annual Computer Security Applications Conference10.1145/3627106.3627198(150-162)Online publication date: 4-Dec-2023
https://dl.acm.org/doi/10.1145/3627106.3627198
De Moraes Firmino EKhurshid A(2023)A Model-Based Approach for Expansion of Androids Status-Bar-NotificationDesign, Operation and Evaluation of Mobile Communications 10.1007/978-3-031-35921-7_2(18-28)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.1007/978-3-031-35921-7_2
Hennig ADietmann HLehr FMutter MVolkamer MMayer P(2022)“Your Cookie Disclaimer is Not in Line with the Ideas of the GDPR. Why?”Human Aspects of Information Security and Assurance10.1007/978-3-031-12172-2_17(218-227)Online publication date: 22-Jul-2022
https://doi.org/10.1007/978-3-031-12172-2_17

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents