More Web Proxy on the site http://driver.im/

research-article

Taking 5 minutes protects you for 5 months: : Evaluating an anti-phishing awareness video

Authors:

Benjamin M. Berens,

Mattia Mossano,

Melanie VolkamerAuthors Info & Claims

Volume 137, Issue C

https://doi.org/10.1016/j.cose.2023.103620

Published: 12 April 2024 Publication History

Abstract

Phishing is one of the biggest security threats to organizations. Anti-phishing awareness measures can improve phishing email detection rates. These measures need to be efficient, effective, and have an enduring impact over months, rather than days. Related research provides evidence of their effectiveness in the short term. However, questions remain as to how long this impact endures. We conducted a retention user study in two phases, with almost 200 participants in the first phase and almost 80 in the second phase, to determine whether a five-minute video retains its effectiveness five months after the intervention (similar to related work on more time-intensive measures). Our results suggest that short videos can indeed still exert a positive influence five months later. We also report on the video's influence on phishing detection strategies, as well as on viewers' confidence in this respect. Based on our results, we propose recommendations to inform the content of future awareness refreshment measures.

Highlights

•

Anti-phishing knowledge from a short video still effective 5 months after watching it.

•

Tooltip interfaces reduce the loss of anti-phishing knowledge after 5 months.

•

Effect of the short anti-phishing video more prevalent for group with URL in the status bar, but more easily forgotten.

•

Mangle (e.g., typos) phishing technique is the most difficult to notice.

References

[1]

J. Abawajy, User preference of cyber security awareness delivery methods, Behav. Inf. Technol. 33 (2014) 237–248,.

[2]

S. Albakry, K. Vaniea, M.K. Wolters, What is this URL's destination? Empirical evaluation of users' URL reading, in: Conference on Human Factors in Computing Systems, ACM, New York, NY, US, 2020, pp. 1–12,.

Digital Library

[3]

K. Althobaiti, N. Meng, K. Vaniea, I don't need an expert! Making URL phishing features human comprehensible, in: Conference on Human Factors in Computing Systems, ACM, New York, NY, US, 2021, pp. 1–17,.

Digital Library

[4]

Anti-Phishing Working Group (2022): Phishing activity trends report. Technical Report. Anti-Phishing Working Group https://docs.apwg.org/reports/apwg_trends_report_q3_2022.pdf.

[5]

N.A.G. Arachchilage, I. Flechais, K. Beznosov, Poster: a game storyboard design for avoiding phishing attacks, in: Tenth Symposium on Usable Privacy and Security, USENIX, Berkeley, CA, US, 2014, pp. 1–2. https://cups.cs.cmu.edu/soups/2014/posters/soups2014_posters-paper39.pdf.

[6]

N.A.G. Arachchilage, S. Love, K. Beznosov, Phishing threat avoidance behaviour: an empirical investigation, Comput. Hum. Behav. 60 (2016) 185–197,.

Digital Library

[7]

B.M. Berens, K. Dimitrova, M. Mossano, M. Volkamer, Phishing awareness and education – when to best remind?, in: Symposium on Usable Security and Privacy, Internet Society, Reston, VA, US, 2022, pp. 1–15,.

[8]

Beyer, M.; Ahmed, S.; Doerlemann, K.; Arnell, S.; Parkin, S.; Sasse, A.; Passingham, N. (2015): Awareness is only the first step: a framework for progressive engagement of staff in cyber security. Business white paper: Hewlett Packard https://www.riscs.org.uk/wp-content/uploads/2015/12/Awareness-is-Only-the-First-Step.pdf.

[9]

C. Canfield, B. Fischhoff, A. Davis, Poster: using signal detection theory to measure phishing detection ability and behavior, in: Eleventh Symposium on Usable Privacy and Security, USENIX, Berkeley, CA, US, 2015, pp. 1–2. https://cups.cs.cmu.edu/soups/2015/posters/soups2015_posters-final13.pdf.

[10]

G. Canova, M. Volkamer, C. Bergmann, B. Berens, NoPhish app evaluation: lab and retention study, in: Workshop on Usable Security, Internet Society, Reston, VA, US, 2015, pp. 1–10,.

[11]

G. Canova, M. Volkamer, C. Bergmann, R. Borza, B. Reinheimer, S. Stockhardt, R. Tenberg, Learn to spot phishing URLs with the Android NoPhish App, in: Ninth World Conference on Information Security Education, Springer, Cham, 2015, pp. 87–100,.

[12]

L.Y. Chang, N. Coppel, Building cyber security awareness in a developing country: lessons from Myanmar, Comput. Secur. 97 (2020),.

Digital Library

[13]

J. Cohen, Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit, Psychol. Bull. 70 (1968) 213–220.

[14]

A. Field, Discovering Statistics Using IBM SPSS Statistics, Sage Publications, Thousand Oaks, CA, US, 2013.

Digital Library

[15]

A. Franz, V. Zimmermann, G. Albrecht, K. Hartwig, C. Reuter, A. Benlian, J. Vogt, SoK: still plenty of phish in the sea — a taxonomy of user-oriented phishing interventions and avenues for future research, in: Seventeenth Symposium on Usable Privacy and Security, USENIX, Berkeley, CA, US, 2021, pp. 339–358. https://www.usenix.org/conference/soups2021/presentation/franz.

[16]

V. Garg, L.J. Camp, L. Mae, K. Connelly, Designing risk communication for older adults, in: Seventh Symposium on Usable Privacy and Security, USENIX, Berkeley, CA, US, 2011, pp. 20–22.

[17]

C.J. Gokul, S. Pandit, S. Vaddepalli, H. Tupsamudre, V. Banahatti, S. Lodha, PHISHY - a serious game to train enterprise users on phishing awareness, in: Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts, ACM, New York, NY, US, 2018, pp. 169–181,.

Digital Library

[18]

Gonzalez, R.; Locasto, M.E. (2015): An interdiscplinary study of phishing and spear-phishing attacks. In: Eleventh USENIX Conference on Usable Privacy and Security http://cups.cs.cmu.edu/soups/2015/papers/eduGonzales.pdf.

[19]

K.J. Hamdani, M.I.E. Mustafa, Effectiveness of Online Anti-Phishing Delivery methods in raising Awareness among Internet Users, Master's thesis Luleå University of Technology, Department of Computer Science, Electrical and Space Engineering, 2021.

[20]

S. Hart, A. Margheri, F. Paci, V. Sassone, Riskio: a serious game for cyber security awareness and education, Comput. Secur. 95 (2020),.

[21]

D. Jampen, G. Gür, T. Sutter, B. Tellenbach, Don't click: towards an effective anti-phishing training. A comparative literature review, Hum.-Cent. Comput. Inf. Sci. 10 (2020),.

Digital Library

[22]

I. Kirlappos, M.A. Sasse, Security education against phishing: a modest proposal for a major rethink, IEEE Secur. Priv. 10 (2012) 24–32,.

Digital Library

[23]

P. Kumaraguru, J. Cranshaw, A. Acquisti, L. Cranor, J. Hong, M.A. Blair, T. Pham, School of phish: a real-world evaluation of anti-phishing training, in: Fifth Symposium on Usable Privacy and Security, ACM, New York, NY, US, 2009, pp. 1–12,.

Digital Library

[24]

A. Kunz, M. Volkamer, S. Stockhardt, S. Palberg, T. Lottermann, E. Piegert, Nophish: evaluation of a web application that teaches people being aware of phishing attacks, in: Informatik 2016 P-259, 2016, pp. 509–518,.

[25]

E. Lastdrager, I.C. Gallardo, P. Hartel, M. Junger, How effective is anti-phishing training for children?, in: Thirteenth Symposium on Usable Privacy and Security, USENIX, Berkeley, CA, US, 2017, pp. 229–239. https://www.usenix.org/conference/soups2017/technical-sessions/presentation/lastdrager.

[26]

P. Mayer, M. Volkamer, M. Kauer, Authentication schemes - comparison and effective password spaces, in: International Conference on Information Systems Security, Springer, Cham, 2014, pp. 204–225,.

[27]

M.L. McHugh, Interrater reliability: the kappa statistic, Biochem. Med. 22 (2012) 276–282. https://hrcak.srce.hr/89395.

[28]

Misra, G.; Arachchilage, N.A.G.; Berkovsky, S. : Phish phinder: a game design approach to enhance user confidence in mitigating phishing attacks. http://arxiv.org/abs/1710.06064 CoRR (2017): Phish phinder: a game design approach to enhance user confidence in mitigating phishing attacks. arXiv:1710.06064.

[29]

M. Mossano, B. Berens, P. Heller, C. Beckmann, L. Aldag, P. Mayer, M. Volkamer, SMILE - smart eMaIl link domain extractor, in: Computer Security. ESORICS 2021 International Workshops, Springer, Cham, 2022, pp. 403–412,.

Digital Library

[30]

M. Mossano, K. Vaniea, L. Aldag, R. Düzgün, P. Mayer, M. Volkamer, Analysis of publicly available anti-phishing webpages: contradicting information, lack of concrete advice and very narrow attack vector, in: European Symposium on Security and Privacy Workshops, IEEE, New York, NY, US, 2020, pp. 130–139,.

[31]

S. Neumann, B. Reinheimer, M. Volkamer, Don't be deceived: the message might be fake, in: Trust, Privacy and Security in Digital Business, Springer, Cham, 2017, pp. 199–214,.

[32]

R.E. Nisbett, T.D. Wilson, The halo effect: evidence for unconscious alteration of judgments, J. Pers. Soc. Psychol. 35 (1977) 250–256,.

[33]

A.S. Onashoga, O.E. Ojo, O.O. Soyombo, Securix: a 3D game-based learning approach for phishing attack awareness, J. Cybersecurity Technol. 3 (2019) 108–124,.

[34]

J. Petelka, Y. Zou, F. Schaub, Put your warning where your link is: improving and evaluating email phishing warnings, in: 2019 Conference on Human Factors in Computing Systems, ACM, New York, NY, US, 2019, pp. 1–15,.

Digital Library

[35]

D. Pienta, J.B. Thatcher, A.C. Johnston, A taxonomy of phishing: attack types spanning economic, temporal, breadth, and target boundaries, in: Workshop on Information Security and Privacy, Association for Information Systems, Atlanta, GA, US, 2018, pp. 1–18. https://aisel.aisnet.org/wisp2018/19.

[36]

B. Reinheimer, L. Aldag, P. Mayer, M. Mossano, R. Duezguen, B. Lofthouse, T. von Landesberger, M. Volkamer, An investigation of phishing awareness and education over time: when and how to best remind users, in: Sixteenth Symposium on Usable Privacy and Security, USENIX, Berkeley, CA, US, 2020, pp. 259–284. https://www.usenix.org/conference/soups2020/presentation/reinheimer.

[37]

J. Reynolds, D. Kumar, Z. Ma, R. Subramanian, M. Wu, M. Shelton, J. Mason, E. Stark, M. Bailey, Measuring identity confusion with uniform resource locators, in: Conference on Human Factors in Computing Systems, ACM, New York, NY, US, 2020, pp. 1–12,.

Digital Library

[38]

A. Sasse, J. Hielscher, J. Friedauer, M. Peiffer, U. Menges, Warum IT-Sicherheit in Organisationen einen Neustart braucht, in: BSI - 18. Deutscher IT-Sicherheitskongress 2022, BSI, Berlin, DE, 2022, pp. 1–15. https://www.researchgate.net/publication/358277373_Warum_IT-Sicherheit_in_Organisationen_eien_Neustart_braucht.

[39]

S. Sheng, M. Holbrook, P. Kumaraguru, L.F. Cranor, J. Downs, Who falls for phish? A demographic analysis of phishing susceptibility and effectiveness of interventions, in: Conference on Human Factors in Computing Systems, ACM, New York, NY, US, 2010, pp. 373–382,.

Digital Library

[40]

M. Singhal, D. Levine, Analysis and categorization of drive-by download malware, in: Fourth International Conference on Computing, Communications and Security, IEEE, New York, NY, US, 2019, pp. 1–4,.

[41]

A.K. Sood, S. Zeadally, Drive-by download attacks: a comparative study, IT Prof. 18 (2016) 18–25,.

[42]

K.F. Tschakert, S. Ngamsuriyaroj, Effectiveness of and user preferences for security awareness training methodologies, Heliyon 5 (2019),.

[43]

M. Volkamer, K. Renaud, B. Reinheimer, TORPEDO: TOoltip-poweRed phishing email DetectiOn, in: Thirtyfirst ICT Systems Security and Privacy Protection, Springer, Cham, 2016, pp. 161–175,.

[44]

M. Volkamer, K. Renaud, B. Reinheimer, P. Rack, M. Ghiglieri, P. Mayer, A. Kunz, N. Gerber, Developing and evaluating a five minute phishing awareness video, in: Trust, Privacy and Security in Digital Business, Springer, Cham, 2018, pp. 119–134,.

[45]

D. Votipka, D. Abrokwa, M.L. Mazurek, Building and validating a scale for secure software development self-efficacy, in: Conference on Human Factors in Computing Systems, ACM, New York, NY, US, 2020, pp. 1–20,.

Digital Library

[46]

J. Wang, Y. Li, H.R. Rao, Overconfidence in phishing email detection, J. Assoc. Inf. Syst. 17 (2016),.

[47]

R. Wash, M.M. Cooper, Who provides phishing training? Facts, stories, and people like me, in: 2018 CHI Conference on Human Factors in Computing Systems, ACM, New York, NY, US, 2018, pp. 1–12,.

Digital Library

[48]

Z.A. Wen, Z. Lin, R. Chen, E. Andersen, What.Hack: engaging anti-phishing training through a role-playing phishing simulation game, in: Conference on Human Factors in Computing Systems, ACM, New York, NY, US, 2019, pp. 1–12,.

Digital Library

[49]

O.A. Zielinska, R. Tembe, K.W. Hong, X. Ge, E. Murphy-Hill, C.B. Mayhorn, One phish, two phish, how to avoid the Internet phish: analysis of training strategies to detect phishing emails, in: Human Factors and Ergonomics Society Annual Meeting, vol. 58, 2014, pp. 1466–1470,.

Cited By

Schöni LCarles VStrohmeier MMayer PZimmermann V(2024)You Know What? - Evaluation of a Personalised Phishing Training Based on Users' Phishing Knowledge and Detection SkillsProceedings of the 2024 European Symposium on Usable Security10.1145/3688459.3688460(1-14)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3688459.3688460
Al Qahtani EStory PShehab M(2024)The Impact of Risk Appeal Approaches on Users’ Sharing Confidential InformationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642524(1-21)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642524

Recommendations

Overcome EMail Overload with Eudora 5: Get through Your Electronic Mail Faster
You Know What? - Evaluation of a Personalised Phishing Training Based on Users' Phishing Knowledge and Detection Skills
EuroUSEC '24: Proceedings of the 2024 European Symposium on Usable Security
Training is important to support users in phishing detection. To better match phishing training content with users’ current skills, personalised training has huge potential. Therefore, we evaluated personalised training with N=96 participants in an online ...
Working 9-5?: Professional Differences in Email and Boundary Management Practices
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Technology not only brings benefits such as flexible working practices but can also have negative stressful consequences such as increasing email overload and the blurring of work-home boundaries. We report on an exploratory study that extends the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computers and Security

Computers and Security Volume 137, Issue C

Feb 2024

818 pages

Issue’s Table of Contents

The Author(s).

Publisher

Elsevier Advanced Technology Publications

United Kingdom

Publication History

Published: 12 April 2024

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Schöni LCarles VStrohmeier MMayer PZimmermann V(2024)You Know What? - Evaluation of a Personalised Phishing Training Based on Users' Phishing Knowledge and Detection SkillsProceedings of the 2024 European Symposium on Usable Security10.1145/3688459.3688460(1-14)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3688459.3688460
Al Qahtani EStory PShehab M(2024)The Impact of Risk Appeal Approaches on Users’ Sharing Confidential InformationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642524(1-21)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642524

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents