More Web Proxy on the site http://driver.im/

research-article

Visualization evaluation for cyber security: trends and future directions

Authors:

R. Jordan Crouser,

Suresh Damodaran,

Lane HarrisonAuthors Info & Claims

VizSec '14: Proceedings of the Eleventh Workshop on Visualization for Cyber Security

Pages 49 - 56

https://doi.org/10.1145/2671491.2671492

Published: 10 November 2014 Publication History

Abstract

The Visualization for Cyber Security research community (VizSec) addresses longstanding challenges in cyber security by adapting and evaluating information visualization techniques with application to the cyber security domain. This research effort has created many tools and techniques that could be applied to improve cyber security, yet the community has not yet established unified standards for evaluating these approaches to predict their operational validity. In this paper, we survey and categorize the evaluation metrics, components, and techniques that have been utilized in the past decade of VizSec research literature. We also discuss existing methodological gaps in evaluating visualization in cyber security, and suggest potential avenues for future research in order to help establish an agenda for advancing the state-of-the-art in evaluating cyber security visualizations.

References

[1]

D. Afergan, E. M. Peck, E. T. Solovey, a. Jenkins, S. W. Hincks, E. T. Brown, R. Chang, and R. J. Jacob. Dynamic difficulty using brain metrics of workload. In Proc. 32nd annual ACM Conf. on Human factors in computing systems, pages 3797--3806. ACM, 2014.

Digital Library

[2]

W. Albert and T. Tullis. Measuring the user experience: collecting, analyzing, and presenting usability metrics. Newnes, 2013.

Digital Library

[3]

R. Ball, G. A. Fink, and C. North. Home-centric visualization of network traffic for security administration. In Proc. 2004 ACM workshop on Visualization and data mining for computer security, pages 55--64. ACM, 2004.

Digital Library

[4]

L. Barkhuus and J. A. Rode. From mice to men-24 years of evaluation in chi. In Proc. SIGCHI Conf. on Human factors in computing systems. ACM, 2007.

Digital Library

[5]

S. Bateman, C. Gutwin, N. Osgood, and G. McCalla. Interactive usability instrumentation. In Proc. 1st ACM SIGCHI symposium on Engineering interactive computing systems, pages 45--54. ACM, 2009.

Digital Library

[6]

H. Beyer and K. Holtzblatt. Contextual design: defining customer-centered systems. Elsevier, 1997.

Digital Library

[7]

S. Carpendale. Evaluating information visualizations. In Information Visualization, pages 19--45. Springer, 2008.

Digital Library

[8]

Guide to a successful archive submission. Online, July 2014.

[9]

V. P. Committee. Paper submission guidelines: Paper types. Online, jul 2014.

[10]

W. Dou, D. H. Jeong, F. Stukes, W. Ribarsky, H. R. Lipford, and R. Chang. Recovering reasoning process from user interactions. IEEE Computer Graphics & Applications, 2009.

Digital Library

[11]

N. Elmqvist and J. S. Yi. Patterns for visualization evaluation. Information Visualization, page 1473871613513228, 2013.

[12]

G. A. Fink, C. L. North, A. Endert, and S. Rose. Visualizing cyber security: Usable workspaces. In Visualization for Cyber Security, 2009. VizSec 2009. 6th Int'l Workshop on, pages 45--56. IEEE, 2009.

[13]

C. Gates and S. Engle. Reflecting on visualization for cyber security. In Intelligence and Security Informatics (ISI), 2013 IEEE Int'l Conf. on, pages 275--277. IEEE, 2013.

[14]

M. Gleicher. Why ask why?: considering motivation in visualization evaluation. In Proc. 2012 BELIV Workshop: Beyond Time and Errors-Novel Evaluation Methods for Visualization, page 10. ACM, 2012.

Digital Library

[15]

D. Gotz and M. X. Zhou. Characterizing users' visual analytic activity for insight provenance. Information Visualization, 8(1):42--55, 2009.

Digital Library

[16]

S. Greenberg and B. Buxton. Usability evaluation considered harmful (some of the time). In Proc. SIGCHI Conf. on Human Factors in Computing Systems, pages 111--120. ACM, 2008.

Digital Library

[17]

L. Hao, C. G. Healey, and S. E. Hutchinson. Flexible web visualization for alert-based network security analytics. In Proc. Tenth Workshop on Visualization for Cyber Security, pages 33--40. ACM, 2013.

Digital Library

[18]

T. Isenberg, P. Isenberg, J. Chen, M. Sedlmair, and T. Moller. A systematic review on the practice of evaluating visualization. Visualization and Computer Graphics, IEEE Trans. on, 19(12):2818--2827, 2013.

Digital Library

[19]

B. Jackson, D. Coffey, L. Thorson, D. Schroeder, A. M. Ellingson, D. J. Nuckley, and D. F. Keefe. Toward mixed method evaluations of scientific visualizations and design process as an evaluation tool. In Proc. 2012 BELIV Workshop: Beyond Time and Errors-Novel Evaluation Methods for Visualization, page 4. ACM, 2012.

Digital Library

[20]

T. Jankun-Kelly, J. Franck, D. Wilson, J. Carver, D. Dampier, and J. E. Swan Ii. Show me how you see: Lessons from studying computer forensics experts for visualization. In Visualization for Computer Security, pages 80--86. Springer, 2008.

Digital Library

[21]

H. Lam, E. Bertini, P. Isenberg, C. Plaisant, and S. Carpendale. Empirical studies in information visualization: Seven scenarios. Visualization and Computer Graphics, IEEE Trans. on, 18(9):1520--1536, 2012.

Digital Library

[22]

M. D. McNeese, K. Perusich, and J. R. Rentsch. Advancing socio-technical systems design via the living laboratory. In Proc. Human Factors and Ergonomics Society Annual Meeting, volume 44 Number 12, pages 2--610. SAGE Publications, 2000.

[23]

M. J. Muller. Participatory design: the third space in hci. Human-computer interaction: Development process, pages 165--185, 2003.

[24]

T. Munzner. Process and pitfalls in writing information visualization research papers. In Information visualization, pages 134--153. Springer, 2008.

Digital Library

[25]

T. Munzner. A nested model for visualization design and validation. Visualization and Computer Graphics, IEEE Trans. on, 15(6):921--928, 2009.

Digital Library

[26]

J. Nielsen. Usability inspection methods. In Conf. companion on Human factors in computing systems, pages 413--414. ACM, 1994.

Digital Library

[27]

C. North. Toward measuring visualization insight. Computer Graphics and Applications, IEEE, 26(3):6--9, 2006.

Digital Library

[28]

A. A. Ozok. Survey design and implementation in hci. Human-Computer Interaction: Development Process, page 253, 2009.

[29]

E. M. M. Peck, B. F. Yuksel, A. Ottley, R. J. Jacob, and R. Chang. Using fnirs brain sensing to evaluate information visualization interfaces. In Proc. SIGCHI Conf. on Human Factors in Computing Systems, pages 473--482. ACM, 2013.

Digital Library

[30]

A. Perer and B. Shneiderman. Integrating statistics and visualization: case studies of gaining clarity during exploratory data analysis. In Proc. SIGCHI Conf. on Human Factors in computing systems, pages 265--274. ACM, 2008.

Digital Library

[31]

R. W. Picard and S. B. Daily. Evaluating affective interactions: Alternatives to asking what users feel. In CHI Workshop on Evaluating Affective Interfaces: Innovative Approaches, pages 2119--2122, 2005.

[32]

W. A. Pike, J. Stasko, R. Chang, and T. A. O'Connell. The science of interaction. Information Visualization, 8(4):263--274, 2009.

Digital Library

[33]

C. Plaisant. The challenge of information visualization evaluation. In Proc. working Conf. on Advanced visual interfaces, pages 109--116. ACM, 2004.

Digital Library

[34]

J. Rasmussen, K. Ehrlich, S. Ross, S. Kirk, D. Gruen, and J. Patterson. Nimble cybersecurity incident management through visualization and defensible recommendations. In Proc. Seventh Int'l Symposium on Visualization for Cyber Security, pages 102--113. ACM, 2010.

Digital Library

[35]

N. Riche et al. Beyond system logging: human logging for evaluating information visualization. In Proc. BELIV 2010 Workshop, 2010.

[36]

J. Sauro and J. R. Lewis. Quantifying the user experience: Practical statistics for user research. Elsevier, 2012.

Digital Library

[37]

J. Scholtz. Beyond usability: Evaluation aspects of visual analytic environments. In Visual Analytics Science and Technology, 2006 IEEE Symposium On, pages 145--150. IEEE, 2006.

[38]

K. Sedig, P. Parsons, and A. Babanski. Towards a characterization of interactivity in visual analytics. JMPT, 3(1):12--28, 2012.

[39]

M. Sedlmair, M. Meyer, and T. Munzner. Design study methodology: Reflections from the trenches and the stacks. Visualization and Computer Graphics, IEEE Trans. on, 18(12):2431--2440, 2012.

Digital Library

[40]

B. Shneiderman and C. Plaisant. Strategies for evaluating information visualization tools: multi-dimensional in-depth long-term case studies. In Proc. 2006 AVI workshop on BEyond time and errors: novel evaluation methods for information visualization, pages 1--7. ACM, 2006.

Digital Library

[41]

B. Steichen, G. Carenini, and C. Conati. User-adaptive information visualization: using eye gaze data to infer visualization tasks and user cognitive abilities. In Proc. 2013 Int Conf. on Intelligent user interfaces, pages 317--328. ACM, 2013.

Digital Library

[42]

M. Wattenberg and D. Fisher. Analyzing perceptual organization in information graphics. Information Visualization, 3(2):123--133, 2004.

Digital Library

[43]

L. E. Wood. Semi-structured interviewing for user-centered design. interactions, 4(2):48--61, 1997.

Digital Library

[44]

X. S. Zheng, I. Chakraborty, J. J.-W. Lin, and R. Rauschenberger. Correlating low-level image statistics with users-rapid aesthetic and affective judgments of web pages. In Proc. SIGCHI Conf. on Human Factors in Computing Systems, pages 1--10. ACM, 2009.

Digital Library

Cited By

Adeyinka A(2024)The Role of Program Managers in Ensuring Successful Cybersecurity InitiativesInternational Journal of Innovative Science and Research Technology (IJISRT)10.38124/ijisrt/IJISRT24MAY2350(2942-2944)Online publication date: 15-Jun-2024
https://doi.org/10.38124/ijisrt/IJISRT24MAY2350
Rakotondravony NPöhls HPfeifer JHarrison L(2024)Viz$$^4$$NetSec: Visualizing Dynamic Network Security Configurations of Everyday Interconnected Objects in Home NetworksHCI for Cybersecurity, Privacy and Trust10.1007/978-3-031-61382-1_11(164-185)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-61382-1_11
(2023)Cyber Security Awareness (CSA) and Cyber Crime in Bangladesh: A Statistical Modeling ApproachAustralian Journal of Engineering and Innovative Technology10.34104/ajeit.023.015025(15-25)Online publication date: 9-Feb-2023
https://doi.org/10.34104/ajeit.023.015025
Show More Cited By

Visualization evaluation for cyber security: trends and future directions
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

7 key challenges for visualization in cyber network defense
VizSec '14: Proceedings of the Eleventh Workshop on Visualization for Cyber Security

What does it take to be a successful visualization in cyber security? This question has been explored for some time, resulting in many potential solutions being developed and offered to the cyber security community. However, when one reflects upon the ...
Government regulations in cyber security: Framework, standards and recommendations
Abstract
Cyber security refers to the protection of Internet-connected systems, such as hardware, software as well as data (information) from cyber attacks (adversaries). A cyber security regulation is needed in order to protect information ...
Highlights
- We list and discuss the cyber attacks, security requirements and measures. We then discuss the cyber security incident management framework and its various ...
Evaluating Climate Visualization: An Information Visualization Approach
IV '10: Proceedings of the 2010 14th International Conference Information Visualisation

To meet the growing demand of communicating climate science and policy research, the interdisciplinary field of climate visualization has increasingly extended its traditional use of 2D representations and techniques from the field of scientific ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

VizSec '14: Proceedings of the Eleventh Workshop on Visualization for Cyber Security

November 2014

105 pages

ISBN:9781450328265

DOI:10.1145/2671491

General Chair:
Kirsten Whitley
US Department of Defense
,
Program Chair:
Sophie Engle
University of San Francisco
,
Publications Chairs:
Lane Harrison,
Fabian Fischer,
Nicolas Prigent

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 November 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Assistant Secretary of Defense for Research & Engineering under Air Force Contract

Conference

VizSec '14

VizSec '14: Visualization for Cyber Security

November 10, 2014

Paris, France

Acceptance Rates

VizSec '14 Paper Acceptance Rate 12 of 43 submissions, 28%;

Overall Acceptance Rate 39 of 111 submissions, 35%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

48
Total Citations
View Citations
2,497
Total Downloads

Downloads (Last 12 months)182
Downloads (Last 6 weeks)40

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Adeyinka A(2024)The Role of Program Managers in Ensuring Successful Cybersecurity InitiativesInternational Journal of Innovative Science and Research Technology (IJISRT)10.38124/ijisrt/IJISRT24MAY2350(2942-2944)Online publication date: 15-Jun-2024
https://doi.org/10.38124/ijisrt/IJISRT24MAY2350
Rakotondravony NPöhls HPfeifer JHarrison L(2024)Viz$$^4$$NetSec: Visualizing Dynamic Network Security Configurations of Everyday Interconnected Objects in Home NetworksHCI for Cybersecurity, Privacy and Trust10.1007/978-3-031-61382-1_11(164-185)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-61382-1_11
(2023)Cyber Security Awareness (CSA) and Cyber Crime in Bangladesh: A Statistical Modeling ApproachAustralian Journal of Engineering and Innovative Technology10.34104/ajeit.023.015025(15-25)Online publication date: 9-Feb-2023
https://doi.org/10.34104/ajeit.023.015025
Morrison BNicholson JCoventry LBriggs P(2023)Recognising Diversity in Older Adults' Cybersecurity NeedsProceedings of the 2023 ACM Conference on Information Technology for Social Good10.1145/3582515.3609565(437-445)Online publication date: 6-Sep-2023
https://dl.acm.org/doi/10.1145/3582515.3609565
Vikas BKarthikeya NChandu GRaja GSai N(2023)Study of Enhancing Usage of Data Visualization in Cyber Security- Quick, Efficient, and Complete2023 Third International Conference on Artificial Intelligence and Smart Energy (ICAIS)10.1109/ICAIS56108.2023.10073891(410-413)Online publication date: 2-Feb-2023
https://doi.org/10.1109/ICAIS56108.2023.10073891
Dawar SKudal PDawar P(2022)Cybersecurity as a Digital and Economic EnablerCybersecurity Issues, Challenges, and Solutions in the Business World10.4018/978-1-6684-5827-3.ch003(37-45)Online publication date: 14-Oct-2022
https://doi.org/10.4018/978-1-6684-5827-3.ch003
Wang HLau N(2022)DURESS SCADA: A simulation platform to study user interface design for cybersecurity of industrial control systemsProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/107118132266117366:1(115-119)Online publication date: 27-Oct-2022
https://doi.org/10.1177/1071181322661173
Komadina AMihajlovic ZGros S(2022)Analysis of the Design Space for Cybersecurity Visualizations in VizSec2022 IEEE Symposium on Visualization for Cyber Security (VizSec)10.1109/VizSec56996.2022.9941422(1-11)Online publication date: 19-Oct-2022
https://doi.org/10.1109/VizSec56996.2022.9941422
Puentes MLei YRakotondravony NHarrison LShue C(2022)Visualizing Web Application Execution Logs to Improve Software Security Defect Localization2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER53432.2022.00138(1183-1190)Online publication date: Mar-2022
https://doi.org/10.1109/SANER53432.2022.00138
Muhati ERawat D(2022)Hidden-Markov-Model-Enabled Prediction and Visualization of Cyber Agility in IoT EraIEEE Internet of Things Journal10.1109/JIOT.2021.30561189:12(9117-9127)Online publication date: 15-Jun-2022
https://doi.org/10.1109/JIOT.2021.3056118
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents