research-article

Dimensions of Concern: A Method to Use Cognitive Dimensions to Evaluate Interfaces

Authors:

Mark A. Cohen,

Frank E. Ritter,

Steven R. HaynesAuthors Info & Claims

ACM Transactions on Computer-Human Interaction (TOCHI), Volume 19, Issue 2

Article No.: 9, Pages 1 - 18

https://doi.org/10.1145/2240156.2240157

Published: 01 July 2012 Publication History

Get Access

Abstract

Producing useful and usable software often requires continuous and iterative evaluation. This paper introduces a novel usability evaluation method based on the Cognitive Dimensions of Notations framework. The target of our evaluation is Herbal a suite of tools designed to simplify agent development by providing a high-level language and maintenance-oriented development environment. The method introduced here uncovers dimensions of concern, which are used to measure the usability of Herbal and to identify areas for improvement in the design. In this article, we demonstrate how we used dimensions of concern to effectively evaluate and improve usability, and we discuss ways in which our method can be adapted, extended, and applied to improving the usability of other interactive systems.

References

[1]

Blackwell, A. F. and Green, T. R. G. 2000. A cognitive dimensions questionnaire optimised for users. In Procedings of the 12th Workshop of the Psychology of Programming Interest Group.

Google Scholar

[2]

Blackwell, A. F. and Green, T. R. G. 2003. Notational systems: The cognitive dimensions of notations frameworks. In HCI Models, Theories, and Frameworks, J. M. Carroll Ed., Morgan Kaufmann, San Francisco, CA, 103--133

Google Scholar

[3]

Cohen, J. 1960. A coefficient for agreement for nominal scales. Edu. Psych. Measure. 20, 37--46.

Crossref

Google Scholar

[4]

Cohen, M. A. 2005. Teaching agent programming using custom environments and Jess. Newslet. Soc. Study Artif. Intell. Simul. Behav. 120, 4.

Google Scholar

[5]

Cohen, M. A., Ritter, F. E., and Haynes, S. R. 2010. Applying software engineering to agent development. AI Mag., 31, 2, 25--44.

Crossref

Google Scholar

[6]

Ericsson, K. A. and Simon, H. A. 1993. Protocol Analysis: Verbal Reports as Data. MIT Press, Cambridge, MA.

Crossref

Google Scholar

[7]

Haynes, S. R., Cohen, M. A., and Ritter, F. E. 2008. Design patterns for explaining intelligent agents. Int. J. Hum.-Comput. Stud.

Digital Library

Google Scholar

[8]

Kadoda, G., Stone, R., and Diaper, D. 1999. Desirable features of educational theorem provers: A cognitive dimensions viewpoint. In Collected Papers of the 11th Annual Workshop of the Psychology of Programming Interest Group, T. R. G. Green, R. Abdullah, and P. Brna Eds., Leeds: Leeds University Press, Leeds, 18--23.

Google Scholar

[9]

Ko, A. J., Aung, H. H., and Myers, B. A. 2005. Eliciting design requirements for maintenance-oriented IDEs: A detailed study of corrective and perfective maintenance tasks. In Proceedings of the International Conference on Software Engineering.

Digital Library

Google Scholar

[10]

Lehman, J. F., Laird, J. E., and Rosenbloom, P. S. 1996. A gentle introduction to Soar: An architecture for human cognition. In An Invitation to Cognitive Science, Vol. 4, D. Scarborough and S. Sternberg Eds., MIT Press.

Google Scholar

[11]

Newell, A., and Simon, H. A. 1972. Human Problem Solving. Prentice Hall, Englewood Cliffs, NJ.

Digital Library

Google Scholar

[12]

Newell, A. Yost, G. R., Laird, J. E., Rosenbloom, P., and Altmann, E. 1991. Formulating the problem space computational model. In Carnegie Mellon Computer Science: A 25-Year Commemorative, R. F. Rashid Ed., ACM Press-Addison-Wesley, Reading, MA, 255--293.

Google Scholar

[13]

Pew, R. W. and Mavor, A. S. Eds. 2007. Human-System Integration in the System Development Process: A New Look. National Academies Press, Washington, DC.

Google Scholar

[14]

Russell, S. and Norvig, P. (2003). Artificial Intelligence: A Modern Approach 2nd Ed. Prentice Hall, Upper Saddle River, NJ.

Digital Library

Google Scholar

Cited By

View all

Paik JKim JRitter FReitter D(2015)Predicting User Performance and Learning in Human--Computer Interaction with the Herbal CompilerACM Transactions on Computer-Human Interaction10.1145/277689122:5(1-26)Online publication date: 24-Aug-2015
https://dl.acm.org/doi/10.1145/2776891

Index Terms

Dimensions of Concern: A Method to Use Cognitive Dimensions to Evaluate Interfaces
1. Computing methodologies
  1. Artificial intelligence
    1. Philosophical/theoretical foundations of artificial intelligence
      1. Cognitive science
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI

Recommendations

Using cognitive dimensions to evaluate the usability of security APIs: An empirical investigation
Highlights
- This research evaluates the Cognitive Dimensions Framework based Usability Evaluation Methodology for security APIs by measuring its thoroughness, validity, ...
Abstract Context
Usability issues of security Application Programming Interfaces (APIs) are a main factor for mistakes programmers make that could result in introducing security vulnerabilities into applications they develop. This ...
Applying cognitive dimensions to evaluate and improve the usability of Z formalism
SEKE '02: Proceedings of the 14th international conference on Software engineering and knowledge engineering

Cognitive Dimensions framework is a useful tool with which the usability of information artifact can be evaluated. We have used this frame work to evaluate the usability of Z formalism in TranZit environment. The target group for this usability exercise ...
We Need Numbers!: Heuristic Evaluation during Demonstrations (HED) for Measuring Usability in IT System Procurement
CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

We introduce a new usability inspection method called HED (heuristic evaluation during demonstrations) for measuring and comparing usability of competing complex IT systems in public procurement. The method presented enhances traditional heuristic ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Computer-Human Interaction

ACM Transactions on Computer-Human Interaction Volume 19, Issue 2

July 2012

226 pages

ISSN:1073-0516

EISSN:1557-7325

DOI:10.1145/2240156

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2012

Accepted: 01 October 2011

Revised: 01 October 2011

Received: 01 March 2010

Published in TOCHI Volume 19, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
753
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)2

Reflects downloads up to 21 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Paik JKim JRitter FReitter D(2015)Predicting User Performance and Learning in Human--Computer Interaction with the Herbal CompilerACM Transactions on Computer-Human Interaction10.1145/277689122:5(1-26)Online publication date: 24-Aug-2015
https://dl.acm.org/doi/10.1145/2776891

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Using cognitive dimensions to evaluate the usability of security APIs: An empirical investigation

Applying cognitive dimensions to evaluate and improve the usability of Z formalism

We Need Numbers!: Heuristic Evaluation during Demonstrations (HED) for Measuring Usability in IT System Procurement