More Web Proxy on the site http://driver.im/

research-article

Detecting Users’ Cognitive Load by Galvanic Skin Response with Affective Interference

Authors:

Nargess Nourbakhsh,

Rafael A. CalvoAuthors Info & Claims

ACM Transactions on Interactive Intelligent Systems (TiiS), Volume 7, Issue 3

Article No.: 12, Pages 1 - 20

https://doi.org/10.1145/2960413

Published: 19 September 2017 Publication History

Abstract

Experiencing high cognitive load during complex and demanding tasks results in performance reduction, stress, and errors. However, these could be prevented by a system capable of constantly monitoring users’ cognitive load fluctuations and adjusting its interactions accordingly. Physiological data and behaviors have been found to be suitable measures of cognitive load and are now available in many consumer devices. An advantage of these measures over subjective and performance-based methods is that they are captured in real time and implicitly while the user interacts with the system, which makes them suitable for real-world applications. On the other hand, emotion interference can change physiological responses and make accurate cognitive load measurement more challenging. In this work, we have studied six galvanic skin response (GSR) features in detection of four cognitive load levels with the interference of emotions. The data was derived from two arithmetic experiments and emotions were induced by displaying pleasant and unpleasant pictures in the background. Two types of classifiers were applied to detect cognitive load levels. Results from both studies indicate that the features explored can detect four and two cognitive load levels with high accuracy even under emotional changes. More specifically, rise duration and accumulative GSR are the common best features in all situations, having the highest accuracy especially in the presence of emotions.

References

[1]

J. L. Andreassi. 2007. Psychophysiology: Human behavior and physiological response. Lawrence Erlbaum.

[2]

C. Berka, D. Levendowski, M. Lumicao, A. Yau, G. Davis, and V. Zivkovic et al. 2007. EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviation, Space, and Environmental Medicine. 78, 5, B231--44.

[3]

W. Boucsein. 2012. Electrodermal Activity. Springer.

[4]

Y. Cao, M. Theune, and A. Nijholt. 2009. Modality effects on cognitive load and performance in high-load information presentation. International Conference on Intelligent User Interfaces. New York, 335--344.

[5]

R. A. Calvo and D. Peters. 2014. Positive computing: Technology for wellbeing and human potential. MIT Press.

[6]

P. Chandler and J. Sweller. 1996. Cognitive load while learning to use a computer program. Applied Cognitive Psychology. 10, 2, 151--170.

[7]

F. Chen, N. Ruiz, E. Choi, J. Epps, M. A. Khawaja, and R. Taib et al. 2012. Multimodal behaviour and interaction as indicators of cognitive load. ACM Transactions on Interactive Intelligent Systems 4, 2.

[8]

S. Chen, J. Epps, N. Ruiz, and F. Chen. 2011. Eye activity as a measure of human mental effort in HCI. In Proceedings of the 16th International Conference on Intelligent User Interfaces. ACM, 315--318.

Digital Library

[9]

N. Cowan. 2001. The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences. 24, 1, 87--114.

[10]

D. E. Crundall and G. Underwood. 1998. Effects of experience and processing demands on visual information acquisition in drivers. Ergonomics. 41, 4, 448--459.

[11]

M. E. Dawson, A. M. Schell, and D. L. Filion. 2007. The electrodermal system. Handbook of Psychophysiology. 159--181.

[12]

J. Engstrom, E. Johansson, and J. Ostlund. 2005. Effects of visual and cognitive load in real and simulated motorway driving. Transportation Research. 8, 2, 97--120.

[13]

E. Haapalainen, S. Kim, J. F. Forlizzi, and A. K. Dey. 2010. Psycho-physiological measures for assessing cognitive load. International Conference on Ubiquitous Computing. ACM Press, 26--29.

Digital Library

[14]

S. Hart and L. Staveland. 1988. Development of NASA-TLX (task load index): Results of empirical and theoretical research. Human Mental Workload. 52, 139--183.

[15]

P. Hu, P. C. Ma, and P. Y. Chau. 1999. Evaluation of user interface designs for information retrieval systems: A computer-based experiment. Decision Support Systems. 27, 1, 125--143.

Digital Library

[16]

M. S. Hussain, O. AlZoubi, R. A. Calvo, and S. D'Mello. 2011. Affect detection from multichannel physiology during learning sessions with autotutor. International Conference on Artificial Intelligence in Education. Springer, Auckland, New Zealand, 131--138.

[17]

M. S. Hussain, R. Calvo, and F. Chen. 2013. Automatic cognitive load detecting from face, physiology, task performance and fusion during affective interference. Interacting with Computers. 25, 4.

[18]

K. Huttunen, H. Keränen, E. Väyrynen, R. Pääkkönen, and T. Leino. 2011. Effect of cognitive load on speech prosody in aviation: Evidence from military simulator flights. Applied ergonomics. 42, 2, 348--57.

[19]

C. S. Ikehara and M. E. Crosby. 2005. Assessing cognitive load with physiological sensors. Hawaii International Conference on System Sciences. 295a--295a.

Digital Library

[20]

L. C. Johnson and A. Lubin. 1966. Spontaneous electrodermal activity during waking and sleeping. Psychophysiology. 3, 8--17.

[21]

P. Lang, M. Bradley, and B. Cuthbert. 2008. International affective picture system (IAPS): Affective ratings of pictures and instruction manual. University of Florida, Gainesville, FL.

[22]

E. Leyman, G. Mirka, D. Kaber, and C. Sommerich. 2004. Cervicobrachial muscle response to cognitive load in a dual-task scenario. Ergonomics. 47, 6, 625--645.

[23]

F. Lotte, M. Congedo, A. Lécuyer, F. Lamarche, and B. Arnaldi. 2007. A review of classification algorithms for EEG-based brain-computer interfaces. Journal of Neural Engineering. 4, 2, R1--R13.

[24]

B. Mehler, B. Reimer, and J. F. Coughlin. 2012. Sensitivity of physiological measures for detecting systematic variations in cognitive demand from a working memory task: An on-road study across three age groups. Human Factors: The Journal of Human Factors and Ergonomics Society. 54, 3, 396--412.

[25]

K. R. Muller, M. Tangermann, G. Dornhege, M. Krauledat, G. Curio, and B. Blankertz. 2008. Machine learning for real-time single-trial EEG-analysis: From brain-computer interfacing to mental state monitoring. Journal of Neuroscience Methods. 167, 1, 82--90.

[26]

S. Nakagawa. 2004. A farewell to Bonferroni: The problems of low statistical power and publication bias. Behavioral Ecology. 15, 6, 1044--1045.

[27]

A. Nakasone, H. Prendinger, and M. Ishizuka. 2005. Emotion recognition from electromyography and skin conductance. 5th International Workshop on Biosignal Interpretation. Tokyo, Japan, 219--222.

[28]

N. Nourbakhsh, Y. Wang, and F. Chen. 2013. GSR and blink features for cognitive load classification. In Proceedings of the 14th IFIP International Conference on Human-Computer Interaction. Lecture Notes on Computer Science. Vol. 8117. Springer, 159--166.

[29]

N. Nourbakhsh, Y. Wang, F. Chen, and R. A. Calvo. 2012. Using galvanic skin response for cognitive load measurement in arithmetic and reading tasks. In Proceedings of the Australian Computer-Human Interaction Conference. ACM, 420--423.

Digital Library

[30]

F. G. Paas. 1992. Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology 84, 4, 429--434.

[31]

F. Paas and J. Van Merriënboer. 1994a. Instructional control of cognitive load in the training of complex cognitive tasks. Educational Psychology Review. 6, 4, 351--371.

[32]

F. Paas and J. Van Merriënboer. 1994b. Variability of worked examples and transfer of geometrical problem-solving skills: A cognitive-load approach. Journal of Educational Psychology. 86, 1, 122--133.

[33]

T. V. Perneger. 1998. What's wrong with Bonferroni adjustments. British Medical Journal. 316, (7139), 1236--1238.

[34]

N. Ruiz, R. Taib, Y. Shi, Y. E. Choi, and F. Chen. 2007. Using pen input features as indices of cognitive load. In Proceedings of the 9th International Conference on Multimodal Interfaces. ACM Press, 315--318.

Digital Library

[35]

G. R. Saadé and A. C. Otrakji. 2007. First impressions last a lifetime: Effect of interface type on disorientation and cognitive load. Computers in Human Behavior. 23, 1, 525--535.

[36]

W. Schnotz and C. Kürschner. 2007. A reconsideration of cognitive load theory. Educational Psychology Review 19, 4, 469--508.

[37]

C. Setz, B. Arnrich, J. Schumm, R. La Marca, and G. Tröster. 2010. Discriminating stress from cognitive load using a wearable EDA device. Technology. 14, 2, 410--417.

Digital Library

[38]

Y. Shi, N. Ruiz, R. Taib, E. H. Choi, and F. Chen. 2007. Galvanic skin response (GSR) as an index of cognitive load. Computer Human Interaction 2007 Conference Work-in-Progress. 2651--2656.

[39]

R. M. Stern, W. J. Ray, and K. S. Quigley. 2001. Psychophysiological Recording. Oxford University Press.

[40]

J. Sweller. 1988. Cognitive load during problem solving: Effects on learning. Cognitive Science. 12, 2, 257--285.

[41]

J. Sweller. 1994. Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction. 4, 4, 295--312.

[42]

J. Sweller, J. J. Van Merrienboer, and F. G. Paas. 1998. Cognitive architecture and instructional design. Educational Psychology Review. 10, 3, 251--296.

[43]

J. J. Vogel-Walcutt, J. B. Gebrim, C. Bowers, T. M. Carper, and D. Nicholson. 2010. Cognitive load theory vs. constructivist approaches: Which best leads to efficient, deep learning? Journal of Computer Assisted Learning. 27, 2, 133--145.

[44]

G. F. Wilson. 2002. An analysis of mental workload in pilots during flight using multiple psychophysiological measures. The International Journal of Aviation Psychology 12, 1, 3--18.

[45]

J. Xu, Y. Wang, F. Chen, H. Choi, G. Li, and S. Chen et al. 2011. Pupillary response based cognitive workload index under luminance and emotional changes. In Proceedings of the ACM Conference on Human Factors in Computing Systems. 1627--1632.

Digital Library

[46]

B. Yin, F. Chen, N. Ruiz, and E. Ambikairajah. 2008. Speech-based cognitive load monitoring system. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. 2041--2044.

[47]

K. Yu, J. Epps, and F. Chen. 2013. Mental workload classification via online writing features. In Proceedings of the International Conference on Document Analysis 8 Recognition. 1110--1114.

Digital Library

Cited By

Alsharif AKhraiwish A(2024)Tools in Marketing Research: Exploring Emotional Responses to StimuliScientific Annals of Economics and Business10.47743/saeb-2024-000971:2(173-192)Online publication date: 4-Jun-2024
https://doi.org/10.47743/saeb-2024-0009
Dörner ABures MSimon MPirkl G(2024)Making Cognitive Ergonomics in the Human–Computer Interaction of Manufacturing Execution Systems Assessable: Experimental and Validation Approaches to Closing Research GapsMachines10.3390/machines1203019512:3(195)Online publication date: 16-Mar-2024
https://doi.org/10.3390/machines12030195
Alchieri LAbdalazim NAlecci LGashi SGjoreski MSantini S(2024)Lateralization Effects in Electrodermal Activity Data Collected Using Wearable DevicesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435418:1(1-30)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643541
Show More Cited By

Recommendations

Using Galvanic Skin Response (GSR) to Measure Trust and Cognitive Load in the Text-Chat Environment
CHI EA '15: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems

Exchanging text messages via software on smart phones and computers has recently become one of the most popular ways for people to communicate and accomplish their tasks. However, there are negative aspects to using this kind of software, for example, ...
Using galvanic skin response for cognitive load measurement in arithmetic and reading tasks
OzCHI '12: Proceedings of the 24th Australian Computer-Human Interaction Conference

Galvanic Skin Response (GSR) has recently attracted researchers' attention as a prospective physiological indicator of cognitive load and emotions. However, it has commonly been investigated through single or few measures and in one experimental ...
Galvanic skin response (GSR) as an index of cognitive load
CHI EA '07: CHI '07 Extended Abstracts on Human Factors in Computing Systems

Multimodal user interfaces (MMUI) allow users to control computers using speech and gesture, and have the potential to minimise users. experienced cognitive load, especially when performing complex tasks. In this paper, we describe our attempt to use a ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Interactive Intelligent Systems

ACM Transactions on Interactive Intelligent Systems Volume 7, Issue 3

September 2017

164 pages

ISSN:2160-6455

EISSN:2160-6463

DOI:10.1145/3143523

Editor:
Michelle Zhou
Juji, Inc., USA

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 September 2017

Accepted: 01 March 2017

Revised: 01 February 2017

Received: 01 December 2015

Published in TIIS Volume 7, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Australian Government
Department of Broadband, Communications and the Digital Economy and the Australian Research Council through the ICT Centre of Excellence program

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

56
Total Citations
View Citations
1,436
Total Downloads

Downloads (Last 12 months)167
Downloads (Last 6 weeks)29

Reflects downloads up to 21 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Alsharif AKhraiwish A(2024)Tools in Marketing Research: Exploring Emotional Responses to StimuliScientific Annals of Economics and Business10.47743/saeb-2024-000971:2(173-192)Online publication date: 4-Jun-2024
https://doi.org/10.47743/saeb-2024-0009
Dörner ABures MSimon MPirkl G(2024)Making Cognitive Ergonomics in the Human–Computer Interaction of Manufacturing Execution Systems Assessable: Experimental and Validation Approaches to Closing Research GapsMachines10.3390/machines1203019512:3(195)Online publication date: 16-Mar-2024
https://doi.org/10.3390/machines12030195
Alchieri LAbdalazim NAlecci LGashi SGjoreski MSantini S(2024)Lateralization Effects in Electrodermal Activity Data Collected Using Wearable DevicesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435418:1(1-30)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643541
Reddy AK KM V HSridevi VBegum B(2024)EmoClass: Subject-Independent Emotion Classification for BCI Systems Using EEG Signals2024 IEEE International Conference on Signal Processing, Informatics, Communication and Energy Systems (SPICES)10.1109/SPICES62143.2024.10779770(1-6)Online publication date: 20-Sep-2024
https://doi.org/10.1109/SPICES62143.2024.10779770
Shao SHan GWang TLin CSong CYao C(2024)EEG-Based Mental Workload Classification Method Based on Hybrid Deep Learning Model Under IoTIEEE Journal of Biomedical and Health Informatics10.1109/JBHI.2023.328179328:5(2536-2546)Online publication date: May-2024
https://doi.org/10.1109/JBHI.2023.3281793
Suryani MSantoso HSchrepp MAji RHadi SSensuse DSuryono RKautsarina (2024)Role, Methodology, and Measurement of Cognitive Load in Computer Science and Information Systems ResearchIEEE Access10.1109/ACCESS.2024.351435512(190007-190024)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3514355
Chakraborty SKiefer PRaubal M(2024)The influence of uncertainty visualization on cognitive load in a safety- and time-critical decision-making taskInternational Journal of Geographical Information Science10.1080/13658816.2024.234874738:8(1583-1610)Online publication date: 9-May-2024
https://doi.org/10.1080/13658816.2024.2348747
Qiao YGao XLiao WChen D(2024)Assessment of dynamic cognitive performance under different thermal conditions by multiple physiological signalsJournal of Building Engineering10.1016/j.jobe.2024.10932589(109325)Online publication date: Jul-2024
https://doi.org/10.1016/j.jobe.2024.109325
Balta EPsarrakis AVatakis A(2024)The effects of increased mental workload of air traffic controllers on time perception: Behavioral and physiological evidenceApplied Ergonomics10.1016/j.apergo.2023.104162115(104162)Online publication date: Feb-2024
https://doi.org/10.1016/j.apergo.2023.104162
Appriou ALotte F(2024)Riemannian ElectroCardioGraphic Signal ClassificationComputational Neuroscience10.1007/978-3-031-63848-0_3(25-39)Online publication date: 27-Jun-2024
https://doi.org/10.1007/978-3-031-63848-0_3
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents