More Web Proxy on the site http://driver.im/

short-paper

Assessment of Situation Awareness during Robotic Surgery using Multimodal Data

Authors:

Aurelien Lechappe,

Mathieu Chollet,

Jérôme Rigaud,

Caroline G.L. CaoAuthors Info & Claims

ICMI '20 Companion: Companion Publication of the 2020 International Conference on Multimodal Interaction

Pages 412 - 416

https://doi.org/10.1145/3395035.3425205

Published: 27 December 2020 Publication History

Abstract

The use of robotic surgical systems disrupts existing team dynamics inside operating rooms and constitutes a major challenge for the development of crucial non-technical skills such as situation awareness (SA). Techniques for assessing SA mostly rely on subjective assessments and questionnaires; few leverage multimodal measures combining physiological, behavioural, and subjective indicators. We propose a conceptual model relating SA with mental workload, stress and communication, supported by measurable behaviours and physiological signals. To validate this model, we collect subjective, behavioural, and physiological data from surgical teams performing radical prostatectomy using robotic surgical systems. Statistical analyses will be performed to establish relationships between SA, subjective assessment of stress and mental workload, communication processes, and the surgeons' physiological signals.

References

[1]

Brittany L Anderson-Montoya and Mark W Scerbo. 2019. Human Factors Psychology in Surgery. In Comprehensive Healthcare Simulation: Surgery and Surgical Subspecialties. Springer, 153--167.

[2]

Abdullatif Aydin, Nicholas Raison, Muhammad Shamim Khan, Prokar Dasgupta, and Kamran Ahmed. 2016. Simulation-based training and assessment in urological surgery. Nature Reviews Urology 13, 9 (2016), 503.

[3]

Jason J Braithwaite, Derrick G Watson, Robert Jones, and Mickey Rowe. 2013. A guide for analysing electrodermal activity (EDA) & skin conductance responses (SCRs) for psychological experiments. Psychophysiology 49, 1 (2013).

[4]

James C Byers. 1989. Traditional and raw task load index (TLX) correlations: Are paired comparisons necessary? Advances in Industrial Erfonomics and Safety l: Taylor and Francis. (1989).

[5]

Nancy J Cooke, R Stout, Eduardo Salas, et al. 2001. A knowledge elicitation approach to the measurement of team situation awareness. New trends in cooperative activities: Understanding system dynamics in complex environments (2001), 114--139.

[6]

Roberto et Buckingham Shum Simon Echeverria, Vanessa et Martinez-Maldonado. 2019. Vers une collaboration translucide: Donner du sens aux données d?un groupe multimodal. In Actes de la conférence CHI 2019 sur les facteurs humains dans les systèmes informatiques. 1--16.

[7]

M.R. Endsley. 1988. Situation awareness global assessment technique (SAGAT). In Proceedings of the IEEE 1988 National Aerospace and Electronics Conference. 789--795 vol.3. https://doi.org/10.1109/NAECON.1988.195097

[8]

Mica R Endsley. 1993. Situation awareness and workload-Flip sides of the same coin. In International Symposium on Aviation Psychology, 7th, Columbus, OH. 906--911.

[9]

Mica R Endsley. 1995. Measurement of situation awareness in dynamic systems. Human factors 37, 1 (1995), 65--84.

[10]

Mica R. Endsley. 1995. Toward a Theory of Situation Awareness in Dynamic Systems. 37, 1 (1995), 32--64. https://doi.org/10.1518/001872095779049543 Publisher: SAGE Publications Inc.

[11]

Mica R. Endsley. 2020. The Divergence of Objective and Subjective Situation Awareness: A Meta-Analysis. 14, 1 (2020), 34--53. https://doi.org/10.1177/1555343419874248 Publisher: SAGE Publications.

[12]

Raul Fernandez Rojas, Essam Debie, Justin Fidock, Michael Barlow, Kathryn Kasmarik, Sreenatha Anavatti, Matthew Garratt, and Hussein Abbass. 2019. Encephalographic Assessment of Situation Awareness in Teleoperation of Human-Swarm Teaming. In Neural Information Processing (Cham) (Communications in Computer and Information Science), Tom Gedeon, Kok Wai Wong, and Minho Lee (Eds.). Springer International Publishing, 530--539. https://doi.org/10.1007/978- 3-030--36808--1_58

[13]

Maik Friedrich, Maresa Biermann, Patrick Gontar, Marcus Biella, and Klaus Bengler. 2018. The influence of task load on situation awareness and control strategy in the ATC tower environment. 20, 2 (2018), 205--217. https://doi.org/10.1007/s10111-018-0464--4

[14]

Aimee K. Gardner, Matthew Kosemund, and Joseph Martinez. 2017. Examining the Feasibility and Predictive Validity of the SAGAT Tool to Assess Situation Awareness Among Medical Trainees. 12, 1 (2017), 17--21. https://doi.org/10.1097/SIH.0000000000000181

[15]

Sandra G Hart. 2006. NASA-task load index (NASA-TLX); 20 years later. In Proceedings of the human factors and ergonomics society annual meeting, Vol. 50. Sage publications Sage CA: Los Angeles, CA, 904--908.

[16]

Andrew N Healey, Shabnam Undre, Nick Sevdalis, Maria Koutantji, and Charles A Vincent. 2006. The complexity of measuring interprofessional teamwork in the operating theatre. Journal of interprofessional care 20, 5 (2006), 485--495.

[17]

Keith C Hendy. 1995. Situation awareness and workload: Birds of a feather? Defence and Civil Institute of Environmental Medicine.

[18]

Brittany E Holthausen, Zoe M Becerra, M David Keller, John M Ziriax, and Bruce N Walker. 2019. Performance in Noise: The Relationship Between Workload And Situation Awareness in Navy Tactical Teams. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 63. SAGE Publications Sage CA: Los Angeles, CA, 1585--1589.

[19]

Daniel Kahneman and Kahneman Daniel. 1973. Attention and Effort.

[20]

Reza Ul Karim. 2012. The Effect of Stress on Task Capacity and Situational Awareness. (2012). https://library.ndsu.edu/ir/handle/10365/26480 Accepted: 2017-09--12T21:02:07Z Publisher: North Dakota State University.

[21]

T Koester. 2019. Situation awareness and situation dependent behaviour adjustment in the maritime work domain. In Human-Centered Computing: Cognitive, Social, and Ergonomic Aspects. Vol. 3. #255.

[22]

Richard S. Lazarus. 1990. Theory-Based Stress Measurement. 1, 1 (1990), 3-- 13. https://doi.org/10.1207/s15327965pli0101_1 Publisher: Routledge _eprint: https://doi.org/10.1207/s15327965pli0101_1.

[23]

Yeong Heok Lee, Jeong-Dae Jeon, and Youn-Chul Choi. 2012. Air Traffic Con- trollers? Situation Awareness and Workload under Dynamic Air Traffic Situations. 51, 3 (2012), 338--352. https://doi.org/10.5325/transportationj.51.3.0338 Publisher: Penn State University Press.

[24]

Theresa M. Marteau and Hilary Bekker. 1992. The development of a six-item short- form of the state scale of the Spielberger State?Trait Anxiety Inventory (STAI). 31, 3 (1992), 301--306. https://doi.org/10.1111/j.2044--8260.1992.tb00997.x_eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.2044--8260.1992.tb00997.x.

[25]

Ranjana K. Mehta, S. Camille Peres, Ashley E. Shortz, Wimberly Hoyle, Melissa Lee, Gurtej Saini, Hong-Chih Chan, and Mitchell W. Pryor. 2018. Operator situa- tion awareness and physiological states during offshore well control scenarios. 55 (2018), 332--337. https://doi.org/10.1016/j.jlp.2018.07.010

[26]

Lucy Mitchell, Rhona Flin, Steven Yule, Janet Mitchell, Kathy Coutts, and George Youngson. 2013. Development of a behavioural marker system for scrub practi- tioners? non-technical skills (SPLINTS system). Journal of evaluation in clinical practice 19, 2 (2013), 317--323.

[27]

Albert A. Nofi. 2000. Defining and Measuring Shared Situational Awareness. https://apps.dtic.mil/sti/citations/ADA390136 Section: Technical Reports.

[28]

Avi Parush, Chelsea Kramer, Tara Foster-Hunt, Kathryn Momtahan, Aren Hunter, and Benjamin Sohmer. 2011. Communication and team situation awareness in the OR: Implications for augmentative information display. 44, 3 (2011), 477--485. https://doi.org/10.1016/j.jbi.2010.04.002

Digital Library

[29]

Jason C. Pradarelli, Steven Yule, and Douglas S. Smink. 2020. Evaluating Non- Technical Skills in Surgery. In Health Services Research, Justin B. Dimick and Carrie C. Lubitz (Eds.). Springer International Publishing, 125--135. https://doi.org/10.1007/978--3-030--28357--5_12

[30]

Tom Price, Matthew Tenan, James Head, William Maslin, and Michael LaFiandra. 2016. Acute stress causes over confidence in situation awareness. In 2016 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA). IEEE, 1--6.

[31]

Nicholas Raison, Thomas Wood, Oliver Brunckhorst, Takashige Abe, Talisa Ross, Ben Challacombe, Mohammed Shamim Khan, Giacomo Novara, Nicolo Buffi, Henk Van Der Poel, et al. 2017. Development and validation of a tool for non- technical skills evaluation in robotic surgery?the ICARS system. Surgical en- doscopy 31, 12 (2017), 5403--5410.

[32]

R. Randell, S. A. Honey, J. Hindmarsh, N. Alvarado, J. Greenhalgh, A. Pearman, A. Long, A. Cope, A. Gill, P. Gardner, A. Kotze, D. Wilkinson, D. Jayne, J. Croft, and D. Dowding. 2017. A realist process evaluation of robot-assisted surgery: integration into routine practice and impacts on communication, collaboration and decision- making. 5, 20 (2017). https://www.journalslibrary.nihr.ac.uk/hsdr/hsdr05200/#/ abstract Number: 20 Publisher: National Institute for Health Research.

[33]

Eduardo Salas, Carolyn Prince, David P Baker, and Lisa Shrestha. 1995. Situation awareness in team performance: Implications for measurement and training. Human factors 37, 1 (1995), 123--136.

[34]

Christian M Schulz, Mica R Endsley, Eberhard F Kochs, Adrian W Gelb, and Klaus J Wagner. 2013. Situation Awareness in AnesthesiaConcept and Research. Anesthesiology: The Journal of the American Society of Anesthesiologists 118, 3 (2013), 729--742.

[35]

Stephen J Selcon and RM Taylor. 1990. Evaluation of the Situational Awareness Rating Technique(SART) as a tool for aircrew systems design. AGARD, Situational Awareness in Aerospace Operations 8 p(SEE N 90--28972 23--53) (1990).

[36]

Fred Shaffer and JP Ginsberg. 2017. An overview of heart rate variability metrics and norms. Frontiers in public health 5 (2017), 258.

[37]

Kip Smith and P. A. Hancock. 1995. Situation Awareness Is Adaptive, Exter- nally Directed Consciousness. 37, 1 (1995), 137--148. https://doi.org/10.1518/ 001872095779049444 Publisher: SAGE Publications Inc.

[38]

Renée J Stout, Janis A Cannon-Bowers, Eduardo Salas, and Dana M Milanovich. 1999. Planning, shared mental models, and coordinated performance: An empiri- cal link is established. Human Factors 41, 1 (1999), 61--71.

[39]

Geoffrey Tien, M Stella Atkins, Bin Zheng, and Colin Swindells. 2010. Measuring situation awareness of surgeons in laparoscopic training. In Proceedings of the 2010 symposium on eye-tracking research & applications. 149--152.

Digital Library

[40]

Michael A Vidulich. 2000. The relationship between mental workload and situa- tion awareness. In Proceedings of the human factors and ergonomics society annual meeting, Vol. 44. SAGE Publications Sage CA: Los Angeles, CA, 3--460.

[41]

Ana Wheelock, Amna Suliman, Rupert Wharton, ED Babu, Louise Hull, Charles Vincent, Nick Sevdalis, and Sonal Arora. 2015. The impact of operating room distractions on stress, workload, and teamwork. Annals of surgery 261, 6 (2015), 1079--1084.

[42]

S Yule, Rhona Flin, S Paterson-Brown, N Maran, and D Rowley. 2006. Development of a rating system for surgeons? non-technical skills. Medical education 40, 11 (2006), 1098--1104.

[43]

S Yule, A Gupta, D Gazarian, A Geraghty, DS Smink, J Beard, T Sundt, G Youngson, C McIlhenny, and S Paterson-Brown. 2018. Construct and criterion validity testing of the Non-Technical Skills for Surgeons (NOTSS) behaviour assessment tool using videos of simulated operations. British Journal of Surgery 105, 6 (2018), 719--727.

[44]

Yuliya Y. Yurko, Mark W. Scerbo, Ajita S. Prabhu, Christina E. Acker, and Dimitrios Stefanidis. 2010. Higher Mental Workload is Associated With Poorer Laparoscopic Performance as Measured by the NASA-TLX Tool. 5, 5 (2010), 267--271. https://doi.org/10.1097/SIH.0b013e3181e3f329

Cited By

AlNaim AAlNaim NAl Nasser FAlbash LAlmulhim MAlbinsaad L(2024)Public’s Perception and Knowledge of Using Robotics in General Surgery in Eastern Region, Saudi ArabiaAnnals of African Medicine10.4103/aam.aam_182_2323:3(299-306)Online publication date: 20-Jul-2024
https://doi.org/10.4103/aam.aam_182_23
Takács KLukács ELevendovics RPekli DSzijártó AHaidegger T(2024)Assessment of Surgeons’ Stress Levels with Digital Sensors during Robot-Assisted Surgery: An Experimental StudySensors10.3390/s2409291524:9(2915)Online publication date: 2-May-2024
https://doi.org/10.3390/s24092915
Liu TWang JWong SRazjigaev ABeier SPeng SDo TSong SChu DWang CLovell NWu L(2024)A Review on the Form and Complexity of Human–Robot Interaction in the Evolution of Autonomous SurgeryAdvanced Intelligent Systems10.1002/aisy.2024001976:11Online publication date: 9-Jul-2024
https://doi.org/10.1002/aisy.202400197
Show More Cited By

Index Terms

Assessment of Situation Awareness during Robotic Surgery using Multimodal Data

Index terms have been assigned to the content through auto-classification.

Recommendations

Impacts of Telemanipulation in Robotic Assisted Surgery
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Robotic-assisted Minimally Invasive Surgery (MIS) is adopted more and more as it overcomes the shortcomings of classic MIS for surgeons while keeping the benefits of small incisions for patients. However, introducing new technology oftentimes affects ...
Stress recognition in human-computer interaction using physiological and self-reported data: a study of gender differences
PCI '15: Proceedings of the 19th Panhellenic Conference on Informatics

This paper investigates gender differences in stress recognition in human computer interaction (HCI) for both objective (i.e., skin conductance) and subjective (i.e., valence-arousal VA ratings) metrics. To this end, 31 healthy participants, 18 females, ...
Robotic surgery: From autonomous systems to intelligent tools

A brief history of robotic surgery is provided, which describes the transition from autonomous robots to hands-on systems that are under the direct control of the surgeon. An example of the latter is the Acrobot (for active-constraint robot) system used ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '20 Companion: Companion Publication of the 2020 International Conference on Multimodal Interaction

October 2020

548 pages

ISBN:9781450380027

DOI:10.1145/3395035

General Chairs:
Khiet Truong
University of Twente, the Netherlands
,
Dirk Heylen
University of Twente, the Netherlands
,
Mary Czerwinski
Microsoft Research, USA
,
Program Chairs:
Nadia Berthouze
University College London, United Kingdom
,
Mohamed Chetouani
Sorbonne University, France
,
Mikio Nakano
C4A Research Institute, Japan

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 December 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

ICMI '20

Sponsor:

SIGCHI

ICMI '20: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

October 25 - 29, 2020

Virtual Event, Netherlands

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
236
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)2

Reflects downloads up to 10 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

AlNaim AAlNaim NAl Nasser FAlbash LAlmulhim MAlbinsaad L(2024)Public’s Perception and Knowledge of Using Robotics in General Surgery in Eastern Region, Saudi ArabiaAnnals of African Medicine10.4103/aam.aam_182_2323:3(299-306)Online publication date: 20-Jul-2024
https://doi.org/10.4103/aam.aam_182_23
Takács KLukács ELevendovics RPekli DSzijártó AHaidegger T(2024)Assessment of Surgeons’ Stress Levels with Digital Sensors during Robot-Assisted Surgery: An Experimental StudySensors10.3390/s2409291524:9(2915)Online publication date: 2-May-2024
https://doi.org/10.3390/s24092915
Liu TWang JWong SRazjigaev ABeier SPeng SDo TSong SChu DWang CLovell NWu L(2024)A Review on the Form and Complexity of Human–Robot Interaction in the Evolution of Autonomous SurgeryAdvanced Intelligent Systems10.1002/aisy.2024001976:11Online publication date: 9-Jul-2024
https://doi.org/10.1002/aisy.202400197
Allemang--Trivalle A(2023)Enhancing Surgical Team Collaboration and Situation Awareness through Multimodal SensingProceedings of the 25th International Conference on Multimodal Interaction10.1145/3577190.3614233(716-720)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3577190.3614233
Küster DPutze FSt-Onge DFortin PUrrestilla NSchultz T(2021)3rd Workshop on Modeling Socio-Emotional and Cognitive Processes from Multimodal Data in the WildProceedings of the 2021 International Conference on Multimodal Interaction10.1145/3462244.3480978(860-861)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3462244.3480978
Milef NRyason AQi DAlfred SJackson CDe S(2021)Disruptions to shared mental models from poor quality of service in collaborative virtual environmentsScientific Reports10.1038/s41598-021-02567-711:1Online publication date: 7-Dec-2021
https://doi.org/10.1038/s41598-021-02567-7
Lechappe AChollet MRigaud JCao C(2021)Using Multimodal Data to Predict Surgeon Situation AwarenessProceedings of the 21st Congress of the International Ergonomics Association (IEA 2021)10.1007/978-3-030-74614-8_37(308-316)Online publication date: 13-Jun-2021
https://doi.org/10.1007/978-3-030-74614-8_37

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents