More Web Proxy on the site http://driver.im/

research-article

Open access

The Impact of Different Levels of Autonomy and Training on Operators’ Drone Control Strategies

Authors:

Mary L. CummingsAuthors Info & Claims

ACM Transactions on Human-Robot Interaction (THRI), Volume 8, Issue 4

Article No.: 22, Pages 1 - 15

https://doi.org/10.1145/3344276

Published: 15 November 2019 Publication History

All formats PDF

Abstract

Unmanned Aerial Vehicles (UAVs), also known as drones, have extensive applications in civilian rescue and military surveillance realms. A common drone control scheme among such applications is human supervisory control, in which human operators remotely navigate drones and direct them to conduct high-level tasks. However, different levels of autonomy in the control system and different operator training processes may affect operators’ performance in task success rate and efficiency. An experiment was designed and conducted to investigate such potential impacts. The results showed us that a dedicated supervisory drone control interface tended toward increased operator successful task completion as compared to an enhanced teleoperation control interface, although this difference was not statistically significant. In addition, using Hidden Markov Models, operator behavior models were developed to further study the impact of operators’ drone control strategies as a function of differing levels of autonomy. These models revealed that people with both supervisory and enhanced teleoperation control training were not able to determine the right control action at the right time to the same degree that people with just training in the supervisory control mode. Future work is needed to determine how trust plays a role in such settings.

References

[1]

Leonard E. Baum and Ted Petrie. 1966. Statistical inference for probabilistic functions of finite state Markov chains. Ann. Math. Stat. 37, 6 (Dec. 1966), 1554--1563.

[2]

Yves Boussemart. 2011. Predictive Models of Procedural Human Supervisory Control Behavior. Ph.D. Dissertation. Massachusetts Institute of Technology, Cambridge, MA.

[3]

Yves Boussemart and M. L. Cummings. 2011. Predictive models of human supervisory control behavioral patterns using hidden semi-Markov models. Eng. Appl. Artific. Intell. 24, 7 (2011), 1252--1262. https://doi.org/10.1016/j.engappai.2011.04.008

Digital Library

[4]

Yves Boussemart, M. L. Cummings, Jonathan Las Fargeas, and Nicholas Roy. 2011. Supervised vs. Unsupervised learning for operator state modeling in unmanned vehicle settings. J. Aerospace Comput. Info. Commun. 8, 3 (2011), 71--85. https://doi.org/10.2514/1.46767

[5]

Jan Bredereke and Axel Lankenau. 2002. A rigorous view of mode confusion. In Proceedings of the Conference on Computer Safety, Reliability and Security (SAFECOMP’02), Stuart Anderson, Massimo Felici, and Sandro Bologna (Eds.), Vol. 2434. Springer, Berlin, 19--31. https://doi.org/10.1007/3-540-45732-1_4

[6]

M. L. Cummings, K. Jackson, P. Quimby, and D. Pitman. 2012. Field testing of a quad rotor smartphone control system. Int. J. Micro Air Vehicles 4, 3 (2012), 165--177. http://hdl.handle.net/1721.1/86945

[7]

Cristina Gómez and David R. Green. 2017. Small unmanned airborne systems to support oil and gas pipeline monitoring and mapping. Arab. J. Geosci. 10, 9 (May 2017), 202.

[8]

A. Gupta and B. Dhingra. 2012. Stock market prediction using hidden Markov models. In Proceedings of the Students Conference on Engineering and Systems. IEEE, Allahabad, Uttar Pradesh, India, 1--4.

[9]

Minwoo Kim. 2018. The Impact of Skill-based Training Across Different Levels of Autonomy for Drone Inspection Tasks. Master’s thesis. Duke University, Durham, NC.

[10]

Nataliya Kos’myna, Franck Tarpin-Bernard, and Bertrand Rivet. 2014. Bidirectional feedback in motor imagery BCIs: Learn to control a drone within 5 minutes. In CHI ’14 Extended Abstracts on Human Factors in Computing Systems (CHI EA’14). ACM, New York, NY.

[11]

Vianey Leos Barajas and Theo Michelot. 2018. An Introduction to Animal Movement Modeling with Hidden Markov Models using Stan for Bayesian Inference. arXiv: 1806.10639.

[12]

Dahai Liu, Elizabeth L. Blickensderfer, Nikolas D. Macchiarella, and Dennis A. Vincenzi. 2008. Transfer of training. In Human Factors in Simulation and Training, Peter A. Hancock, Dennis A. Vincenzi, John A. Wise, and Mustapha Mouloua (Eds.). CRC Press, Boca Raton, FL, 49--60.

[13]

Gonzalo Pajares. 2015. Overview and current status of remote sensing applications based on unmanned aerial vehicles (UAVs). Photogram. Eng. Remote Sens. 81, 4 (2015), 281--330. https://doi.org/10.14358/PERS.81.4.281

[14]

Raja Parasuraman and Thomas B. Sheridan. 2000. A model for types and levels of human interaction with automation. IEEE Trans. Syst. Man Cybernet. Part A: Syst. Hum. 30, 3 (May 2000), 286--297. http://dx.doi.org/10.1109/3468.844354

Digital Library

[15]

Alex Pentland and Andrew Liu. 1999. Modeling and prediction of human behavior. Neural Comput. 11, 1 (1999), 229--242.

Digital Library

[16]

David Pitman and M. L. Cummings. 2012. Collaborative exploration with a micro aerial vehicle: A novel interaction method for controlling a MAV with a hand-held device. Adv. Hum.-Comput. Interact. 2012 (2012), 18. http://dx.doi.org/10.1155/2012/768180

Digital Library

[17]

Julie C. Prinet, Andrew Terhune, and Nadine B. Sarter. 2016. Supporting dynamic re-planning in multiple Uav control: A comparison of three levels of automation. Proc. Hum. Factors Ergonom. Soc. Ann. Meet. 56, 1 (Dec. 2016), 423--427.

[18]

L. R. Rabiner. 1989. A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77, 2 (Feb. 1989), 257--286.

Digital Library

[19]

L. R. Rabiner and B. Juang. 1986. An introduction to hidden Markov models. IEEE ASSP Mag. 3, 1 (1986), 4--16.

[20]

Narayanan Ramanathan. 2006. Applications of Hidden Markov Models. Retrieved from http://www.cs.umd.edu/djacobs/CMSC828/ApplicationsHMMs.pdf.

[21]

R. Schaufele, L. Ding, Nick Miller, H. Barlett, Michael Lukacs, and D. Bhadra. 2017. FAA aerospace forecast: Fiscal years 2017--2037. Technical Report. Federal Aviation Administration, Washington DC.

[22]

Thomas B. Sheridan. 1992. Telerobotics, Automation, and Human Supervisory Control. MIT Press, Cambridge, MA.

[23]

Greg Zacharias and Mark Maybury. 2011. Operating next-generation remotely piloted aircraft for irregular warfare. Technical Report. United States Air Force Scientific Advisory Board, Washington, DC.

[24]

J. Zhang, J. Jung, G. Sohn, and M. Cohen. 2015. Thermal infrared inspection of roof insulation using unmanned aerial vehicles. Int. Arch. Photogram. Remote Sens. Spatial Info. Sci. XL-1/W4 (2015), 381--386.

[25]

Yong Zhang, Xiuxiao Yuan, Yi Fang, and Shiyu Chen. 2017. UAV low altitude photogrammetry for power line inspection. ISPRS Int. J. Geo-Info. 6, 1 (2017), 14. https://doi.org/10.3390/ijgi6010014

[26]

Haibei Zhu, Mary Cummings, Mahmoud Elfar, Ziyao Wang, and Miroslav Pajic. 2019. Operator strategy model development in UAV hacking detection. IEEE Trans. Hum.-Mach. Syst. (Feb. 2019), 1--10.

Cited By

Turovsky YSurovtsev AAlekseev VRybak L(2024)Monitoring the State of the Operator of the Ergatic System in UAV Control TasksMachines10.3390/machines1205029112:5(291)Online publication date: 26-Apr-2024
https://doi.org/10.3390/machines12050291
Cheng JMahmud SMohammed MSingh AKim J(2024)Design of a Novice-Friendly Drone Control System2024 IEEE 14th Annual Computing and Communication Workshop and Conference (CCWC)10.1109/CCWC60891.2024.10427804(0184-0190)Online publication date: 8-Jan-2024
https://doi.org/10.1109/CCWC60891.2024.10427804
French ACummings MZhu HPajic M(2024)Determining Novice and Expert Status in Human–Automation Interaction Through Hidden Markov ModelsApplied Artificial Intelligence10.1080/08839514.2024.240217438:1Online publication date: 10-Sep-2024
https://doi.org/10.1080/08839514.2024.2402174
Show More Cited By

Index Terms

The Impact of Different Levels of Autonomy and Training on Operators’ Drone Control Strategies
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
    2. HCI theory, concepts and models

Recommendations

Demonstration of wireless synchronisation methods in autonomously controlled fleet of drones

This paper describes methods used in synchronising the behaviour of multiple autonomous flight vehicles utilising custom external hardware and firmware injected into current aviation flight systems. To prove the reliability and consistency of an ...
Towards a Better Understanding of Drone Forensics: A Case Study of Parrot AR Drone 2.0

Unmanned aerial vehicles (drones) have gained increased popularity as their innovative uses continue to expand across various fields. Despite their numerous beneficial uses, drones have unfortunately been misused, through many reported cases, to ...
Oculus Rift Application for Training Drone Pilots
SIMUTOOLS '17: Proceedings of the 10th EAI International Conference on Simulation Tools and Techniques

The research described in this paper, focuses on a virtual reality headset system that integrates the Oculus Rift VR headset with a low cost Unmanned Aerial Vehicle (UAV) to allow for drone teleoperation and telepresence using the Robot Operating System ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Human-Robot Interaction

ACM Transactions on Human-Robot Interaction Volume 8, Issue 4

Survey Paper and Special Issue on Representation Learning for Human and Robot Cognition

December 2019

108 pages

EISSN:2573-9522

DOI:10.1145/3372354

Editors:
Odest Chadwicke Jenkins
University of Michigan, USA
,
Selma Sabanovic
Indiana University, USA

Issue’s Table of Contents

Copyright © 2019 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 November 2019

Accepted: 01 July 2019

Revised: 01 June 2019

Received: 01 July 2018

Published in THRI Volume 8, Issue 4

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Office of Naval Research under the Science of Autonomy program

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
2,072
Total Downloads

Downloads (Last 12 months)653
Downloads (Last 6 weeks)46

Reflects downloads up to 04 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Turovsky YSurovtsev AAlekseev VRybak L(2024)Monitoring the State of the Operator of the Ergatic System in UAV Control TasksMachines10.3390/machines1205029112:5(291)Online publication date: 26-Apr-2024
https://doi.org/10.3390/machines12050291
Cheng JMahmud SMohammed MSingh AKim J(2024)Design of a Novice-Friendly Drone Control System2024 IEEE 14th Annual Computing and Communication Workshop and Conference (CCWC)10.1109/CCWC60891.2024.10427804(0184-0190)Online publication date: 8-Jan-2024
https://doi.org/10.1109/CCWC60891.2024.10427804
French ACummings MZhu HPajic M(2024)Determining Novice and Expert Status in Human–Automation Interaction Through Hidden Markov ModelsApplied Artificial Intelligence10.1080/08839514.2024.240217438:1Online publication date: 10-Sep-2024
https://doi.org/10.1080/08839514.2024.2402174
Znakovska YAveryanova YOstroumov IZaliskyi MHolubnychyi OSushchenko OPogurelskiy OVoliansky R(2024)The Information Technologies Use for UAS Operators’ TrainingInformation Technology for Education, Science, and Technics10.1007/978-3-031-71804-5_22(327-338)Online publication date: 8-Oct-2024
https://doi.org/10.1007/978-3-031-71804-5_22
Lijcklama à Nijeholt LKronshorst TTeeffelen Kvan Manen BEmaus RKnotter JMersha A(2023)Utilizing Drone-Based Ground-Penetrating Radar for Crime Investigations in Localizing and Identifying Clandestine GravesSensors10.3390/s2316711923:16(7119)Online publication date: 11-Aug-2023
https://doi.org/10.3390/s23167119
Lee JYu K(2023)Wearable Drone Controller: Machine Learning-Based Hand Gesture Recognition and Vibrotactile FeedbackSensors10.3390/s2305266623:5(2666)Online publication date: 28-Feb-2023
https://doi.org/10.3390/s23052666
Popović KSchlafly MPrabhakar AKim CMurphey T(2023)Measuring Human-Robot Team Benefits Under Time Pressure in a Virtual Reality Testbed2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)10.1109/IROS55552.2023.10341794(5410-5417)Online publication date: 1-Oct-2023
https://doi.org/10.1109/IROS55552.2023.10341794
Ellwart TSchauffel N(2023)Human-Autonomy Teaming in Ship Inspection: Psychological Perspectives on the Collaboration Between Humans and Self-Governing SystemsSmart Ports and Robotic Systems10.1007/978-3-031-25296-9_18(343-362)Online publication date: 3-Apr-2023
https://doi.org/10.1007/978-3-031-25296-9_18
Li SCummings MWelton B(2022)Assessing the impact of autonomy and overconfidence in UAV first-person view trainingApplied Ergonomics10.1016/j.apergo.2021.10358098(103580)Online publication date: Jan-2022
https://doi.org/10.1016/j.apergo.2021.103580
Dönmez Özkan YMirnig AMeschtscherjakov ADemir CTscheligi M(2021)Mode Awareness Interfaces in Automated Vehicles, Robotics, and Aviation: A Literature Review13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3409118.3475125(147-158)Online publication date: 9-Sep-2021
https://dl.acm.org/doi/10.1145/3409118.3475125
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents