[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3379337.3415872acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article

Authr: A Task Authoring Environment for Human-Robot Teams

Published: 20 October 2020 Publication History

Abstract

Collaborative robots promise to transform work across many industries and promote human-robot teaming as a novel paradigm. However, realizing this promise requires the understanding of how existing tasks, developed for and performed by humans, can be effectively translated into tasks that robots can singularly or human-robot teams can collaboratively perform. In the interest of developing tools that facilitate this process we present Authr, an end-to-end task authoring environment that assists engineers at manufacturing facilities in translating existing manual tasks into plans applicable for human-robot teams and simulates these plans as they would be performed by the human and robot. We evaluated Authr with two user studies, which demonstrate the usability and effectiveness of Authr as an interface and the benefits of assistive task allocation methods for designing complex tasks for human-robot teams. We discuss the implications of these findings for the design of software tools for authoring human-robot collaborative plans.

Supplementary Material

VTT File (ufp7124pv.vtt)
VTT File (ufp7124vf.vtt)
VTT File (3379337.3415872.vtt)
SRT File (ufp7124pvc.srt)
Preview video captions
SRT File (ufp7124vfc.srt)
Video figure captions
MOV File (ufp7124vf.mov)
Video figure
MP4 File (ufp7124pv.mp4)
Preview video
MP4 File (3379337.3415872.mp4)
Presentation Video

References

[1]
H. Akrout, D. Anson, G. Bianchini, A. Neveur, C. Trinel, M. Farnsworth, and T. Tomiyama. 2013. Maintenance Task Classification: Towards Automated Robotic Maintenance for Industry. Procedia CIRP 11 (2013), 367 -- 372.
[2]
Sonya Alexandrova, Zachary Tatlock, and Maya Cakmak. 2015. Roboflow: A Flow-Based Visual Programming Language for Mobile Manipulation Tasks. In 2015 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 5537--5544.
[3]
Aaron Bangor, Philip Kortum, and James Miller. 2008. An Empirical Evaluation of the System Usability Scale. Intl. Journal of Human-Computer Interaction 24, 6 (2008), 574--594.
[4]
Mario Bollini, Stefanie Tellex, Tyler Thompson, Nicholas Roy, and Daniela Rus. 2013. Interpreting and Executing Recipes with a Cooking Robot. In Experimental Robotics. Springer, 481--495.
[5]
John Brooke. 1996. SUS-A Quick and Dirty Usability Scale. Usability Evaluation in Industry 189, 194 (1996), 4--7.
[6]
Guido Bugmann, Ewan Klein, Stanislao Lauria, and Theocharis Kyriacou. 2004. Corpus-Based Robotics: A Route Instruction Example. In Proceedings of Intelligent Autonomous Systems. 96--103.
[7]
Sachin Chitta, Ioan Sucan, and Steve Cousins. 2012. Moveit![ros topics]. IEEE Robotics & Automation Magazine 19, 1 (2012), 18--19.
[8]
Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Springer, 337--340.
[9]
Johann Dréo, Pierre Savéant, Marc Schoenauer, and Vincent Vidal. 2011. Divide-and-Evolve: The Marriage of Descartes and Darwin. Proceedings of the 7th International Planning Competition (IPC). Freiburg, Germany 91 (2011), 155.
[10]
Shirine El Zaatari, Mohamed Marei, Weidong Li, and Zahid Usman. 2019. Cobot Programming for Collaborative Industrial Tasks: An Overview. Robotics and Autonomous Systems 116 (2019), 162--180.
[11]
K. Anders Ericsson and Herbert Simon. 1998. How to Study Thinking in Everyday Life: Contrasting Think-Aloud Protocols with Descriptions and Explanations of Thinking. Mind, Culture, and Activity 5, 3 (1998), 178--186.
[12]
C. Ailie Fraser, Tovi Grossman, and George Fitzmaurice. 2017. WeBuild: Automatically Distributing Assembly Tasks Among Collocated Workers to Improve Coordination. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 1817--1830.
[13]
Frank Gilbreth and Lilian Gilbreth. 1924. Classifying the Elements of Work. Management and Administration 8, 2 (1924), 151--154.
[14]
Matthew Gombolay, Ronald Wilcox, and Julie Shah. 2018. Fast Scheduling of Robot Teams Performing Tasks With Temporospatial Constraints. IEEE Transactions on Robotics 34, 1 (2018), 220--239.
[15]
Google. 2019. Angular. (Jan 2019). https://angular.io/
[16]
Kelleher Guerin, Colin Lea, Chris Paxton, and Gregory Hager. 2015. A Framework for End-User Instruction of a Robot Assistant for Manufacturing. In Robotics and Automation (ICRA), 2015 IEEE International Conference on. IEEE, 6167--6174.
[17]
Sandra Hart and Lowell Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Advances in Psychology. Vol. 52. Elsevier, 139--183.
[18]
Justin Huang. 2017. Enabling Rapid End-to-End Programming of Mobile Manipulators. In Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction. ACM, 343--344.
[19]
Jun Kato, Daisuke Sakamoto, Takeo Igarashi, and Masataka Goto. 2014. Sharedo: To-do List Interface for Human-agent Task Sharing. In Proceedings of the Second International Conference on Human-agent Interaction (HAI '14). ACM, New York, NY, USA, 345--351.
[20]
Mostepha Khouadjia, Marc Schoenauer, Vincent Vidal, Johann Dréo, and Pierre Savéant. 2013. Pareto-Based Multiobjective AI Planning. In International Joint Conference on Artificial Intelligence. AAAI.
[21]
Seung Han Kim and Jae Wook Jeon. 2007. Programming LEGO Mindstorms NXT with Visual Programming. In 2007 International Conference on Control, Automation and Systems. 2468--2472.
[22]
Frank Klassner and Scott Anderson. 2003. LEGO MindStorms: Not Just for K-12 Anymore. IEEE Robotics Automation Magazine 10, 2 (June 2003), 12--18.
[23]
Seung kook Jun, Madusudanan Narayanan, Pooja Agarwal, Abeer Eddib, Pragya Singhal, Satyanarayana Garimella, and Venkat Krovi. 2012. Robotic Minimally Invasive Surgical Skill Assessment Based on Automated Video-Analysis Motion Studies. 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob) (2012), 25--31.
[24]
Hsien-I Lin and Chia-Hsien Cheng. 2014. Behavior-Based Manipulator Programming Based on Extensible Agent Behavior Specification Language. In 2014 14th International Conference on Control, Automation and Systems (ICCAS 2014). 808--813.
[25]
Hsien-I Lin and YP Chiang. 2015. Understanding Human Hand Gestures for Learning Robot Pick-and-Place Tasks. International Journal of Advanced Robotic Systems 12, 5 (2015), 49.
[26]
Arnold Lund. 2001. Measuring Usability with the USE Questionnaire. Usability and User Experience Newsletter of the STC Usability SIG 8 (01 2001).
[27]
John Maloney, Mitchel Resnick, Natalie Rusk, Brian Silverman, and Evelyn Eastmond. 2010. The Scratch Programming Language and Environment. ACM Transactions on Computing Education (TOCE) 10, 4 (2010), 16.
[28]
Carlos Mateo, Alberto Brunete, Ernesto Gambao, and Miguel Hernando. 2014. Hammer: An Android Based Application for End-User Industrial Robot Programming. In 2014 IEEE/ASME 10th International Conference on Mechatronic and Embedded Systems and Applications (MESA). IEEE, 1--6.
[29]
Joseph Michaelis, Amanda Siebert-Evenstone, David Shaffer, and Bilge Mutlu. 2020. Collaborative or Simply Uncaged? Understanding Human-Cobot Interactions in Automation. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). 1--12.
[30]
George Michalos, Sotiris Makris, Jason Spiliotopoulos, Ioannis Misios, Panagiota Tsarouchi, and George Chryssolouris. 2014. ROBO-PARTNER: Seamless Human-Robot Cooperation for Intelligent, Flexible and Safe Operations in the Assembly Factories of the Future. Procedia CIRP 23 (2014), 71--76.
[31]
George Michalos, Sotiris Makris, Panagiota Tsarouchi, Toni Guasch, Dimitris Kontovrakis, and George Chryssolouris. 2015. Design Considerations for Safe Human-Robot Collaborative Workplaces. Procedia CIRP 37 (2015), 248--253.
[32]
Arne Muxfeldt, Jan-Henrik Kluth, and Daniel Kubus. 2014. Kinesthetic Teaching in Assembly Operations--A User Study. In International Conference on Simulation, Modeling, and Programming for Autonomous Robots. Springer, 533--544.
[33]
Hai Nguyen, Matei Ciocarlie, Kaijen Hsiao, and Charles Kemp. 2013. ROS Commander (ROSCO): Behavior Creation for Home Robots. In 2013 IEEE International Conference on Robotics and Automation. IEEE, 467--474.
[34]
Chris Paxton, Andrew Hundt, Felix Jonathan, Kelleher Guerin, and Gregory Hager. 2017. CoSTAR: Instructing Collaborative Robots with Behavior Trees and Vision. In Robotics and Automation (ICRA), 2017 IEEE International Conference on. IEEE, 564--571.
[35]
Chris Paxton, Felix Jonathan, Andrew Hundt, Bilge Mutlu, and Gregory D Hager. 2018. Evaluating Methods for End-User Creation of Robot Task Plans. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 6086--6092.
[36]
Margaret Pearce, Bilge Mutlu, Julie Shah, and Robert Radwin. 2018. Optimizing Makespan and Ergonomics in Integrating Collaborative Robots into Manufacturing Processes. IEEE Transactions on Automation Science and Engineering 99 (2018), 1--13.
[37]
Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, and Andrew Y Ng. 2009. ROS: An Open-Source Robot Operating System. In ICRA Workshop on Open Source Software, Vol. 3. Kobe, Japan, 5.
[38]
Dominik Riedelbauch and Dominik Henrich. 2018. Fast Graphical Task Modelling for Flexible Human-Robot Teaming. In ISR 2018; 50th International Symposium on Robotics. 1--6.
[39]
Allison Sauppé and Bilge Mutlu. 2014. Design Patterns for Exploring and Prototyping Human-robot Interactions. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 1439--1448.
[40]
Alexander Skoglund, Boyko Iliev, Bourhane Kadmiry, and Rainer Palm. 2007. Programming by Demonstration of Pick-and-Place Tasks for Industrial Manipulators using Task Primitives. In 2007 International Symposium on Computational Intelligence in Robotics and Automation. IEEE, 368--373.
[41]
Neville Stanton. 2006. Hierarchical Task Analysis: Developments, Applications, and Extensions. Applied Ergonomics 37, 1 (2006), 55--79.
[42]
Frank Steinmetz, Verena Nitsch, and Freek Stulp. 2019. Intuitive Task-Level Programming by Demonstration Through Semantic Skill Recognition. IEEE Robotics and Automation Letters 4, 4 (Oct 2019), 3742--3749.
[43]
Frank Steinmetz, Annika Wollschl"ager, and Roman Weitschat. 2018. RAZER-A HRI for Visual Task-Level Programming and Intuitive Skill Parameterization. IEEE Robotics and Automation Letters 3, 3 (July 2018), 1362--1369.
[44]
Ioan cS ucan, Mark Moll, and Lydia Kavraki. 2012. The Open Motion Planning Library. IEEE Robotics & Automation Magazine 19, 4 (December 2012), 72--82.
[45]
Russell Toris, Julius Kammerl, David Lu, Jihoon Lee, Odest Chadwicke Jenkins, Sarah Osentoski, Mitchell Wills, and Sonia Chernova. 2015. Robot Web Tools: Efficient Messaging for Cloud Robotics. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 4530--4537.
[46]
Maartin Van Someren, Yvonne Barnard, and Jacobijn Sandberg. 1994. The Think Aloud Method : A Practical Guide to Modelling Cognitive Processes. London: AcademicPress (1994).
[47]
Chadwick Wingrave and Joseph LaViola. 2010. Reflecting on the Design and Implementation Issues of Virtual Environments. Presence 19, 2 (April 2010), 179--195.
[48]
Chongjie Zhang and Julie Shah. 2016. Co-optimizating Multi-agent Placement with Task Assignment and Scheduling. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI'16). AAAI Press, 3308--3314.

Cited By

View all
  • (2024)Therblig to RobotSPOOL10.47982/spool.2024.1.0311:1(49-64)Online publication date: 1-Jul-2024
  • (2024)Goal-Oriented End-User Programming of RobotsProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634974(582-591)Online publication date: 11-Mar-2024
  • (2023)Measuring Temporal Awareness for Human-Aware AIProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/2169506723119263567:1(1817-1823)Online publication date: 25-Oct-2023
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
UIST '20: Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology
October 2020
1297 pages
ISBN:9781450375146
DOI:10.1145/3379337
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 October 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. authoring
  2. human-robot collaboration
  3. task allocation
  4. visual programming

Qualifiers

  • Research-article

Funding Sources

  • National Science Foundation

Conference

UIST '20

Acceptance Rates

Overall Acceptance Rate 432 of 1,968 submissions, 22%

Upcoming Conference

UIST '25
The 38th Annual ACM Symposium on User Interface Software and Technology
September 28 - October 1, 2025
Busan , Republic of Korea

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)70
  • Downloads (Last 6 weeks)8
Reflects downloads up to 19 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Therblig to RobotSPOOL10.47982/spool.2024.1.0311:1(49-64)Online publication date: 1-Jul-2024
  • (2024)Goal-Oriented End-User Programming of RobotsProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634974(582-591)Online publication date: 11-Mar-2024
  • (2023)Measuring Temporal Awareness for Human-Aware AIProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/2169506723119263567:1(1817-1823)Online publication date: 25-Oct-2023
  • (2023)Visual Programming of Robot Tasks with Product and Process VarietyAnnals of Scientific Society for Assembly, Handling and Industrial Robotics 202210.1007/978-3-031-10071-0_20(241-252)Online publication date: 11-Jul-2023
  • (2023)Analysis of quality standards for industrial collaborative robots based on user‐centered design frameworkHuman Factors and Ergonomics in Manufacturing & Service Industries10.1002/hfm.2101434:2(100-117)Online publication date: 26-Sep-2023
  • (2022)CoFrameProceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction10.5555/3523760.3523788(185-194)Online publication date: 7-Mar-2022
  • (2022)Human Robot Collaboration for Enhancing Work ActivitiesHuman Factors: The Journal of the Human Factors and Ergonomics Society10.1177/0018720822107772266:1(158-179)Online publication date: 28-Mar-2022
  • (2022)Mimic: In-Situ Recording and Re-Use of Demonstrations to Support Robot TeleoperationProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545639(1-13)Online publication date: 29-Oct-2022
  • (2022)CoFrame: A System for Training Novice Cobot Programmers2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI)10.1109/HRI53351.2022.9889345(185-194)Online publication date: 7-Mar-2022
  • (2021)Situated Live Programming for Human-Robot CollaborationThe 34th Annual ACM Symposium on User Interface Software and Technology10.1145/3472749.3474773(613-625)Online publication date: 10-Oct-2021
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media