More Web Proxy on the site http://driver.im/

research-article

Open access

FLEX-SDK: An Open-Source Software Development Kit for Creating Social Robots

Authors:

Patricia Alves-Oliveira,

Elin A. Bjorling,

Maya CakmakAuthors Info & Claims

UIST '22: Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology

Article No.: 66, Pages 1 - 10

https://doi.org/10.1145/3526113.3545707

Published: 28 October 2022 Publication History

All formats PDF

Abstract

We present FLEX-SDK: an open-source software development kit that allows creating a social robot from two simple tablet screens. FLEX-SDK involves tools for designing the robot face and its facial expressions, creating screens for input/output interactions, controlling the robot through a Wizard-of-Oz interface, and scripting autonomous interactions through a simple text-based programming interface. We demonstrate how this system can be used to replicate an interaction study and we present nine case studies involving controlled experiments, observational studies, participatory design sessions, and outreach activities in which our tools were used by researchers and participants to create and interact with social robots. We discuss common observations and lessons learned from these case studies. Our work demonstrates the potential of FLEX-SDK to lower the barrier to entry for Human-Robot Interaction research.

Supplementary Material

MP4 File (uist22-95.mp4)

Supplemental video

Download
9.92 MB

References

[1]

Gopika Ajaykumar, Maureen Steele, and Chien-Ming Huang. 2021. A Survey on End-User Robot Programming. arXiv preprint arXiv:2105.01757(2021).

[2]

Sonya Alexandrova, Zachary Tatlock, and Maya Cakmak. 2015. RoboFlow: A flow-based visual programming language for mobile manipulation tasks. In 2015 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 5537–5544.

[3]

Patrícia Alves-Oliveira, Matthew Bavier, Samrudha Malandkar, Ryan Eldridge, Julie Sayigh, Elin A Björling, and Maya Cakmak. 2022. FLEXI: A Robust and Flexible Social Robot Embodiment Kit. In Designing Interactive Systems Conference. 1177–1191.

[4]

Patrícia Alves-Oliveira, Elin Björling, Patriya Wiesmann, Dwikat Heba, Simran Bhatia, Kai Mihata, and Maya Cakmak. 2022. Robots for Connection: A Co-Design Study with Adolescents. In (IEEE International Conference on Robot and Human Interactive Communication (Ro-Man)).

[5]

Patrícia Alves-Oliveira, Tanya Budhiraja, Samuel So, Raida Karim, Elin A Björling, and Maya Cakmak. 2022. Robot-Mediated Interventions for Youth Mental Health. Design for Health (2022).

[6]

Patrícia Alves-Oliveira, Alaina Orr, Elin A Björling, and Maya Cakmak. 2022. Connecting the Dots of Social Robot Design From Interviews With Robot Creators. Frontiers in Robotics and AI(2022), 127.

[7]

Elin A Björling, Honson Ling, Simran Bhatia, and Kimberly Dziubinski. 2020. The Experience and Effect of Adolescent to Robot Stress Disclosure: A Mixed-Methods Exploration. In International Conference on Social Robotics. Springer, 604–615.

Digital Library

[8]

Elin A Björling, Honson Ling, Simran Bhatia, and Jeff Matarrese. 2021. Sharing stressors with a social robot prototype: What embodiment do adolescents prefer?International Journal of Child-Computer Interaction 28 (2021), 100252.

[9]

Elin A Björling, Kyle Thomas, Emma J Rose, and Maya Cakmak. 2020. Exploring teens as robot operators, users and witnesses in the wild. Frontiers in Robotics and AI 7 (2020), 5.

[10]

Michael Jae-Yoon Chung, Justin Huang, Leila Takayama, Tessa Lau, and Maya Cakmak. 2016. Iterative design of a system for programming socially interactive service robots. In International Conference on Social Robotics. Springer, 919–929.

[11]

Chandan Datta, Chandimal Jayawardena, I Han Kuo, and Bruce A MacDonald. 2012. RoboStudio: A visual programming environment for rapid authoring and customization of complex services on a personal service robot. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2352–2357.

[12]

Sarah Gillet, Ronald Cumbal, André Pereira, José Lopes, Olov Engwall, and Iolanda Leite. 2021. Robot Gaze Can Mediate Participation Imbalance in Groups with Different Skill Levels. In Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction. 303–311.

Digital Library

[13]

D Glas, Satoru Satake, Takayuki Kanda, and Norihiro Hagita. 2012. An interaction design framework for social robots. In Robotics: Science and Systems, Vol. 7. 89.

[14]

Dylan F Glas, Takayuki Kanda, and Hiroshi Ishiguro. 2016. Human-Robot Interaction Design Using Interaction Composer: Eight Years of Lessons Learned. In The Eleventh ACM/IEEE International Conference on Human Robot Interaction. IEEE Press, 303–310.

[15]

Kelleher R Guerin, Colin Lea, Chris Paxton, and Gregory D Hager. 2015. A framework for end-user instruction of a robot assistant for manufacturing. In 2015 IEEE international conference on robotics and automation (ICRA). IEEE, 6167–6174.

[16]

Justin Huang and Maya Cakmak. 2016. Programming by Demonstration with User-Specified Perceptual Landmarks. arXiv preprint arXiv:1612.00565(2016).

[17]

Justin Huang, Tessa Lau, and Maya Cakmak. 2016. Design and evaluation of a rapid programming system for service robots. In 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 295–302.

[18]

Alisa Kalegina, Grace Schroeder, Aidan Allchin, Keara Berlin, and Maya Cakmak. 2018. Characterizing the design space of rendered robot faces. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction. 96–104.

Digital Library

[19]

Tino Lourens. 2004. TiViPE-Tino’s visual programming environment. In Computer Software and Applications Conference, 2004. COMPSAC 2004. Proceedings of the 28th Annual International. IEEE, 10–15.

[20]

Tino Lourens and Emilia Barakova. 2011. User-friendly robot environment for creation of social scenarios. In International Work-Conference on the Interplay between Natural and Artificial Computation. Springer, 212–221.

Digital Library

[21]

Hai Nguyen, Matei Ciocarlie, Kaijen Hsiao, and Charles C. Kemp. 2013. ROS Commander (ROSCo): Behavior creation for home robots. In Robotics and Automation (ICRA), 2013 IEEE International Conference on. IEEE, 467–474.

[22]

Chris Paxton, Andrew Hundt, Felix Jonathan, Kelleher Guerin, and Gregory D Hager. 2017. CoSTAR: Instructing collaborative robots with behavior trees and vision. In 2017 IEEE international conference on robotics and automation (ICRA). IEEE, 564–571.

Digital Library

[23]

David Porfirio, Allison Sauppé, Aws Albarghouthi, and Bilge Mutlu. 2018. Authoring and verifying human-robot interactions. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. 75–86.

Digital Library

[24]

Emmanuel Pot, Jérôme Monceaux, Rodolphe Gelin, and Bruno Maisonnier. 2009. Choregraphe: a graphical tool for humanoid robot programming. In RO-MAN 2009-The 18th IEEE International Symposium on Robot and Human Interactive Communication. IEEE, 46–51.

[25]

Finn Rietz, Alexander Sutherland, Suna Bensch, Stefan Wermter, and Thomas Hellström. 2021. WoZ4U: An Open-Source Wizard-of-Oz Interface for Easy, Efficient and Robust HRI Experiments. Frontiers in Robotics and AI 8 (2021).

[26]

Allison Sauppé and Bilge Mutlu. 2014. Design patterns for exploring and prototyping human-robot interactions. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems. ACM, 1439–1448.

Digital Library

[27]

Katie Winkle, Emmanuel Senft, and Séverin Lemaignan. 2021. LEADOR: A Method for End-to-End Participatory Design of Autonomous Social Robots. arXiv preprint arXiv:2105.01910(2021).

Cited By

Stegner LHwang YPorfirio DMutlu B(2024)Understanding On-the-Fly End-User Robot ProgrammingProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660721(2468-2480)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660721
Schiffer SLiu AGrollman DBroadbent EJu WSoh HWilliams T(2024)MoBi-LE - A Low-Cost 3D-printable Robot to Educate Children in Waste DisposalCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640680(955-959)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610978.3640680
Jamshad RHaripriyan ASonti ASimkins SRiek LGrollman DBroadbent EJu WSoh HWilliams T(2024)Taking Initiative in Human-Robot Action Teams: How Proactive Robot Behaviors Affect TeamworkCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640640(559-562)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610978.3640640

Index Terms

FLEX-SDK: An Open-Source Software Development Kit for Creating Social Robots
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interactive systems and tools
      1. User interface programming

Recommendations

GUI Robots: Using Off-the-Shelf Robots as Tangible Input and Output Devices for Unmodified GUI Applications
DIS '17: Proceedings of the 2017 Conference on Designing Interactive Systems

Traditional GUI applications provide limited support for tangible interaction, as most applications are not programmed to support tangible input, and most input devices do not provide haptic feedback. To address this limitation, we introduce GUI Robots, ...
Multi-touch interaction for tasking robots
HRI '10: Proceedings of the 5th ACM/IEEE international conference on Human-robot interaction

The objective is to develop a mobile human-robot interface that is optimized for multi-touch input. Our existing interface was designed for mouse and keyboard input and was later adopted for voice and touch interaction. A new multi-touch interface ...
PATI: a projection-based augmented table-top interface for robot programming
IUI '19: Proceedings of the 24th International Conference on Intelligent User Interfaces

As robots begin to provide daily assistance to individuals in human environments, their end-users, who do not necessarily have substantial technical training or backgrounds in robotics or programming, will ultimately need to program and "re-task" their ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '22: Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology

October 2022

1363 pages

ISBN:9781450393201

DOI:10.1145/3526113

Editors:
Maneesh Agrawala
Stanford University, USA
,
Jacob O. Wobbrock
University of Washington, USA
,
Eytan Adar
University of Michigan, USA
,
Vidya Setlur
Tableau Research, USA

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 October 2022

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Science Foundation

Conference

UIST '22

Sponsor:

UIST '22: The 35th Annual ACM Symposium on User Interface Software and Technology

October 29 - November 2, 2022

OR, Bend, USA

Acceptance Rates

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
1,368
Total Downloads

Downloads (Last 12 months)631
Downloads (Last 6 weeks)71

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Stegner LHwang YPorfirio DMutlu B(2024)Understanding On-the-Fly End-User Robot ProgrammingProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660721(2468-2480)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660721
Schiffer SLiu AGrollman DBroadbent EJu WSoh HWilliams T(2024)MoBi-LE - A Low-Cost 3D-printable Robot to Educate Children in Waste DisposalCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640680(955-959)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610978.3640680
Jamshad RHaripriyan ASonti ASimkins SRiek LGrollman DBroadbent EJu WSoh HWilliams T(2024)Taking Initiative in Human-Robot Action Teams: How Proactive Robot Behaviors Affect TeamworkCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640640(559-562)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610978.3640640

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents