More Web Proxy on the site http://driver.im/

research-article

Open access

Robiot: A Design Tool for Actuating Everyday Objects with Automatically Generated 3D Printable Mechanisms

Authors:

Xiang 'Anthony' ChenAuthors Info & Claims

UIST '19: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology

Pages 673 - 685

https://doi.org/10.1145/3332165.3347894

Published: 17 October 2019 Publication History

Abstract

Users can now easily communicate digital information with an Internet of Things; in contrast, there remains a lack of support to automate physical tasks that involve legacy static objects, e.g. adjusting a desk lamp's angle for optimal brightness, turning on/off a manual faucet when washing dishes, sliding a window to maintain a preferred indoor temperature. Automating these simple physical tasks has the potential to improve people's quality of life, which is particularly important for people with a disability or in situational impairment.

We present Robiot -- a design tool for generating mechanisms that can be attached to, motorized, and actuating legacy static objects to perform simple physical tasks. Users only need to take a short video manipulating an object to demonstrate an intended physical behavior. Robiot then extracts requisite parameters and automatically generates 3D models of the enabling actuation mechanisms by performing a scene and motion analysis of the 2D video in alignment with the object's 3D model. In an hour-long design session, six participants used Robiot to actuate seven everyday objects, imbuing them with the robotic capability to automate various physical tasks.

Supplementary Material

MP4 File (ufp3692pv.mp4)

Preview video

Download
11.65 MB

MP4 File (p673-li.mp4)

Download
482.08 MB

References

[1]

Gift a lot. 2019. Smart Wifi Electric Curtains Tracks and Motorized Roller Blinds. https://giftalot.com/. (2019). (Accessed on 04/01/2019).

[2]

N. S. Altman. 1992. An introduction to kernel and nearest-neighbor nonparametric regression. American Statistician (1992).

[3]

Rahul Arora, Rubaiat Habib Kazi, Tovi Grossman, George Fitzmaurice, and Karan Singh. 2018. SymbiosisSketch. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems CHI '18. ACM Press, New York, New York, USA, 1--15.

[4]

Daniel Ashbrook, Shitao Stan Guo, and Alan Lambie. 2016. Towards augmented fabrication: Combining fabricated and existing objects. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 1510--1518.

Digital Library

[5]

M Bacher, S Coros, and B Thomaszewski. 2015. LinkEdit: Interactive Linkage Editing using Symbolic Kinematics. Acm Transactions on Graphics (2015).

Digital Library

[6]

Aude Billard, Sylvain Calinon, Ruediger Dillmann, and Stefan Schaal. 2008. Robot programming by demonstration. Springer handbook of robotics (2008), 1371--1394.

[7]

Mayara Bonani, Raquel Oliveira, Filipa Correia, André Rodrigues, Tiago Guerreiro, and Ana Paiva. 2018. What My Eyes Can't See, A Robot Can Show Me. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility -ASSETS '18. ACM Press, New York, New York, USA, 15--27.

Digital Library

[8]

Xiang 'Anthony' Chen, Stelian Coros, Jennifer Mankoff, and Scott E Hudson. 2015. Encore: 3D printed augmentation of everyday objects with printed-over, affixed and interlocked attachments. In Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology. ACM, 73--82.

[9]

Xiang 'Anthony' Chen, Jeeeun Kim, Jennifer Mankoff, Tovi Grossman, Stelian Coros, and Scott E Hudson. 2016. Reprise: A Design Tool for Specifying, Generating, and Customizing 3D Printable Adaptations on Everyday Objects. In the 29th Annual ACM Symposium on User Interface Software and Technology. ACM.

[10]

Xiang 'Anthony' Chen, Ye Tao, Guanyun Wang, Runchang Kang, Tovi Grossman, Stelian Coros, and Scott E Hudson. 2018. Forte: User-Driven Generative Design. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 496.

Digital Library

[11]

Gabe Cohn, Sidhant Gupta, Tien-Jui Lee, Dan Morris, Joshua R. Smith, Matthew S. Reynolds, Desney S. Tan, and Shwetak N. Patel. 2012. An Ultra-low-power Human Body Motion Sensor Using Static Electric Field Sensing. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing (UbiComp '12). ACM, New York, NY, USA, 99--102.

[12]

Stelian Coros, Bernhard Thomaszewski, Gioacchino Noris, Shinjiro Sueda, Moira Forberg, Robert W. Sumner, Wojciech Matusik, and Bernd Bickel. 2013. Computational design of mechanical characters. ACM Transactions on Graphics 32, 4 (jul 2013), 1.

Digital Library

[13]

Scott Davidoff, Nicolas Villar, Alex S Taylor, and Shahram Izadi. 2011. Mechanical hijacking: how robots can accelerate UbiComp deployments. In Proceedings of the 13th international conference on Ubiquitous computing. ACM, 267--270.

Digital Library

[14]

Ruta Desai, James McCann, and Stelian Coros. 2018. Assembly-aware Design of Printable Electromechanical Devices. In The 31st Annual ACM Symposium on User Interface Software and Technology -UIST '18. ACM Press, New York, New York, USA, 457--472.

[15]

dezeen. 2018. Reconfigurable apartment allows residents to transform their living spaces. https: //www.dezeen.com/2018/02/15/video-white-arkitekterdream-home-reconfigurable-apartment-movie/. (2018). (Accessed on 04/01/2019).

[16]

Markus Ehrenmann, Oliver Rogalla, Raoul Zöllner, and Rüdiger Dillmann. 2001. Teaching service robots complex tasks: Programming by demonstration for workshop and household environments. In Proceedings of the 2001 International Conference on Field and Service Robots (FSR), Vol. 1. 397--402.

[17]

Chaitanya P. Gharpure and Vladimir A. Kulyukin. 2008. Robot-assisted shopping for the blind: issues in spatial cognition and product selection. Intelligent Service Robotics 1, 3 (jul 2008), 237--251.

[18]

Anhong Guo, Jeeeun Kim, Xiang 'Anthony' Chen, Tom Yeh, Scott E Hudson, Jennifer Mankoff, and Jeffrey P Bigham. 2017. Facade: Auto-generating tactile interfaces to appliances. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 5826--5838.

Digital Library

[19]

Guy Hoffman and Wendy Ju. 2014. Designing Robots With Movement in Mind. Journal of Human-Robot Interaction (2014).

Digital Library

[20]

Google Home. 2019. Smart Speaker & Home Assistant Google Store. https: //store.google.com/us/product/google_home?hl=en-US. (2019). (Accessed on 04/04/2019).

[21]

Xiaojun Huang, Liangh Wang, Junjun Huang, Dongxiao Li, and Ming Zhang. 2009. A Depth Extraction Method Based on Motion and Geometry for 2D to 3D Conversion. In 2009 Third International Symposium on Intelligent Information Technology Application, Vol. 3. 294--298.

Digital Library

[22]

Philips Hue. 2019. Smart home automation light. https: //www2.meethue.com/en-us/smart-home-automation-light. (2019). (Accessed on 04/04/2019).

[23]

Jianbo Shi and Tomasi. 2002. Good features to track.

[24]

Rubaiat Habib Kazi, Tovi Grossman, Hyunmin Cheong, Ali Hashemi, and George Fitzmaurice. 2017. DreamSketch: Early Stage 3D Design Explorations with Sketching and Generative Design. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology -UIST '17.

Digital Library

[25]

Han-Jong Kim, Chang Min Kim, and Tek-Jin Nam. 2018. SketchStudio: Experience Prototyping with 2.5-Dimensional Animated Design Scenarios. Proceedings of the 2018 on Designing Interactive Systems Conference 2018 (2018).

[26]

Robert Kovacs, Alexandra Ion, Pedro Lopes, Tim Oesterreich, Johannes Filter, Philipp Otto, Tobias Arndt, Nico Ring, Melvin Witte, Anton Synytsia, and others. 2018. Trussformer: 3d printing large kinetic structures. In The 31st Annual ACM Symposium on User Interface Software and Technology. ACM, 113--125.

Digital Library

[27]

Yuki Koyama, Shinjiro Sueda, Emma Steinhardt, Cal Poly, and I D C Herzliya. 2015. AutoConnect : Computational Design of 3D-Printable Connectors. ACM Transactions on Graphics (2015).

[28]

Andrey Kurenkov. DeepCrop : Directed Object Segmentation with Deep Learning. In arXiv.

[29]

TP-Link Laos. 2019. Smart Wi-Fi Light Switch. https://www.tp-link.com/la/products/details/cat5622_HS200.html. (2019). (Accessed on 04/01/2019).

[30]

LEGO. 2019. Homes -Mindstorms LEGO.com. (2019). https://www.lego.com/en-us/mindstorms

[31]

TV Lift. 2019. Motorized TV Lift & TV Automation System -Nexus 21. https://www.tvlift.com/. (2019). (Accessed on 04/01/2019).

[32]

Chao Liu and Lauren Christopher. 2012. Depth map estimation from motion for 2D to 3D conversion. In 2012 IEEE International Conference on Electro/Information Technology. 1--4.

[33]

Bruce D. Lucas and Takeo Kanade. 1981. An Iterative Image Registration Technique with an Application to Stereo Vision. Robotics (1981).

Digital Library

[34]

Blair MacIntyre, Maribeth Gandy, Steven Dow, and Jay David Bolter. 2005. DART: A Toolkit for Rapid Design Exploration of Augmented Reality Experiences Blair. ACM Transactions on Graphics (2005).

Digital Library

[35]

M. Manti, A. Pratesi, E. Falotico, M. Cianchetti, and C. Laschi. 2016. Soft assistive robot for personal care of elderly people. In 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob). IEEE, 833--838.

Digital Library

[36]

Justin Matejka, Ali Hashemi, Michael Glueck, Tovi Grossman, Erin Bradner, and George Fitzmaurice. 2018. Dream Lens.

[37]

Vittorio Megaro, Bernhard Thomaszewski, Maurizio Nitti, Otmar Hilliges, Markus Gross, and Stelian Coros. 2015. Interactive design of 3D-printable robotic creatures. ACM Transactions on Graphics 34, 6 (oct 2015), 1--9.

Digital Library

[38]

Stefanie Mueller, Alexander Teibrich, Stefan Neubert, Patrick Baudisch, François Guimbretière, and Robert Kovacs. 2015. Patching Physical Objects.

[39]

MySmartBlinds. 2019. Make your blinds smart. https://www.mysmartblinds.com/. (2019). (Accessed on 04/01/2019).

[40]

Michael Nebeling, Janet Nebeling, Ao Yu, and Rob Rumble. 2018. ProtoAR. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems -CHI '18. ACM Press, New York, New York, USA, 1--12.

[41]

BBC News. 2016. Nissan's self-parking robot chairs tidy up offices. https://www.youtube.com/watch?v=FLEgvD7iG-M. (2016). (Accessed on 04/04/2019).

[42]

RT News. 2018. Smart slippers: Nissan extends its self-parking technology. https://www.youtube.com/watch?v=BtGqi0C2SSU. (2018). (Accessed on 04/04/2019).

[43]

Georgia Peleka, Ioannis Kostavelis, Andreas Kargakos, Dimitrios Giakoumis, Manolis Vasileiadis, Dimitrios Tzovaras, and Evangelos Skartados. 2019. RAMCIP Robot: A Personal Robotic Assistant; Demonstration of a Complete Framework. Springer, Cham, 96--111.

[44]

Huaishu Peng, Jimmy Briggs, Cheng-Yao Wang, Kevin Guo, Joseph Kider, Stefanie Mueller, Patrick Baudisch, and François Guimbretière. 2018. RoMA: Interactive fabrication with augmented reality and a robotic 3D printer. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 579.

Digital Library

[45]

Raf Ramakers, Fraser Anderson, Tovi Grossman, and George Fitzmaurice. 2016. RetroFab. 409--419.

[46]

Ahunzyanov Rasim and Tropchenko Alex. 1999. Hand Detection Based on Skin Color Segmentation and Classification of Image Local Features. (1999).

[47]

Autodesk Research. 2019. Project Dreamcatcher. https: //www.autodeskresearch.com/projects/dreamcatcher. (2019). (Accessed on 04/01/2019).

[48]

Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. 2004. "GrabCut"-Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics -- Proceedings of ACM SIGGRAPH 2004 (2004).

[49]

Thijs Jan Roumen, Willi Mueller, and Patrick Baudisch. 2018. Grafter: Remixing 3D-printed machines. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 63.

Digital Library

[50]

Iason Sarantopoulos, Yannis Koveos, and Zoe Doulgeri. 2018. Grasping Flat Objects by Exploiting Non-Convexity of the Object and Support Surface. In 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 1--6.

[51]

Munehiko Sato, Ivan Poupyrev, and Chris Harrison. 2012. Touché: Enhancing Touch Interaction on Humans, Screens, Liquids, and Everyday Objects. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 483--492.

Digital Library

[52]

Eric Schweikardt and Mark D. Gross. 2008. Learning about Complexity with Modular Robots. In 2008 Second IEEE International Conference on Digital Game and Intelligent Toy Enhanced Learning. IEEE, 116--123.

Digital Library

[53]

Evan Strasnick, Jackie Yang, Kesler Tanner, Alex Olwal, and Sean Follmer. 2017. shiftIO: Reconfigurable Tactile Elements for Dynamic Affordances and Mobile Interaction. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 5075--5086.

Digital Library

[54]

Xiang Zhi Tan and Aaron Steinfeld. 2017. Using Robot Manipulation to Assist Navigation by People Who Are Blind or Low Vision. In Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction -HRI '17. ACM Press, New York, New York, USA, 379--380.

Digital Library

[55]

Markus Vincze, Tobias Koertner, Astrid Weiss, Walter Wohlkinger, Peter Einramhof, Paul Panek, Konstantinos Papoutsakis, Antonis Argyros, David Fischinger, Stefan Hofmann, and Peter Mayer. 2014. Hobbit, a care robot supporting independent living at home: First prototype and lessons learned. Robotics and Autonomous Systems 75, PA (jan 2014), 60--78.

[56]

Luke Vink, Viirj Kan, Ken Nakagaki, Daniel Leithinger, Sean Follmer, Philipp Schoessler, Amit Zoran, and Hiroshi Ishii. 2015. TRANSFORM As Adaptive and Dynamic Furniture. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '15). ACM, New York, NY, USA, 183--183.

Digital Library

[57]

Philipp Wacker, Simon Voelker, Adrian Wagner, and Jan Borchers. 2018. Physical Guides. In Proceedings of the Symposium on Spatial User Interaction -SUI '18. ACM Press, New York, New York, USA, 25--35.

[58]

Weitian Wang, Rui Li, Yi Chen, Z. Max Diekel, and Yunyi Jia. 2018. Facilitating Human-Robot Collaborative Tasks by Teaching-Learning-Collaboration From Human Demonstrations. (2018).

[59]

Victor Zykov, Andrew Chan, and Hod Lipson. 2007. Molecubes: An open-source modular robotics kit. In IROS-2007 Self-Reconfigurable Robotics Workshop. 3--6.

Cited By

Lebloch KRafetseder A(2024)Laying foundations for a “Right to Improve”Frontiers in the Internet of Things10.3389/friot.2024.13212633Online publication date: 19-Feb-2024
https://doi.org/10.3389/friot.2024.1321263
Yang WZou YHuang JAbujaber RNakagaki K(2024)TorqueCapsules: Fully-Encapsulated Flywheel Actuation Modules for Designing and Prototyping Movement-Based and Kinesthetic InteractionProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676364(1-15)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676364
Mei YJones BCascaval DMankoff JVouga ESchulz A(2024)FabHacks: Transform Everyday Objects into Home Hacks Leveraging a Solver-aided DSLProceedings of the 9th ACM Symposium on Computational Fabrication10.1145/3639473.3665788(1-16)Online publication date: 7-Jul-2024
https://dl.acm.org/doi/10.1145/3639473.3665788
Show More Cited By

Index Terms

Robiot: A Design Tool for Actuating Everyday Objects with Automatically Generated 3D Printable Mechanisms
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies
    2. Interactive systems and tools
      1. User interface toolkits
  2. Interaction design
    1. Interaction design process and methods
      1. User interface design

Recommendations

Roman: Making Everyday Objects Robotically Manipulable with 3D-Printable Add-on Mechanisms
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

One important vision of robotics is to provide physical assistance by manipulating different everyday objects, e.g., hand tools, kitchen utensils. However, many objects designed for dexterous hand-control are not easily manipulable by a single robotic ...
Reprise: A Design Tool for Specifying, Generating, and Customizing 3D Printable Adaptations on Everyday Objects
UIST '16: Proceedings of the 29th Annual Symposium on User Interface Software and Technology

Everyday tools and objects often need to be customized for an unplanned use or adapted for specific user, such as adding a bigger pull to a zipper or a larger grip for a pen. The advent of low-cost 3D printing offers the possibility to rapidly construct ...
Becoming Q: Using Design Workshops to Explore Everyday Objects as Interaction Devices for an Augmented Reality Spy Game
SUI '24: Proceedings of the 2024 ACM Symposium on Spatial User Interaction

The work in this paper extends state of the art research in the field of interaction design for everyday objects as interaction devices in Augmented Reality (AR), by taking a user defined approach to explore how users understand everyday objects as ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '19: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology

October 2019

1229 pages

ISBN:9781450368162

DOI:10.1145/3332165

General Chair:
François Guimbretière
Cornell University, USA
,
Program Chairs:
Michael Bernstein
Stanford University, USA
,
Katharina Reinecke
University of Washington, USA

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UIST '19

Sponsor:

UIST '19: The 32nd Annual ACM Symposium on User Interface Software and Technology

October 20 - 23, 2019

LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

36
Total Citations
View Citations
1,614
Total Downloads

Downloads (Last 12 months)256
Downloads (Last 6 weeks)25

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lebloch KRafetseder A(2024)Laying foundations for a “Right to Improve”Frontiers in the Internet of Things10.3389/friot.2024.13212633Online publication date: 19-Feb-2024
https://doi.org/10.3389/friot.2024.1321263
Yang WZou YHuang JAbujaber RNakagaki K(2024)TorqueCapsules: Fully-Encapsulated Flywheel Actuation Modules for Designing and Prototyping Movement-Based and Kinesthetic InteractionProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676364(1-15)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676364
Mei YJones BCascaval DMankoff JVouga ESchulz A(2024)FabHacks: Transform Everyday Objects into Home Hacks Leveraging a Solver-aided DSLProceedings of the 9th ACM Symposium on Computational Fabrication10.1145/3639473.3665788(1-16)Online publication date: 7-Jul-2024
https://dl.acm.org/doi/10.1145/3639473.3665788
Deshpande HHan BMoon KBianchi AZheng CKim J(2024)Reconfigurable Interfaces by Shape Change and Embedded MagnetsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642802(1-12)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642802
Kwon NLu QQazi MLiu JOh CKong SKim J(2024)AccessLens: Auto-detecting Inaccessibility of Everyday ObjectsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642767(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642767
Sermarini JMichlowitz RLaViola JWalters LAzevedo RKider J(2024)Exploring Augmented Reality’s Role in Enhancing Spatial Perception for Building Facade Retrofit Design for Non-experts2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00046(233-243)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00046
Lebloch KRafetseder A(2023)SerIoT: The Interface That Speaks Upgradeability By DefaultProceedings of the 13th International Conference on the Internet of Things10.1145/3627050.3627062(66-73)Online publication date: 7-Nov-2023
https://dl.acm.org/doi/10.1145/3627050.3627062
Arabi AWang XZhang YKim J(2023)E3DProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108977:3(1-31)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610897
Yan ZWu YLuo DZhang CJin QChen WWu YChen XWang GMi H(2023)NaCanva: Exploring and Enabling the Nature-Inspired Creativity for ChildrenProceedings of the ACM on Human-Computer Interaction10.1145/36042627:MHCI(1-25)Online publication date: 13-Sep-2023
https://dl.acm.org/doi/10.1145/3604262
Chi HSha JZhang Y(2023)Bring Environments to People – A Case Study of Virtual Tours in Accessibility Assessment for People with Limited MobilityProceedings of the 20th International Web for All Conference10.1145/3587281.3587292(96-103)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3587281.3587292
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents