More Web Proxy on the site http://driver.im/

research-article

deForm: an interactive malleable surface for capturing 2.5D arbitrary objects, tools and touch

Authors:

Edward Adelson,

Hiroshi IshiiAuthors Info & Claims

UIST '11: Proceedings of the 24th annual ACM symposium on User interface software and technology

Pages 527 - 536

https://doi.org/10.1145/2047196.2047265

Published: 16 October 2011 Publication History

Abstract

We introduce a novel input device, deForm, that supports 2.5D touch gestures, tangible tools, and arbitrary objects through real-time structured light scanning of a malleable surface of interaction. DeForm captures high-resolution surface deformations and 2D grey-scale textures of a gel surface through a three-phase structured light 3D scanner. This technique can be combined with IR projection to allow for invisible capture, providing the opportunity for co-located visual feedback on the deformable surface. We describe methods for tracking fingers, whole hand gestures, and arbitrary tangible tools. We outline a method for physically encoding fiducial marker information in the height map of tangible tools. In addition, we describe a novel method for distinguishing between human touch and tangible tools, through capacitive sensing on top of the input surface. Finally we motivate our device through a number of sample applications.

Supplementary Material

JPG File (fp220.jpg)

Download
5.58 KB

MP4 File (fp220.mp4)

Download
17.38 MB

References

[1]

Xbox.com | Kinect. 2010. http://www.xbox.com/en-US/kinect.

[2]

ofxStructured Light. http://code.google.com/p/structured-light/.

[3]

Cassinelli, A. and Ishikawa, M. Khronos projector. ACM SIGGRAPH 2005 Emerging Technologies, ACM Press (2005), 10.

Digital Library

[4]

Chan, L.W., Wu, H.T., Kao, H.S., Lin, H.R., Chen, M.Y,Hsu, Jane, Hung, Y.P. Enabling beyond-surface interactions for interactive surface with an invisible projection. Proc. UIST 2010, ACM Press (2010), 263--272.

Digital Library

[5]

Han, J.Y. Low-cost multi-touch sensing through frustrated total internal reflection. Proc. UIST 2005, ACM Press (2005), 115.

Digital Library

[6]

Hilliges, O., Izadi, S., Wilson, A.D., Hodges, S., Garcia-Mendoza, A., and Butz, A. Interactions in the Air : Adding Further Depth to Interactive Tabletops. Proc. UIST 2009, ACM Press (2009), 139--148.

Digital Library

[7]

Hilliges, O., Kim, D., and Izadi, S. Creating malleable interactive surfaces using liquid displacement sensing. Proc. Tabletop 2008, IEEE Press (2008), 157--160.

[8]

Hook, J., Taylor, S., Butler, A., Villar, N., and Izadi, S. A reconfigurable ferromagnetic input device. Proc. UIST 2009, ACM Press (2009), 51.

Digital Library

[9]

Ishii, H. and Ullmer, B. Tangible bits. Proc. CHI 1997, ACM Press (1997), 234--241.

Digital Library

[10]

Izadi, S., Hodges, S., Taylor, S., et al. Going beyond the display. Proc. UIST 2008, ACM Press (2008), 269.

Digital Library

[11]

Jansen, Y., Karrer, T., and Borchers, J. MudPad: tactile feedback and haptic texture overlay for touch surfaces. Proc. ITS 2010, ACM Press (2010), 11--14.

Digital Library

[12]

Johnson, M.K. and Adelson, E.H. Retrographic sensing for the measurement of surface texture and shape. Proc. IEEE CVPR 2009, IEEE Press (2009), 1070--1077.

[13]

Kaltenbrunner, M. and Bencina, R. reacTIVision. Proc. TEI 2007, ACM Press (2007), 69.

Digital Library

[14]

Lanman, D. and Taubin, G. Build your own 3D scanner. ACM SIGGRAPH 2009 Courses, ACM Press (2009), 1--94.

Digital Library

[15]

Large, M.J., Large, T., and Travis, A.R.L. Parallel Optics in Waveguide Displays: A Flat Panel Autostereoscopic Display. Journal of Display Technology 6, 10 (2010), 431--437.

[16]

Lecuyer, A., Coquillart, S., Kheddar, A., Richard, P., and Coiffet, P. Pseudo-haptic feedback: can isometric input devices simulate force feedback? Proc. IEEE Virtual Reality 2000, IEEE Comput. Soc, 83--90.

Digital Library

[17]

Lei, S. and Zhang, S. Flexible 3-D shape measurement using projector defocusing. Optics Letters 34, 20 (2009), 3080.

[18]

Malik, S. and Laszlo, J. Visual touchpad. Proc. ICMI 2004, ACM Press (2004), 289.

Digital Library

[19]

Massie, T.H. and Salisbury, J.K. The PHANTOM Haptic Interface: A Device for Probing Virtual Objects. Proceedings of the ASME Winter Annual Meeting Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, (1994), 295--300.

[20]

Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. Efficiently combining positions and normals for precise 3D geometry. ACM Transactions on Graphics 24, 3 (2005), 536.

Digital Library

[21]

Otsuki, M., Sugihara, K., Kimura, A., Shibata, F., and Tamura, H. MAI painting brush: an interactive device that realizes the feeling of real painting. Proc. UIST 2010, ACM Press (2010), 97--100.

Digital Library

[22]

Overholt, D. The MATRIX: a novel controller for musical expression. Proc. NIME 2001, AMC (2001), 1--4.

Digital Library

[23]

Piper, B., Ratti, C., and Ishii, H. Illuminating clay. Proc. CHI 2002, ACM Press (2002), 355.

Digital Library

[24]

Rosenberg, I. and Perlin, K. The UnMousePad. Proc. SIGGRAPH 2009, ACM Press (2009), 1.

Digital Library

[25]

Sato, T., Mamiya, H., Tokui, T., Koike, H., and Fukuchi, K. PhotoelasticTouch: transparent rubbery interface using a LCD and photoelasticity. ACM SIGGRAPH 2009 Emerging Technologies, ACM (2009), 1--1.

Digital Library

[26]

Sheng, J., Balakrishnan, R., and Singh, K. An interface for virtual 3D sculpting via physical proxy. Computer graphics and interactive techniques in Australasia and South East Asia, (2006), 213.

Digital Library

[27]

Sile O'Modhrain. Playing by Feel: Incorporating Haptic Feedback into Computer-Based musical Instruments. 2000. https://ccrma.stanford.edu/~sile/thesis.html.

Digital Library

[28]

Sinclair, M. The haptic lens. Ext. Abstracts ACM SIGGRAPH '97, ACM Press (1997), 179.

Digital Library

[29]

Smith, J.D., Graham, T.C.N., Holman, D., and Borchers, J. Low-Cost Malleable Surfaces with Multi-Touch Pressure Sensitivity. TABLETOP 2007, IEEE (2007), 205--208.

[30]

Tom White. Introducing Liquid Haptics in High Bandwidth Human Computer Interfaces. 1998. http://dspace.mit.edu/handle/1721.1/62938.

[31]

Valino Koh, J.T.K., Karunanayaka, K., Sepulveda, J., Tharakan, M.J., Krishnan, M., and Cheok, A.D. Liquid interface. Proc. ACE 2010, ACM Press (2010), 45.

Digital Library

[32]

Vandoren, P., Van Laerhoven, T., Claesen, L., Taelman, J., Raymaekers, C., and Van Reeth, F. IntuPaint: Bridging the gap between physical and digital painting. TABLETOP 2008, IEEE (2008), 65--72.

[33]

Viciana-Abad, R., Lecuona, A.R., and Poyade, M. The Influence of Passive Haptic Feedback and Difference Interaction Metaphors on Presence and Task Performance. Presence: Teleoperators and Virtual Environments 19, 3 (2010), 197--212.

Digital Library

[34]

Vlack, K., Mizota, T., Kawakami, N., Kamiyama, K., Kajimoto, H., and Tachi, S. Gelforce: a vision-based traction field computer interface. Ext. Abstracts CHI 2005, ACM Press (2005), 1154--1155.

Digital Library

[35]

Vogt, F., Chen, T., Hoskinson, R., and Fels, S. A malleable surface touch interface. ACM SIGGRAPH 2004 Sketches, ACM Press (2004), 36.

Digital Library

[36]

Weiss, M., Wagner, J., Jansen, Y., et al. SLAP widgets. Proc. CHI 2009, ACM Press (2009), 481.

Digital Library

[37]

Wilson, A.D., Izadi, S., Hilliges, O., Garcia-Mendoza, A., and Kirk, D. Bringing physics to the surface. Proc. UIST 2008, ACM Press (2008), 67.

Digital Library

[38]

Wilson, A.D. TouchLight. Proc. ICMI 2004, ACM Press (2004), 69.

Digital Library

[39]

Zhang, S. and Yau, S.-T. Generic nonsinusoidal phase error correction for three-dimensional shape measurement using a digital video projector. Applied Optics 46, 1 (2007), 36.

[40]

Zhang, S. Recent progresses on real-time 3D shape measurement using digital fringe projection techniques. Optics and Lasers in Engineering 48, 2 (2010), 149--158.

Cited By

Moyer IBourgault SFrost DJacobs J(2024)Don't Mesh Around: Streamlining Manual-Digital Fabrication Workflows with Domain-Specific 3D ScanningProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676385(1-16)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676385
Getschmann CEchtler FCiolfi LHogan TDöring TJenkins Tvan Dijk JHuron SLi ZCoyle DSigner B(2024)LensLeech: On-Lens Interaction for Arbitrary Camera DevicesProceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3623509.3633382(1-10)Online publication date: 11-Feb-2024
https://dl.acm.org/doi/10.1145/3623509.3633382
Hasenfuss H(2022)Reshaping Thinking for Shape-Shifting Technology: Adapting a MAS Agent Design to Encourage User EngagementComputer-Human Interaction Research and Applications10.1007/978-3-031-22015-9_3(32-53)Online publication date: 13-Dec-2022
https://doi.org/10.1007/978-3-031-22015-9_3
Show More Cited By

Index Terms

deForm: an interactive malleable surface for capturing 2.5D arbitrary objects, tools and touch
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens

Recommendations

Non-Rigid HCI: A Review of Deformable Interfaces and Input
DIS '19: Proceedings of the 2019 on Designing Interactive Systems Conference

Deformable interfaces are emerging in HCI and prototypes show potential for non-rigid interactions. Previous reviews looked at deformation as a material property of shape-changing interfaces and concentrated on output. As such,deformable input was under-...
Sinkpad: a malleable mouse pad consisted of an elastic material
CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing Systems

The computer mouse has been used for more than 40 years; users, however, can only perform simple actions. To solve this problem, we present "Sinkpad", an augmented mouse pad that has a malleable surface consisted of an elastic material. Sinkpad augments ...
Deformation Techniques for Shape Changing Interfaces
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

Since the introduction of shape changing interfaces, research in the field has contributed a number of taxonomies to classify shape changing interfaces according to different characteristics, including shape, interaction mapping, material, actuation, and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '11: Proceedings of the 24th annual ACM symposium on User interface software and technology

October 2011

654 pages

ISBN:9781450307161

DOI:10.1145/2047196

General Chair:
Jeff Pierce
IBM Research, USA
,
Program Chairs:
Maneesh Agrawala
University of California, Berkeley, USA
,
Scott Klemmer
Stanford University, USA

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 October 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UIST '11

Sponsor:

UIST '11: The 24th Annual ACM Symposium on User Interface Software and Technology

October 16 - 19, 2011

California, Santa Barbara, USA

Acceptance Rates

UIST '11 Paper Acceptance Rate 67 of 262 submissions, 26%;

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
1,835
Total Downloads

Downloads (Last 12 months)71
Downloads (Last 6 weeks)7

Reflects downloads up to 12 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Moyer IBourgault SFrost DJacobs J(2024)Don't Mesh Around: Streamlining Manual-Digital Fabrication Workflows with Domain-Specific 3D ScanningProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676385(1-16)Online publication date: 13-Oct-2024
Getschmann CEchtler FCiolfi LHogan TDöring TJenkins Tvan Dijk JHuron SLi ZCoyle DSigner B(2024)LensLeech: On-Lens Interaction for Arbitrary Camera DevicesProceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3623509.3633382(1-10)Online publication date: 11-Feb-2024
Hasenfuss H(2022)Reshaping Thinking for Shape-Shifting Technology: Adapting a MAS Agent Design to Encourage User EngagementComputer-Human Interaction Research and Applications10.1007/978-3-031-22015-9_3(32-53)Online publication date: 13-Dec-2022
Ullmer BShaer OMazalek AHummels C(2022)Weaving Fire into FormundefinedOnline publication date: 20-Jul-2022
Hasenfuss H(2021)Through the Looking Glass: Designing Agents for MAS Based Shape-Shifting Technology Using the STEAM ApproachComputer-Human Interaction Research and Applications10.1007/978-3-030-67108-2_5(80-101)Online publication date: 22-Jan-2021
Wallace LDelaurante TSimon MAustin RRolich TKhadka RBanic A(2020)Squishy Volumes: Evaluation of Silicone as Camera-less Pressure-Based Input for 3-Dimensional Interaction2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW50115.2020.00013(29-34)Online publication date: Mar-2020
Boem ATroiano GHarrison SBardzell SNeustaedter CTatar D(2019)Non-Rigid HCIProceedings of the 2019 on Designing Interactive Systems Conference10.1145/3322276.3322347(885-906)Online publication date: 18-Jun-2019
Vandenberghe BGerling KGeurts LVanden Abeele VKuznetsov SSaakes DWakkary RGeurts LHayes LLau M(2019)Skweezee for ProcessingProceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3294109.3300984(375-381)Online publication date: 17-Mar-2019
Reusser DKnoop ESiegwart RBeardsley PBrewster SFitzpatrick GCox AKostakos V(2019)Feeling FireworksProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300659(1-11)Online publication date: 2-May-2019
Siriborvornratanakul T(2018)Enhancing User Experiences of Mobile-Based Augmented Reality via Spatial Augmented RealityAdvances in Multimedia10.1155/2018/81947262018Online publication date: 1-Jan-2018
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents