More Web Proxy on the site http://driver.im/

research-article

Perception-aware modeling and fabrication of digital drawing tools

Authors:

Michal Piovarči,

David I. W. Levin,

Danny M. Kaufman,

Piotr DidykAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 37, Issue 4

Article No.: 123, Pages 1 - 15

https://doi.org/10.1145/3197517.3201322

Published: 30 July 2018 Publication History

Abstract

Digital drawing is becoming a favorite technique for many artists. It allows for quick swaps between different materials, reverting changes, and applying selective modifications to finished artwork. These features enable artists to be more efficient and creative. A significant disadvantage of digital drawing is poor haptic feedback. Artists are usually limited to one surface and a few different stylus nibs, and while they try to find a combination that suits their needs, this is typically challenging. In this work, we address this problem and propose a method for designing, evaluating, and optimizing different stylus designs. We begin with collecting a representative set of traditional drawing tools. We measure their physical properties and conduct a user experiment to build a perceptual space that encodes perceptually-relevant attributes of drawing materials. The space is optimized to both explain our experimental data and correlate it with measurable physical properties. To embed new drawing tool designs into the space without conducting additional experiments and measurements, we propose a new, data-driven simulation technique for characterizing stylus-surface interaction. We finally leverage the perceptual space, our simulation, and recent advancements in multi-material 3D printing to demonstrate the application of our system in the design of new digital drawing tools that mimic traditional drawing materials.

Supplementary Material

ZIP File (123-271.zip)

Supplemental files.

Download
11.96 MB

MP4 File (123-271.mp4)

Download
176.47 MB

MP4 File (a123-piovarci.mp4)

Download
206.77 MB

References

[1]

Michelle Annett, Fraser Anderson, Walter F Bischof, and Anoop Gupta. 2014. The pen is mightier: understanding stylus behaviour while inking on tablets. In Proceedings of Graphics Interface 2014. Canadian Information Processing Society, 193--200.

Digital Library

[2]

Atakan Arasan, Cagatay Basdogan, and Tevfik Metin Sezgin. 2013. Haptic stylus with inertial and vibro-tactile feedback. In 2013 World Haptics Conference (WHC). 425--430.

[3]

John F. Archard. 1953. Contact and Rubbing of Flat Surfaces. Journal of Applied Physics 24, 8 (1953), 981--988.

[4]

Brian Armstrong-Hélouvry. 1991. Control of Machines with Friction. Kluwer Academic Publishers, Boston, Massachusetts.

[5]

Mordecai Avriel. 2003. Nonlinear Programming: Analysis and Methods. Dover Publications.

[6]

Olivier Bau, Ivan Poupyrev, Ali Israr, and Chris Harrison. 2010. TeslaTouch: Electro-vibration for Touch Surfaces. In Proceedings of the 23Nd Annual ACM Symposium on User Interface Software and Technology (UIST '10). ACM, New York, NY, USA, 283--292.

Digital Library

[7]

Sliman Bensmaïa, Mark Hollins, and Jeffrey Yau. 2005. Vibrotactile intensity and frequency information in the Pacinian system: A psychophysical model. Attention, Perception, & Psychophysics 67, 5 (2005), 828--841.

[8]

Wouter M Bergmann Tiest and Astrid ML Kappers. 2006. Analysis of haptic perception of materials by multidimensional scaling and physical measurements of roughness and compressibility. Acta psychologica 121, 1 (2006), 1--20.

[9]

Florence Bertails-Descoubes, Florent Cadoux, Gilles Daviet, and Vincent Acary. 2011. A nonsmooth Newton solver for capturing exact Coulomb friction in fiber assemblies. ACM Transactions on Graphics 30, 1 (Feb. 2011), 6:1--6:14.

Digital Library

[10]

Brian Blood. 2009. Physics of musical instruments. (2009).

[11]

Carlos Canudas de Wit, Henrik Olsson, Karl J. Astrom, and Pablo Lischinsky. 1995. A new model for control of systems with friction. IEEE Trans. Automat. Control 40, 3 (Mar 1995), 419--425.

[12]

Desai Chen, David I. W. Levin, Wojciech Matusik, and Danny M. Kaufman. 2017. Dynamics-aware Numerical Coarsening for Fabrication Design. ACM Trans. Graph. 36, 4, Article 84 (July 2017), 15 pages.

Digital Library

[13]

Youngjun Cho, Andrea Bianchi, Nicolai Marquardt, and Nadia Bianchi-Berthouze. 2016. RealPen: Providing Realism in Handwriting Tasks on Touch Surfaces Using Auditory-Tactile Feedback. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, New York, NY, USA, 195--205.

Digital Library

[14]

Seungmoon Choi and Hong Z. Tan. 2005. Toward Realistic Haptic Rendering of Surface Textures. In ACM SIGGRAPH 2005 Courses (SIGGRAPH '05). ACM, New York, NY, USA, Article 125.

Digital Library

[15]

Vasilios G Chouvardas, Amalia N Miliou, and Miltiadis K Hatalis. 2008. Tactile displays: Overview and recent advances. Displays 29, 3 (2008), 185--194.

[16]

Vasilios G Chouvardas, Amalia N Miliou, Miltiadis K Hatalis, et al. 2005. Tactile displays: a short overview and recent developments. In Proceedings of the 5th International Conference on Technology and Automation (ICTA'05). 246--251.

[17]

Theresa Cooke, Sebastian Kannengiesser, Christian Wallraven, and Heinrich H Bülthoff.

[18]

2006. Object feature validation using visual and haptic similarity ratings. ACM Transactions on Applied Perception (TAP) 3, 3 (2006), 239--261.

Digital Library

[19]

Theresa Cooke, Christian Wallraven, and Heinrich H Bülthoff. 2010. Multidimensional scaling analysis of haptic exploratory procedures. ACM Transactions on Applied Perception (TAP) 7, 1 (2010), 7.

Digital Library

[20]

Philip R. Dahl. 1976. Solid Friction Damping of Mechanical Vibrations. AIAA Journal 14, 12 (1976), 1675--1682.

[21]

Jérémy Danna and Jean-Luc Velay. 2015. Basic and supplementary sensory feedback in handwriting. Frontiers in Psychology 6 (2015), 169.

[22]

Nicholas Fellion, Thomas Pietrzak, and Audrey Girouard. 2017. FlexStylus: leveraging flexion input for pen interaction. In UIST 2017 (UIST 2017). Québec, Canada, 2482 -- 2489.

Digital Library

[23]

Claire Watson Garcia. 2003. Drawing for the Absolute and Utter Beginner. Watson-Guptill.

[24]

Ioannis Gkioulekas, Bei Xiao, Shuang Zhao, Edward H Adelson, Todd Zickler, and Kavita Bala. 2013. Understanding the role of phase function in translucent appearance. ACM Transactions on Graphics (TOG) 32, 5 (2013), 147.

Digital Library

[25]

Avraham Harnoy, Bernard Friedland, and Simon Cohn. 2008. Modeling and measuring friction effects. IEEE Control Systems 28, 6 (Dec 2008), 82--91.

[26]

Vincent Hayward and M Cruz-Hernandez. 2000. Tactile display device using distributed lateral skin stretch. In Proceedings of the haptic interfaces for virtual environment and teleoperator systems symposium, Vol. 69. ASME, 1309--1314.

[27]

Richard Helps and Clarissa Helps. 2016. Measuring Stylus and Tablet Performance for Usability in Sketching. In Proceedings of the 5th Annual Conference on Research in Information Technology. ACM, 19--24.

Digital Library

[28]

Marlis Hochbruck and Alexander Ostermann. 2010. Exponential integrators. Acta Numerica 19 (2010), 209--286.

[29]

Mark Hollins, Sliman Bensmaïa, Kristie Karlof, and Forrest Young. 2000. Individual differences in perceptual space for tactile textures: Evidence from multidimensional scaling. Perception & Psychophysics 62, 8 (2000), 1534--1544.

[30]

Mark Hollins, Florian Lorenz, and Daniel Harper. 2006. Somatosensory Coding of Roughness: The Effect of Texture Adaptation in Direct and Indirect Touch. Journal of Neuroscience 26, 20 (2006), 5582--5588.

[31]

Ali Israr, Seungmoon Choi, and Hong Z. Tan. 2006a. Detection Threshold and Mechanical Impedance of the Hand in a Pen-Hold Posture. In 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems. 472--477.

[32]

Ali Israr, Hong Z. Tan, and Charlotte M. Reed. 2006b. Frequency and amplitude discrimination along the kinesthetic-cutaneous continuum in the presence of masking stimuli. The Journal of the Acoustical Society of America 120, 5 (2006), 2789--2800.

[33]

Lynette A Jones and Ian W Hunter. 1990. A perceptual analysis of stiffness. Experimental Brain Research 79, 1 (1990), 150--156.

[34]

Dean Karnopp. 1985. Computer Simulation of Stick-Slip Friction in Mechanical Dynamic Systems. ASME Journal of dynamic Systems, Measurement and Control 107, 1 (Mar 1985), 100--103.

[35]

Danny M. Kaufman, Timothy Edmunds, and Dinesh K. Pai. 2005. Fast Frictional Dynamics for Rigid Bodies. ACM Trans. Graph. 24, 3 (July 2005), 946--956.

Digital Library

[36]

Danny M. Kaufman, Shinjiro Sueda, Doug L. James, and Dinesh K. Pai. 2008. Staggered Projections for Frictional Contact in Multibody Systems. ACM Trans. Graph. 27, 5, Article 164 (Dec. 2008), 11 pages.

Digital Library

[37]

Seung-Chan Kim, Ali Israr, and Ivan Poupyrev. 2013. Tactile Rendering of 3D Features on Touch Surfaces. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). ACM, New York, NY, USA, 531--538.

Digital Library

[38]

Roberta L. Klatzky and Susan J. Lederman. 2002. Perceiving texture through a probe. Touch in virtual environments (2002), 180--193.

[39]

Roberta L. Klatzky and Susan J. Lederman. 2008. Perceiving object properties through a rigid link. Haptic Rendering: Algorithms and Applications 1 (2008), 7--19.

[40]

Roberta L. Klatzky, Susan J. Lederman, Cheryl Hamilton, Molly Grindley, and Robert H. Swendsen. 2003. Feeling textures through a probe: Effects of probe and surface geometry and exploratory factors. Attention, Perception, & Psychophysics 65, 4 (2003), 613--631.

[41]

Johnny C. Lee, Paul H. Dietz, Darren Leigh, William S. Yerazunis, and Scott E. Hudson. 2004. Haptic Pen: A Tactile Feedback Stylus for Touch Screens. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (UIST '04). ACM, New York, NY, USA, 291--294.

Digital Library

[42]

Dingzeyu Li, David I. W. Levin, Wojciech Matusik, and Changxi Zheng. 2016. Acoustic Voxels: Computational Optimization of Modular Acoustic Filters. ACM Trans. Graph. 35, 4, Article 88 (July 2016), 12 pages.

Digital Library

[43]

Thomas H. Massie, J. Kenneth Salisbury, et al. 1994. The phantom haptic interface: A device for probing virtual objects. In Proceedings of the ASME winter annual meeting, symposium on haptic interfaces for virtual environment and teleoperator systems, Vol. 55. Chicago, IL, 295--300.

[44]

Ralph Mayer. 1991. Artist's Handbook of Materials and Techniques: Revised and Updated. Viking Books.

[45]

Dominik L. Michels, Gerrit A. Sobottka, and Andreas G. Weber. 2014. Exponential Integrators for Stiff Elastodynamic Problems. ACM Trans. Graph. 33, 1, Article 7 (Feb. 2014), 20 pages.

Digital Library

[46]

Brian Mirtich and John Canny. 1995. Impulse-based Simulation of Rigid Bodies. In Proceedings of the 1995 Symposium on Interactive 3D Graphics (I3D '95). ACM, New York, NY, USA, 181--ff.

Digital Library

[47]

Ilana Nisky, Assaf Pressman, Carla M. Pugh, Ferdinando A. Mussa-Ivaldi, and Amir Karniel. 2011. Perception and Action in Teleoperated Needle Insertion. EEE Trans. Haptics 4, 3 (July 2011), 155--166.

Digital Library

[48]

Miguel A. Otaduy and Ming C. Lin. 2004. A Perceptually-inspired Force Model for Haptic Texture Rendering. In Proceedings of the 1st Symposium on Applied Perception in Graphics and Visualization (APGV '04). ACM, New York, NY, USA, 123--126.

Digital Library

[49]

Fabio Pellacini, James A. Ferwerda, and Donald P. Greenberg. 2000. Toward a psychophysically-based light reflection model for image synthesis. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., 55--64.

Digital Library

[50]

Alvaro G. Perez, Daniel Lobo, Francesco Chinello, Gabriel Cirio, Monica Malvezzi, Jose San Martin, Domenico Prattichizzo, and Miguel A. Otaduy. 2017. Optimization-based wearable tactile rendering. IEEE transactions on haptics (2017).

[51]

Henry Petroski. 1992. The pencil: A history of design and circumstance. Alfred a Knopf Incorporated.

[52]

Michal Piovarči, David I.W. Levin, Jason Rebello, Desai Chen, Roman Ďurikovič, Hanspeter Pfister, Wojciech Matusik, and Piotr Didyk. 2016. An Interaction-Aware, Perceptual Model For Non-Linear Elastic Objects. ACM Transactions on Graphics (Proc. SIGGRAPH) 35, 4 (2016).

Digital Library

[53]

Helena Pongrac. 2008. Vibrotactile perception: examining the coding of vibrations and the just noticeable difference under various conditions. Multimedia Systems 13, 4 (2008), 297--307.

Digital Library

[54]

Ivan Poupyrev, Makoto Okabe, and Shigeaki Maruyama. 2004. Haptic Feedback for Pen Computing: Directions and Strategies. In CHI '04 Extended Abstracts on Human Factors in Computing Systems (CHI EA '04). ACM, New York, NY, USA, 1309--1312.

Digital Library

[55]

Joseph M. Romano and Katherine J. Kuchenbecker. 2012. Creating Realistic Virtual Textures from Contact Acceleration Data. IEEE Transactions on Haptics 5, 2 (April 2012), 109--119.

Digital Library

[56]

Leonid Sigal, Moshe Mahler, Spencer Diaz, Kyna McIntosh, Elizabeth Carter, Timothy Richards, and Jessica Hodgins. 2015. A Perceptual Control Space for Garment Simulation. ACM Trans. Graph. 34, 4, Article 117 (July 2015), 10 pages.

Digital Library

[57]

D. Amnon Silverstein and Joyce E. Farrell. 2001. Efficient method for paired comparison. Journal of Electronic Imaging 10, 2 (2001), 394--398.

[58]

Hyunyoung Song, Hrvoje Benko, Francois Guimbretiere, Shahram Izadi, Xiang Cao, and Ken Hinckley. 2011. Grips and Gestures on a Multi-Touch Pen. 1323--1332. https://www.microsoft.com/en-us/research/publication/grips-and-gestures-on-a-multi-touch-pen/

Digital Library

[59]

Gwidon W Stachowiak. 2006. Wear: materials, mechanisms and practice. John Wiley & Sons.

[60]

Luis L. Thurstone. 1927. A law of comparative judgment. Psychological Review 34, 4 (1927).

[61]

Wouter M Bergmann Tiest and Astrid ML Kappers. 2009. Cues for haptic perception of compliance. Haptics, IEEE Transactions on 2, 4 (2009), 189--199.

Digital Library

[62]

Stephen Van Dulken. 2002. Inventing the 20th Century: 100 Inventions that Shaped the World from the Airplane to the Zipper. NYU Press.

[63]

Hector Vazquez-Leal, Roberto Castaneda-Sheissa, Uriel Filobello-Nino, Arturo Sarmiento-Reyes, and Jesus Sanchez Orea. 2012. High Accurate Simple Approximation of Normal Distribution Integral. Mathematical Problems in Engineering 2012 (2012).

[64]

Qinglong Wang, Xiangshi Ren, Sayan Sarcar, and Xiaoying Sun. 2016. EV-Pen: Leveraging Electrovibration Haptic Feedback in Pen Interaction. In Proceedings of the 2016 ACM on Interactive Surfaces and Spaces (ISS '16). ACM, New York, NY, USA, 57--66.

Digital Library

[65]

Rhiannon Williams. 2015. Jony Ive interview: The story of the Apple Pencil. http://www.telegraph.co.uk/technology/apple/11988396/Jony-Ive-interview-The-story-of-the-Apple-Pencil.html. (2015). accessed on: 8 January 2018.

[66]

Josh Wills, Sameer Agarwal, David Kriegman, and Serge Belongie. 2009. Toward a perceptual space for gloss. ACM Transactions on Graphics 28, 4 (2009), 1--15.

Digital Library

[67]

Takashi Yoshioka, Sliman Bensmaïa, James C. Craig, and Shen-Fu. Hsiao. 2007. Texture perception through direct and indirect touch: An analysis of perceptual space for tactile textures in two modes of exploration. Somatosensory & Motor Research 24, 1--2 (2007), 53--70.

[68]

Takashi Yoshioka and Julia Zhou. 2009. Factors Involved in Tactile Texture Perception Through Probes. Advanced Robotics 23, 6 (2009), 747--766.

Cited By

Burnah TImtiaz MGuesgen HRudolph GBlagojevic R(2024)Passive Stylus Tracking: A Systematic Literature ReviewProceedings of the ACM on Human-Computer Interaction10.1145/36981448:ISS(427-461)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3698144
Berns SVolz VTokarchuk LSnodgrass SGuckelsberger C(2024)Not All the Same: Understanding and Informing Similarity Estimation in Tile-Based Video GamesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642077(1-23)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642077
Liu NWang QNai WSun X(2024)Haptic Feedback Relative Fidelity Measure for Stylus Writing Based on Triplet-CNNIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.347001473(1-12)Online publication date: 2024
https://doi.org/10.1109/TIM.2024.3470014
Show More Cited By

Index Terms

Perception-aware modeling and fabrication of digital drawing tools
1. Computing methodologies
  1. Computer graphics
    1. Animation
      1. Physical simulation
    2. Graphics systems and interfaces
      1. Graphics input devices
      2. Perception
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Haptic devices

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 37, Issue 4

August 2018

1670 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/3197517

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 July 2018

Published in TOG Volume 37, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
528
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)4

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Burnah TImtiaz MGuesgen HRudolph GBlagojevic R(2024)Passive Stylus Tracking: A Systematic Literature ReviewProceedings of the ACM on Human-Computer Interaction10.1145/36981448:ISS(427-461)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3698144
Berns SVolz VTokarchuk LSnodgrass SGuckelsberger C(2024)Not All the Same: Understanding and Informing Similarity Estimation in Tile-Based Video GamesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642077(1-23)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642077
Liu NWang QNai WSun X(2024)Haptic Feedback Relative Fidelity Measure for Stylus Writing Based on Triplet-CNNIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.347001473(1-12)Online publication date: 2024
https://doi.org/10.1109/TIM.2024.3470014
Huang XRitschel TSeidel HMemari PSingh G(2023)Patternshop: Editing Point Patterns by Image ManipulationACM Transactions on Graphics10.1145/359241842:4(1-14)Online publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1145/3592418
Tao YWang SJi JCai LXia HWang ZHe JFan YPan SXu JYang CSun LWang G(2023)4Doodle: 4D Printing Artifacts Without 3D PrintersProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581321(1-16)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581321
Ogura AIto KItoh Y(2022)Transtiff: A Stick Interface with Various Stiffness by Artificial Muscle MechanismAdjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526114.3558777(1-3)Online publication date: 29-Oct-2022
https://dl.acm.org/doi/10.1145/3526114.3558777
Kim JZhou QGhassaei AChen XKaltenbrunner MMurer MWolf KOakley I(2021)OmniSoft: A Design Tool for Soft Objects by ExampleProceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3430524.3440634(1-13)Online publication date: 14-Feb-2021
https://dl.acm.org/doi/10.1145/3430524.3440634
Ogata MKoyama YKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)A Computational Approach to Magnetic Force Feedback DesignProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445631(1-12)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445631
Piovarči MKaufman DLevin DDidyk P(2020)Fabrication-in-the-loop co-optimization of surfaces and styli for drawing hapticsACM Transactions on Graphics10.1145/3386569.339246739:4(116:1-116:16)Online publication date: 12-Aug-2020
https://dl.acm.org/doi/10.1145/3386569.3392467
Sahli RProt AWang AMüser MPiovarči MDidyk PBennewitz R(2020)Tactile perception of randomly rough surfacesScientific Reports10.1038/s41598-020-72890-y10:1Online publication date: 25-Sep-2020
https://doi.org/10.1038/s41598-020-72890-y
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents