More Web Proxy on the site http://driver.im/

research-article

OpenMPD: A Low-Level Presentation Engine for Multimodal Particle-Based Displays

Authors:

Roberto Montano-Murillo,

Ryuji Hirayama,

Diego Martinez PlasenciaAuthors Info & Claims

ACM Transactions on Graphics, Volume 42, Issue 2

Article No.: 24, Pages 1 - 13

https://doi.org/10.1145/3572896

Published: 11 April 2023 Publication History

Abstract

Phased arrays of transducers have been quickly evolving in terms of software and hardware with applications in haptics (acoustic vibrations), display (levitation), and audio. Most recently, Multimodal Particle-based Displays (MPDs) have even demonstrated volumetric content that can be seen, heard, and felt simultaneously, without additional instrumentation. However, current software tools only support individual modalities and they do not address the integration and exploitation of the multi-modal potential of MPDs. This is because there is no standardized presentation pipeline tackling the challenges related to presenting such kind of multi-modal content (e.g., multi-modal support, multi-rate synchronization at 10 KHz, visual rendering or synchronization and continuity). This article presents OpenMPD, a low-level presentation engine that deals with these challenges and allows structured exploitation of any type of MPD content (i.e., visual, tactile, audio). We characterize OpenMPD’s performance and illustrate how it can be integrated into higher-level development tools (i.e., Unity game engine). We then illustrate its ability to enable novel presentation capabilities, such as support of multiple MPD contents, dexterous manipulations of fast-moving particles, or novel swept-volume MPD content.

Supplementary Material

tog-22-0030-File002 (tog-22-0030-file002.m4v)

Supplementary video

Download
23.18 MB

References

[1]

2022. Photonics Voxon. (2022). Retrieved from voxon.co/voxon-vx1-available-for-purchase/.

[2]

Johann Berthelot and Nicolas Bonod. 2019. Free-space micro-graphics with electrically driven levitated light scatterers. Optics Letters 44, 6 (Mar.2019), 1476–1479. DOI:

[3]

E. H. Brandt. 1989. Levitation in Physics. Science 243, 4889 (1989), 349–355. DOI:

[4]

Tom Carter, Sue Ann Seah, Benjamin Long, Bruce Drinkwater, and Sriram Subramanian. 2013. UltraHaptics: Multi-point Mid-Air Haptic feedback for touch surfaces. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST’13). Association for Computing Machinery, New York, NY, 505–514. DOI:

Digital Library

[5]

Oliver S. Cossairt, Joshua Napoli, Samuel L. Hill, Rick K. Dorval, and Gregg E. Favalora. 2007. Occlusion-capable multiview volumetric three-dimensional display. Applied Optics 46, 8 (Mar.2007), 1244–1250. DOI:

[6]

Andreas Rene Fender, Diego Martinez Plasencia, and Sriram Subramanian. 2021. ArticuLev: An integrated self-assembly pipeline for articulated multi-bead levitation primitives. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI’21). Association for Computing Machinery, New York, NY, Article 422, 12 pages.DOI:

Digital Library

[7]

Dan Foisy. 2021. How a Basement inventor builds volumetric displays. In Proceedings of the ACM SIGGRAPH 2021 Labs (SIGGRAPH’21). Association for Computing Machinery, New York, NY, Article 2, 2 pages.DOI:

Digital Library

[8]

Euan Freeman, Asier Marzo, Praxitelis B. Kourtelos, Julie R. Williamson, and Stephen Brewster. 2019. Enhancing physical objects with actuated levitating particles. In Proceedings of the 8th ACM International Symposium on Pervasive Displays (PerDis’19). Association for Computing Machinery, New York, NY, Article 2, 7 pages.DOI:

Digital Library

[9]

Euan Freeman, Julie Williamson, Sriram Subramanian, and Stephen Brewster. 2018. Point-and-Shake: Selecting from levitating object displays. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI’18). Association for Computing Machinery, New York, NY, 1–10.DOI:

Digital Library

[10]

William Frier, Damien Ablart, Jamie Chilles, Benjamin Long, Marcello Giordano, Marianna Obrist, and Sriram Subramanian. 2018. Using spatiotemporal modulation to draw tactile patterns in mid-air. In Proceedings of the International Conference on Human Haptic Sensing and Touch Enabled Computer Applications. Springer, 270–281. DOI:

Digital Library

[11]

Tatsuki Fushimi, Asier Marzo, Bruce W. Drinkwater, and Thomas L. Hill. 2019. Acoustophoretic volumetric displays using a fast-moving levitated particle. Applied Physics Letters 115, 6 (2019), 064101. DOI:

[12]

Yunsil Heo and Hyunwoo Bang. 2014. Levitate. Leonardo 47, 4 (082014), 402–403.DOI:

[13]

Ryuji Hirayama, Giorgos Christopoulos, Diego Martinez Plasencia, and Sriram Subramanian. 2022. High-speed acoustic holography with arbitrary scattering objects. Science Advances 8, 24 (2022), eabn7614. DOI:

[14]

Ryuji Hirayama, Diego Martinez Plasencia, Nobuyuki Masuda, and Sriram Subramanian. 2019. A volumetric display for visual, tactile and audio presentation using acoustic trapping. Nature 575, 7782 (2019), 320–323.

[15]

Takayuki Hoshi, Masafumi Takahashi, Takayuki Iwamoto, and Hiroyuki Shinoda. 2010. Noncontact tactile display based on radiation pressure of airborne ultrasound. IEEE Transactions on Haptics 3, 3 (2010), 155–165. DOI:

Digital Library

[16]

Seki Inoue, Yasutoshi Makino, and Hiroyuki Shinoda. 2018. Scalable architecture for airborne ultrasound tactile display. In Haptic Interaction. Shoichi Hasegawa, Masashi Konyo, Ki-Uk Kyung, Takuya Nojima, and Hiroyuki Kajimoto (Eds.), Springer, 99–103. DOI:

[17]

Jarrod Knibbe, Hrvoje Benko, and Andrew D. Wilson. 2015. Juggling the effects of latency: Software approaches to minimizing latency in dynamic projector-camera systems. In Proceedings of the Adjunct Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology (UIST’15 Adjunct). Association for Computing Machinery, New York, NY, 93–94.DOI:

Digital Library

[18]

Jinha Lee, Rehmi Post, and Hiroshi Ishii. 2011. ZeroN: Mid-Air tangible interaction enabled by computer controlled magnetic levitation. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST’11). Association for Computing Machinery, New York, NY, 327–336.DOI:

Digital Library

[19]

Benjamin Long, Sue Ann Seah, Tom Carter, and Sriram Subramanian. 2014. Rendering volumetric haptic shapes in mid-air using ultrasound. ACM Transactions on Graphics (TOG) 33, 6 (2014), 1–10.

Digital Library

[20]

Asier Marzo, Tom Corkett, and Bruce W. Drinkwater. 2017. Ultraino: An open phased-array system for narrowband airborne ultrasound transmission. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control 65, 1 (2017), 102–111.

[21]

Asier Marzo and Bruce W. Drinkwater. 2019. Holographic acoustic tweezers. Proceedings of the National Academy of Sciences 116, 1 (2019), 84–89.

[22]

Asier Marzo, Sue Ann Seah, Bruce W. Drinkwater, Deepak Ranjan Sahoo, Benjamin Long, and Sriram Subramanian. 2015. Holographic acoustic elements for manipulation of levitated objects. Nature Communications 6, 1 (2015), 8661. DOI:

[23]

Rafael Morales, Iñigo Ezcurdia, Josu Irisarri, Marco A. B. Andrade, and Asier Marzo. 2021. Generating airborne ultrasonic amplitude patterns using an open hardware phased array. Applied Sciences 11, 7 (2021), 2981.DOI:

[24]

Rafael Morales, Asier Marzo, Sriram Subramanian, and Diego Martinez. 2019. LeviProps: Animating levitated optimized fabric structures using holographic acoustic tweezers. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. Association for Computing Machinery, New York, NY, 651–661.

Digital Library

[25]

Yoichi Ochiai, Takayuki Hoshi, and Jun Rekimoto. 2014. Pixie dust: Graphics generated by levitated and animated objects in computational acoustic-potential field. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1–13.

Digital Library

[26]

Themis Omirou, Asier Marzo, Sue Ann Seah, and Sriram Subramanian. 2015. LeviPath: Modular acoustic levitation for 3D path visualisations. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 309–312.

Digital Library

[27]

Themis Omirou, Asier Marzo Perez, Sriram Subramanian, and Anne Roudaut. 2016. Floating charts: Data plotting using free-floating acoustically levitated representations. In Proceedings of the 2016 IEEE Symposium on 3D User Interfaces (3DUI’16). IEEE, IEEE, New York, NY, 187–190.

[28]

Viktorija Paneva, Arthur Fleig, Diego Martínez Plasencia, Timm Faulwasser, and Jörg Müller. 2022. OptiTrap: Optimal trap trajectories for acoustic levitation displays. ACM Transactions on Graphics 41, 5 (Feb.2022), 14 pages.DOI:

Digital Library

[29]

Diego Martinez Plasencia, Ryuji Hirayama, Roberto Montano-Murillo, and Sriram Subramanian. 2020. GS-PAT: High-speed multi-point sound-fields for phased arrays of transducers. ACM Transactions on Graphics 39, 4, Article 138 (Jul.2020), 12 pages.DOI:

Digital Library

[30]

Deepak Ranjan Sahoo, Takuto Nakamura, Asier Marzo, Themis Omirou, Michihiro Asakawa, and Sriram Subramanian. 2016. Joled: A mid-air display based on electrostatic rotation of levitated janus objects. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. Association for Computing Machinery, New York, NY, 437–448. DOI:

Digital Library

[31]

D. E. Smalley, E. Nygaard, K. Squire, J. Van Wagoner, J. Rasmussen, S. Gneiting, K. Qaderi, J. Goodsell, W. Rogers, M. Lindsey, et al. 2018. A photophoretic-trap volumetric display. Nature 553, 7689 (2018), 486. DOI:

Digital Library

[32]

W. Le Conte Stevens. 1899. Review of a text-book of physics–sound. Science 9, 234 (1899), 872–874. Retrieved fromhttp://www.jstor.org/stable/1626789.

[33]

Ryoko Takahashi, Keisuke Hasegawa, and Hiroyuki Shinoda. 2018. Lateral modulation of midair ultrasound focus for intensified vibrotactile stimuli. In Haptics: Science, Technology, and Applications. Domenico Prattichizzo, Hiroyuki Shinoda, Hong Z. Tan, Emanuele Ruffaldi, and Antonio Frisoli (Eds.), Springer International Publishing, Cham, 276–288.

[34]

Ultraleap. 2022. Developer SDK. (2022). Retrieved from www.ultraleap.com/developers/.

[35]

Sebastian Zehnter and Christoph Ament. 2019. A modular FPGA-based phased array system for ultrasonic levitation with MATLAB. In Proceedings of the 2019 IEEE International Ultrasonics Symposium (IUS’19). 654–658. DOI:

Cited By

Hoshi TO-oka Y(2024)Optical fiber-based acoustic intensity microphone for high-intensity airborne ultrasound measurementJapanese Journal of Applied Physics10.35848/1347-4065/ad38c863:4(04SP67)Online publication date: 18-Apr-2024
https://doi.org/10.35848/1347-4065/ad38c8
Gao LTokuda YBansal SSubramanian S(2024)Computational Gastronomy and Eating with AcoustophoresisCompanion Proceedings of the 26th International Conference on Multimodal Interaction10.1145/3686215.3686218(113-116)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3686215.3686218
Christopoulos GGao LPlasencia DBetcke MHirayama RSubramanian S(2024)Temporal acoustic point holographyACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657443(1-11)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657443
Show More Cited By

Index Terms

OpenMPD: A Low-Level Presentation Engine for Multimodal Particle-Based Displays
1. Computing methodologies
  1. Computer graphics
    1. Animation
      1. Physical simulation

Recommendations

Avoiding Collisions when Interacting with Levitating Particle Displays
CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

Levitating particle displays are an emerging technology where content is composed of physical pixels. Unlike digital displays, manipulating the content is not straightforward because physical constraints affect the placement and movement of each particle:...
Visual Authoring Tool for Presentation Agent Based on Multimodal Presentation Markup Language
IV '01: Proceedings of the Fifth International Conference on Information Visualisation

Abstract: This paper proposes a visual authoring tool for presentation agent system on the Web. In presentation systems, it is more fascinating when we conduct the presentation stream by the explanation or conversation of presenters. And in order to ...
Levitating Particle Displays with Interactive Voxels
PerDis '18: Proceedings of the 7th ACM International Symposium on Pervasive Displays

Levitating objects can be used as the primitives in a new type of display. We present levitating particle displays and show how research into object levitation is enabling a new way of presenting and interacting with information. We identify novel ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 42, Issue 2

April 2023

210 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/3563904

Editor:
Carol O'Sullivan
Trinity College Dublin, Ireland

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 April 2023

Online AM: 14 February 2023

Accepted: 09 November 2022

Revised: 01 November 2022

Received: 17 May 2022

Published in TOG Volume 42, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

AHRC UK-China Research-Industry Creative Partnerships
EU Horizon 2020 research and innovation programme
EPSRC through their prosperity partnership program

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
963
Total Downloads

Downloads (Last 12 months)451
Downloads (Last 6 weeks)54

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hoshi TO-oka Y(2024)Optical fiber-based acoustic intensity microphone for high-intensity airborne ultrasound measurementJapanese Journal of Applied Physics10.35848/1347-4065/ad38c863:4(04SP67)Online publication date: 18-Apr-2024
https://doi.org/10.35848/1347-4065/ad38c8
Gao LTokuda YBansal SSubramanian S(2024)Computational Gastronomy and Eating with AcoustophoresisCompanion Proceedings of the 26th International Conference on Multimodal Interaction10.1145/3686215.3686218(113-116)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3686215.3686218
Christopoulos GGao LPlasencia DBetcke MHirayama RSubramanian S(2024)Temporal acoustic point holographyACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657443(1-11)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657443
Shen ZMorgan ZVasudevan MObrist MMartinez Plasencia D(2024)Controlled-STM: A Two-stage Model to Predict User’s Perceived Intensity for Multi-point Spatiotemporal Modulation in Ultrasonic Mid-air HapticsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642439(1-12)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642439
Gao LChristopoulos GMittal PHirayama RSubramanian S(2024)StableLev: Data-Driven Stability Enhancement for Multi-Particle Acoustic LevitationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642286(1-11)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642286
Jiang LWang YChen YChen ZLi X(2024)Effects and selection of update rates in acoustic levitatorMeasurement Science and Technology10.1088/1361-6501/ad561435:9(096203)Online publication date: 20-Jun-2024
https://doi.org/10.1088/1361-6501/ad5614
Chen HBansal SPlasencia DDi‐Silvio LHuang JSubramanian SHirayama R(2024)Omnidirectional and Multi‐Material In Situ 3D Printing Using Acoustic LevitationAdvanced Materials Technologies10.1002/admt.202401792Online publication date: 10-Dec-2024
https://doi.org/10.1002/admt.202401792

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents