More Web Proxy on the site http://driver.im/

research-article

HoloFlex: A Flexible Light-Field Smartphone with a Microlens Array and a P-OLED Touchscreen

Authors:

Juan Pablo Carrascal,

Roel VertegaalAuthors Info & Claims

UIST '16: Proceedings of the 29th Annual Symposium on User Interface Software and Technology

Pages 69 - 79

https://doi.org/10.1145/2984511.2984524

Published: 16 October 2016 Publication History

Abstract

We present HoloFlex, a 3D flexible smartphone featuring a light-field display consisting of a high-resolution P-OLED display and an array of 16,640 microlenses. HoloFlex allows mobile users to interact with 3D images featuring natural visual cues such as motion parallax and stereoscopy without glasses or head tracking. Its flexibility allows the use of bend input for interacting with 3D objects along the z axis. Images are rendered into 12-pixel wide circular blocks-pinhole views of the 3D scene-which enable ~80 unique viewports at an effective resolution of 160 × 104. The microlens array distributes each pixel from the display in a direction that preserves the angular information of light rays in the 3D scene. We present a preliminary study evaluating the effect of bend input vs. a vertical touch screen slider on 3D docking performance. Results indicate that bend input significantly improves movement time in this task. We also present 3D applications including a 3D editor, a 3D Angry Birds game and a 3D teleconferencing system that utilize bend input.

Supplementary Material

suppl.mov (uist2051-file3.mp4)

Supplemental video

Download
72.27 MB

MP4 File (p69-gotsch.mp4)

Download
307.15 MB

References

[1]

Adihikarla, V. K., Wozniak, P., and Teather, R. Hololeap: Towards efficient 3d object manipulation on light field displays. In Proceedings of the 2nd ACM symposium on Spatial user interaction, ACM (2014), 158--158.

Digital Library

[2]

Ahmaniemi, T. T., Kildal, J., and Haveri, M. What is a device bend gesture really good for' In Proceedings of the 32nd annual ACM conference on Human factors in computing systems, ACM (2014), 3503--3512.

Digital Library

[3]

Balakrishnan, R., Baudel, T., Kurtenbach, G., and Fitzmaurice, G. The rockin-mouse: integral 3d manipulation on a plane. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems, ACM (1997), 311--318.

Digital Library

[4]

Benko, H., Jota, R., and Wilson, A. Miragetable: freehand interaction on a projected augmented reality tabletop. In Proceedings of the SIGCHI conference on human factors in computing systems, ACM (2012), 199--208.

Digital Library

[5]

Boritz, J., and Booth, K. S. A study of interactive 6 dof docking in a computerised virtual environment. In Virtual Reality Annual International Symposium, 1998. Proceedings., IEEE 1998, IEEE (1998), 139--146.

Digital Library

[6]

Burstyn, J., Banerjee, A., and Vertegaal, R. Flexview: An evaluation of depth navigation on deformable mobile devices. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction, ACM (2013), 193--200.

Digital Library

[7]

Daiber, F., Falk, E., and Krüger, A. Balloon selection revisited: multi-touch selection techniques for stereoscopic data. In Proceedings of the International Working Conference on Advanced Visual Interfaces, ACM (2012), 441--444.

Digital Library

[8]

Fattal, D., Peng, Z., Tran, T., Vo, S., Fiorentino, M., Brug, J., and Beausoleil, R. G. A multi-directional backlight for a wide-angle, glasses-free threedimensional display. Nature 495, 7441 (2013), 348--351.

[9]

Froehlich, B., Hochstrate, J., Skuk, V., and Huckauf, A. The globefish and the globemouse: two new six degree of freedom input devices for graphics applications. In Proceedings of the SIGCHI conference on Human Factors in computing systems, ACM (2006), 191--199.

Digital Library

[10]

Gomes, A., and Vertegaal, R. Paperfold: evaluating shape changes for viewport transformations in foldable thin-film display devices. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction, ACM (2015), 153--160.

Digital Library

[11]

Google. Tango. https://get.google.com/tango/.

[12]

Guiard, Y. Asymmetric division of labor in human skilled bimanual action: The kinematic chain as a model. Journal of motor behavior 19, 4 (1987), 486--517.

[13]

Hachet, M., Bossavit, B., Cohé, A., and de la Rivière, J.-B. Toucheo: multitouch and stereo combined in a seamless workspace. In Proceedings of the 24th annual ACM symposium on User interface software and technology, ACM (2011), 587--592.

Digital Library

[14]

Hirsch, M., Izadi, S., Holtzman, H., and Raskar, R. 8d: interacting with a relightable glasses-free 3d display. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2013), 2209--2212.

Digital Library

[15]

Hirsch, M., Lanman, D., Holtzman, H., and Raskar, R. Bidi screen: a thin, depth-sensing lcd for 3d interaction using light fields. In ACM Transactions on Graphics (TOG), vol. 28, ACM (2009), 159.

Digital Library

[16]

Hirsch, M., Wetzstein, G., and Raskar, R. A compressive light field projection system. ACM Transactions on Graphics (TOG) 33, 4 (2014), 58.

Digital Library

[17]

HTC. Evo 3d. https://en.wikipedia.org/wiki/HTC_ Evo_3Dwiki/HTC_Evo_3D.

[18]

ISORG. Isorg and plastic logic co-develop the world's first image sensor on plastic. http://www.isorg.fr/actu/4/isorg-and-plastic-logic-codevelop-the-world-s-first-image-sensor-onplastic_149.htm.

[19]

Ives, F. E. Parallax stereogram and process of making same., Apr. 14 1903. US Patent 725,567.

[20]

Jacob, R. J., Sibert, L. E., McFarlane, D. C., and Mullen Jr, M. P. Integrality and separability of input devices. ACM Transactions on Computer-Human Interaction (TOCHI) 1, 1 (1994), 3--26.

Digital Library

[21]

Ji, C.-C., Luo, C.-G., Deng, H., Li, D.-H., and Wang, Q.-H. Tilted elemental image array generation method for moiré-reduced computer generated integral imaging display. Optics express 21, 17 (2013), 19816--19824.

[22]

Johnston, E. B., Cumming, B. G., and Landy, M. S. Integration of stereopsis and motion shape cues. Vision research 34, 17 (1994), 2259--2275.

[23]

Khalilbeigi, M., Lissermann, R., Kleine, W., and Steimle, J. Foldme: interacting with double-sided foldable displays. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction, ACM (2012), 33--40.

Digital Library

[24]

Kildal, J., Lucero, A., and Boberg, M. Twisting touch: combining deformation and touch as input within the same interaction cycle on handheld devices. In Proceedings of the 15th international conference on Human-computer interaction with mobile devices and services, ACM (2013), 237--246.

Digital Library

[25]

Kim, D., Hilliges, O., Izadi, S., Butler, A. D., Chen, J., Oikonomidis, I., and Olivier, P. Digits: freehand 3d interactions anywhere using a wrist-worn gloveless sensor. In Proceedings of the 25th annual ACM symposium on User interface software and technology, ACM (2012), 167--176.

Digital Library

[26]

Kim, S., Cao, X., Zhang, H., and Tan, D. Enabling concurrent dual views on common lcd screens. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2012), 2175--2184.

Digital Library

[27]

Lahey, B., Girouard, A., Burleson, W., and Vertegaal, R. Paperphone: understanding the use of bend gestures in mobile devices with flexible electronic paper displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2011), 1303--1312.

Digital Library

[28]

Lanman, D., and Luebke, D. Near-eye light field displays. ACM Transactions on Graphics (TOG) 32, 6 (2013), 220.

Digital Library

[29]

Lee, J. C., Hudson, S. E., and Tse, E. Foldable interactive displays. In Proceedings of the 21st annual ACM symposium on User interface software and technology, ACM (2008), 287--290.

Digital Library

[30]

Levoy, M., and Hanrahan, P. Light field rendering. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, ACM (1996), 31--42.

Digital Library

[31]

Li, H., Trutoiu, L., Olszewski, K., Wei, L., Trutna, T., Hsieh, P.-L., Nicholls, A., and Ma, C. Facial performance sensing head-mounted display. ACM Transactions on Graphics (TOG) 34, 4 (2015), 47.

Digital Library

[32]

Lippmann, G. Epreuves reversibles. photographies integrals. Comptes-Rendus Academie des Sciences 146 (1908), 446--451.

[33]

Liu, J., Au, O. K.-C., Fu, H., and Tai, C.-L. Two-finger gestures for 6dof manipulation of 3d objects. In Computer Graphics Forum, vol. 31, Wiley Online Library (2012), 2047--2055.

Digital Library

[34]

Luxexcel. Luxexcel -- 3d printed optics. https://www.luxexcel.com/.

[35]

Martinet, A., Casiez, G., and Grisoni, L. The design and evaluation of 3d positioning techniques for multitouch displays. In 3D User Interfaces (3DUI), 2010 IEEE Symposium on, IEEE (2010), 115--118.

Digital Library

[36]

Martinez Plasencia, D., Karnik, A., Martinez Muñoz, J., and Subramanian, S. Portallax: bringing 3d displays capabilities to handhelds. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services, ACM (2014), 145--154.

Digital Library

[37]

Masliah, M. R., and Milgram, P. Measuring the allocation of control in a 6 degree-of-freedom docking experiment. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, ACM (2000), 25--32.

Digital Library

[38]

Microsoft. Hololens. https://www.microsoft.com/microsoft-hololens/en-us.

[39]

NASA TLX. http://humansystems.arc.nasa.gov/groups/tlx/paperpenc il.html.

[40]

Nintendo. 3ds. http://www.nintendo.com/3ds.

[41]

Oculus. Rift. https://www3.oculus.com/en-us/rift/.

[42]

Oikonomidis, I., Kyriazis, N., and Argyros, A. A. Efficient model-based 3d tracking of hand articulations using kinect. In BmVC, vol. 1 (2011), 3.

[43]

Ogniewski, J., and Ragnemalm, I. Autostereoscopy and motion parallax for mobile computer games using commercially available hardware. International Journal of Computer Information Systems and Industrial Management Applications 3 (2011), 480--488.

[44]

Peng, Y., Zhong, Q., Han, X., Yuan, L., Wang, R., Bao, H., Li, H., and Liu, X. 42.4 l: Late-news paper: Footprint of scalable 3d telecommunication: Using integral light field display and kinect-based capture. In SID Symposium Digest of Technical Papers, vol. 44, Wiley Online Library (2013), 589--592.

[45]

Reisman, J. L., Davidson, P. L., and Han, J. Y. A screen-space formulation for 2d and 3d direct manipulation. In Proceedings of the 22nd annual ACM symposium on User interface software and technology, ACM (2009), 69--78.

Digital Library

[46]

Rendl, C., Kim, D., Fanello, S., Parzer, P., Rhemann, C., Taylor, J., Zirkl, M., Scheipl, G., Rothländer, T., Haller, M., et al. Flexsense: a transparent self-sensing deformable surface. In Proceedings of the 27th annual ACM symposium on User interface software and technology, ACM (2014), 129--138.

Digital Library

[47]

Rendl, C., Kim, D., Parzer, P., Fanello, S., Zirkl, M., Scheipl, G., Haller, M., and Izadi, S. Flexcase: Enhancing mobile interaction with a flexible sensing and display cover. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, ACM (2016), 5138--5150.

Digital Library

[48]

RFDuino. http://www.rfduino.com/.

[49]

Rio. Angry birds. https://www.angrybirds.com/.

[50]

Scheurich, D., and Stuerzlinger, W. A one-handed multi-touch mating method for 3d rotations. In CHI'13 Extended Abstracts on Human Factors in Computing Systems, ACM (2013), 1623--1628.

Digital Library

[51]

Schwesig, C., Poupyrev, I., and Mori, E. Gummi: a bendable computer. In Proceedings of the SIGCHI conference on Human factors in computing systems, ACM (2004), 263--270.

Digital Library

[52]

Sharp, T., Keskin, C., Robertson, D., Taylor, J., Shotton, J., Kim, D., Rhemann, C., Leichter, I., Vinnikov, A., Wei, Y., et al. Accurate, robust, and flexible real-time hand tracking. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM (2015), 3633--3642.

Digital Library

[53]

Steimle, J., Jordt, A., and Maes, P. Flexpad: highly flexible bending interactions for projected handheld displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2013), 237--246.

Digital Library

[54]

Techradar. Philips to launch glasses-free 3d tv in 2013. http://www.techradar.com/us/news/television/hdtv/phili ps-to-launch-glasses-free-3d-tv-in-2013--713951.

[55]

Tagliasacchi, A., Schröder, M., Tkach, A., Bouaziz, S., Botsch, M., and Pauly, M. Robust articulated-icp for real-time hand tracking. In Computer Graphics Forum, vol. 34, Wiley Online Library (2015), 101--114.

[56]

Telkenaroglu, C., and Capin, T. Dual-finger 3d interaction techniques for mobile devices. Personal and ubiquitous computing 17, 7 (2013), 1551--1572.

Digital Library

[57]

Tompkin, J., Muff, S., Jakuschevskij, S., McCann, J., Kautz, J., Alexa, M., and Matusik, W. Interactive light field painting. In ACM SIGGRAPH 2012 Emerging Technologies, ACM (2012), 12.

Digital Library

[58]

Valkov, D., Giesler, A., and Hinrichs, K. H. Imperceptible depth shifts for touch interaction with stereoscopic objects. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2014), 227--236.

Digital Library

[59]

Zhai, S. Human performance in six degree of freedom input control. PhD thesis, University of Toronto, 1995.

[60]

Zhai, S., Milgram, P., and Buxton, W. The influence of muscle groups on performance of multiple degree-offreedom input. In Proceedings of the SIGCHI conference on Human factors in computing systems, ACM (1996), 308--315.

Digital Library

Cited By

Eady AGuerrero ACheung VThue DGirouard A(2024)BendAide: A Deformable Interface to Augment Touchscreen Mobile DevicesProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670960(1-10)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670960
Yeh YIrudayaraj AVogel D(2024)Single-handed Folding Interactions with a Modified Clamshell Flip PhoneProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642554(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642554
Wang JQin ZMa YYang YHuang HGuo CYue Q(2023)Micro-resonant cavity organic light-emitting diode with high refractive index contrast dielectric metasurfaces for naked-eye 3D displayOptics Communications10.1016/j.optcom.2022.129251532(129251)Online publication date: Apr-2023
https://doi.org/10.1016/j.optcom.2022.129251
Show More Cited By

Index Terms

HoloFlex: A Flexible Light-Field Smartphone with a Microlens Array and a P-OLED Touchscreen
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

HoloFlex: A Flexible Holographic Smartphone with Bend Input
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems

We present HoloFlex, a 3D flexible smartphone featuring a lightfield display with 16,640 microlenses. HoloFlex renders "holographic" images providing motion parallax and stereoscopy to multiple users with a resolution of 160x104 pixels, without glasses. ...
Rendering for an interactive 360° light field display
SIGGRAPH '07: ACM SIGGRAPH 2007 papers

We describe a set of rendering techniques for an autostereoscopic light field display able to present interactive 3D graphics to multiple simultaneous viewers 360 degrees around the display. The display consists of a high-speed video projector, a ...
BiDi screen: a thin, depth-sensing LCD for 3D interaction using light fields
SIGGRAPH Asia '09: ACM SIGGRAPH Asia 2009 papers

We transform an LCD into a display that supports both 2D multi-touch and unencumbered 3D gestures. Our BiDirectional (BiDi) screen, capable of both image capture and display, is inspired by emerging LCDs that use embedded optical sensors to detect ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '16: Proceedings of the 29th Annual Symposium on User Interface Software and Technology

October 2016

908 pages

ISBN:9781450341899

DOI:10.1145/2984511

General Chairs:
Jun Rekimoto
The University of Tokyo
,
Takeo Igarashi
The University of Tokyo
,
Program Chairs:
Jacob O. Wobbrock
University of Washington
,
Daniel Avrahami
FXPAL

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UIST '16

Sponsor:

UIST '16: The 29th Annual ACM Symposium on User Interface Software and Technology

October 16 - 19, 2016

Tokyo, Japan

Acceptance Rates

UIST '16 Paper Acceptance Rate 79 of 384 submissions, 21%;

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
678
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)2

Reflects downloads up to 21 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Eady AGuerrero ACheung VThue DGirouard A(2024)BendAide: A Deformable Interface to Augment Touchscreen Mobile DevicesProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670960(1-10)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670960
Yeh YIrudayaraj AVogel D(2024)Single-handed Folding Interactions with a Modified Clamshell Flip PhoneProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642554(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642554
Wang JQin ZMa YYang YHuang HGuo CYue Q(2023)Micro-resonant cavity organic light-emitting diode with high refractive index contrast dielectric metasurfaces for naked-eye 3D displayOptics Communications10.1016/j.optcom.2022.129251532(129251)Online publication date: Apr-2023
https://doi.org/10.1016/j.optcom.2022.129251
Pan XZheng MXu XXu ZCampbell A(2022)How can the Additional Motion Parallax along the y and z-axis Affect Viewer’s 3D Perception?: A Generic Approach and Evaluation2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR55827.2022.00040(252-259)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR55827.2022.00040
Liu HSun ZChen YZhang WChen XWong C(2022)Laser Processing of Flexible In-Plane Micro-supercapacitors: Progresses in Advanced Manufacturing of Nanostructured ElectrodesACS Nano10.1021/acsnano.2c0281216:7(10088-10129)Online publication date: 5-Jul-2022
https://doi.org/10.1021/acsnano.2c02812
Panzirsch MWeber BBechtel NGrabner NLingenauber M(2022)Light‐field head‐mounted displays reduce the visual effort: A user studyJournal of the Society for Information Display10.1002/jsid.111230:4(319-334)Online publication date: Mar-2022
https://doi.org/10.1002/jsid.1112
Yin JBai SHan YZhang XDeng SWang S(2021)The Study of Bending and Twisting Input Modalities in Deformable InterfacesElectronics10.3390/electronics1023299110:23(2991)Online publication date: 1-Dec-2021
https://doi.org/10.3390/electronics10232991
Zeng JDeng HZhu YWessely MKilian AMueller S(2021)Lenticular Objects: 3D Printed Objects with Lenticular Lens Surfaces That Can Change their Appearance Depending on the ViewpointThe 34th Annual ACM Symposium on User Interface Software and Technology10.1145/3472749.3474815(1184-1196)Online publication date: 10-Oct-2021
https://dl.acm.org/doi/10.1145/3472749.3474815
Ortega MGoguey ALee BLee GScott STory MKim J(2019)BEXHI: A Mechanical Structure for Prototyping Bendable and EXpandable Handheld InterfacesProceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces10.1145/3343055.3359703(269-273)Online publication date: 10-Nov-2019
https://dl.acm.org/doi/10.1145/3343055.3359703
Boem ATroiano GHarrison SBardzell SNeustaedter CTatar D(2019)Non-Rigid HCIProceedings of the 2019 on Designing Interactive Systems Conference10.1145/3322276.3322347(885-906)Online publication date: 18-Jun-2019
https://dl.acm.org/doi/10.1145/3322276.3322347
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents