Enhancing hand-object interactions in virtual reality for precision manual tasks
Authors: 
Madhur Mangalam, Sanjay Oruganti, Gavin Buckingham, Christoph W. BorstAuthors Info & Claims
Abstract
The realism and immersion of Virtual Reality (VR) experiences depend on the quality of interactions between the virtual hand and virtual objects. The current drawbacks, such as seemingly artificial hand-object interactions and erratic post-collision behaviors of both virtual objects and the virtual hand, curtail the effectiveness of VR in tasks requiring precise manipulation, precluding sustained and successful adoption of VR in precision manual tasks. To address these limitations, we advocate a strategic approach across three related domains: (1) Developing a sufficiently realistic virtual hand model (i.e., a set of rigid bodies or deformable meshes) that can implement the complex movements of a biological hand. (2) Exploiting synergistic patterns of multi-digit motion and contact forces revealed by research on neuroscience, psychophysics, and manual actions to develop hand-object collision handling algorithms. (3) Implementing seamless and fluid releases of whole-hand virtual grasps, especially involving complex grasps and in-hand manipulation tasks. This article explores various aspects of virtual grasping that go beyond traditional physics simulations, proposing innovative solutions to overcome technical barriers across these domains.
References
[1]
Abi-Rafeh J, Zammit D, Jaberi MM, Al-Halabi B, and Thibaudeau S Nonbiological microsurgery simulators in plastic surgery training: a systematic review Plast Reconstr Surg 2019 144 3 496e-507e
[2]
Andrychowicz OM, Baker B, Chociej M, Jozefowicz R, McGrew B, Pachocki J, Petron A, Plappert M, Powell G, Ray A, et al. Learning dexterous in-hand manipulation Int J Robot Res 2020 39 1 3-20
[3]
Argelaguet F and Andujar C A survey of 3D object selection techniques for virtual environments Comput Graph 2013 37 3 121-136
[4]
Argelaguet F, Hoyet L, Trico M, and Lécuyer A (2016) The role of interaction in virtual embodiment: Effects of the virtual hand representation. In 2016 IEEE Virtual Reality (VR), pages 3–10.
[5]
Aslandere T, Dreyer D, and Pankratz F (2015) Virtual hand-button interaction in a generic virtual reality flight simulator. In 2015 IEEE Aerospace Conference, pages 1–8.
[6]
Badash I, Burtt K, Solorzano CA, and Carey JN Innovations in surgery simulation: a review of past, current and future techniques Ann Translat Med 2016 4 23 453
[7]
Belter JT, Segil JL, and SM, B. Mechanical design and performance specifications of anthropomorphic prosthetic hands: a review J Rehab Res Dev 2013 50 5 599-617
[8]
Bergamasco M, Degl’Innocenti P, and Bucciarelli D (1994) A realistic approach for grasping and moving virtual objects. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’94), volume 1, pages 717–724.
[9]
Bianchi M, Salaris P, and Bicchi A Synergy-based hand pose sensing: Reconstruction enhancement Int J Robot Res 2013 32 4 396-406
[10]
Bianchi M, Salaris P, Turco A, Carbonaro N, and Bicchi A (2012) On the use of postural synergies to improve human hand pose reconstruction. In 2012 IEEE Haptics Symposium (HAPTICS), pages 91–98.
[11]
Bicchi A Hands for dexterous manipulation and robust grasping: a difficult road toward simplicity IEEE Trans Robot Autom 2000 16 6 652-662
[12]
Bicchi A, Gabiccini M, and Santello M Modelling natural and artificial hands with synergies Philos Trans Royal Soc b: Biol Sci 2011 366 1581 3153-3161
[13]
Biddiss EA and Chau TT Upper limb prosthesis use and abandonment: a survey of the last 25 years Prosthet Orthot Int 2007 31 3 236-257
[14]
Blaga AD, Frutos-Pascual M, Creed C and Williams I (2021) A grasp on reality: Understanding grasp- ing patterns for object interaction in real and virtual environments. In 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pages 391–396. IEEE.
[15]
Borst CW and Indugula AP Realistic virtual grasping Virtual Reality 2005 2005 91-98
[16]
Borst CW and Indugula AP A spring model for whole-hand virtual grasping Presence 2006 15 1 47-61
[17]
Borst CW and Volz RA Evaluation of a haptic mixed reality system for interactions with a virtual control panel Presence: Teleop Virtual Environ 2005 14 6 677-696
[18]
Boulic R, Rezzonico S, and Thalmann D (1996) Multi-finger manipulation of virtual objects. In VRST ’96: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, pages 67–74.
[19]
Bouwsema H, van der Sluis CK, and Bongers RM Learning to control opening and closing a myoelectric hand Arch Phys Med Rehabil 2010 91 9 1442-1446
[20]
Brown CY and Asada HH (2007) Inter-finger coordination and postural synergies in robot hands via mechanical implementation of principal components analysis. In 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 2877–2882.
[21]
Buckingham G Hand tracking for immersive virtual reality: opportunities and challenges Front Virtual Reality 2021 2 728561
[22]
Burdea GC (2000) Haptics issues in virtual environments. In Proceedings Computer Graphics International 2000, pages 295–302
[23]
Canales R, Normoyle A, Sun Y, Ye Y, Luca MD, and Jörg S (2019) Virtual grasping feedback and virtual hand ownership. In SAP ’19: ACM Symposium on Applied Perception 2019, page 4.
[24]
Catalano MG, Grioli G, Farnioli E, Serio A, Piazza C, and Bicchi A Adaptive synergies for the design and control of the Pisa/IIT softhand Int J Robot Res 2014 33 5 768-782
[25]
Catalano MG, Grioli G, Serio A, Farnioli E, Piazza C, and Bicchi A (2012) Adaptive synergies for a humanoid robot hand. In 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012), pages 7–14.
[26]
Crajé C, Lukos JR, Ansuini C, Gordon AM, and Santello M The effects of task and content on digit placement on a bottle Exp Brain Res 2011 212 119-124
[27]
Crajé C, Santello M, and Gordon AM Effects of visual cues of object density on perception and anticipatory control of dexterous manipulation PLoS ONE 2013 8 10 e76855
[28]
Culbertson H, Schorr SB, and Okamura AM Haptics: The present and future of artificial touch sensation Annu Rev Control, Robot, Auton Syst 2018 1 385-409
[29]
D’Anna E, Valle G, Mazzoni A, Strauss I, Iberite F, Patton J, Petrini FM, Raspopovic S, Granata G, Di Iorio R, et al. A closed-loop hand prosthesis with simultaneous intraneural tactile and position feedback Sci Robot 2019 4 27 eaau8892
[30]
Della Santina C, Bianchi M, Averta G, Ciotti S, Arapi V, Fani S, Battaglia E, Catalano MG, Santello M, and Bicchi A Postural hand synergies during environmental constraint exploitation Front Neurorobot 2017 11 41
[31]
Delrieu T, Weistroffer V, and Gazeau JP (2020) Precise and realistic grasping and manipulation in Virtual Reality without force feedback. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pages 266–274.
[32]
Endo Y, Tada M, and Mochimaru M Reconstructing individual hand models from motion capture data J Comput Design Eng 2014 1 1 1-12
[33]
Engdahl SM, Meehan SK, and Gates DH Differential experiences of embodiment between body-powered and myoelectric prosthesis users Sci Rep 2020 10 15471
[34]
Erez T, Tassa Y, and Todorov E (2015) Simulation tools for model-based robotics: Comparison of Bullet, Havok, MuJoCo, ODE and PhysX. In 2015 IEEE International Conference on Robotics and Automation (ICRA), pages 4397–4404.
[35]
Fabiani L, Burdea G, Langrana N, and Gomez D (1996) Human interface using the Rutgers Master II force feedback interface. In Proceedings of the IEEE 1996 Virtual Reality Annual International Symposium, pages 54–59.
[36]
Fu Q and Santello M Coordination between digit forces and positions: interactions between anticipatory and feedback control J Neurophysiol 2014 111 7 1519-1528
[37]
Fu Q, Zhang W, and Santello M Anticipatory planning and control of grasp positions and forces for dexterous two-digit manipulation J Neurosci 2010 30 27 9117-9126
[38]
Funahashi K, Yasuda T, Yokoi S, and Toriwaki J-I A model for manipulation of objects with virtual hand in 3D virtual space Syst Comput Jpn 1999 30 11 22-32
[39]
Furmanek MP, Schettino LF, Yarossi M, Kirkman S, Adamovich SV, and Tunik E Coordination of reach-to-grasp in physical and haptic-free virtual environments J Neuroeng Rehabil 2019 16 78
[40]
Furmanek MP, Mangalam M, Lockwood K, Smith A, Yarossi M, and Tunik E Effects of sensory feedback and collider size on reach-to-grasp coordination in haptic-free virtual reality Front Virtual Reality 2021 2 648529
[41]
Gabiccini M, Bicchi A, Prattichizzo D, and Malvezzi M On the role of hand synergies in the optimal choice of grasping forces Auton Robot 2011 31 235-252
[42]
Gan C, Schwartz J, Alter S, Mrowca D, Schrimpf M, Traer J, De Freitas J, Kubilius J, Bhandwaldar A, Haber N, et al. (2020) ThreeDWorld: a platform for interactive multi-modal physical simulation. arXiv preprint arXiv:2007.04954. https://arxiv.org/abs/2007.04954
[43]
Ganias G, Lougiakis C, Katifori A, Roussou M, Ioannidis Y, and Ioannidis IP Comparing different grasping visualizations for object manipulation in VR using controllers IEEE Trans Visual Comput Graphics 2023
[44]
Garre C, Hernández F, Gracia A, and Otaduy MA (2011) Interactive simulation of a deformable hand for haptic rendering. In 2011 IEEE World Haptics Conference, pages 239–244. 5945492
[45]
Geiger A, Bewersdorf I, Brandenburg E, and Stark R (2018) Visual feedback for grasping in virtual reality environments for an interface to instruct digital human models. In Ahram T and Falcão C, editors, Advances in Usability and User Experience. AHFE 2017. Advances in Intelligent Systems and Computing, Vol 607, pages 228–239. Springer, Cham, Switzerland.
[46]
Geng T, Lee M, and Hülse M Transferring human grasping synergies to a robot Mechatronics 2011 21 1 272-284
[47]
Gentner R and Classen J Modular organization of finger movements by the human central nervous system Neuron 2006 52 4 731-742
[48]
Gomelsky V (2016) Virtual reality takes fans inside the world of watches. https://www.nytimes.com/2016/11/ 15/fashion/watches-virtual-reality-roger-dubuis.html
[49]
Gustus A, Stillfried G, Visser J, Jörntell H, and van der Smagt P Human hand modelling: Kinematics, dy-namics, applications Biol Cybern 2012 106 741-755
[50]
Häger-Ross C and Schieber MH Quantifying the independence of human finger movements: comparisons of digits, hands, and movement frequencies J Neurosci 2000 20 22 8542-8550
[51]
Hallett M and Marsden C Ballistic flexion movements of the human thumb J Physiol 1979 294 1 33-50
[52]
Hameed A, Möller S, and Perkis A How good are virtual hands? Influences of input modality on motor tasks in virtual reality J Environ Psychol 2023 92 102137
[53]
Hamza-Lup FG, Bergeron K, and Newton D (2019) Haptic systems in user interfaces: State of the art survey. In ACM SE ’19: Proceedings of the 2019 ACM Southeast Conference, pages 141–148.
[54]
Han S, Liu B, Cabezas R, Twigg CD, Zhang P, Petkau J, Yu T-H, Tai C-J, Akbay M, Wang Z, et al. MEgATrack: monochrome egocentric articulated hand-tracking for virtual reality ACM Trans Gr 2020 39 4 87
[55]
Hight MP, Fussell SG, Kurkchubasche MA, and Hummell IJ Effectiveness of virtual reality simulations for civilian, ab initio pilot training J Aviat/aerosp Educ Res 2022 31 1 1
[56]
Hilliges O, Izadi S, Wilson AD, Hodges S, Garcia-Mendoza A, and Butz A (2009) Interactions in the air: Adding further depth to interactive tabletops. In UIST ’09: Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology, pages 139–148.
[57]
Hilliges O, Kim D, Izadi S, Weiss M, and Wilson A (2012) HoloDesk: Direct 3D interactions with a situated see-through display. In CHI ’12: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 2421–2430.
[58]
Hirota K and Hirose M (2003) Dexterous object manipulation based on collision response. In IEEE Virtual Reality, 2003. Proceedings. pages 232–239.
[59]
Hirota K and Tagawa K (2016) Interaction with virtual object using deformable hand. In 2016 IEEE Virtual Reality (VR), pages 49–56.
[60]
Höll M, Oberweger M, Arth C, and Lepetit V (2018) Efficient physics-based implementation for realistic hand-object interaction in virtual reality. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pages 175–182.
[61]
Holz D, Ullrich S, Wolter M, and Kuhlen T Multi-contact grasp interaction for virtual environments J Virtual Reality Broadcast 2008
[62]
Huagen W, Shuming G, and Qunsheng P (2004) Virtual grasping for virtual assembly tasks. In Third International Conference on Image and Graphics (ICIG’04), pages 448–451.
[63]
Isaac JH, Manivannan M, and Ravindran B Corrective filter based on kinematics of human hand for pose estimation Front Virtual Reality 2021 2 663618
[64]
Iwata H Artificial reality with force-feedback: development of desktop virtual space with compact master manipulator ACM SIGGRAPH Comput Gr 1990 24 4 165-170
[65]
Jacobs J and Froehlich B (2011) A soft hand model for physically-based manipulation of virtual objects. In 2011 IEEE Virtual Reality Conference, pages 11–18. IEEE.
[66]
Jacobs J, Stengel M, and Froehlich B (2012) A generalized god-object method for plausible finger-based interactions in virtual environments. In 2012 IEEE Symposium on 3D User Interfaces (3DUI), pages 43–51. IEEE.
[67]
Jiang L, Xia H, and Guo C A model-based system for real-time articulated hand tracking using a simple data glove and a depth camera Sensors 2019 19 21 4680
[68]
Jones LA and Lederman SJ Human Hand Function 2006 New York Oxford University Press
[69]
Jörg S, Ye Y, Mueller F, Neff M, and Zordan V (2020) Virtual hands in VR: Motion capture, synthesis, and perception. In SA ’20: SIGGRAPH Asia 2020 Courses, page 2. ACM New York, NY, USA.
[70]
Joy T, Ugur E, and Ayhan I Trick the body trick the mind: avatar representation affects the perception of available action possibilities in virtual reality Virtual Reality 2022 26 615-629
[71]
Jung S, Sandor C, Wisniewski PJ, and Hughes CE (2017) Realme: The influence of body and hand representations on body ownership and presence. In SUI ’17: Proceedings of the 5th Symposium on Spatial User Interaction, pages 3–11.
[72]
Jung S, Bruder G, Wisniewski PJ, Sandor C, and Hughes CE (2018) Over my hand: Using a personalized hand in VR to improve object size estimation, body ownership, and presence. In SUI ’18: Proceedings of the 2018 ACM Symposium on Spatial User Interaction, pages 60–68.
[73]
Kamper DG, Hornby TG, and Rymer WZ Extrinsic flexor muscles generate concurrent flexion of all three finger joints J Biomech 2002 35 12 1581-1589
[74]
Kamper DG, Cruz EG, and Siegel MP Stereotypical fingertip trajectories during grasp J Neurophysiol 2003 90 6 3702-3710
[75]
Kilteni K, Normand J-M, Sanchez-Vives MV, and Slater M Extending body space in immersive virtual reality: a very long arm illusion PLoS ONE 2012 7 7 e40867
[76]
Kim JS and Park JM (2015) Physics-based hand interaction with virtual objects. In 2015 IEEE International Conference on Robotics and Automation (ICRA), pages 3814–3819.
[77]
Kim JS and Park JM (2016) Direct and realistic handover of a virtual object. In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems. (IROS), pages 994–999.
[78]
Kim D, Hilliges O, Izadi S, Butler AD, Chen J, Oikonomidis I, and Olivier P (2012) Digits: Freehand 3D interactions anywhere using a wrist-worn gloveless sensor. In UIST ’12: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, pages 167–176.
[79]
Kim CE and Vance JM Using VPS (Voxmap PointShell) as the basis for interaction in a virtual assembly environment In Int Design Eng Tech Conf Comput Inf Eng Conf 2003 36991 1153-1161
[80]
Kim CE and Vance JM Collision detection and part interaction modeling to facilitate immersive virtual assembly methods J Comput Inf Sci Eng 2004 4 2 83-90
[81]
Kitamura Y, Higashi T, Masaki T, and Kishino F (1999) Virtual chopsticks: Object manipulation using multiple exact interactions. In Proceedings IEEE Virtual Reality, pages 198–204.
[82]
Konnaris C, Gavriel C, Thomik AA, and Faisal AA (2016) Ethohand: A dexterous robotic hand with ball-joint thumb enables complex in-hand object manipulation. In 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics, pages 1154–1159.
[83]
Korkut EH and Surer E Visualization in virtual reality: a systematic review Virtual Reality 2023 27 2 1447-1480
[84]
Koulieris GA, Aks¸it K, Stengel M, Mantiuk RK, Mania K, and Richardt C Near-eye display and tracking technologies for virtual and augmented reality Comput Gr Forum 2019 38 2 493-519
[85]
Laffranchi M, Boccardo N, Traverso S, Lombardi L, Canepa M, Lince A, Semprini M, Saglia JA, Naceri A, Sacchetti R, et al. The hannes hand prosthesis replicates the key biological properties of the human hand Sci Robot 2020 5 46 eabb0467
[86]
Laha B, Bowman DA, and Socha JJ Effects of VR system fidelity on analyzing isosurface visualization of volume datasets IEEE Trans Visual Comput Gr 2014 20 4 513-522
[87]
Lam MC, Arshad H, Prabuwono AS, Tan SY, and Kahaki SMM Interaction techniques in desktop virtual environment: The study of visual feedback and precise manipulation method Multimed Tools Appl 2018 77 16367-16398
[88]
Latash ML, Scholz JP, and Schöner G Toward a new theory of motor synergies Mot Control 2007 11 3 276-308
[89]
Lee-Miller T, Marneweck M, Santello M, and Gordon AM Visual cues of object properties differentially affect anticipatory planning of digit forces and placement PLoS ONE 2016 11 4 e0154033
[90]
Leo A, Handjaras G, Bianchi M, Marino H, Gabiccini M, Guidi A, Scilingo EP, Pietrini P, Bicchi A, Santello M, et al. A synergy-based hand control is encoded in human motor cortical areas Elife 2016 5 e13420
[91]
Li Y, Fu JL, and Pollard NS Data-driven grasp synthesis using shape matching and task-based pruning IEEE Trans Visual Comput Graphics 2007 13 4 732-747
[92]
Li S, Sheng X, Liu H, and Zhu X Design of a myoelectric prosthetic hand implementing postural synergy mechanically Ind Robot 2014 41 5 447-455
[93]
Li J, Xu Y, Ni J, and Wang Q Glove-based virtual hand grasping for virtual mechanical assembly Assem Autom 2016 36 4 349-361
[94]
Lim WN, Yap KM, Lee Y, Wee C, and Yen CC A systematic review of weight perception in virtual reality: techniques, challenges, and road ahead IEEE Access 2021 9 163253-163283
[95]
Lin L and Jörg, S (2016a) The effect of realism on the virtual hand illusion. In 2016 IEEE Virtual Reality (VR), pages 217–218.
[96]
Lin L and Jörg S (2016b) Need a hand? How appearance affects the virtual hand illusion. In SAP ’16: Proceedings of the ACM Symposium on Applied Perception, pages 69–76.
[97]
Lin L, Normoyle A, Adkins A, Sun Y, Robb A, Ye Y, Di Luca M, and Jörg S (2019) The effect of and size and interaction modality on the virtual hand illusion. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pages 510–518.
[98]
Liu H, Zhang Z, Xie X, Zhu Y, Liu Y, Wang Y, and Zhu SC (2019) High-fidelity grasping in virtual reality using a glove-based system. In 2019 International Conference on Robotics and Automation (ICRA), pages 5180–5186.
[99]
Lougiakis C, Katifori A, Roussou M, and Ioannidis IP (2020) Effects of virtual hand representation on interaction and embodiment in HMD-based virtual environments using controllers. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pages 510–518.
[100]
Maekawa H and Hollerbach JM (1998) Haptic display for object grasping and manipulating in virtual environment. In 1998 IEEE International Conference on Robotics and Automation), pages 2566–2573.
[101]
Mangalam M, Yarossi M, Furmanek MP, and Tunik E Control of aperture closure during reach- to-grasp movements in immersive haptic-free virtual reality Exp Brain Res 2021 239 5 1651-1665
[102]
McKnight RR, Pean CA, Buck JS, Hwang JS, Hsu JR, and Pierrie SN Virtual reality and augmented reality—translating surgical training into surgical technique Curr Rev Musculoskelet Med 2020 13 663-674
[103]
McMahan RP, Bowman DA, Zielinski DJ, and Brady RB Evaluating display fidelity and interaction fidelity in a virtual reality game IEEE Trans Visual Comput Graphics 2012 18 4 626-633
[104]
Melmoth DR and Grant S Advantages of binocular vision for the control of reaching and grasping Exp Brain Res 2006 171 371-388
[105]
Miller AT, Knoop S, Christensen HI, and Allen PK (2003) Automatic grasp planning using shape primitives. In 2003 IEEE International Conference on Robotics and Automation, pages 1824–1829.
[106]
Moehring M and Froehlich B (2010) Enabling functional validation of virtual cars through natural interaction metaphors. In 2010 IEEE Virtual Reality Conference (VR), pages 27–34.
[107]
Moehring M and Froehlich B (2011) Effective manipulation of virtual objects within arm’s reach. In 2011 IEEE Virtual Reality Conference, pages 131–138.
[108]
Moustakas K, Tzovaras D, and Strintzis MG SQ-map: Efficient layered collision detection and haptic rendering IEEE Trans Visual Comput Gr 2006 13 1 80-93
[109]
Mulatto S, Formaglio A, Malvezzi M, and Prattichizzo D (2010) Animating a synergy-based deformable hand avatar for haptic grasping. In EuroHaptics 2010. Lecture Notes in Computer Science, pages 203–210. Springer, Berlin, Heidelberg, Germany.
[110]
Nabioyuni M and Bowman DA (2015) An evaluation of the effects of hyper-natural components of interaction fidelity on locomotion performance in virtual reality. In ICAT - EGVE ’15: Proceedings of the 25th International Conference on Artificial Reality and Telexistence and 20th Eurographics Symposium on Virtual Environments, pages 167–174.
[111]
Napier JR The prehensile movements of the human hand J Bone Joint Surg 1956 38 4 902-913
[112]
Nasim K and Kim YJ (2016) Physics-based interactive virtual grasping. In HCIK ’16: Proceedings of HCI Korea, pages 114–120. Nurimedia.
[113]
Nilsson NC, Zenner A, and Simeone AL Propping up virtual reality with haptic proxies IEEE Comput Graphics Appl 2021 41 5 104-112
[114]
Ogawa N, Narumi T, and Hirose M (2018) Object size perception in immersive virtual reality: Avatar realism affects the way we perceive. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pages 647–648.
[115]
Oprea S, Martinez-Gonzalez P, Garcia-Garcia A, Castro-Vargas JA, Orts-Escolano S, and Garcia-Rodriguez J A visually realistic grasping system for object manipulation and interaction in virtual reality environments Comput Graph 2019 83 77-86
[116]
Osawa N (2005) Enhanced hand manipulation for efficient and precise positioning and release. In EGVE’05: Proceedings of the 11th Eurographics Conference on Virtual Environments, pages 221–222
[117]
Osawa N (2006) Automatic adjustments for efficient and precise positioning and release of virtual objects. In VRCIA ’06: Proceedings of the 2006 ACM International Conference on Virtual Reality Continuum and its Applications, pages 121–128.
[118]
Owlchemylabs (2017). Tomato presence! http://owlchemylabs.com/tomatopresence/.
[119]
Pan,Y and Steed A (2019) Avatar type affects performance of cognitive tasks in virtual reality. In Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology, page 6.
[120]
Patel GK, Castellini C, Hahne JM, Farina D, and Dosen S A classification method for myoelectric control of hand prostheses inspired by muscle coordination IEEE Trans Neural Syst Rehabil Eng 2018 26 9 1745-1755
[121]
Perez AG, Cirio G, Hernandez F, Garre C, and Otaduy MA (2013) Strain limiting for soft finger contact simulation. In 2013 World Haptics Conference (WHC), pages 79–84.
[122]
Pollard NS and Zordan VB (2005) Physically based grasping control from example. In SCA ’05: Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pages 311–318.
[123]
Prachyabrued M and Borst CW (2011) Dropping the ball: Releasing a virtual grasp. In 2011 IEEE Symposium on 3D User Interfaces (3DUI), pages 59–66.
[124]
Prachyabrued, M. and Borst, C. W. (2012b). Visual interpenetration tradeoffs in whole-hand virtual grasping. In 2012 IEEE Symposium on 3D User Interfaces (3DUI), pages 39–42
[125]
Prachyabrued M and Borst, CW (2014) Visual feedback for virtual grasping. In 2014 IEEE Symposium on 3D User Interfaces (3DUI), pages 19–26.
[126]
Prachyabrued M and Borst CW Virtual grasp release method and evaluation Int J Hum Comput Stud 2012 70 11 828-848
[127]
Prachyabrued M and Borst CW Design and evaluation of visual interpenetration cues in virtual grasping IEEE Trans Visual Comput Graphics 2016 22 6 1718-1731
[128]
Prasetya F, Fajri BR, Wulansari RE, Primawati P, and Fortuna A (2023) Virtual reality adventures as an effort to improve the quality of welding technology learning during a pandemic. International Journal of Online and Biomedical (iJOE). https://ssrn.com/abstract=4746890
[129]
Pyasik M, Tieri G, and Pia L Visual appearance of the virtual hand affects embodiment in the virtual hand illusion Sci Rep 2020 10 5412
[130]
Rearick MP, Casares A, and Santello M Task-dependent modulation of multi-digit force coordination patterns J Neurophysiol 2003 89 3 1317-1326
[131]
Reilly KT and Schieber MH Incomplete functional subdivision of the human multitendoned finger muscle flexor digitorum profundus: an electromyographic study J Neurophysiol 2003 90 4 2560-2570
[132]
Rijpkema H and Girard M Computer animation of knowledge-based human grasping ACM SIGGRAPH Comput Gr 1991 25 4 339-348
[133]
Roth D, Lugrin JL, Büser J, Bente G, Fuhrmann A, and Latoschik ME (2016) A simplified inverse kinematic approach for embodied VR applications. In 2016 IEEE Virtual Reality (VR), pages 275–276.
[134]
Santello M, Flanders M, and Soechting JF Postural hand synergies for tool use J Neurosci 1998 18 23 10105-10115
[135]
Santello M, Bianchi M, Gabiccini M, Ricciardi E, Salvietti G, Prattichizzo D, Ernst M, Moscatelli A, Jörntell H, Kappers AM, et al. Hand synergies: Integration of robotics and neuroscience for understanding the control of biological and artificial hands Phys Life Rev 2016 17 1-23
[136]
Schettino LF, Adamovich SV, and Poizner H Effects of object shape and visual feedback on hand configuration during grasping Exp Brain Res 2003 151 158-166
[137]
Schieber MH and Santello M Hand function: peripheral and central constraints on performance J Appl Physiol 2004 96 6 2293-2300
[138]
Schwartz A and Reimer D (2015) Being there: designing standing VR experiences with tracked controllers. https://www.gdcvault.com/play/1024256/-Job-Simulator-Postmortem-VR
[139]
Schwartz A and Reimer D (2017) ‘Job Simulator’ postmortem: VR design, tech, and business lessons learned. https://www.gdcvault.com/play/1024256/-Job-Simulator-Postmortem-VR
[140]
Schwind V, Lin L, Di Luca M, Jörg S, and Hillis J (2018) Touch with foreign hands: The effect of virtual hand appearance on visual-haptic integration. In: SAP ’18: Proceedings of the 15th ACM Symposium on Applied Perception, page 9.
[141]
Smith B, Wu C, Wen H, Peluse P, Sheikh Y, Hodgins JK, and Shiratori T Constraining dense hand surface tracking with elasticity ACM Trans Gr 2020 39 6 219
[142]
Sobinov AR and Bensmaia SJ The neural mechanisms of manual dexterity Nat Rev Neurosci 2021 22 12 741-757
[143]
Steed A, Takala TM, Archer D, Lages W, and Lindeman RW Directions for 3D user interface research from consumer VR games IEEE Trans Visual Comput Graphics 2021 27 11 4171-4182
[144]
Sueda S, Kaufman A, and Pai DK (2008) Musculotendon simulation for hand animation. In SIGGRAPH ’08: ACM SIGGRAPH 2008 papers, page 83. ACM New York, NY, USA.
[145]
Sundaram S, Kellnhofer P, Li Y, Zhu J-Y, Torralba A, and Matusik W Learning the signatures of the human grasp using a scalable tactile glove Nature 2019 569 7758 698-702
[146]
Syukur A, Andono PN, Hastuti K, and Syarif AM Immersive and challenging experiences through a virtual reality musical instruments game: an approach to gamelan preservation J Metaverse 2023 3 1 34-42
[147]
Talvas A, Marchal M, Duriez C, and Otaduy MA Aggregate constraints for virtual manipulation with soft fingers IEEE Trans Visual Comput Gr 2015 21 4 452-461
[148]
Taylor J, Bordeaux L, Cashman T, Corish B, Keskin C, Sharp T, Soto E, Sweeney D, Valentin J, Luff B, et al. Efficient and precise interactive hand tracking through joint, continuous optimization of pose and correspondences ACM Trans Gr 2016 35 4 1-12
[149]
Tieri G, Gioia A, Scandola M, Pavone EF, and Aglioti SM Visual appearance of a virtual upper limb modulates the temperature of the real hand: a thermal imaging study in immersive virtual reality Eur J Neurosci 2017 45 9 1141-1151
[150]
Ting LH and McKay JL Neuromechanics of muscle synergies for posture and movement Curr Opin Neurobiol 2007 17 6 622-628
[151]
Todorov E, Erez T, and Tassa Y (2012) Mujoco: a physics engine for model-based control. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026–5033.
[152]
Tzafestas CS Whole-hand kinesthetic feedback and haptic perception in dextrous virtual manipulation IEEE Trans Syst, Man, Cybernet-Part a: Syst Humans 2003 33 1 100-113
[153]
Ullmann T and Sauer J (2000). Intuitive virtual grasping for non haptic environments. In Proceedings the Eighth Pacific Conference on Computer Graphics and Applications, pages 373–457.
[154]
Vaillant R, Barthe L, Guennebaud G, Cani M-P, Rohmer D, Wyvill B, Gourmel O, and Paulin M Implicit skinning: real-time skin deformation with contact modeling ACM Trans Gr 2013 32 4 125
[155]
van Duinen H and Gandevia SC Constraints for control of the human hand J Physiol 2011 589 23 5583-5593
[156]
van Veldhuizen M and Yang X The effect of semi-transparent and interpenetrable hands on object manipulation in virtual reality IEEE Access 2021 9 17572-17583
[157]
Vizcay S, Kourtesis P, Argelaguet F, Pacchierotti C, and Marchal M (2021) Electrotactile feedback in virtual reality for precise and accurate contact rendering. arXiv preprint arXiv:2102.00259.
[158]
Vosinakis S and Koutsabasis P Evaluation of visual feedback techniques for virtual grasping with bare hands using Leap Motion and Oculus Rift Virtual Reality 2018 22 1 47-62
[159]
Wan H, Luo Y, Gao S, and Peng Q (2004) Realistic virtual hand modeling with applications for virtual grasping. In VRCAI ’04: Proceedings of the 2004 ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry, pages 81–87.
[160]
Wee C, Yap KM, and Lim WN Haptic interfaces for virtual reality: challenges and research directions IEEE Access 2021 9 112145-112162
[161]
Weiss EJ and Flanders M Muscular and postural synergies of the human hand J Neurophysiol 2004 92 1 523-535
[162]
Wheatland N, Wang Y, Song H, Neff M, Zordan V, and Jörg S State of the art in hand and finger modeling and animation Comput Gr Forum 2015 34 2 735-760
[163]
Wilson FR The Hand: How Its Use Shapes the Brain, Language, and Human Culture 1999 New York Vintage
[164]
Winges SA, Weber DJ, and Santello M The role of vision on hand preshaping during reach to grasp Exp Brain Res 2003 152 489-498
[165]
Wolfartsberger J, Zimmermann R, Obermeier G, and Niedermayr D Analyzing the potential of virtual reality-supported training for industrial assembly tasks Comput Ind 2023 147 103838
[166]
Wu X and Song J 3D modeling and design effect optimization of ceramics using virtual reality Comput-Aided Design Appl 2024 21 S12 34-48
[167]
Xiong J, Hsiang EL, He Z, Zhan T, and Wu ST Augmented reality and virtual reality displays: emerging technologies and future perspectives Light: Sci Appl 2021 10 1-13
[168]
Yin K, He Z, Xiong J, Zou J, Li K, and Wu S-T Virtual reality and augmented reality displays: Advances and future perspectives J Phys: Photon 2021 3 2 022010
[169]
Yu S, Liu Q, Johnson-Glenberg MC, Han M, Ma J, Ba S, and Wu L Promoting musical instrument learning in virtual reality environment: Effects of embodiment and visual cues Comput Educ 2023 198 104764
[170]
Yuan Y and Steed A (2010) Is the rubber hand illusion induced by immersive virtual reality? In: 2010 IEEE Virtual Reality Conference (VR), pages 95–102.
[171]
Yuan K, Xu C, Du Q, and Fu Y (2003) Collision detection for a haptic interface. In 2003 IEEE International Conference on Robotics, Intelligent Systems and Signal Processing, pages 278–283.
[172]
Zachmann G (2000) Virtual reality in assembly simulation-collision detection, simulation algorithms, and interaction techniques. PhD thesis, Graz University of Technology, Styria, Austria.
[173]
Zachmann G and Rettig A (2001) Natural and robust interaction in virtual assembly simulation. In: 8th ISPE International Conference on Concurrent Engineering: Research and Applications (CE2001), pages 425–434
[174]
Zecca M, Micera S, Carrozza MC, and Dario P Control of multifunctional prosthetic hands by processing the electromyographic signal Critic Rev Biomed Eng 2002 30 4–6 459-485
[175]
Zhan T, Yin K, Xiong J, He Z, and Wu S-T Augmented reality and virtual reality displays: perspectives and challenges Iscience 2020
[176]
Zhang H, Ye Y, Shiratori T, and Komura T Manipnet: Neural manipulation synthesis with a hand- object spatial representation ACM Trans Gr 2021 40 4 1-14
[177]
Zhao W, Zhang J, Min J, and Chai J Robust realtime physics-based motion control for human grasping ACM Trans Gr 2013 32 6 207
[178]
Zhu Z, Gao S, Wan H, Yang W (2006) Trajectory-based grasp interaction for virtual environments. In: Computer Graphics International Conference (pp. 300-311). Berlin, Heidelberg: Springer Berlin Heidelberg.
Index Terms
- Enhancing hand-object interactions in virtual reality for precision manual tasks
Index terms have been assigned to the content through auto-classification.
Recommendations
Enabling Reusable Haptic Props for Virtual Reality by Hand Displacement
MuC '21: Proceedings of Mensch und Computer 2021Virtual Reality (VR) enables compelling visual experiences. However, providing haptic feedback is still challenging. Previous work suggests utilizing haptic props to overcome such limitations and presents evidence that props could function as a single ...
The Gesture Authoring Space: Authoring Customised Hand Gestures for Grasping Virtual Objects in Immersive Virtual Environments
MuC '22: Proceedings of Mensch und Computer 2022Natural user interfaces are on the rise. Manufacturers for Augmented, Virtual, and Mixed Reality head mounted displays are increasingly integrating new sensors into their consumer grade products, allowing gesture recognition without additional hardware. ...
Can Haptic Feedback on One Virtual Object Increase the Presence of Another Virtual Object?
VRST '22: Proceedings of the 28th ACM Symposium on Virtual Reality Software and TechnologyThis paper investigated whether increased presence from experiencing haptic feedback on one virtual object can transfer to another virtual object. Two similar studies were run in different environments: an immersive virtual environment and a mixed ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© The Author(s) 2024.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 05 November 2024
Accepted: 20 September 2024
Received: 11 March 2023
Author Tags
Qualifiers
- Research-article
Funding Sources
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 18 Dec 2024