Abstract
This paper explores the relationship between facial expressions of emotions and the sense of touch, with the future aim of investigating their impact on emotion recognition deficits in individuals with neurological disorders. The developed application involves a two-part system: a test designer for experts and a training simulator for patients. Using a haptic device, patients assess emotions, textures, and sensations through a survey. The system employs a client-server architecture, allowing remote monitoring by experts and promoting patient expression. The application gathers subjective and objective data for subsequent analysis. To validate the system, a study was conducted with two psychologists to identify potential usability issues prior to testing with end users.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
3D Systems: Phantom Premium (2024). https://www.3dsystems.com/haptics-devices/3d-systems-phantom-premium
Adobe: Fuse (2024). https://www.adobe.com/es/products/fuse.htmlIn
Ahmed, I., Harjunen, V.J., Jacucci, G., Ravaja, N., Ruotsalo, T., Spapé, M.M.: Touching virtual humans: haptic responses reveal the emotional impact of affective agents. IEEE Trans. Affect. Comput. 14(1), 331–342 (2020)
Burdea, G.C., Coiffet, P.: Virtual Reality Technology. John Wiley & Sons, Hoboken (2017)
Ekman, P., Friesen, W.V.: Unmasking the face: a guide to recognizing emotions from facial clues. Ishk (1975)
Fernández-Sotos, P., García, A.S., Vicente-Querol, M.A., Lahera, G., Rodriguez-Jimenez, R., Fernandez-Caballero, A.: Validation of dynamic virtual faces for facial affect recognition. PLoS ONE 16(1), e0246001 (2021). https://doi.org/10.1371/journal.pone.0246001
Haptics House: Novint’s Falcon Haptic Device (2024). https://hapticshouse.com/pages/novints-falcon-haptic-device/
Jamil, M.H., Park, W., Eid, M.: Emotional responses to watching and touching 3d emotional face in a virtual environment. Virt. Real. 25(2), 553–564 (2021)
Katz, D., Krueger, L.E.: The World of Touch. Psychology press, Abingdon (1986)
Martin, S., Hillier, N.: Characterisation of the novint falcon haptic device for application as a robot manipulator. In: Australasian Conference on Robotics and Automation (ACRA), pp. 291–292. Citeseer (2009)
Martínez, J., García, A.S., Oliver, M., González, P., Molina, J.P.: The sense of touch as the last frontier in virtual reality technology. In: Virtual Reality Designs, pp. 218–246. CRC Press (2020)
Martínez, J., Martínez, D., Molina, J.P., González, P., García, A.S.: Comparison of force and vibrotactile feedback with direct stimulation for texture recognition. In: 2011 International Conference on Cyberworlds, pp. 62–68 (2011) https://doi.org/10.1109/CW.2011.23
Mixamo: Fuse (2024). https://store.steampowered.com/app/257400/Fuse/
Révész, G.: Psychology and art of the blind (1950)
Robles-De-La-Torre, G.: The importance of the sense of touch in virtual and real environments. IEEE Multimedia 13(3), 24–30 (2006). https://doi.org/10.1109/MMUL.2006.69
Rus-Calafell, M., Gutiérrez-Maldonado, J., Ribas-Sabaté, J.: A virtual reality-integrated program for improving social skills in patients with schizophrenia: a pilot study. J. Behav. Ther. Exp. Psychiat. 45(1), 81–89 (2014). https://doi.org/10.1016/j.jbtep.2013.09.002
Sachs, G., et al.: Training of affect recognition (TAR) in schizophrenia-impact on functional outcome. Schizophr. Res. 138(2–3), 262–267 (2012). https://doi.org/10.1016/j.schres.2012.03.005
Touchable Universe: 3D Haptics for Novint Falcon (2021). https://assetstore.unity.com/packages/tools/physics/3d-haptics-for-novint-falcon-119281/
Unity Technologies: Unity Real-Time Development Platform (2024). https://unity.com/
Unity Technologies: Unity User Manual: Prefabs (2024). https://docs.unity3d.com/Manual/Prefabs.html
Wegrzyn, M., Vogt, M., Kireclioglu, B., Schneider, J., Kissler, J.: Mapping the emotional face: how individual face parts contribute to successful emotion recognition. PloS one 12(5), e0177239 (2017). https://doi.org/10.1371/journal.pone.0177239
Acknowledgments
Grants PID2020-115220RB-C21 and PID2023-149753OB-C21, funded by MCIN/AEI/10.13039/501100011033, and 2022-GRIN-34436, funded by UCLM, all also funded by “ERDF: A way to make Europe”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ramínez, L.M., Molina, J.P., López, Á.L., de la Paz, F., Fernández-Caballero, A., García, A.S. (2024). Using Touch to Improve Emotion Recognition: Proposed System and Expert Validation. In: Ferrández Vicente, J.M., Val Calvo, M., Adeli, H. (eds) Artificial Intelligence for Neuroscience and Emotional Systems. IWINAC 2024. Lecture Notes in Computer Science, vol 14674. Springer, Cham. https://doi.org/10.1007/978-3-031-61140-7_42
Download citation
DOI: https://doi.org/10.1007/978-3-031-61140-7_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-61139-1
Online ISBN: 978-3-031-61140-7
eBook Packages: Computer ScienceComputer Science (R0)