Search Results (375)

The emotional response of robotics is crucial for promoting the socially intelligent level of human–robot interaction (HRI). The development of machine learning has extensively stimulated research on emotional recognition for robots. Our research focuses on emotional gaits, a type of simple modality that stores a series of joint coordinates and is easy for humanoid robots to execute. However, a limited amount of research investigates emotional HRI systems based on gaits, indicating an existing gap in human emotion gait recognition and robotic emotional gait response. To address this challenge, we propose a Gait-to-Gait Emotional HRI system, emphasizing the development of an innovative emotion classification model. In our system, the humanoid robot NAO can recognize emotions from human gaits through our Trajectories-Aware and Skeleton-Graph-Aware Spatial–Temporal Transformer (TS-ST) and respond with pre-set emotional gaits that reflect the same emotion as the human presented. Our TS-ST outperforms the current state-of-the-art human-gait emotion recognition model applied to robots on the Emotion-Gait dataset. Full article

Background: This paper presents TheraSense, a system developed within the Supporting Mental Health in Young People: Integrated Methodology for cLinical dEcisions and evidence (Smile) and Sensor Enabled Affective Computing for Enhancing Medical Care (SenseCare) projects. TheraSense is designed to enhance teleconsultation services by leveraging deep learning for real-time emotion recognition through facial expressions. It integrates with the Knowledge Management-Ecosystem Portal (SenseCare KM-EP) platform to provide mental health practitioners with valuable emotional insights during remote consultations. Method: We describe the conceptual design of TheraSense, including its use case contexts, architectural structure, and user interface layout. The system’s interoperability is discussed in detail, highlighting its seamless integration within the teleconsultation workflow. The evaluation methods include both quantitative assessments of the video-based emotion recognition system’s performance and qualitative feedback through heuristic evaluation and survey analysis. Results: The performance evaluation shows that TheraSense effectively recognizes emotions in video streams, with positive user feedback on its usability and integration. The system’s real-time emotion detection capabilities provide valuable support for mental health practitioners during remote sessions. Conclusions: TheraSense demonstrates its potential as an innovative tool for enhancing teleconsultation services. By providing real-time emotional insights, it supports better-informed decision-making in mental health care, making it an effective addition to remote telehealth platforms. Full article

Cultural and religious diversity in Spain, driven by recent decades of migratory flows, has not been exempt from generating social tensions and, unfortunately, an increasing stigmatization of migrant children. This article examines how power dynamics and exclusion impact the identity construction of these young people, particularly within the child protection system. Through interviews and focus groups with young people and professionals in Barcelona and Melilla, this study highlights the resistance strategies these young individuals employ to counteract stigmatizing narratives. The findings indicate that, despite inclusion policies, imposed labels reinforce their vulnerability and limit their social and community participation. Although interfaith dialogue is proposed as a tool to mitigate these tensions, professionals working with these children emphasize the lack of institutional support and insufficient training in socio-cultural diversity, which hinders their efforts. The study underscores the importance of developing interfaith competencies that foster mutual respect and recognition, concluding with a critique of the current protection system. It advocates for a comprehensive approach to addressing these young people’s emotional, social, and spiritual needs beyond solely legal and educational aspects. Full article

The metaverse, where users interact through avatars, is evolving to closely mirror the real world, requiring realistic object responses based on users’ emotions. While technologies like eye-tracking and hand-tracking transfer physical movements into virtual spaces, accurate emotion detection remains challenging. This study proposes the “Face and Voice Recognition-based Emotion Analysis System (EAS)” to bridge this gap, assessing emotions through both voice and facial expressions. EAS utilizes a microphone and camera to gauge emotional states, combining these inputs for a comprehensive analysis. It comprises three neural networks: the Facial Emotion Analysis Model (FEAM), which classifies emotions using facial landmarks; the Voice Sentiment Analysis Model (VSAM), which detects vocal emotions even in noisy environments using MCycleGAN; and the Metaverse Emotion Recognition Model (MERM), which integrates FEAM and VSAM outputs to infer overall emotional states. EAS’s three primary modules—Facial Emotion Recognition, Voice Emotion Recognition, and User Emotion Analysis—analyze facial features and vocal tones to detect emotions, providing a holistic emotional assessment for realistic interactions in the metaverse. The system’s performance is validated through dataset testing, and future directions are suggested based on simulation outcomes. Full article

Affective computing aims to develop systems capable of effectively interacting with people through emotion recognition. Neuroscience and psychology have established models that classify universal human emotions, providing a foundational framework for developing emotion recognition systems. Brain activity related to emotional states can be captured through electroencephalography (EEG), enabling the creation of models that classify emotions even in uncontrolled environments. In this study, we propose an emotion recognition model based on EEG signals using deep learning techniques on a proprietary database. To improve the separability of emotions, we explored various data transformation techniques, including Fourier Neural Networks and quantum rotations. The convolutional neural network model, combined with quantum rotations, achieved a 95% accuracy in emotion classification, particularly in distinguishing sad emotions. The integration of these transformations can further enhance overall emotion recognition performance. Full article

Background: Human emotions are subjective psychophysiological processes that play an important role in the daily interactions of human life. Emotions often do not manifest themselves in isolation; people can experience a mixture of them and may not express them in a visible or perceptible way; Methods: This study seeks to uncover EEG patterns linked to emotions, as well as to examine brain activity across emotional states and optimise machine learning techniques for accurate emotion classification. For these purposes, the DEAP dataset was used to comprehensively analyse electroencephalogram (EEG) data and understand how emotional patterns can be observed. Machine learning algorithms, such as SVM, MLP, and RF, were implemented to predict valence and arousal classifications for different combinations of frequency bands and brain regions; Results: The analysis reaffirms the value of EEG as a tool for objective emotion detection, demonstrating its potential in both clinical and technological contexts. By highlighting the benefits of using fewer electrodes, this study emphasises the feasibility of creating more accessible and user-friendly emotion recognition systems; Conclusions: Further improvements in feature extraction and model generalisation are necessary for clinical applications. This study highlights not only the potential of emotion classification to develop biomedical applications, but also to enhance human–machine interaction systems. Full article

Affective computing is an emerging area of education research and has the potential to enhance educational outcomes. Despite the growing number of literature studies, there are still deficiencies and gaps in the domain of affective computing in education. In this study, we systematically review affective computing in the education domain. Methods: We queried four well-known research databases, namely the Web of Science Core Collection, IEEE Xplore, ACM Digital Library, and PubMed, using specific keywords for papers published between January 2010 and July 2023. Various relevant data items are extracted and classified based on a set of 15 extensive research questions. Following the PRISMA 2020 guidelines, a total of 175 studies were selected and reviewed in this work from among 3102 articles screened. The data show an increasing trend in publications within this domain. The most common research purpose involves designing emotion recognition/expression systems. Conventional textual questionnaires remain the most popular channels for affective measurement. Classrooms are identified as the primary research environments; the largest research sample group is university students. Learning domains are mainly associated with science, technology, engineering, and mathematics (STEM) courses. The bibliometric analysis reveals that most publications are affiliated with the USA. The studies are primarily published in journals, with the majority appearing in the Frontiers in Psychology journal. Research gaps, challenges, and potential directions for future research are explored. This review synthesizes current knowledge regarding the application of affective computing in the education sector. This knowledge is useful for future directions to help educational researchers, policymakers, and practitioners deploy affective computing technology to broaden educational practices. Full article

The cognitive emotions of individuals during tasks largely determine the success or failure of tasks in various fields such as the military, medical, industrial fields, etc. Facial video data can carry more emotional information than static images because emotional expression is a temporal process. Video-based Facial Expression Recognition (FER) has received increasing attention from the relevant scholars in recent years. However, due to the high cost of marking and training video samples, feature extraction is inefficient and ineffective, which leads to a low accuracy and poor real-time performance. In this paper, a cognitive emotion recognition method based on video data is proposed, in which 49 emotion description points were initially defined, and the spatial–temporal features of cognitive emotions were extracted from the video data through a feature extraction method that combines geodesic distances and sample entropy. Then, an active learning algorithm based on complexity and uncertainty was proposed to automatically select the most valuable samples, thereby reducing the cost of sample labeling and model training. Finally, the effectiveness, superiority, and real-time performance of the proposed method were verified utilizing the MMI Facial Expression Database and some real-time-collected data. Through comparisons and testing, the proposed method showed satisfactory real-time performance and a higher accuracy, which can effectively support the development of a real-time monitoring system for cognitive emotions. Full article

Emotions play a significant role in shaping psychological activities, behaviour, and interpersonal communication. Reflecting this importance, automated emotion classification has become a vital research area in artificial intelligence. Electroencephalogram (EEG)-based emotion recognition is particularly promising due to its high temporal resolution and resistance to manipulation. This study introduces an advanced fuzzy inference algorithm for EEG data-driven emotion recognition, effectively addressing the ambiguity of emotional states. By combining adaptive fuzzy rule generation, feature evaluation, and weighted fuzzy rule interpolation, the proposed approach achieves accurate emotion classification while handling incomplete knowledge. Experimental results demonstrate that the integrated fuzzy system outperforms state-of-the-art techniques, offering improved recognition accuracy and robustness under uncertainty. Full article

Since the early 21st century, wildlands have witnessed an effusion of wildfires, with climate and social changes resulting in unanticipated wildfire activity and impact. For forest fires to be prevented and suppressed effectively, forest firefighting forces have adopted a specific administrative system for organizing and managing the fighting force. Under the administrative system, a debate on desired “leadership and management qualities” arises, and hence, this study sought to identify the leadership and management traits that should distinguish individuals in the forest fire incident command system (FFICS) applied by the Department of Forests (Cyprus). The research subject was addressed using mixed method research, employing quantitative and qualitative data. Both datasets were used to distinguish the purposes of the applied triangulation, enabling the examination of differentiation between the trends/positions recorded in terms of the object of study. These findings point to ideal forms of transformational leadership and neoclassical management. The outcomes suggest that at the individual level, the leaders of each of the operating structures should develop leadership qualities related to emotional intelligence, empathy, judgment, critical thinking, and especially self-awareness of strengths and weaknesses. At the stage of pre-suppression, a democratic leadership style (or guiding style) is supported, while during the operational progress stage of the FFICS, a “hybrid” leadership style is suggested, borrowing elements from the democratic and authoritarian (or managerial) leadership styles. The administrative skills of FFICS leaders should include the moral and psychological rewards of subordinates, job satisfaction and recognition, and two-way communication. The current study illustrates the need for divergent leadership and management traits and styles among the different hierarchical structures of the FFICS. Full article

This paper presents a study on enhancing human–robot interaction (HRI) through multimodal emotional recognition within social robotics. Using the humanoid robot Pepper as a testbed, we integrate visual, auditory, and textual analysis to improve emotion recognition accuracy and contextual understanding. The proposed framework combines pretrained neural networks with fine-tuning techniques tailored to specific users, demonstrating that high accuracy in emotion recognition can be achieved by adapting the models to the individual emotional expressions of each user. This approach addresses the inherent variability in emotional expression across individuals, making it feasible to deploy personalized emotion recognition systems. Our experiments validate the effectiveness of this methodology, achieving high precision in multimodal emotion recognition through fine-tuning, while maintaining adaptability in real-world scenarios. These enhancements significantly improve Pepper’s interactive and empathetic capabilities, allowing it to engage more naturally with users in assistive, educational, and healthcare settings. This study not only advances the field of HRI but also provides a reproducible framework for integrating multimodal emotion recognition into commercial humanoid robots, bridging the gap between research prototypes and practical applications. Full article

The field of emotion recognition from physiological signals is a growing area of research with significant implications for both mental health monitoring and human–computer interaction. This study introduces a novel approach to detecting emotional states based on fractal analysis of electrodermal activity (EDA) signals. We employed detrended fluctuation analysis (DFA), Hurst exponent estimation, and wavelet entropy calculation to extract fractal features from EDA signals obtained from the CASE dataset, which contains physiological recordings and continuous emotion annotations from 30 participants. The analysis revealed significant differences in fractal features across five emotional states (neutral, amused, bored, relaxed, and scared), particularly those derived from wavelet entropy. A cross-correlation analysis showed robust correlations between fractal features and both the arousal and valence dimensions of emotion, challenging the conventional view of EDA as a predominantly arousal-indicating measure. The application of machine learning for emotion classification using fractal features achieved a leave-one-subject-out accuracy of 84.3% and an F1 score of 0.802, surpassing the performance of previous methods on the same dataset. This study demonstrates the potential of fractal analysis in capturing the intricate, multi-scale dynamics of EDA signals for emotion recognition, opening new avenues for advancing emotion-aware systems and affective computing applications. Full article

Emotion recognition through artificial intelligence and smart sensing of physical and physiological signals (affective computing) is achieving very interesting results in terms of accuracy, inference times, and user-independent models. In this sense, there are applications related to the safety and well-being of people (sexual assaults, gender-based violence, children and elderly abuse, mental health, etc.) that require even more improvements. Emotion detection should be done with fast, discrete, and non-luxurious systems working in real time and real life (wearable devices, wireless communications, battery-powered). Furthermore, emotional reactions to violence are not equal in all people. Then, large general models cannot be applied to a multi-user system for people protection, and health and social workers and law enforcement agents would welcome customized and lightweight AI models. These semi-personalized models will be applicable to clusters of subjects sharing similarities in their emotional reactions to external stimuli. This customization requires several steps: creating clusters of subjects with similar behaviors, creating AI models for every cluster, continually updating these models with new data, and enrolling new subjects in clusters when required. An initial approach for clustering labeled data compiled (physiological data, together with emotional labels) is presented in this work, as well as the method to ensure the enrollment of new users with unlabeled data once the AI models are generated. The idea is that this complete methodology can be exportable to any other expert systems where unlabeled data are added during in-field operation and different profiles exist in terms of data. Experimental results demonstrate an improvement of 5% in accuracy and 4% in F1 score with respect to our baseline general model, along with a 32% to 58% reduction in variability, respectively. Full article

Visual biosignals can be used to analyze human behavioral activities and serve as a primary resource for Facial Expression Recognition (FER). FER computational systems face significant challenges, arising from both spatial and temporal effects. Spatial challenges include deformations or occlusions of facial geometry, while temporal challenges involve discontinuities in motion observation due to high variability in poses and dynamic conditions such as rotation and translation. To enhance the analytical precision and validation reliability of FER systems, several datasets have been proposed. However, most of these datasets focus primarily on spatial characteristics, rely on static images, or consist of short videos captured in highly controlled environments. These constraints significantly reduce the applicability of such systems in real-world scenarios. This paper proposes the Facial Biosignals Time–Series Dataset (FBioT), a novel dataset providing temporal descriptors and features extracted from common videos recorded in uncontrolled environments. To automate dataset construction, we propose Visual–Temporal Facial Expression Recognition (VT-FER), a method that stabilizes temporal effects using normalized measurements based on the principles of the Facial Action Coding System (FACS) and generates signature patterns of expression movements for correlation with real-world temporal events. To demonstrate feasibility, we applied the method to create a pilot version of the FBioT dataset. This pilot resulted in approximately 10,000 s of public videos captured under real-world facial motion conditions, from which we extracted 22 direct and virtual metrics representing facial muscle deformations. During this process, we preliminarily labeled and qualified 3046 temporal events representing two emotion classes. As a proof of concept, these emotion classes were used as input for training neural networks, with results summarized in this paper and available in an open-source online repository. Full article

Speech emotion recognition (SER) is important in facilitating natural human–computer interactions. In speech sequence modeling, a vital challenge is to learn context-aware sentence expression and temporal dynamics of paralinguistic features to achieve unambiguous emotional semantic understanding. In previous studies, the SER method based on the single-scale cascade feature extraction module could not effectively preserve the temporal structure of speech signals in the deep layer, downgrading the sequence modeling performance. To address these challenges, this paper proposes a novel multi-scale feature pyramid network. The enhanced multi-scale convolutional neural networks (MSCNNs) significantly improve the ability to extract multi-granular emotional features. Experimental results on the IEMOCAP corpus demonstrate the effectiveness of the proposed approach, achieving a weighted accuracy (WA) of 71.79% and an unweighted accuracy (UA) of 73.39%. Furthermore, on the RAVDESS dataset, the model achieves an unweighted accuracy (UA) of 86.5%. These results validate the system’s performance and highlight its competitive advantage. Full article