Search Results (415)

This work proposes an approach that uses a feature space by combining the representation obtained in the unsupervised learning process and manually selected features defining the prosody of the utterances. In the experiments, we used two time-frequency representations (Mel and CQT spectrograms) and EmoDB and RAVDESS databases. As the results show, the proposed system improved the classification accuracy of both representations: 1.29% for CQT and 3.75% for Mel spectrogram compared to the typical CNN architecture for the EmoDB dataset and 3.02% for CQT and 0.63% for Mel spectrogram in the case of RAVDESS. Additionally, the results present a significant increase of around 14% in classification performance in the case of happiness and disgust emotions using Mel spectrograms and around 20% in happiness and disgust emotions for CQT in the case of best models trained on EmoDB. On the other hand, in the case of models that achieved the highest result for the RAVDESS database, the most significant improvement was observed in the classification of a neutral state, around 16%, using the Mel spectrogram. For CQT representation, the most significant improvement occurred for fear and surprise, around 9%. Additionally, the average results for all prepared models showed the positive impact of the method used on the quality of classification of most emotional states. For the EmoDB database, the highest average improvement was observed for happiness—14.6%. For other emotions, it ranged from 1.2% to 8.7%. The only exception was the emotion of sadness, for which the classification quality was average decreased by 1% when using the Mel spectrogram. In turn, for the RAVDESS database, the most significant improvement also occurred for happiness—7.5%, while for other emotions ranged from 0.2% to 7.1%, except disgust and calm, the classification of which deteriorated for the Mel spectrogram and the CQT representation, respectively. Full article

Misophonia is a condition characterized by intense, involuntary distress or anger in response to specific sounds, often leading to irritation or aggression. While the condition is recognized for its emotional and behavioral impacts, little is known about its physiological and perceptual effects. The current study aimed to explore the physiological correlates and perceptual consequences of misophonia through a combination of electrophysiological, perceptual, and cognitive assessments. Seventeen individuals with misophonia and sixteen control participants without the condition were compared. Participants completed a comprehensive battery of tests, including (a) cortical event-related potentials (ERPs) to assess neural responses to standard and deviant auditory stimuli, (b) the spatial release from the speech-on-speech masking (SRM) paradigm to evaluate speech segregation in background noise, and (c) the flanker task to measure selective attention and cognitive control. The results revealed that individuals with misophonia exhibited significantly smaller mean peak amplitudes of the N1 and N2 components in response to oddball tones compared to controls. This suggests a potential underlying neurobiological deficit in misophonia patients, as these components are associated with early auditory processing. However, no significant differences between each group were observed in the P1 and P2 components regarding oddball tones or in any ERP components in response to standard tones. Despite these altered neural responses, the misophonia group did not show differences in hearing thresholds, speech perception abilities, or cognitive function compared to the controls. These findings suggest that while misophonia may involve distinct neurophysiological changes, particularly in early auditory processing, it does not necessarily lead to perceptual deficits in speech perception or cognitive function. Full article

Background/Objectives: Stuttering is a speech disorder involving fluency disruptions like repetitions, prolongations, and blockages, often leading to emotional distress and social withdrawal. Here, we present Augmented Multisensory Feedback Stimulation (AMFS), a novel personalized intervention to improve speech fluency in people who stutter (PWS). AMFS includes a five-day intensive phase aiming at acquiring new skills, plus a reinforcement phase designed to facilitate the transfer of these skills across different contexts and their automatization into effortless behaviors. The concept of our intervention derives from the prediction of the neurocomputational model Directions into Velocities of Articulators (DIVA). The treatment applies dynamic multisensory stimulation to disrupt PWS’ maladaptive over-reliance on sensory feedback mechanisms, promoting the emergence of participants’ natural voices. Methods: Forty-six PWS and a control group, including twenty-four non-stuttering individuals, participated in this study. Stuttering severity and physiological measures, such as heart rate and electromyographic activity, were recorded before and after the intensive phase and during the reinforcement stage in the PWS but only once in the controls. Results: The results showed a significant reduction in stuttering severity at the end of the intensive phase, which was maintained during the reinforcement training. Crucially, worse performance was found in PWS than in the controls at baseline but not after the intervention. In the PWS, physiological signals showed a reduction in activity during the training phases compared to baseline. Conclusions: Our findings show that AMFS provides a promising approach to enhancing speech fluency. Future studies should clarify the mechanisms underlying such intervention and assess whether effects persist after the treatment conclusion. Full article

This research investigates the utilization of entertainment approaches, such as serious games and gamification technologies, to address various challenges and implement targeted tasks. Specifically, it details the design and development of an innovative gamified application named “J-Plus”, aimed at both professionals and non-professionals in journalism. This application facilitates the enjoyable, efficient, and high-quality collection of emotionally tagged speech samples, enhancing the performance and robustness of speech emotion recognition (SER) systems. Additionally, these approaches offer significant educational benefits, providing users with knowledge about emotional speech and artificial intelligence (AI) mechanisms while promoting digital skills. This project was evaluated by 48 participants, with 44 engaging in quantitative assessments and 4 forming an expert group for qualitative methodologies. This evaluation validated the research questions and hypotheses, demonstrating the application’s diverse benefits. Key findings indicate that gamified features can effectively support learning and attract users, with approximately 70% of participants agreeing that serious games and gamification could enhance their motivation to practice and improve their emotional speech. Additionally, 50% of participants identified social interaction features, such as collaboration, as most beneficial for fostering motivation and commitment. The integration of these elements supports reliable and extensive data collection and the advancement of AI algorithms while concurrently developing various skills, such as emotional speech articulation and digital literacy. This paper advocates for the creation of collaborative environments and digital communities through crowdsourcing, balancing technological innovation in the SER sector. Full article

Introduction: In patients refractory to vestibular rehabilitation in the management of bilateral vestibulopathy, the cochleo-vestibular implant has emerged as a viable alternative to enhance both audiovestibular function and quality of life. The main objective of this study is to pioneer the use of PET to assess cortical modifications in patients with cochleo-vestibular implants, aiming to evaluate the safety and functional improvements in individuals with bilateral vestibulopathy and severe to profound hearing loss. Methods: A phase I pilot clinical trial was conducted with participants who received a BIONIC-VEST CI24RE cochleo-vestibular implant, with pre- and post-implantation assessments conducted for twelve months. Audiovestibular testing and two PET studies with 18F-FDG under baseline conditions and with active stimulus to observe cortical-level differences were performed. Results: Five patients were included in the study, all of them treated with a cochleo-vestibular implant, none of whom presented postoperative adverse effects. Audiologically, the mean post-implant gain was 56.63 ± 14.53 dB and 50.40 ± 35.54% in terms of speech intelligibility. From a vestibular perspective, the most remarkable findings were observed at the graviceptive pathway level, where a mean posturographic improvement was observed, with a sensory organization test score of 24.20 ± 13.74 and a subjective visual vertical of 1.57° ± 0.79°, achieving, in most cases, results within the normal range (<2.3°) by the end of the follow-up. PET images confirmed that with the electrical stimulus active (implant ON), there was a supratentorial activation pattern, particularly in areas related to somatosensory integration, emotional regulation, and autonomic control. Conclusions: The BIONIC-VEST implant significantly improved the vestibular system, particularly the graviceptive pathway, enhancing balance and SVV and reducing fall risk. PET revealed distinct uptake patterns in baseline and activated conditions, highlighting a cortical-level response with the use of the cochleo-vestibular implant. Full article

In recent years, substantial research has focused on emotion recognition using multi-stream speech representations. In existing multi-stream speech emotion recognition (SER) approaches, effectively extracting and fusing speech features is crucial. To overcome the bottleneck in SER caused by the fusion of inter-feature information, including challenges like modeling complex feature relations and the inefficiency of fusion methods, this paper proposes an SER framework based on multi-task learning, named AFEA-Net. The framework consists of a speech emotion alignment learning (SEAL), an acoustic feature excitation-and-aggregation mechanism (AFEA), and a continuity learning. First, SEAL aligns sentiment information between WavLM and Fbank features. Then, we design an acoustic feature excitation-and-aggregation mechanism to adaptively calibrate and merge the two features. Furthermore, we introduce a continuity learning strategy to explore the distinctiveness and complementarity of dual-stream features from intra- and inter-speech. Experimental results on the publicly available IEMOCAP and RAVDESS sentiment datasets show that our proposed approach outperforms state-of-the-art SER approaches. Specifically, we achieve 75.1% WA, 75.3% UAR, 76% precision, and 75.4% F1-score on IEMOCAP, and 80.3%, 80.6%, 80.8%, and 80.4% on RAVDESS, respectively. Full article

Background/Objectives: Loneliness among older adults is a prevalent issue, significantly impacting their quality of life and increasing the risk of physical and mental health complications. The application of artificial intelligence (AI) technologies in behavioral interventions offers a promising avenue to overcome challenges in designing and implementing interventions to reduce loneliness by enabling personalized and scalable solutions. This study systematically reviews the AI-enabled interventions in addressing loneliness among older adults, focusing on the effectiveness and underlying technologies used. Methods: A systematic search was conducted across eight electronic databases, including PubMed and Web of Science, for studies published up to 31 January 2024. Inclusion criteria were experimental studies involving AI applications to mitigate loneliness among adults aged 55 and older. Data on participant demographics, intervention characteristics, AI methodologies, and effectiveness outcomes were extracted and synthesized. Results: Nine studies were included, comprising six randomized controlled trials and three pre–post designs. The most frequently implemented AI technologies included speech recognition (n = 6) and emotion recognition and simulation (n = 5). Intervention types varied, with six studies employing social robots, two utilizing personal voice assistants, and one using a digital human facilitator. Six studies reported significant reductions in loneliness, particularly those utilizing social robots, which demonstrated emotional engagement and personalized interactions. Three studies reported non-significant effects, often due to shorter intervention durations or limited interaction frequencies. Conclusions: AI-driven interventions show promise in reducing loneliness among older adults. Future research should focus on long-term, culturally competent solutions that integrate quantitative and qualitative findings to optimize intervention design and scalability. Full article

This paper investigates the effectiveness of various data augmentation techniques for enhancing Arabic speech emotion recognition (SER) using convolutional neural networks (CNNs). Utilizing the Saudi Dialect and BAVED datasets, we address the challenges of limited and imbalanced data commonly found in Arabic SER. To improve model performance, we apply augmentation techniques such as noise addition, time shifting, increasing volume, and reducing volume. Additionally, we examine the optimal number of augmentations required to achieve the best results. Our experiments reveal that these augmentations significantly enhance the CNN’s ability to recognize emotions, with certain techniques proving more effective than others. Furthermore, the number of augmentations plays a critical role in balancing model accuracy. The Saudi Dialect dataset achieved its best results with two augmentations (increasing volume and decreasing volume), reaching an accuracy of 96.81%. Similarly, the BAVED dataset demonstrated optimal performance with a combination of three augmentations (noise addition, increasing volume, and reducing volume), achieving an accuracy of 92.60%. These findings indicate that carefully selected augmentation strategies can greatly improve the performance of CNN-based SER systems, particularly in the context of Arabic speech. This research underscores the importance of tailored augmentation techniques to enhance SER performance and sets a foundation for future advancements in this field. Full article

Over the past five years, the proliferation of virtual reality platforms and the advancement of metahuman technologies have underscored the importance of natural interaction and emotional expression. As a result, there has been significant research activity focused on developing emotion recognition techniques based on speech data. Despite significant progress in emotion recognition research for the Korean language, a shortage of speech databases applicable to such research has been regarded as the most critical problem in this field, leading to overfitting issues in several models developed by previous studies. To address the issue of overfitting caused by limited data availability in the field of Korean speech emotion recognition (SER), this study focuses on integrating the data augmentation–saturation (DA-S) technique into a traditional ResNet model to enhance SER performance. The DA-S technique enhances data augmentation by adjusting the saturation of an image. We used 11,192 utterance numbers provided by AI-HUB, which were converted into images to extract features such as pitch and intensity of speech. The DA-S technique was then applied to this dataset, using weights of 0 and 2, to augment the utterance numbers to 33,576. This augmented dataset was utilized to classify four emotion categories: happiness, sadness, anger, and neutrality. The results of this study showed that the proposed model using the DA-S technique overcame overfitting issues. Furthermore, its performance for SER increased by 34.19% compared to that of existing ResNet models not using the DA-S technique. This demonstrates that the DA-S technique effectively enhances model performance with limited data and may be applicable to specific areas such as stress monitoring and mental health support. Full article

Black children in the U.S. learn from scaffolded parental teachings to help manage racial discrimination. Middle childhood is an understudied developmental period for this research. This paper builds upon research on culturally informed practices Black caregivers use to rear their young with a healthy identity and socio-emotional skills to navigate racism Guided by a phenomenological qualitative approach, we conducted focus groups with 39 Black children (Mean_age = 7.67, 54% girls, 46% boys). Children reported that their parents imparted a sense of positive identity in terms of their cultural heritage, skin, and hair—areas in which they experienced frequent bullying. A uniqueness of our study is that Black children also reported learning emotion-centered coping strategies that focus on their inner strengths and private speech. They adopted a range of adaptive coping mechanisms such as kindness, ignoring perpetrators, centering their positive identity, identity framing, and fighting back. Through children’s voices, we build upon previous research integrating racial–ethnic socialization (RES) with socio-emotional competencies in response to discrimination. We underscore the importance of exploring racial–ethnic identity development and socialization in childhood, a developmental period in which these processes are understudied. Full article

This paper investigates the integration of affective computing techniques using biophysical data to advance emotionally aware machines and enhance child–robot interaction (CRI). By leveraging interdisciplinary insights from neuroscience, psychology, and artificial intelligence, the study focuses on creating adaptive, emotion-aware systems capable of dynamically recognizing and responding to human emotional states. Through a real-world CRI pilot study involving the NAO robot, this research demonstrates how facial expression analysis and speech emotion recognition can be employed to detect and address negative emotions in real time, fostering positive emotional engagement. The emotion recognition system combines handcrafted and deep learning features for facial expressions, achieving an 85% classification accuracy during real-time CRI, while speech emotions are analyzed using acoustic features processed through machine learning models with an 83% accuracy rate. Offline evaluation of the combined emotion dataset using a Dynamic Bayesian Mixture Model (DBMM) achieved a 92% accuracy for facial expressions, and the multilingual speech dataset yielded 98% accuracy for speech emotions using the DBMM ensemble. Observations from psychological and technological aspects, coupled with statistical analysis, reveal the robot’s ability to transition negative emotions into neutral or positive states in most cases, contributing to emotional regulation in children. This work underscores the potential of emotion-aware robots to support therapeutic and educational interventions, particularly for pediatric populations, while setting a foundation for developing personalized and empathetic human–machine interactions. These findings demonstrate the transformative role of affective computing in bridging the gap between technological functionality and emotional intelligence across diverse domains. Full article

The emotional expressions in live streaming e-commerce possess a strong contagious effect, enabling viewers to easily resonate with the specific emotions conveyed by the streamers and consciously build an empathy transmission chain. This study constructs a regression model based on the emotional contagion theory and explores the impact of empathy between streamers and viewers on sales performance. Using data from 30 live streams, totaling 22,707 min, from one of China’s most popular live streaming rooms, “East Buy”, between February and April 2024, we demonstrate the significant positive impact of empathy between streamers and viewers on sales. Additionally, product type positively moderates this relationship. The unexpected thing is that live streaming time does not significantly affect the relationship between empathy and sales. This study employs text sentiment analysis methods to extract emotional features from the streamers’ speech and real-time comments from viewers. Our research extends the application of emotional contagion theory to the context of live-streaming e-commerce, enriches the literature on emotional interaction in service marketing, and provides practical insights for live-streaming platforms and streamers. Streamers can optimize marketing strategies and achieve sales goals by creating a more engaging and empathetic live-streaming experience. Full article

Touch, particularly affective touch mediated by C-tactile fibers, plays a key role in emotional regulation and therapeutic interventions. However, tactile stimulation is underutilized in sensory stimulation (SS) protocols for brain injury patients, despite its potential to enhance consciousness and promote recovery through neural and autonomic regulation. Tools like the Neurowave enable advanced multisensory stimulation, including audio-visual and emotional inputs, but lack tactile components. Integrating gentle touch stimulation with such systems could further enhance neuroplasticity, improve heart rate regulation, and support recovery in patients with disorders of consciousness. In this study, twenty patients affected by minimally conscious state (MCS) were divided into two groups: an experimental group (EG n.10) and a control group (CG n.10). Both groups underwent standard neurorehabilitation, including conventional physiotherapy and speech therapy. The key difference was in the type of sensory stimulation. The EG received advanced sensory stimulation with the Neurowave system (which provides audio-visual and emotional sensory stimulation) in addition to gentle touch stimulation. The CG received conventional sensory stimulation without the Neurowave and neutral gentle touch stimulation. Each patient was evaluated by a multidisciplinary rehabilitation team, using clinical scales such as coma recovery scale—revised (CSR-R), level of cognitive functioning (LCF), before (T0) and after (T1) treatment. Additionally, heart rate (HR) and neurophysiological outcomes (P300) were also recorded for both groups (EG and CG). The MANOVA model revealed a significant interaction effect between group and phase on P300 latency (F (1, 18) = 10.23, p < 0.001, η² = 0.09), indicating that the intervention involving gentle touch stimulation significantly influenced the P300 latency in the EG. The findings of this study contribute to our understanding of the therapeutic potential of emotional multisensory stimulation, which also includes gentle touch stimulation, in MCS rehabilitation. By demonstrating significant effects on both neurophysiological and functional measures, our results support the integration of tactile interventions into comprehensive neurorehabilitation programs. Full article

Artificial intelligence (AI) has emerged as a transformative force in psychiatry, improving diagnostic precision, treatment personalization, and early intervention through advanced data analysis techniques. This review explores recent advancements in AI applications within psychiatry, focusing on EEG and ECG data analysis, speech analysis, natural language processing (NLP), blood biomarker integration, and social media data utilization. EEG-based models have significantly enhanced the detection of disorders such as depression and schizophrenia through spectral and connectivity analyses. ECG-based approaches have provided insights into emotional regulation and stress-related conditions using heart rate variability. Speech analysis frameworks, leveraging large language models (LLMs), have improved the detection of cognitive impairments and psychiatric symptoms through nuanced linguistic feature extraction. Meanwhile, blood biomarker analyses have deepened our understanding of the molecular underpinnings of mental health disorders, and social media analytics have demonstrated the potential for real-time mental health surveillance. Despite these advancements, challenges such as data heterogeneity, interpretability, and ethical considerations remain barriers to widespread clinical adoption. Future research must prioritize the development of explainable AI models, regulatory compliance, and the integration of diverse datasets to maximize the impact of AI in psychiatric care. Full article

Background/Objectives: Voice artificial intelligence (AI) technology is becoming increasingly common. Recent work indicates that middle-aged to older adults are less able to identify modern AI speech compared to younger adults, but the underlying causes are unclear. Methods: The current study with younger and middle-aged to older adults investigated factors that could explain the age-related reduction in AI speech identification. Experiment 1 investigated whether high-frequency information in speech—to which middle-aged to older adults often have less access due sensitivity loss at high frequencies—contributes to age-group differences. Experiment 2 investigated whether an age-related reduction in the ability to process prosodic information in speech predicts the reduction in AI speech identification. Results: Results for Experiment 1 show that middle-aged to older adults are less able to identify AI speech for both full-bandwidth speech and speech for which information above 4 kHz is removed, making the contribution of high-frequency hearing loss unlikely. Experiment 2 shows that the ability to identify AI speech is greater in individuals who also show a greater ability to identify emotions from prosodic speech information, after accounting for hearing function and self-rated experience with voice-AI systems. Conclusions: The current results suggest that the ability to identify AI speech is related to the accurate processing of prosodic information. Full article