AI

Background: Integrating nonlinear behavior into the architecture of artificial neural networks is regarded as essential requirement to constitute their effectual learning capacity for solving complex tasks. This claim seems to be true for moderate-sized networks, i.e., with a lower double-digit number of layers. However, going deeper with neural networks regularly turns into destructive tendencies of gradual performance degeneration during training. To circumvent this degradation problem, the prominent neural architectures Residual Network and Highway Network establish skip connections with additive identity mappings between layers. Methods: In this work, we unify the mechanics of both architectures into Capsule Networks (CapsNet)s by showing their inherent ability to learn skip connections. As a necessary precondition, we introduce the concept of Adaptive Nonlinearity Gates (ANG)s which dynamically steer and limit the usage of nonlinear processing. We propose practical methods for the realization of ANGs including biased batch normalization, the Doubly-Parametric ReLU (D-PReLU) activation function, and Gated Routing (GR) dedicated to extremely deep CapsNets. Results: Our comprehensive empirical study using MNIST substantiates the effectiveness of our developed methods and delivers valuable insights for the training of very deep nets of any kind. The final experiments on Fashion-MNIST and SVHN demonstrate the potential of pure capsule-driven networks with GR. Full article

Among gynecological pathologies, cervical cancer has always represented a health problem with great social impact. The giant strides made as a result of both the screening programs perfected and implemented over the years and the use of new and accurate technological equipment have in fact significantly improved our clinical approach in the management and personalized diagnosis of precancerous lesions of the cervix. In this context, the advent of artificial intelligence and digital algorithms could represent new directions available to gynecologists and pathologists for the following: (i) the standardization of screening procedures, (ii) the identification of increasingly early lesions, and (iii) heightening the diagnostic accuracy of targeted biopsies and prognostic analysis of cervical cancer. The purpose of our review was to evaluate to what extent artificial intelligence can be integrated into current protocols, to identify the strengths and/or weaknesses of this method, and, above all, determine what we should expect in the future to develop increasingly safer solutions, as well as increasingly targeted and personalized screening programs for these patients. Furthermore, in an innovative way, and through a multidisciplinary vision (gynecologists, pathologists, and computer scientists), with this manuscript, we highlight a key role that AI could have in the management of HPV-positive patients. In our vision, AI will move from being a simple diagnostic device to being used as a tool for performing risk analyses of HPV-related disease progression. This is thanks to the ability of new software not only to analyze clinical and histopathological images but also to evaluate and integrate clinical elements such as vaccines, the composition of the microbiota, and the immune status of patients. In fact, the single-factor evaluation of high-risk HPV strains represents a limitation that must be overcome. Therefore, AI, through multifactorial analysis, will be able to generate a risk score that will better stratify patients and will support clinicians in choosing highly personalized treatments overall. Our study remains an innovative proposal and idea, as the literature to date presents a limitation in that this topic is considered niche, but we believe that the union of common efforts can overcome this limitation. Full article

Background: In the last decade, numerous methods have been proposed to define and detect outliers, particularly in complex environments like networks, where anomalies significantly deviate from normal patterns. Although defining a clear standard is challenging, anomaly detection systems have become essential for network administrators to efficiently identify and resolve irregularities. Methods: This study develops and evaluates a machine learning-based system for network anomaly detection, focusing on point anomalies within network traffic. It employs both unsupervised and supervised learning techniques, including change point detection, clustering, and classification models, to identify anomalies. SHAP values are utilized to enhance model interpretability. Results: Unsupervised models effectively captured temporal patterns, while supervised models, particularly Random Forest (94.3%), demonstrated high accuracy in classifying anomalies, closely approximating the actual anomaly rate. Conclusions: Experimental results indicate that the system can accurately predict network anomalies in advance. Congestion and packet loss were identified as key factors in anomaly detection. This study demonstrates the potential for real-world deployment of the anomaly detection system to validate its scalability. Full article

Soybean is a critical agricultural commodity, serving as a vital source of protein and vegetable oil, and contributing significantly to the economies of producing nations. However, soybean yields are frequently compromised by disease and pest infestations, which, if not identified early, can lead to substantial production losses. To address this challenge, we propose AgriNAS, a method that integrates a Neural Architecture Search (NAS) framework with an adaptive convolutional architecture specifically designed for plant pathology. AgriNAS employs a novel data augmentation strategy and a Spatial–Time Augmentation (STA) method, and it utilizes a multi-stage convolutional network that dynamically adapts to the complexity of the input data. The proposed AgriNAS leverages powerful GPU resources to handle the intensive computational tasks involved in NAS and model training. The framework incorporates a bi-level optimization strategy and entropy-based regularization to enhance model robustness and prevent overfitting. AgriNAS achieves classification accuracies superior to VGG-19 and a transfer learning method using convolutional neural networks. Full article

(1) Background: Existing Vehicle travel time prediction applications face challenges in modeling complex road network and handling irregular spatiotemporal traffic state propagation. (2) Methods: To address these issues, we propose a Graph Attention-based Multi-Spatiotemporal Features for Travel Time Prediction (GMTP) model, which integrates an enhanced graph attention network (GATv2) and Bidirectional Encoder Representations from Transformers (BERT) to analyze dynamic correlations across spatial and temporal dimensions. The pre-training process consists of two blocks: the Road Segment Interaction Pattern to Enhance GATv2, which generates road segment representation vectors, and a traffic congestion-aware trajectory encoder by incorporating a shared attention mechanism for high computational efficiency. Additionally, two self-supervised tasks are designed for improved model accuracy and robustness. (3) Results: The fine-tuned model had comparatively optimal performance metrics with significant reductions in Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), and Root Mean Squared Error (RMSE). (4) Conclusions: Ultimately, the integration of this model into travel time prediction, based on two large-scale real-world trajectory datasets, demonstrates enhanced performance and computational efficiency. Full article

The estimation of energy expenditure (EE) is often an integral part of algorithms for wearable electronics. In field practice, procedures based on an indirect estimation of EE from the heart rate (using empirically or statistically based relationships) work correctly only in a narrow range of physical loads, yet they are current considered state of the art. This pilot study aims to experimentally assess novel method using a wide range of input sensors and parameters (heart rate, RR intervals, and 3D motion activity in several places on the body) and neural network (NN) algorithms. Our proposed method consists of training an NN on a specific subject, with a specific set and placement of sensors during the so-called training run, using the golden standard method of indirect calorimetry as a reference. Then, the subject’s EE can be estimated using trained NN. The results of the experiments (carried out on a total of 12 subjects during various physical activities) show a statistically significant improvement in EE estimation with the new prospective method, and it outperforms the state-of-the-art method based on the heart rate and regression model. Full article

Background: Three-dimensional skeleton-based human motion prediction is an essential and challenging task for human–machine interactions, aiming to forecast future poses given a history of previous motions. However, existing methods often fail to effectively model dynamic changes and optimize spatial–temporal features. Methods: In this paper, we introduce Dynamic Differencing-based Hybrid Networks (2DHnet), which addresses these issues with two innovations: the Dynamic Differential Dependencies Extractor (2D-DE) for capturing dynamic features like velocity and acceleration, and the Attention-based Spatial–Temporal Dependencies Extractor (AST-DE) for enhancing spatial–temporal correlations. The 2DHnet combines these into a dual-branch network, offering a comprehensive motion representation. Results: Experiments on the Human3.6M and 3DPW datasets show that 2DHnet significantly outperforms existing methods, with average improvements of 4.7% and 26.6% in MPJPE, respectively. Full article

Machine learning (ML) is increasingly used to support or automate decision processes in critical decision systems such as self driving cars or systems for medical diagnosis. These systems require decisions in which human lives are at stake and the decisions should therefore be well founded and very reliable. This need for reliability contrasts with the black-box nature of many ML models, making it difficult to ensure that they always behave as intended. In face of the high stakes involved, the resulting uncertainty is a significant challenge. Explainable artificial intelligence (XAI) addresses the issue by making black-box models more interpretable, often to increase user trust. However, many current XAI applications focus more on transparency and usability than on enhancing safety of ML applications. In this work, we therefore conduct a systematic literature review to examine how XAI can be leveraged to increase safety of ML applications in critical decision systems. We strive to find out for what purposes XAI is currently used in critical decision systems, what are the most common XAI techniques in critical decision systems and how XAI can be harnessed to increase safety of ML applications in critical decision systems. Using the SPAR-4-SLR protocol, we are able to answer these questions and provide a foundational resource for researchers and practitioners seeking to mitigate risks of ML applications. Essentially, we identify promising approaches of XAI which go beyond increasing trust to actively ensure correctness of decisions. Our findings propose a three-layered framework to enhance safety of ML in critical decision systems by means of XAI. The approach consists of Reliability, Validation and Verification. Furthermore, we point out gaps in research and propose future directions of XAI research for enhancing safety of ML applications in critical decision systems. Full article

In the digital era, social media platforms have seen a substantial increase in the volume of online comments. While these platforms provide users with a space to express their opinions, they also serve as fertile ground for the proliferation of hate speech. Hate comments can be categorized into various types, including discrimination, violence, racism, and sexism, all of which can negatively impact mental health. Among these, sexism poses a significant challenge due to its various forms and the difficulty in defining it, making detection complex. Nevertheless, detecting and preventing sexism on social networks remains a critical issue. Recent studies have leveraged language models such as transformers, known for their ability to capture the semantic nuances of textual data. In this study, we explore different transformer models, including multiple versions of RoBERTa (A Robustly Optimized BERT Pretraining Approach), to detect sexism. We hypothesize that combining a sentiment-focused language model with models specialized in sexism detection can improve overall performance. To test this hypothesis, we developed two approaches. The first involved using classical transformers trained on our dataset, while the second combined embeddings generated by transformers with a Long Short-Term Memory (LSTM) model for classification. The probabilistic outputs of each approach were aggregated through various voting strategies to enhance detection accuracy. The LSTM with embeddings approach improved the F1-score by 0.2% compared to the classical transformer approach. Furthermore, the combination of both approaches confirms our hypothesis, achieving a 1.6% improvement in the F1-score in each case. We determined that an F1 score of over 0.84 effectively measures sexism. Additionally, we constructed our own dataset to train and evaluate the models. Full article

Cryptocurrency is recognized as a leading digital currency by its peer-to-peer transfer capabilities and secure features. Accurately forecasting cryptocurrency price trends holds substantial significance for investors and traders, as they inform critical decisions regarding the acquisition, divestment, or retention of cryptocurrencies, guided by expectations of value, risk assessment, and potential returns. This study also aims to identify a resourceful technique to efficiently forecast prices of cryptocurrencies such as Bitcoin (BTC), Binance (BNB), Ripple (XRP), and Tether (USDT) using optimal data-driven models (LSTM, GRU, and BiLSTM models) using bias correction. The proposed methodology includes collecting cryptocurrency data and precious metal data from Coindesk and BullionVault, respectively, and then finding the optimal model input combination for each cryptocurrency by lag adjustment and correlating feature selection. Hyperparameter tuning was performed by trial-and-error technique, and an early stopping function was applied to minimize time and space complexity. Bias correction (BC) is applied to model-forecasted price trends to reduce errors in evaluation and to enhance accuracy by adjusting model outputs to reduce prediction bias, providing a refined alternative to traditional unadjusted deep learning methods. GRU-BC outperformed other models in forecasting Bitcoin (with MAE 25.291, RMSE 31.266, MAPE 2.999) and USDT (with MAE 0.0006, RMSE 0.0012, MAPE 0.0622) price trends, while BiLSTM-BC was superior in predicting XRP (with MAE 0.0129, RMSE 0.0171, MAPE 2.9013) and BNB (with MAE 2.2759, RMSE 2.8357, MAPE 1.9785) market price flow. Full article

There are multiple uses for single-channel images, such as infrared imagery, depth maps, and others. To automatically classify objects in such images, an algorithm suited for single-channel image processing is required. This study explores the application of deep learning techniques for the recognition of wild animals using infrared images. Traditional methods of wildlife monitoring often rely on visible light imaging, which can be hindered by various environmental factors such as darkness, fog, and dense foliage. In contrast, infrared imaging captures the thermal signatures of animals, providing a robust alternative for wildlife detection and identification. We test a Convolutional Neural Network (CNN) model specifically designed to analyze infrared images, leveraging the unique thermal patterns emitted by different animal species. The model is trained and tested on a diverse dataset of infrared images, demonstrating high accuracy in distinguishing between multiple species. In this paper, we also present a comparison of several well-known artificial neural networks on this data. To ensure accurate testing, we introduce a new dataset containing infrared photos of Slovak wildlife, specifically including classes such as bear, deer, boar, and fox. To complement this dataset, the Fashion MNIST dataset was also used. Our results indicate that deep learning approaches significantly enhance the capability of infrared imaging for wildlife monitoring, offering a reliable and efficient tool for conservation efforts and ecological studies. Full article

Large language models (LLMs) have transformed Natural Language Processing (NLP) by enabling robust text generation and understanding. However, their deployment in sensitive domains like healthcare, finance, and legal services raises critical concerns about privacy and data security. This paper proposes a comprehensive framework for embedding trust mechanisms into LLMs to dynamically control the disclosure of sensitive information. The framework integrates three core components: User Trust Profiling, Information Sensitivity Detection, and Adaptive Output Control. By leveraging techniques such as Role-Based Access Control (RBAC), Attribute-Based Access Control (ABAC), Named Entity Recognition (NER), contextual analysis, and privacy-preserving methods like differential privacy, the system ensures that sensitive information is disclosed appropriately based on the user’s trust level. By focusing on balancing data utility and privacy, the proposed solution offers a novel approach to securely deploying LLMs in high-risk environments. Future work will focus on testing this framework across various domains to evaluate its effectiveness in managing sensitive data while maintaining system efficiency. Full article

Curriculum learning is a method of prioritizing learning data to improve learning performance. In this paper, we propose a new algorithm that determines how to select learning data and when to start and stop curriculum learning by considering learning errors. We use entropy to select data samples with less consistent predictions and automatically determine the warming-up period based on the characteristics of the data. Additionally, to mitigate learning bias, we introduced a variable that adjusts the range of sample selection according to the progress of the training. To validate our method, we conducted extensive experiments on both balanced and imbalanced data classification tasks, and our proposed approach showed an average improvement of about 1.8%, with a maximum improvement of up to 4.4%, compared to previously suggested methods. Full article

The prioritization of bug reports based on severity is a crucial aspect of bug triaging, enabling a focus on more critical issues. Traditional methods for assessing bug severity range from manual inspection to the application of machine and deep learning techniques. However, manual evaluation tends to be resource-intensive and inefficient, while conventional learning models often lack contextual understanding. This study explores the effectiveness of large language models (LLMs) in predicting bug report severity. We propose a novel approach called SevPredict using GPT-2, an advanced LLM, and compare it against state-of-the-art models. The comparative analysis between the proposed approach and state-of-the-art approaches suggests that the proposed approach outperforms the state-of-the-art approaches in terms of performance evaluation metrics. SevPredict shows improvements over the best-performing state-of-the-art approach (BERT-SBR) with 1.72% higher accuracy, 2.18% higher precision, and 4.94% higher MCC. The improvements are even more substantial when compared to the approach by Ramay et al., with SevPredict demonstrating 10.66% higher accuracy, 10.39% higher precision, 3.29% higher recall, 7.19% higher F1-score, and a remarkable 41.27% higher MCC. These findings not only demonstrate the superiority of our GPT-2-based approach in predicting the severity of bug reports but also highlight its potential to significantly advance automated bug triaging and software maintenance. This research introduces a severity prediction tool named SevPredict. Full article

Substance use disorders affect 17.3% of Americans. Digital health solutions that use machine learning to detect substance use from wearable biosignal data can eventually pave the way for real-time digital interventions. However, difficulties in addressing severe between-subject data heterogeneity have hampered the adaptation of machine learning approaches for substance use detection, necessitating more robust technological solutions. We tested the utility of personalized machine learning using participant-specific convolutional neural networks (CNNs) enhanced with self-supervised learning (SSL) to detect drug use. In a pilot feasibility study, we collected data from 9 participants using Fitbit Charge 5 devices, supplemented by ecological momentary assessments to collect real-time labels of substance use. We implemented a baseline 1D-CNN model with traditional supervised learning and an experimental SSL-enhanced model to improve individualized feature extraction under limited label conditions. Results: Among the 9 participants, we achieved an average area under the receiver operating characteristic curve score across participants of 0.695 for the supervised CNNs and 0.729 for the SSL models. Strategic selection of an optimal threshold enabled us to optimize either sensitivity or specificity while maintaining reasonable performance for the other metric. Conclusion: These findings suggest that Fitbit data have the potential to enhance substance use monitoring systems. However, the small sample size in this study limits its generalizability to diverse populations, so we call for future research that explores SSL-powered personalization at a larger scale. Full article

This article proposes a framework for integrating ethical reasoning into AI systems through Continuous Logic Programming (CLP), emphasizing the improvement of transparency and accountability in automated decision-making. The study highlights requirements for AI that respects human values and societal norms by examining concerns such as algorithmic bias, data privacy, and ethical dilemmas in fields like healthcare and autonomous systems. The proposed CLP-based methodology offers a systematic, elucidative framework for ethical decision-making, allowing AI systems to balance operational efficiency with ethical principles. Important contributions include strategies for the integration of ethical frameworks, stakeholder engagement, and transparency, as well as discussion on artificial moral agents and their function in addressing ethical dilemmas in AI. The paper presents practical examples that illustrate the application of CLP in ethical reasoning, highlighting its ability to bring together AI performance with responsible AI practices. Full article

We report on the effectiveness of using generative neural networks in an antenna design. We considered the modeling of microstrip antennas as they have significant advantages, such as a low profile, lightness, and ease of manufacture, which make them versatile for various applications. We designed, trained, and analyzed generative models applied to the modeling of these antennas without losing the generalizability of the application of these models to any antenna. We started with a Generative Adversarial Network (GAN), which was trained with data related to the antenna models for operation within the frequency range of 1 to 30 GHz. Using the synthetic data produced by the GAN resulted in antenna designs with dimensions and electromagnetic properties that were very close to the expected values. Next, a model was developed using a Conditional GAN (CGAN), which was trained to generate antenna characteristic data conditioned on an arbitrary central frequency, i.e., 2.4 GHz (generally used for Bluetooth, Wi-Fi, and ZigBee technologies), to enable better control over the process of generating these synthetic data. The CGAN model could satisfactorily generate synthetic data for this frequency range, simultaneously considering substrates with different dielectric permittivities. This study reveals that both generative models could produce synthetic data that were very close to the expected data, as evidenced by the low error values. Additionally, in terms of application, the models could provide both geometries and more than one antenna characteristic (resonance, bandwidth, and quality factor), which is very useful for direct application to practical designs. Full article

Background: Large language models (LLMs) like GPT-3.5-Turbo and GPT-4 show potential to transform medical diagnostics through their linguistic and analytical capabilities. This study evaluates their diagnostic proficiency using English and German medical examination datasets. Methods: We analyzed 452 English and 637 German medical examination questions using GPT models. Performance metrics included broad and exact accuracy rates for primary and three-model generated guesses, with an analysis of performance against varying question difficulties based on student accuracy rates. Results: GPT-4 demonstrated superior performance, achieving up to 95.4% accuracy when considering approximate similarity in English datasets. While GPT-3.5-Turbo showed better results in English, GPT-4 maintained consistent performance across both languages. Question difficulty was correlated with diagnostic accuracy, particularly in German datasets. Conclusions: The study demonstrates GPT-4’s significant diagnostic capabilities and cross-linguistic flexibility, suggesting potential for clinical applications. However, further validation and ethical consideration are necessary before widespread implementation. Full article

This work proposes the use of a micro genetic algorithm to optimize the architecture of fully connected layers in convolutional neural networks, with the aim of reducing model complexity without sacrificing performance. Our approach applies the paradigm of transfer learning, enabling training without the need for extensive datasets. A micro genetic algorithm requires fewer computational resources due to its reduced population size, while still preserving a substantial degree of the search capabilities found in algorithms with larger populations. By exploring different representations and objective functions, including classification accuracy, hidden neuron ratio, minimum redundancy, and maximum relevance for feature selection, eight algorithmic variants were developed, with six variants performing both hidden layers reduction and feature-selection tasks. Experimental results indicate that the proposed algorithm effectively reduces the architecture of the fully connected layers in the convolutional neural network. The variant achieving the best reduction used only 44% of the convolutional features in the input layer, and only 9.7% of neurons in the hidden layers, without negatively impacting (statistically confirmed) classification accuracy when compared to a network model based on a full reference architecture and a representative method from the literature. Full article