Applications of Machine Learning in Large-Scale Optimization and High-Dimensional Learning

The global community is awaiting the advent of a self-driving vehicle that is safe, reliable, and capable of navigating a diverse range of road conditions and terrains. This requires a lot of research, study, and optimization. Thus, this work focused on implementing, training, and optimizing a convolutional neural network (CNN) model, aiming to predict the steering angle during driving (one of the main issues). The considered dataset comprises images collected inside a car-driving simulator and further processed for augmentation and removal of unimportant details. In addition, an innovative data-balancing process was previously performed. A CNN model was trained with the dataset, conducting a comparison between several different standard optimizers. Moreover, evolutionary optimization was applied to optimize the model’s weights as well as the optimizers themselves. Several experiments were performed considering different approaches of genetic algorithms (GAs) along with other optimizers from the state of the art. The obtained results demonstrate that the GA is an effective optimization tool for this problem. Full article

Maximum entropy (MaxEnt) models are a class of statistical models that use the maximum entropy principle to estimate probability distributions from data. Due to the size of modern data sets, MaxEnt models need efficient optimization algorithms to scale well for big data applications. State-of-the-art algorithms for MaxEnt models, however, were not originally designed to handle big data sets; these algorithms either rely on technical devices that may yield unreliable numerical results, scale poorly, or require smoothness assumptions that many practical MaxEnt models lack. In this paper, we present novel optimization algorithms that overcome the shortcomings of state-of-the-art algorithms for training large-scale, non-smooth MaxEnt models. Our proposed first-order algorithms leverage the Kullback–Leibler divergence to train large-scale and non-smooth MaxEnt models efficiently. For MaxEnt models with discrete probability distribution of n elements built from samples, each containing m features, the stepsize parameter estimation and iterations in our algorithms scale on the order of $O\left(mn\right)$ operations and can be trivially parallelized. Moreover, the strong ${\ell }_1$ convexity of the Kullback–Leibler divergence allows for larger stepsize parameters, thereby speeding up the convergence rate of our algorithms. To illustrate the efficiency of our novel algorithms, we consider the problem of estimating probabilities of fire occurrences as a function of ecological features in the Western US MTBS-Interagency wildfire data set. Our numerical results show that our algorithms outperform the state of the art by one order of magnitude and yield results that agree with physical models of wildfire occurrence and previous statistical analyses of wildfire drivers. Full article

With the continuous growth of electricity demand, the safety and stability of transmission lines have become increasingly important. To ensure the reliability of power supply, it is essential to promptly detect and address foreign object intrusions on transmission lines, such as tree branches, kites, and balloons. Addressing the issues where foreign objects can cause power outages and severe safety accidents, as well as the inefficiency, time consumption, and labor-intensiveness of traditional manual inspection methods, especially in large-scale power transmission lines, we propose an enhanced YOLOv8-based model for detecting foreign objects. This model incorporates the Swin Transformer, AFPN (Asymptotic Feature Pyramid Network), and a novel loss function, Focal SIoU, to improve both the accuracy and real-time detection of hazards. The integration of the Swin Transformer into the YOLOv8 backbone network significantly improves feature extraction capabilities. The AFPN enhances the multi-scale feature fusion process, effectively integrating information from different levels and improving detection accuracy, especially for small and occluded objects. The introduction of the Focal SIoU loss function optimizes the model’s training process, enhancing its ability to handle hard-to-classify samples and uncertain predictions. This method achieves efficient automatic detection of foreign objects by comprehensively utilizing multi-level feature information and optimized label matching strategies. The dataset used in this study consists of images of foreign objects on power transmission lines provided by a power supply company in Jilin, China. These images were captured by drones, offering a comprehensive view of the transmission lines and enabling the collection of detailed data on various foreign objects. Experimental results show that the improved YOLOv8 network has high accuracy and recall rates in detecting foreign objects such as balloons, kites, and bird nests, while also possessing good real-time processing capabilities. Full article

As the demand for efficient and lightweight models in image classification grows, knowledge distillation has emerged as a promising technique to transfer expertise from complex teacher models to simpler student models. However, the efficacy of knowledge distillation is intricately linked to the choice of datasets used during training. Datasets are pivotal in shaping a model’s learning process, influencing its ability to generalize and discriminate between diverse patterns. While considerable research has independently explored knowledge distillation and image classification, a comprehensive understanding of how different datasets impact knowledge distillation remains a critical gap. This study systematically investigates the impact of diverse datasets on knowledge distillation in image classification. By varying dataset characteristics such as size, domain specificity, and inherent biases, we aim to unravel the nuanced relationship between datasets and the efficacy of knowledge transfer. Our experiments employ a range of datasets to comprehensively explore their impact on the performance gains achieved through knowledge distillation. This study contributes valuable guidance for researchers and practitioners seeking to optimize image classification models through kno-featured applications. By elucidating the intricate interplay between dataset characteristics and knowledge distillation outcomes, our findings empower the community to make informed decisions when selecting datasets, ultimately advancing the field toward more robust and efficient model development. Full article