Search Results (3,269)

Wildfires pose a severe threat to ecological systems, human life, and infrastructure, making early detection critical for timely intervention. Traditional fire detection systems rely heavily on single-sensor approaches and are often hindered by environmental conditions such as smoke, fog, or nighttime scenarios. This paper proposes Adaptive Multi-Sensor Oriented Object Detection with Space–Frequency Selective Convolution (AMSO-SFS), a novel deep learning-based model optimized for drone-based wildfire and smoke detection. AMSO-SFS combines optical, infrared, and Synthetic Aperture Radar (SAR) data to detect fire and smoke under varied visibility conditions. The model introduces a Space–Frequency Selective Convolution (SFS-Conv) module to enhance the discriminative capacity of features in both spatial and frequency domains. Furthermore, AMSO-SFS utilizes weakly supervised learning and adaptive scale and angle detection to identify fire and smoke regions with minimal labeled data. Extensive experiments show that the proposed model outperforms current state-of-the-art (SoTA) models, achieving robust detection performance while maintaining computational efficiency, making it suitable for real-time drone deployment. Full article

The fusion of multiple sensors’ data in real-time is a crucial process for autonomous and assisted driving, where high-level controllers need classification of objects in the surroundings and estimation of relative positions. This paper presents an open-source framework to estimate the distance between a vehicle equipped with sensors and different road objects on its path using the fusion of data from cameras, radars, and LiDARs. The target application is an Advanced Driving Assistance System (ADAS) that benefits from the integration of the sensors’ attributes to plan the vehicle’s speed according to real-time road occupation and distance from obstacles. Based on geometrical projection, a low-level sensor fusion approach is proposed to map 3D point clouds into 2D camera images. The fusion information is used to estimate the distance of objects detected and labeled by a Yolov7 detector. The open-source pipeline implemented in ROS consists of a sensors’ calibration method, a Yolov7 detector, 3D point cloud downsampling and clustering, and finally a 3D-to-2D transformation between the reference frames. The goal of the pipeline is to perform data association and estimate the distance of the identified road objects. The accuracy and performance are evaluated in real-world urban scenarios with commercial hardware. The pipeline running on an embedded Nvidia Jetson AGX achieves good accuracy on object identification and distance estimation, running at 5 Hz. The proposed framework introduces a flexible and resource-efficient method for data association from common automotive sensors and proves to be a promising solution for enabling effective environment perception ability for assisted driving. Full article

The accuracy of satellite positioning results depends on the number of available satellites in the sky. In complex environments such as urban canyons, the effectiveness of satellite positioning is often compromised. To enhance the positioning accuracy of low-cost sensors, this paper combines the visual odometer data output by Xtion with the GNSS/IMU integrated positioning data output by the satellite receiver and MEMS IMU both in the mobile phone through adaptive Kalman filtering to improve positioning accuracy. Studies conducted in different experimental scenarios have found that in unobstructed environments, the RMSE of GNSS/IMU/visual fusion positioning accuracy improves by 50.4% compared to satellite positioning and by 24.4% compared to GNSS/IMU integrated positioning. In obstructed environments, the RMSE of GNSS/IMU/visual fusion positioning accuracy improves by 57.8% compared to satellite positioning and by 36.8% compared to GNSS/IMU integrated positioning. Full article

Odor source localization (OSL) technology allows autonomous agents like mobile robots to localize a target odor source in an unknown environment. This is achieved by an OSL navigation algorithm that processes an agent’s sensor readings to calculate action commands to guide the robot to locate the odor source. Compared to traditional ‘olfaction-only’ OSL algorithms, our proposed OSL algorithm integrates vision and olfaction sensor modalities to localize odor sources even if olfaction sensing is disrupted by non-unidirectional airflow or vision sensing is impaired by environmental complexities. The algorithm leverages the zero-shot multi-modal reasoning capabilities of large language models (LLMs), negating the requirement of manual knowledge encoding or custom-trained supervised learning models. A key feature of the proposed algorithm is the ‘High-level Reasoning’ module, which encodes the olfaction and vision sensor data into a multi-modal prompt and instructs the LLM to employ a hierarchical reasoning process to select an appropriate high-level navigation behavior. Subsequently, the ‘Low-level Action’ module translates the selected high-level navigation behavior into low-level action commands that can be executed by the mobile robot. To validate our algorithm, we implemented it on a mobile robot in a real-world environment with non-unidirectional airflow environments and obstacles to mimic a complex, practical search environment. We compared the performance of our proposed algorithm to single-sensory-modality-based ‘olfaction-only’ and ‘vision-only’ navigation algorithms, and a supervised learning-based ‘vision and olfaction fusion’ (Fusion) navigation algorithm. The experimental results show that the proposed LLM-based algorithm outperformed the other algorithms in terms of success rates and average search times in both unidirectional and non-unidirectional airflow environments. Full article

The detection of human activities is an important step in automated systems to understand the context of given situations. It can be useful for applications like healthcare monitoring, smart homes, and energy management systems for buildings. To achieve this, a sufficient data basis is required. The presented dataset contains labeled recordings of 25 different activities of daily living performed individually by 14 participants. The data were captured by five multisensors in supervised sessions in which a participant repeated each activity several times. Flawed recordings were removed, and the different data types were synchronized to provide multi-modal data for each activity instance. Apart from this, the data are presented in raw form, and no further filtering was performed. The dataset comprises ambient audio and vibration, as well as infrared array data, light color and environmental measurements. Overall, 8615 activity instances are included, each captured by the five multisensor devices. These multi-modal and multi-channel data allow various machine learning approaches to the recognition of human activities, for example, federated learning and sensor fusion. Full article

Object detection is an essential computer vision task that identifies and locates objects within images or videos and is crucial for applications such as autonomous driving, robotics, and augmented reality. Light Detection and Ranging (LiDAR) and camera sensors are widely used for reliable object detection. These sensors produce heterogeneous data due to differences in data format, spatial resolution, and environmental responsiveness. Existing review articles on object detection predominantly focus on the statistical analysis of fusion algorithms, often overlooking the complexities of aligning data from these distinct modalities, especially dynamic environment data alignment. This paper addresses the challenges of heterogeneous LiDAR-camera alignment in dynamic environments by surveying over 20 alignment methods for three-dimensional (3D) object detection, focusing on research published between 2019 and 2024. This study introduces the core concepts of multimodal 3D object detection, emphasizing the importance of integrating data from different sensor modalities for accurate object recognition in dynamic environments. The survey then delves into a detailed comparison of recent heterogeneous alignment methods, analyzing critical approaches found in the literature, and identifying their strengths and limitations. A classification of methods for aligning heterogeneous data in 3D object detection is presented. This paper also highlights the critical challenges in aligning multimodal data, including dynamic environments, sensor fusion, scalability, and real-time processing. These limitations are thoroughly discussed, and potential future research directions are proposed to address current gaps and advance the state-of-the-art. By summarizing the latest advancements and highlighting open challenges, this survey aims to stimulate further research and innovation in heterogeneous alignment methods for multimodal 3D object detection, thereby pushing the boundaries of what is currently achievable in this rapidly evolving domain. Full article

Human engagement is a vital test research area actively explored in cognitive science and user experience studies. The rise of big data and digital technologies brings new opportunities into this field, especially in autonomous systems and smart applications. This article reviews the latest sensors, current advances of estimation methods, and existing domains of application to guide researchers and practitioners to deploy engagement estimators in various use cases from driver drowsiness detection to human–robot interaction (HRI). Over one hundred references were selected, examined, and contrasted in this review. Specifically, this review focuses on accuracy and practicality of use in different scenarios regarding each sensor modality, as well as current opportunities that greater automatic human engagement estimation could unlock. It is highlighted that multimodal sensor fusion and data-driven methods have shown significant promise in enhancing the accuracy and reliability of engagement estimation. Upon compiling the existing literature, this article addresses future research directions, including the need for developing more efficient algorithms for real-time processing, generalization of data-driven approaches, creating adaptive and responsive systems that better cater to individual needs, and promoting user acceptance. Full article

This systematic review evaluates the integration of advanced radar technologies into unmanned ground vehicles (UGVs), focusing on their role in enhancing autonomy in defense, transportation, and exploration. A comprehensive search across IEEE Xplore, Google Scholar, arXiv, and Scopus identified relevant studies from 2007 to 2024. The studies were screened, and 54 were selected for full analysis based on inclusion criteria. The review details advancements in radar perception, machine learning integration, and sensor fusion while also discussing the challenges of radar deployment in complex environments. The findings reveal both the potential and limitations of radar technology in UGVs, particularly in adverse weather and unstructured terrains. The implications for practice, policy, and future research are outlined. Full article

Pine wilt disease (PWD), caused by pine wood nematodes, is a major forest disease that poses a serious threat to global pine forest resources. Therefore, the prompt identification of PWD-discolored trees is crucial for controlling its spread. Currently, remote sensing is the primary approach for monitoring PWD. This study comprehensively reviews advances in the global remote sensing monitoring of PWD. It explores the remote sensing platforms and identification methods used in the detection of PWD-discolored trees, evaluates their precision, and provides prospects for existing problems. Three observations were made from existing studies: First, unmanned aerial vehicles (UAVs) are the dominant remote sensing platforms, and RGB data sources are the most commonly used for identifying PWD-discolored trees. Second, deep-learning methods are increasingly applied to identify PWD-discolored trees. Third, the early monitoring of PWD-discolored trees has gained increasing attention. This study reveals the problems associated with the acquisition of remote sensing images and identification algorithms. Future research directions include the fusion of multiple sensors to enhance the identification precision and early monitoring of PWD-discolored trees to obtain an optimal detection window period. This study aimed to provide technical references and scientific foundations for the comprehensive monitoring and control of PWD. Full article

The Internet of Things (IoT) has emerged as a transformative technology with a variety of applications across various industries. However, the development of IoT systems is hindered by challenges such as interoperability, system complexity, and the need for streamlined development and maintenance processes. In this study, we introduce a robust architecture grounded in discrete event system specification (DEVS) as a model-driven development solution to overcome these obstacles. Our proposed architecture utilizes the publish/subscribe paradigm, and it also adds to the robustness of the proposed solution with the incorporation of the Brooks–Iyengar algorithm to enhance fault tolerance against unreliable sensor readings. We detail the DEVS specification that is used to define this architecture and validate its effectiveness through a detailed home automation case study that integrates multiple sensors and actuators. Full article

This article introduces an advanced theoretical approach, named the Trigonometric Kalman Filter (TKF), to enhance measurement accuracy for multi-head rotational encoders. Leveraging the processing capabilities of a Field-Programmable Gate Array (FPGA), the proposed TKF algorithm uses trigonometric functions and sophisticated signal fusion techniques to provide highly accurate real-time angle estimation with rapid response. The inclusion of the Coordinate Rotation Digital Computer (CORDIC) algorithm enables swift and efficient computation of trigonometric values, facilitating precise tracking of angular position and rotational speed. This approach represents a notable advancement in control systems, where high accuracy and minimal latency are essential for optimal performance. The paper addresses key challenges in angle measurement, particularly the signal fusion inaccuracies that often impede precision in high-demand applications. Implementing the TKF with an FPGA-based pure fixed-point method not only enhances computational efficiency but also significantly reduces latency when compared to conventional software-based solutions. This FPGA-based implementation is particularly advantageous in real-time applications where processing speed and accuracy are critical, and it demonstrates the effective integration of hardware acceleration in improving measurement fidelity. To validate the effectiveness of this approach, the TKF was rigorously tested on a precision drive control system, configured for a direct PMSM drive in an astronomical telescope mount equipped with a standard $0.5$m telescope frequently used by astronomers. This real-world application highlights the TKF’s ability to meet the stringent positioning and measurement accuracy requirements characteristic of astronomical observation, a field where minute angular adjustments are critical. The FPGA-based design enables high-frequency updates, essential for managing the minor, precise adjustments required for telescope control. The study includes a comprehensive computational analysis and experimental testing on an Altera Stratix FPGA board, presenting a detailed comparison of the TKF’s performance with other known methods, including fusion techniques such as differential methods, $\alpha$–$\beta$ filters, and related Kalman filtering applied to one sensors. The study demonstrates that the four-head fusion configuration of the TKF outperforms traditional methods in terms of measurement accuracy and responsiveness. Full article

The remaining useful life (RUL) prediction of rolling bearings is crucial for optimizing maintenance schedules, reducing downtime, and extending machinery lifespan. However, existing multi-channel feature fusion methods do not fully capture the correlations between channels and time points in multi-dimensional sensor data. To address the above problems, this paper proposes a multi-channel feature fusion algorithm based on a hybrid attention mechanism and temporal convolutional networks (TCNs), called MCHA-TFCN. The model employs a dual-channel hybrid attention mechanism, integrating self-attention and channel attention to extract spatiotemporal features from multi-channel inputs. It uses causal dilated convolutions in TCNs to capture long-term dependencies and incorporates enhanced residual structures for global feature fusion, effectively extracting high-level spatiotemporal degradation information. The experimental results on the PHM2012 dataset show that MCHA-TFCN achieves excellent performance, with an average Root-Mean-Square Error (RMSE) of 0.091, significantly outperforming existing methods like the DANN and CNN-LSTM. Full article

Millions of people around the globe are impacted by falls annually, making it a significant public health concern. Falls are particularly challenging to detect in real time, as they often occur suddenly and with little warning, highlighting the need for innovative detection methods. This study aimed to assist in the advancement of an accurate and efficient fall detection system using electroencephalogram (EEG) data to recognize the reaction to a postural disturbance. We employed a state-space-based system identification approach to extract features from EEG signals indicative of reactions to postural perturbations and compared its performance with those of traditional autoregressive (AR) and Shannon entropy (SE) methods. Using EEG epochs starting from 80 ms after the onset of the event yielded improved performance compared with epochs that started from the onset. The classifier trained on the EEG data achieved promising results, with a sensitivity of up to 90.9%, a specificity of up to 97.3%, and an accuracy of up to 95.2%. Additionally, a real-time algorithm was developed to integrate the EEG and accelerometer data, which enabled accurate fall detection in under 400 ms and achieved an over 99% accuracy in detecting unexpected falls. This research highlights the potential of using EEG data in conjunction with other sensors for developing more accurate and efficient fall detection systems, which can improve the safety and quality of life for elderly adults and other vulnerable individuals. Full article

This research proposes a novel modeling method for integrating IMU arrays into multi-sensor kinematic positioning/navigation systems. This method characterizes sensor errors (biases/scale factor errors) for each IMU in an IMU array, leveraging the novel Generic Multisensor Integration Strategy (GMIS) and the framework for comprehensive error analysis in Discrete Kalman filtering developed through the authors’ previous research. This work enables the time-varying estimation of all individual sensor errors for an IMU array, as well as rigorous fault detection and exclusion for outlying measurements from all constituent sensors. This research explores the feasibility of applying Variance Component Estimation (VCE) to IMU array data, using separate variance components to characterize the performance of each IMU’s gyroscopes and accelerometers. This analysis is only made possible by directly modeling IMU inertial measurements under the GMIS. A real land-vehicle kinematic dataset was used to demonstrate the proposed technique. The a posteriori positioning/attitude standard deviations were compared between multi-IMU and single IMU solutions, with the multi-IMU solution providing an average accuracy improvement of ca. 14–16% in the estimated position, 30% in the estimated roll and pitch, and 40% in the estimated heading. The results of this research demonstrate that IMUs in an array do not generally exhibit homogeneous behavior, even when using the same model of tactical-grade MEMS IMU. Furthermore, VCE was used to compare the performance of three IMU sensors, which is not possible under other IMU array data fusion techniques. This research lays the groundwork for the future evaluation of IMU array sensor configurations. Full article

The dispersion of structural waves, where wave speed varies with frequency, introduces significant challenges in accurately localizing occupants in a building based on vibrations caused by their movements. This study presents a novel multi-sensor vibro-localization technique that accounts for dispersion effects, enhancing the accuracy and robustness of occupant localization. The proposed method utilizes a model-based approach to parameterize key propagation phenomena, including wave dispersion and attenuation, which are fitted to observed waveforms. The localization is achieved by maximizing the joint likelihood of the occupant’s location based on sensor measurements. The effectiveness of the proposed technique is validated using two experimental datasets: one from a controlled environment involving an aluminum plate and the other from a building-scale experiment conducted at Goodwin Hall, Virginia Tech. Results for the proposed algorithm demonstrates a significant improvement in localization accuracy compared to benchmark algorithms. Specifically, in the aluminum plate experiments, the proposed technique reduced the average localization precision from 7.77 cm to 1.97 cm, representing a ∼74% improvement. Similarly, in the Goodwin Hall experiments, the average localization error decreased from 0.67 m to 0.3 m, with a ∼55% enhancement in accuracy. These findings indicate that the proposed approach outperforms existing methods in accurately determining occupant locations, even in the presence of dispersive wave propagation. Full article