[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Next Article in Journal
Modeling and Experimental Verification of In-House Built Portable Ultrafiltration (PUF) System to Maintain Water Quality
Next Article in Special Issue
Estimated Tardiness-Based Reinforcement Learning Solution to Repeatable Job-Shop Scheduling Problems
Previous Article in Journal
Study on Dense Phase Separation of Associated Gas with High Carbon Dioxide Content
Previous Article in Special Issue
Automated Dual-Side Leather Defect Detection and Classification Using YOLOv11: A Case Study in the Finished Leather Industry
You seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Processes
Volume 12
Issue 12
10.3390/pr12122925
processes-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Parallel Prognostic Method Integrating Uncertainty Quantification for Probabilistic Remaining Useful Life Prediction of Aero-Engine

by
Rongqiu Wang
1,
Ya Zhang
1,*,
Chen Hu
2,
Zhengquan Yang
3,
Huchang Li
1,
Fuqi Liu
1,
Linling Li
1 and
Junyu Guo
4,*
1
College of Electronic Information and AI, Yibin Vocational and Technical College, Yibin 644003, China
2
China Oil & Gas Pipeline Network Corporation Central China Branch, Wuhan 430000, China
3
Sichuan Yibin Minjiang Machinery Manufacturing Co., Ltd., Yibin 644000, China
4
School of Mechatronic Engineering, Southwest Petroleum University, Chengdu 610500, China
*
Authors to whom correspondence should be addressed.
Processes 2024, 12(12), 2925; https://doi.org/10.3390/pr12122925
Submission received: 25 November 2024 / Revised: 12 December 2024 / Accepted: 17 December 2024 / Published: 20 December 2024
(This article belongs to the Special Issue Applications of Artificial Intelligence Technologies in Energy, Manufacturing and Automatic Control Processes)
Browse Figures
Figure 1
<p>The flowchart of the proposed method.</p> "> Figure 2
<p>The structure of the TCN model.</p> "> Figure 3
<p>The visualization of dilated causal convolution.</p> "> Figure 4
<p>The architecture of the Transformer.</p> "> Figure 5
<p>The principle of multi-head self-attention.</p> "> Figure 6
<p>The comparison between the deterministic model and BNN-like model.</p> "> Figure 7
<p>The simplified diagram of C-MAPSS.</p> "> Figure 8
<p>The visualization of selected normalized sensor measurements.</p> "> Figure 9
<p>The schematic of sliding window processing.</p> "> Figure 10
<p>The rectified piece-wise RUL label.</p> "> Figure 11
<p>The performance comparison with other hyperparameters.</p> "> Figure 12
<p>The visualization of comparison results with other methods.</p> "> Figure 13
<p>The prediction results for the testing engines.</p> "> Figure 14
<p>The visualization of the prediction results on testing datasets.</p> "> Figure 15
<p>The visualization of prediction performance comparison with different methods.</p> ">
Versions Notes

Abstract

:
Remaining useful life (RUL) prediction plays a fundamental role in the prognostics and health management of mechanical equipment. Consequently, extensive research has been devoted to estimating the RUL of mechanical equipment. Owing to the development of modern advanced sensor technologies, a significant amount of monitoring data is recorded. Traditional methods, such as machine-learning-based methods and statistical-data-driven methods, are ineffective in matching when faced with big data thus leading to poor predictions. As a result, deep-learning-based methods are extensively utilized due to their efficient capability to excavate deep features and realize accurate predictions. However, most deep-learning-based methods only provide point estimations and ignore the prediction uncertainty. To address this limitation, this paper proposes a parallel prognostic network to sufficiently excavate the degradation features from multiple dimensions for more accurate RUL prediction. In addition, accurate calculation of model evidence is extremely difficult when dealing with big data so the Monte Carlo dropout is employed to infer the model weights under low computational cost and high scalability to obtain a probabilistic RUL prediction. Finally, the C-MAPSS aero-engine dataset is employed to validate the proposed dual-channel framework. The experimental results illustrate its superior prediction performance compared to other deep learning methods and the ability to quantify prediction uncertainty.

1. Introduction

Prognostics and health management is a critical technology that reduces operating and maintenance costs and improves the availability of mechanical structures [1,2,3]. PHM continuously monitors equipment status, analyzes data, and predicts faults, enabling early detection of potential issues, preventing equipment failures and unplanned downtime, and improving system reliability and maintenance efficiency [4,5,6,7]. Moreover, the remaining useful life (RUL) prediction as a central component of PHM has received increasing attention and research [8,9]. However, because of complex operating procedures, mechanical equipment tends to deteriorate, leading to various types of failures [10,11,12,13]. Therefore, it is of vital importance to enhance the RUL prediction accuracy of mechanical equipment to ensure operational safety.
Nowadays, RUL prediction methods are categorized into two types: model-based methods and data-driven methods [14,15,16]. Model-based methods normally utilize the theoretical or physical model to represent the degradation of the mechanical equipment [17,18]. However, the accuracy of degradation modeling is significantly dependent on the prior physical knowledge, which is particularly tough to obtain when the system is complex [19,20].
Due to the advanced feature extraction capability of data-driven methods, they can easily overcome the above limitations. Among the data-driven methods, machine learning has been extensively employed and developed in recent years due to the fact that it does not require prior knowledge and has better generalization performance [21,22]. According to different learning algorithms, machine learning is broadly categorized into shallow machine learning algorithms and deep learning (DL) algorithms [23]. Shallow machine learning algorithms are routinely utilized in regression and classification tasks with small amounts of data, such as the support vector machine, relevance vector machine, and hidden Markov model [24,25]. Due to the advanced feature extraction and mapping capabilities, DL-based methods are extensively adopted in the PHM field. Normally employed DL-based methods contain the convolutional neural network (CNN), recurrent neural network (RNN), deep belief network, and Transformer. To be specific, Peng et al. [26] constructed the health indicator of an aero-engine by utilizing the deep belief network, and the double exponential function was employed to depict the degradation process. Wang et al. [27] proposed an improved Inception module with a gated recurrent unit (GRU) based on a multi-branch parallel structure. The experimental results illustrate that the model can improve feature extraction capacity and realize accurate RUL prediction. Lu et al. [28] utilized a multi-scale CNN and bidirectional GRU with temporal attention to predict the RUL of aero-engines. When processing the long-term sequence, traditional RNN-based methods may have problems such as gradient disappearance and explosion. However, due to the special mechanism called multi-head self-attention, the Transformer network could overcome the above limitations. To be specific, Liu et al. [29] employed a dual attention-based CNN and Transformer model to forecast the RUL of aircraft engines, and the comparison results illustrate the effectiveness of the model. Jiang et al. [30] employed a convolutional dual-channel Transformer to realize the RUL prediction of rolling bearings, and the experimental evaluation showed that the algorithm could accurately extract the bearing degradation features to ensure the prediction accuracy. Moreover, with the development of artificial intelligence, more and more improved networks are proposed. The temporal convolutional network (TCN), as an improved form of the CNN, is able to effectively capture a longer receptive field than the traditional CNN when using the same number of layers. This results from the fact that the TCN utilizes the residual connection and dilated causal convolution. Lin et al. [31] implemented a double attention TCN framework to predict the RUL of aircraft engines. It was learned from the experimental results that the proposed framework can achieve accurate RUL prediction, especially in complex failure modes. Chen et al. [32] proposed an improved GRU-TCN model to realize the RUL prediction of aero-engines. The experiment results on the aero-engine dataset illustrate the effectiveness of the proposed model.
Although the above-mentioned methods have achieved superior performance, the above-mentioned methods do not consider the prediction uncertainty, which is of vital importance to make the maintenance decisions, shown in Table 1. To solve the above limitations, this paper proposes a dual-channel Bayesian network on the basis of the Transformer and TCN. The dual-channel architecture can combine the benefits of both the TCN and Transformer. Moreover, the Monte Carlo (MC) dropout is employed to infer the model weights and realize the prediction uncertainty quantification. The main contributions of this paper are as follows:
(1)
A dual-channel framework is proposed to adequately combine the benefits of both the TCN and Transformer and obtain a deep degradation representation to realize more accurate RUL prediction.
(2)
The MC dropout is introduced into the framework to realize the Bayesian approximation and obtain the uncertainty quantification results.
The remainder of this paper is organized as follows: Section 2 presents the proposed dual-channel framework. The experimental evaluation and analysis are detailed in Section 3. Finally, the conclusions are provided in Section 4.

2. Methodology

In this section, the proposed method is introduced. Section 2.1 illustrates the overview of the proposed method. The TCN and Transformer are introduced in Section 2.2 and Section 2.3, respectively. Finally, Section 2.4 describes how the MC dropout extends the model to a Bayesian-neural-network-like model.

2.1. Overview of the Proposed Method

The proposed method contains four parts: (1) data acquisition, (2) data pre-processing, (3) model construction, and (4) RUL prediction, as illustrated in Figure 1.
Step 1: First, multiple sensors are employed to measure the operating state of aero-engines, including some physical information (such as speed, temperature, pressure, etc.). Then, data normalization is performed on the raw condition monitoring signals to keep the data fluctuating within the same range. After the data normalization, the data are screened to obtain the features that better portray the degradation process of aero-engines.
Step 2: Once the screened data are obtained, they are transferred into the model for training by combining the strength of the TCN in capturing short-term temporal dependencies with the excellent ability of the Transformer in capturing long-range dependencies. The proposed method benefits from the complementary strengths of both models, leading to superior overall performance in modeling techniques for aero-engine data. So, this paper proposes a dual-channel framework to better exploit the degradation representations to achieve more accurate RUL predictions. Moreover, the MC dropout is introduced into the framework to achieve probabilistic prediction of RUL.
Step 3: Then, the MC dropout is also applied during the online testing process to realize the prediction uncertainty quantification. Moreover, the prediction performance under other hyperparameters is employed for comparison. Finally, the predicted RUL is evaluated against other widely implemented methods in different operational scenarios to confirm the robustness and efficacy of the proposed method.

2.2. TCN

The TCN is a variant version of the CNN, which can efficiently capture the cumulative information when processing the long-term sequence [31]. As shown in Figure 2, the TCN comprises the dilated causal convolution, the activation function, dropout, and WeightNorm. The central element of the TCN is the dilated causal convolution, which builds on causal convolution to capture the long-term performance degradation. Figure 3 illustrates the process of dilated causal convolution. The dilation rate generally tripled in each hidden layer to capture a broader receptive field. The convolutional structure of the TCN facilitates efficient parallel computation, and by stacking multiple layers of dilated convolutions, the TCN achieves a flexible and adjustable receptive field, capable of capturing dependencies across various time scales. Furthermore, TCNs employ causal convolutions, ensuring each output time step depends only on the current and previous time steps, thus guaranteeing real-time prediction accuracy. Unlike RNN-based methods, TCNs effectively capture long-range dependencies through dilated convolutions, avoiding vanishing and exploding gradient issues and demonstrating superior stability and training efficiency. While CNNs offer some parallel computation and sequence data processing capabilities, TCNs are better suited for capturing temporal dependencies and handling long-sequence data with greater stability.

2.3. Transformer

The Transformer is a DL model utilized in natural language processing and various sequence-to-sequence tasks, such as machine translation, text generation, and image processing [33]. Unlike RNN-based methods, the Transformer does not have a recursive structure, which eliminates the issue of forgetting previous states when managing long sequences. By leveraging parallel data processing and self-attention mechanisms, the Transformer effectively captures long-term dependencies within sequences. Its architecture consists of an encoder–decoder structure, multi-head self-attention, and positional encoding. The encoder and decoder consist of several layers, each containing several self-attention heads that compute the relationships between each element in the input sequence and all other elements, thus capturing global information. Following the self-attention layer, a feed-forward neural network further processes and transforms the data. To introduce positional information, positional encoding is added to the input. Additionally, residual connections and layer normalization are embedded within the Transformer to enhance prediction performance and facilitate the training of deeper networks. Through its unique architecture and powerful self-attention mechanism, the Transformer has significantly advanced the fields of natural language processing and other sequence-to-sequence tasks. Figure 4 illustrates the architecture of the Transformer. The following sub-sections present the three principal parts of the Transformer in detail.

2.3.1. Encoder–Decoder Architecture

From Figure 4, The encoders and decoders of the transformer are normally arranged in a stacked configuration on top of each other. Moreover, the encoder and decoder of the Transformer are similar but have distinct functions. The encoder is employed to transform the input feature into a hidden representation, whereas the decoder is utilized to decode the hidden representation to acquire the output. Moreover, the difference between the encoder and decoder lies in the multi-head self-attention. To be specific, the encoder in the Transformer receives and transforms the input sequences into context vectors, capturing the global information mainly through the multi-head self-attention mechanism and feed-forward neural network. The decoder, on the other hand, utilizes these context vectors to generate the output sequence step by step, which includes a causal self-attention mechanism, encoder–decoder attention mechanism, and feed-forward neural network to ensure the accuracy and coherence of the generation. The encoder focuses on global understanding, and the decoder focuses on incremental output generation.

2.3.2. Multi-Head Self-Attention

As mentioned before, the Transformer has no recurrent structure so it only processes the input embedding using the multi-head self-attention. The mechanism has been extensively applied in processing time series and has gained enormous success. The self-attention mechanism can allocate various weights to individual pieces, enabling the model to devote more attention to these important pieces. The calculation processes are listed as follows:
Attention ( Q , K , V ) = Softmax ( QK T d ) V
Q = X P W Q K = X P W K V = X P W V
Softmax ( x i ) = exp ( x i ) ∑ j exp ( x j )
where X P represents the vector obtained after positional encoding of input X . W Q , W K , W V represent the learnable weight parameters. d is the dimension of K . S o f t m a x denotes the activation function.
However, only those pieces of information that are relevant within a single representational space could be accessed by a single attention mechanism. Therefore, the multi-head self-attention mechanism is employed to acquire more comprehensive positional information within multiple representation spaces, as shown in Figure 5. The definitions are denoted as follows:
MultiHead ( Q , K , V ) = Concat ( Head 1 , Head 2 , … , Head n ) W O
Head i = Attention ( X P W i Q , X P W i K , X P W i V )
where n denotes the head number of multi-head self-attention. Head i represents the attention matrix of the i-th head. W O denotes the learnable weight parameter to concat the Head i .

2.3.3. Positional Encoding

In the raw time series, there are only numerical values at each time point. Therefore, it is of vital importance to introduce an algorithm to represent the positional correlation at each time point. The positional encoding is employed to obtain the positional information and inject it into the input embedding. The positional encoding algorithm can be expressed as follows:
PE ( pos , 2 i ) = sin ( pos 10 , 000 2 i / d model ) PE ( pos , 2 i + 1 ) = cos ( pos 10 , 000 2 i / d model )
where pos denotes the position of each time point in input embedding. d model represents the dimension of input embedding. 2 i and 2 i + 1 denote the odd and even positions of the input embedding, respectively.

2.4. Bayesian Neural Network

The Bayesian neural network (BNN) has significant advantages over traditional deterministic models: it is able to quantify prediction uncertainty, provide confidence intervals, prevent overfitting, improve data efficiency, and exhibit higher robustness in noise and anomalous data handling. In addition, the BNN is able to dynamically adjust its interpretation of data by introducing prior distributions, which is highly adaptive and particularly suitable for dynamic environments and application scenarios with constantly changing data. These advantages enable the BNN to excel in tasks with high complexity and uncertainty.
As shown in Figure 6, compared with the traditional deterministic models, the weights and biases of the BNN model are random variables denoted by a probability distribution. Therefore, the BNN model can be regarded as an aggregate model with a variety of weighted models.
Assume that the training dataset is D = { ( X i , Y i ) } i = 0 N , where X i = { x i ( j ) } j = 0 N i and Y i = { y i ( j ) } j = 0 N i represent the condition monitoring data and the corresponding actual RUL label, respectively. Then, the predicted probability of the model is denoted as
p ( y ∗ | x ∗ , D ) = ∫ p ( y ∗ | x ∗ , X , ω ) p ( ω | D ) d ω
p ( ω | D ) = p ( ω ) p ( D | ω ) p ( D ) = p ( ω ) p ( D | ω ) ∫ p ( D | ω ) p ( ω ) d ω
where p ( y ∗ | x ∗ , D ) represents the predicted probability. p ( y ∗ | x ∗ , X , ω ) denotes the likelihood estimation of the model. p ( ω | D ) represents the posterior distribution of the model. p ( ω ) represents the prior distribution of model parameters. x ∗ and y ∗ identify the input and output, respectively.
However, when processing high-dimension condition monitoring data, accurately calculating model evidence is extremely tricky. In this paper, the MC dropout is introduced in the proposed framework to achieve the RUL prediction with epistemic uncertainty quantification, shown in Figure 6. To be specific, the model parameters are sampled by the MC dropout using random dropping and considered as a Bernoulli distribution. The calculation process can be expressed as
q θ ( ω ) = Π i q θ i ( ω i )
q θ i ( ω i ) = p              ω i = θ i 1 − p      ω i = 0
where the p denotes the probability of an element being zeroed.
In this manuscript, the MC dropout is utilized to extend the model to a BNN-like model. In traditional inference processes, the dropout is usually disabled. On the contrary, by using the MC dropout, the dropout is enabled even in the inference phase, thus randomly blocking some neurons in each inference.
By conducting k forward propagations, k different predictions are obtained, denoted as { y ^ 1 ,   y ^ 2 , … ,   y ^ k } . Then, the mean and variance of {   y ^ 1 ,   y ^ 2 , … ,   y ^ k } are represented as
m = mean ( y ^ 1 ,   y ^ 2 , … ,   y ^ k ) = 1 k ∑ i = 1 k   y ^ i σ = std ( y ^ 1 ,   y ^ 2 , … ,   y ^ k ) = 1 k ∑ i = 1 k (   y ^ i − m ) 2
Assuming that the predictions follow a Gaussian distribution, the 95% confidence interval (CI) based on the mean and standard deviation can be expressed as [ m − 1 . 96 ∗ σ , m + 1 . 96 ∗ σ ] .

3. Experimental Results and Discussion

In this section, the experimental analysis and discussion are presented. Section 3.1 details the employed dataset, while Section 3.2 introduces the procedure of data pre-processing, including data normalization, data screening, sliding window processing, and RUL label rectification. The performance metrics used to evaluate the model are introduced in Section 3.3. Then, the selection of model hyperparameters is listed in Section 3.4. Finally, Section 3.5 highlights the superiority of the proposed method by comparing it with other methods.

3.1. Data Description

To confirm the feasibility of the proposed dual-channel Bayesian network, the experimental analysis is performed on aero-engines, which is called the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS), shown in Figure 7 [34]. As listed in Table 2, the C-MAPSS dataset originates from NASA and contains four datasets. Moreover, the FD001 and FD003 only have one operating condition, but the FD002 and FD004 have six operating conditions, where the working conditions are specified by three operating parameters. In addition, the FD001 and FD002 only have one fault mode while the FD003 and FD004 have two fault modes. In this dataset, 21 sensors are employed to monitor the status of the aero-engines, including physical information (e.g., temperature, pressure, speed, etc.) collected from different positions during the running cycle.

3.2. Data Pre-Processing

In this section, the procedures of data pre-processing are presented. First, the data normalization in Section 3.2.1 is used to normalize the monitoring signals into the same range to keep some important degradation information with narrower fluctuation ranges. Then, in Section 3.2.2, the signals of a constant trend are removed because these signals cannot portray the degradation process of aero-engines. After that, the sliding window processing in Section 3.2.3 is introduced to strengthen the relationship between neighboring points at distinct times. Finally, the RUL labels of aero-engines are set based on their degradation characteristics in Section 3.2.4.

3.2.1. Data Normalization

There are considerable differences in the fluctuation ranges between different monitoring sensor signals. Therefore, if the monitoring sensor signals without normalization are fed into the model, the signals with narrower fluctuation ranges will be ignored, and then some degradation information will be lost, leading to inaccurate RUL predictions. To enhance the RUL prediction accuracy and model convergence speed, the min-max normalization is employed to normalize the monitoring signals within the range [0, 1]. The definitions are as follows:
x norm i , j = x i , j − x min j x max j − x min j
where x i , j and x norm i , j denote the monitoring sensor signals before and after normalization, respectively. i represents the i-th data point. j denotes the j-th sensor. x max j and x min j denote the maximum and minimum data points of the j-th sensor, respectively.

3.2.2. Data Screening

From Figure 8a, the normalized data can be categorized into three groups based on trend: increasing, decreasing, and constant, as shown in Table 3. However, the signals of a constant trend cannot portray the degradation process of aero-engines. In this paper, these constant signals (Sensors 1, 5, 6, 16, 18, and 19) are dropped, and the remaining fourteen sensor signals are transmitted into the model to predict the RUL [29], shown in Figure 8b.

3.2.3. Sliding Window Processing

When analyzing the time series, it is of vital importance to explore the relationships between the neighboring points at distinct times. Therefore, sliding window processing is introduced to envelop the neighboring data points at each time step to capture these relationships. As shown in Figure 9, the window step is set to 1, and the sliding window is slid along the columns of each sensor’s data to generate samples.

3.2.4. RUL Label Rectification

According to [35], the early degradation of aero-engines can be neglected, so the aero-engine can be assumed to be in a constant state for a certain period of time. If the degradation process of aero-engines is straightforwardly set to a linear degradation process, then it cannot accurately describe the current health state of the aero-engine. In this paper, similar to [29,31], the threshold is set to 125, i.e., the RUL label is treated as a constant when it exceeds the threshold, as shown in Figure 10.

3.3. Performance Metrics

In the evaluation of the effectiveness and superiority of the method under consideration, three diffusely utilized performance metrics are introduced, namely root mean square error (RMSE), Score, and the coefficient of determination R 2 . The definitions of these performance metrics are as follows:
RMSE = 1 N ∑ i = 1 N ( y pre i − y act i ) 2
Score = ∑ i = 1 N ( exp ( − e i 13 ) − 1 )   if   e i < 0 ∑ i = 1 N ( exp ( e i 10 ) − 1 )   if   e i > 0
e i = y pre i − y act i
R 2 = 1 − ∑ i = 1 N ( y pre i − y act i ) 2 ∑ i = 1 N ( y act i − y ¯ act ) 2
where y pre i and y act i represent the predicted and actual values, respectively. N denotes the number of samples.

3.4. Hyperparameter Selection

The prediction accuracy is remarkably affected by the hyperparameters of the DL models. Therefore, the selection of model hyperparameters is of particular importance. To justify the selection of the model hyperparameters, prediction performance with other hyperparameters is measured and employed for comparison. The model hyperparameter configuration is presented in Table 4, and the performance comparison with other hyperparameters is illustrated in Figure 11. In this paper, two parameters named “Window size” and “Encoder Layer” are explored. As illustrated in Figure 11a, the RMSE of the RUL prediction progressively decreases as the number of encoder layers increases, but when the encoder layer is equal to three, the RMSE increases instead of decreasing. This phenomenon indicates that the number of encoder layers selected in Table 4 is optimal compared to other encoder layers. Moreover, as the window size increases, the RMSE of the RUL prediction dramatically decreases and reaches a minimum when the window size is equal to thirty. However, since the minimum life cycle in the testing dataset of FD001 is thirty-one, the window size must be smaller than that of the minimum life cycle. Hence, the window size is set to 30 in this paper. In summary, the proposed method with the hyperparameters presented in Table 4 exhibits superior prediction performance.

3.5. Experimental Result Analysis

To further evaluate the prediction capability of the proposed method, three widely adopted DL-based methods are introduced for comparison, namely CNN-Transformer, Transformer, and TCN-BiGRU. From Table 5 and Figure 12, compared to other advanced methods, the method proposed in this paper exhibits superior RUL prediction performance. For FD001, the proposed method achieves a maximum reduction of 11.66% and 45.56% in RMSE and Score when compared to other methods, respectively. For FD003, the proposed method achieves a maximum reduction of 48.98% and 92.49% in RMSE and Score when compared to other methods, respectively. It is worth mentioning that among all the methods, the Transformer is the worst at predicting, while the prediction performance of TCN-BiGRU is the second best, which convincingly proves that the proposed method can efficiently combine the merits of the TCN and Transformer thus achieving the most accurate RUL prediction.
Moreover, to provide a more intuitive illustration of the RUL prediction performance of the proposed method, four engine units from the testing dataset are randomly selected. The numbers are FD001#34 and #36 and FD003#10 and #94. As can be seen in Figure 13, the error between the predicted value and the actual value is relatively small, and both have the same degradation trend. Moreover, the predicted value is approximately constant in the early operating state and then degrades as the operating cycle increases. It is worth mentioning that the prediction results of the proposed method are closer to the actual RUL as the operating cycle increases.
Moreover, the constructed confidence intervals (CIs) contain the actual RUL adequately. In the RUL prediction of aero-engines, there often exist uncertainties due to factors such as environmental conditions, material property variations, and differences in operational settings. By quantifying these uncertainties, the maintenance engineers are able to obtain more accurate decision support. For example, for a specific engine component, uncertainty analysis can help maintenance engineers better assess the optimal time for maintenance, selecting a maintenance window that balances safety and cost-effectiveness. Compared to traditional maintenance decisions based solely on deterministic predictions, the proposed method significantly reduces the risks of over-maintenance or deferred maintenance.
Figure 14 illustrates the predicted RUL and actual RUL of each aero-engine unit in the testing datasets for FD001 and FD003. To facilitate the analysis of prediction performance, the testing units of FD001 and FD003 are sorted according to the actual RUL values from smallest to largest. Moreover, the X-axis represents the testing unit with increasing RUL, and the Y-axis indicates the predicted RUL using the proposed method. In Figure 14, the red line refers to the predicted RUL, the blue line denotes the actual RUL, and the pink region represents the CIs of the predicted RUL. It can be concluded from Figure 14 that the proposed method behaves satisfactorily on FD001 and FD003, and the constructed 95% CIs incorporate the vast majority of the actual RUL of testing units. Furthermore, it is worth noting that the RUL prediction results tend to be more accurate when the actual RUL is smaller. As the actual RUL increases, the prediction error gradually increases. The reason for this phenomenon is that when the RUL is small, its fault deterioration characteristics become progressively more obvious, making it simpler for the DL-based method to extract the features that are more relevant to the degeneration process, thus enabling more accurate RUL prediction. In summary, the proposed method achieves accurate RUL prediction and efficiently quantifies the RUL prediction uncertainty.
In addition, the other two state-of-the-art methods are employed to further validate the proposed method. As listed in Table 6 and Figure 15, the proposed method displays outstanding RUL prediction performance compared to the other two state-of-the-art methods under four engine units. To be specific, the values of the proposed method for the three indicators on FD001 are lower than the values obtained by the other two methods. For FD003, the proposed method is sub-optimal but closer to the optimal method.

4. Conclusions

In this paper, a dual-channel framework that combines the temporal convolutional network and Transformer network is proposed. Firstly, data normalization is performed on raw condition monitoring signals to keep them fluctuating within the same range, and then, feature screening is employed to identify the features that can characterize the degradation process of aero-engines. Secondly, the selected features are sent into the dual-channel framework for deep feature mining. This dual-channel framework can efficiently extract the deterioration features for more accurate RUL prediction. Moreover, to quantify the epistemic uncertainty, the Monte Carlo dropout is introduced into the framework to realize the Bayesian approximation and uncertainty quantification. Finally, the effectiveness and superiority of the proposed dual-channel framework are validated on the C-MAPSS aero-engine dataset. The experimental results illustrate that the proposed method is capable of achieving more accurate RUL prediction as well as uncertainty quantification compared to other state-of-the-art methods. In future work, transfer learning will be introduced into the framework to realize the cross-working condition or cross-machine RUL prediction.

Author Contributions

Conceptualization, R.W. and Y.Z.; methodology, Z.Y.; software, R.W.; validation, R.W., Y.Z. and Z.Y.; formal analysis, F.L.; investigation, Z.Y.; resources, H.L.; data curation, L.L.; writing—original draft preparation, R.W.; writing—review and editing, R.W. and C.H.; visualization, R.W.; supervision, R.W.; project administration, J.G.; funding acquisition, Y.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Yibin Science and Technology Bureau Project, grant number 2022SF002; Natural Science Project of Yibin Vocational and Technical College, grant number ZR22YB-10; and Natural Science Foundation of Sichuan, China, grant number 2023NSFSC856.

Data Availability Statement

The data used in the manuscript comes from https://www.nasa.gov/intelligent-systems-division/discovery-and-systems-health/pcoe/pcoe-data-set-repository/ (accessed on 16 December 2024). Data Set Citation: Saxena, A.; Goebel, K.; Simon, D.; Eklund, N. Damage propagation modeling for aircraft engine run-to-failure simulation. In Proceedings of the 2008 International Conference on Prognostics and Health Management, Denver, CO, USA, 6–9 October 2008; IEEE: Piscataway, NJ, USA, 2008; pp. 1–9.

Conflicts of Interest

Author Chen Hu has received research grants from China Oil & Gas Pipeline Network Corporation Central China Branch. Author Zhengquan Yang has received research grants from Sichuan Yibin Minjiang Machinery Manufacturing Co., Ltd. The China Oil & Gas Pipeline Network Corporation Central China Branch and Sichuan Yibin Minjiang Machinery Manufacturing Co., Ltd. had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Li, F.; Wu, H.; Liu, L.; Ye, Y.; Wang, Y.; Wu, P. Nonlinear optimal frequency control for dynamic vibration absorber and its application. Mech. Syst. Sig. Process. 2025, 223, 111932. [Google Scholar] [CrossRef]
  2. Wu, H.; Wei, J.; Wu, P.; Zhang, F.; Liu, Y. Dynamic response analysis of high-speed train gearboxes excited by wheel out-of-round: Experiment and simulation. Veh. Syst. Dyn. 2024, 1–27. [Google Scholar] [CrossRef]
  3. Wang, X.; Liu, X.; Yang, H.; Wang, Z.; Wen, X.; He, X.; Qing, L.; Chen, H. Degradation Modeling for Restoration-enhanced Object Detection in Adverse Weather Scenes. IEEE Trans. Intell. Veh. 2024, 1–17. [Google Scholar] [CrossRef]
  4. Zhao, Y.; Teng, Q.; Chen, H.; Zhang, S.; He, X.; Li, Y.; Sheriff, R.E. Activating more information in arbitrary-scale image super-resolution. IEEE Trans. Multimed. 2024, 26, 7946–7961. [Google Scholar] [CrossRef]
  5. Wang, X.; Chen, H.; Gou, H.; He, J.; Wang, Z.; He, X.; Qing, L.; Sheriff, R.E. RestorNet: An efficient network for multiple degradation image restoration. Knowl.-Based Syst. 2023, 282, 111116. [Google Scholar] [CrossRef]
  6. Li, Y.; Sun, L.; Geng, J.; Zhao, X. Semi-analytical investigation on hydrodynamic efficiency and loading of perforated breakwater-integrated OWCs. Ocean Eng. 2024, 309, 118460. [Google Scholar] [CrossRef]
  7. Shao, Z.; Yin, Y.; Lyu, H.; Soares, C.G. A robust method for multi object tracking in autonomous ship navigation systems. Ocean Eng. 2024, 311, 118560. [Google Scholar] [CrossRef]
  8. Guo, J.; Li, D.; Du, B. A stacked ensemble method based on TCN and convolutional bi-directional GRU with multiple time windows for remaining useful life estimation. Appl. Soft Comput. 2024, 150, 111071. [Google Scholar] [CrossRef]
  9. Xia, M.; Zheng, X.; Imran, M.; Shoaib, M. Data-driven prognosis method using hybrid deep recurrent neural network. Appl. Soft Comput. 2020, 93, 106351. [Google Scholar] [CrossRef]
  10. Wang, L.; Guo, W.; Guo, J.; Zheng, S.; Wang, Z.; Kang, H.S.; Li, H. An Integrated Deep Learning Model for Intelligent Recognition of Long-distance Natural Gas Pipeline Features. Reliab. Eng. Syst. Saf. 2024, 255, 110664. [Google Scholar] [CrossRef]
  11. Lin, X.; Liu, X.; Yang, H.; He, X.; Chen, H. Perception-and Fidelity-aware Reduced-Reference Super-Resolution Image Quality Assessment. IEEE Trans. Broadcast. 2024. [Google Scholar] [CrossRef]
  12. Guo, J.; Yang, Y.; Li, H.; Wang, J.; Tang, A.; Shan, D.; Huang, B. A hybrid deep learning model towards fault diagnosis of drilling pump. Appl. Energy 2024, 372, 123773. [Google Scholar] [CrossRef]
  13. Huang, C.G.; Li, H.; Peng, W.; Tang, L.C.; Ye, Z.S. Personalized Federated Transfer Learning for Cycle-Life Prediction of Lithium-Ion Batteries in Heterogeneous Clients with Data Privacy Protection. IEEE Internet Things J. 2024, 11, 36895–36906. [Google Scholar] [CrossRef]
  14. Hu, G.; Deng, J.; Wang, G.; Tang, K.; Ren, G.; Wen, X. Experimental and Simulation Study on Fatigue Damage Characteristics of HNBR by HTHP Aging in Oil-Based Mud Environment. In Fatigue & Fracture of Engineering Materials & Structures; John Wiley & Sons Ltd: Hoboken, NJ, USA, 2025. [Google Scholar] [CrossRef]
  15. Liu, H.; Sun, Y.; Ding, W.; Wu, H.; Zhang, H. Enhancing non-stationary feature learning for remaining useful life prediction of aero-engine under multiple operating conditions. Measurement 2024, 227, 114242. [Google Scholar] [CrossRef]
  16. Huang, C.; Bu, S.; Lee, H.H.; Chan, C.H.; Kong, S.W.; Yung, W.K. Prognostics and health management for predictive maintenance: A review. J. Manuf. Syst. 2024, 75, 78–101. [Google Scholar] [CrossRef]
  17. Li, W.; Shang, Z.; Gao, M.; Qian, S.; Feng, Z. Remaining useful life prediction based on transfer multi-stage shrinkage attention temporal convolutional network under variable working conditions. Reliab. Eng. Syst. Saf. 2022, 226, 108722. [Google Scholar] [CrossRef]
  18. Guo, J.; Wang, Z.; Yang, Y.; Song, Y.; Wan, J.L.; Huang, C.G. A dual-channel transferable RUL prediction method integrated with Bayesian deep learning and domain adaptation for rolling bearings. Qual. Reliab. Eng. Int. 2024, 40, 2348–2366. [Google Scholar] [CrossRef]
  19. Lei, Y.; Li, N.; Guo, L.; Li, N.; Yan, T.; Lin, J. Machinery health prognostics: A systematic review from data acquisition to RUL prediction. Mech. Syst. Sig. Process. 2018, 104, 799–834. [Google Scholar] [CrossRef]
  20. Shao, Z.; Yin, Y.; Lyu, H.; Soares, C.G.; Cheng, T.; Jing, Q.; Yang, Z. An efficient model for small object detection in the maritime environment. Appl. Ocean Res. 2024, 152, 104194. [Google Scholar] [CrossRef]
  21. Guo, J.; Yang, Y.; Li, H.; Dai, L.; Huang, B. A parallel deep neural network for intelligent fault diagnosis of drilling pumps. Eng. Appl. Artif. Intell. 2024, 133, 108071. [Google Scholar] [CrossRef]
  22. Huang, C.G.; Huang, H.Z.; Li, Y.F.; Peng, W. A novel deep convolutional neural network-bootstrap integrated method for RUL prediction of rolling bearing. J. Manuf. Syst. 2021, 61, 757–772. [Google Scholar] [CrossRef]
  23. Wang, Y.; Deng, L.; Zheng, L.; Gao, R.X. Temporal convolutional network with soft thresholding and attention mechanism for machinery prognostics. J. Manuf. Syst. 2021, 60, 512–526. [Google Scholar] [CrossRef]
  24. Guo, J.; Zan, X.; Wang, L.; Lei, L.; Ou, C.; Bai, S. A random forest regression with Bayesian optimization-based method for fatigue strength prediction of ferrous alloys. Eng. Fract. Mech. 2023, 293, 109714. [Google Scholar] [CrossRef]
  25. Soualhi, M.; Nguyen, K.T.P.; Medjaher, K. Explainable RUL estimation of turbofan engines based on prognostic indicators and heterogeneous ensemble machine learning predictors. Eng. Appl. Artif. Intell. 2024, 133, 108186. [Google Scholar] [CrossRef]
  26. Peng, K.; Jiao, R.; Dong, J.; Pi, Y. A deep belief network based health indicator construction and remaining useful life prediction using improved particle filter. Neurocomputing 2019, 361, 19–28. [Google Scholar] [CrossRef]
  27. Wang, T.; Li, B.; Fei, Q.; Xu, S.; Ma, Z. Parallel processing of sensor signals using deep learning method for aero-engine remaining useful life prediction. Meas. Sci. Technol. 2024, 35, 096129. [Google Scholar] [CrossRef]
  28. Lu, X.; Pan, H.; Zhang, L.; Ma, L.; Wan, H. A dual path hybrid neural network framework for remaining useful life prediction of aero-engine. Qual. Reliab. Eng. Int. 2024, 40, 1795–1810. [Google Scholar] [CrossRef]
  29. Liu, L.; Song, X.; Zhou, Z. Aircraft engine remaining useful life estimation via a double attention-based data-driven architecture. Reliab. Eng. Syst. Saf. 2022, 221, 108330. [Google Scholar] [CrossRef]
  30. Jiang, L.; Zhang, T.; Lei, W.; Zhuang, K.; Li, Y. A new convolutional dual-channel Transformer network with time window concatenation for remaining useful life prediction of rolling bearings. Adv. Eng. Inf. 2023, 56, 101966. [Google Scholar] [CrossRef]
  31. Lin, L.; Wu, J.; Fu, S.; Zhang, S.; Tong, C.; Zu, L. Channel attention & temporal attention based temporal convolutional network: A dual attention framework for remaining useful life prediction of the aircraft engines. Adv. Eng. Inf. 2024, 60, 102372. [Google Scholar]
  32. Chen, F.; Yu, Y.; Li, Y. An aero–engine remaining useful life prediction model based on clustering analysis and the improved GRU–TCN. Meas. Sci. Technol. 2024, 36, 016001. [Google Scholar] [CrossRef]
  33. Guo, J.; Wang, Z.; Li, H.; Yang, Y.; Huang, C.G.; Yazdi, M.; Kang, H.S. A hybrid prognosis scheme for rolling bearings based on a novel health indicator and nonlinear Wiener process. Reliab. Eng. Syst. Saf. 2024, 245, 110014. [Google Scholar] [CrossRef]
  34. Saxena, A.; Goebel, K.; Simon, D.; Eklund, N. Damage propagation modeling for aircraft engine run-to-failure simulation. In Proceedings of the 2008 International Conference on Prognostics and Health Management, Denver, CO, USA, 6–9 October 2008; IEEE: Piscataway, NJ, USA, 2008; pp. 1–9. [Google Scholar]
  35. Wen, L.; Su, S.; Wang, B.; Ge, J.; Gao, L.; Lin, K. A new multi-sensor fusion with hybrid Convolutional Neural Network with Wiener model for remaining useful life estimation. Eng. Appl. Artif. Intell. 2023, 126, 106934. [Google Scholar] [CrossRef]
  36. Zhao, C.; Huang, X.; Li, Y.; Li, S. A novel remaining useful life prediction method based on gated attention mechanism capsule neural network. Measurement 2022, 189, 110637. [Google Scholar] [CrossRef]
Processes 12 02925 g001
Figure 1. The flowchart of the proposed method.
Figure 1. The flowchart of the proposed method.
Processes 12 02925 g001
Processes 12 02925 g002
Figure 2. The structure of the TCN model.
Figure 2. The structure of the TCN model.
Processes 12 02925 g002
Processes 12 02925 g003
Figure 3. The visualization of dilated causal convolution.
Figure 3. The visualization of dilated causal convolution.
Processes 12 02925 g003
Processes 12 02925 g004
Figure 4. The architecture of the Transformer.
Figure 4. The architecture of the Transformer.
Processes 12 02925 g004
Processes 12 02925 g005
Figure 5. The principle of multi-head self-attention.
Figure 5. The principle of multi-head self-attention.
Processes 12 02925 g005
Processes 12 02925 g006
Figure 6. The comparison between the deterministic model and BNN-like model.
Figure 6. The comparison between the deterministic model and BNN-like model.
Processes 12 02925 g006
Processes 12 02925 g007
Figure 7. The simplified diagram of C-MAPSS.
Figure 7. The simplified diagram of C-MAPSS.
Processes 12 02925 g007
Processes 12 02925 g008
Figure 8. The visualization of selected normalized sensor measurements.
Figure 8. The visualization of selected normalized sensor measurements.
Processes 12 02925 g008
Processes 12 02925 g009
Figure 9. The schematic of sliding window processing.
Figure 9. The schematic of sliding window processing.
Processes 12 02925 g009
Processes 12 02925 g010
Figure 10. The rectified piece-wise RUL label.
Figure 10. The rectified piece-wise RUL label.
Processes 12 02925 g010
Processes 12 02925 g011
Figure 11. The performance comparison with other hyperparameters.
Figure 11. The performance comparison with other hyperparameters.
Processes 12 02925 g011
Processes 12 02925 g012
Figure 12. The visualization of comparison results with other methods.
Figure 12. The visualization of comparison results with other methods.
Processes 12 02925 g012
Processes 12 02925 g013
Figure 13. The prediction results for the testing engines.
Figure 13. The prediction results for the testing engines.
Processes 12 02925 g013
Processes 12 02925 g014
Figure 14. The visualization of the prediction results on testing datasets.
Figure 14. The visualization of the prediction results on testing datasets.
Processes 12 02925 g014
Processes 12 02925 g015
Figure 15. The visualization of prediction performance comparison with different methods.
Figure 15. The visualization of prediction performance comparison with different methods.
Processes 12 02925 g015
Table 1. Applied studies of relevant literature.
Table 1. Applied studies of relevant literature.
LiteratureModelAdvantagesResearch Gaps
Wang et al. [27]Improved Inception module with gated recurrent unitReduces feature omission problem; high feature extraction capacityLack of uncertainty quantification
Lu et al. [28]A multi-scale CNN and bidirectional GRU with temporal attentionHigh prediction efficiency; high feature extraction capacityLack of uncertainty quantification
Liu et al. [29]A dual attention-based CNN and TransformerLittle prior knowledge; integration of channel and temporal informationLack of uncertainty quantification
Jiang et al. [30]A convolutional dual-channel TransformerLow amounts of trainable parameters; extraction of local features from different domainsLack of uncertainty quantification
Lin et al. [31]A double attention TCN frameworkIncrease in the attention to key time points; high feature extraction capacityLack of uncertainty quantification; parallel channel
Chen et al. [32]An improved GRU-TCN modelExtraction of deeper degradation features; incorporation of the recognition resultsLack of uncertainty quantification; parallel channel
Table 2. The description of the C-MAPSS dataset.
Table 2. The description of the C-MAPSS dataset.
DatasetFD001FD002FD003FD004
Number of training engines100260100249
Number of test engines100259100248
Operating conditions1616
Fault modes1122
Maximum cycles362378525543
Minimum cycles128128145128
Table 3. The classification results of sensor signals.
Table 3. The classification results of sensor signals.
CategoriesSensor Number
Increasing trend2, 3, 4, 8, 9, 11, 13, 14, 15, 17
Decreasing trend7, 12, 20, 21
Constant trend1, 5, 6, 10, 16, 18, 19
Table 4. The configuration of the model hyperparameters.
Table 4. The configuration of the model hyperparameters.
ParameterValue
TCN block 1Filters = 8, kernel size = 4, dilation rate = 1
TCN block 2Filters = 6, kernel size = 4, dilation rate = 2
TCN block 3Filters = 4, kernel size = 4, dilation rate = 4
d_model14
Heads2
Encoder layer2
Dropout0.2
Learning rate0.001
Batch size128
OptimizerAdam
Epoch50
Window size30
Table 5. The comparison results with other methods.
Table 5. The comparison results with other methods.
FD001FD003
RMSEScoreRMSEScore
CNN-Transformer14.90450.8013.55475.37
Transformer15.44445.8326.464637.81
TCN-BiGRU13.76357.8314.07360.02
Proposed method13.64245.4213.50347.78
Table 6. The prediction performance comparison with different methods.
Table 6. The prediction performance comparison with different methods.
GAM-Capsnet [36]AM-Capsnet [36]The Proposed Method
DatasetNO.RMSEScore R 2 RMSEScore R 2 RMSEScore R 2
FD001#3412.015230.9119.2515660.769.832130.93
#8110.052200.9317.047170.8110.522480.92
FD003#34.5575.790.9716.329180.7310.513050.93
#8213.084600.9721.2520090.7110.663130.93
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Wang, R.; Zhang, Y.; Hu, C.; Yang, Z.; Li, H.; Liu, F.; Li, L.; Guo, J. A Parallel Prognostic Method Integrating Uncertainty Quantification for Probabilistic Remaining Useful Life Prediction of Aero-Engine. Processes 2024, 12, 2925. https://doi.org/10.3390/pr12122925

AMA Style

Wang R, Zhang Y, Hu C, Yang Z, Li H, Liu F, Li L, Guo J. A Parallel Prognostic Method Integrating Uncertainty Quantification for Probabilistic Remaining Useful Life Prediction of Aero-Engine. Processes. 2024; 12(12):2925. https://doi.org/10.3390/pr12122925

Chicago/Turabian Style

Wang, Rongqiu, Ya Zhang, Chen Hu, Zhengquan Yang, Huchang Li, Fuqi Liu, Linling Li, and Junyu Guo. 2024. "A Parallel Prognostic Method Integrating Uncertainty Quantification for Probabilistic Remaining Useful Life Prediction of Aero-Engine" Processes 12, no. 12: 2925. https://doi.org/10.3390/pr12122925

APA Style

Wang, R., Zhang, Y., Hu, C., Yang, Z., Li, H., Liu, F., Li, L., & Guo, J. (2024). A Parallel Prognostic Method Integrating Uncertainty Quantification for Probabilistic Remaining Useful Life Prediction of Aero-Engine. Processes, 12(12), 2925. https://doi.org/10.3390/pr12122925

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop