基于双重自注意力机制和长短时记忆网络的剩余寿命预测

doi:10.12305/j.issn.1001-506X.2024.06.16

摘要/Abstract

摘要：

剩余使用寿命(remaining useful life, RUL)预测是产品故障预测与健康管理的重要内容。传统长短期记忆(long short-term memory, LSTM) 网络无法主动选择关键特征、难以高效提取大数据所蕴含的退化信息。提出一种基于改进LSTM网络的RUL预测方法, 采用随机森林(random forest, RF) 算法筛选输入特征, 以主动选取关键特征; 采用双重自注意力机制分别从特征维度和时间维度完成权重自适应分配, 使模型在学习过程中关注主要特征和历史时间点; 通过融合统计特征, 以提高RUL预测精度。以航空发动机数据集为例完成案例分析, 验证方法有效性。结果表明, 所提方法能有效提高基于复杂数据集的RUL预测精度。

关键词: 剩余寿命预测, 随机森林, 双重自注意力机制, 长短期记忆网络, 航空发动机

Abstract:

Prediction of remaining useful life (RUL) is an important part of fault prognostic and health management. Traditional long short-term memory (LSTM) network cannot select the key features actively, and it is difficult to effectively extract the degradation information contained in big data. This paper proposes an RUL prediction approach based on an improved LSTM network, where the random forest (RF) algorithm is adopted to filter the input features in order to select key features actively. A double self-attention mechanism is used to complete the adaptive weight assignment from feature dimension and the time dimension. Thus, the proposed approach can focus on the key features and historical time during the learning process. By fusing the statistical features, the model can improve the accuracy of RUL prediction. To illustrate the effectiveness of the proposed method, a case study is conducted with a data set of aircraft engine. The results indicate that the proposed method can effectively improve the accuracy of RUL prediction with complicated data sets.

Key words: remaining useful life (RUL) prediction, random forest (RF), double self-attention mechanism, long short-term memory (LSTM) network, aircraft engine

中图分类号:

TH17

吴嘉俊, 苏春, 张玉茹. 基于双重自注意力机制和长短时记忆网络的剩余寿命预测[J]. 系统工程与电子技术, 2024, 46(6): 1986-1994.

Jiajun WU, Chun SU, Yuru ZHANG. Remaining useful life prediction based on double self-attention mechanism and long short-term memory network[J]. Systems Engineering and Electronics, 2024, 46(6): 1986-1994.

图/表 16

图1

图2

图3

图4

表1

图5

图6

表2

表3

图7

表4

图8

图9

表5

图10

图11

参考文献 33

1	OMSHI E M , GRALL A . Replacement and imperfect repair of deteriorating system: study of a CBM policy and impact of repair efficiency[J]. Reliability Engineering & System Safety, 2021, 215, 107905.
2	LEI Y G , LI N P , GUO L , et al. Machinery health prognostics: a systematic review from data acquisition to RUL prediction[J]. Mechanical Systems and Signal Processing, 2018, 104, 799- 834. doi: 10.1016/j.ymssp.2017.11.016
3	NIETO P J G , GARCIA-GONZALO E , LASHERAS F S , et al. Hybrid PSO-SVM-based method for forecasting of the remaining useful life for aircraft engines and evaluation of its reliability[J]. Reliability Engineering & System Safety, 2015, 138, 219- 231.
4	SCHONLAU M , ZOU R Y . The random forest algorithm for statistical learning[J]. The Stata Journal, 2020, 20 (1): 3- 29. doi: 10.1177/1536867X20909688
5	LIN M Q , YOU Y Q , WANG W , et al. Battery health prognosis with gated recurrent unit neural networks and hidden Markov model considering uncertainty quantification[J]. Reliability Engineering & System Safety, 2023, 230, 108978.
6	XU Z Y , SALEH J H . Machine learning for reliability engineering and safety applications: review of current status and future opportunities[J]. Reliability Engineering & System Safety, 2021, 211, 107530.
7	LI X , DING Q , SUN J Q . Remaining useful life estimation in prognostics using deep convolution neural networks[J]. Reliability Engineering & System Safety, 2018, 172, 1- 11.
8	ZHU J , CHEN N , PENG W W . Estimation of bearing remaining useful life based on multiscale convolutional neural network[J]. IEEE Trans.on Industrial Electronics, 2019, 66 (4): 3208- 3216. doi: 10.1109/TIE.2018.2844856
9	SHANG Z W , ZHANG B R , LI W X , et al. Machine remaining life prediction based on multi-layer self-attention and temporal convolution network[J]. Complex & Intelligent Systems, 2022, 8 (2): 1409- 1424.
10	ZHANG J J , WANG P , YAN R Q , et al. Long short-term memory for machine remaining life prediction[J]. Journal of Manufacturing Systems, 2018, 48, 78- 86. doi: 10.1016/j.jmsy.2018.05.011
11	SU C , LI L , WEN Z J . Remaining useful life prediction via a variational autoencoder and a time-window-based sequence neural network[J]. Quality and Reliability Engineering International, 2020, 36 (5): 1639- 1656. doi: 10.1002/qre.2651
12	李京峰, 陈云翔, 项华春, 等. 基于LSTM-DBN的航空发动机剩余寿命预测[J]. 系统工程与电子技术, 2020, 42 (7): 1637- 1644.
	LI J F , CHEN Y X , XIANG H C , et al. Remaining useful life prediction for aero-engine based on the similarity of degradation characteristics[J]. Systems Engineering and Electronics, 2020, 42 (7): 1637- 1644.
13	CHEN Z H , WU M , ZHAO R , et al. Machine remaining useful life prediction via an attention-based deep learning approach[J]. IEEE Trans.on Industrial Electronics, 2021, 68 (3): 2521- 2531. doi: 10.1109/TIE.2020.2972443
14	ZHANG Y , XIN Y Q , LIU Z W , et al. Health status assessment and remaining useful life prediction of aero-engine based on BiGRU and MMoE[J]. Reliability Engineering & System Safety, 2022, 220, 108263.
15	闫啸家, 梁伟阁, 张钢, 等. 基于RCNN-ABiLSTM的机械设备剩余寿命预测方法[J]. 系统工程与电子技术, 2023, 45 (3): 931- 940.
	YAN X J , LIANG W G , ZHANG G , et al. Prediction method for mechanical equipment based on RCNN-ABiLSTM[J]. Systems Engineering and Electronics, 2023, 45 (3): 931- 940.
16	MA X L , TAO Z M , WANG Y H , et al. Long short-term memory neural for traffic speed prediction using remote microwave sensor data[J]. Transportation Research Part C: Emerging Technologies, 2015, 54, 187- 197. doi: 10.1016/j.trc.2015.03.014
17	WANG H , PENG M J , MIAO Z , et al. Remaining useful life prediction techniques for electric valves based on convolution auto encoder and long short term memory[J]. ISA Transactions, 2021, 108, 333- 342. doi: 10.1016/j.isatra.2020.08.031
18	GU L D , ZHENG R , ZHOU Y F , et al. Remaining useful life prediction using composite health index and hybrid LSTM-SVR model[J]. Quality and Reliability Engineering International, 2022, 38 (7): 3559- 3578. doi: 10.1002/qre.3151
19	DU W B , WANG Y L , QIAO Y . Recurrent spatial-temporal attention network for action recognition in videos[J]. IEEE Trans.on Image Processing, 2018, 27 (3): 1347- 1360. doi: 10.1109/TIP.2017.2778563
20	PAN T Y , CHEN J L , YE Z S , et al. A multi-head attention network with adaptive meta-transfer learning for RUL prediction of rocket engines[J]. Reliability Engineering & System Safety, 2022, 225, 108610.
21	LIU L , SONG X , ZHOU Z T . Aircraft engine remaining useful life estimation via a double attention-based data-driven architecture[J]. Reliability Engineering & System Safety, 2022, 221, 108330.
22	KHELIF R , CHEBEL-MORELLO B , MALINOWSKI S , et al. Direct remaining useful life estimation based on support vector regression[J]. IEEE Trans.on Industrial Electronics, 2017, 64 (3): 2276- 2285. doi: 10.1109/TIE.2016.2623260
23	汪千程, 苏春, 文泽军. 基于协整分析的风力机多工况监测与故障诊断[J]. 中国机械工程, 2022, 33 (13): 1596- 1603. doi: 10.3969/j.issn.1004-132X.2022.13.010
	WANG Q C , SU C , WEN Z J . Multi-condition monitoring and fault diagnosis of wind turbines based on cointegration analysis[J]. China Mechanical Engineering, 2022, 33 (13): 1596- 1603. doi: 10.3969/j.issn.1004-132X.2022.13.010
24	张紫嫣, 卢文科, 左锋, 等. 基于分段线性插值法的应变片式扭矩传感器的温度补偿[J]. 电子测量技术, 2022, 45 (8): 143- 147.
	ZHANG Z Y , LU W K , ZUO F , et al. Temperature compensation of strain gauge torque sensor based on piecewise linear interpolation method[J]. Electronic Measurement Technology, 2022, 45 (8): 143- 147.
25	CHENG Y J , ZENG J Y , WANG Z L , et al. A health state-related ensemble deep learning method for aircraft engine remaining useful life prediction[J]. Applied Soft Computing, 2023, 135, 110041. doi: 10.1016/j.asoc.2023.110041
26	GONG R , LI J X , WANG C L . Remaining useful life prediction based on multisensor fusion and attention TCN-BiGRU model[J]. IEEE Sensors Journal, 2022, 22 (21): 21101- 21110. doi: 10.1109/JSEN.2022.3208753
27	SAXENA A, GOEBEL K, SIMON D, et al. Damage propagation modeling for aircraft engine run-to-failure simulation[C]//Proc. of the International Conference on Prognostics and Health Management, 2008: 1-9.
28	LI H , ZHAO W , ZHANG Y X , et al. Remaining useful life prediction using multi-scale deep convolutional neural network[J]. Applied Soft Computing, 2020, 89, 106113. doi: 10.1016/j.asoc.2020.106113
29	ZHANG C , LIM P , QIN A K , et al. Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics[J]. IEEE Trans.on Neural Networks and Learning Systems, 2017, 28 (10): 2306- 2318. doi: 10.1109/TNNLS.2016.2582798
30	XIA M , ZHENG X , IMRAN M , et al. Data-driven prognosis method using hybrid deep recurrent neural network[J]. Applied Soft Computing, 2020, 93, 106351. doi: 10.1016/j.asoc.2020.106351
31	REMADNA I, TERRISSA S L, ZEMOURI R, et al. Leveraging the power of the combination of CNN and bi-directional LSTM networks for aircraft engine RUL estimation[C]//Proc. of the Prognostics and Health Management Conference, 2020: 116-121.
32	XIANG S , QIN Y , LUO J , et al. Multicellular LSTM-based deep learning model for aero-engine remaining useful life prediction[J]. Reliability Engineering & System Safety, 2021, 216, 107927.
33	LI Y F , CHEN Y F , SHAO H N , et al. A novel dual attention mechanism combined with knowledge for remaining useful life prediction based on gated recurrent units[J]. Reliability Engineering & System Safety, 2023, 239, 109514.

指标数量	子数据集
指标数量	FD001	FD002	FD003	FD004
工作条件数量	1	6	1	6
故障模式数量	1	1	2	2
训练集数量	100	260	100	249
测试集数量	100	259	100	248

子数据集	选取的传感器特征编号
FD001	11, 9, 4, 12, 7, 14, 15, 21, 2, 3, 20, 13, 8, 17, 6
FD002	13, 11, 15, 4, 14, 9, 8, 2, 12, 3, 7, 21, 20, 17, 6, 16
FD003	11, 9, 4, 12, 7, 14, 15, 21, 2, 3, 20, 13, 8, 17, 6
FD004	13, 11, 15, 4, 14, 9, 8, 2, 12, 3, 7, 21, 20, 17, 6, 16

参数	取值
决策树个数	200
随机失活率	0.2
批量大小	128
迭代轮次	50
LSTM隐藏层单元数	50
第1层全连接层单元数	50
第2层全连接层单元数	10
第3层全连接层单元数	10

方法	FD001				FD004
方法	RMSE	STD	Score	STD	RMSE	STD	Score	STD
LSTM	13.21	0.51	306.06	54.58	19.46	1.05	3 786.78	1 691.87
RF+LSTM	13.20	0.33	306.42	45.93	19.14	0.56	3 658.44	1 498.73
RF+Feats+LSTM	12.52	0.57	253.09	26.14	18.81	0.45	2 381.83	468.51
本文方法	12.357	0.31	269.28	22.76	18.27	0.44	1 793.52	171.05

方法	年份	FD001		FD002		FD003		FD004
方法	年份	RMSE	Score	RMSE	Score	RMSE	Score	RMSE	Score
MLP^[29]	2017	16.78	561	28.78	14 027	18.47	480	30.96	10 444
DCNN^[7]	2018	12.61	273	22.36	10 412	12.64	284	23.31	12 466
RNN^[7]	2018	13.44	339	24.03	14 245	13.36	316	24.02	13 931
DLSTM^[30]	2020	14.57	—	23.20	—	14.92	—	28.72	—
VAE+LSTM^[11]	2020	15.88	322	25.78	4 990	14.29	309	23.93	4 720
CNN+Bi-LSTM^[31]	2020	10.74	—	15.20	—	13.85	—	18.60	—
MCLSTM^[32]	2021	13.71	315	—	—	—	—	23.81	4 826
Attention+LSTM^[13]	2021	14.54	322	—	—	—	—	27.08	5 649
Multi-attention+TCN^[9]	2022	13.25	235	19.57	1 655	13.43	239	21.69	2 415
Double attention-based architecture^[21]	2022	12.25	198	17.08	1 575	13.39	290	19.86	1 741
Dual attention+GRU^[33]	2023	11.77	—	16.09	—	11.66	—	20.10	—
MMoE+BiGRU^[14]	2022	13.22	—	18.26	—	13.79	—	18.38	—
RCNN+ABi-LSTM^[15]	2023	12.98	258	19.16	2 980	13.24	246	22.29	3 795
本文方法	2023	12.35	269	15.13	924	13.24	487	18.27	1 794