基于改进稀疏KELM的在线非平稳动态系统状态预测方法

doi:10.3969/j.issn.1001-506X.2020.09.18

摘要/Abstract

摘要：

本文针对基于核的增量超限学习机(kernel based incremental extreme learning machine，KB-IELM)对非平稳动态系统的时变状态跟踪能力不足的问题，提出一种新型的状态预测方法。通过融合遗忘因子和自适应时变正则化因子构建新的目标函数。通过最小化字典的快速留一交叉验证(fast leave-one-out cross-validation, FLOO-CV)误差，选择具有预定规模的关键节点以构成字典。通过融合遗忘因子，为字典中各关键节点按时间顺序分配不同权重。基于FLOO-CV原则，使用天牛须搜索(beetle antennae search，BAS)算法为不同的非线性区域赋予不同的正则化参数。通过矩阵初等变换和分块求逆，实现核权重向量的在线递推更新。将模型应用于非平稳Mackey-Glass混沌时间序列预测和某型飞机发动机的状态预测。所提算法相比于最新的非平稳在线序列核超限学习机(non-stationary online sequential kernel extreme learning machine，NOS-KELM)和融合自适应正则化因子的在线稀疏核超限学习机(online sparse kernel extreme learning machine with adaptive regulation factor, OSKELM-ARF)两种方法，其训练精度分别提升了66.67%、50.72%、预测精度提升了67.02%、56.34%，最大预测误差减少了67.27%、51.09%，平均相对误差率分别减少了67.18%、59.62%。实验证明所提算法有效提升了在线预测的精度。

关键词: 超限学习机, 遗忘因子, 自适应时变正则化因子, 状态预测

Abstract:

A new state prediction method for the problem is proposed that the kernel based incremental extreme learning machine (KB-IELM) has insufficient ability to track the time-varying state of non-stationary dynamic systems. A new objective function is constructed by integrating the forgetting factor and the adaptive time-varing regulation factor. The key nodes with the predetermined scale are selected to form the dictionary by minimizing the fast leave-one-out cross-validation (FLOO-CV) error of the dictionary. Different weights are assigned to the key nodes in the dictionary according to the sequential order via mixing the forgetting factor. Based on the FLOO-CV principle, the different regulation parameters are assigned to different nonlinear regions by utilizing the algorithm of the beetle antennae search (BAS). The online recursive update of weight vectors is realized by matrix elementary transformation and block inversion. The model is applied to the non-stationary Mackey-Glass chaotic time series prediction and the state prediction of an aircraft engine. Compared with the latest two methods of the non-stationary online sequential kernel extreme learning machine (NOS-KELM) and the online sparse kernel extreme learning machine with adaptive regulation factor (OSKELM-ARF), the training accuracy is improved by 66.67% and 50.72 %. The prediction accuracy is improved by 67.02% and 56.34%, and the maximum prediction error is reduced by 67.27% and 51.09%. Then the average relative error rate is reduced by 67.18% and 59.62% respectively. It is proved that the proposed algorithm effectively improve the accuracy of online prediction.

Key words: extreme learning machine (ELM), forgetting factor, adaptive time-varying regulation factor, state prediction

中图分类号:

V243

刘星, 熊厚情, 赵建印, 朱敏. 基于改进稀疏KELM的在线非平稳动态系统状态预测方法[J]. 系统工程与电子技术, 2020, 42(9): 2022-2032.

Xing LIU, Houqing XIONG, Jianyin ZHAO, Min ZHU. State prediction method of online non-stationary dynamic system based on improved sparse KELM[J]. Systems Engineering and Electronics, 2020, 42(9): 2022-2032.

图/表 15

图1

图2

图3

图4

表1

图5

表2

图6

图7

图8

图9

表3

表4

图10

图11

参考文献 19

1	HUANG G B , ZHU Q Y , SIEW C K . Extreme learning machine: theory and application[J]. Neurocomputing, 2006, 70 (1): 489- 501.
2	HUANG G B, ZHU Q Y, SIEW C K. Extreme learning machine: a new learning scheme of feed forward neural networks[C]//Proc.of the IEEE International Joint Conference on Neural Networks, 2004: 985-990.
3	LIANG N Y , HUANG G B , SARATCHANDRAN P , et al. A fast and accurate online sequential learning algorithm for feed forward networks[J]. IEEE Tran.on Neural Networks, 2006, 17 (6): 1411- 1423. doi: 10.1109/TNN.2006.880583
4	HUANG G B , ZHOU H , DING X , et al. Extreme learning machine for regression and multiclass classification[J]. IEEE Tran.on Systems, Man, and Cybernetics—Part B: Cybernetics, 2011, 42 (2): 513- 529.
5	SCARDAPANE S , COMMINIELLO D , SCARPINITI M , et al. Online sequential extreme learning machine with kernels[J]. IEEE Tran.on Neural Networks and Learning Systems, 2015, 26 (9): 2214- 2220. doi: 10.1109/TNNLS.2014.2382094
6	GUO L , HAO J H , LIU M . An incremental extreme learning machine for online sequential learning problems[J]. Neurocomputing, 2014, 128 (27): 50- 58.
7	ZHAO S L , CHEN B D , ZHU P P , et al. Fixed budget quantized kernel least-mean-square algorithm[J]. Signal Processing, 2013, 93 (9): 2759- 2770. doi: 10.1016/j.sigpro.2013.02.012
8	ZHOU X R , LIU Z J , ZHU C X . Online regularized and kerne-lized extreme learning machines with forgetting mechanism[J]. Mathematical Problems in Engineering, 2014, 56 (19): 1- 11.
9	ZHOU X R , WANG C S . Cholesky factorization based online regularized and kernelized extreme learning machines with forgetting mechanism[J]. Neurocomputing, Part B, 2016, 174, 1147- 1155. doi: 10.1016/j.neucom.2015.10.033
10	GU Y , LIU J F , CHEN Y Q , et al. TOSELM: timeliness online sequential extreme learning machine[J]. Neurocomputing, 2014, 128, 119- 127. doi: 10.1016/j.neucom.2013.02.047
11	张英堂, 马超, 李志宁, 等. 基于快速留一交叉验证的核极限学习机在线建模[J]. 上海交通大学学报, 2014, 48 (5): 641- 646.
	ZHANG Y T , MA C , LI Z N , et al. Online modeling of kernel extreme learning machine based on fast leave-one-out cross-validation[J]. Journal of Shanghai Jiaotong University, 2014, 48 (5): 641- 646.
12	CEDRIC R , JOSE C M , BERMUDEZ P H . Online prediction of time series data with kernels[J]. IEEE Tran.on Signal Processing, 2009, 57 (3): 1058- 1067. doi: 10.1109/TSP.2008.2009895
13	张伟, 许爱强, 高明哲. 一种基于积累一致性测量的在线状态预测算法[J]. 上海交通大学学报, 2017, 51 (11): 1391- 1398.
	ZHANG W , XU A Q , GAO M Z . An online condition prediction algorithm based on cumulative coherence measurement[J]. Journal of Shanghai Jiaotong University, 2017, 51 (11): 1391- 1398.
14	张伟, 许爱强, 高明哲. 基于稀疏核增量超限学习机的机载设备在线状态预测[J]. 北京航空航天大学学报, 2017, 43 (10): 2089- 2098.
	ZHANG W , XU A Q , GAO M Z . Online condition prediction of avionic devices based on sparse kernel incremental extreme learning machine[J]. Journal of Beijing University of Aeronautics and Astronautics, 2017, 43 (10): 2089- 2098.
15	朱敏, 许爱强, 陈强强, 等. 一种基于改进KELM的在线状态预测方法[J]. 北京航空航天大学学报, 2019, 45 (7): 1370- 1379.
	ZHU M , XU A Q , CHEN Q Q , et al. An improved KELM based online condition prediction method[J]. Journal of Beijing University of Aeronautics and Astronautics, 2019, 45 (7): 1370- 1379.
16	ZHANG W , XU A Q , PING D F , et al. An improved kernel-based incremental extreme learning machine with fixed budget for non-stationary time series prediction[J]. Neural Computing & Applications, 2019, 31, 637- 652.
17	LIM J S , LEE S , PANG H S , et al. Low complexity adaptive forgetting factor for online sequential extreme learning machine (OS-ELM) for application to non-stationary system estimations[J]. Neural Computing & Applications, 2013, 22 (3/4): 569- 576.
18	LI M , XIAN W M , LONG B , et al. Prognostics of analog filters based on particle filters using frequency features[J]. Journal of Electronic Testing: Theory and Application, 2013, 29 (4): 567- 584. doi: 10.1007/s10836-013-5383-y
19	郭威, 徐涛, 于建江, 等. 基于M-estimator与可变遗忘因子的在线贯序超限学习机[J]. 电子与信息学报, 2018, 40 (6): 1360- 1367.
	GUO W , XU T , YU J J , et al. Online sequential extreme learning machine based on M-estimator and variable forgetting factor[J]. Journal of Electronics & Information Technology, 2018, 40 (6): 1360- 1367.

算法	正则化参数C	核参数σ	其他
ReOS-ELM	2×10³	—	L=50
ALD-KOS-ELM	1×10³	10	b=0.000 01
FOKELM	2×10⁴	10	z=50
KB-IELM	10	10	—
NOS-KELM	2×10⁴	10	m=50, η=0.8
OSKELM-ARF	2×10⁵	10	m=50, η=0.8
本文算法	—	10	m=50, δ=0.95

算法	训练		测试
算法	时间/s	RMSE	时间/s	RMSE	MAPE	MRPE
ReOS-ELM	7.011 6	0.016 6	9.00E-04	0.015 2	0.036 8	0.009 8
ALD-KOS-ELM	0.102 1	0.011 2	3.61E-04	0.010 9	0.025 0	0.008 1
FOKELM	0.081 6	0.026 9	4.32E-04	0.012 3	0.031 7	0.008 3
KB-IELM	8.121 3	0.017 5	9.80E-04	0.015 3	0.042 0	0.010 8
NOS-KELM	0.421 1	0.010 2	2.32E-04	0.009 4	0.027 5	0.006 4
OSKELM-ARF	3.668 1	0.006 9	6.96E-04	0.007 1	0.018 4	0.005 2
本文算法	1.512 1	0.003 4	2.77E-04	0.003 1	0.009 2	0.002 1

算法	正则化参数	核参数	其他
ALD-KOS-ELM	1×10³	10	m=30
NOS-KELM	2×10⁴	10	b=0.000 01
OSKELM-ARF	2×10⁴	10	m=30, η=0.8
本文算法	2×10⁴	10	m=30, δ=0.95

算法	训练	测试
算法	RMSE	RMSE	MAPE	MRPE
ALD-KOS-ELM	0.062 3	0.039 0	0.062 9	0.011 7
NOS-KELM	0.077 5	0.050 4	0.075 7	0.015 2
OSKELM-ARF	0.075 2	0.045 9	0.082 4	0.082 4
本文算法	0.010 0	0.009 1	0.040 3	0.001 7

[1]	董一平, 刘宁, 苏中, 王靖骁, 白宏阳. 基于AEKF的高速自旋飞行体组合导航方法[J]. 系统工程与电子技术, 2022, 44(6): 1977-1983.
[2]	刘星, 王文双, 赵建印, 朱敏. 自适应在线增量ELM的故障诊断模型研究[J]. 系统工程与电子技术, 2021, 43(9): 2678-2687.
[3]	朱敏, 刘奇, 刘星, 许晴. 基于LMKL和OC-ELM的航空电子部件故障检测方法[J]. 系统工程与电子技术, 2020, 42(6): 1424-1432.
[4]	李晨, 杨俊安, 刘辉. 基于信息熵和GA-ELM的调制识别算法[J]. 系统工程与电子技术, 2020, 42(1): 223-229.
[5]	张袁鹏, 郑岱堃, 李昕哲, 孙永健. 基于隐马尔可夫模型的动态规划检测前跟踪算法[J]. 系统工程与电子技术, 2019, 41(11): 2479-2487.
[6]	高明哲, 许爱强, 张伟. 自适应建模相关向量机及其在电子系统状态预测中的应用[J]. 系统工程与电子技术, 2017, 39(8): 1898-1904.
[7]	张伟, 许爱强, 平殿发. 基于稀疏化核方法的非线性动态系统在线辨识[J]. 系统工程与电子技术, 2017, 39(1): 223-230.
[8]	孙瑜, 夏鹏, 林辉. 含状态时滞的非线性间歇过程迭代学习控制研究[J]. Journal of Systems Engineering and Electronics, 2011, 33(2): 380-384.