基于GPR模型的用户量预测优化方法

doi:10.12305/j.issn.1001-506X.2024.08.19

系统工程与电子技术 ›› 2024, Vol. 46 ›› Issue (8): 2721-2729.doi: 10.12305/j.issn.1001-506X.2024.08.19

• 系统工程 • 上一篇

基于GPR模型的用户量预测优化方法

刘学浩¹^,², 刘文学¹, 杨超三¹, 祝文晶¹^,², 宋玉¹^,², 李金海¹^,²^,*

1. 中国科学院微电子研究所通信与信息工程研发中心, 北京 100029
2. 中国科学院大学集成电路学院, 北京 100049

收稿日期:2024-01-10 出版日期:2024-07-25 发布日期:2024-08-07
通讯作者: 李金海
作者简介:刘学浩 (1999—), 男, 硕士研究生, 主要研究方向为通信与卫星导航技术、信息与信号处理
刘文学 (1985—), 男, 高级工程师, 博士, 主要研究方向为卫星导航相关的软硬件设计、信号处理算法
杨超三 (1989—), 男, 助理研究员, 博士, 主要研究方向为卫星通信与自组网通信
祝文晶 (1999—), 女, 博士研究生, 主要研究方向为智能无线通信
宋玉 (1999—), 女, 博士研究生, 主要研究方向为智能无线通信、自组网通信
李金海 (1978—), 男, 高级工程师, 博士, 主要研究方向为卫星导航、宽带无线通信
基金资助:
地球观测与导航(2022YFB3903900);中国科学院“西部之光”交叉团队项目(xbzg-zdsys-202120)

Optimization method of user quantity prediction based on GPR model

Xuehao LIU¹^,², Wenxue LIU¹, Chaosan YANG¹, Wenjing ZHU¹^,², Yu SONG¹^,², Jinhai LI¹^,²^,*

1. Communication and Information Engineering Research and Development Center, Institute of Microelectronics of the Chinese Academy of Sciences, Beijing 100029, China
2. School of Integrated Circuits, University of Chinese Academy of Sciences, Beijing 100049, China

Received:2024-01-10 Online:2024-07-25 Published:2024-08-07
Contact: Jinhai LI

摘要/Abstract

摘要：

高斯过程回归(Gaussian process regression, GPR)是一种基于高斯过程的非参数化贝叶斯回归方法, 其可以灵活适应不同类型数据, 用于建模和预测数据之间的复杂关系, 具有拟合能力强、泛化能力好等特点。针对海量用户场景下用户量实时预测问题, 提出一种基于GPR的用户量预测优化方法。在滑动窗口方法处理数据的基础上, 选择合适的核函数, 基于k折交叉验证得到最佳超参数组合以实现GPR模型训练, 完成在线用户量的实时预测并进行性能评估。实验结果表明, 相比于采用训练集中输出数据方差的50%作为信号噪声估计量的传统方案, 所提方法具有较高的预测准确度, 并且在测试集均方根误差(root mean square, RMS)、平均绝对误差(mean absolute error, MAE)、平均偏差(mean bias error, MBE)和决定系数R²这4个评估指标方面均有提升, 其中MBE至少提升了43.3%。

关键词: 高斯过程回归, 用户量预测, 滑动窗口, 交叉验证, 超参数优化

Abstract:

Gaussian process regression (GPR) is a non-parametric Bayesian regression method based on Gaussian processes. It is flexible in adapting to different types of data, and it is used to model and predict complex relationships between different types of data. It has strong fitting capabilities and good generalization abilities. A user quantity prediction optimization method based on GPR is proposed to tackle the problem of real-time user quantity prediction in the context of massive user scenario. Building upon the sliding window method for data processing, the method selects a suitable kernel function and uses k-fold cross-validation to determine the optimal hyperparameter combination for training the GPR model, which enables the real-time prediction of online user quantity. Finally, the performance of the model is evaluated. The experimental results demonstrate that compared with the traditional approach that uses half of the variance of the output data in the training set as the signal noise estimator, the proposed method has higher prediction accuracy and improvements in the four following evaluation metrics of root mean square (RMS), mean absolute error (MAE), mean bias error (MBE) and determination coefficient R² on the test set. Specifically, the MBE shows an improvement of at least 43.3%.

Key words: Gaussian process regression (GPR), user quantity prediction, sliding window, cross-validation, hyperparameter optimization

中图分类号:

TP391.7

刘学浩, 刘文学, 杨超三, 祝文晶, 宋玉, 李金海. 基于GPR模型的用户量预测优化方法[J]. 系统工程与电子技术, 2024, 46(8): 2721-2729.

Xuehao LIU, Wenxue LIU, Chaosan YANG, Wenjing ZHU, Yu SONG, Jinhai LI. Optimization method of user quantity prediction based on GPR model[J]. Systems Engineering and Electronics, 2024, 46(8): 2721-2729.

图/表 21

图1

表1

表2

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

表3

图13

图14

表4

图15

图16

表5

参考文献 31

11	LIANG Y Z. An automatic detection method for epileptic EEG signals based on empirical mode decomposition and permutation entropy[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2023.
12	杨雨亭. 基于RF特征优选的ISSA-SVM变压器故障诊断方法[D]. 南京: 南京邮电大学, 2023.
	YANG Y T. ISSA-SVM transformer fault diagnosis method based on RF feature selection[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2023.
13	ZHANG S Y, TAN W A, LI Y B. Survey of kernel extreme learning machine kernel function based on the perspective of kernel parameter optimization time[C]//Proc. of the 4th Annual International Conference on Network and Information Systems for Computers, 2018: 430-433.
14	PARAL P , GHOSH S , CHATTERJEE A , et al. Automatic relevance determination kernel-embedded Gaussian process regression for sonar-based human leg localization with a mobile robot[J]. IEEE Sensors Letters, 2022, 7 (1): 6000504.
15	DONG W H , LI X F , BI D J , et al. Matern kernel adaptive filtering with Nystrom approximation for indoor localization[J]. IEEE Trans.on Instrumentation and Measurement, 2023,
16	TRONARP F, KARVONEN T, SARKKA S. Mixture representation of the Matern class with applications in state space approximations and Bayesian quadrature[C]//Proc. of the IEEE 28th International Workshop on Machine Learning for Signal Processing, 2018.
17	LIU K L , LI Y , HU X S , et al. Gaussian process regression with automatic relevance determination kernel for calendar aging prediction of lithiumion batteries[J]. IEEE Trans.on Industrial Informatics, 2019, 16 (6): 3767- 3777.
18	TAGHAVIFAR H , MARDANI A . Gaussian process with automatic relevance determination predictive model for energy management of electric direct-drive wheels: experimental validation[J]. IEEE Trans.on Vehicular Technology, 2023, 73 (2): 1910- 1917.
19	LIU T, CHAI W, WANG C C. Soft-sensors based on Gaussian process regression for wastewater treatment plants[C]//Proc. of the IEEE 11th Data Driven Control and Learning Systems Conference, 2022: 437-442.
20	杨成飞. 基于高斯过程的智能采样策略研究[D]. 合肥: 中国科学技术大学, 2019.
	YANG C F. Adaptive sampling strategy based on Gaussian process[D]. Hefei: University of Science and Technology of China, 2019.
1	王继民, 彭波. 搜索引擎用户访问量模型[J]. 计算机工程与应用, 2004, (25): 9-11, 30. doi: 10.3321/j.issn:1002-8331.2004.25.003
	WANG J M , PENG B . Modeling quantity of users' access for search engine[J]. Computer Engineering and Applications, 2004, (25): 9-11, 30. doi: 10.3321/j.issn:1002-8331.2004.25.003
2	程鹏超, 杜军平, 薛哲. 基于多路交叉的用户金融行为预测[J]. 智能系统学报, 2021, 16 (2): 378- 384.
	CHENG P C , DU J P , XUE Z . Prediction of user financial behavior based on multi-way crossing[J]. CAAI Transactions on Intelligent Systems, 2021, 16 (2): 378- 384.
3	XIAO H B, XIAO J H, DENG X W, et al. Traffic flow prediction based on traffic and meteorological data fusion in non-stationary environments[C]//Proc. of the International Conference on Electronic Information Technology and Smart Agriculture, 2021: 154-158.
4	LEE J, LEE H Y, KIM N W, et al. A study on online arima algorithms applying various gradient descent optimization algorithms for time series prediction[C]//Proc. of the International Conference on Information and Communication Technology Convergence, 2021: 1104-1106.
5	PENG Y N , XIANG W L . Short-term traffic volume prediction using GA-BP based on wavelet denoising and phase space reconstruction[J]. Physica A: Statistical Mechanics and its Applications, 2020, 549, 123913. doi: 10.1016/j.physa.2019.123913
6	LI R M , HU Y C , LIANG Q H . T2F-LSTM method for long-term traffic volume prediction[J]. IEEE Trans.on Fuzzy Systems, 2020, 28 (12): 3256- 3264. doi: 10.1109/TFUZZ.2020.2986995
7	BECKERS T. An introduction to Gaussian process models[EB/OL]. [2024-01-10]. https://arXivpreprintarXiv:2102.05497, 2021.
8	KRISHNAN V , CHANDRA K . Probability and random proce-sses[M]. Hoboken: John Wiley & Sons, 2016.
9	JAKKALA K. Deep Gaussian processes: a survey[EB/OL]. [2024-01-10]. https://arXivpreprintarXiv:2106.12135, 2021.
10	WILLIAMS C K I , RASMUSSEN C E . Gaussian processes for machine learning[M]. Cambridge: Massachusetts Institute of Technology press, 2006.
11	梁袁泽. 基于经验模态分解与排列熵的癫痫脑电信号自动检测方法[D]. 南京: 南京邮电大学, 2023.
21	ALBRECHT T , RAUSCH T M , DERRA N D . Call me maybe: methods and practical implementation of artificial intelligence in call center arrivals' forecasting[J]. Journal of Business Research, 2021, 123, 267- 278. doi: 10.1016/j.jbusres.2020.09.033
22	BERGMEIR C , COSTANTINI M , BENITEZ J M . On the usefulness of cross-validation for directional forecast evaluation[J]. Computational Statistics & Data Analysis, 2014, 76, 132- 143.
23	CERQUEIRA V , TORGO L , MOZETIC I . Evaluating time series forecasting models: an empirical study on performance estimation methods[J]. Machine Learning, 2020, 109 (11): 1997- 2028. doi: 10.1007/s10994-020-05910-7
24	ARLOT S , CELISSE A . A survey of cross-validation procedures for model selection[J]. Statistics Surveys, 2010, 4 (2010): 40- 79.
25	BERGMEIR C , HYNDMAN R J , KOO B . A note on the validity of cross-validation for evaluating autoregressive time series prediction[J]. Computational Statistics & Data Analysis, 2018, 120, 70- 83.
26	GUI C, SUN B L, SONG Y, et al. Variable length sliding window-based network coding algorithm in MANETs[C]//Proc. of the International Conference on Mobile and Ubiquitous Systems: Networking and Services, 2017.
27	MENG J F , GONG L , XU J . Sliding-window QPS (SW-QPS) a perfect parallel iterative switching algorithm for input-queued switches[J]. ACM Sigmetrics Performance Evaluation Review, 2021, 48 (3): 71- 76. doi: 10.1145/3453953.3453969
28	LI L G, ZHAO J S, QIANG B H, et al. Prediction method of fan main shaft fault state based on sliding window characteristics[C]//Proc. of the 10th International Conference on Internet Computing for Science and Engineering, 2021: 67-73.
29	VARDHAN B V S, KHEDKAR M, SURESH V. Hyper-parameter tuned short term load forecasting using stochastic classifier-regression mapping for power system operator[C]//Proc. of the IEEE PES 14th Asia-Pacific Power and Energy Engineering Conference, 2022.
30	LI Z H, SHOEMAKER C A. Hyper-parameter optimization for deep learning by surrogate-based model with weighted distance exploration[C]//Proc. of the IEEE Congress on Evolutionary Computation, 2021: 917-925.
31	SODA上海开放数据创新应用大赛. 摩拜上海城区用户使用数据[EB/OL]. [2024-01-10]. https://shanghai.sodachallenges.com/data.html.

核函数名	k(z, z′)=
常数核	φ₁²
线性核	z^Tz′
Matérn核$\check{p}=p+\frac{1}{2}, p \in \mathbf{N}$	$\begin{gathered}\varphi_1^2 \exp \left(-\frac{\sqrt{2 \check{p}}\left\\|z-z^{\prime}\right\\|}{\varphi_2}\right) \frac{p!}{(2 p)!} \\\sum\limits_{i=0}^p \frac{(p+i)!}{i!(p-i)!}\left(\frac{\sqrt{8 \check{p}}\left\\|z-z^{\prime}\right\\|}{\varphi_2}\right)^{p-i}\end{gathered}$
平方指数核	$\varphi_1^2 \exp \left(-\frac{\left\\|z-z^{\prime}\right\\|^2}{2 \varphi_2^2}\right)$
由平方指数核得到的周期核	$\varphi_1^2 \exp \left(-\frac{\sin \left(\frac{{\rm{ \mathsf{ π} }}}{p}\left\\|z-z^{\prime}\right\\|\right)^2}{2 \varphi_2^2}\right)$
平方指数自动相关性确定(automatic relevance determination, ARD)核	$\begin{gathered}\varphi_1^2 \exp \left(-\left(z-z^{\prime}\right)^{\mathrm{T}} \boldsymbol{P}^{-1}\left(z-z^{\prime}\right)\right), \\\boldsymbol{P}=\operatorname{diag}\left(\varphi_2^2, \varphi_3^2, \cdots, \varphi_{1+n_z}^2\right)\end{gathered}$

性能评估指标	表达式
RMS	$\sqrt{\frac{\sum\limits_{i=1}^n\left(\hat{x}_i-x_i\right)^2}{n}}$
MAE	$\frac{\sum\limits_{i=1}^n\left\|\hat{x}_i-x_i\right\|}{n}$
MBE	$\frac{\sum\limits_{i=1}^n \hat{x}_i-x_i}{n}$
R²	$1-\frac{\sum\limits_{i=1}^n\left(\hat{x}_i-x_i\right)^2}{\sum\limits_{i=1}^n\left(x_i-\bar{x}_i\right)^2}$

性能指标	不优化Sigma测试集评价	CV优化Sigma测试集评价	优化方法性能提升/%
RMS	20.942 7	15.584 9	25.6
MAE	12.803 2	10.007 6	21.8
MBE	-4.602 8	-2.611 9	43.3
R²	0.917 34	0.954 22	4.0

性能指标	不优化Sigma测试集评价	CV优化Sigma测试集评价	优化方法性能提升/%
RMS	22.412 4	14.593 5	34.9
MAE	13.718 9	9.375 9	31.7
MBE	-5.536	-2.040 9	63.1
R²	0.905 33	0.959 86	6.0

性能指标	不优化Sigma测试集评价	CV优化Sigma测试集评价	优化方法性能提升/%
RMS	20.089 1	15.517 3	22.8
MAE	12.687	10.066 2	20.7
MBE	-4.529 9	-2.391 8	47.2
R²	0.923 94	0.954 62	3.3

基于GPR模型的用户量预测优化方法

Optimization method of user quantity prediction based on GPR model

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 21

参考文献 31

相关文章 12

编辑推荐

Metrics

本文评价

[1]	张然, 刘天宇, 金光. 基于核自构建-高斯过程回归的锂离子电池剩余使用寿命预测[J]. 系统工程与电子技术, 2023, 45(8): 2623-2633.
[2]	王高岳, 张慧君, 陈贤, 李昊. 基于高斯过程回归的大气进入段航天器飞行能力预测方法[J]. 系统工程与电子技术, 2020, 42(10): 2334-2339.
[3]	孟阳, 高社生, 王维. 基于高斯过程回归的平方根UPF算法[J]. 系统工程与电子技术, 2015, 37(12): 2817-2822.
[4]	范庚，马登武，吴明辉，孟上. 电子系统状态时间序列预测的优化相关向量机方法[J]. 系统工程与电子技术, 2013, 35(9): 2011-2015.
[5]	张毅, 张强, 王宇, 欧阳志新. 抑制SAR图像相干斑的自适应红黑窗滤波算法[J]. Journal of Systems Engineering and Electronics, 2012, 34(8): 1576-1580.
[6]	魏喜庆，宋申民，张保群. 基于无模型无迹粒子滤波的编队卫星相对运动估计[J]. Journal of Systems Engineering and Electronics, 2012, 34(6): 1215-1219.
[7]	贺亚鹏, 庄珊娜, 张燕洪, 朱晓华. 一种基于交叉验证的稳健SL0目标参数提取算法[J]. Journal of Systems Engineering and Electronics, 2012, 34(1): 64-68.
[8]	孙林凯, 金家善, 耿俊豹. 基于加权精度的ε-SVR组合参数优化[J]. Journal of Systems Engineering and Electronics, 2011, 33(8): 1820-1823.
[9]	姚雯1，陈小前1，罗文彩1，郭健2. 基于部分交叉验证的多准则序贯近似建模方法[J]. Journal of Systems Engineering and Electronics, 2010, 32(7): 1462-1467.
[10]	李鹏,宋申民,陈兴林,段广仁. 联合高斯回归的平方根UKF方法[J]. Journal of Systems Engineering and Electronics, 2010, 32(6): 1281-1285.
[11]	敖富江, 杜静, 颜跃进, 黄柯棣. 在线挖掘数据流滑动窗口中频繁闭项集[J]. Journal of Systems Engineering and Electronics, 2009, 31(5): 1235-1240.
[12]	侯伟, 杨炳儒, 吴晨生, 周谆. 一种基于滑动窗口的多关系模式频度更新算法[J]. Journal of Systems Engineering and Electronics, 2009, 31(3): 671-676.