基于DQN的探测干扰一体化波形优化设计

doi:10.12305/j.issn.1001-506X.2023.03.03

摘要/Abstract

摘要：

由于侦察干扰机设备具有发射功能, 为使发射的干扰信号还具有探测的效果，考虑将探测信号隐藏在干扰信号中, 提出一种基于非均匀间歇采样重复转发的探测干扰一体化信号波形。首先，建立一体化信号模型, 并利用非均匀间歇采样重复转发技术实现幅度编码调制；然后，在优化过程中，从模糊函数以及雷达检测环节分析一体化信号的特征, 根据距离、速度分辨率以及脉压后幅度的均值与标准差之比，构造相应的目标函数；最后，利用深度Q学习算法求解目标函数, 获取最优的幅度编码方式。仿真结果表明, 当编码状态量小时, 深度Q网络(deep Q-network, DQN)算法与强化学习算法收敛效果一致。与遗传算法相比, DQN算法最优解的质量提高了13.10%;当编码状态量增大时, 相对于遗传算法和强化学习算法, DQN算法的收敛值更优, 最优解更稳定。

关键词: 探测干扰一体化信号, 非均匀间歇采样重复转发, 模糊函数, 脉冲幅度编码, 深度Q学习

Abstract:

In view that the reconnaissance jammer has the transmitting function, with the aim that the transmitted interference signal can also have the detection effect. The detection signal can be hidden in the jamming signal. A detection and jamming integrated signal waveform based on non-uniform intermittent sampling and repeated forwarding is proposed. Firstly, an integrated signal model is established, and then the non-uniform intermittent sampling and repeating forwarding technology is used to realize amplitude coding modulation. Secondly, in the optimization process, the characteristics of the integrated signal are analyzed from the fuzzy function and the radar detection link, and the corresponding objective function is constructed according to the distance, velocity resolution, and the ratio of the mean value to the standard deviation value of the post-pulse compression amplitude. Finally, the deep Q-learning algorithm is used to solve the objective function to obtain the optimal amplitude coding. The simulation results show that when the number of coding states is small, the convergence effect of deep Q-network(DQN) algorithm is consistent with that of the reinforcement learning algorithm. Compared with the genetic algorithm, the quality of optimal solution of DQN algorithm is improved by 13.10%. When the number of coding states increases, compared with genetic algorithm and reinforcement learning algorithm, the convergence value of DQN algorithm is larger and the optimal solution is more stable.

Key words: detection and jamming integrated signal, non-uniform intermittent sampling and repeated forwarding, fuzzy function, pulse amplitude coding, deep Q-learning

中图分类号:

TN97

陈涛, 张颖, 胡学晶, 肖易寒. 基于DQN的探测干扰一体化波形优化设计[J]. 系统工程与电子技术, 2023, 45(3): 638-646.

Tao CHEN, Ying ZHANG, Xuejing HU, Yihan XIAO. Integrated waveform optimization design of detection and jamming based on DQN[J]. Systems Engineering and Electronics, 2023, 45(3): 638-646.

图/表 19

图1

图2

图3

图4

图5

图6

表1

表2

图7

图8

图9

图10

表3

表4

图11

表5

图12

图13

表6

参考文献 20

1	曹兰英, 郭明明, 罗美方. 雷达与电子战的认知博弈[J]. 雷达科学与技术, 2021, 19 (5): 552- 557. doi: 10.3969/j.issn.1672-2337.2021.05.011
	CAO L Y , GUO M M , LUO M F . Cognitive game between radar and electronic warfare[J]. Radar Science and Technology, 2021, 19 (5): 552- 557. doi: 10.3969/j.issn.1672-2337.2021.05.011
2	李其虎, 王颖, 商开拴. 干扰探测一体化信号波形设计与性能仿真[J]. 探测与控制学报, 2020, 42 (1): 39- 43.
	LI Q H , WANG Y , SHANG K X . Waveform design and perfor-mance simulation of interference detection integrated signal[J]. Journal of Detection and Control, 2020, 42 (1): 39- 43.
3	熊国淼, 李云鹏, 李鹏蛟, 等. 基于PNFM-LFM复合调制的探测干扰共享波形设计[J]. 航空学报, 2022, 42 (8): 347- 357.
	XIONG G M , LI Y P , LI P J , et al. Design of detecting interference shared waveform based on PNFM-LFM composite modulation[J]. Acta Aeronautica et Astronautica Sinica, 2022, 42 (8): 347- 357.
4	朱晟坤, 杨瑞娟, 左家骏, 等. 雷达通信干扰一体化OFDM共享波形优化方法[J]. 兵器装备工程学报, 2020, 41 (11): 218- 223. doi: 10.11809/bqzbgcxb2020.11.040
	ZHU S K , YANG R J , ZUO J J , et al. Integrative waveform optimization of radar communication jamming based on OFDM signal[J]. Journal of Ordnance Equipment Engineering, 2020, 41 (11): 218- 223. doi: 10.11809/bqzbgcxb2020.11.040
5	TIAN L Y , HE M , LIU B , et al. Novel non-coherent integration method using binary phase-coded radar signal[J]. Journal of Beijing Institute of Technology, 2013, 22 (1): 60- 66.
6	SHI C G , WANG F , SALOUS S , et al. Adaptive jamming waveform design for distributed multiple-radar architectures based on low probability of intercept[J]. Radio Science, 2019, 54 (1/2): 72- 90.
7	GE M M , YU X X , YAN Z X , et al. Joint cognitive optimization of transmit waveform and receive filter against deceptive interference[J]. Signal Processing, 2021, 185 (3): 108084.
8	CUI G L , YU X X , YANG J , et al. An overview of waveform optimization methods for cognitive radar[J]. Journal of Radars, 2019, 8 (5): 537- 557.
9	邢强, 贾鑫, 朱卫纲. 基于Q-学习的智能雷达对抗[J]. 系统工程与电子技术, 2018, 40 (5): 1031- 1035.
	XING Q , JIA X , ZHU W G . Intelligent radar countermeasure based on Q-learning[J]. Systems Engineeering and Electronics, 2018, 40 (5): 1031- 1035.
10	武忠国. 认知电子战干扰波形设计[D]. 西安: 西安电子科技大学, 2018.
	WU Z G. Waveform design of cognitive electronic warfare interference[D]. Xi'an: Xidian University, 2018.
11	张柏开, 朱卫纲. 对多功能雷达的DQN认知干扰决策方法[J]. 系统工程与电子技术, 2020, 42 (4): 819- 825.
	ZHANG B K , ZHU W G . DQN based decision-making method of cognitive jamming against multifunctional radar[J]. Systems Engineering and Electronics, 2020, 42 (4): 819- 825.
12	FAN J Q, WANG Z R, XIE Y C, et al. A theoretical analysis of deep Q-learning[C]//Proc. of the 2nd Conference on Learning for Dynamics and Control, 2020: 486-489.
13	张荣霞, 武长旭, 孙同超, 等. 深度强化学习及在路径规划中的研究进展[J]. 计算机工程与应用, 2021, 57 (19): 44- 56.
	ZHANG R X , WU C X , SUN T C , et al. Research progress of deep reinforcement learning and path planning[J]. Computer Engineering and Applications, 2021, 57 (19): 44- 56.
14	李宏, 薛冰, 赵艳丽. 雷达欺骗干扰的现状与困惑[J]. 航天电子对抗, 2019, 35 (4): 1- 5.
	LI H , XUE B , ZHAO Y L . Status and confusion of radar deception jamming[J]. Aerospace Electronic Countermeasures, 2019, 35 (4): 1- 5.
15	金德亚, 张玮. 线性调频脉冲压缩信号旁瓣抑制分析[J]. 舰船电子对抗, 2021, 44 (3): 100- 102.100-102, 106
	JIN D Y , ZHANG W . Analysis of sidelobe suppression of chirp compressed signal[J]. Ship Electronic Countermeasures, 2021, 44 (3): 100- 102.100-102, 106
16	KYLE D , JOEY B . Understanding digital radio frequency memory performance in countermeasure design[J]. Applied Sciences, 2020, 10 (12): 4123.
17	张养瑞, 李云杰, 李曼玲, 等. 间歇采样非均匀重复转发实现多假目标压制干扰[J]. 电子学报, 2016, 44 (1): 46- 53.
	ZHANG Y R , LI Y J , LI M L , et al. Suppress jamming technique of multiple false targets on interrupted-sampling and non-uniform periodic repeater[J]. Acta Electronica Sinica, 2016, 44 (1): 46- 53.
18	张亮, 王国宏, 张翔宇, 等. 基于分数阶字典的间歇采样转发干扰自适应抑制算法[J]. 系统工程与电子技术, 2020, 42 (7): 1439- 1448.
	ZHANG L , WANG G H , ZHANG X Y , et al. Interrupted-sampling repeater jamming adaptive suppression algorithm based on fractional dictionary[J]. Systems Engineering and Electronics, 2020, 42 (7): 1439- 1448.
19	刘东青, 孙陈刚, 刘和飞. 间歇采样移频非均匀重复转发干扰[J]. 电子信息对抗技术, 2019, 34 (4): 70- 75.
	LIU D Q , SUN C G , LIU H F . Intermittent sampling and frequency shift non-uniform and periodic repeating jamming[J]. Electronic Information Countermeasures Technology, 2019, 34 (4): 70- 75.
20	WANG W J , WANG R Y , JIANG R K , et al. Modified reference window for two-dimensional CFAR in radar target detection[J]. The Journal of Engineering, 2019, 2019 (21): 7924- 7927.

LFM调频信号参数	参数大小
信号长度T/μs	20
带宽B/MHz	10
采样频率f₀/MHz	40
调频斜率K	5×10¹¹
最小采样时间τ/μs	0.125

DQN网络参数	参数大小
学习率α	0.01
折现因子γ	0.9
记忆库总量	1 000
探索概率ε	0.9

指标	分组
指标	固定组	随机组1	随机组2	随机组3
收敛时间/s	111	99	324	225
收敛值	0.8251	0.8013	0.7928	0.8315

遗传算法参数	参数大小
种群个数	40
选择概率	0.5
杂交概率	0.7
变异率	0.001

DQN网络参数	参数大小
学习率α	0.01
折现因子γ	0.9
探索概率ε	0.9