基于A3C的多功能雷达认知干扰决策方法

doi:10.12305/j.issn.1001-506X.2023.01.11

系统工程与电子技术 ›› 2023, Vol. 45 ›› Issue (1): 86-92.doi: 10.12305/j.issn.1001-506X.2023.01.11

• 传感器与信号处理 • 上一篇

基于A3C的多功能雷达认知干扰决策方法

邹玮琦, 牛朝阳, 刘伟, 高欧阳, 张浩波

信息工程大学数据与目标工程学院, 河南郑州 450000

收稿日期:2021-10-25 出版日期:2023-01-01 发布日期:2023-01-03
通讯作者: 牛朝阳
作者简介:邹玮琦 (1998—), 男, 硕士研究生, 主要研究方向为认知干扰决策、干扰效果评估
牛朝阳 (1981—), 男, 副教授, 博士, 主要研究方向为雷达信息处理与对抗
刘伟 (1980—), 男, 副教授, 博士, 主要研究方向为智能信息处理、遥感图像分析
高欧阳 (1998—), 男, 硕士研究生, 主要研究方向为雷达信息处理
张浩波 (1996—), 男, 硕士研究生, 主要研究方向为雷达信息处理

Cognitive jamming decision-making method against multifunctional radar based on A3C

Weiqi ZOU, Chaoyang NIU, Wei LIU, Ouyang GAO, Haobo ZHANG

School of Data and Target Engineering, Information Engineering University, Zhengzhou 450000, China

Received:2021-10-25 Online:2023-01-01 Published:2023-01-03
Contact: Chaoyang NIU

摘要/Abstract

摘要：

在多功能雷达对抗领域, 目前基于强化学习理论的认知干扰决策方法难以满足雷达对抗高实时性要求。对此, 将异步优势行动者-评论家(asynchronous advantage actor-critic, A3C)算法引入到认知干扰决策领域, 设计了包括干扰机模型、环境模型(目标方多功能雷达)以及交互机制的认知干扰决策整体框架, 制定了干扰决策流程, 干扰机模型利用异步多线程方式与环境模型进行交互训练。仿真实验表明, 在扩充雷达任务转换关系表的基础上, 所提方法与基于深度Q网络(deep Q network, DQN)的认知干扰决策系列方法相比, 极大地提高了时间效率, 平均决策时间降低70%以上, 并且在决策准确度上有着明显优势, 表明所提方法能够为多功能雷达对抗决策提供更有力的技术支撑。

关键词: 干扰决策, 异步优势, 行动者-评论家, 时间效率, 决策准确度

Abstract:

In the field of multifunctional radar countermeasures, the current cognitive jamming decision-making method based on reinforcement learning theory is difficult to meet the high real-time requirements of radar countermeasures. Therefore, we apply asynchronous advantage actor-critic (A3C) to the field of cognitive jamming decision-making, design the cognitive jamming decision-making overall framework including the jammer model, the environment model (the target multifunctional radar) and its interaction mechanism, and develop a decision-making process, the jammer model uses asynchronous multi-threaded way to interact with the environment model. Simulation results show that on the basis of expanding the radar task transformation relationship table, compared with the cognitive jamming decision-making series methods based on deep Q network (DQN), theproposedmethod can significantly improve time efficiency, decreases the average decision-making time by more than 70%, and has obvious advantages in decision-making degree of accuracy, it shows that theproposed method can provide more powerful technical support for multifunctional radar countermeasures decision-making.

Key words: jamming decision-making, asynchronous advantage, actor-critic, time efficiency, decision-making degree of accuracy

中图分类号:

TN974

邹玮琦, 牛朝阳, 刘伟, 高欧阳, 张浩波. 基于A3C的多功能雷达认知干扰决策方法[J]. 系统工程与电子技术, 2023, 45(1): 86-92.

Weiqi ZOU, Chaoyang NIU, Wei LIU, Ouyang GAO, Haobo ZHANG. Cognitive jamming decision-making method against multifunctional radar based on A3C[J]. Systems Engineering and Electronics, 2023, 45(1): 86-92.

图/表 8

图1

图2

图3

表1

表2

表3

图4

图5

参考文献 15

1	FARINA A, TIMMONERI L. Live data test of electronic counter counter measures (ECCM) on a multifunctional prototype radar[C]// Proc. of the IEEE Metrology for Aerospace, 2016.
2	KOWALSKI E, CONWAY D, MORRIS A, et al. Multifunction phased array radar advanced technology demonstrator (MPAR ATD) nearfield testing and fielding[C]//Proc. of the IEEE Radar Conference, 2019.
3	周脉成. 基于博弈论的雷达干扰决策技术研究[D]. 西安: 西安电子科技大学, 2014.
	ZHOU M C. Research on radar jamming decision technology based on game theory[D]. Xi'an: Xidian University, 2014.
4	孙宏伟, 童宁宁, 孙富君. 基于D-S证据理论的电子干扰模式选择[J]. 弹箭与制导学报, 2003, 23 (2): 218- 220.
	SUN H W , TONG N N , SUN F J . Jamming design selection based on D-S theory[J]. Journal of Projectiles, Rockets, Missiles and Guidance, 2003, 23 (2): 218- 220.
5	李云杰, 朱云鹏, 高梅国. 基于Q-学习算法的认知雷达对抗过程设计[J]. 北京理工大学学报, 2015, 35 (11): 1194- 1199.
	LI Y J , ZHU Y P , GAO M G . Design of cognitive radar jamming based on Q-learning algorithm[J]. Transactions of Beijing Institute of Technology, 2015, 35 (11): 1194- 1199.
6	MOUSAVI S S, SCHUKAT M, HOWLEY E. Deep reinforcement learning: an overview[C]//Proc. of the SAI Intelligent Systems Conference, 2016: 426-440.
7	邢强, 贾鑫, 朱卫纲. 基于Q-学习的智能雷达对抗[J]. 系统工程与电子技术, 2018, 40 (5): 1031- 1035.
	XING Q , JIA X , ZHU W G . Intelligent radar countermeasure based on Q-learning[J]. Systems Engineering and Electronics, 2018, 40 (5): 1031- 1035.
8	张柏开, 朱卫纲. 基于Q-Learning的多功能雷达认知干扰决策方法[J]. 电讯技术, 2020, 60 (2): 129- 136.
	ZHANG B K , ZHU W G . A cognitive jamming decision method for multi-functional radar based on Q-learning[J]. Telecommunication Engineering, 2020, 60 (2): 129- 136.
9	张柏开, 朱卫纲. 对多功能雷达的DQN认知干扰决策方法[J]. 系统工程与电子技术, 2020, 42 (4): 819- 825.
	ZHANG B K , ZHU W G . DQN based decision-making method of cognitive jamming against multifunction radar[J]. Systems Engineering and Electronics, 2020, 42 (4): 819- 825.
10	MNIH V, BADIA A P, MIRZA M, et al. Asynchronous methods for deep reinforcement learning[C]//Proc. of the International Conference on Machine Learning, 2016: 1928-1937.
11	HUSSEIN A , ELYAN E , GABER M , et al. Deep imitation learning for 3D navigation tasks[J]. Neural Computing and Applications, 2018, 29 (7): 389- 404.
12	GENDERS W , RAZAVI S . Evaluating reinforcement learning state representations for adaptive traffic signal control[J]. Procedia Computer Science, 2018, 130, 26- 33.
13	KONDA V R , TSITSIKLIS J N . Onactor-critic algorithms[J]. SIAM Journal on Control and Optimization, 2003, 42 (4): 1143- 1166.
14	马爽. 多功能雷达电子情报信号处理关键技术研究[D]. 长沙: 国防科技大学, 2013.
	MA S. Research on ELINT signal processing key technologies for multifunction radar[D]. Changsha: National University of Defense Technology, 2013.
15	张柏开, 朱卫纲. MFR认知干扰决策体系构建及关键技术[J]. 系统工程与电子技术, 2020, 42 (9): 1969- 1975.
	ZHANG B K , ZHU W G . Construction and key technologies of cognitive jamming decision-making system against MFR[J]. Systems Engineering and Electronics, 2020, 42 (9): 1969- 1975.

功能层	任务层
功能层	1	2	3	4
T₁	T₁^A	-	-	-
T₂	T₂^A	T₂^B	-	-
T₃	T₃^A	T₃^B	T₃^C	-
				

雷达任务状态	干扰样式
雷达任务状态	1	2	3	4	5	6	7	8	9	10	11	12
T₁^A	-	T₂^B	-	T₂^A	-	T₁^A	-	-	-	T₁^A	-	-
T₂^A	T₁^A	-	-	-	T₂^A	T₃^A	-	T₃^B	-	-	-	-
T₂^B	T₁^A	-	-	-	T₂^B	-	T₃^C	-	T₃^A	-	-	-
T₃^A~T₁₃^C												
T₁₄^A	-	T₁₅^A	-	-	T₁₃^B	-	-	T₁₄^A	-	-	-	-
T₁₄^B	-	-	-	T₁₃^C	-	T₁₄^B	-	-	T₁₃^B	-	-	T₁₅^A
T₁₅^A	-	-	-	-	-	-	-	-	-	-	-	-

认知干扰决策方法	平均每回合所用时间/ms
DQN	160.579 5
Double DQN	77.591 8
Prioritized Replay DQN	127.440 2
A3C	2.654 2

[1]	张柏开, 朱卫纲. MFR认知干扰决策体系构建及关键技术[J]. 系统工程与电子技术, 2020, 42(9): 1969-1975.
[2]	张柏开, 朱卫纲. 对多功能雷达的DQN认知干扰决策方法[J]. 系统工程与电子技术, 2020, 42(4): 819-825.
[3]	邢强, 朱卫纲, 贾鑫, 郑光勇. 干扰规则库未知条件下的干扰决策[J]. 系统工程与电子技术, 2019, 41(2): 298-303.
[4]	颛孙少帅, 杨俊安, 刘辉, 黄科举. 基于正强化学习和正交分解的干扰策略选择算法[J]. 系统工程与电子技术, 2018, 40(3): 518-525.
[5]	高晓光,胡明,郑景嵩. 突防任务中的单机对多目标干扰决策[J]. Journal of Systems Engineering and Electronics, 2010, 32(6): 1239-1243.

基于A3C的多功能雷达认知干扰决策方法

Cognitive jamming decision-making method against multifunctional radar based on A3C

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 15

相关文章 5

编辑推荐

Metrics

本文评价