知识牵引与数据驱动的兵棋AI设计及关键技术

doi:10.12305/j.issn.1001-506X.2021.10.26

摘要/Abstract

摘要：

在分析知识推理型与数据学习型兵棋人工智能(artifical intelligence, AI)优缺点的基础上, 提出了基于知识牵引与数据驱动的AI设计框架。针对框架中涉及的基于数据补全的战场态势感知，基于遗传模糊系统的关键点推理，基于层次任务网的任务规划、计划修复与重规划，基于深度强化学习的算子动作策略优化等关键技术进行深入探讨。结果表明，所提框架具有较强的适应性, 不仅能够满足分队、群队、人机混合等兵棋推演的应用需求, 而且适用于解决一般回合制或即时策略性的博弈对抗问题。

关键词: 知识, 数据, 兵棋, 态势感知, 任务规划, 强化学习

Abstract:

Based on the analysis of the advantages and disadvantages of knowledge-based reasoning and data-learning wargame artifical intelligence (AI), an AI design framework based on knowledge traction and data-driven is proposed. The key technologies involved in the framework such as battlefield situation awareness based on data completion, key point reasoning based on genetic fuzzy system, mission planning based on hierarchical task network, plan repair and replanning, and operator action strategy optimization based on deep reinforcement learning are discussed in depth. The result shows that the proposed framework is highly adaptable. It can not only meet the application requirements of team, group, man-machine mixed wargaming, but also be suitable for solving general turn-based or real-time strategic game confrontation problems.

Key words: knowledge, data, wargame, situational awareness, mission planning, reinforcement learning

中图分类号:

TP181

程恺, 陈刚, 余晓晗, 刘满, 邵天浩. 知识牵引与数据驱动的兵棋AI设计及关键技术[J]. 系统工程与电子技术, 2021, 43(10): 2911-2917.

Kai CHENG, Gang CHEN, Xiaohan YU, Man LIU, Tianhao SHAO. Knowledge traction and data-driven wargame AI design and key technologies[J]. Systems Engineering and Electronics, 2021, 43(10): 2911-2917.

图/表 8

图1

图2

图3

图4

图5

图6

图7

图8

参考文献 32

1	SCHRITTWIESER J , ANTONOGLOU I , HUBERT T , et al. Mastering Atari, Go, chess and shogi by planning with a learned model[J]. Nature, 2020, 588 (7839): 604- 609. doi: 10.1038/s41586-020-03051-4
2	SILVER D , HUBERT T , SCHRITTWIESER J , et al. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play[J]. Science, 2018, 362 (6419): 1140- 1144. doi: 10.1126/science.aar6404
3	SILVER D , SCHRITTWIESER J , SIMONYAN K , et al. Mastering the game of go without human knowledge[J]. Nature, 2017, 550 (7676): 354- 359. doi: 10.1038/nature24270
4	SILVER D , HUANG A , MADDISON C , et al. Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016, 529 (7589): 484- 489.
5	NOAM B , TUOMAS S . Superhuman AI for multiplayer poker[J]. Science, 2019, 365 (6456): 885- 890. doi: 10.1126/science.aay2400
6	MATEJ M , MARTIN S , NEIL B , et al. Deep-stack: expert-level artificial intelligence in heads-up no-limit poker[J]. Science, 2017, 356 (6337): 508- 513. doi: 10.1126/science.aam6960
7	VINYALS O , BABUSCHKIN I , CZARNECKI W M , et al. Grandmaster level in starcraft Ⅱ using multi-agent reinforcement learning[J]. Nature, 2019, 575 (7782): 350- 369. doi: 10.1038/s41586-019-1724-z
8	胡晓峰, 贺筱媛, 陶九阳. AlphaGo的突破与兵棋推演的挑战[J]. 科技导报, 2017, (21): 51- 62.
	HU X F , HE X Y , TAO J Y . AlphaGo's breakthrough and challenges of wargaming[J]. Science & Technology Review, 2017, (21): 51- 62.
9	胡晓峰, 荣明. 智能化作战研究值得关注的几个问题[J]. 指挥与控制学报, 2018, 4 (3): 195- 200. doi: 10.3969/j.issn.2096-0204.2018.03.0195
	HU X F , RONG M . Several important questions of intelligent warfare research[J]. Journal of Command and Control, 2018, 4 (3): 195- 200. doi: 10.3969/j.issn.2096-0204.2018.03.0195
10	刘海洋, 唐宇波, 胡晓峰, 等. 面向联合作战评估的兵棋推演实验研究[J]. 指挥与控制学报, 2018, 4 (4): 272- 280.
	LIU H Y , TANG Y B , HU X F , et al. Wargaming experiment oriented evaluation of joint operations[J]. Journal of Command and Control, 2018, 4 (4): 272- 280.
11	YUNA H W, JOHN Y, ROBERT W, et al. Deterrence in the age of thinking machines[R]. California: RAND Corporation, 2020.
12	KELSEY A. DARPA wants wargame AI to never fight fair[EB/OL]. [2020-11-13]. https://breakingdefense.com/2020/08/darpa-wants-wargame-ai-to-never-fight-fair/.
13	周超, 胡晓峰, 郑书奎, 等. 战略战役兵棋演习系统兵力聚合问题研究[J]. 指挥与控制学报, 2017, 3 (1): 19- 26.
	ZHOU C , HU X F , ZHENG S K , et al. Force integration in strategic and operational war-game maneuver system[J]. Journal of Command and Control, 2017, 3 (1): 19- 26.
14	黄凯奇, 兴军亮, 张俊格, 等. 人机对抗智能技术[J]. 中国科学: 信息科学, 2020, 50 (4): 540- 550.
	HUANG K Q , XING J L , ZHANG J G , et al. Intelligent technologies of human-computer gaming (in Chinese)[J]. Scientia Sinica Informationis, 2020, 50 (4): 540- 550.
15	中国科学院自动化研究所. 即时策略人机对抗平台[EB/OL]. [2020-11-13]. http://wargame.ia.ac.cn.
	Institute of Automation, Chinese Academy of Sciences. Real-time strategy man-machine confrontation platform[EB/OL]. [2020-11-13]. http://wargame.ia.ac.cn.
16	MOY G, SHEKH S. The application of alphazero to wargaming[C]//Proc. of the 32nd Australasian Joint Conference on Artificial Intelligence, 2019: 3-14.
17	殷昌盛, 杨若鹏, 邹小飞, 等. 指挥智能化研究综述[C]//第8届中国指挥控制大会, 2020: 110-115.
	YIN C S, YANG R P, ZOU X F, et al. A survey on military intelligent command[C]//Proc. of the 8th China Command and Control Conference, 2020: 110-115.
18	李琛, 黄炎焱, 张永亮, 等. Actor-Critic框架下的多智能体决策方法及其在兵棋上的应用[J]. 系统工程与电子技术, 2021, 43 (3): 755- 762.
	LI C , HUANG Y Y , ZHANG Y L , et al. Multi-agent decision-making method based on Actor-Critic framework and its application in wargame[J]. Systems Engineering and Electronics, 2021, 43 (3): 755- 762.
19	刘满, 张宏军, 郝文宁, 等. 战术级兵棋实体作战行动智能决策方法研究[J]. 控制与决策, 2020, 35 (12): 2977- 2985.
	LIU M , ZHANG H J , HAO W N , et al. Research on intelligent decision-making method of tactical-level wargames[J]. Control and Decision, 2020, 35 (12): 2977- 2985.
20	李航, 刘代金, 刘禹. 军事智能博弈对抗系统设计框架研究[J]. 火力与指挥控制, 2020, 45 (9): 116- 121. doi: 10.3969/j.issn.1002-0640.2020.09.021
	LI H , LIU D J , LIU Y . Architecture design research of military intelligent wargame system[J]. Fire Control & Command Control, 2020, 45 (9): 116- 121. doi: 10.3969/j.issn.1002-0640.2020.09.021
21	张永亮, 董浩洋, 刘勇. 基于知识的智能指挥决策运行机制及其支撑技术研究[J]. 军事运筹与系统工程, 2020, 34 (2): 5- 12. doi: 10.3969/j.issn.1672-8211.2020.02.002
	ZHANG Y L , DONG H Y , LIU Y . Research on the support technology and its intelligent command decision-making mechanism based on knowledge[J]. Military Operations Research and Systems Engineering, 2020, 34 (2): 5- 12. doi: 10.3969/j.issn.1672-8211.2020.02.002
22	PANDEY G, DUKKIPATI A. Variational methods for conditional multimodal deep learning[C]//Proc. of the International Joint Conference on Neural Networks, 2017: 308-315.
23	HYUNJIK K, ANDRIY M, JONATHAN S, et al. Attentive neural processes[C]//Proc. of the 7th International Conference on Learning Representations, 2019.
24	张可, 郝文宁, 余晓晗, 等. 基于遗传模糊系统的兵棋推演关键点推理方法[J]. 系统工程与电子技术, 2020, 42 (10): 2303- 2311. doi: 10.3969/j.issn.1001-506X.2020.10.19
	ZHANG K , HAO W N , YU X H , et al. Wargame key point reasoning method based on genetic fuzzy system[J]. Systems Engineering and Electronics, 2020, 42 (10): 2303- 2311. doi: 10.3969/j.issn.1001-506X.2020.10.19
25	JARRAYA Y , BOUAZIZ S , HAGRAS H , et al. A multi-agent architecture for the design of hierarchical interval type-2 beta fuzzy system[J]. IEEE Trans.on Fuzzy Systems, 2019, 27 (6): 1174- 1188. doi: 10.1109/TFUZZ.2018.2871800
26	ILCHE G I , MARCO A . HTN planning: overview, comparison, and beyond[J]. Artificial Intelligence, 2015, 222 (2015): 124- 156.
27	HOLLER D, BERCHER P, BEHNKE G, et al. On guiding search in htn planning with classical planning heuristics[C]//Proc. of the 28th International Joint Conference on Artificial Intelligence, 2019: 6171-6175.
28	邵天浩, 张宏军, 程恺, 等. 层次任务网络中的重新规划研究综述[J]. 系统工程与电子技术, 2020, 42 (12): 2833- 2846. doi: 10.3969/j.issn.1001-506X.2020.12.21
	SHAO T H , ZHANG H J , CHENG K , et al. Review of replanning in hierarchical task network[J]. Systems Engineering and Electronics, 2020, 42 (12): 2833- 2846. doi: 10.3969/j.issn.1001-506X.2020.12.21
29	MUNOZ A H, DANNENHAUER D, REIFSNYDER N. Is everything going according to plan?Expectations in goal reasoning agents[C]//Proc. of the 33rd AAAI Conference on Artificial Intelligence, 2019.
30	HOLLER D, BERCHER P, BEHNKE G, et al. HTN plan repair using unmodified planning systems[C]//Proc. of the Inter-national Conference on Automated Planning and Scheduling, 2018: 26-30.
31	YE D, LIU Z, SUN M, et al. Mastering complex control in moba games with deep reinforcement learning[C]//Proc. of the 34th AAAI Conference on Artificial Intelligence, 2020.
32	WU B, FU Q, LIANG J, et al. Hierarchical macro strategy model for MOBA Game AI[EB/OL]. [2020-11-14]. https://arxiv.org/abs/1812.07887v1.

[1]	王磊, 张志勇, 曾维贵, 曹司磊, 张天赫. 基于数据场联合决策图改进的GMM聚类[J]. 系统工程与电子技术, 2022, 44(9): 2743-2751.
[2]	刘晓楠, 王昆, 孙晗伟, 鲁耀兵. Ka频段合成孔径雷达高度计机载数据处理[J]. 系统工程与电子技术, 2022, 44(9): 2800-2808.
[3]	余婧, 雍恩米, 陈汉洋, 郝东, 张显才. 面向多无人机协同对地攻击的双层任务规划方法[J]. 系统工程与电子技术, 2022, 44(9): 2849-2857.
[4]	封皓君, 段立, 张碧莹, 刘海潮. 双向循环进化的实体链接及知识推理框架[J]. 系统工程与电子技术, 2022, 44(9): 2878-2885.
[5]	胡泰洋, 张晋宇, 卢海梁, 李鹏飞, 李一楠, 吕容川. 基于数据融合的分布式综合孔径微波辐射高分辨率成像算法[J]. 系统工程与电子技术, 2022, 44(8): 2403-2409.
[6]	郭军成, 万刚, 胡欣杰, 严发宝, 王帅. 太阳射电频谱爆发识别的元学习方法[J]. 系统工程与电子技术, 2022, 44(8): 2410-2418.
[7]	杨立儒, 刘永祥, 杨威. 基于迁移学习的雷达杂波幅度统计模型选择[J]. 系统工程与电子技术, 2022, 44(8): 2457-2467.
[8]	王彩云, 吴钇达, 王佳宁, 马璐, 赵焕玥. 基于改进的CNN和数据增强的SAR目标识别[J]. 系统工程与电子技术, 2022, 44(8): 2483-2487.
[9]	朱霸坤, 朱卫纲, 李伟, 杨莹, 高天昊. 基于马尔可夫的多功能雷达认知干扰决策建模研究[J]. 系统工程与电子技术, 2022, 44(8): 2488-2497.
[10]	刘满, 张宏军, 徐有为, 冯欣亮, 冯玉芳. 群队级兵棋实体智能行为决策方法研究[J]. 系统工程与电子技术, 2022, 44(8): 2562-2569.
[11]	徐瑞麟, 耿伯英, 刘树衎. 多模态公文的结构知识抽取与组织研究[J]. 系统工程与电子技术, 2022, 44(7): 2241-2250.
[12]	王冠, 茹海忠, 张大力, 马广程, 夏红伟. 弹性高超声速飞行器智能控制系统设计[J]. 系统工程与电子技术, 2022, 44(7): 2276-2285.
[13]	孟泠宇, 郭秉礼, 杨雯, 张欣伟, 赵柞青, 黄善国. 基于深度强化学习的网络路由优化方法[J]. 系统工程与电子技术, 2022, 44(7): 2311-2318.
[14]	张东东, 艾小川, 刘畅. 基于相似样本特征提取的装备性能退化研究[J]. 系统工程与电子技术, 2022, 44(7): 2374-2380.
[15]	郭冬子, 黄荣, 许河川, 孙立伟, 崔乃刚. 再入飞行器深度确定性策略梯度制导方法研究[J]. 系统工程与电子技术, 2022, 44(6): 1942-1949.