基于RHTC网络的飞机目标检测与精细识别

doi:10.12305/j.issn.1001-506X.2021.12.04

摘要/Abstract

摘要：

飞机目标的方向检测和精细识别是高分辨率光学遥感图像解译领域的一个重要任务。针对遥感图像中多方向密集排布飞机的方向检测和识别困难问题, 提出一种基于旋转混合任务级联(rotated hybrid task cascade, RHTC)网络的飞机检测识别方法。首先, 基于混合任务级联(hybrid task cascade, HTC)网络, 扩展分割分支数量, 并将分割分支与包围框分支多层级联以不断加强语义特征。其次, 设计了一个新的斜框回归器, 将其添加在掩膜分支的最后一层以完成目标方向预测。最后, 增加一个新的方向损失函数以优化训练过程, 从而完成RHTC网络构建。在数据预处理阶段, 构建了每类型号飞机目标的精细掩膜以增强目标细节和提高掩膜预测精度。基于DOTA和公开Google图像构建的飞机数据集开展了多组实验。结果表明, 与其他多种先进的方法相比, 所提方法在飞机检测方向精准度和类别平均精准度上性能更优。此外, 所设计的斜框回归器和方向损失函数在嵌入到其他分割网络时也具有良好的性能。

关键词: 高分辨率遥感图像, 飞机目标, 旋转混合任务级联网络, 方向检测, 精细识别

Abstract:

Direction detection and fine-gained recognition of aircraft targets is an important task in the field of high-resolution optical remote sensing image interpretation. Aiming at the difficulty of direction detection and recognition of multi-directional densely arranged aircraft in remote sensing images, an aircraft detection and recognition method based on rotating hybrid task cascade (RHTC) network is proposed. Firstly, based on hybrid task cascade (HTC) network, the number of segmented branches is expanded, and the segmented branches and bounding box branches are cascaded at multiple levels to continuously strengthen semantic features. Secondly, a new slant frame regressor is designed and added to the last layer of the mask branch to complete the target direction prediction. Finally, a new directional loss function is added to optimize the training process, so as to complete the construction of RHTC network. In the data preprocessing stage, the fine mask of each type of aircraft target is constructed to enhance the target detail and improve the mask prediction accuracy. Several groups of experiments were carried out on aircraft data sets constructed based on DOTA and public Google images. The results show that compared with other advanced methods, the proposed method has better performance in aircraft detection direction accuracy and category average accuracy. In addition, the designed skew frame regressor and directional loss function also have good performance when embedded into other segmented networks.

Key words: high resolution remote sensing image, aircraft target, rotated hybrid task cascade network, direction detection, fine-grained recognition (RHTC)

中图分类号:

TN957

曹旭, 邹焕新, 成飞, 李润林, 贺诗甜. 基于RHTC网络的飞机目标检测与精细识别[J]. 系统工程与电子技术, 2021, 43(12): 3439-3451.

Xu CAO, Huanxin ZOU, Fei CHENG, Runlin LI, Shitian HE. Aircraft target detection and fine-grained recognition based on RHTC network[J]. Systems Engineering and Electronics, 2021, 43(12): 3439-3451.

图/表 23

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

表1

图13

图14

表2

表3

图15

表4

图16

表5

表6

表7

参考文献 35

1	唐小佩, 杨小冈, 刘云峰, 等. 基于深度卷积神经网络的飞机识别研究[J]. 电光与控制, 2018, 25 (5): 72- 76.
	TANG X P , YANG X G , LIU Y F , et al. Research on aircraft recognition based on deep convolutional neural network[J]. Electronics Optics and Control, 2018, 25 (5): 72- 76.
2	MIAO W X, LUO Z. Aircraft detection based on multiple scale faster-RCNN[C]//Proc. of the International Conference on Virtual Reality and Visualization, 2018: 90-93.
3	ZHANG M C , WU X , WANG P . Aircraft recognition via extraction of closed contour and partial feature matching[J]. Computer Simulation, 2006, doi: 10.1016/S1005-8885(07)60042-9
4	刘莉, 吴秀清, 牛宗标, 等. 模糊图像中的飞机识别方法[J]. 计算机仿真, 2006, 23 (3): 164- 166.
	LIU L , WU X Q , NIU Z B , et al. Aircraft recognition method in fuzzy image[J]. Computer simulation, 2006, 23 (3): 164- 166.
5	QIU J B , LI S J , WANG W . A new approach to detect aircrafts in remote sensing images based on corner and edge information fusion[J]. Microelectronics & Computer, 2011, 28 (9): 214- 216.
6	ZHANG Y X , JIAN H U , ZHANG Z , et al. Research on aircraft recognition based on local detection and affine invariant feature[J]. Remote Sensing Information, 2013, 28 (3): 3- 102.
7	ZHANG F , LIU S Q , WANG D B , et al. Aircraft recognition in infrared image using wavelet moment invariants[J]. Image and Vision Computing, 2009, 27 (4): 313- 318. doi: 10.1016/j.imavis.2008.08.007
8	CAI D . Study on aircraft recognition in high spatial resolution remote sensing image based on skeleton characteristics analysis[J]. Advanced Materials Research, 2011, 268-270, 1982- 1985. doi: 10.4028/www.scientific.net/AMR.268-270.1982
9	LIU G , SUN X , FU K , et al. Aircraft recognition in high-resolution satellite images using coarse-to-fine shape prior[J]. IEEE Geoscience and Remote Sensing Letters, 2012, 10 (3): 573- 577.
10	GUO Z H , LI S H . One-dimensional frequency-domain features for aircraft recognition from radar range profiles[J]. IEEE Trans.on Aerospace and Electronic Systems, 2010, 46 (4): 1880- 1892. doi: 10.1109/TAES.2010.5595601
11	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time target detection with region proposal networks[C]//Proc. of the Advances in Neural Information Processing Systems, 2015: 91-99.
12	TANG T Y , ZHOU S L , DENG Z P , et al. Vehicle detection in aerial images based on region convolutional neural networks and hard negative example mining[J]. Sensors, 2017, 17 (2): 336. doi: 10.3390/s17020336
13	DENG Z P , SUN H , ZHOU S , et al. Toward fast and accurate vehicle detection in aerial images using coupled region-based convolutional neural networks[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10 (8): 3652- 3664. doi: 10.1109/JSTARS.2017.2694890
14	孙嘉赤, 邹焕新, 邓志鹏, 等. 基于级联卷积神经网络的港口多方向舰船检测与分类[J]. 系统工程与电子技术, 2020, 42 (9): 1903- 1910.
	SUN J C , ZOU H X , DENG Z P , et al. Oriented inshore ship detection and classification based on cascade RCNN[J]. Systems Engineering and Electronics, 2020, 42 (9): 1903- 1910.
15	REN S Q , HE K M , GIRSHICK R , et al. Faster R-CNN: towards real-time target detection with region proposal networks[J]. IEEE Trans.on Pattern Analysis & Machine Intelligence, 2017, 39 (6): 1137- 1149.
16	DING J, XUE N, LONG Y, et al. Learning RoI transformer for oriented target detection in aerial images[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2019: 2849-2858.
17	YU D H, ZHAO C, XU J F, et al. A method of aircraft detection using fully convolutional network[C]//Proc. of the International Conference on Computer Systems, Electronics and Control, 2017: 914-918.
18	WANG B, ZHOU Y, ZHANG H, et al. An aircraft target detection method based on regional convolutional neural network for remote sensing images[C]//Proc. of the IEEE 9th International Conference on Electronics Information and Emergency Communication, 2019: 474-478.
19	LI Y , FU K , SUN H , et al. An aircraft detection framework based on reinforcement learning and convolutional neural networks in remote sensing images[J]. Remote Sensing, 2018, 10 (2): 243. doi: 10.3390/rs10020243
20	ZHANG Y H , SUN H , ZUO J , et al. Aircraft type recognition in remote sensing images based on feature learning with conditional generative adversarial networks[J]. Remote Sensing, 2018, 10 (7): 1123. doi: 10.3390/rs10071123
21	YANG J C , ZHU Y , JIANG B , et al. Aircraft detection in remote sensing images based on a deep residual network and super-vector coding[J]. Remote Sensing Letters, 2018, 9 (3): 228- 236. doi: 10.1080/2150704X.2017.1415474
22	LIN J X, LI X D, PAN H. Aircraft recognition in remote sensing images based on deep learning[C]//Proc. of the 33rd Youth Academic Annual Conference of Chinese Association of Automation, 2018: 895-899.
23	ZUO J W , XU G L , FU K , et al. Aircraft type recognition based on segmentation with deep convolutional neural networks[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15 (2): 282- 286. doi: 10.1109/LGRS.2017.2786232
24	俞汝劼, 杨贞, 熊惠霖. 基于深度卷积神经网络的航空器检测与识别[J]. 计算机应用, 2017, 37 (6): 1702- 1707.
	YU R J , YANG Z , XIONG H L . Aircraft detection and recognition based on deep convolutional neural network[J]. Journal of Computer Applications, 2017, 37 (6): 1702- 1707.
25	CHEN K, PANG J, WANG J, et al. Hybrid task cascade for instance segmentation[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2019: 4974-4983.
26	XIA G S, BAI X, DING J, et al. DOTA: a large-scale dataset for target detection in aerial images[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 3974-3983.
27	MA J Q , SHAO W , YE H , et al. Arbitrary-oriented scene text detection via rotation proposals[J]. IEEE Trans.on Multimedia, 2018, 20 (11): 3111- 3122. doi: 10.1109/TMM.2018.2818020
28	JIANG Y Y, ZHU X, WANG X, et al. R2CNN: rotational region cnn for orientation robust scene text detection[EB/OL]. [2020-11-30]. https://arxiv.org/abs/1706.09579v2
29	WANG X , SUN H , FU K , et al. Automatic ship detection in remote sensing images from google earth of complex scenes based on multiscale rotation dense feature pyramid networks[J]. Remote Sensing, 2018, 10 (1): 132- 140. doi: 10.3390/rs10010132
30	HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 2961-2969.
31	CAI Z , VASCONCELOS N . Cascade R-CNN: high quality target detection and instance segmentation[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2019,
32	CAI Z, VASCONCELOS N. Cascade R-CNN: delving into high quality target detection[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 6154-6162.
33	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2117-2125.
34	安静. 基于数学形态学的图像增强算法及其应用[D]. 兰州: 西北师范大学, 2016.
	AN J. Image enhancement algorithm based on mathematical morphology and its application[D]. Lanzhou: Northwest Normal University, 2015.
35	OTSU N . A threshold selection method from gray-level histograms[J]. IEEE Trans.on Systems, Man, and Cybernetics, 1979, 9 (1): 62- 66. doi: 10.1109/TSMC.1979.4310076

参数	型号
参数	A-50	An-12	An-24	B-1B	B-2	B-52	C-5	C-17	C-130
S₁₂	398	435	216	879	772	156	1 026	1 005	163
S₃₄	31	15	40	21	14	16	44	63	13

参数	型号
参数	E-3	EP-3	F-15	F-16	Il-76	KC-10	KC-135	MiG-31	P-8A
S₁₂	865	311	256	123	494	1 303	244	287	164
S₃₄	25	4	54	27	22	1	14	1	2

参数	型号
参数	RC-135S	RQ-4	Su-24	Su-27	Tu-22M	Tu-95	Tu-160	Typhoon	其他
S₁₂	460	267	205	353	675	218	863	189	3 272
S₃₄	36	41	3	73	69	6	7	37	34

数量类别	型号
数量类别	A-50	An-12	An-24	B-1B	B-2	B-52	C-5	C-17	C-130
原始数量	75	281	377	43	40	101	116	85	149
扩增数量	1 029	1 335	1 427	1 087	881	1 159	984	1 188	1 621

数量类别	型号
数量类别	E-3	EP-3	F-15	F-16	Il-76	KC-10	KC-135	MiG-31	P-8A
原始数量	36	247	137	85	886	75	202	479	36
扩增数量	798	2 120	1 156	1 090	1 657	1 263	1 370	1 505	1 236

数量类别	型号
数量类别	RC-135S	RQ-4	Su-24	Su-27	Tu-22M	Tu-95	Tu-160	Typhoon	其他
原始数量	34	9	741	1 101	440	189	93	19	2 418
扩增数量	1 017	471	1 317	1 774	2 561	870	687	1 665	5 762

数据集	精准度	方法
数据集	精准度	FRO^[26]	ROI transformer^[16]	RRPN^[28]	R2CNN^[29]	RDFPN^[30]	RHTC
自建数据集	DP	25.50	46.88	47.62	43.50	37.66	79.82
自建数据集	AP	71.08	89.56	84.25	87.20	85.63	93.53
DOTA	DP	40.39	68.33	65.24	66.87	63.50	76.89
DOTA	AP	73.82	86.25	80.92	85.98	81.14	87.13

参数	方法
参数	RHTC	FRO^[26]	ROI transformer^[16]	RRPN^[28]	R2CNN^[29]	RDFPN^[30]
Δθ_max/(°)	135.097	139.088	172.873	157.69	160.256	175.644
Δθ_median/(°)	3.13	61.845	18.294	25.652	17.21	24.251
Δθ_mean/(°)	10.587	71.083	38.904	50.334	40.603	47.247
Δθ_std/(°)	23.15	55.865	46.681	52.79	48.663	51.98

参数	方法
参数	FRO^[26]	ROI transformer^[16]	RRPN^[28]	R2CNN^[29]	RDFPN^[30]	RHTC
mAP	25.5	46.88	47.62	43.5	37.66	79.82