基于Faster R-CNN的电子海图和雷达图像的数据融合

doi:10.3969/j.issn.1001-506X.2020.06.09

摘要/Abstract

摘要：

船载导航雷达和电子海图(electronic navigational chart, ENC)是船舶重要的导航仪器,雷达图像和ENC图像的融合能够给出更加丰富的航行和避碰信息。为此,提出了一种基于深度学习理论的提取雷达图像中鲁棒特征的数据融合算法,实现了ENC和雷达图像较高层次的数据融合。首先,利用深度学习算法对雷达图像进行目标检测,识别船舶雷达的特征目标。其次,对检测到的特征区域执行图像处理,并确定用于ENC和船舶雷达图像配准的参考点。最后,根据参考点进行仿射变换,实现融合算法。利用连续时间段的狭窄水域中的真实船舶雷达图像数据对融合算法进行验证,结果显示船舶雷达图像和ENC的海岸线边缘信息匹配良好且满足实时性要求。该算法与简单的像素级图像融合算法相比鲁棒性更强,实现了ENC与雷达图像的特征级融合。

关键词: 深度学习, 船舶雷达图像, 电子海图, 数据融合

Abstract:

Marine narigation radar and electronic navigational chart (ENC) are important navigation instruments. The fusion of radar images and electronic navigational chart data can give more abundant navigation and collision avoidance information. Therefore, this paper proposes a data fusion algorithm based on deep learning to extract robust features from radar images. At first the ability of deep learning is exploited to perform target detection for the identification of marine radar targets. Then, the image processing is performed on the identified targets to determine the reference points for consistent data fusion of ENC and marine radar information. Finally, an affine transformation-fusion algorithm is built to merge the marine radar and electronic chart data according to the determined reference points. The proposed fusion algorithn is verified through simulations using ENC data and marine radar images from real ships in narrow waters in a continuous period. The results show a suitable edge matching performance of the shoreline and real-time applicability. The algorithm is more robust than the simple pixel-level image fusion, and it realizes the feature level fusion of ENC and radar images.

Key words: deep learning, marine radar image, electronic navigational chart (ENC), data fusion

中图分类号:

TN957.51

张大恒, 张英俊, 张闯. 基于Faster R-CNN的电子海图和雷达图像的数据融合[J]. 系统工程与电子技术, 2020, 42(6): 1267-1273.

Daheng ZHANG, Yingjun ZHANG, Chuang ZHANG. Data fusion of electronic navigational chart and radar images based on Faster R-CNN[J]. Systems Engineering and Electronics, 2020, 42(6): 1267-1273.

图/表 11

图1

图2

图3

图4

图5

表1

图6

图7

图8

表2

图9

参考文献 29

1	LLINAS J, HALL D L.An introduction to multi-sensor data fusion[C]//Proc.of the IEEE International Symposium on Circuits & Systems, 1998: 537-540.
2	AL-SHARMAN M , EMRAN B J , JARADAT M A , et al. Precision landing using an adaptive fuzzy multi-sensor data fusion architecture[J]. Applied Soft Computing, 2018, 69, 149- 164. doi: 10.1016/j.asoc.2018.04.025
3	阎海峰, 胡高歌, 高社生, 等. 基于极大似然准则与滚动时域估计的自适应UKF算法[J]. 系统工程与电子技术, 2016, 38 (7): 1629- 1637.
	YAN H F , HU G G , GAO S S , et al. Adaptive UKF based on maximum likelihood principle and receding horizon estimation[J]. Systems Engineering and Electronics, 2016, 38 (7): 1629- 1637.
4	孟阳, 高社生, 王维. 基于高斯过程回归的平方根UPF算法[J]. 系统工程与电子技术, 2015, 37 (12): 2817- 2822. doi: 10.3969/j.issn.1001-506X.2015.12.23
	MENG Y , GAO S S , WANG W . Square-root unscented particle filter based on Gaussian process regression[J]. Systems Engineering and Electronics, 2015, 37 (12): 2817- 2822. doi: 10.3969/j.issn.1001-506X.2015.12.23
5	赵蓝飞, 王爱丽, 王博, 等. 基于子图融合技术的图像增强算法[J]. 系统工程与电子技术, 2017, 39 (12): 2840- 2848. doi: 10.3969/j.issn.1001-506X.2017.12.30
	ZHAO L F , WANG A L , WANG B , et al. Image enhancement algorithm based on sub-image fusion[J]. Systems Engineering and Electronics, 2017, 39 (12): 2840- 2848. doi: 10.3969/j.issn.1001-506X.2017.12.30
6	MA J Y , JIANG J J , ZHOU H B , et al. Guided locality preserving feature matching for remote sensing image registration[J]. IEEE Trans.on Geoscience and Remote Sensing, 2018, PP (99): 1- 13.
7	李广儒, 张闯, 胡景峰. 基于Harris特征点检测法的雷达图像与电子海图的数据融合[J]. 大连海事大学学报, 2009, 35 (2): 55- 58.
	LI G R , ZHANG C , HU J F . Data fusion of radar image and ECDIS based on matching of Harris feature points[J]. Journal of Dalian Maritime University, 2009, 35 (2): 55- 58.
8	操文芷.基于电子海图和航海雷达的无人水面艇路径规划研究[D].辽宁:大连海事大学, 2017.
	CAO W Z.Research on path planning of USV based on electronic chart and radar image[D].Liaoning: Dalian Maritime University, 2017.
9	ZHAO Y , GAO S S , ZHANG J , et al. Robust predictive augmented unscented Kalman filter[J]. International Journal of Control, Automation and Systems, 2014, 12 (5): 996- 1004. doi: 10.1007/s12555-013-0048-2
10	GOYAL M, REEVES N, RAJBHANDARI S, et al.Fully convolutional networks for diabetic foot ulcer segmentation[C]//Proc.of the IEEE International Conference on Systems, 2017: 618-623.
11	KAZIMIERSKI W , STATECZNY A . Radar and automatic identification system track fusion in an electronic chart display and information system[J]. Journal of Navigation, 2015, 68 (6): 1141- 1154. doi: 10.1017/S0373463315000405
12	彭书娟, 曲长文, 李健伟. 基于ROEWA算子局部活动轮廓的SAR图像分割算法[J]. 系统工程与电子技术, 2019, 41 (2): 280- 291.
	PENG S J , QU C W , LI J W . Local motion contour segmentation of SAR image based on ROEWA operator[J]. Systems Engineering and Electronics, 2019, 41 (2): 280- 291.
13	刘赢, 田润澜, 王晓峰. 基于深层卷积神经网络和双谱特征的雷达信号识别方法[J]. 系统工程与电子技术, 2019, 41 (9): 1998- 2005.
	LIU Y , TIAN R L , WANG X F . Radar signal recognition method based on deep convolutional neural network and bispectrum feature[J]. Systems Engineering and Electronics, 2019, 41 (9): 1998- 2005.
14	才志.雷达图像与电子海图叠加的研究[D].辽宁:大连海事大学, 2011.
	CAI Z.Research of superposition between radar image and electronic chart[D].Liaoning: Dalian Maritime University, 2011.
15	IEC 62388.Maritime navigation and radiocommunication equipment and systems-shipborne radar-performance requirements, methods of testing and required test results[S].Geneva: International Electrotechnology Commission, 2013.
16	DAI J F, LI Y, HE K M, et al.R-FCN: object detection via region-based fully convolutional networks[C]//Proc.of the 30th International Conference on Neural Information Processing Systems, 2016: 379-387.
17	SUN X , WU P , HOI S C H . Face detection using deep learning:an improved faster RCNN approach[J]. Neurocomputing, 2018, 299, 42- 50. doi: 10.1016/j.neucom.2018.03.030
18	HOANG NGAN LE T, ZHENG Y, ZHU C, et al.Multiple scale faster-rcnn approach to driver's cell-phone usage and hands on steering wheel detection[C]//Proc.of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2016: 46-53.
19	REN S, HE K, GIRSHICK R, et al.Faster R-CNN: towards real-time object detection with region proposal networks[C]//Proc.of the Advances in Neural Information Processing Systems, 2015: 91-99.
20	XU Y Z , YU G Z , WANG Y P , et al. Car detection from low-altitude UAV imagery with the faster R-CNN[J]. Journal of Advanced Transportation, 2017, 2823619.
21	ZHAO X T, LI W, ZHANG Y F, et al.A faster RCNN-based pedestrian detection system[C]//Proc.of the IEEE 84th Vehicular Technology Conference (VTC-Fall), 2016: 1-5.
22	REN Y , ZHU C R , XIAO S P . Deformable faster R-CNN with aggregating multi-layer features for partially occluded object detection in optical remote sensing images[J]. Remote Sensing, 2018, 10 (9): 1470. doi: 10.3390/rs10091470
23	SALVADOR A, GIRó-I-NIETO X, MARQUéS F, et al.Faster R-CNN features for instance search[C]//Proc.of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2016: 9-16.
24	WANG Q , ZHANG X D , CHEN G Z , et al. Change detection based on faster R-CNN for high-resolution remote sensing images[J]. Remote Sensing Letters, 2018, 9 (10): 923- 932. doi: 10.1080/2150704X.2018.1492172
25	谢兴澜, 李源惠. 电子海图中不同基准纬度海图墨卡托坐标转换[J]. 大连海事大学学报, 2003, 29 (2): 19- 21.
	XIE X L , LI Y H . Mercator coordinate transformation of charts with different datum latitude in electronic chart[J]. Journal of Dalian Maritime University, 2003, 29 (2): 19- 21.
26	郑尧. 电子海图与雷达图像的融合显示[J]. 舰船科学技术, 2015, 37 (3): 181- 184. doi: 10.3404/j.issn.1672-7649.2015.03.040
	ZHENG Y . Research of integration display on electronic charts and radar images[J]. Ship Science and Technology, 2015, 37 (3): 181- 184. doi: 10.3404/j.issn.1672-7649.2015.03.040
27	OTSU N . A threshold selection method from gray-level histograms[J]. IEEE Trans.on Systems Man & Cybernetics, 1979, 9 (1): 62- 66.
28	杨功流, 窦玉宝, 郑荣才. 一种雷达和电子海图图像叠加方法[J]. 中国惯性技术学报, 2010, 18 (2): 181- 184.
	YANG G L , DOU Y B , ZHENG R C . Method of image overlay on radar and electronic chart[J]. Journal of Chinese Inertial Technology, 2010, 18 (2): 181- 184.
29	熊恒昌.基于深度学习的实时目标检测[D].湖南:湖南大学, 2018.
	Xiong H C.Real-time object detection using deep learning[D].Changsha: Hunan University of Computer Science and Technology, 2018.

检测状态	MAP	岬角	防波堤	船舶	岸线	桥梁	雷康
类别减少前	0.297 3	0.385 6	0.393 1	0.226 0	0.498 6	0.435 6	0.065 0
类别减少后	0.364 9	0.417 4	NaN	NaN	0.513 2	0.504 2	NaN

物标种类	MAP	桥梁	陆地	岬角
精确度/%	0.517 8	0.534 8	0.584 7	0.433 8

[1]	韩啸, 陈世文, 陈蒙, 杨锦程. 基于互易点学习的LPI信号开集识别[J]. 系统工程与电子技术, 2022, 44(9): 2752-2759.
[2]	胡泰洋, 张晋宇, 卢海梁, 李鹏飞, 李一楠, 吕容川. 基于数据融合的分布式综合孔径微波辐射高分辨率成像算法[J]. 系统工程与电子技术, 2022, 44(8): 2403-2409.
[3]	张立民, 谭凯文, 闫文君, 张聿远. 基于多级跳线残差网络的雷达辐射源识别[J]. 系统工程与电子技术, 2022, 44(7): 2148-2156.
[4]	金国栋, 薛远亮, 谭力宁, 许剑锟. 基于孪生神经网络的目标跟踪算法进展研究[J]. 系统工程与电子技术, 2022, 44(6): 1805-1822.
[5]	赵晓枫, 徐叶斌, 吴飞, 牛家辉, 蔡伟, 张志利. 基于全局感知机制的地面红外目标检测方法[J]. 系统工程与电子技术, 2022, 44(5): 1461-1467.
[6]	邹虹, 白陈阳, 何鹏, 崔亚平, 王汝言, 吴大鹏. 基于分布式深度学习的边缘服务放置策略[J]. 系统工程与电子技术, 2022, 44(5): 1728-1737.
[7]	陈冬, 句彦伟. 基于语义分割实现的SAR图像舰船目标检测[J]. 系统工程与电子技术, 2022, 44(4): 1195-1201.
[8]	孙晶明, 虞盛康, 孙俊. 基于深度学习的HRRP识别姿态敏感性分析[J]. 系统工程与电子技术, 2022, 44(3): 802-807.
[9]	姚云翔, 陈莹. 注意力机制下双模态交互融合的目标跟踪网络[J]. 系统工程与电子技术, 2022, 44(2): 410-419.
[10]	高涌荇, 王旭东, 汪玲, 朱岱寅, 郭军, 孟凡旺. 基于RCNN的双极化气象雷达天气信号检测[J]. 系统工程与电子技术, 2022, 44(11): 3380-3387.
[11]	周毅恒, 杨军, 夏赛强, 吕明久. 闪烁现象下旋翼目标微动参数估计方法[J]. 系统工程与电子技术, 2022, 44(1): 54-63.
[12]	付蓉, 黄天耀, 刘一民. 基于深度学习的捷变相参雷达1-Bit块稀疏重构[J]. 系统工程与电子技术, 2022, 44(1): 70-75.
[13]	张聿远, 张立民, 闫文君. 基于互相关特征图和扩张稠密卷积网络的SFBC-OFDM识别方法[J]. 系统工程与电子技术, 2021, 43(9): 2657-2664.
[14]	张路路, 陈思雅, 金光. 航天器抗辐射等效试验评估建模[J]. 系统工程与电子技术, 2021, 43(9): 2673-2677.
[15]	王彩云, 李阳雨, 李晓飞, 王佳宁, 魏文怡. 基于稀疏性深度学习的航拍图像超分辨重构[J]. 系统工程与电子技术, 2021, 43(8): 2045-2050.