基于异源影像匹配的无人机在线快速定位方法

doi:10.12305/j.issn.1001-506X.2023.10.02

摘要/Abstract

摘要：

卫星导航定位是支撑无人机(unmanned aerial vehicle, UAV)安全飞行的关键技术之一, 当卫星信号弱或受干扰时, 方法定位将影响甚至危及UAVs正常飞行。视觉方法可通过影像匹配实现UAVs定位, 但异源图像差异大, 现有特征匹配方法定位的鲁棒性和实时性不能满足UAVs应用需求。提出了一种快速特征匹配定位方法, 采用残差网络提取多尺度鲁棒特征, 通过最小欧氏距离加速低分辨率特征图粗匹配, 并在高分辨率特征图上结合小型Transformer实现精细匹配, 最后利用单应矩阵校正匹配点对实现定位, 改善了UAVs定位的鲁棒性和实时性。经在武汉城郊区域实飞数据集上测试, 所提方法平均定位精度为2.86 m, 与现有典型匹配算法相比提升了2.1%, 同时在定位鲁棒性和计算速度上具有明显优势。算法在Jetson Xavier NX上完成数据集内所有影像的匹配定位, 且定位频率达到最优(1 Hz)。

关键词: 无人机, 快速视觉定位, 异源影像匹配, 残差模块, Transformer

Abstract:

Satellite navigation and positioning is one of the critical modules to ensure unmanned aerial vehicle (UAV) flight safety. When the satellite signal is weak or interfered, failure location will affect or even endanger the normal flight of UAVs. Vision-based methods can be used to locate UAVs with image matching. However, current image matching methods are unable to extract robust features due to the great difference between heterogenous images in time, which results in inadequate accuracy and efficiency. This paper proposes a fast feature matching method, which uses residual network to extract multiscale robust features and accelerates coarse matching of low-resolution feature maps using minimum Euclidean distance. The module above is used on high-resolution feature maps to achieve secondary fine matching. The homography matrix is used to correct matching pairs and achieve UAVs localization. The proposed method improves robustness and efficiency of localization for UAVs. The experimental results with Wuhan Suburb actural dataset show that the average accuracy of the proposed method is 2.86 m, which is 2.1% higher than the current typical matching algorithm. This method has obvious advantages on localization robustness and computing speed, which completes all images matching and localization on Jetson Xavier NX, while the localization frequency is optimal with the frequency of 1 Hz.

Key words: unmanned aerial vehicle (UAV), fast visual localization, heterogenous image matching, residual module, Transformer

中图分类号:

TP391
TP751

眭海刚, 李嘉杰, 苟国华. 基于异源影像匹配的无人机在线快速定位方法[J]. 系统工程与电子技术, 2023, 45(10): 3008-3015.

Haigang SUI, Jiajie LI, Guohua GOU. Online fast localization method of UAVs based on heterologous image matching[J]. Systems Engineering and Electronics, 2023, 45(10): 3008-3015.

图/表 11

图1

图2

图3

图4

图5

图6

表1

表2

图7

表3

表4

参考文献 33

1	GYAGENDA N , HATILIMA J V , ROTH H , et al. A review of GNSS-independent UAV navigation techniques[J]. Robotics and Autonomous Systems, 2022, 152, 104069. doi: 10.1016/j.robot.2022.104069
2	REZWAN S , CHOI W . Artificial intelligence approaches for UAV navigation: recent advances and future challenges[J]. IEEE Access, 2022, 10, 26320- 26339. doi: 10.1109/ACCESS.2022.3157626
3	COUTURIER A , AKHLOUFI M A . A review on absolute vi-sual localization for UAV[J]. Robotics and Autonomous Systems, 2021, 135, 103666. doi: 10.1016/j.robot.2020.103666
4	KAZEROUNI I A , FITZGERALD L , DOOLY G , et al. A survey of state-of-the-art on visual SLAM[J]. Expert Systems with Applications, 2022, 205, 117734. doi: 10.1016/j.eswa.2022.117734
5	JIA G W , LI X Y , ZHANG D M , et al. Visual-SLAM classical framework and key techniques: a review[J]. Sensors, 2022, 22 (12): 4582. doi: 10.3390/s22124582
6	SHAN M, WANG F, LIN F, et al. Google map aided visual navigation for UAVs in GPS-denied environment[C]//Proc. of the IEEE International Conference on Robotics and Biomimetics, 2015: 114-119.
7	COUTURIER A , AKHLOUFI M A . A review on absolute vi-sual localization for UAV[J]. Robotics and Autonomous Systems, 2021, 135, 103666. doi: 10.1016/j.robot.2020.103666
8	AL SAID N , GORBACHEV Y , AVDEENKO A . An unmanned aerial vehicles navigation system on the basis of pattern recognition applications—review of implementation options and prospects for development[J]. Software: Practice and Experience, 2021, 51 (7): 1509- 1517. doi: 10.1002/spe.2964
9	MUGHAL M H , KHOKHAR M J , SHAHZAD M . Assisting UAV localization via deep contextual image matching[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14, 2445- 2457. doi: 10.1109/JSTARS.2021.3054832
10	MA J Y , JIANG X Y , FAN A X , et al. Image matching from handcrafted to deep features: a survey[J]. International Journal of Computer Vision, 2021, 129 (1): 23- 79. doi: 10.1007/s11263-020-01359-2
11	CHEN L , ROTTENSTEINER F , HEIPKE C . Feature detection and description for image matching: from hand-crafted design to deep learning[J]. Geo-spatial Information Science, 2021, 24 (1): 58- 74. doi: 10.1080/10095020.2020.1843376
12	陈世伟, 夏海, 杨小冈, 等. 基于风格迁移不变特征的SAR与光学图像配准算法[J]. 系统工程与电子技术, 2022, 44 (5): 1536- 1542.
	CHEN S W , XIA H , YANG X G , et al. SAR and optical image registration algorithm based on style transfer invariable fea-tures[J]. Systems Engineering and Electronics, 2022, 44 (5): 1536- 1542.
13	YANG W , XU C , MEI L Y , et al. LPSO: multi-source image matching considering the description of local phase sharpness orientation[J]. IEEE Photonics Journal, 2022, 14 (1): 1- 9.
14	SARLIN P E, DETONE D, MALISIEWICZ T, et al. SuperGlue: learning feature matching with graph neural networks[C]// Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 4938-4947.
15	ROCCO I , CIMPOI M , ARANDJELOVIC R , et al. Ncnet: neighbourhood consensus networks for estimating image corres- pondences[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2020, 44 (2): 1020- 1034.
16	EFE U, INCE K G, ALATAN A. Dfm: a performance baseline for deep feature matching[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 4284-4293.
17	ONO Y, TRULLS E, FUA P, et al. LF-Net: learning local features from images[C]//Proc. of the 32nd International Conference on Neural Information Processing Systems, 2018: 6237-6247.
18	DUSMANU M, ROCCO I, PAJDLA T, et al. D2-net: a trainable CNN for joint description and detection of local features[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 8092-8101.
19	CHEN H K, LUO Z X, ZHANG J H, et al. Learning to match features with seeded graph matching network[C]//Proc. of the IEEE/CVF International Conference on Computer Vision, 2021: 6301-6310.
20	REVAUD J, LEROY V, WEINZAEPFEL P, et al. PUMP: pyramidal and uniqueness matching priors for unsupervised learning of local descriptors[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 3926-3936.
21	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Proc. of the 31st International Conferenceon Neural Information Processing Systems, 2017: 6000-6010.
22	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proc. of the IEEE/CVF Confe- rence on Computer Vision and Pattern Recognition, 2016: 770-778.
23	韩子硕, 王春平, 付强, 等. 基于超密集特征金字塔网络的SAR图像舰船检测[J]. 系统工程与电子技术, 2020, 42 (10): 2214- 2222.
	HAN Z S , WANG C P , FU Q , et al. Ship detection in SAR images based on super dense feature pyramid networks[J]. Systems Engineering and Electronics, 2020, 42 (10): 2214- 2222.
24	KATHAROPOULOS A, VYAS A, PAPPAS N, et al. Transformers are RNNs: fast autoregressive transformers with linear attention[C]//Proc. of the International Conference on Machine Learning, 2020, 119: 5156-5165.
25	NIBALI A, HE Z, MORGAN S, et al. Numerical coordinate regression with convolutional neural networks[EB/OL]. [2022-09-06]. https://arxiv.org/abs/1801.07372v2.
26	LI Z Q, SNAVELY N. Megadepth: learning single-view depth prediction from internet photos[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 2041-2050.
27	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 2980-2988.
28	FISCHLER M A , BOLLES R C . Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24 (6): 381- 395. doi: 10.1145/358669.358692
29	BALNTAS V, LENC K, VEDALDI A, et al. HPatches: a benchmark and evaluation of handcrafted and learned local descriptors[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017: 5173-5182.
30	LOWE D G . Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60 (2): 91- 110. doi: 10.1023/B:VISI.0000029664.99615.94
31	DETONE D, MALISIEWICZ T, RABINOVICH A. Superpoint: self-supervised interest point detection and description[C]// Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2018: 224-236.
32	SUN J M, SHEN Z H, WANG Y, et al. LoFTR: detector-free local feature matching with transformers[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 8922-8931.
33	ZHOU Q J, SATTLER T, LEAL-TAIXE L. Patch2pix: epipolar-guided pixel-level correspondences[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 4669-4678.

方法	匹配点数量
LoFTR	2 937
SuperGlue	503
Patch2Pix	1 347
SIFT	610
本文所提算法	3 139

方法	1像素	3像素	5像素
LoFTR	58.8(67.8)	80.2(86.4)	85.9(90.9)
SuperGlue	43.8(55.6)	76.5(87.4)	84.6(93.4)
Patch2pix	34.9(50.7)	67.0(82.2)	78.5(89.9)
SIFT	35.9(44.1)	61.5(72.0)	71.0(83.2)
本文所提算法	52.6(80.1)	71.6(91.3)	81.1(95.5)

方法	平均误差/m	最大误差/m	匹配个数
LoFTR/m	2.92	10.05	870/910
SuperGlue/m	3.04	10.32	850/910
Patch2Pix/m	2.95	10.59	910/910
Proposed/m	2.86	10.04	910/910

方法	时间
LoFTR	1.6
SuperGlue	1.0
Patch2Pix	2.3
本文所提算法	1.0

[1]	王中宝, 尹奎英. 基于联合域滤波的无人机载SAR图像块效应抑制方法[J]. 系统工程与电子技术, 2023, 45(9): 2768-2776.
[2]	张洪海, 任真苹, 冯讴歌, 王非, 刘皞. 城市低空物流无人机飞行计划预先调配[J]. 系统工程与电子技术, 2023, 45(9): 2802-2811.
[3]	梁玉峰, 赵景朝, 刘旺魁, 王雷, 王世鹏, 阮仕龙. 基于顶层滚动优化和底层跟踪的空战导引方法[J]. 系统工程与电子技术, 2023, 45(9): 2866-2872.
[4]	韩蕙竹, 黄仰超, 胡航, 安琪, 刘世豪. 无人机通信中基于短包传输的能/谱效折中优化[J]. 系统工程与电子技术, 2023, 45(9): 2956-2964.
[5]	齐小刚, 周雨桐, 刘立芳. 无人机集群对地作战任务可靠性评估[J]. 系统工程与电子技术, 2023, 45(9): 2971-2978.
[6]	刘子昌, 白永生, 李思雨, 贾希胜. 基于小波时频图与Swin Transformer的柴油机故障诊断方法[J]. 系统工程与电子技术, 2023, 45(9): 2986-2998.
[7]	盛磊, 时满红, 亓迎川, 李浩, 庞明军. 基于态势演化博弈的无人机集群动态攻防[J]. 系统工程与电子技术, 2023, 45(8): 2332-2342.
[8]	许彤, 陈亚洲, 王玉明, 赵敏. 无人机数据链宽带白噪声电磁干扰效应研究[J]. 系统工程与电子技术, 2023, 45(7): 1965-1973.
[9]	吴立尧, 苏析超, 王垒, 潘子双. 有人/无人机编队队形集结控制研究[J]. 系统工程与电子技术, 2023, 45(7): 2192-2202.
[10]	吴冯国, 陶伟, 李辉, 张建伟, 郑成辰. 基于深度强化学习算法的无人机智能规避决策[J]. 系统工程与电子技术, 2023, 45(6): 1702-1711.
[11]	刘瑶, 夏阳升, 石建迈, 陈超, 黄金才. 车载多无人机协同多区域覆盖路径规划方法[J]. 系统工程与电子技术, 2023, 45(5): 1380-1390.
[12]	杨小草, 都延丽, 步雨浓, 刘燕斌, 高程. 基于层次分解的在线三维RRT^*协同航路规划[J]. 系统工程与电子技术, 2023, 45(5): 1409-1419.
[13]	唐进, 梁彦刚, 白志会, 黎克波. 基于DQN的旋翼无人机着陆控制算法[J]. 系统工程与电子技术, 2023, 45(5): 1451-1460.
[14]	庞阳, 王明, 闫子仪, 岳童尧, 周哲. 多视图融合的无人机定位方法[J]. 系统工程与电子技术, 2023, 45(4): 1127-1133.
[15]	白嘉琪, 王彦恺, 邢昊. 无人艇与四旋翼无人机固定时间异构编队控制[J]. 系统工程与电子技术, 2023, 45(4): 1152-1163.