基于无锚框的红外多类别多目标实时跟踪网络

doi:10.12305/j.issn.1001-506X.2022.02.06

摘要/Abstract

摘要：

本文提出一种改进的红外多类别多目标实时跟踪网络, 在确保跟踪精度的同时, 重新设计无锚框网络结构, 进一步降低网络的参数量与推理时间。通过优化目标特征向量, 进一步提高识别精度, 同时简化与改进跟踪流程。此外, 通过细化分析相关流程执行时间, 选用GPU与CPU分别执行最优运算, 提升跟踪整体运行速度。上述方法被应用于低空海面红外目标跟踪数据集中。结果表明, 在本文所提的综合评价指标下, 所设计的网络相较其他轻量级网络评分提高1.78, 且运行速度在NVIDIA Jetson Xavier NX中达到52.37 FPS, 满足边缘端实时运行需求。

关键词: 多目标跟踪, 无锚框, 红外目标, 多类别, 边缘端实时, 重识别

Abstract:

In this paper, an improved real-time infrared multi-class multi-target tracking network is proposed, which not only ensures the tracking accuracy, but also redesigns the anchor-free network structure to further reduce the number of network parameters and the network inference latency. By optimizing the target feature vector, the recognition accuracy is further improved, and the tracking process is simplified and developed. In addition, through detailed analysis of the execution time of related processes, GPU and CPU are selected to perform the optimal operation respectively to improve the overall tracking speed. The above method is applied to the low altitude sea surface infrared target tracking dataset. The results show that under the comprehensive evaluation metric proposed in this paper, the score is increased by 1.78 and the running speed of the algorithm reaches 52.37 FPS in NVIDIA Jetson Xavier NX compared with other lightweight network, which meets the real-time operation requirements of edge devices.

Key words: multi-target tracking, anchor-free, infrared target, multi-class, real time in edge devices, re-identification

中图分类号:

TP391.4

宋子壮, 杨嘉伟, 张东方, 王诗强, 张硕. 基于无锚框的红外多类别多目标实时跟踪网络[J]. 系统工程与电子技术, 2022, 44(2): 401-409.

Zizhuang SONG, Jiawei YANG, Dongfang ZHANG, Shiqiang WANG, Shuo ZHANG. Real-time infrared multi-class multi-target anchor-free tracking network[J]. Systems Engineering and Electronics, 2022, 44(2): 401-409.

图/表 14

图1

图2

图3

图4

图5

表1

图6

表2

表3

表4

表5

表6

表7

图7

参考文献 31

1	王法胜, 鲁明羽, 赵清杰, 等. 粒子滤波算法[J]. 计算机学报, 2014, 37 (8): 1679- 1694.
	WANG F S , LU M Y , ZHAO Q J , et al. Particle filtering algorithm[J]. Chinese Journal of Computers, 2014, 37 (8): 1679- 1694.
2	COLLINS R T. Mean-shift blob tracking through scale space[C]// Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. DOI: 10.1109/CVPR.2003.1211475.
3	WELCH G, BISHOP G. An introduction to the Kalman filter[C]// Proc. of the ACM Special Interest Group on Computer Graphics and Interactive Techniques, 2001.
4	BOLME D S, BEVERIDGE J R, DRAPER B A, et al. Visual object tracking using adaptive correlation filters[C]//Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010: 2544-2550.
5	HENRIQUES J F, CASEIRO R, MARTINS P, et al. Exploiting the circulant structure of tracking-by-detection with kernels[C]// Proc. of the European Conference on Computer Vision, 2012: 702-715.
6	HENRIQUES J F , CASEIRO R , MARTINS P , et al. High-speed tracking with kernelized correlation filters[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2014, 37 (3): 583- 596.
7	杜若鹏, 张磊, 卢杨. 上下文感知相关滤波的红外目标跟踪改进算法[J]. 激光与红外, 2020, 50 (7): 839- 845.
	DU R P , ZHANG L , LU Y . Improved infrared target tracking algorithm based on context-aware correlation filter[J]. Laser & Infrared, 2020, 50 (7): 839- 845.
8	DANELLJAN M, ROBINSON A, SHAHBAZ K F, et al. Beyond correlation filters: learning continuous convolution operators for visual tracking[C]//Proc. of the European Conference on Computer Vision, 2016: 472-488.
9	王承赟, 张龙杰, 李相民, 等. 基于改进的核相关滤波算法的红外目标跟踪[J]. 电光与控制, 2021, 28 (7): 9- 13.
	WANG C Y , ZHANG L J , LI X M , et al. Infrared target tracking based on improved kernel correlation filter algorithm[J]. Electronics Optics & Control, 2021, 28 (7): 9- 13.
10	宋国鹏. 基于孪生网络的单目标跟踪算法研究及其应用[D]. 北京: 北京交通大学, 2020.
	SONG G P. Research and application of single target tracking algorithm based on siamese network[D]. Beijing: Beijing Jiaotong University, 2020.
11	柳赟, 孙淑艳. 基于自适应模板更新的改进孪生卷积网络目标跟踪算法[J]. 计算机应用与软件, 2021, 38 (4): 145- 151, 230. doi: 10.3969/j.issn.1000-386x.2021.04.024
	LIU Y , SUN S Y . Object tracking algorithm based on improved siamese convolutional networks combined with adaptive template updating[J]. Computer Applications and Software, 2021, 38 (4): 145- 151, 230. doi: 10.3969/j.issn.1000-386x.2021.04.024
12	DANELLJAN M, BHAT G, SHAHBAZ K F, et al. Eco: efficient convolution operators for tracking[C]//Proc. of the Conference on Computer Vision and Pattern Recognition, 2017: 6638-6646.
13	BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional siamese networks for object tracking[C]//Proc. of the European Conference on Computer Vision, 2016: 850-865.
14	LI B, YAN J, WU W, et al. High performance visual tracking with siamese region proposal network[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 8971-8980.
15	ZHU Z, WANG Q, LI B, et al. Distractor-aware siamese networks for visual object tracking[C]//Proc. of the European Conference on Computer Vision, 2018: 101-117.
16	LI B, WU W, WANG Q, et al. SiamRPN++: evolution of siamese visual tracking with very deep networks[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 4282-4291.
17	BEWLEY A, GE Z, OTT L, et al. Simple online and realtime tracking[C]//Proc. of the IEEE International Conference on Image Processing, 2016: 3464-3468.
18	WOJKE N, BEWLEY A, PAULUS D. Simple online and realtime tracking with a deep association metric[C]//Proc. of the IEEE International Conference on Image Processing, 2017: 3645-3649.
19	WANG Z D, ZHENG L, LIU Y X, et al. Towards real-time multi-object tracking[C]// Proc. of the European Conference on Computer Vision, 2020: 107-122.
20	ZHANG Y F , WANG C Y , WANG X G , et al. FairMOT: on the fairness of detection and re-identification in multiple object tracking[J]. arXiv preprint, 2020,
21	ZHOU X , WANG D . KRÄHENBVHL P. Objects as points[J]. arXiv preprint, 2019, arXiv, 1904.07850.
22	李震霄, 孙伟, 刘明明, 等. 交通监控场景中的车辆检测与跟踪算法研究[J]. 计算机工程与应用, 2021, 57 (8): 103- 111.
	LI Z X , SUN W , LIU M M , et al. Research on vehicle detection and tracking algorithms in traffic monitoring scenes[J]. Computer Engineering and Applications, 2021, 57 (8): 103- 111.
23	赵朵朵, 章坚武, 傅剑峰. 基于深度学习的实时人流统计方法研究[J]. 传感技术学报, 2020, 33 (8): 1161- 1168. doi: 10.3969/j.issn.1004-1699.2020.08.013
	ZHAO D D , ZHANG J W , FU J F . Research on real-time statistics of people flow based on deep learning[J]. Chinese Journal of Sensors and Actuators, 2020, 33 (8): 1161- 1168. doi: 10.3969/j.issn.1004-1699.2020.08.013
24	张宏鸣, 汪润, 董佩杰, 等. 基于DeepSORT算法的肉牛多目标跟踪方法[J]. 农业机械学报, 2021, 52 (4): 248- 256.
	ZHANG H M , WANG R , DONG P J , et al. Beef cattle multi-target tracking based on DeepSORT algorithm[J]. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52 (4): 248- 256.
25	MILAN A , LEAL-TAIXÉ L , REID I , et al. MOT16: a benchmark for multi-object tracking[J]. arXiv preprint, 2016, arXiv, 1603.00831.
26	DING X, ZHANG X, MA N, et al. RepVGG: making VGG-style ConvNets great again[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 13733-13742.
27	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
28	SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint, 2014, arXiv, 1409.1556.
29	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2117-2125.
30	LUO H, GU Y, LIAO X, et al. Bag of tricks and a strong baseline for deep person re-identification[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019.
31	KENDALL A, GAL Y, CIPOLLA R. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7482-7491.

处理单元	FP	RD	TT	总时间
GPU+CPU	28.26	81.39	7.09	116.74
GPU	28.26	48.93	17.56	94.75

网络结构	MOTA	IDs	IDF1	MT	ML	Score
ResNet18+DCN	59.52	361	63.15	38/71	10/71	65.53
ResNet18+DFPN	63.47	306	65.64	40/71	9/71	68.19
HRNet18	66.51	227	68.81	41/71	7/71	70.80
DLA34	67.18	221	69.26	42/71	7/71	71.43
RepVGG+DCN(Deploy)	62.03	284	64.04	38/71	9/71	66.73
RepVGG+FFPN(Train)	66.24	267	66.72	40/71	8/71	69.51
RepVGG+FFPN(Deploy)	66.24	267	66.72	40/71	8/71	69.51

网络结构	参数量 (×10⁶)	计算复杂度/ GFLOPs	网络推理时间/ms
ResNet18+DCN	15.46	58.74	28.68
ResNet18+DFPN	16.72	60.24	35.08
DLA34	19.84	72.12	63.89
HRNet18	11.62	127.54	184.12
RepVGG+DCN(Deploy)	13.51	63.99	32.62
RepVGG+FFPN(Train)	13.02	61.06	47.32
RepVGG+FFPN(Deploy)	11.91	57.64	28.26

改进方法	MOTA	IDs	IDF1	MT	ML	Score
CE	66.24	267	66.72	40/71	8/71	69.51
CE+Triplet	67.01	232	68.44	41/71	6/71	71.19
LSCE+Triplet	67.17	226	69.54	42/71	6/71	71.85
LSCE+Triplet+Center	67.24	190	70.18	42/71	6/71	72.03
LSCE+Triplet+Center+BNNeck	68.23	176	71.56	42/71	6/71	72.62

跟踪方法	MOTA	IDs	IDF1	MT	ML	Score	TT/ms
特征向量+IoU+卡尔曼滤波	62.19	6 952	17.08	35/71	8/71	54.32	25.65
特征向量+IoU	68.23	176	71.56	42/71	6/71	72.62	7.09
特征向量	69.18	266	66.51	44/71	5/71	72.65	5.83
改进特征向量(C++)	69.73	253	67.45	46/71	5/71	73.73	1.29