基于持续无监督域适应策略的水面漂浮物目标检测方法

doi:10.12305/j.issn.1001-506X.2023.11.04

摘要/Abstract

摘要：

针对漂浮物检测中小尺度目标和域转移问题, 提出一种基于持续无监督域适应策略的漂浮物检测方法。该方法通过删除低分辨率特征图, 增强高分辨率特征图, 提升小尺度漂浮物的特征提取能力。同时, 该方法整合无监督域适应、缓冲区和样本重放, 降低应用场景中不断变化的域转移差异。并将改进检测网络与持续无监督域适应相结合, 提升模型检测精度和泛化能力。通过在漂浮物数据集上实验验证, 对比现有方法, 该方法的检测精度达到82.2%, 检测速度达到68.5 f/s, 浮点数的算量减少到33亿, 模型大小达到25.3 MB, 扩展了目标检测在水面视觉中的应用。

关键词: 深度学习, 水面漂浮物, 目标检测, 无监督域适应, 持续学习

Abstract:

For small-scale targets and domain transfer problems, a method based on a continuous unsupervised domain adaptation strategy is proposed. By removing low-resolution feature maps and enhancing high-resolution feature maps, the method improves the ability of small-scale floaters to extract features. This study proposes a continuous unsupervised domain adaptation method that integrates unsupervised domain adaptation, buffering, and sample replay to reduce the constantly varying domain transfer variance in application scenarios. Meanwhile, this study combines the improved detection network with continual unsupervised domain adaption to improve model detection precision and generalization capabilities. Through the experimental verification on the data set of the floating targets, compared with the mainstream methods, the detection accuracy of the proposed method reaches 82.2%, the detection speed can reach 68.5 f/s, the computation amount of floating-point numbers reaches 3.3 billion, and the size of the model reaches 25.3 MB. This study extends the application of object detection in water surface vision.

Key words: deep learning, floating materials, object detection, unsupervised domain adaptation, continual learning

中图分类号:

陈任飞, 彭勇, 李忠文. 基于持续无监督域适应策略的水面漂浮物目标检测方法[J]. 系统工程与电子技术, 2023, 45(11): 3391-3401.

Renfei CHEN, Yong PENG, Zhongwen LI. A novel detector for floating objects based on continual unsupervised domain adaptation strategy[J]. Systems Engineering and Electronics, 2023, 45(11): 3391-3401.

图/表 13

图1

图2

图3

图4

表1

表2

图5

图6

表3

图7

图8

图9

表4

参考文献 32

1	WANG X , ZHANG M , LIU L , et al. Using EEM-PARAFAC to identify and trace the pollution sources of surface water with receptor models in taihu lake basin, China[J]. Journal of Environmental Management, 2022, 321, 115925. doi: 10.1016/j.jenvman.2022.115925
2	ZHU Y N , WANG K X , LIN Y X , et al. An online contaminant classification method based on MF-DCCA using conventional water quality indicators[J]. Processes, 2020, 8 (2): 178. doi: 10.3390/pr8020178
3	RUANGPAYOONGSAK N, SUMROENGRIT J, LEANGLUM M. A floating waste scooper robot on water surface[C]// Proc. of the International Conference on Control, Automation and Systems (ICCAS), 2017: 1543-1548.
4	GOUTAM M A , RAMESH W U , CHAKRABORTY R , et al. A review on modern and smart technologies for efficient waste disposal and management[J]. Journal of Environmental Management, 2021, 297, 113347. doi: 10.1016/j.jenvman.2021.113347
5	ZHOU Z G , SUN J E , YU J B , et al. An image-based benchmark dataset and a novel object detector for water surface object detection[J]. Frontiers in Neurorobotics, 2021,
6	REN S Q, HE K M, GIRSHICK R, et al. Faster RCNN: towards real-time object detection with region proposal networks[C]// Proc. of the IEEE International Conference on Neural Information Processing Systems, 2015: 91-99.
7	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]//Proc. of the IEEE Conference on Computer Vision And Pattern Recognition, 2016: 779-788.
8	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[C]//Proc. of the 14th European Conference on Computer Vision, 2016: 21-37.
9	CHENG Y , JIANG M , ZHU J , et al. Are we ready for unmanned surface vehicles in inland waterways? The USV in land multisensor dataset and benchmark[J]. IEEE Robotics and Automation Letters, 2021, 6 (2): 3964- 3970. doi: 10.1109/LRA.2021.3067271
10	BAI Y, ZHANG Y, DING M, et al. SOD-MTGAN: small object detection via multi-task generative adversarial network[C]// Proc. of the Computer Vision, 2018: 210-226.
11	TSUNG Y L, PIOTR D, ROSS G, et al. Feature pyramid networks for object detection[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2117-2125.
12	LI J, LIANG X, WEI Y, et al. Perceptual generative adversarial networks for small object detection[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017: 1951-1959.
13	LIU Z , LIU J , ZUO X , et al. Multi-scale iterative refinement network for RGB-D salient object detection[J]. Engineering Applications of Artificial Intelligence, 2021, 106, 104473. doi: 10.1016/j.engappai.2021.104473
14	汪云云, 孙顾威, 赵国祥, 等. 基于自监督知识的无监督新集域适应学习[J]. 软件学报, 2022, 33 (4): 1170- 1182.
	WANG Y Y , SUN G W , ZHAO G X , et al. Unsupervised new-set domain adaptation with self-supervised knowledge[J]. Journal of Software, 2022, 33 (4): 1170- 1182.
15	LI X , YE M , LIU Y , et al. Adaptive deep convolutional neural networks for scene-specific object detection[J]. IEEE Trans.on Circuits and Systems for Video Technology, 2019, 29 (9): 2538- 2551. doi: 10.1109/TCSVT.2017.2749620
16	JIA Y , ZHANG J , SHAN S , et al. Unified unsupervised and semi-supervised domain adaptation network for cross-scenario face anti-spoofing[J]. Pattern Recognition, 2021, 115, 107888. doi: 10.1016/j.patcog.2021.107888
17	CHEN Y, LI W, SAKARIDIS C, et al. Domain adaptive faster R-CNN for object detection in the wild[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 3339-3348.
18	JIANG N , FANG J , XU J , et al. SSD based on contour-material level for domain adaptation[J]. Pattern Analysis and Applications, 2021, 24 (3): 1221- 1229. doi: 10.1007/s10044-021-00986-w
19	XIONG L , YE M , ZHANG D , et al. Source data-free domain adaptation for a faster R-CNN[J]. Pattern Recognition, 2022, 124, 108436. doi: 10.1016/j.patcog.2021.108436
20	KIM T, JEONG M, KIM S, et al. Diversify and match: a domain adaptive representation learning paradigm for object detection[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019: 12448-12457.
21	范苍宁, 刘鹏, 肖婷, 等. 深度域适应综述: 一般情况与复杂情况[J]. 自动化学报, 2021, 47 (3): 515- 548.
	FAN C N , LIU P , XIAO T , et al. A review of deep domain adaptation: general situation and complex situation[J]. Acta Automatica Sinica, 2021, 47 (3): 515- 548.
22	TAUFIQUE A M, JAHAN C S, SAVAKIS A E. ConDA: continual unsupervised domain adaptation[EB/OL]. [2022-09-18]. https://arxiv.org/abs/2103.11056.
23	HOFFMAN J, DARRELL T, SAENKO K. Continuous manifold based adaptation for evolving visual domains[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2014: 867-874.
24	WULFMEIER M, BEWLEY A, POSNER I. Incremental adversarial domain adaptation for continually changing environments[C]//Proc. of the IEEE International Conference on Robotics and Automation (ICRA), 2018: 4489-4495.
25	VOLPI R, LARLUS D, ROGEZ G. Continual adaptation of visual representations via domain randomization and meta-learning[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021: 4441-4451.
26	LENGA M, SCHULZ H, SAALBACH A. Continual learning for domain adaptation in chest x-ray classification[EB/OL]. [2022-09-18]. https://arxiv.org/abs/2001.05922.
27	魏文晓, 刘洁瑜, 沈强, 等. 基于人眼视点图的特征融合小目标检测算法[J]. 系统工程与电子技术, 2022, 44 (4): 1120- 1127. doi: 10.12305/j.issn.1001-506X.2022.04.07
	WEI W X , LIU J Y , SHEN Q , et al. Feature fusion small target detection algorithm based on human eye view-point map[J]. Systems Engineering and Electronics, 2022, 44 (4): 1120- 1127. doi: 10.12305/j.issn.1001-506X.2022.04.07
28	FISHER Y, VLADLEN K. Multi-scale context aggregation by dilated convolutions[EB/OL]. [2022-09-18]. https://arxiv.org/arXiv:1511.07122.
29	LIANG J, HU D, FENG J. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation[EB/OL]. [2022-09-18]. https://arxiv.org/abs/2002.08546.
30	ANDREAS K, PIETRO P, RYAN G. Discriminative clustering by regularized information maximization[C]//Proc. of the 23rd International Conference on Neural Information Processing Systems, 2010: 775-783.
31	HU W, MIYATO T, TOKUI S, et al. Learning discrete representations via information maximizing self-augmented training[C]// Proc. of the 34th International Conference on Machine Learning, 2017: 1558-1567.
32	JIANG J, CHEN B, WANG J, et al. Decoupled adaptation for cross-domain object detection[EB/OL]. [2022-09-18]. https://arxiv.org/abs/2110.02578.

方法	渔船	水葫芦	漂浮杂草	塑料瓶	mAP/%
Data1→Data2
SSD-FT	63.2	72.3	65.8	66.3	66.9
CM-SSD	73.9	78.2	76.2	69.2	74.4
DA-FRCNN	68.1	77.4	74.1	73.8	73.4
D-adapt	79.8	80.3	76.3	76.4	78.2
CDA-SSD-FT	83.9	87.8	85.6	84.2	85.4

Data2→Data3
SSD-FT	58.3	67.5	66.8	56.3	62.2
CM-SSD	52.2	65.2	64.2	60.2	60.5
DA-FRCNN	63.5	68.3	70.4	66.6	67.2
D-adapt	59.4	72.5	72.5	72.4	69.2
CDA-SSD-FT	79.2	78.4	82.8	79.1	79.9

Data3→Data4
SSD-FT	70.2	72.5	68.3	62.5	68.4
CM-SSD	72.4	68.3	70.4	66.3	69.4
DA-FRCNN	76.9	77.2	73.9	70.5	74.6
D-adapt	71.7	75.1	76.7	72.6	74.0
CDA-SSD-FT	84.9	83.1	81.3	79.4	82.2

Data4→Data5
SSD-FT	72.3	69.2	73.5	69.3	71.0
CM-SSD	68.4	70.3	72.7	71.6	70.8
DA-FRCNN	70.2	73.6	77.5	74.7	74.0
D-adapt	74.9	71.7	74.2	75.9	74.2
CDA-SSD-FT	82.5	79.4	81.3	81.0	81.1

方法	目标域	实验1	实验2	实验3	实验4	mAP/%
SSD	Full¹⁾	53.2	48.9	51.9	48.4	50.6
SSD-FT	Full	66.9	62.2	68.4	71.1	67.2
SSD-FT	Cont²⁾	63.8	64.1	65.3	68.2	65.4
CDA-SSD-FT	Cont	85.4	79.9	82.2	81.1	82.2

实验	删除深层特征图	增强浅层特征图	mAP/%	FPS/(f/s)
实验1	—	—	50.8	11.5
实验2	√	—	52.4	15.3
实验3	—	√	57.2	13.1
本算法	√	√	67.1	25.8

方法	mAP/%	FLOPs/Billion	Model/MB	FPS/(f/s)
CM-SSD	68.8	5.4	35.2	65.2
DA-FRCNN	72.3	11.4	78.3	7.2
D-adapt	73.9	3.4	26.8	66.1
CDA-SSD-FT	82.2	3.3	25.3	68.5
CDA-YOLOv5	70.3	7.8	64.9	10.3

[1]	汪萌, 诸兵. 不确定性建模在2D和3D目标检测中的应用[J]. 系统工程与电子技术, 2023, 45(8): 2370-2376.
[2]	邵凯, 杜自群, 王光宇. 基于模型剪枝动态调整压缩率的CSI反馈方法[J]. 系统工程与电子技术, 2023, 45(8): 2615-2622.
[3]	崔天舒, 王栋, 黄振. 面向空间认知通信的轻量化网络自动调制分类方法[J]. 系统工程与电子技术, 2023, 45(7): 2220-2226.
[4]	姜雨, 袁琪, 胡志韬, 吴薇薇, 顾欣. 基于气象因素的机场进离港延误预测[J]. 系统工程与电子技术, 2023, 45(6): 1722-1731.
[5]	成倩, 李佳, 杜娟. 基于YOLOv5的光学遥感图像舰船目标检测算法[J]. 系统工程与电子技术, 2023, 45(5): 1270-1276.
[6]	朱晶晶, 朱圣棋, 廖桂生, 许京伟, 兰岚, 曾操. 相控阵和频率分集阵双模式雷达联合目标检测[J]. 系统工程与电子技术, 2023, 45(5): 1342-1350.
[7]	杨宇超, 方明, 赵晨帆, 方刚. 高速机动目标长时间相参积累算法[J]. 系统工程与电子技术, 2023, 45(5): 1359-1370.
[8]	张冬冬, 王春平, 付强. 基于特征增强网络的SAR图像舰船目标检测[J]. 系统工程与电子技术, 2023, 45(4): 1032-1039.
[9]	孟自强, 高伟, 李晓明. 机载雷达地面静止目标二维检测算法[J]. 系统工程与电子技术, 2023, 45(4): 1040-1048.
[10]	陈洋, 廖灿辉, 张锟, 刘建, 王鹏举. 基于自监督对比学习的信号调制识别算法[J]. 系统工程与电子技术, 2023, 45(4): 1200-1206.
[11]	张晔, 侯毅, 欧阳克威, 周石琳. 单变量序列数据分类方法综述[J]. 系统工程与电子技术, 2023, 45(2): 313-335.
[12]	邵正途, 许登荣, 徐文利, 王晗中. 基于LSTM和残差网络的雷达有源干扰识别[J]. 系统工程与电子技术, 2023, 45(2): 416-423.
[13]	张昀普, 单甘霖, 黄燕, 付强. 考虑盲区的多移动传感器地面目标检测跟踪调度方法[J]. 系统工程与电子技术, 2023, 45(2): 453-464.
[14]	周剑雄, 朱永锋, 陈冀, 吴宏铭, 吴堃, 张永杰. SAR图像辅助的雷达目标距离像检测识别[J]. 系统工程与电子技术, 2023, 45(11): 3428-3436.
[15]	陶臣嵩, 陈思伟, 肖顺平. 基于深度学习模型的SAR图像间歇采样转发干扰检测[J]. 系统工程与电子技术, 2023, 45(11): 3465-3473.