基于迁移学习和注意力机制的伪装图像分割

doi:10.12305/j.issn.1001-506X.2022.02.03

摘要/Abstract

摘要：

不同于常规目标, 伪装目标特征模糊、尺度信息复杂多变、检测和分割难度更高。在现有伪装数据集基础上, 提出了一种结合迁移学习和有效通道注意力的UNet网络伪装图像分割方法。首先, 针对伪装目标特征模糊难以有效提取的问题, 在UNet的下采样和上采样过程中, 引入一种有效通道注意力机制, 在不增加网络参数的同时, 提高有效区域的特征权重; 并将在ImageNet预训练好的视觉几何组(visual geometry group, VGG)系列网络迁移到UNet网络中, 实现特征迁移和参数共享, 提高模型的泛化能力, 降低训练效果对数据集的依赖, 减少训练成本; 在训练过程中引入FocalLoss函数, 增加难挖掘样本权重, 提高对困难样本关注度; 最后通过解码网络得到分割结果。在CHAMELEON、CAMO和COD10K数据集上进行了测试, 相比原始算法, 性能指标有显著提升。

关键词: 伪装图像, 图像分割, 注意力机制, 迁移学习

Abstract:

Different from conventional targets, camouflage targets have fuzzy features and complex scale information, resulting in more difficult detection and segmentation. Based on the existing camouflage dataset, a UNet network camouflage image segmentation method combining transfer learning and effective channel attention is proposed. First of all, in view of the problem that the camouflage target feature is difficult to effectively extract, in the down-sampling and up-sampling process of UNet, an effective channel attention mechanism is introduced to increase the feature weight of the effective area without increasing the network parameters. Transfer the visual geometry group (VGG) series network pre-trained on ImageNet to the UNet network to realize feature transfer and parameter sharing, improve the generalization ability of the model, reduce the dependence of the training effect on the dataset, and reduce the training cost. The FocalLoss function is introduced in the training process to increase the weight of difficult samples and increase the attention to difficult samples. Finally get the segmentation through the decoding network result. Tested on the datasets of CHAMELEON, CAMO and COD10K, compared with the original algorithm, the performance indicators of the method proposed have been significantly improved.

Key words: camouflage image, image segmentation, attention mechanism, transfer learning

中图分类号:

TN957.52

吴涛, 王伦文, 朱敬成. 基于迁移学习和注意力机制的伪装图像分割[J]. 系统工程与电子技术, 2022, 44(2): 376-384.

Tao WU, Lunwen WANG, Jingcheng ZHU. Camouflage image segmentation based on transfer learning and attention mechanism[J]. Systems Engineering and Electronics, 2022, 44(2): 376-384.

图/表 18

图1

图2

表1

图3

表2

表3

图4

图5

表4

图6

表5

图7

表6

图8

表7

图9

图10

图11

参考文献 26

1	GRUSON H . Digest: evolution of camouflage patterns in geckos[J]. Evolution, 2020, 74 (6): 1224- 1225. doi: 10.1111/evo.13999
2	CUTHILL I C . Camouflage[J]. Zool, 2019, 308, 75- 92.
3	杨晔, 夏前军, 钱坤, 等. 单兵伪装服及其伪装方法的研究进展[J]. 纺织导报, 2019, (8): 57- 60. doi: 10.3969/j.issn.1003-3025.2019.08.016
	YANG Y , XIA Q K , QIAN K , et al. Research progress of individual camouflage clothing and camouflage methods[J]. Textile Herald, 2019, (8): 57- 60. doi: 10.3969/j.issn.1003-3025.2019.08.016
4	SHELHAMER E , LONG J , DARRELL T . Fully convolutional networks for semantic segmentation[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2017, 39 (4): 640- 651. doi: 10.1109/TPAMI.2016.2572683
5	MO S W , DENG X P , WANG S , et al. Moving object detection algorithm based on improved visual background extractor[J]. Acta Optica Sinica, 2016, 36 (6): 0615001. doi: 10.3788/AOS201636.0615001
6	CARREIRA J C . CPMC: automatic object segmentation using constrained parametric min-cuts[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2012, 34 (7): 1312- 1328. doi: 10.1109/TPAMI.2011.231
7	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2014: 580-587.
8	HARIHARAN B, ARBEL E Z P, GIRSHICK R, et al. Simultaneous detection and segmentation[C]//Proc. of the European Conference on Computer Vision, 2014: 297-312.
9	LONG J , SHELHAMER E , DARRELL T , et al. Fully convolutional networks for semantic segmentation[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2017, 39 (4): 640- 651. doi: 10.1109/TPAMI.2016.2572683
10	BADRINARAYANAN V , KENDALL A , CIPOLLA R . SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2017, 39 (12): 2481- 2495. doi: 10.1109/TPAMI.2016.2644615
11	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proc. of the International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015: 234-241.
12	WANG Z. Theoretical guarantees of transfer learning[EB/OL]. [2021-02-25]. https://arxiv.org/abs/1810.05986V2.
13	HU J, SHEN L, SAMUEL A, et al. Gather-excite: exploiting feature context in convolutional neural networks[EB/OL]. [2021-05-25]. https://arxiv.org/abs/1810.12348V2.
14	WOO S, PARK J, LEE J Y, et al. Cbam: convolutional block attention module[C]//Proc. of the European Conference on Computer Vision, 2018: 3-19.
15	LIU F J, TIA H J. Dual attention network for scene segmentation[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 3146-3154.
16	HU J, SHEN L, SUN G, et al. Squeeze-and-excitation networks[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7132-7141.
17	WANG Q Q, WU B, ZHU P, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proc. of the CVF Conference on Computer Vision and Pattern Recognition, 2020.
18	HU J, SHEN L S, SUN G H, et al. Squeeze-and-excitation networks[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7132-7141.
19	AMIR R Z, ALEXANDER S, SHEN W, et al. Taskonomy: disentangling task transfer learning. In IEEE CVPR, 2018: 3712-3722.
20	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2021-02-25]. https://arxiv.org/abs/14091556.
21	KRIZHEVSKY A , SUTSKEVER I , HINTON G . ImageNet classfication with deep neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25 (2): 381- 394.
22	FAN D P, JI G P, SUN G, et al. Camouflaged object detection[C]// Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 2777-2787.
23	SKUROWSKI P , ABDULAMEER H , BŁASZCZYK J , et al. Animal camouflage analysis: chameleon database[J]. Unpublished Manuscript, 2018, 2 (6): 7- 15.
24	LE T N , NGUYEN T V , NIE Z , et al. Anabranch network for camouflaged object segmentation[J]. Computer Vision and Image Understanding, 2019, 184, 45- 56. doi: 10.1016/j.cviu.2019.04.006
25	LIN T Y . Focal loss for dense object detection[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2020, 42 (2): 318- 327. doi: 10.1109/TPAMI.2018.2858826
26	LIN T Y , WELLS W M , GRIMSON W E L , et al. Adaptive segmentation of MRI data[J]. IEEE Trans.on Medical Imaging, 1996, 15 (4): 429- 442. doi: 10.1109/42.511747

层数	VGG11	VGG13	VGG16	VGG19
第1层		Conv2d	Conv2d	Conv2d
	Conv2d	(3, 64)	(3, 64)	(3, 64)
	(3, 64)	Conv2d	Conv2d	Conv2d
		(3, 64)	(3, 64)	(3, 64)
	MaxPool2d
第2层		Conv2d	Conv2d	Conv2d
	Conv2d	(64, 128)	(64, 128)	(64, 128)
	(64, 128)	Conv2d	Conv2d	Conv2d
		(128, 128)	(128, 128)	(128, 128)
	MaxPool2d
第3层				Conv2d
			Conv2d	(128, 256)
	Conv2d	Conv2d	(128, 256)	Conv2d
	(128, 256)	(128, 256)	Conv2d	(256, 256)
	Conv2d	Conv2d	(256, 256)	Conv2d
	(256, 256)	(256, 256)	Conv2d	(256, 256)
			(256, 256)	Conv2d
				(256, 256)
	MaxPool2d
第4层				Conv2d
			Conv2d	(256, 512)
	Conv2d	Conv2d	(256, 512)	Conv2d
	(512, 512)	(512, 512)	Conv2d	(512, 512)
	Conv2d	Conv2d	(512, 512)	Conv2d
	(512, 512)	(512, 512)	Conv2d	(512, 512)
			(512, 512)	Conv2d
				(512, 512)
	MaxPool2d
第5层				Conv2d
			Conv2d	(512, 512)
	Conv2d	Conv2d	(512, 512)	Conv2d
	(512, 512)	(512, 512)	Conv2d	(512, 512)
	Conv2d	Conv2d	(512, 512)	Conv2d
	(512, 512)	(512, 512)	Conv2d	(512, 512)
			(512, 512)	Conv2d
				(512, 512)

参数	数值
学习率(Learning_rate)	初始学习率1e-3, 在训练100 轮后学习率降为原来的1/10
迭代次数(Epochs)	150
批处理规模(Batch_size)	4

类型	训练集	测试集	合计
CAM	3 040	2 026	5 066
NonCAM	2 960	1 974	4 934
合计	6 000	4 000	10 000

数量		真实类
数量		(positive，P)	(negative，N)
预测类	正确(T)	TP	TN
预测类	错误(F)	FP	FN

γ	CHAMELEON			CAMO			COD10K
γ	PA	Dice	Sensitivity	PA	Dice	Sensitivity	PA	Dice	Sensitivity
0	0.917 6	0.574 2	0.554 0	0.834 1	0.257 9	0.269 0	0.918 8	0.357 7	0.517 4
0.5	0.913 0	0.536 8	0.506 0	0.835 4	0.252 1	0.256 9	0.918 2	0.327 7	0.404 8
1	0.916 6	0.556 4	0.525 4	0.833 7	0.228 5	0.208 0	0.918 9	0.341 2	0.410 8
2	0.930 6	0.626 4	0.630 0	0.831 5	0.312 6	0.329 8	0.917 2	0.326 5	0.532 0
5	0.914 2	0.540 0	0.526 4	0.833 1	0.224 1	0.204 0	0.914 2	0.250 0	0.281 3