基于感知融合机制的渐进式去雾网络

doi:10.12305/j.issn.1001-506X.2023.11.07

摘要/Abstract

摘要：

智能辅助驾驶应用场景对图像去雾的准确性和实时性要求较高。提出了一种新颖的基于感知融合机制的渐进式去雾网络(progressive dehaze network, PD-Net), 将降质图像恢复的任务分解为多阶段的子任务, 通过轻量级的子网分块学习特征图的不同区域语义信息, 以提升去雾效率。在此基础上, 基于注意力机制和导向滤波设计跨阶段感知融合模块(perception fusion module, PFM), 自适应感知各阶段提取的多尺度特征并进行融合, 而不损失图像细节信息及边缘结构信息。实验结果表明, 与现有主流的端对端去雾模型相比, 所提出的算法在处理户外图像时具有更高的准确度和实时性, 在公开的合成对象测试集(synthetic object testing set, SOTS)上的峰值信噪比(peak signal to noise ratio, PSNR)与现有最好结果相比提升了0.93 dB, 处理单幅图像仅需72 ms, 提出的网络模型有望应用于智能交通等现实领域。

关键词: 图像去雾, 注意力机制, 导向滤波, 感知融合

Abstract:

Intelligent assisted driving application scenarios require higher accuracy and real-time performance for image dehazing. This paper proposes a novel progressive dehaze network (PD-Net) based on perceptual fusion mechanism to improve the efficiency and accuracy of image dehaze. This method decomposes the task of degraded image recovery into multi-stage subtasks and uses lightweight subnetwork chunk to learn semantic information of different regions of the feature map to ensure image dehazing efficiency. On this basis, the cross-stage perception fusion module (PFM) is introduced based on the attention mechanism and guided filtering. It adaptively percepts semantic features and fuse the features in a cascading manner without losing image edges and texture details. Experimental results show that compared with the existing mainstream end-to-end dehazing models, the algorithm proposed in this paper has higher accuracy and real-time performance in processing outdoor images, with peaksignal to noise ratio (PSNR) improved by 0.93 dB compared with the best results available on the public synthetic object testing set (SOTS), which consumes only 72 ms when processing a single image.

Key words: image dehaze, attention mechanism, guided filter, perception fusion

中图分类号:

TP39

齐城慧, 张登银. 基于感知融合机制的渐进式去雾网络[J]. 系统工程与电子技术, 2023, 45(11): 3419-3427.

Chenghui QI, Dengyin ZHANG. Progressive image dehaze based on perceptual fusion[J]. Systems Engineering and Electronics, 2023, 45(11): 3419-3427.

图/表 10

图1

图2

图3

图4

图5

表1

表2

表3

图6

图7

参考文献 32

1	郑凤仙, 王夏黎, 何丹丹, 等. 单幅图像去雾算法研究综述[J]. 计算机工程与应用, 2022, 58 (3): 1- 14.
	ZHENG F X , WANG X L , HE D D , et al. A review of research on single-image dehaze algorithms[J]. Computer Engineering and Applications, 2022, 58 (3): 1- 14.
2	CAI B L , XU X M , JIA K , et al. DehazeNet: an end-to-end system for single image haze removal[J]. IEEE Trans.on Image Processing, 2016, 25 (11): 5187- 5198. doi: 10.1109/TIP.2016.2598681
3	QIN X, WANG Z L, BAI Y C, et al. FFA-Net: feature fusion attention network for single image dehazing[EB/OL]. [2019-11-18]. https://arxiv.org/abs/1911.07559.
4	NARASIMHAN S G , NAYAR S K . Contrast restoration of weather degraded images[J]. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2003, 25 (6): 713- 724. doi: 10.1109/TPAMI.2003.1201821
5	ZHU Q S , MAI J M , SHAO L . A fast single image haze removal algorithm using color attenuation prior[J]. IEEE Trans. on Image Processing, 2015, 24 (11): 3522- 3533. doi: 10.1109/TIP.2015.2446191
6	HE K M , SUN J , TANG X O , et al. Single image haze removal using dark channel prior[J]. IEEE Trans. on Pattern Analysis & Machine Intelligence, 2011, 33 (12): 2341- 2353.
7	JU M Y , DING C , REN W Q , et al. IDE: Image dehazing and exposure using enhanced atmospheric scattering model[J]. IEEE Trans. on Image Processing, 2021, 30, 2180- 2192. doi: 10.1109/TIP.2021.3050643
8	LI B Y, PENG X L, WANG Z Y, et al. AOD-Net: all-in-one dehazing network[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 4770 - 4778.
9	ZHANG H, PATEL V M. Densely connected pyramid dehazing network[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 3194-3203.
10	REN W Q, MA L, ZHANG J W, et al. Gated fusion network for single image dehazing[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 3253-3261.
11	汪美琴, 袁伟伟, 张继业. 生成对抗网络GAN的研究综述[J]. 计算机工程与设计, 2021, 42 (12): 3389- 3395. doi: 10.16208/j.issn1000-7024.2021.12.012
	WANG M Q , YUAN W W , ZHANG J Y . A review of research on generative adversarial network GANs[J]. Computer Engineering and Design, 2021, 42 (12): 3389- 3395. doi: 10.16208/j.issn1000-7024.2021.12.012
12	QU Y Y, CHEN Y Z, HUANG J Y, et al. Enhanced pix2pix dehazing network[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 8160-8168.
13	SHAO Y J, LI L H, REN W Q, et al. Domain adaptation for image dehazing[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 2808-2817.
14	LIU X H, MA Y R, SHI Z H, et al. Griddehazenet: attention-based multi-scale network for image dehazing[C]//Proc. of the IEEE/CVF International Conference on Computer Vision, 2019: 7314-7323.
15	CUI T, ZHANG Z, TANG Y D, et al. Multi-scale densely connected dehazing network[C]//Proc. of the International Conference on Intelligent Robotics and Applications, 2019: 594-604.
16	HUA Z , FAN G D , LI J J . Iterative residual network for image dehazing[J]. IEEE Access, 2020, 8, 167693- 167710. doi: 10.1109/ACCESS.2020.3023906
17	范新南, 赵忠鑫, 严炜, 等. 结合注意力机制的多尺度特征融合图像去雾算法[J]. 计算机科学, 2022, 49 (5): 50- 57.
	FAN X N , ZHAO Z X , YAN W , et al. Multi-scale feature fusion image defog algorithm combined with attention mechanism[J]. Computer Science, 2022, 49 (5): 50- 57.
18	DONG H, PAN J S, XIANG L, et al. Multi-scale boosted dehazing network with dense feature fusion[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 2157-2167.
19	ZHENG J H , LIU X Y , WANG X D . Single image cloud removal using U-Net and generative adversarial networks[J]. IEEE Trans.on Geoscience and Remote Sensing, 2020, 59, 6371- 6385.
20	田元, 李方迪. 基于深度信息的人体姿态识别研究综述[J]. 计算机工程与应用, 2020, 56 (4): 1- 8.
	TIAN Y , LI F D . A review of human posture recognition research based on depth information[J]. Computer Engineering and Applications, 2020, 56 (4): 1- 8.
21	鞠默然, 罗江宁, 王仲博, 等. 融合注意力机制的多尺度目标检测算法[J]. 光学学报, 2020, 40 (13): 1315002.
	JU M R , LUO J N , WANG Z B , et al. Multi-scale object detection algorithm that incorporates attention mechanisms[J]. Acta Optica Sinica, 2020, 40 (13): 1315002.
22	FU X Y , LIANG B R , HUANG Y , et al. Lightweight pyramid networks for image deraining[J]. IEEE Trans.on Neural Networks and Learning Systems, 2019, 31 (6): 1794- 1807.
23	SUIN M, PUROHIT K, RAJAGOPALANA N. Spatially-attentive patch-hierarchical network for adaptive motion deblurring[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 3606-3615.
24	REN W Q , PAN J S , ZHANG H , et al. Single image dehazing via multi-scale convolutional neural networks with holistic edges[J]. International Journal of Computer Vision, 2020, 128 (1): 240- 259.
25	ZHAO S Y , ZHANG L , SHEN Y , et al. RefineDNet: a weakly supervised refinement framework for single image dehazing[J]. IEEE Trans.on Image Processing, 2021, 30, 3391- 3404.
26	WOO S, PARK J C, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proc. of the European Conference on Computer Vision, 2018: 3-19.
27	TANG K , XU D H , LIU H , et al. Context module based multi-patch hierarchical network for motion deblurring[J]. Neural Processing Letters, 2021, 53 (1): 211- 226.
28	HE K M , SUN J , TANG X . Guided image filtering[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2013, 35 (6): 1397- 1409.
29	ZHANG S H, FU H Z, YAN Y G, et al. Attention guided network for retinal image segmentation[C]//Proc. of the International Conference on Medical Image Computing and Computer-assisted Intervention, 2019: 797-805.
30	JUSTIN J, ALEXANDRE A, LI F F. Perceptual losses for real-time style transfer and super-resolution[C]//Proc. of the European Conference on Computer Vision, 2016: 694-711.
31	ADAM P , SAM G , FRANCISCO M , et al. Pytorch: an imperative style, high-performance deep learning library[J]. Advances in Neural Information Processing Systems, 2019, 32, 8026- 8037.
32	LI B Y , REN W Q , FU D P , et al. Benchmarking single-image dehazing and beyond[J]. IEEE Trans.on Image Processing, 2018, 28 (1): 492- 505.

算法	PSNR/dB	SSIM
DCP	19.31	0.814 8
CAP	18.49	0.798 2
DehazeNet	22.46	0.851 4
AOD-Net	20.29	0.876 5
GridDehazeNet	30.86	0.981 9
MSBDN	32.09	0.976 0
本文算法	33.02	0.986 5

损失函数	L_rec	L_per	L_col	PSNR/dB	SSIM
1	√	-	-	30.65	0.964 5
2	√	√	-	32.38	0.977 3
3	√	√	√	33.02	0.986 5

算法	Runtime
DCP	1.254
CAP	0.982
DehazeNet	1.031
AOD-Net	0.265
Grid-DNet	0.122
MSBDN	0.124
本文算法	0.072

[1]	赵晓枫, 牛家辉, 刘春桐, 夏玉婷. 基于三维注意力与混合卷积的高光谱图像分类[J]. 系统工程与电子技术, 2023, 45(9): 2673-2680.
[2]	李海军, 孔繁程, 林云. 基于改进YOLOv5s的红外舰船检测算法[J]. 系统工程与电子技术, 2023, 45(8): 2415-2422.
[3]	邓喆, 雷菁, 孙承哲. 跳频信号盲检测的半监督干扰对消方法[J]. 系统工程与电子技术, 2023, 45(7): 2236-2248.
[4]	孙景荣, 陈哲哲, 谢林昌, 杜梦欣, 宋诗斌. 基于图像天空区域分割的去雾算法[J]. 系统工程与电子技术, 2023, 45(6): 1606-1615.
[5]	赵庆媛, 赵志强, 叶春茂, 鲁耀兵. 基于自注意力的双波段预警雷达微动融合识别[J]. 系统工程与电子技术, 2023, 45(3): 708-716.
[6]	曹鹏宇, 杨承志, 陈泽盛, 王露, 石礼盟. 基于深度残差收缩注意力网络的雷达信号识别方法[J]. 系统工程与电子技术, 2023, 45(3): 717-725.
[7]	闫啸家, 梁伟阁, 张钢, 佘博, 田福庆. 基于RCNN-ABiLSTM的机械设备剩余寿命预测方法[J]. 系统工程与电子技术, 2023, 45(3): 931-940.
[8]	黄妍妍, 盖绍彦, 达飞鹏. 三分支空间变换注意力机制的图像匹配算法[J]. 系统工程与电子技术, 2023, 45(11): 3363-3373.
[9]	孙隽丰, 李成海, 曹波. 基于TCN-BiLSTM的网络安全态势预测[J]. 系统工程与电子技术, 2023, 45(11): 3671-3679.
[10]	韩啸, 陈世文, 陈蒙, 杨锦程. 基于互易点学习的LPI信号开集识别[J]. 系统工程与电子技术, 2022, 44(9): 2752-2759.
[11]	王健, 何自豪, 刘洁, 杨珂. 基于梯度域导向滤波器和改进PCNN的图像融合算法[J]. 系统工程与电子技术, 2022, 44(8): 2381-2392.
[12]	徐平亮, 崔亚奇, 熊伟, 熊振宇, 顾祥岐. 生成式中断航迹接续关联方法[J]. 系统工程与电子技术, 2022, 44(5): 1543-1552.
[13]	吴涛, 王伦文, 朱敬成. 基于迁移学习和注意力机制的伪装图像分割[J]. 系统工程与电子技术, 2022, 44(2): 376-384.
[14]	金涛, 王晓峰, 田润澜, 张歆东. 基于改进1DCNN+TCN的雷达辐射源快速识别方法[J]. 系统工程与电子技术, 2022, 44(2): 463-469.
[15]	马钰棠, 孙鹏, 张杰勇, 王鹏, 闫云飞, 赵亮. 样本不平衡下的空中群组意图识别方法[J]. 系统工程与电子技术, 2022, 44(12): 3747-3755.