基于图像特征的红外与低照度图像融合

doi:10.12305/j.issn.1001-506X.2023.08.13

摘要/Abstract

摘要：

针对难以独立分析红外图像和低照度图像场景中目标和背景信息的问题, 提出了基于图像特征的红外与低照度图像融合算法。首先, 针对性地对红外和低照度图像进行图像处理, 优化各自特性。其次, 采用非下采样剪切波变换对两者进行高低频分解。然后, 采用改进的拉普拉斯加权算法融合低频图像, 采用改进的脉冲耦合神经网络融合高频图像。最后, 经非下采样剪切波逆变换获取融合图像。实验结果表明, 所提算法可有效融合红外与低照度图像, 降低了噪声对融合图像的影响, 提高了融合图像的清晰度, 且图像对比度适中, 符合人眼视觉感知效果。

关键词: 低照度图像, 红外图像, 非下采样剪切波变换, 脉冲耦合神经网络, 图像融合算法

Abstract:

Aiming at the problem that it is difficult to analyze the target and background information in the scene of infrared images and low illumination images independently, an infrared and low illumination image fusion algorithm based on image features is proposed. Firstly, the infrared and low illumination images are processed to optimize their characteristics respectively. Secondly, the non-subsampled shearlet transform is used to decompose them in high and low frequency. Thirdly, the improved Laplace weighted algorithm is used to fuse the low-frequency images. The improved pulse coupled neural network is used to fuse the high-frequency images. Finally, the fused image is obtained by non-subsampled shearlet inverse transform. Experimental results show that the proposed algorithm can effectively fuse infrared and low illumination images, reduce the impact of noise on the fused image, improve the clarity of the fused image, and the contrast of the image is moderate, which is suifable the visual perception effect of human eyes.

Key words: low illumination image, infrared image, non-subsampled shearlet transform, pulse-coupled neural network, image fusion algorithm

中图分类号:

TP391.41

王慧赢, 王春平, 付强, 韩子硕, 张冬冬. 基于图像特征的红外与低照度图像融合[J]. 系统工程与电子技术, 2023, 45(8): 2395-2404.

Huiying WANG, Chunping WANG, Qiang FU, Zishuo HAN, Dongdong ZHANG. Infrared and low illumination image fusion based on image features[J]. Systems Engineering and Electronics, 2023, 45(8): 2395-2404.

图/表 14

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

图13

表1

参考文献 31

1	邢雅琼, 王晓丹, 毕凯, 等. 基于NSCT和ICA的红外和可见光图像融合方法[J]. 系统工程与电子技术, 2013, 35 (11): 2251- 2256.
	XING Y Q , WANG X D , BI K , et al. Infrared and visible image fusion method based on NSCT and ICA[J]. Systems Engineering and Electronics, 2013, 35 (11): 2251- 2256.
2	江泽涛, 覃露露, 秦嘉奇, 等. 一种基于MDARNet的低照度图像增强方法[J]. 软件学报, 2021, 32 (12): 3977- 3991. doi: 10.13328/j.cnki.jos.006112
	JIANG Z T , QIN L L , QIN J Q , et al. Low-light image enhancement method based on MDARNet[J]. Journal of Software, 2021, 32 (12): 3977- 3991. doi: 10.13328/j.cnki.jos.006112
3	周晓玲, 江泽涛. 结合脉冲耦合神经网络与引导滤波的红外与可见光图像融合[J]. 光学学报, 2019, 39 (11): 1110003.
	ZHOU X L , JIANG Z T . Infrared and visible image fusion combining pulse-coupled neural network and guided filtering[J]. Acta Optical Sinica, 2019, 39 (11): 1110003.
4	LIU K , GUO L , LI H H , et al. Fusion of infrared and visible light images based on region segmentation[J]. Chinese Journal of Aeronautics, 2009, 22 (1): 75- 80. doi: 10.1016/S1000-9361(08)60071-0
5	GAI D , SHEN X J , CHEN H P , et al. Medical image fusion using the PCNN based on IQPSO in NSST domain[J]. IET Image Processing, 2020, 14 (9): 1870- 1880. doi: 10.1049/iet-ipr.2020.0040
6	CHEN J , LI X J , LUO L B , et al. Infrared and visible image fusion based on target-enhanced multiscale transform decomposition[J]. Information Sciences, 2020, 508, 64- 78. doi: 10.1016/j.ins.2019.08.066
7	BHAT S , KOUNDAL D . Multi-focus image fusion using neutrosophic based wavelet transform[J]. Applied Soft Computing, 2021, 106, 107307. doi: 10.1016/j.asoc.2021.107307
8	GUO L Q , CAO X , LIU L . Dual-tree biquaternion wavelet transform and its application to color image fusion[J]. Signal Processing, 2020, 171, 107513. doi: 10.1016/j.sigpro.2020.107513
9	AGHAMALEKI J A , GHORBANI A . Image fusion using dual tree discrete wavelet transform and weights optimization[J]. The Visual Computer, 2023, 39, 1181- 1191. doi: 10.1007/s00371-021-02396-9
10	ZHAO C H , GUO Y T , WANG Y L . A fast fusion scheme for infrared and visible light images in NSCT domain[J]. Infrared Physics and Technology, 2015, 72, 266- 275. doi: 10.1016/j.infrared.2015.07.026
11	YUE J J , LI M Z , CHEN J , et al. Multi-focus infrared image fusion based on pulse coupled neural networks in a nonsubsampled contourlet transform domain[J]. Infrared Technology, 2017, 39 (9): 798- 806.
12	JIAO J , WU L D . Pansharpening with a gradient domain GIF based on NSST[J]. Electronics, 2019, 8 (2): 229. doi: 10.3390/electronics8020229
13	LI L L , SI Y J , WANG L L , et al. A novel approach for multi-focus image fusion based on SF-PAPCNN and ISML in NSST domain[J]. Multimedia Tools and Applications, 2020, 79 (6): 24303- 24328.
14	WEI T , LIU Y , CHENG J , et al. A phase congruency-based green fluorescent protein and phase contrast image fusion method in nonsubsampled shearlet transform domain[J]. Microscopy Research and Technique, 2020, 83 (10): 1225- 1234.
15	WANG J , YANG K , REN P , et al. Multi-source image fusion algorithm based on fast weighted guided filter[J]. Journal of Systems Engineering and Electronics, 2019, 30 (5): 831- 840. doi: 10.21629/JSEE.2019.05.02
16	GANASALA P , PRASAD A D . Contrast enhanced multi sensor image fusion based on guided image filter and NSST[J]. IEEE Trans.on Sensors Journal, 2020, 20 (2): 939- 946. doi: 10.1109/JSEN.2019.2944249
17	ZHANG S , HUANG F Y , ZHONG H , et al. Multi-modal image fusion via sparse representation and multi-scale anisotropic guided measure[J]. IEEE Access, 2020, (8): 35638- 35649.
18	BIRGIT K , LORENZ D A , LENA V . Denoising of image gradients and total generalized variation denoising[J]. Journal of Mathematical Imaging & Vision, 2019, 61, 21- 39.
19	EASLEY G , LABATE D , LIM W Q . Sparse directional image representations using the discrete shearlet transform[J]. Applied Computational Harmonic Analysis, 2008, 25 (1): 25- 46. doi: 10.1016/j.acha.2007.09.003
20	ECKHORN R , REIBOECK H J , ARNDT M , et al. Feature linking via synchronization among distributed assemblies: simulation of results from cat cortex[J]. Neural Computation, 1990, 2 (3): 293- 307. doi: 10.1162/neco.1990.2.3.293
21	YIN M , LIU X N , LIU Y , et al. Medical image fusion with parameter adaptive pulse coupled neural network in nonsubsampled shearlet transform domain[J]. IEEE Trans.on Instrumentation and Measurement, 2019, 68 (1): 49- 64. doi: 10.1109/TIM.2018.2838778
22	LI X X , GUO X P , HAN P F , et al. Laplacian re-decomposition for multimodal medical image fusion[J]. IEEE Trans.on Instrumentation and Measurement, 2020, 69 (9): 6880- 6890. doi: 10.1109/TIM.2020.2975405
23	LI Y Q , ZHAO H T , HU Z W , et al. IVFuseNet: fusion of infrared and visible light images for depth prediction[J]. Information Fusion, 2019, 58, 1- 12.
24	TOET Alexander. TNO image fusion dataset[EB/OL]. [2021-07-30]. https://figshare.com/articles/TNO-Image-Fusion-Dataset/1008029.
25	WU C , CHEN L . Infrared and visible image fusion method of dual NSCT and PCNN[J]. PLoS One, 2020, 15 (9): e0239535. doi: 10.1371/journal.pone.0239535
26	WANG L , LUO W , CHEN J H , et al. Fusion of infrared and visible images based on adaptive PCNN and information extraction[J]. Computer Engineering and Applications, 2018, 54 (4): 192- 198.
27	XING X X, LIU C, LUO C, et al. Infrared and visible image fusion based on nonlinear enhancement and NSST decomposition[EB/OL]. [2022-03-25]. https//www.research.square.com/article/rs-15346v1.
28	DU Q L , XU H , MA Y , et al. Fusing infrared and visible images of different resolutions via total variation model[J]. Sensors, 2018, 18 (11): 3827. doi: 10.3390/s18113827
29	MA J Y , CHEN C , LI C , et al. Infrared and visible image fusion via gradient transfer and total variation minimization[J]. Information Fusion, 2016, 31, 100- 109. doi: 10.1016/j.inffus.2016.02.001
30	JIN X , JIANG Q , YAO S W , et al. A survey of infrared and visual image fusion methods[J]. Infrared Physics and Technology, 2017, 85, 478- 501. doi: 10.1016/j.infrared.2017.07.010
31	KONG W W , LEI Y , REN M M . Fusion method for infrared and visible images based on improved quantum theory model[J]. Neuro Computing, 2016, 212, 12- 21.

源图像	指标	NSCT_PCNN	PCNN	NSST	本文算法
第1组	IE	6.561 8	6.668 0	6.569 1	7.791 8
	SD	45.329 1	45.423 4	45.356 8	60.576 2
	AG	7.453 2	7.612 8	7.642 0	8.921 0
	Q^AB/F	0.504 0	0.564 1	0.521 0	0.582 8
第2组	IE	6.803 2	5.521 9	6.674 2	7.313 0
	SD	38.751 7	38.721 0	38.854 2	74.012 5
	AG	5.781 6	5.359 6	5.742 9	8.672 0
	Q^AB/F	0.340 2	0.561 6	0.551 3	0.584 2
第3组	IE	6.541 2	6.211 2	6.421 3	7.294 6
	SD	26.324 0	24.104 2	26.214 0	50.321 1
	AG	4.401 5	3.014 2	4.025 6	6.121 0
	Q^AB/F	0.391 8	0.401 2	0.354 2	0.471 3
第4组	IE	6.701 2	7.399 5	7.066 8	7.651 6
	SD	23.191 2	26.146 2	25.486 9	30.151 6
	AG	2.295 2	4.352 8	3.961 6	5.659 6
	Q^AB/F	0.382 4	0.401 5	0.375 2	0.542 1
第5组	IE	6.954 8	6.707 8	6.575 3	7.213 4
	SD	31.492 6	27.451 2	25.617 1	45.142 3
	AG	4.809 8	4.754 3	4.017 6	5.013 6
	Q^AB/F	0.409 6	0.391 2	0.301 9	0.553 8

[1]	李海军, 孔繁程, 林云. 基于改进YOLOv5s的红外舰船检测算法[J]. 系统工程与电子技术, 2023, 45(8): 2415-2422.
[2]	王健, 何自豪, 刘洁, 杨珂. 基于梯度域导向滤波器和改进PCNN的图像融合算法[J]. 系统工程与电子技术, 2022, 44(8): 2381-2392.
[3]	李旭, 丁萌, 魏东辉, 吴晓舟, 曹云峰. VDAS中基于单目红外图像的深度估计方法[J]. 系统工程与电子技术, 2021, 43(5): 1210-1217.
[4]	花思齐, 赵伟, 刘建业. 基于改进滤波器和图像多尺度变换的背景抑制算法[J]. 系统工程与电子技术, 2020, 42(8): 1679-1684.
[5]	李翠芸, 曹潇男, 廖良雄, 江舟. 高斯粒子PHD滤波的多个弱小目标TBD算法[J]. 系统工程与电子技术, 2015, 37(4): 740-745.
[6]	邢雅琼, 王晓丹, 梁兵杰, 秦卓. 基于非下采样剪切波变换域方向信息测度的多聚焦图像融合方法[J]. 系统工程与电子技术, 2015, 37(1): 191-198.
[7]	郑皓天，顾晓东. 基于带宽剩余率的脉冲耦合神经网络最短路径算法[J]. Journal of Systems Engineering and Electronics, 2013, 35(4): 859-863.
[8]	刘刚, 梁晓庚, 张京国. 基于轮廓波变换和改进模糊c均值聚类的红外图像分割[J]. Journal of Systems Engineering and Electronics, 2011, 33(2): 443-448.
[9]	朱英宏, 李俊山, 汤雨. 基于CSS角点提取的红外与可见光图像匹配算法[J]. Journal of Systems Engineering and Electronics, 2011, 33(11): 2540-2545.