基于三维注意力与混合卷积的高光谱图像分类

doi:10.12305/j.issn.1001-506X.2023.09.04

系统工程与电子技术 ›› 2023, Vol. 45 ›› Issue (9): 2673-2680.doi: 10.12305/j.issn.1001-506X.2023.09.04

基于三维注意力与混合卷积的高光谱图像分类

赵晓枫¹^,², 牛家辉¹^,²^,*, 刘春桐¹^,², 夏玉婷¹^,²

1. 火箭军工程大学导弹工程学院, 陕西西安 710025
2. 兵器发射理论与技术国家重点学科实验室, 陕西西安 710025

收稿日期:2022-05-09 出版日期:2023-08-30 发布日期:2023-09-05
通讯作者: 牛家辉
作者简介:赵晓枫 (1979—), 男, 副教授, 博士, 主要研究方向为兵器发射理论与技术
牛家辉 (1994—), 男, 硕士研究生, 主要研究方向为高光谱图像处理、光电隐身伪装与防护
刘春桐 (1974—), 男, 教授, 博士, 主要研究方向为定位定向与光电防护
夏玉婷 (1998—), 女, 硕士研究生, 主要研究方向为图像处理、光电隐身伪装防护
基金资助:
国家自然科学基金(41404022)

Hyperspectral image classification based on hybrid convolution with three-dimensional attention mechanism

Xiaofeng ZHAO¹^,², Jiahui NIU¹^,²^,*, Chuntong LIU¹^,², Yuting XIA¹^,²

1. Missile Engineering College, Rocket Force Engineering University, Xi'an 710025, China
2. Armament Launch Theory and Technology Key Discipline Laboratory of China, Xi'an 710025, China

Received:2022-05-09 Online:2023-08-30 Published:2023-09-05
Contact: Jiahui NIU

摘要/Abstract

摘要：

针对现有高光谱图像分类模型在特征提取的过程中有效特征关注缺乏的问题, 提出了一种基于三维卷积注意力与混合卷积的高光谱图像分类方法。该方法使用三维卷积和二维卷积串联完成对高光谱图像空谱特征的提取, 并在三维卷积阶段引入注意力机制, 使得模型在提取底层空谱特征的同时实现对有效特征的关注和激活。相对于传统三维卷积模型, 提出的分类模型减小了运算复杂度, 提升了模型噪声抑制能力, 提高了分类效果。针对该方法的消融实验证明了提出的三维卷积注意力机制的有效性, 在印第安松树林和帕维亚大学两个公开数据集上与其他5种分类模型的对比实验中取得了最优的分类精度。

关键词: 高光谱图像分类, 三维注意力机制, 混合卷积, 空谱特征提取

Abstract:

Aiming at the lack of effective attention in the process of feature extraction in existing hyperspectral image classification models, a classification model based on hybrid convolution and three-dimensional attention mechanism is proposed. The method realizes the extraction of spatial-spectral features of hyperspectral images by tandem three-dimensioal (3D) convolution and two-dimensional (2D) convolution. An attention mechanism in the 3D convolution stage is designed, and the attention mechanism is implemented in the 3D convolution stage to realize the attention and activation of the effective spatial-spectral features of hyperspectral images while the model is extracting the underlying features. Compared with the traditional 3D convolution-based model, the classification model proposed in this paper reduces the complexity of operations, improves the model's ability to suppress interference noise, and enhances the classification effect. Ablation experiments against the method demonstrate the effectiveness of the proposed 3D convolution attention mechanism, and the optimal classification accuracy is achieved in comparison experiments with five other classification models on two publicly available datasets, Indian Pines and Pavia University.

Key words: hyperspectral image classification, three-dimensional attention mechanism, hybrid convolution, spatial-spectral feature extraction

中图分类号:

TP751

赵晓枫, 牛家辉, 刘春桐, 夏玉婷. 基于三维注意力与混合卷积的高光谱图像分类[J]. 系统工程与电子技术, 2023, 45(9): 2673-2680.

Xiaofeng ZHAO, Jiahui NIU, Chuntong LIU, Yuting XIA. Hyperspectral image classification based on hybrid convolution with three-dimensional attention mechanism[J]. Systems Engineering and Electronics, 2023, 45(9): 2673-2680.

图/表 11

图1

图2

图3

图4

图5

表1

表2

图6

表3

图7

图8

参考文献 32

1	GHAMISI P , RASTI B , YOKOYA N , et al. Multisource and multitemporal data fusion in remote sensing: a comprehensive review of the state of the art[J]. IEEE Geoscience and Remote Sensing Magazine, 2019, 7 (1): 6- 39. doi: 10.1109/MGRS.2018.2890023
2	ZHU Q Q , DENG W H , ZHENG Z , et al. A spectral-spatial-dependent global learning framework for insufficient and imbalanced hyperspectral image classification[J]. IEEE Trans.on Cybernetics, 2021, 52 (11): 11709- 11723.
3	AHMAD M, SHABBIR S, RAZA R A, et al. Hyperspectral image classification: artifacts of dimension reduction on hybrid CNN[C]//Proc. of the Computer Vision and Pattern Recognition Conference, 2021.
4	张朝阳, 程海峰, 陈朝辉, 等. 高光谱遥感的发展及其对军事装备的威胁[J]. 光电技术应用, 2008, (1): 10- 12. doi: 10.3969/j.issn.1673-1255.2008.01.003
	ZHANG C Y , CHENG H F , CHEN C H , et al. The development of hyperspectral remote sensing and its threatening to military equipments[J]. Electro-optic Technology Application, 2008, (1): 10- 12. doi: 10.3969/j.issn.1673-1255.2008.01.003
5	YANG X G , YU Y . Estimating soil salinity under various moisture conditions: an experimental study[J]. IEEE Trans.on Geoscience and Remote Sensing, 2017, 55 (5): 2525- 2533. doi: 10.1109/TGRS.2016.2646420
6	GOVENDER M , CHETTY K , BULCOCK H . A review of hyperspectral remote sensing and its application in vegetation and water resource studies[J]. Water SA, 2007, 33 (2): 145- 151.
7	MOLERO J M , GARZON E M , GARCIA I , et al. Efficient implementation of hyperspectral anomaly detection techniques on GPUs and multicore processors[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7 (6): 2256- 2266. doi: 10.1109/JSTARS.2014.2328614
8	SUMARSONO A , DU Q . Low-rank subspace representation for estimating the number of signal subspaces in hyperspectral imagery[J]. IEEE Trans.on Geoscience and Remote Sensing, 2015, 53 (11): 6286- 6292. doi: 10.1109/TGRS.2015.2438079
9	CAMPSVALLS G , GOMEZCHOVA L , MUNOZMARI J , et al. Composite kernels for hyperspectral image classification[J]. IEEE Geoscience Remote Sensing Letters, 2006, 105 (1): 23- 33.
10	LI J , HUANG X , GAMBA P , et al. Multiple feature learning for hyperspectral image classification[J]. IEEE Trans.on Geoscience and Remote Sensing, 2015, 53 (3): 1592- 1606. doi: 10.1109/TGRS.2014.2345739
11	CIOTOLA M , VITALE S , MAZZA A , et al. Pansharpening by convolutional neural networks in the full resolution framework[J]. IEEE Trans.on Geoscience and Remote Sensing, 2022, 60, 5408717.
12	GUO M H , XU T X , LIU J J , et al. Attention mechanisms in computer vision: a survey[J]. Computational Visual Media, 2022, 8 (3): 331- 368. doi: 10.1007/s41095-022-0271-y
13	HU W , HUANG Y Y , LI W , et al. Deep convolutional neural networks for hyperspectral image classification[J]. Journal of Sensors, 2015, 2015, 258619.
14	BOULCH A, TROUVE P, KOENIGUER E, et al. Learning speckle suppression in SAR images without ground truth: application to Sentinel-1 time-series[C]//Proc. of the IEEE International Geoscience and Remote Sensing Symposium, 2018: 2366-2369.
15	MOU L , GHAMISI P , ZHU X X . Deep recurrent neural networks for hyperspectral image classification[J]. IEEE Trans.on Geoscience and Remote Sensing, 2017, 55 (7): 3639- 3655. doi: 10.1109/TGRS.2016.2636241
16	MAKANTASIS K, KARANTZALOS K, DOULAMIS A, et al. Deep supervised learning for hyperspectral data classification through convolutional neural networks[C]//Proc. of the IEEE International Geoscience and Remote Sensing Symposium, 2015: 4959-4962.
17	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778
18	SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
19	韩彦岭, 崔鹏霞, 杨树瑚, 等. 基于残差网络特征融合的高光谱图像分类[J]. 国土资源遥感, 2021, 33 (2): 11- 19.
	HAN Y L , CUI P X , YANG S H , et al. Classification of hyperspectral image based on feature fusion of residual network[J]. Remote Sensing for Land and Resources, 2021, 33 (2): 11- 19.
20	王溢琴, 董云云, 刘慧玲. 基于GoogLeNet和空间谱变换的高光谱图像超分辨率方法[J]. 光学技术, 2022, 48 (1): 93- 101.
	WANG Y Q , DONG Y Y , LIU H L . Super-resolution method of hyperspectral image based on GoogLeNet and spatial spectrum transformation[J]. Optical Technique, 2022, 48 (1): 93- 101.
21	HAMIDA A B , BENOIT A , LAMBERT P , et al. 3-D deep learning approach for remote sensing image classification[J]. IEEE Trans.on Geoscience and Remote Sensing, 2018, 56 (8): 4420- 4434. doi: 10.1109/TGRS.2018.2818945
22	CHEN Y S , JIANG H L , LI C Y , et al. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks[J]. IEEE Trans.on Geoscience and Remote Sensing, 2016, 54 (10): 6232- 6251. doi: 10.1109/TGRS.2016.2584107
23	ROY S K , KRISHNA G , DUBEY S R , et al. HybridSN: exploring 3D-2D CNN feature hierarchy for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17 (2): 277- 281.
24	张祥东, 王腾军, 朱劭俊, 等. 基于扩张卷积注意力神经网络的高光谱图像分类[J]. 光学学报, 2021, 41 (3): 49- 59.
	ZHANG X D , WANG T J , ZHU S J , et al. Hyperspectral image classification based on dilated convolutional attention neural network[J]. Acta Optica Sinica, 2021, 41 (3): 49- 59.
25	DING Y , ZHAO X F , ZHANG Z L , et al. Graph sample and aggregate-attention network for hyperspectral image classification[J]. IEEE Geoscience Remote Sensing Letters, 2021, 19, 5504205.
26	LIU Q C , XIAO L , YANG J X , et al. CNN-enhanced graph convolutional network with pixel and superpixel level feature fusion for hyperspectral image classification[J]. IEEE Trans. on Geoscience and Remote Sensing, 2021, 59 (10): 8657- 8671.
27	HONG D F , HAN Z , YAO J , et al. SpectralFormer: rethinking hyperspectral image classification with Transformers[J]. IEEE Trans.on Geoscience and Remote Sensing, 2022, 60 (5518615)
28	HU J , SHEN L , ALBANIE S , et al. Squeeze-and-excitation networks[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2020, 42 (8): 2011- 2023.
29	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proc. of the European Conference on Computer Vision, 2018.
30	HAMIDA A B , BENOIT A , LAMBERT P , et al. 3-D deep learning approach for remote sensing image classification[J]. IEEE Trans.on Geoscience and Remote Sensing, 2018, 56 (8): 4420- 4434.
31	HE M Y, BO L, CHEN H H. Multi-scale 3D deep convolutional neural network for hyperspectral image classification[C]//Proc. of the IEEE International Conference on Image Processing, 2018: 3904-3908.
32	SUN H , ZHENG X T , LU X Q , et al. Spectral-spatial attention networks for hyperspectral image classification[J]. Remote Sensing, 2019, 11 (8): 3232- 3245.

模型	分类精度
模型	OA	AA	Kappa
2D-CNN-DF	77.33	71.95	68.71
FE-3D-CNN	80.74	85.04	83.00
Muti-3DCNN	79.39	84.29	82.10
3D-2D-ADCNN	97.25	94.89	96.32
SSAN	97.26	94.88	96.31
3DA-FCNN	97.73	95.24	97.41

模型	分类精度
模型	OA	AA	Kappa
2D-CNN-DF	94.15	94.23	91.25
FE-3D-CNN	93.17	91.49	88.7
Muti-3DCNN	97.57	94.69	93.10
3D-2D-ADCNN	98.74	98.93	98.60
SSAN	99.12	98.03	98.11
3DA-FCNN	99.61	98.96	99.48

方法	IP			PU
方法	OA	AA	Kappa	OA	AA	Kappa
FCNN	94.84	90.89	94.09	99.19	98.06	98.93
3DA-FCNN	97.73	95.24	97.41	99.61	98.96	99.48

[1]	陈善学, 王欣欣. 基于空间预处理联合稀疏表示高光谱图像分类[J]. 系统工程与电子技术, 2021, 43(9): 2422-2429.
[2]	陈善学, 陈雯雯. 基于二级字典的联合稀疏表示高光谱图像分类[J]. 系统工程与电子技术, 2020, 42(3): 550-556.
[3]	陈善学, 周艳发, 漆若兰. 基于核函数的联合稀疏表示高光谱图像分类[J]. 系统工程与电子技术, 2018, 40(3): 692-698.
[4]	陈善学, 屈龙瑶, 胡灿. 基于空间约束加权条件稀疏表示高光谱图像分类[J]. 系统工程与电子技术, 2016, 38(2): 442-449.

基于三维注意力与混合卷积的高光谱图像分类

Hyperspectral image classification based on hybrid convolution with three-dimensional attention mechanism

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 32

相关文章 4

编辑推荐

Metrics

本文评价