基于多任务学习的建筑毁伤评估方法

doi:10.12305/j.issn.1001-506X.2024.10.15

摘要/Abstract

摘要：

建筑毁伤评估在灾害救援过程中对制定救援策略、优化资源调度等方面具有重要作用。现阶段, 基于语义分割的毁伤评估方法难以提取毁伤建筑细粒度语义信息, 对此提出一种基于多任务学习的建筑毁伤评估方法, 将毁伤评估分为粗粒度的建筑区域提取与细粒度的毁伤分割两个子任务, 通过共用编解码器, 与上下文信息融合模块实现建筑区域的粗粒度提取和建筑毁伤的细粒度分割, 将两个结果使用Hadamard积融合, 得到最终评估结果。实验结果表明, 所提的基于多任务学习的建筑毁伤评估方法有较好的性能。

关键词: 建筑毁伤评估, 深度学习, 多任务学习

Abstract:

Building damage assessment plays an important role in the disaster relief process, influencing the formulation of rescue strategies and optimization of resource allocation. Currently, damage assessment methods based on semantic segmentation face challenges in extracting fine-grained semantic information for damaged buildings. Thus, a multi-task learning based approach for building damage assessment is proposed, dividing the damage assessment into two subtasks as coarse-grained building area extraction and fine-grained damage segmentation. The proposed method utilizes a shared encoder-decoder and context fusion module to achieve coarse-grained extraction of building areas and fine-grained segmentation of building damage. The results of these two tasks are fused using the Hadamard product to obtain the final assessment. Experimental results demonstrate that the proposed multi-task learning based building damage assessment method performs well.

Key words: building damage assessment, deep learning, multi-task learning

中图分类号:

TP391.41

王一博, 张乐飞, 李新德. 基于多任务学习的建筑毁伤评估方法[J]. 系统工程与电子技术, 2024, 46(10): 3375-3382.

Yibo WANG, Lefei ZHANG, Xinde LI. Multi-task learning based building damage assessment method[J]. Systems Engineering and Electronics, 2024, 46(10): 3375-3382.

图/表 10

图1

图2

图3

图4

表1

图5

表2

图6

图7

表3

参考文献 31

1	ZHENG S Q, LI X D, ZHU L L. Optimal time-efficient UAV area coverage path planning based on raster map[C]//Proc. of the International Conference on Advanced Robotics and Mechatronics, 2023: 727-732.
2	王建峰, 贾高伟, 郭正, 等. 多无人机协同任务规划方法研究综述[J/OL]. 系统工程与电子技术: 1-18[2024-09-23]. http://kns.cnki.net/kcms/detail/11.2422.TN.20230419.1331.010.html.
	WANG J F, JIA G W, GUO Z, et al. Research status and development of multi-UAV system mission planning[J/OL]. Systems Engineering and Electronics: 1-18[2024-09-23]. http://kns.cnki.net/kcms/detail/11.2422.TN.20230419.1331.010.html.
3	TCHINDA N E , PANOFF K M , KWADJO T D , et al. Semi-supervised image stitching from unstructured camera arrays[J]. Sensors, 2023, 23 (23): 9481- 9501. doi: 10.3390/s23239481
4	杨丹宁. 基于受灾点应急需求分析的地震初期应急物资分配问题研究[D]. 北京: 北京交通大学, 2023.
	YANG D N. A study on the distribution of emergency supplies at the early stage of earthquake based on the analysis of emergency needs at the affected sites[D]. Beijing: Beijing Jiao Tong University, 2023.
5	李阳, 张玉梅, 赵彦东, 等. 基于云贝叶斯网络的毁伤效果评估方法[J]. 火力与指挥控制, 2020, 45 (3): 144- 149.
	LI Y , ZHANG Y M , ZHAO Y D , et al. Method of battle da-mage assessment based on cloudy Bayesian network[J]. Fire Control & Command Control, 2020, 45 (3): 144- 149.
6	LI X D , FIR D K , JEAN D . Multi-source information fusion: progress and future[J]. Chinese Journal of Aeronautics, 2024, 37 (7): 24- 58. doi: 10.1016/j.cja.2023.12.009
7	王广学, 黄晓涛, 周智敏. 基于图像分割的VHF SAR叶簇隐蔽目标差值变化检测[J]. 电子学报, 2020, 38 (9): 1969- 1974.
	WANG G X , HUANG X T , ZHOU Z M . VHF SAR difference change detection of target in foliage based on image segmentation[J]. Acta Electronica Sinica, 2020, 38 (9): 1969- 1974.
8	CHEN J , CHEN X H , CUI X H , et al. Change vector analysis in posterior probability space: a new method for land cover change detection[J]. IEEE Geoscience and Remote Sensing Letters, 2010, 8 (2): 317- 321.
9	MARIUS P , ANDREAS D , TOBIAS U , et al. Automated extraction of annual erosion rates for arctic permafrost coasts using sentinel-1, deep learning, and change vector analysis[J]. Remote Sensing, 2022, 14 (15): 3656- 3681. doi: 10.3390/rs14153656
10	苗添, 曾虹程, 王贺, 等. 基于迭代阈值分割的星载SAR洪水区域快速提取[J]. 系统工程与电子技术, 2022, 44 (9): 2760- 2768. doi: 10.12305/j.issn.1001-506X.2022.09.08
	MIAO T , ZENG H C , WANG H , et al. A fast extraction method of flood areas based on iterative threshold segmentation using spaceborne SAR data[J]. Systems Engineering and Electronics, 2022, 44 (9): 2760- 2768. doi: 10.12305/j.issn.1001-506X.2022.09.08
11	杨青青, 樊桂花. 基于改进模糊综合评判法的建筑物毁伤效果评估[J]. 系统工程与电子技术, 2018, 40 (9): 2026- 2031. doi: 10.3969/j.issn.1001-506X.2018.09.19
	YANG Q Q , FAN G H . Battle damage assessment of building based on improved fuzzy comprehensive evaluation method[J]. Systems Engineering and Electronics, 2018, 40 (9): 2026- 2031. doi: 10.3969/j.issn.1001-506X.2018.09.19
12	RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]//Proc. of the Medical Image Computing and Computer-assisted Intervention, 2015: 234-241.
13	XIE E Z , WANG W H , YU Z D , et al. SegFormer: simple and efficient design for semantic segmentation with transfor-mers[J]. Advances in Neural Information Processing Systems, 2021, 34, 12077- 12090.
14	ZHONG A Q , FU Q , HUANG D F , et al. A topology based automatic registration method for infrared and polarized coupled imaging[J]. Applied Sciences, 2022, 12 (24): 12596- 12607. doi: 10.3390/app122412596
15	DENG X , LIU E P , LI S X , et al. Interpretable multi-model image registration network based on disentangled convolutional sparse coding[J]. IEEE Trans.on Image Processing, 2023, 32, 1078- 1091. doi: 10.1109/TIP.2023.3240024
16	徐艺博, 于清华, 王炎娟, 等. 基于多源信息融合的巡飞弹对地目标识别与毁伤评估[J]. 系统仿真学报, 2024, 36 (2): 511- 521.
	XU Y B , YU Q H , WANG Y J , et al. Ground target recognition and damage assessment of patrol missiles based on multi-source information fusion[J]. Journal of System Simulation, 2024, 36 (2): 511- 521.
17	李琳, 孙世岩, 曾雅琴, 等. 基于改进CRITIC-Bayes网络的激光武器毁伤效果评估方法[J]. 兵器装备工程学报, 2023, 44 (7): 109- 115.
	LI L , SUN S Y , ZENG Y Q , et al. Evaluation methods of laser weapon damage effect based on improved CRITIC-Bayes network[J]. Journal of Ordnance Equipment Engineering, 2023, 44 (7): 109- 115.
18	NIA K R, MORI G. Building damage assessment using deep learning and ground-level image data[C]//Proc. of the 14th Conference on Computer and Robot, 2017: 95-102.
19	麻连伟, 宁卫远, 焦利伟, 等. 基于U-Net卷积神经网络的遥感影像变化检测方法研究[J]. 能源与环保, 2022, 44 (11): 102- 106.
	MA L W , NING W Y , JIAO L W , et al. Research on remote sensing image change detection method based on U-Net convolutional neural network[J]. China Energy and Environmental Protection, 2022, 44 (11): 102- 106.
20	FAN Z P , WANG S Q , PU X T , et al. Fusion-former: fusion features across transformer and convolution for building change detection[J]. Electronics, 2023, 12 (23): 4823- 4836. doi: 10.3390/electronics12234823
21	WANG X D , TIAN M L , ZHANG Z J , et al. SDSNet: building extraction in high-resolution remote sensing images using a deep convolutional network with cross-layer feature information interaction filtering[J]. Remote Sensing, 2023, 16 (1): 169- 191. doi: 10.3390/rs16010169
22	王凌霄, 贾婧. 基于改进DeepLabv3plus算法的遥感图像海岛建筑提取方法[J]. 热带地理, 2021, 41 (4): 834- 844.
	WANG L X , JIA J . Extraction method of island buildings in remote sensing images based on improved DeepLabv3plus algorithm[J]. Tropical Geography, 2021, 41 (4): 834- 844.
23	ZHENG Z , ZHONG Y F , WANG J J , et al. Building damage assessment for rapid disaster response with a deep object-based semantic change detection framework: from natural disasters to man-made disasters[J]. Remote Sensing of Environment, 2021, 265, 112636- 112653. doi: 10.1016/j.rse.2021.112636
24	SHENG W J , LI X D . Multi-task learning for gait-based identity recognition and emotion recognition using attention enhanced temporal graph convolutional network[J]. Pattern Recognition, 2021, 114, 107868- 107880. doi: 10.1016/j.patcog.2021.107868
25	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
26	LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]// Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2117-2125.
27	MILLETARI F, NAVAB N, AHMADI S. V-net: fully convolutional neural networks for volumetric medical image segmentation[C]//Proc. of the 4th Conference on 3D Vision, 2016: 565-571.
28	GUPTA R, GOODMAN B, PATEL N, et al. Creating xBD: a dataset for assessing building damage from satellite imagery[C]// Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019: 10-17.
29	LOSHCHILOV I, HUTTER F. Decoupled weight decay regularization[C]// Proc. of the International Conference on Learning Representations, 2018.
30	GUPTA R, HOSFELT R, SAJEEV S, et al. xBD: a dataset for assessing building damage from satellite imagery[EB/OL]. [2024-01-30]. https://arXiv preprint arXiv:1911.09296, 2019.
31	WEBER E, KANE H. Building disaster damage assessment in sate- llite imagery with multi-temporal fusion[EB/OL]. [2024-01-30]. https://arXiv preprint arXiv:2004.05525, 2020.

方法	主干网	F₁^overall	F₁^loc	F₁^dam	F₁^dam_1	F₁^dam_2	F₁^dam_3	F₁^dam_4
xBD-Baseline^[30]	ResNet50	26.5	-	-	66.3	14.4	0.9	46.6
方法a^[31]	ResNet50	74.1	-	-	90.6	49.3	72.2	83.7
方法b^[23]	ResNet50	74.5	85.4	69.9	88.9	51.4	71.6	79.6
MLDA(本文)	ResNet50	76.1	83.1	73.1	93.5	56.1	70.3	83.3

方法	F₁^overall	F₁^loc	F₁^dam	F₁^dam_1	F₁^dam_2	F₁^dam_3	F₁^dam_4
Res-U-Net	72.6	80.1	69.5	87.2	51.6	69.6	80.9
MLDA-	71.5	81.4	67.2	91.5	50.1	63.2	78.1
MLDA	76.1	83.1	73.1	93.5	56.1	70.3	83.3

方法	λ₁	λ₂	F₁^overall	F₁^loc	F₁^dam
MLDA	0.1	2	73.4	80.3	70.5
MLDA	1	2	76.1	83.1	73.1

[1]	蔡伟, 王鑫, 蒋昕昊, 杨志勇, 陈栋. 基于解耦的小样本目标检测方法研究[J]. 系统工程与电子技术, 2024, 46(9): 2941-2950.
[2]	陈晓萱, 徐书文, 胡绍海, 马晓乐. 基于卷积与自注意力的红外与可见光图像融合[J]. 系统工程与电子技术, 2024, 46(8): 2641-2649.
[3]	汪强龙, 高晓光, 吴必聪, 胡子剑, 万开方. 受限玻尔兹曼机及其变体研究综述[J]. 系统工程与电子技术, 2024, 46(7): 2323-2345.
[4]	孙先涛, 江汪洋, 陈文杰, 陈伟海, 智亚丽. 基于感兴趣区域的物体抓取位姿检测[J]. 系统工程与电子技术, 2024, 46(6): 1867-1877.
[5]	陈雪梅, 刘志恒, 周绥平, 余航, 刘彦明. 基于HRNet的高分辨率遥感影像道路提取方法[J]. 系统工程与电子技术, 2024, 46(4): 1167-1173.
[6]	张天文, 张晓玲, 邵子康, 曾天娇. 基于掩模注意型交互的SAR舰船实例分割[J]. 系统工程与电子技术, 2024, 46(3): 831-838.
[7]	季然, 肖茂森, 李硕, 刘宇, 罗湛仪, 程嘉维. 基于机器学习的MRTD客观测试方法研究[J]. 系统工程与电子技术, 2024, 46(10): 3265-3270.
[8]	聂千祁, 沙明辉, 朱应申. 基于改进残差神经网络的雷达信号识别方法[J]. 系统工程与电子技术, 2024, 46(10): 3356-3364.
[9]	施端阳, 林强, 胡冰, 杜小帅. 基于YOLO的航管一次雷达目标检测方法[J]. 系统工程与电子技术, 2024, 46(1): 143-151.
[10]	汪萌, 诸兵. 不确定性建模在2D和3D目标检测中的应用[J]. 系统工程与电子技术, 2023, 45(8): 2370-2376.
[11]	邵凯, 杜自群, 王光宇. 基于模型剪枝动态调整压缩率的CSI反馈方法[J]. 系统工程与电子技术, 2023, 45(8): 2615-2622.
[12]	崔天舒, 王栋, 黄振. 面向空间认知通信的轻量化网络自动调制分类方法[J]. 系统工程与电子技术, 2023, 45(7): 2220-2226.
[13]	姜雨, 袁琪, 胡志韬, 吴薇薇, 顾欣. 基于气象因素的机场进离港延误预测[J]. 系统工程与电子技术, 2023, 45(6): 1722-1731.
[14]	陈洋, 廖灿辉, 张锟, 刘建, 王鹏举. 基于自监督对比学习的信号调制识别算法[J]. 系统工程与电子技术, 2023, 45(4): 1200-1206.
[15]	张晔, 侯毅, 欧阳克威, 周石琳. 单变量序列数据分类方法综述[J]. 系统工程与电子技术, 2023, 45(2): 313-335.