基于视差范围估计和改进代价的半全局匹配

doi:10.12305/j.issn.1001-506X.2022.02.05

摘要/Abstract

摘要：

针对传统半全局算法对视差范围内未知场景通常人为地设定一个视差范围造成计算资源浪费, 同时利用传统Census变换进行代价计算限制视差精度的不足, 提出了基于视差范围估计和改进代价的半全局匹配算法。首先, 采用多种特征算子同时提取图像对的特征点, 通过快速最近邻搜索进行特征点匹配, 利用立体匹配的约束条件筛选匹配点, 计算匹配点对的视差值, 估计视差范围; 然后, 在此基础上, 分别对图像的亮度、梯度和边缘信息进行Census变换, 构建新的代价计算函数。实验结果表明, 与传统算法相比, 改进算法的平均误匹配率降低了6.37%, 计算时间缩短了95%以上。

关键词: 立体匹配, 半全局匹配, 视差范围估计, 代价计算, Census变换

Abstract:

Traditional semi-global method usually sets a disparity range artificially for scenes with unknown disparity range, resulting in a waste of computing resources, and uses the traditional Census transform for cost computation, limiting the lack of parallax aceuracy. A semi-global matching method based on disparity range estimation and improved cost is proposed. First, a variety of feature operators are used to extract the feature points of the image pairs at the same time, and the fast nearest neighbor search method is used to match the feature points, the matching points are filtered by the constraints of stereo matching, and the disparity of the matching point pairs are computed to estimate the disparity range. Then, on this basis, the intensity, gradient and edge information of the image are processed by Census transform respectively to construct a new cost computation function. The experimental results demonstrate that, compared with the traditional method, the average error matching rate of the improved method is reduced by 6.37%, and the computation time is shortened by more than 95%.

Key words: stereo matching, semi-global matching, disparity range estimation, cost computation, Census transform

中图分类号:

TP391

彭妍, 郭君斌, 于传强, 李静波. 基于视差范围估计和改进代价的半全局匹配[J]. 系统工程与电子技术, 2022, 44(2): 394-400.

Yan PENG, Junbin GUO, Chuanqiang YU, Jingbo LI. Semi-global matching method based on disparity range estimation and improved cost[J]. Systems Engineering and Electronics, 2022, 44(2): 394-400.

图/表 13

图1

图2

图3

表1

图4

图5

表2

图6

表3

表4

表5

表6

表7

参考文献 32

1	HAMZAH R A , KADMIN A F , HAMID M S , et al. Improvement of stereo matching algorithm for 3D surface reconstruction[J]. Signal Processing: Image Communication, 2018, 65, 165- 172. doi: 10.1016/j.image.2018.04.001
2	PON A D, KU J, LI C Y, et al. Objectcentric stereo matching for 3D object detection[C]//Proc. of the IEEE International Conference on Robotics and Automation, 2020: 8383-8389.
3	YANG G R, SONG X, HUANG C Q, et al. Driving stereo: a large-scale dataset for stereo matching in autonomous driving scenarios[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 899-908.
4	RAUTER M, ABSEHER C, SAFAR M. Augmenting virtual reality with near real world objects[C]//Proc. of the IEEE Conference on Virtual Reality and 3D User Interfaces, 2019: 1134-1135.
5	SCHARSTEIN D , SZELISKI R . A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[J]. International Journal of Computer Vision, 2002, 47 (3): 37- 42.
6	HIRSCHMULLER H . Stereo processing by semiglobal mat-ching and mutual information[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2007, 30 (2): 328- 341.
7	CAMBUIM L F S , OLIVEIRA L A , BARROS E N S , et al. An FPGA-based real-time occlusion robust stereo vision system using semi-global matching[J]. Journal of Real-Time Image Processing, 2020, 17 (4): 1447- 1468. doi: 10.1007/s11554-019-00902-w
8	SEKI A, POLLEFEYS M. SGM-Nets: semi-global matching with neural networks[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 231-240.
9	毛苗, 吴艳楠, 秦开怀. 快速视差范围估计算法及其应用[J]. 计算机辅助设计与图形学学报, 2015, 27 (1): 120- 127.
	MAO M , WU Y N , QIN K H . Fast disparity range estimation and its applications[J]. Journal of Computer-Aided Design and Computer Graphics, 2015, 27 (1): 120- 127.
10	董本志, 张丽君, 景维鹏. 基于前向搜索的快速视差范围估计方法[J]. 计算机工程与应用, 2017, 53 (24): 213- 218. doi: 10.3778/j.issn.1002-8331.1706-0217
	DONG B Z , ZHANG L J , JING W P . Method of fast disparity range estimation based on forward search[J]. Computer Engineering and Applications, 2017, 53 (24): 213- 218. doi: 10.3778/j.issn.1002-8331.1706-0217
11	李海超, 李莹, 顾征. 利用平面约束的月球车立体匹配[J]. 光学精密工程, 2019, 27 (2): 433- 439.
	LI H C , LI Y , GU Z . Stereo matching method for lunar rover using the plane constraint[J]. Optics and Precision Engineering, 2019, 27 (2): 433- 439.
12	YANG W P , LI X W , YANG B , et al. A novel stereo matching algorithm for digital surface model (DSM) generation in water areas[J]. Remote Sensing, 2020, 12 (5): 870- 884. doi: 10.3390/rs12050870
13	MEI X X, SUN X, ZHOU M J, et al. On building an accurate stereo matching system on graphics hardware[C]//Proc. of the IEEE International Conference on Computer Vision Workshops, 2011: 467-474.
14	闫利, 王芮, 刘华, 等. 基于改进代价计算和自适应引导滤波的立体匹配[J]. 光学学报, 2018, 38 (11): 257- 267.
	YAN L , WANG R , LIU H , et al. Stereo matching method based on improved cost computation and adaptive guided filter[J]. Acta Optica Sinica, 2018, 38 (11): 257- 267.
15	ZHU S P , YAN L . Local stereo matching algorithm with efficient matching cost and adaptive guided image filter[J]. The Visual Computer, 2017, 33 (9): 1087- 1102. doi: 10.1007/s00371-016-1264-6
16	LIU H M , WANG R , XIA Y , et al. Improved cost computation and adaptive shape guided filter for local stereo matching of low texture stereo images[J]. Applied Sciences, 2020, 10 (5): 1869- 1874. doi: 10.3390/app10051869
17	GUO C , JIA F X , TANG W Z , et al. A fast method for image matching and registration based on SIFT algorithm and image pyramid[J]. Journal of Physics: Conference Series, 2020, 1449 (1): 012119. doi: 10.1088/1742-6596/1449/1/012119
18	BAY H , ESS A , TUYTELAARS T , et al. Speeded-up robust features[J]. Computer Vision and Image Understanding, 2008, 110 (3): 346- 359. doi: 10.1016/j.cviu.2007.09.014
19	LIU J , BU F L . Improved RANSAC features image-matching method based on SURF[J]. The Journal of Engineering, 2019, 2019 (23): 9118- 9122. doi: 10.1049/joe.2018.9198
20	XIONG W B , TIAN W H , YANG Z , et al. Improved FAST corner-detection method[J]. The Journal of Engineering, 2019, 2019 (19): 5493- 5497. doi: 10.1049/joe.2019.0400
21	LEUTENEGGER S, CHLI M, SIEGWART R Y. BRISK: binary robust invariant scalable keypoints[C]//Proc. of the International Conference on Computer Vision, 2011: 2548-2555.
22	LUO C , YANG W , HUANG P , et al. Overview of image matching based on ORB algorithm[J]. Journal of Physics: Conference Series, 2019, 1237 (3): 032020. doi: 10.1088/1742-6596/1237/3/032020
23	ZABIH R, WOODFILL J. Non-parametric local transforms for computing visual correspondence[C]//Proc. of the European Conference on Computer Vision, 1994: 151-158.
24	SCHARSTEIN D, SZELISKI R. High-accuracy stereo depth maps using structured light[C]//Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003: 195-202.
25	YANG Q, WANG L M, YANG R Y, et al. Real-time global stereo matching using hierarchical belief propagation[C]// Proc. of the British Machine Vision Conference, 2006: 989-998.
26	GUPTA R K, CHO S Y. Real-time stereo matching using adaptive binary window[C]//Proc. of the International Symposium on 3D Data Processing, Visualization, and Transmission, 2010.
27	TOMBARI F, MATTOCCIA S, STEFANO L D, et al. Near real-time stereo based on effective cost aggregation[C]//Proc. of the 19th International Conference on Pattern Recognition, 2008.
28	SALMEN J, SCHLIPSING M, EDELBRUNNER J, et al. Real-time stereo vision: making more out of dynamic programming[C]//Proc. of the International Conference on Computer Analysis of Images and Patterns, 2009: 1096-1103.
29	KOSOV S, THORMAHLEN T, SEIDEL H P. Accurate real-time disparity estimation with variational methods[C]//Proc. of the 5th International Symposium on Visual Computing, 2009: 796-807.
30	WANG L, LIAO M, GONG M, et al. High-quality real-time stereo using adaptive cost aggregation and dynamic programming[C]//Proc. of the 3rd International Symposium on 3D Data Processing, Visualization, and Transmission, 2006: 798-805.
31	GONG M, YANG Y H. Near real-time reliable stereo matching using programmable graphics hardware[C]//Proc. of the Computer Society Conference on Computer Vision and Pattern Recognition, 2005: 924-931.
32	VEKSLER O. Stereo correspondence by dynamic programming on a tree[C]//Proc. of the Computer Society Conference on Computer Vision and Pattern Recognition, 2005: 384-390.

视差范围	锥面
基于特征点的视差范围估计	[18, 52]
真实的视差范围	[0, 59]

参数	参数值	参数	参数值
α	10	μ	2
β	10	m	5
λ	150	n	5

算法	筑波			金星			泰迪			锥面			均值
算法	non	all	disc	non	all	disc	non	all	disc	non	all	disc	均值
传统算法	7.48	8.62	16.9	7.16	8.00	16.8	13.2	20.0	22.4	10.8	17.0	20.0	14.0
所提算法	2.67	4.17	11.7	0.89	1.71	12.2	6.12	13.1	17.4	3.18	9.21	9.49	7.65

算法	筑波	金星	泰迪	锥面
传统算法	24.308	36.549	36.836	37.048
所提算法	0.954	1.370	1.636	1.821

图像对	筛选前特征点对	筛选后特征点对	特征处理时间/s (检测+匹配+筛选)
筑波	294 6	838	0.227
金星	469 2	115 0	0.369
泰迪	387 9	996	0.339
锥面	499 5	105 3	0.406