基于语义约束与Graph Cuts的稠密三维场景重建

本文提出一种新颖、有效的稠密三维场景重建算法.在城市建筑场景的重建中,为了快速恢复稠密、准确的深度信息,本文算法首先在视图中对建筑区域进行了语义分割以降低非重建区域(如天空、地面等)的干扰,在提高整体重建速度的同时也增强了采用平面模型对其进行重建的可靠性;然后,在通过基于DAISY特征的空间点扩散方法获取的初始深度图的基础上,针对传统算法难以重建的弱纹理、倾斜表面等区域,本文算法依据场景分段平滑的假设,在超像素级MRF能量优化框架中对其相应的空间平面进行了推断.由于能量函数融合了初始深度图的约束、空间平面先验及空间平面间的几何关系等信息,而且候选平面集通过平面拟合和已知平面约束下的多方向平面扫描两种方法构造,使得相应的两阶段迭代Graph Cuts对能量函数的求解更快速和精确.在标准数据集和真实数据上的实验表明,本文算法能有效克服光照变化、透视畸变、弱纹理区域等因素的影响,快速恢复建筑区域完整的深度图.

Dense 3D scene reconstruction based on semantic constraint and graph cuts

A new and effective dense 3D scene reconstruction algorithm is proposed in this work. At first, in order to speedup the reconstruction process and reduce possible unnecessary interference by unrelated regions, such as sky, ground, etc., to the desired architectural scene, a semantic image segmentation step is carried out at first to mask those uninterested parts. After that, an initial depth map is obtained by Daisy-feature based 3D point propagation. To tackle the difficult dense reconstruction problem of poorly textured regions and slanted surfaces, the problem is fornmlated as a plane inference problem under the MRF energy minimization framework to assign a possible planar patch to each image superpixel. More specifically, at first, the plane principal directions and possible non-principal are clustered by the affinity propagation from the reconstructed point clouds, then the plane inference is carried out along the principal directions first, followed by the non-principal directions. In each of the above two plane inference steps, a Graph Cuts based algorithm is employed to minimize a properly designed cost function which takes into account a variety of factors, such as appearance similarity, the constraints from the initial depth map, spatial planes priors, as well as geometric relations between spatial planes etc. Experiments on the standard data set as well as our own data set show that the our proposed algorithm can satisfactorily handle many challenging factors, e.g. illumination variation, perspective distortion, poorly textured regions, etc., to efficiently recover complete dense depth map of buildings.