基于引导对抗网络的人体深度图像修补方法

移动设备配备的小型深度相机采集到的人体深度图像存在严重的孔洞问题。针对该问题,提出基于深度学习的引导对抗网络。使用基于堆叠沙漏网络的引导器从RGB图像中提取人体部分分割特征和深度类别特征;在上述人体特征引导下,使用独特的生成器修复人体深度图像中的孔洞。为使结果更加逼真,加入判别器在网络训练过程中对生成器进行优化调整。实验结果显示,该方法在现有的人体数据集和小型深度相机采集的数据集上,都能很好解决孔洞问题,均取得比现有方法更好的效果。

Human Depth Maps Restoration Based on Guided GAN

The depth maps captured by a small depth camera on mobile devices suffer from the problem of severe holes. The Guided Generative Adversarial Network (Guided GAN) based on deep learning is proposed to restore human depth maps with above problems. The high-precision human segmentation features and depth class features are extracted from the monocular RGB image by the guider based on the stacked hourglass network. The holes in the human depth maps are filled by the special generator under the guidance of the extracted human features. In order to get the more realistic results, the discriminator is introduced to optimize the generator. The experimental results show that the proposed method can restore the human depth maps effectively in the existing human datasets and the dataset collected by the small depth camera. It achieves better results than the existing method.