基于改进Faster R-CNN的手部位姿估计方法

基于视觉的手部位姿估计技术应用于诸多领域，具备着广泛的国际应用市场前景和巨大发展潜力。然而，手部自身存在检测目标过小、手指高自由度以及手部自遮挡等问题。通过对目前存在的难点分析，将手部位姿估计任务分为手部检测和手部关键点检测，提出基于改进的Faster R-CNN的手部位姿估计方法。首先提出基于改进的Faster R-CNN手部检测网络，将传统Faster R-CNN网络中的对ROI(regional of interest)的最大值池化，更改为ROI Align,并增加损失函数用于区分左右手。在此基础上增加了头网络分支用以训练输出MANO(hand model with articulated and non-rigid deformations)手部模型的姿态参数和形状参数，得到手部关键点三维坐标，最终得到手部的三维位姿估计结果。实验表明，手部检测结果中存在的自遮挡和尺度问题得到了解决，并且检测结果的准确性有所提高，本文手部检测算法准确率为85%,比传统Faster R-CNN算法提升13%。手部关键点提取算法在MSRA、ICVL、NYU三个数据集分别取得关键点坐标的均方误差值(k...

Research on Hand Pose Estimation Based on Improved Faster R-CNN Method

The visual-based hand pose estimation technology has been applied in many fields and has extensive international application market prospects and great development potential. However, the problems existing in hand, such as too small detection target, high degree of freedom of finger, and self-occlusion of hand. Through analyzing the existing difficulties, this paper divided the hand pose estimation tasks into hand detection and hand key point detection, and proposed the improved Faster R-CNN hand pose estimation method. Firstly, the improved Faster R-CNN hand detection network was proposed, which changes max pooling for ROI in the traditional Faster R-CNN network to ROI Align, and adds a loss function to distinguish left and right hands. On this basis, the head network branch was added to train the attitude parameters and shape parameters of output Mano hand model. Finally, the 3d coordinates of hand key points were obtained, and then the three-dimensional pose estimation results of hand were obtained. Experiments show that the problems of self-occlusion and scale in the hand detection results are solved, and the accuracy of the detection results are improved. The accuracy of the hand detection algorithm in this paper is 85%, which is 13% higher than the traditional Faster R-CNN algorithm. The hand key point extraction algorithm achieved KMSE results of 7.50, 6.32, and 8.50 in MSRA, ICVL, and NYU datasets, respectively.