基于膨胀卷积的多模态融合视线估计

基于表观的视线估计方法主要是在二维的三原色(red green blue,RGB)图像上进行,当头部在自由运动时视线估计精度较低,且目前基于卷积神经网络的表观视线估计都普遍使用池化来增大特征图中像素点的感受野,导致了特征图的信息损失,提出一种基于膨胀卷积神经网络的多模态融合视线估计模型.在该模型中,利用膨胀卷积设计了一种叫GENet(gaze estimation network)的网络提取眼睛的RGB和深度图像的特征图,并利用卷积神经网络的全连接层自动融合头部姿态和2种图像的特征图,从而进行视线估计.实验部分在公开数据集Eyediap上验证了设计的模型,并将设计的模型同其他视线估计模型进行比较.实验结果表明,提出的视线估计模型可以在自由的头部运动下准确地估计视线方向.

Dilated convolution-based gaze estimation via multimodel fusion

Appearance-based gaze estimation is mainly performed on two-dimensional red green blue(RGB)images. When the head is free to move, the gaze estimation accuracy is low, and the current appearance-based gaze estimation via the convolutional neural network generally uses pooling to increase the receptive field of pixel points in the feature map, resulting in information loss of the feature map. In this paper, a dilated convolution-based gaze estimation via multimodel fusion is proposed. In this model, a feature map of RGB and depth images of the eye are extracted by a network called gaze estimation network (GENet) using dilated convolution, and the head pose and the feature map of the two images are automatically fused using the fully connected layer of the convolutional neural network to make gaze estimation. The experimental validated the designed model on the public dataset Eyediap and compared the designed model with other gaze estimation models. The experimental results show that the proposed gaze estimation model can accurately estimate the direction of gaze under free head motion.