Aiming at the problems of fatigue driving detection, we propose a deep convolutional neural network algorithm combining softmax loss and center loss to detect the facial fatigue state of the driver. Firstly,a pre-trained face detector in Dlib library containing HOG(Histogram of Oriented Gradient) and SVM (Support Vector Machine)algorithm was used to detect the presence of driver’s faces. Then, the driver’s face of the 68 key points, eyes and mouth are located through the ERT(Ensemble of Regression Trees) algorithm. Finally, in order to optimize the problem that the sortmax loss has large intra-class spacing in deep convolutional network classification. The center loss function was introduced to optimize this problem and improve the difference of inter-class, the compactness of intra-class and the recognition accuracy of the driver’s facial fatigue state. The experimental results on the self-built test set and the YawDD yawn data set demonstrated that the method can accurately identify the driver’s fatigue state, and the identification average accuracy rate of our algorithm was about 98.81%. This method can automatically extract fatigue features and improve training accuracy and test recognition rate compared with the traditional fatigue driving detection algorithm. Robustness was significantly improved. The recognition accuracy increased by approximately 5.09% compared with the unimproved deep convolution neural network.