Robust vehicle detection in complex environment has both high theory and application value. Focus on the shortage of low identification ability of traditional descriptor, this paper proposed a visual saliency feature (VSF) and sparse representation based vehicle detection algorithm. Firstly, inspired by the human visual characteristics, based on the eye’s gazing mechanism, the training samples are extracted by visual saliency features information. By using compressed sensing mechanism, the samples are expressed as an over complete dictionary through sparse representation. Then the over complete dictionary is trained with LC - KSVD to reconstruct the sample signals. Finally candidate targets are judged to be a vehicle or not by reconstructed residuals in the dictionary. Experimental results demonstrate that, with 0.5/frame false detection rate, the method can reach 94.7% detection rate in good conditions; with the same false detection rate, this method is still able to achieve detection rate of 92.2% in adverse conditions. Comparison results show that this method is superior to conventional vehicle detection methods.