According to the fact that digital images mainly contain Gauss and salt pepper noise, an anisotropic hybrid diffusion model with source term was proposed in this paper. Traditional edge-enhancing and coherence-enhancing models also had the contrast between coherent structures and the background decreased while having the image denoised. In this paper a source term was introduced into the diffusion equation, and its impacts on each term of diffusion system were fully considered. The improved model can not only perfectly denoise the noisy image, but also effectively maintain the contrast between coherent structures and the background. Experimental results demonstrated that the proposed method can achieve a desired result of denoising and restoring image texture information, and create a better visual effect.