Ultra-dense Heterogeneous network, as an effective means to meet the needs of exponential growth in data traffic and high user rate, has become a member of 5G's key technologies. While ultra-densely deployed small cell base stations will inevitably result in increased power consumption and access fairness problem. The problem of user association (also called user access) with energy efficiency(EE) optimization under given rate demands was investigated in this paper. The original power consumption model was improved to make it more accurately reflect the actual transmit power and whole power consumption of the base station. And then, Lagrange duality and bisection method were used to optimize and adjust the proportion of resource allocated to users and the transmit power provided by the base station for each associated user. Combined with matching theory, a user association algorithm considering both EE and access fairness was proposed. Performance simulation and comparison results show that the proposed approach performs well in access fairness and effectively improves EE of the system.