基于Horseshoe+先验的惩罚置信区域变量选择方法

针对高维稀疏线性回归问题,相关变量的数量远远少于不相关变量.相关变量的变量选择问题对于传统的频率论正则化方法是一大挑战.现有的贝叶斯惩罚置信区域法通过将模型拟合与变量选择分离,在联合后验置信区域内搜索最稀疏解,从而得到稀疏模型解.且该方法在高维变量选择效果上优于常用的变量选择方法.在此基础上,针对高维稀疏模型,将原方法中依赖的共轭正态先验替换成针对"稀疏信号勘测问题"提出的Horseshoe+先验,利用Horseshoe+先验对小系数"重"压缩与大系数几乎零压缩的理论特性,实现对稀疏回归系数的稳健估计.通过数据仿真模拟不同稀疏程度下的高维稀疏线性回归,并将基于Horseshoe+先验的惩罚置信区域法分别与基于正态先验以及Laplace先验的该方法进行比较,结果表明基于Horseshoe+先验的惩罚置信区域法在高维稀疏线性回归问题具有更好的变量选择效果与预测效果.

Variable selection methods for penalized credible regions based on the Horseshoe+prior

For high-dimensional sparse linear regression, the number of relevant predictors is much less than that of irrelevant predictors. Selection of relevant predictors is a challenge to the traditional frequentist regularization methods. The existing method of Bayesian penalized credible regions separates the model fitting and the variable selection procedure, obtains the sparse model solution by searching for the sparsest solution within the joint posterior credible regions, which is shown to outperform common methods in the high-dimensional linear regressions. Based on the high-dimensional sparse model, the conjugate normal prior dependent on the original method is replaced by the Horseshoe+prior, which is born for the detection problem of spares signals. Using the theoretical property of the Horseshoe+prior that has little shrinkage on large coefficients but great shrinkage on small coefficients, a robust estimation of the sparse coefficients can be achieved. The high-dimensional sparse linear regressions under different scarcity degrees are simulated by data simulations, and a comparison is conducted among selection methods of penalized credible regions based on the Horseshoe+prior, the normal prior and the Laplace prior. The results show that this method behaves better in variable selection and prediction in high-dimensional sparse linear regression.