| 结合加权KNN和自适应牛顿法的稳健Boosting方法 |
| |
| 引用本文： | 罗森林,赵惟肖,潘丽敏.结合加权KNN和自适应牛顿法的稳健Boosting方法[J].北京理工大学学报,2021,41(1):112-120. |
| |
| 作者姓名： | 罗森林 赵惟肖 潘丽敏 |
| |
| 作者单位： | 北京理工大学信息与电子学院,北京100081;北京理工大学信息与电子学院,北京100081;北京理工大学信息与电子学院,北京100081 |
| |
| 基金项目： | 国家"十三五"科技支撑计划项目（SQ2018YFC200004） |
| |
| 摘 要： | Boosting是机器学习领域中重要的集成学习方法,以AdaBoost为代表的Boosting算法通过在组合弱学习器时不断加强对错分类样本的关注以构建性能优异的强学习器,而该训练机制对噪声点的无差别对待易引发学习器对噪声过拟合,从而削弱算法的稳健性.针对该问题,提出结合加权KNN和自适应牛顿法的稳健Boosting方法.该方法首先通过加权KNN估计样本的噪声先验概率,然后使用噪声先验概率修正Logit损失构建一种新的损失函数,最后采用自适应牛顿法进行损失函数的优化求解.提出方法引导分类器在给予错分类样本更高权重的同时,对噪声先验概率大的样本给予相应的惩罚,使噪声样本的权重得到有效的缩减.结果表明,与其他稳健Boosting方法对比,在不同噪声水平下以及真实的医疗数据集的不同评价指标下,该方法表现出更好的稳健性,具有明显的应用价值.
|
| 关 键 词： | AdaBoost算法 噪声先验概率 加权KNN 损失函数 自适应牛顿法 |
| 收稿时间： | 2019/6/19 0:00:00 |
Robust Boosting Method Combining Weighted KNN and Adaptive Newton Method |
| |
| LUO Senlin,ZHAO Weixiao,PAN Limin.Robust Boosting Method Combining Weighted KNN and Adaptive Newton Method[J].Journal of Beijing Institute of Technology(Natural Science Edition),2021,41(1):112-120. |
| |
| Authors: | LUO Senlin ZHAO Weixiao PAN Limin |
| |
| Affiliation: | School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China |
| |
| Abstract: | Boosting is an essential ensemble learning method in the field of machine learning. continuously strengthening the attention of misclassified samples when combined with weak learners. Strengthening the attention continuously to misclassified samples with weak learners,the Boosting algorithms represented by AdaBoost are capable of building strong learners with excellent performance. However, there is an indiscriminate treatment of noise in the training mechanism, causing the learners likely to over-fit the noise and thus reducing the robustness of the algorithms. Aiming at the problem, a robust Boosting method combining weighted KNN and adaptive Newton method was proposed. Firstly, a weighted KNN method was used to estimate the noise prior probability of the sample. And then, the Logit loss was modified with the noise prior probability to construct a new loss function. Finally, the loss function was optimized based on an adaptive Newton method. The proposed method was arranged to give a corresponding penalty to the samples with a high probability of noise when the misclassified samples got a higher weight from the classifier, so as to make the weight of the noise samples be effectively reduced. The experiment results show that, compared with other robust Boosting methods, the proposed method has better robustness under different noise levels as well as under different evaluation criterions in a real medical data set, having obvious application value. |
| |
| Keywords: | AdaBoost algorithm noise prior probability weighted KNN loss function adaptive Newton method |
| 本文献已被 CNKI 万方数据 等数据库收录！ |
| 点击此处可从《北京理工大学学报》浏览原始摘要信息 |
|
点击此处可从《北京理工大学学报》下载全文 |
|
