结合加权KNN和自适应牛顿法的稳健Boosting方法

Boosting is an essential ensemble learning method in the field of machine learning. continuously strengthening the attention of misclassified samples when combined with weak learners. Strengthening the attention continuously to misclassified samples with weak learners, the Boosting algorithms represented by AdaBoost are capable of building strong learners with excellent performance. However, there is an indiscriminate treatment of noise in the training mechanism, causing the learners likely to over-fit the noise and thus reducing the robustness of the algorithms. Aiming at the problem, a robust Boosting method combining weighted KNN and adaptive Newton method was proposed. Firstly, a weighted KNN method was used to estimate the noise prior probability of the sample. And then, the Logit loss was modified with the noise prior probability to construct a new loss function. Finally, the loss function was optimized based on an adaptive Newton method. The proposed method was arranged to give a corresponding penalty to the samples with a high probability of noise when the misclassified samples got a higher weight from the classifier, so as to make the weight of the noise samples be effectively reduced. The experiment results show that, compared with other robust Boosting methods, the proposed method has better robustness under different noise levels as well as under different evaluation criterions in a real medical data set, having obvious application value.