arXiv research

We introduce a dynamical system which we call the AdaBoost flow. The flow is defined by a system of ODEs with control. We show that three algorithms of the AdaBoost family (i) the AdaBoost algorithm of Schapire and Freund (ii) the arc-gv algorithm of Breiman (iii) the confidence rated prediction of Schapire and Singer …

AdaBoost's success explained through noise influence measure.

AdaBoost's classifier and margins converge to a known value.

Paper explains AdaBoost's overfitting resistance from feature learning perspective.

Boosting with tempered exponential measures improves AdaBoost's convergence rate.

AdaBoost cycles in probability simplex dynamics.

Paper proposes AdaBoost-assisted ELM for efficient online sequential classification.

The significance of the study of the theoretical and practical properties of AdaBoost is unquestionable, given its simplicity, wide practical use, and effectiveness on real-world datasets. Here we present a few open problems regarding the behavior of "Optimal AdaBoost," a term coined by Rudin, Daubechies, and Schapire …

AdaBoost is one of the most popular ML algorithms. It is simple to implement and often found very effective by practitioners, while still being mathematically elegant and theoretically sound. AdaBoost's interesting behavior in practice still puzzles the ML community. We address the algorithm's stability and establish m…

This paper reconciles different views on AdaBoost to better understand its dynamics.

Improved AdaBoost method handles label uncertainty and class overlap.

There is a large literature explaining why AdaBoost is a successful classifier. The literature on AdaBoost focuses on classifier margins and boosting's interpretation as the optimization of an exponential likelihood function. These existing explanations, however, have been pointed out to be incomplete. A random forest …

AdaGCN uses AdaBoost to efficiently integrate high-order neighbor knowledge in graph neural networks.

We design a randomised parallel version of Adaboost based on previous studies on parallel coordinate descent. The algorithm uses the fact that the logarithm of the exponential loss is a function with coordinate-wise Lipschitz continuous gradient, in order to define the step lengths. We provide the proof of convergence …

AdaBoost improves binary classification in robust one-bit compressed sensing with adversarial errors.

Boosting is known to be sensitive to label noise. We studied two approaches to improve AdaBoost's robustness against labelling errors. One is to employ a label-noise robust classifier as a base learner, while the other is to modify the AdaBoost algorithm to be more robust. Empirical evaluation shows that a committee of…

We introduce a useful tool for analyzing boosting algorithms called the ``smooth margin function,'' a differentiable approximation of the usual margin for boosting algorithms. We present two boosting algorithms based on this smooth margin, ``coordinate ascent boosting'' and ``approximate coordinate ascent boosting,'' w…

The AdaBoost algorithm was designed to combine many "weak" hypotheses that perform slightly better than random guessing into a "strong" hypothesis that has very low error. We study the rate at which AdaBoost iteratively converges to the minimum of the "exponential loss." Unlike previous work, our proofs do not require …

We offer a novel view of AdaBoost in a statistical setting. We propose a Bayesian model for binary classification in which label noise is modeled hierarchically. Using variational inference to optimize a dynamic evidence lower bound, we derive a new boosting-like algorithm called VIBoost. We show its close connections …

We present a new online boosting algorithm for adapting the weights of a boosted classifier, which yields a closer approximation to Freund and Schapire's AdaBoost algorithm than previous online boosting algorithms. We also contribute a new way of deriving the online algorithm that ties together previous online boosting…

Boosting methods are highly popular and effective supervised learning methods which combine weak learners into a single accurate model with good statistical performance. In this paper, we analyze two well-known boosting methods, AdaBoost and Incremental Forward Stagewise Regression (FS$_\varepsilon$), by establishing t…

MP-Boost boosts accuracy faster and more interpretable than AdaBoost.

Optimal algorithm converts weak to strong learner with less data.

New algorithms improve boosting by optimizing chance-corrected measures.

It is known that Boosting can be interpreted as a gradient descent technique to minimize an underlying loss function. Specifically, the underlying loss being minimized by the traditional AdaBoost is the exponential loss, which is proved to be very sensitive to random noise/outliers. Therefore, several Boosting algorith…

CUSBoost improves imbalanced classification using clustering and boosting.

A new model of learning corrects for chance to improve learning outcomes.

The paper explores variations of ensemble methods for cancer survival prediction.

Improved risk assessment for UBI using telematics data and AdaBoost.

We consider the problem of learning a forest of nonlinear decision rules with general loss functions. The standard methods employ boosted decision trees such as Adaboost for exponential loss and Friedman's gradient boosting for general loss. In contrast to these traditional boosting algorithms that treat a tree learner…

Learning structured outputs with general structures is computationally challenging, except for tree-structured models. Thus we propose an efficient boosting-based algorithm AdaBoost.MRF for this task. The idea is based on the realization that a graph is a superimposition of trees. Different from most existing work, our…

Unified framework for cost-sensitive ensemble learning.

Improved multi-class AdaBoost algorithm with stronger weak learnability condition.

Paper develops DL methods for signal demodulation in wireless comms.

SBPMT combines bagging and boosting for improved classification.

New margin bound improves generalization for voting classifiers.

Paper explores ML demodulation for VLC, presenting three methods and testing accuracy.

Boosting is a general method of generating many simple classification rules and combining them into a single, highly accurate rule. In this talk, I will review the AdaBoost boosting algorithm and some of its underlying theory, and then look at how this theory has helped us to face some of the challenges of applying Ada…

Adaptive weighting improves Deep Forest classifier's performance.

We describe and analyze a new boosting algorithm for deep learning called SelfieBoost. Unlike other boosting algorithms, like AdaBoost, which construct ensembles of classifiers, SelfieBoost boosts the accuracy of a single network. We prove a $\log(1/ε)$ convergence rate for SelfieBoost under some "SGD success" assumpti…

Boosting algorithm improved by DRO framework for financial prediction.

Designing effective and efficient classifier for pattern analysis is a key problem in machine learning and computer vision. Many the solutions to the problem require to perform logic operations such as `and', `or', and `not'. Classification and regression tree (CART) include these operations explicitly. Other methods s…

Paper introduces new indicators for forecasting crude oil prices using short news headlines.

APA improves brain decoding accuracy for visual stimuli.

Automated method identifies key hyperparameters across datasets.

Paper explores connections between loss functions and consistency in binary classification and regression.

Study improves cryptocurrency volatility forecasting using multiple data sources.

LoAdaBoost boosts federated machine learning efficiency for ICU data.