arXiv research

Multivariate boosted trees improve forecasting and control by capturing correlated predictions.

Boosted Control Functions improve prediction under distributional shifts.

We study the question of how to aggregate controllers for dynamical systems in order to improve their performance. To this end, we propose a framework of boosting for online control. Our main result is an efficient boosting algorithm that combines weak controllers into a provably more accurate one. Empirical evaluation…

Infinitesimal gradient boosting is a new algorithm derived from gradient boosting.

Gradient boosting is a prediction method that iteratively combines weak learners to produce a complex and accurate model. From an optimization point of view, the learning procedure of gradient boosting mimics a gradient descent on a functional variable. This paper proposes to build upon the proximal point algorithm, wh…

Machine learning boosts RCT efficiency by controlling type I error and improving statistical power.

Gradient boosting for spatial regression models improves prediction accuracy.

Unified theory and practical insights for multicalibration boosting.

Proposes a boosting framework for sparsity in grouped covariates.

Uplift modeling is an area of machine learning which aims at predicting the causal effect of some action on a given individual. The action may be a medical procedure, marketing campaign, or any other circumstance controlled by the experimenter. Building an uplift model requires two training sets: the treatment group, w…

Empirical researchers are increasingly faced with rich data sets containing many controls or instrumental variables, making it essential to choose an appropriate approach to variable selection. In this paper, we provide results for valid inference after post- or orthogonal $L_2$-Boosting is used for variable selection.…

SGLB boosts machine learning with Langevin diffusion for multimodal loss functions.

Gradient boosting algorithm for spatial panel models improves estimation in high-dimensional settings.

In this survey, we discuss several different types of gradient boosting algorithms and illustrate their mathematical frameworks in detail: 1. introduction of gradient boosting leads to 2. objective function optimization, 3. loss function estimations, and 4. model constructions. 5. application of boosting in ranking.

Boosting-inspired framework for federated learning with progressive model personalization.

Boosts Q-learning by using value function bounds.

Paper proposes E/PD-Control for better neural network training.

Boosting can efficiently optimize any loss function without requiring first-order information.

We introduce a novel boosting algorithm called `KTBoost' which combines kernel boosting and tree boosting. In each boosting iteration, the algorithm adds either a regression tree or reproducing kernel Hilbert space (RKHS) regression function to the ensemble of base learners. Intuitively, the idea is that discontinuous …

Kernel-based L2-boosting with structure constraints improves regression efficiency.

New algorithms boost SAT solver performance by optimizing restart strategies.

Boost-R uses gradient boosted trees for analyzing recurrence data.

Modern biotechnologies often result in high-dimensional data sets with much more variables than observations (n $\ll$ p). These data sets pose new challenges to statistical analysis: Variable selection becomes one of the most important tasks in this setting. We assess the recently proposed flexible framework for variab…

Boosting theory extended to handle cost-sensitive and multi-objective losses.

The paper provides a uniform convergence bound for smooth calibration error and its relationship with functional gradient.

Boosting is a generic learning method for classification and regression. Yet, as the number of base hypotheses becomes larger, boosting can lead to a deterioration of test performance. Overfitting is an important and ubiquitous phenomenon, especially in regression settings. To avoid overfitting, we consider using $l_1$…

Gradient boosting adapted for vector inputs.

GrowNet uses shallow neural networks for gradient boosting, outperforming existing methods.

Robust boosting improves regression accuracy in noisy data.

We introduce a useful tool for analyzing boosting algorithms called the ``smooth margin function,'' a differentiable approximation of the usual margin for boosting algorithms. We present two boosting algorithms based on this smooth margin, ``coordinate ascent boosting'' and ``approximate coordinate ascent boosting,'' w…

We present SEBOOST, a technique for boosting the performance of existing stochastic optimization methods. SEBOOST applies a secondary optimization process in the subspace spanned by the last steps and descent directions. The method was inspired by the SESOP optimization method for large-scale problems, and has been ada…

Develop gradient boosting for estimating covariate-dependent GP distributions in insurance.

Classification outperforms regression in portfolio construction, yielding higher Sharpe ratios.

Boosted conformal procedure improves prediction intervals.

New statistical methods improve explainability of boosting models.

We present a new algorithm for boosting generalized additive models for location, scale and shape (GAMLSS) that allows to incorporate stability selection, an increasingly popular way to obtain stable sets of covariates while controlling the per-family error rate (PFER). The model is fitted repeatedly to subsampled data…

New approach uses text generation to boost AI agent development.

Functional central limit theorem for kernel gradient flow and infinitesimal gradient boosting

Statistical boosting algorithms have triggered a lot of research during the last decade. They combine a powerful machine-learning approach with classical statistical modelling, offering various practical advantages like automated variable selection and implicit regularization of effect estimates. They are extremely fle…

This paper examines the role and efficiency of the non-convex loss functions for binary classification problems. In particular, we investigate how to design a simple and effective boosting algorithm that is robust to the outliers in the data. The analysis of the role of a particular non-convex loss for prediction accur…

We consider the problem of learning a forest of nonlinear decision rules with general loss functions. The standard methods employ boosted decision trees such as Adaboost for exponential loss and Friedman's gradient boosting for general loss. In contrast to these traditional boosting algorithms that treat a tree learner…

RFRBoost uses random features to boost deep residual neural networks, improving performance and computational efficiency.

Fair MP-Boost improves fairness and interpretability in boosting methods.

A new gradient tree boosting framework reduces variance and accelerates performance.

Develops gradient boosting for multi-label classification.

It is known that Boosting can be interpreted as a gradient descent technique to minimize an underlying loss function. Specifically, the underlying loss being minimized by the traditional AdaBoost is the exponential loss, which is proved to be very sensitive to random noise/outliers. Therefore, several Boosting algorith…

Gradient boosting method enforced with individual fairness.

Boosted Difference of Convex Functions Algorithm solves VaR constrained portfolio optimization.