arXiv research

Proposes a boosting framework for sparsity in grouped covariates.

We propose a statistical inference framework for the component-wise functional gradient descent algorithm (CFGD) under normality assumption for model errors, also known as $L_2$-Boosting. The CFGD is one of the most versatile tools to analyze data, because it scales well to high-dimensional data sets, allows for a very…

Boosting method for causal SEMs from observational data.

The link between different psychophysiological measures during emotion episodes is not well understood. To analyse the functional relationship between electroencephalography (EEG) and facial electromyography (EMG), we apply historical function-on-function regression models to EEG and EMG data that were simultaneously r…

MIBoost boosts variable selection with multiple imputation.

Multi-output prediction deals with the prediction of several targets of possibly diverse types. One way to address this problem is the so called problem transformation method. This method is often used in multi-label learning, but can also be used for multi-output prediction due to its generality and simplicity. In thi…

Paper proposes a privacy-preserving method for estimating complex models.

We present a new algorithm for boosting generalized additive models for location, scale and shape (GAMLSS) that allows to incorporate stability selection, an increasingly popular way to obtain stable sets of covariates while controlling the per-family error rate (PFER). The model is fitted repeatedly to subsampled data…

Boosting strategies for merging vs. ensembling studies analyzed.

As an adaptive, interpretable, robust, and accurate meta-algorithm for arbitrary differentiable loss functions, gradient tree boosting is one of the most popular machine learning techniques, though the computational expensiveness severely limits its usage. Stochastic gradient boosting could be adopted to accelerates gr…

This research tackles uncertainty in gradient boosting models using ensemble methods.

Gradient boosting with randomized trees reduces discontinuities and complexity.

A method to improve gradient boosting models using stacking.

Wasserstein gradient boosting predicts probability distributions for supervised learning.

In this survey, we discuss several different types of gradient boosting algorithms and illustrate their mathematical frameworks in detail: 1. introduction of gradient boosting leads to 2. objective function optimization, 3. loss function estimations, and 4. model constructions. 5. application of boosting in ranking.

Infinitesimal gradient boosting is a new algorithm derived from gradient boosting.

Boosting algorithms are frequently used in applied data science and in research. To date, the distinction between boosting with either gradient descent or second-order Newton updates is often not made in both applied and methodological research, and it is thus implicitly assumed that the difference is irrelevant. The g…

Functional central limit theorem for kernel gradient flow and infinitesimal gradient boosting

Unified framework for statistical inference in gradient boosting regression.

This paper introduces Stochastic Gradient Langevin Boosting (SGLB) - a powerful and efficient machine learning framework that may deal with a wide range of loss functions and has provable generalization guarantees. The method is based on a special form of the Langevin diffusion equation specifically designed for gradie…

New algorithm improves convergence of gradient boosting trees.

In this paper we present CatBoost, a new open-sourced gradient boosting library that successfully handles categorical features and outperforms existing publicly available implementations of gradient boosting in terms of quality on a set of popular publicly available datasets. The library has a GPU implementation of lea…

Boost-R uses gradient boosted trees for analyzing recurrence data.

Gradient tree boosting is a prediction algorithm that sequentially produces a model in the form of linear combinations of decision trees, by solving an infinite-dimensional optimization problem. We combine gradient boosting and Nesterov's accelerated descent to design a new algorithm, which we call AGB (for Accelerated…

Gradient boosting is a prediction method that iteratively combines weak learners to produce a complex and accurate model. From an optimization point of view, the learning procedure of gradient boosting mimics a gradient descent on a functional variable. This paper proposes to build upon the proximal point algorithm, wh…

Gradient boosting can be seen as Gaussian process inference.

FGBoost boosts gradient boosting for complex data.

Gradient boosting method enforced with individual fairness.

Gradient boosting improved with lassoed trees achieves faster convergence.

Stagewise boosting improves gradient boosting for distributional regression.

sGBM speeds up gradient boosting by parallelizing and adapting base learners.

Boosted Frank-Wolfe accelerates optimization for nonconvex problems.

New GPU algorithm boosts machine learning with larger datasets.

TF Boosted Trees (TFBT) is a new open-sourced frame-work for the distributed training of gradient boosted trees. It is based on TensorFlow, and its distinguishing features include a novel architecture, automatic loss differentiation, layer-by-layer boosting that results in smaller ensembles and faster prediction, princ…

We present Natural Gradient Boosting (NGBoost), an algorithm for generic probabilistic prediction via gradient boosting. Typical regression models return a point estimate, conditional on covariates, but probabilistic regression models output a full probability distribution over the outcome space, conditional on the cov…

Gradient boosting adapted for vector inputs.

Unified probabilistic gradient boosting for entire conditional distribution modeling.

Quaternion self-attention reduces computational cost and improves performance.

Excellent ranking power along with well calibrated probability estimates are needed in many classification tasks. In this paper, we introduce a technique, Calibrated Boosting-Forest that captures both. This novel technique is an ensemble of gradient boosting machines that can support both continuous and binary labels. …

U-Clip clips gradients to reduce bias without discarding them.

The paper provides convergence guarantees for multicalibration gradient boosting.

The paper develops methods to accurately locate change points in high-dimensional mean shift models.

ABHT boosts regression by filtering regions with different smoothness.

GBMixed boosts mixed models for clustered data, estimating mean and variance flexibly.

StructureBoost improves gradient boosting for complex categorical variables efficiently.

Infinitesimal boosting converges to a deterministic process in large sample limit.

Additive models, such as produced by gradient boosting, and full interaction models, such as classification and regression trees (CART), are widely used algorithms that have been investigated largely in isolation. We show that these models exist along a spectrum, revealing never-before-known connections between these t…

Gradient Boosted Mixed Models estimate mean and variance components for clustered data.