arXiv research

Wide Boosting improves GB's performance on multivariate output tasks.

Boosted additive models reveal new insights and potential pathologies.

We introduce a novel boosting algorithm called `KTBoost' which combines kernel boosting and tree boosting. In each boosting iteration, the algorithm adds either a regression tree or reproducing kernel Hilbert space (RKHS) regression function to the ensemble of base learners. Intuitively, the idea is that discontinuous …

This paper introduces Stochastic Gradient Langevin Boosting (SGLB) - a powerful and efficient machine learning framework that may deal with a wide range of loss functions and has provable generalization guarantees. The method is based on a special form of the Langevin diffusion equation specifically designed for gradie…

Boosted decision trees enjoy popularity in a variety of applications; however, for large-scale datasets, the cost of training a decision tree in each round can be prohibitively expensive. Inspired by ideas from the multi-arm bandit literature, we develop a highly efficient algorithm for computing exact greedy-optimal d…

Fair MP-Boost improves fairness and interpretability in boosting methods.

Boosting improves ICA for better component recovery.

Additive models, such as produced by gradient boosting, and full interaction models, such as classification and regression trees (CART), are widely used algorithms that have been investigated largely in isolation. We show that these models exist along a spectrum, revealing never-before-known connections between these t…

Boosts weak online learners to strong ones with sublinear regret.

Boosting combines weak (biased) learners to obtain effective learning algorithms for classification and prediction. In this paper, we show a connection between boosting and kernel-based methods, highlighting both theoretical and practical applications. In the context of $\ell_2$ boosting, we start with a weak linear le…

Unified framework for statistical inference in gradient boosting regression.

Distributionally Robust Optimization (DRO) has been shown to provide a flexible framework for decision making under uncertainty and statistical estimation. For example, recent works in DRO have shown that popular statistical estimators can be interpreted as the solutions of suitable formulated data-driven DRO problems.…

This research tackles uncertainty in gradient boosting models using ensemble methods.

MIBoost boosts variable selection with multiple imputation.

Boosting CNNs with dynamic feature selection and boosting weights improves accuracy and efficiency.

Combines boosting and latent Gaussian models for better predictions.

SMAC method optimizes tree-boosting hyperparameters best.

sGBM speeds up gradient boosting by parallelizing and adapting base learners.

We propose a novel approach for using unsupervised boosting to create an ensemble of generative models, where models are trained in sequence to correct earlier mistakes. Our meta-algorithmic framework can leverage any existing base learner that permits likelihood evaluation, including recent deep expressive models. Fur…

Cyclic Boosting offers detailed prediction understanding for machine learning models.

GBHT uses gradient boosting for density estimation with theoretical guarantees.

Gradient Boosted Mixed Models estimate mean and variance components for clustered data.

Gradient boosting enhances existing Mendelian models for genetic disease risk prediction.

Privacy-preserving boosting algorithm for machine learning.

Early stopping of iterative algorithms is a widely-used form of regularization in statistics, commonly used in conjunction with boosting and related gradient-type algorithms. Although consistency results have been established in some settings, such estimators are less well-understood than their analogues based on penal…

A new gradient boosting method improves interpretability of probabilistic models.

The problem of adversarial robustness has been studied extensively for neural networks. However, for boosted decision trees and decision stumps there are almost no results, even though they are widely used in practice (e.g. XGBoost) due to their accuracy, interpretability, and efficiency. We show in this paper that for…

BayesBoost combines boosting and Bayesian methods for linear mixed models, improving uncertainty estimation and variable selection.

Stochastic Gradient Boosting (SGB) is a widely used approach to regularization of boosting models based on decision trees. It was shown that, in many cases, random sampling at each iteration can lead to better generalization performance of the model and can also decrease the learning time. Different sampling approaches…

BoostForest combines multiple BoostTree models for improved accuracy.

Machine learning tools are commonly used in modern high energy physics (HEP) experiments. Different models, such as boosted decision trees (BDT) and artificial neural networks (ANN), are widely used in analyses and even in the software triggers. In most cases, these are classification models used to select the "signal"…

Treeffuser predicts tabular data distributions using gradient-boosted trees.

SketchBoost accelerates GBDT for multioutput problems up to 40x.

This paper reconciles different views on AdaBoost to better understand its dynamics.

Federated machine learning systems have been widely used to facilitate the joint data analytics across the distributed datasets owned by the different parties that do not trust each others. In this paper, we proposed a novel Gradient Boosting Machines (GBM) framework SecureGBM built-up with a multi-party computation mo…

Proposes new attribution methods for trees with regularization.

Gradient Boosting Machine (GBM) is an extremely powerful supervised learning algorithm that is widely used in practice. GBM routinely features as a leading algorithm in machine learning competitions such as Kaggle and the KDDCup. In this work, we propose Accelerated Gradient Boosting Machine (AGBM) by incorporating Nes…

Accurately predicting customer churn using large scale time-series data is a common problem facing many business domains. The creation of model features across various time windows for training and testing can be particularly challenging due to temporal issues common to time-series data. In this paper, we will explore …

We analyze a new Markov chain model for better sampling and optimization.

In AI research and industry, machine learning is the most widely used tool. One of the most important machine learning algorithms is Gradient Boosting Decision Tree, i.e. GBDT whose training process needs considerable computational resources and time. To shorten GBDT training time, many works tried to apply GBDT on Par…

Boosted CVaR Classification improves tail performance in classification tasks.

Enhances GBDT robustness with one-hot encoding and regularization.

Cascade classifiers are widely used in real-time object detection. Different from conventional classifiers that are designed for a low overall classification error rate, a classifier in each node of the cascade is required to achieve an extremely high detection rate and moderate false positive rate. Although there are …

New test improves tree ensemble pruning for better model performance.

Technology and collaboration enable dramatic increases in the size of psychological and psychiatric data collections, but finding structure in these large data sets with many collected variables is challenging. Decision tree ensembles like random forests (Strobl, Malley, and Tutz, 2009) are a useful tool for finding st…

AF improves classification models by adaptively weighting trees.

Deep tabular models outperform GBDT in medical diagnosis tasks.

DReg boosts large-batch SGD's generalization and convergence.