Overview of SML techniques with banking applications.
problem Credit risk modeling in banking.
method Tree-based ensemble algorithms, Feedforward NNs, hyper-parameter optimization, machine learning interpretability.
result Comparison of ML algorithm features and their application in banking.
Paper enhances haptic signals distinguishability with boosted technique.
problem Lack of large datasets in haptics domain limits feature extraction.
method General framework for haptic signal analysis, using spectral features and boosted embedding.
result Framework needs less training data and outperforms state-of-the-art.
Boosting for label ranking outperforms existing methods.
problem Improving label ranking predictions using boosting techniques.
method Proposed a boosting algorithm tailored for label ranking tasks.
result Significantly outperforms existing label ranking algorithms.
XGBoost outperforms other boosting techniques in training speed and generalization performance.
problem Comparing XGBoost with other boosting techniques.
method Comprehensive comparison of XGBoost, random forests, and gradient boosting using tuned and default models.
result XGBoost is not always the best choice under all circumstances.
TF Boosted Trees (TFBT) is a new open-sourced frame-work for the distributed training of gradient boosted trees. It is based on TensorFlow, and its distinguishing features include a novel architecture, automatic loss differentiation, layer-by-layer boosting that results in smaller ensembles and faster prediction, princ…
Excellent ranking power along with well calibrated probability estimates are needed in many classification tasks. In this paper, we introduce a technique, Calibrated Boosting-Forest that captures both. This novel technique is an ensemble of gradient boosting machines that can support both continuous and binary labels. …
Boosting techniques improved uplift modeling performance.
problem Predicting causal effects of actions on individuals.
method Adapted boosting techniques to uplift modeling.
result Proposed algorithms improved performance of base models.
Boosting method adapted for non-square loss functions.
problem Applying gradient boosting to non-square loss functions.
method Introducing column measure ν^(L_2) to adapt L2-Boosting for different loss functions.
result L2-Boosting can be modified to select variables according to different loss functions.
Combines boosting and latent Gaussian models for better predictions.
problem Boosting's assumptions and latent Gaussian models' limitations.
method Integrates tree-boosting and latent Gaussian models.
result Increased prediction accuracy in simulations and real-world data.
A new gradient tree boosting framework reduces variance and accelerates performance.
problem High variance in stochastic gradient boosting.
method Combining gradient tree boosting with importance sampling and a regularizer.
result Achieves a linear convergence rate on logistic loss and 2.5x--18x acceleration on LogitBoost and LambdaMART.
Fair MP-Boost improves fairness and interpretability in boosting methods.
problem Improving fairness and interpretability in boosting methods.
method Fair MP-Boost uses adaptive sampling of minipatches to balance accuracy and fairness.
result Fair MP-Boost enhances fairness and accuracy while providing interpretable feature importance.
New sampling technique improves boosting model accuracy.
problem Improving generalization performance and learning time in stochastic gradient boosting.
method Formulated optimization problem to maximize estimation accuracy, leading to Minimal Variance Sampling (MVS).
result MVS significantly increases model quality and reduces the number of examples needed.
VeriGB verifies robustness of gradient boosted models.
problem Proving robustness of gradient boosted models.
method VeriGB encodes model and robustness as SMT formula for verification.
result VeriGB proves robustness of large gradient boosted models.
Additive models, such as produced by gradient boosting, and full interaction models, such as classification and regression trees (CART), are widely used algorithms that have been investigated largely in isolation. We show that these models exist along a spectrum, revealing never-before-known connections between these t…
Boosted conformal procedure improves prediction intervals.
problem Enhancing prediction interval properties like coverage and length.
method Gradient boosting to optimize conformity score function.
result Significant improvements in interval length and coverage.
Boosting improves ICA for better component recovery.
problem Improving ICA's reliance on prior knowledge of sources.
method Maximizing likelihood via boosting and fixed-point unmixing.
result Boosting-based ICA outperforms existing methods.
SHAP explains boosted trees with additively modeled features.
problem Explaining predictions of boosted trees models with additively modeled features.
method SHAP values for additively modeled features in boosted trees models.
result SHAP dependence plot matches partial dependence plot for additively modeled features.
New algorithms improve boosting by optimizing chance-corrected measures.
problem Improving boosting algorithms to use chance-corrected measures effectively.
method Developed new algorithms (AdaBook and Multibook) that optimize chance-corrected measures.
result AdaBook and Multibook outperform standard Multiboost or AdaBoost in multiclass situations.
Boosting combines weak (biased) learners to obtain effective learning algorithms for classification and prediction. In this paper, we show a connection between boosting and kernel-based methods, highlighting both theoretical and practical applications. In the context of ℓ2 boosting, we start with a weak linear le…
WOTBoost improves minority class accuracy in imbalanced datasets.
problem Imbalanced datasets lead to low accuracy in minority class classification.
method Combines weighted oversampling and boosting techniques.
result WOTBoost achieves best G mean and highest AUC score on multiple datasets.
This work improves adversarial robustness by boosting model ensembles with margin maximization.
problem Single models are insufficient for defending against adversarial attacks.
method Margin-boosting approach to learn ensembles with maximum margin.
result Our algorithm outperforms existing ensembling techniques and large models trained end-to-end.
We present SEBOOST, a technique for boosting the performance of existing stochastic optimization methods. SEBOOST applies a secondary optimization process in the subspace spanned by the last steps and descent directions. The method was inspired by the SESOP optimization method for large-scale problems, and has been ada…
Paper speeds up boosting algorithms with less memory.
problem Boosting algorithms require large memory for training, limiting scalability.
method Combines early stopping, effective sample size, and stratified sampling.
result Achieves 10-100 speedup over XGBoost with limited memory.
This paper introduces Deep Incremental Boosting, a new technique derived from AdaBoost, specifically adapted to work with Deep Learning methods, that reduces the required training time and improves generalisation. We draw inspiration from Transfer of Learning approaches to reduce the start-up time to training each incr…
It is known that Boosting can be interpreted as a gradient descent technique to minimize an underlying loss function. Specifically, the underlying loss being minimized by the traditional AdaBoost is the exponential loss, which is proved to be very sensitive to random noise/outliers. Therefore, several Boosting algorith…
Gradient boosted decision trees are a popular machine learning technique, in part because of their ability to give good accuracy with small models. We describe two extensions to the standard tree boosting algorithm designed to increase this advantage. The first improvement extends the boosting formalism from scalar-val…
Classifier evasion consists in finding for a given instance x the nearest instance x′ such that the classifier predictions of x and x′ are different. We present two novel algorithms for systematically computing evasions for tree ensembles such as boosted trees and random forests. Our first algorithm uses a Mixe…
sgboost reduces variable selection bias in boosting with balanced group selection.
problem Reduces variable selection bias in boosting algorithms.
method Simulation-based approach to balance selection frequencies of base-learners.
result Demonstrates efficacy through simulations and flexible group variable selection.
In this paper, we introduce and evaluate a data-driven staged mixture modeling technique for building density, regression, and classification models. Our basic approach is to sequentially add components to a finite mixture model using the structural expectation maximization (SEM) algorithm. We show that our technique i…
SMAC method optimizes tree-boosting hyperparameters best.
problem Optimizing hyperparameters for tree-boosting to improve model accuracy.
method Compared and evaluated various hyperparameter optimization methods.
result SMAC method outperforms other methods for hyperparameter tuning.
In this paper we analyze boosting algorithms in linear regression from a new perspective: that of modern first-order methods in convex optimization. We show that classic boosting algorithms in linear regression, namely the incremental forward stagewise algorithm (FSε) and least squares boosting (LS-Boost($…
Gradient Boosted Normalizing Flows improve flexibility of NFs without increasing complexity.
problem Improving flexibility of normalizing flows without increasing complexity.
method Gradient Boosting applied to normalizing flows to create a mixture model structure.
result GBNFs outperform non-boosted NFs and produce better results with simpler components.
Many businesses are using recommender systems for marketing outreach. Recommendation algorithms can be either based on content or driven by collaborative filtering. We study different ways to incorporate content information directly into the matrix factorization approach of collaborative filtering. These content-booste…
Boosting method for causal SEMs from observational data.
problem Learning causal order among variables from observational data.
method Boosting-based approach with early stopping and component-wise gradient descent.
result Boosting with early stopping consistently favors the true causal ordering.
Study uses ML to analyze how interest rates affect fund returns, finding gradient boosting is effective.
problem Understanding how interest rate changes impact fund returns.
method Combines Machine Learning and causal inference, using Double Machine Learning framework.
result Gradient boosting is useful for predicting fund returns, showing a significant negative effect of interest rate increases.
BAE uses boosting to improve autoencoder ensembles for robust outlier detection.
problem Overfitting in autoencoders limits their effectiveness in unsupervised outlier detection.
method Boosting-based Autoencoder Ensemble (BAE) trains autoencoders sequentially with weighted sampling to reduce outliers and inject diversity.
result BAE outperforms state-of-the-art approaches in various outlier detection conditions.
Hyperboost uses gradient boosting for hyperparameter optimization, outperforming state-of-the-art methods.
problem Hyperparameter tuning for machine learning algorithms
method Gradient boosting surrogate model with quantile regression and distance metric
result Hyperboost outperforms state-of-the-art techniques in empirical tests
Boosting methods for interval-censored data improve predictive accuracy in survival analysis.
problem Handling interval-censored data in survival analysis and time-to-event studies.
method Nonparametric boosting methods using censoring unbiased transformations and functional gradient descent.
result Effective boosting methods for regression and classification with interval-censored data, offering robust performance.
Boosted decision trees improved for particle identification in high-energy physics.
problem Overfitting in boosted decision trees hampers their performance in particle identification.
method Meta-learning techniques of boosting and bagging to mitigate overfitting.
result The proposed algorithm achieves performance close to that of deep neural networks on a benchmark data set.
Proposes a Gradient Boosting method for learning adaptive kernel functions.
problem Learning a versatile ensemble of kernel functions for better performance.
method Approximates kernel functions as a weighted sum of Random Fourier Features and optimizes their barycenter at each iteration.
result Shows improved performance compared to Boosting-based and kernel-learning methods.
Online boosting for multilabel ranking with limited feedback.
problem Multilabel ranking with top-k feedback.
method Surrogate loss function and unbiased estimator for weak learners.
result Adapted full information multilabel ranking algorithms to top-k feedback setting with theoretical and experimental support.
Boosting improves accuracy with fewer calls to weak learners for certain concept classes.
problem Improving accuracy of learning algorithms with limited weak learner calls.
method Combines boosting and list-decodable codes to achieve better performance for specific concept classes.
result A new boosting algorithm that achieves strong learning with fewer calls to weak learners and additional samples.
Boosting algorithms improve estimation of personalized treatment rules.
problem Estimating optimal personalized treatment rules in clinical settings.
method Additive regression trees with boosting technique based on XGBoost.
result Efficient and accurate estimation of complex treatment rules.
BayesBoost combines boosting and Bayesian methods for linear mixed models, improving uncertainty estimation and variable selection.
problem Lack of straightforward uncertainty estimation for parameters in high-dimensional linear mixed models.
method BayesBoost: Combines boosting and Bayesian inference for linear mixed models.
result Improves uncertainty estimation and variable selection in linear mixed models.
Dynamic classifier chains with XGBoost reduces multi-label classification costs and improves label dependency handling.
problem Static label ordering in multi-label classification limits model performance.
method Combining dynamic classifier chains with XGBoost for efficient multi-label prediction.
result Dynamic label ordering improves model performance and reduces training costs.
The paper uses ML to predict oil rate post-HF, comparing it to engineers' predictions.
problem Predicting oil rate post-hydraulic fracturing.
method Data-driven model using ML techniques on fracturing job data.
result ML predictions outperform engineers' predictions.
The paper analyzes boosting and minimum-ℓ1-norm classifiers in high dimensions.
problem Understanding the generalization error and optimal Bayes error in boosting.
method High-dimensional asymptotic theory, Gaussian comparison techniques, uniform deviation argument.
result Precise characterizations of boosting test error and optimal Bayes error.
Parallel gradient boosting speeds up multi-output regression.
problem Efficiently predicting entire conditional distributions.
method Modification of gradient boosting to use a common descent direction.
result Significantly faster and comparable performance to state-of-the-art boosting libraries.