Boosted decision trees typically yield good accuracy, precision, and ROC area. However, because the outputs from boosting are not well calibrated posterior probabilities, boosting yields poor squared error and cross-entropy. We empirically demonstrate why AdaBoost predicts distorted probabilities and examine three cali…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “precision boost”
A Precise High-Dimensional Asymptotic Theory for Boosting and Minimum--Norm Interpolated Classifiersmath.ST
This paper establishes a precise high-dimensional asymptotic theory for boosting on separable data, taking statistical and computational perspectives. We consider a high-dimensional setting where the number of features (weak learners) scales with the sample size , in an overparametrized regime. Under a class of …
Boosting combines weak classifiers to form highly accurate predictors. Although the case of binary classification is well understood, in the multiclass setting, the "correct" requirements on the weak classifier, or the notion of the most efficient boosting algorithms are missing. In this paper, we create a broad and ge…
Boosting methods are highly popular and effective supervised learning methods which combine weak learners into a single accurate model with good statistical performance. In this paper, we analyze two well-known boosting methods, AdaBoost and Incremental Forward Stagewise Regression (FS), by establishing t…
BayesBoost combines boosting and Bayesian methods for linear mixed models, improving uncertainty estimation and variable selection.
problem Lack of straightforward uncertainty estimation for parameters in high-dimensional linear mixed models.
method BayesBoost: Combines boosting and Bayesian inference for linear mixed models.
result Improves uncertainty estimation and variable selection in linear mixed models.
We introduce a useful tool for analyzing boosting algorithms called the ``smooth margin function,'' a differentiable approximation of the usual margin for boosting algorithms. We present two boosting algorithms based on this smooth margin, ``coordinate ascent boosting'' and ``approximate coordinate ascent boosting,'' w…
Boosts A/B test precision using auxiliary data from historical users.
problem Small sample sizes and imprecise estimates in A/B tests.
method Coupling design-based causal estimation with machine-learning models of historical user data.
result Effect estimates using auxiliary data are roughly equivalent to increasing sample size by 20%, or up to 50-80% in some cases.
A New Perspective on Boosting in Linear Regression via Subgradient Optimization and Relativesmath.ST
In this paper we analyze boosting algorithms in linear regression from a new perspective: that of modern first-order methods in convex optimization. We show that classic boosting algorithms in linear regression, namely the incremental forward stagewise algorithm (FS) and least squares boosting (LS-Boost($…
Proposes a two-stage method for estimating heterogeneous treatment effects using gradient boosting trees.
problem Estimating heterogeneous treatment effects in randomized clinical trials with high-dimensional predictive markers.
method Two-stage statistical learning procedure using gradient boosting trees (XGBoost) to estimate main effects and HTE.
result Improves efficiency in estimating heterogeneous treatment effects through nonparametric function estimation.
We consider the class of locally boost isotropic spacetimes in arbitrary dimension. For any spacetime with boost isotropy, the corresponding curvature tensor and all of its covariant derivatives must be simultaneously of alignment type relative to some common null frame. Such spacetimes are known as type ${\b…
Dynamic CBDT improves treatment effect estimation in clinical data.
problem Estimating heterogeneous treatment effects in observational data with high accuracy and interpretability.
method Dynamic Regularized Causal Boosted Decision Trees (CBDT) integrating variance regularization and calibration.
result Significantly improved estimation accuracy and reliable coverage of true treatment effects.
Insider Purchase Signals in Microcap Equities: Gradient Boosting Detection of Abnormal Returnsq-fin.ST
Gradient boosting detects insider purchases predicting abnormal returns in microcap stocks.
problem Predicting abnormal returns in U.S. microcap stocks using insider purchase signals.
method Gradient boosting classifier trained on insider identity, transaction history, and market conditions.
result Gradient boosting classifier achieves AUC of 0.70 on out-of-sample data, with precision and recall of 0.38 and 0.69 respectively.
Enhanced Tweedie model for insurance claims using CatBoost.
problem Accurately modeling aggregate claims with zero-inflated data.
method Refined Tweedie model with boosting methods in CatBoost.
result Marked improvement in model performance for insurance analytics.
EMPIR combines low and full precision DNNs to enhance robustness against adversarial attacks.
problem Vulnerability of DNNs to adversarial attacks that misclassify inputs with small perturbations.
method Ensemble of quantized DNN models with different numerical precisions.
result EMPIR ensembles increase adversarial robustness by 42.6% on average across different tasks.
Loan default prediction is one of the most important and critical problems faced by banks and other financial institutions as it has a huge effect on profit. Although many traditional methods exist for mining information about a loan application, most of these methods seem to be under-performing as there have been repo…
Combines coarse learners for nonparametric probabilistic regression.
problem Predicting full probability density functions without strong assumptions.
method Combining gradient boosted forests trained on coarsened target values.
result Prediction intervals have high fidelity and provide valuable insights.
Study improves early warning models for currency and stock market crises.
problem Predicting currency and stock market crises.
method Synthetic review and comparison of early warning models, focusing on crisis identifications and predictive models.
result SWARCH model with elastic thresholding methodology most accurately classifies crisis observations.
In recent years, dynamically growing data and incrementally growing number of classes pose new challenges to large-scale data classification research. Most traditional methods struggle to balance the precision and computational burden when data and its number of classes increased. However, some methods are with weak pr…
End-to-end framework learns precise disparity for activity recognition.
problem Precise portrayal of intraclass disparity in activity recognition.
method Knowledge-directed adversarial learning framework with two competitive encoding distributions.
result Robust and generalizable performance on HAR benchmark datasets.
Develops RKHS framework for analyzing tree ensembles.
problem Analyzing the theoretical properties of tree ensembles.
method Reproducing Kernel Hilbert Spaces (RKHS) for tree ensembles.
result Characterizes Random Forest predictor as minimizer of a penalized empirical risk functional in RKHS.
Fair MP-Boost improves fairness and interpretability in boosting methods.
problem Improving fairness and interpretability in boosting methods.
method Fair MP-Boost uses adaptive sampling of minipatches to balance accuracy and fairness.
result Fair MP-Boost enhances fairness and accuracy while providing interpretable feature importance.
MP-Boost boosts accuracy faster and more interpretable than AdaBoost.
problem Developing a faster, more interpretable boosting method.
method Adaptive selection of minipatches (small subsets of instances and features) at each iteration.
result Achieves comparable accuracy to AdaBoost and gradient boosting but faster and more interpretable.
Boost-R uses gradient boosted trees for analyzing recurrence data.
problem Analyzing recurrence data with static and dynamic features.
method Gradient boosted additive trees with time-dependent functions.
result Estimates the cumulative intensity function of recurrent event processes.
Extends boosting to multiclass online agnostic classification.
problem Online multiclass classification with weak learners.
method Reduces multiclass online agnostic boosting to online convex optimization.
result First boosting algorithm for online agnostic multiclass classification.
Boosting is one of the most significant developments in machine learning. This paper studies the rate of convergence of Boosting, which is tailored for regression, in a high-dimensional setting. Moreover, we introduce so-called \textquotedblleft post-Boosting\textquotedblright. This is a post-selection estimator w…
We present a new boosting algorithm, motivated by the large margins theory for boosting. We give experimental evidence that the new algorithm is significantly more robust against label noise than existing boosting algorithm.
Improved agnostic boosting with better sample efficiency.
problem Agnostic boosting's poor sample efficiency compared to Empirical Risk Minimization.
method Leverages sample reuse across rounds, guarantees better generalization.
result Substantially more sample-efficient agnostic boosting algorithm.
Calibrated Boosting-Foreststat.ML
Excellent ranking power along with well calibrated probability estimates are needed in many classification tasks. In this paper, we introduce a technique, Calibrated Boosting-Forest that captures both. This novel technique is an ensemble of gradient boosting machines that can support both continuous and binary labels. …
Boosting algorithms are frequently used in applied data science and in research. To date, the distinction between boosting with either gradient descent or second-order Newton updates is often not made in both applied and methodological research, and it is thus implicitly assumed that the difference is irrelevant. The g…
The paper classifies coverings and non-Hausdorff extensions of Misner spacetime.
problem Classifying coverings and non-Hausdorff extensions of Misner spacetime.
method Covering constructions of the punctured Minkowski plane, quotient spacetimes, and explicit embeddings.
result A natural family of spacetimes including the Hawking--Ellis extension and its universal-cover analogue.
The study provides statistical guarantees for Bayesian variational boosting.
problem Statistical and convergence issues in variational boosting.
method Proposed a novel variational family and a functional Frank-Wolfe optimization algorithm.
result Demonstrated stochastic boundedness and provided convergence rate for boosting iterates.
High-precision regressors for particle physicsphysics.comp-ph
High-precision machine learning reduces particle physics simulations by orders of magnitude.
problem Reducing computational burden in particle physics simulations.
method Developed optimal training strategies and tuned machine learning regressors, including Deep Neural Networks with skip connections and boosted decision trees.
result Significantly reduced computational time by factors of 10^3 to 10^6 over first-principles simulations.
In this paper we propose using the principle of boosting to reduce the bias of a random forest prediction in the regression setting. From the original random forest fit we extract the residuals and then fit another random forest to these residuals. We call the sum of these two random forests a \textit{one-step boosted …
TF Boosted Trees (TFBT) is a new open-sourced frame-work for the distributed training of gradient boosted trees. It is based on TensorFlow, and its distinguishing features include a novel architecture, automatic loss differentiation, layer-by-layer boosting that results in smaller ensembles and faster prediction, princ…
Gradient boosting algorithms construct a regression predictor using a linear combination of ``base learners''. Boosting also offers an approach to obtaining robust non-parametric regression estimators that are scalable to applications with many explanatory variables. The robust boosting algorithm is based on a two-stag…
As an adaptive, interpretable, robust, and accurate meta-algorithm for arbitrary differentiable loss functions, gradient tree boosting is one of the most popular machine learning techniques, though the computational expensiveness severely limits its usage. Stochastic gradient boosting could be adopted to accelerates gr…
Boosting combines weak (biased) learners to obtain effective learning algorithms for classification and prediction. In this paper, we show a connection between boosting and kernel-based methods, highlighting both theoretical and practical applications. In the context of boosting, we start with a weak linear le…
In this survey, we discuss several different types of gradient boosting algorithms and illustrate their mathematical frameworks in detail: 1. introduction of gradient boosting leads to 2. objective function optimization, 3. loss function estimations, and 4. model constructions. 5. application of boosting in ranking.
Gradient boosting is a prediction method that iteratively combines weak learners to produce a complex and accurate model. From an optimization point of view, the learning procedure of gradient boosting mimics a gradient descent on a functional variable. This paper proposes to build upon the proximal point algorithm, wh…
New algorithm improves convergence of gradient boosting trees.
problem Global convergence of Newton boosting in tabular machine learning.
method Introduces Gradient Regularized Newton Descent for GBDTs, proving linear convergence for smooth, strongly convex losses and rate for general convex losses.
result Achieves globally convergent second-order GBDT algorithm with rate matching first-order boosting.
We show that Residual Networks (ResNet) is equivalent to boosting feature representation, without any modification to the underlying ResNet training algorithm. A regret bound based on Online Gradient Boosting theory is proved and suggests that ResNet could achieve Online Gradient Boosting regret bounds through neural n…
In this paper, we introduce a new machine learning (ML) model for nonlinear regression called the Boosted Smooth Transition Regression Trees (BooST), which is a combination of boosting algorithms with smooth transition regression trees. The main advantage of the BooST model is the estimation of the derivatives (partial…
Proposes a boosting framework for sparsity in grouped covariates.
problem Sparsity and selection bias in grouped covariates.
method Component-wise and group-wise gradient boosting with adjusted degrees of freedom.
result Reduces bias and improves predictability in variable selection.
Fréchet Geodesic Boostingstat.ML
FGBoost boosts gradient boosting for complex data.
problem Gradient boosting struggles with non-Euclidean data.
method Introduces FGBoost for geodesic metric spaces.
result FGBoost performs well on complex data.
Boosting improves trend detection in financial data.
problem Discovering trends in financial data during crises and recoveries.
method Extends boosting to higher order integrated processes and series with roots near unity.
result Boosting captures downturns and recoveries more accurately.
Paper compares AI models for credit scoring and explains them.
problem Lack of interpretability in advanced AI models hinders credit risk management.
method Comparison of logistic regression, AI algorithms, and techniques to interpret AI models.
result Advanced tree-based models provide the best prediction of client default.
This research tackles uncertainty in gradient boosting models using ensemble methods.
problem Quantifying uncertainty in gradient boosting models for high-risk applications.
method Probabilistic ensemble-based framework for gradient boosting classification and regression models.
result Ensembles of gradient boosting models detect anomalous inputs but have limited ability to improve total uncertainty.
We study the task of online boosting--combining online weak learners into an online strong learner. While batch boosting has a sound theoretical foundation, online boosting deserves more study from the theoretical perspective. In this paper, we carefully compare the differences between online and batch boosting, and pr…