Paper explores how feature interactions improve XGBoost models.
problem Improving XGBoost model performance and interpretability.
method Identifies feature interactions as constraints in XGBoost models.
result Accurate identification of feature interactions leads to significant model performance improvements.
Proposes a generalized XGBoost method for nonconvex loss functions.
problem Limited to convex loss functions in XGBoost.
method Extends XGBoost to use nonconvex loss functions and multivariate loss functions.
result Generalized XGBoost method can model multiple parameters in various distributions.
XGBoost implements AFT models for survival regression.
problem Survival regression for time-to-event data.
method Loss functions for AFT models in XGBoost.
result XGBoost's AFT model improves generalization and training speed.
XGBoost is a scalable ensemble technique based on gradient boosting that has demonstrated to be a reliable and efficient machine learning challenge solver. This work proposes a practical analysis of how this novel technique works in terms of training speed, generalization performance and parameter setup. In addition, a…
XGBoost learns a broad class of functions bridging empirical success and theory.
problem Understanding the function class implicitly learned by XGBoost.
method Introduced an infinite-dimensional function class and a complexity measure, showing optimizers of XGBoost are also optimizers of a penalized regression problem.
result Proved the least squares estimator achieves nearly minimax-optimal convergence rate over the function class.
The paper presents Imbalance-XGBoost, a Python package that combines the powerful XGBoost software with weighted and focal losses to tackle binary label-imbalanced classification tasks. Though a small-scale program in terms of size, the package is, to the best of the authors' knowledge, the first of its kind which prov…
A method uses ITD and XGBoost for precise power transformer fault diagnosis.
problem Fault diagnosis of power transformers using DGA data.
method Ranking DGA parameters by skewness, extracting ITD features, and using an XGBoost classifier.
result The method achieves over 95% accuracy in classification.
This paper aims to explore models based on the extreme gradient boosting (XGBoost) approach for business risk classification. Feature selection (FS) algorithms and hyper-parameter optimizations are simultaneously considered during model training. The five most commonly used FS methods including weight by Gini, weight b…
XGBoost detects unlawful insider trading with high accuracy.
problem Detecting unlawful insider trading from large volumes of transactions.
method Applying eXtreme Gradient Boosting (XGBoost) for identifying and ranking key features.
result XGBoost achieves 97% accuracy in detecting unlawful transactions.
Paper introduces arctan pinball loss for XGBoost quantile regression.
problem Efficiently predicting multiple quantiles with XGBoost.
method Smooth approximation of pinball loss for XGBoost, using arctan pinball loss.
result Arctan pinball loss reduces quantile crossings and improves efficiency.
We describe the multi-GPU gradient boosting algorithm implemented in the XGBoost library (https://github.com/dmlc/xgboost). Our algorithm allows fast, scalable training on multi-GPU systems with all of the features of the XGBoost library. We employ data compression techniques to minimise the usage of scarce GPU memory …
XGBoost is often presented as the algorithm that wins every ML competition. Surprisingly, this is true even though predictions are piecewise constant. This might be justified in high dimensional input spaces, but when the number of features is low, a piecewise linear model is likely to perform better. XGBoost was exten…
This study compares logistic regression and XGBoost for predicting credit risk.
problem Predicting credit risk in financial services.
method Advanced machine learning techniques (logistic regression and XGBoost) with data preprocessing.
result XGBoost outperforms logistic regression in predicting credit risk.
Develops monotone tree-based GAMI models using XGBoost.
problem Incorporating monotonicity into GAMI models based on boosted trees.
method Adapting XGBoost algorithm to fit monotone GAMI-Tree models, filtering interactions, and parsing results.
result Fits monotone GAMI-Tree models that are directly interpretable and visualizable.
XGBoost models estimate oil recovery factors with moderate accuracy.
problem Estimating oil recovery factors before exploitation and exploration.
method Applied XGBoost classification algorithm to machine learning models.
result XGBoost models achieved accuracies of up to 0.49 in training datasets.
Enhances XGBoost for better uncertainty quantification in ML predictions.
problem Uncertainty in ML predictions, especially for XGBoost.
method Quantile Extreme Gradient Boosting (QXGBoost) using Huber norm in quantile regression.
result QXGBoost produces more accurate 90% prediction intervals.
State-of-the-art implementations of boosting, such as XGBoost and LightGBM, can process large training sets extremely fast. However, this performance requires that the memory size is sufficient to hold a 2-3 multiple of the training set size. This paper presents an alternative approach to implementing the boosted trees…
C-XGBoost estimates causal effects from observational data.
problem Estimating causal effects from observational data.
method Proposes C-XGBoost, a tree boosting model for causal effect estimation.
result Demonstrates effectiveness through performance profiles and statistical tests.
A novel CAB-XDE framework predicts speculative stock prices with high accuracy.
problem Forecasting speculative stock prices in volatile markets.
method Customized attention BiLSTM with XGBoost, integrating attention mechanism and weight determination theory-error reciprocal method.
result Empirically validated with MAPE of 0.0037, MAE of 84.40, and RMSE of 106.14.
We propose a new framework of XGBoost that predicts the entire conditional distribution of a univariate response variable. In particular, XGBoostLSS models all moments of a parametric distribution (i.e., mean, location, scale and shape [LSS]) instead of the conditional mean only. Choosing from a wide range of continuou…
A new Randomized-Hyperopt method improves XGBoost hyperparameter tuning.
problem Improving the performance of XGBoost through hyperparameter optimization.
method Proposes Randomized-Hyperopt for XGBoost hyperparameter tuning.
result Randomized-Hyperopt outperforms other methods in terms of accuracy and execution time.
XGBoost fails to accurately identify relevant features, while interpretable methods do.
problem Accurately identifying relevant features in black-box models like XGBoost.
method Comparison of variable importance methods (CART, Optimal Trees, XGBoost, SHAP) across various experiments.
result Interpretable methods outperform black-box models in feature selection accuracy.
Dynamic classifier chains with XGBoost reduces multi-label classification costs and improves label dependency handling.
problem Static label ordering in multi-label classification limits model performance.
method Combining dynamic classifier chains with XGBoost for efficient multi-label prediction.
result Dynamic label ordering improves model performance and reduces training costs.
A new federated learning method protects privacy in mobile crowdsensing.
problem Data and model privacy protection in federated extreme gradient boosting for mobile crowdsensing.
method Secret sharing based federated learning architecture FedXGB.
result FedXGB achieves less than 1% accuracy loss while preserving model privacy.
This paper compares preprocessing techniques for XGBoost models on various data sets.
problem Improving predictive performance of XGBoost models through optimal data preprocessing.
method Comparison of feature selection, categorical handling, and null imputation methods.
result XGBoost importance by gain is the most consistent and highest-performing method for feature selection.
Combines Xgboost and transductive SVM for semi-supervised learning.
problem Improving semi-supervised learning performance with heterogeneous tabular data.
method Proposes an optimization-based ensemble method to adaptively combine Xgboost and transductive SVM.
result Significantly improves classification accuracy over state-of-the-art methods.
Study compares machine learning algorithms for predicting SST in the Great Barrier Reef.
problem Predicting sea surface temperature in the Great Barrier Reef region.
method Ridge regression, LASSO, Random Forest, and Extreme Gradient Boosting (XGBoost) algorithms were evaluated.
result XGBoost significantly outperforms other algorithms in terms of predictive accuracy and Kullback-Leibler Divergence.
AF improves classification models by adaptively weighting trees.
problem Improving classification model performance.
method AF combines OP2T for input-dependent weights and MIO for dynamic refinement.
result AF consistently outperforms RF, XGBoost, and other weighted RF.
GPU-accelerated BART speeds up Bayesian regression.
problem Long running time of BART makes it impractical for large datasets.
method GPU-enabled implementation of BART.
result BART is now 200x faster on GPUs.
Have you ever looked at a machine learning classification model and thought, I could have made that? Well, that is what we test in this project, comparing XGBoost trained on human engineered features to training directly on data. The human engineered features do not outperform XGBoost trained di- rectly on the data, bu…
This paper compares machine learning models for pricing European options.
problem Pricing European options using traditional methods like Black Scholes Model.
method Google AutoML Regressor, TensorFlow Neural Networks, and XGBoost Gradient Boosting Decision Trees.
result All models outperformed the Black Scholes Model in terms of mean absolute error.
Hybrid model predicts stock prices with high accuracy.
problem Complex volatility of stock market makes traditional models unsatisfactory.
method Attention-based CNN-LSTM and XGBoost integrated model.
result Hybrid model improves prediction accuracy.
Machine learning predicts Bitcoin returns but trading performance drops with costs.
problem Trading Bitcoin predictions with transaction costs.
method XGBoost, LSTM, iTransformer models evaluated in walk-forward protocol; cost-aware execution filter implemented.
result Cost-aware execution filter restores profitability; XGBoost strategy outperforms.
This paper compares ML algorithms for PD prediction, finding XGBoost to be the most effective.
problem Predicting the probability of default in loan portfolios.
method Comparison of five ML algorithms (Random Forests, Decision Trees, XGBoost, Gradient Boosting, AdaBoost) with logistic regression.
result XGBoost outperforms other ML algorithms for PD prediction.
Framework improves financial predictions with deep learning models.
problem Adverse financial conditions like regime changes and low signal-to-noise ratios.
method Incremental use of decision trees and XGBoost models for robust performance.
result Two-layer deep ensemble of XGBoost models outperforms single models under different market regimes.
XGBoost outperforms other models in predicting housing prices.
problem Accurate housing price prediction for socio-economic development.
method Employed XGBoost and other machine learning algorithms on housing price datasets.
result XGBoost outperformed other models in predicting housing prices.
Predicts S&P 500 trends using machine learning models.
problem Market trend prediction for S&P 500 index.
method Feature engineering, machine learning models (Logistic Regression, Decision Trees, Random Forests, Neural Networks, KNN, XGBoost), data preprocessing, hyperparameter tuning, SMOTE.
result KNN for short-term predictions, XGBoost for long-term forecasts.
This study compares machine learning models for short-term stock price forecasting.
problem Accurate short-term stock price prediction in the NYSE.
method Compared four machine learning models (XGBoost, Random Forest, Multi-layer Perceptron, Support Vector Regression) on NYSE stocks.
result XGBoost model outperformed others with highest accuracy.
TabPFN doesn't outperform GLM and XGBoost for motor insurance pricing.
problem Improving insurance pricing models using Tabular Foundation Models (TFMs).
method Pre-training on synthetic datasets and in-context learning for inference.
result TabPFN does not consistently outperform established baselines, has longer inference times, and is sensitive to training set size.
Research improves LGD approximation using XGBoost for cash-flow-limited data.
problem Accurate LGD calculation with limited cash-flow data.
method Enhanced delta outstanding approach with XGBoost machine learning.
result XGBoost model enhances realized LGD estimation accuracy.
A minimalist approach generates synthetic tabular data with sparse PCA and XGBoost.
problem Generating robust synthetic tabular data for model testing.
method Minimalistic unsupervised SparsePCA encoder with XGBoost decoder.
result The method provides an alternative to raw and quantile perturbation for model robustness testing.
In this short paper we investigate whether meta-learning techniques can be used to more effectively tune the hyperparameters of machine learning models using successive halving (SH). We propose a novel variant of the SH algorithm (MeSH), that uses meta-regressors to determine which candidate configurations should be el…
A novel multi-objective optimization framework improves insurance pricing fairness.
problem Exacerbated trade-offs between competing fairness criteria in insurance pricing using machine learning.
method Proposes a novel multi-objective optimization framework using NSGA-II to jointly optimize accuracy and fairness criteria.
result Consistently achieves a balanced compromise between accuracy and fairness, outperforming single-model approaches.
Adaptive XGBoost improves accuracy on evolving data streams by updating the ensemble dynamically.
problem Concept drift in evolving data streams.
method Adapts XGB to update the ensemble with new data, maintaining consistency with current concept.
result Improves classification accuracy on evolving data streams compared to other methods.
XGBoost predicts NEPSE Index log returns with low error and high directional accuracy.
problem Forecasting daily log-returns in the NEPSE Index with high accuracy.
method XGBoost machine learning, feature engineering, hyperparameter optimization, walk-forward validation.
result Optimal XGBoost configuration achieves lowest log-return RMSE and MAE.
New GPU algorithm boosts machine learning with larger datasets.
problem Limited GPU memory restricts training data size.
method Out-of-core GPU gradient boosting algorithm.
result Training larger datasets on GPUs without accuracy loss.
LDA-XGB1 balances fairness and accuracy in lending models.
problem Fair lending practices and model interpretability in binary classification.
method Biobjective optimization using binning and information value, leveraging XGBoost.
result Achieves effective balance between accuracy, fairness, and interpretability.
Paper predicts future sales using machine learning techniques.
problem Predict future sales of retail products using past data.
method Used eXtreme Gradient Boosting (XGBoost) and Long Short Term Memory (LSTM) for forecasting.
result XGBoost performed better than LSTM in predicting sales.