Adaptive XGBoost improves accuracy on evolving data streams by updating the ensemble dynamically.
problem Concept drift in evolving data streams.
method Adapts XGB to update the ensemble with new data, maintaining consistency with current concept.
result Improves classification accuracy on evolving data streams compared to other methods.
Paper introduces XBART for nonlinear regression, outperforming XGBoost.
problem Nonlinear regression problems, especially in speed and accuracy.
method Combines Bayesian modeling and recursive partitioning for efficient, accurate predictions.
result XBART provides faster and more accurate predictions than XGBoost.
Ensemble GP improves genetic programming by achieving better results with smaller models.
problem Improving genetic programming for binary classification problems.
method Ensemble GP uses an evolved population structure, fitness evaluation, and genetic operators inspired by ensemble learning methods.
result Ensemble GP outperformed standard GP on eight binary classification problems, achieving better results with smaller models.
New framework uses tree ensembles for contextual bandits.
problem Optimizing decisions in dynamic environments with contextual information.
method Adapts Upper Confidence Bound and Thompson Sampling to tree ensemble methods.
result Tree ensemble methods outperform traditional methods in regret minimization and runtime.
XGBoost is a scalable ensemble technique based on gradient boosting that has demonstrated to be a reliable and efficient machine learning challenge solver. This work proposes a practical analysis of how this novel technique works in terms of training speed, generalization performance and parameter setup. In addition, a…
A novel CAB-XDE framework predicts speculative stock prices with high accuracy.
problem Forecasting speculative stock prices in volatile markets.
method Customized attention BiLSTM with XGBoost, integrating attention mechanism and weight determination theory-error reciprocal method.
result Empirically validated with MAPE of 0.0037, MAE of 84.40, and RMSE of 106.14.
Combines Xgboost and transductive SVM for semi-supervised learning.
problem Improving semi-supervised learning performance with heterogeneous tabular data.
method Proposes an optimization-based ensemble method to adaptively combine Xgboost and transductive SVM.
result Significantly improves classification accuracy over state-of-the-art methods.
D2KLab's approach predicts tweet engagement using two stages.
problem Predicting user engagement with tweets.
method Two-stage approach: feature learning and ensemble XGBoost.
result Ranked 22 in the 2020 RecSys Challenge leaderboard.
XGBoost learns a broad class of functions bridging empirical success and theory.
problem Understanding the function class implicitly learned by XGBoost.
method Introduced an infinite-dimensional function class and a complexity measure, showing optimizers of XGBoost are also optimizers of a penalized regression problem.
result Proved the least squares estimator achieves nearly minimax-optimal convergence rate over the function class.
GPU-accelerated BART speeds up Bayesian regression.
problem Long running time of BART makes it impractical for large datasets.
method GPU-enabled implementation of BART.
result BART is now 200x faster on GPUs.
Study compares machine learning algorithms for predicting SST in the Great Barrier Reef.
problem Predicting sea surface temperature in the Great Barrier Reef region.
method Ridge regression, LASSO, Random Forest, and Extreme Gradient Boosting (XGBoost) algorithms were evaluated.
result XGBoost significantly outperforms other algorithms in terms of predictive accuracy and Kullback-Leibler Divergence.
MAC combines models without locking them, improving ensemble performance.
problem Improving ensemble learning performance with flexibility.
method Model agnostic combination technique that dynamically combines models.
result MAC outperforms classical methods and competitive to boosting.
Extends Infinitesimal Jackknife for model covariance, enhancing ensemble model analysis.
problem Estimating uncertainty in combinations of models.
method Extends Infinitesimal Jackknife to estimate covariance between models.
result Theoretical consistency of Infinitesimal Jackknife covariance estimate demonstrated.
This study extends verifiable learning to boosted tree ensembles, enabling efficient security verification.
problem Efficiently verifying the robustness of boosted tree ensembles against norm-based attackers.
method Formal verification of robustness for large-spread boosted tree ensembles, considering L∞-norm and pseudo-polynomial time for Lp-norm verification. result Polynomial time verification for L∞-norm attackers, NP-hard for other norms, and pseudo-polynomial time for Lp-norm verification. KANEL combines models for early hit enrichment in virtual screening.
problem Assessing model accuracy in chemical bioactivity predictions.
method Ensemble workflow using Kolmogorov-Arnold Networks (KANs) and other models.
result Improves early hit enrichment metrics like PPV@N.
Framework improves financial predictions with deep learning models.
problem Adverse financial conditions like regime changes and low signal-to-noise ratios.
method Incremental use of decision trees and XGBoost models for robust performance.
result Two-layer deep ensemble of XGBoost models outperforms single models under different market regimes.
BoostForest combines multiple BoostTree models for improved accuracy.
problem Improving ensemble learning performance.
method BoostTree uses gradient boosting and random cut-points. BoostForest bootstraps training data and randomly samples parameters.
result BoostForest outperforms classical ensemble methods on 35 datasets.
Online leading has disrupted the traditional consumer banking sector with more effective loan processing. Risk prediction and monitoring is critical for the success of the business model. Traditional credit score models fall short in applying big data technology in building risk model. In this manuscript, data with var…
Study benchmarks classical models over quantum in DeFi yield prediction.
problem Accurate yield and performance forecasting for DeFi liquidity allocation.
method Benchmarked six models on Curve Finance pools' historical data.
result Classical models, especially XGBoost, outperform quantum models.
Enhanced Random Forests outperform XGBoost across binary classification datasets.
problem Improving performance of Random Forests in binary classification.
method Adaptive sample and model weighting, iterative algorithm for sample weights, personalized tree weighting schemes.
result Significantly outperforms XGBoost across 15 binary classification datasets.
XGBoost predicts NEPSE Index log returns with low error and high directional accuracy.
problem Forecasting daily log-returns in the NEPSE Index with high accuracy.
method XGBoost machine learning, feature engineering, hyperparameter optimization, walk-forward validation.
result Optimal XGBoost configuration achieves lowest log-return RMSE and MAE.
Study improves detection of cryptocurrency pump-and-dump schemes.
problem Class imbalance in P&D detection due to rare events.
method Synthetic Minority Oversampling Technique (SMOTE) and ensemble learning models.
result XGBoost and LightGBM achieved high recall rates (94.87% and 93.59%) with strong F1-scores.
TabPFN-2.5 boosts tabular AI performance, especially for large datasets.
problem Improving tabular AI models for large datasets and complex feature sets.
method Advanced tabular foundation model with data augmentation and distillation techniques.
result TabPFN-2.5 outperforms tuned tree-based models and AutoGluon 1.4 on benchmark datasets.
Paper explores how feature interactions improve XGBoost models.
problem Improving XGBoost model performance and interpretability.
method Identifies feature interactions as constraints in XGBoost models.
result Accurate identification of feature interactions leads to significant model performance improvements.
Proposes a generalized XGBoost method for nonconvex loss functions.
problem Limited to convex loss functions in XGBoost.
method Extends XGBoost to use nonconvex loss functions and multivariate loss functions.
result Generalized XGBoost method can model multiple parameters in various distributions.
XGBoost implements AFT models for survival regression.
problem Survival regression for time-to-event data.
method Loss functions for AFT models in XGBoost.
result XGBoost's AFT model improves generalization and training speed.
LIFE framework improves model accuracy and interpretability.
problem Achieving high prediction accuracy and interpretability in neural networks.
method Three-step process: subset definition, feature creation, and linear model combination.
result LIFE consistently outperforms other models in prediction accuracy and interpretability.
This study introduces a new GAS blending ensemble model for Bitcoin price prediction.
problem Predicting Bitcoin price fluctuations in the cryptocurrency market.
method Integrates advanced ensemble learning methods, feature selection algorithms, and sentiment analysis.
result The GAS model demonstrates excellent performance in daily Bitcoin trend prediction.
agtboost speeds up gradient tree boosting with automatic complexity adjustment.
problem Speeding up and simplifying gradient tree boosting computations.
method Adaptive gradient tree boosting with automatic complexity adjustment and feature importance.
result Significant decrease in computation time and simplification of model complexity.
The paper presents Imbalance-XGBoost, a Python package that combines the powerful XGBoost software with weighted and focal losses to tackle binary label-imbalanced classification tasks. Though a small-scale program in terms of size, the package is, to the best of the authors' knowledge, the first of its kind which prov…
Transformer pre-training improves stock return prediction accuracy.
problem Improving stock price prediction accuracy for better investment decisions.
method Pre-trained transformer models on TSX index, fine-tuned for individual stocks, compared to LSTM and XGBoost.
result Transformer model achieved lower mean squared error than benchmarks.
A method uses ITD and XGBoost for precise power transformer fault diagnosis.
problem Fault diagnosis of power transformers using DGA data.
method Ranking DGA parameters by skewness, extracting ITD features, and using an XGBoost classifier.
result The method achieves over 95% accuracy in classification.
This paper aims to explore models based on the extreme gradient boosting (XGBoost) approach for business risk classification. Feature selection (FS) algorithms and hyper-parameter optimizations are simultaneously considered during model training. The five most commonly used FS methods including weight by Gini, weight b…
XGBoost detects unlawful insider trading with high accuracy.
problem Detecting unlawful insider trading from large volumes of transactions.
method Applying eXtreme Gradient Boosting (XGBoost) for identifying and ranking key features.
result XGBoost achieves 97% accuracy in detecting unlawful transactions.
Ensemble model predicts AD progression from CN status with high accuracy.
problem Early prediction of clinical progression from cognitively normal to mild cognitive impairment or Alzheimer's disease.
method Ensemble survival analysis combining penalized Cox regression, advanced survival models, and aggregation techniques.
result Ensemble model achieved peak C-index of 0.907 and integrated time-dependent AUC of 0.904, outperforming baseline models.
Study predicts heart failure patient survival using stacked ensemble ML.
problem Predicting survival of heart failure patients.
method Collect and analyze patient data, apply SMOTE, use K-Means, Fuzzy C-Means clustering, Random Forest, XGBoost, Decision Tree, and propose a stacked ensemble model.
result Supervised ML algorithms outperform unsupervised models, achieving high accuracy and F1 score.
Robust cancer screening model using pre-trained ensembles for biomarkers.
problem Detecting early-stage cancer, especially in hard-to-diagnose cases like pancreatic cancer.
method Meta-trained Hyperfast model for robust classification, combined with ensembling of XGBoost and LightGBM.
result Achieved highest AUC of 0.9929 and robust performance on imbalanced datasets.
Model combination, often regarded as a key sub-field of ensemble learning, has been widely used in both academic research and industry applications. To facilitate this process, we propose and implement an easy-to-use Python toolkit, combo, to aggregate models and scores under various scenarios, including classification…
Paper introduces arctan pinball loss for XGBoost quantile regression.
problem Efficiently predicting multiple quantiles with XGBoost.
method Smooth approximation of pinball loss for XGBoost, using arctan pinball loss.
result Arctan pinball loss reduces quantile crossings and improves efficiency.
We describe the multi-GPU gradient boosting algorithm implemented in the XGBoost library (https://github.com/dmlc/xgboost). Our algorithm allows fast, scalable training on multi-GPU systems with all of the features of the XGBoost library. We employ data compression techniques to minimise the usage of scarce GPU memory …
XGBoost is often presented as the algorithm that wins every ML competition. Surprisingly, this is true even though predictions are piecewise constant. This might be justified in high dimensional input spaces, but when the number of features is low, a piecewise linear model is likely to perform better. XGBoost was exten…
NetRCA algorithm locates network faults by analyzing derived features and leveraging unlabeled data.
problem Locating the true root cause of network faults is challenging due to complex architectures and limited labeled data.
method NetRCA algorithm extracts derived features, generates new training data, and combines multiple models to enhance performance.
result NetRCA outperforms existing methods in fault cause localization on real-world data.
In this paper, we present our approach for solving the DEBS Grand Challenge 2018. The challenge asks to provide a prediction for (i) a destination and the (ii) arrival time of ships in a streaming-fashion using Geo-spatial data in the maritime context. Novel aspects of our approach include the use of ensemble learning …
Hybrid quantum-classical model boosts S&P 500 prediction accuracy to 60.14%.
problem Challenges in financial market prediction, especially high noise and non-stationarity.
method Combines quantum sentiment analysis, Decision Transformer, and model selection strategies.
result Achieved 60.14% directional accuracy on S&P 500, a 3.10% improvement.
Study uses machine learning to identify IBD biomarkers from gut microbiota.
problem Identifying biomarkers for Inflammatory Bowel Disease (IBD) from gut microbiota.
method Ensemble feature selection methods (CMIM, FCBF, mRMR, XGBoost) applied to IBD-associated metagenomics dataset.
result XGBoost minimizes microbiota used for IBD diagnosis, improving classification accuracy.
Note that a newer expanded version of this paper is now available at: arXiv:1802.03888 It is critical in many applications to understand what features are important for a model, and why individual predictions were made. For tree ensemble methods these questions are usually answered by attributing importance values to i…
This study compares logistic regression and XGBoost for predicting credit risk.
problem Predicting credit risk in financial services.
method Advanced machine learning techniques (logistic regression and XGBoost) with data preprocessing.
result XGBoost outperforms logistic regression in predicting credit risk.
Interpreting predictions from tree ensemble methods such as gradient boosting machines and random forests is important, yet feature attribution for trees is often heuristic and not individualized for each prediction. Here we show that popular feature attribution methods are inconsistent, meaning they can lower a featur…