ECV method optimizes ensemble parameters for randomized ensembles.
problem Efficient tuning of ensemble parameters in randomized ensembles.
method ECV (Extrapolated Cross-Validation) method for tuning ensemble and subsample sizes.
result ECV yields δ-optimal ensembles for squared prediction risk.
Ensemble validation shows selectivity penalties but variety benefits.
problem Selecting classifiers for ensemble models and their error bounds.
method Forming an ensemble from a set of hypothesis classifiers, selecting randomly, with an error bound formula.
result No penalty for using a richer hypothesis set if same fraction selected.
Proposes a method to generate counterfactuals for ensemble models using entropic risk measures.
problem Finding a single counterfactual explanation for an ensemble of models.
method Incorporates entropic risk measure into a constrained optimization to generate counterfactuals valid for an adjustable fraction of models.
result Entropic risk measure allows generation of counterfactuals valid for all models in the ensemble under a limiting case.
We improve prediction risk estimation for large datasets using sketching and ridge regression.
problem Estimating prediction risks for large datasets efficiently and accurately.
method Random matrix theory, generalized cross validation, sketched ridge regression ensembles, and ensemble trick.
result Consistent risk estimation and prediction intervals for large-scale datasets.
Develops a prediction method based on sampling design.
problem Creating accurate individual predictions.
method Design-based approach using expected cross-validation results.
result Valid inference of unobserved prediction errors defined with respect to sampling design.
Paper uses ensemblers to predict sepsis early from patient records.
problem Early detection of sepsis in patients.
method Imputation and weak ensembler technique applied to 40k patient records.
result Model achieved 93.45% accuracy and 0.271 utility score.
A new test validates ensemble models against the null hypothesis.
problem Validating ensemble models against the null hypothesis of a constant response.
method Randomized permutation test on SVEM model predictions.
result The test maintains Type I error rate even with more parameters than observations.
Combines classifiers from different types to improve ensemble accuracy.
problem Improving ensemble accuracy by combining classifiers of different types.
method Builds heterogeneous ensembles by pooling classifiers from multiple homogeneous ensembles, using cross-validation or out-of-bag data for optimal composition.
result Optimal heterogeneous ensemble compositions can be determined using cross-validation or out-of-bag data.
Corrects GCV for inconsistent risk estimation in finite ensembles of penalized estimators.
problem Inconsistent risk estimation of GCV for finite ensembles of penalized estimators.
method Identifies a correction involving an additional scalar correction based on degrees of freedom adjusted training errors from each ensemble component.
result CGCV maintains computational advantages of GCV and is model-free uniformly consistent for ridge regression.
Study ridge ensembles in proportional feature-to-sample size regime, proving risk equivalence and GCV consistency.
problem Characterizing and optimizing ridge ensembles in proportional feature-to-sample size regimes.
method Proportional asymptotics analysis, GCV for tuning, proving risk equivalence.
result Risk of optimal full ridgeless ensemble matches optimal ridge predictor's risk.
Study develops ensemble machine learning framework for predicting groundwater heavy metal pollution.
problem Statistical complexity and spatial heterogeneity of heavy metal contamination in groundwater.
method Nested cross-validated ensemble machine learning with response transformations (raw, log, Gaussian copula).
result Copula-based models with DBSCAN clustering diagnostics provide the most reliable and interpretable assessments of groundwater contamination.
5th place in video labeling challenge using deep learning.
problem Efficient large-scale video labeling.
method Ensemble of frame and video level models trained on augmented dataset with cross-validation.
result Ranked 5th in the Google Cloud and YouTube-8M Video Understanding Challenge.
The paper proposes a method to evaluate clustering models using ensemble techniques.
problem Lack of robust cluster validity scores for unsupervised learning.
method Cluster ensemble aggregation techniques and normalized mutual information.
result The method can highlight standout clustering and hyperparameter configurations in an ensemble.
Vote-boosting uses weighted training data to build accurate and robust ensembles.
problem Generating accurate and robust ensemble classifiers.
method Sequential ensemble learning with weighted training data and emphasis on instances with high disagreement.
result Vote-boosting is effective for generating accurate and robust ensembles, especially when noise levels are low.
ECG outperforms graph clustering algorithms using ensemble method.
problem Graph clustering challenges.
method ECG combines Louvain algorithm and consensus clustering.
result ECG outperforms leading algorithms on artificial networks.
A new method balances accuracy and diversity in ensemble pruning.
problem Balancing accuracy and diversity in ensemble learning.
method Formalizing ensemble pruning as an objection maximization problem based on information entropy, proposing a distributed framework.
result Achieves less time-consuming execution with minimal accuracy degradation.
Bayesian Neural Networks improve geophysical model ensembles with reduced uncertainty.
problem Improving geophysical model projections and uncertainty quantification.
method Developed a Bayesian Neural Network ensemble strategy for geophysical models.
result Bayesian Neural Network ensemble outperforms existing methods in ozone prediction.
Paper analyzes uncertainty metrics in ensemble learning for healthcare AI.
problem Selecting appropriate uncertainty metrics for ensemble learners in healthcare AI.
method Rigorous analysis of two uncertainty metrics: ensemble mean and variance.
result Ensemble mean is preferable to ensemble variance for decision making in healthcare AI.
Improved neural network ensembles using Stein Variational Newton updates.
problem Lack of efficient second-order information in current ensemble methods.
method Proposes a novel approximate Bayesian inference method integrating Stein Variational Newton updates with scalable Hessian approximations.
result Significantly faster convergence and more accurate posterior distribution approximations.
This paper proves long-time accuracy of ensemble Kalman filters for chaotic and machine-learned systems.
problem Ensuring long-term accuracy of ensemble Kalman filters for complex dynamical systems.
method Established conditions for long-time accuracy of ensemble Kalman filters for chaotic and machine-learned dynamical systems.
result Ensemble Kalman filters maintain small estimation error over long time horizons for chaotic and machine-learned systems.
New deep learning method validated across multiple sleep staging databases.
problem Improving automatic sleep scoring accuracy across different datasets.
method Ensemble of local models using deep learning for automatic sleep staging.
result Good general performance compared to human experts and state-of-the-art methods.
CDST improves ensemble prediction by adjusting model weights based on covariates.
problem Improving ensemble prediction accuracy in complex scenarios.
method Covariate-dependent stacking (CDST) with flexible model weights estimated via cross-validation.
result CDST consistently outperforms conventional model averaging methods in complex datasets.
Ensemble++ uses shared-factor ensembles to scale Thompson Sampling for linear and nonlinear bandits.
problem Computational challenges in Thompson Sampling for large-scale or non-conjugate settings.
method Ensemble++ with shared-factor architecture and random linear combinations.
result Ensemble++ achieves comparable regret to exact Thompson Sampling with significantly smaller ensemble sizes.
In this article, the logic rule ensembles approach to supervised learning is applied to the unsupervised or semi-supervised clustering. Logic rules which were obtained by combining simple conjunctive rules are used to partition the input space and an ensemble of these rules is used to define a similarity matrix. Simila…
Optimal model improves AUC, recall, and F1 score for class-imbalanced business risk.
problem Improving prediction of class-imbalanced business risk.
method Resampling, regularization, and model ensembling techniques.
result Boosting on DT with SMOTE oversampling achieves AUC, recall, and F1 score of 0.8633, 0.9260, and 0.8907, respectively.
Bayesian nonparametric ensemble improves uncertainty quantification in ensemble learning.
problem Accurate quantification of model uncertainty in ensemble learning.
method Bayesian nonparametric ensemble (BNE) approach that augments existing ensemble models.
result BNE achieves accurate uncertainty estimates and decomposes overall predictive uncertainty into distinct components.
The paper extends calibration to sets of probabilistic classifiers, finding many ensembles are poorly calibrated.
problem Evaluating the validity of epistemic uncertainty in sets of probabilistic classifiers.
method Proposed a novel nonparametric calibration test for sets of probabilistic classifiers.
result Ensembles of deep neural networks are often not well calibrated.
The paper connects neural network ensembles to Bayesian inference using variational methods.
problem Explaining the behavior of ensemble methods in neural networks.
method Deriving conditions for ensemble optimization to reduce divergence to the posterior distribution.
result Ensemble methods can be a valid alternative to approximate Bayesian inference.
An ensemble method enhances cryptocurrency trading strategies using deep reinforcement learning.
problem Improving generalization performance in stochastic cryptocurrency trading environments.
method Model selection and mixture distribution policy to ensemble deep reinforcement learning models.
result Improved out-of-sample performance compared to benchmarks.
Automates zero-shot classification by scoring and weighting prompts.
problem Improving zero-shot accuracy through prompt ensembling.
method Automatic prompt scoring and weighting method.
result Method outperforms existing techniques on various benchmarks.
Paper uses stacking with neural networks to predict cryptocurrency price direction.
problem Predicting the direction of cryptocurrency prices.
method Generative and discriminative classifiers stacked over a one-layer neural network, using technical indicators and sentiment analysis.
result Stacking method outperformed individual models in accuracy.
Paper proposes ensemble distillation for well-calibrated structured prediction.
problem Well-calibrated predictions are hard to achieve in structured prediction.
method Ensemble distillation framework for structured prediction.
result Ensemble distillation produces well-calibrated models with similar performance and calibration benefits to ensembles.
Research reveals how diversity impacts ensemble generalization in classification tasks.
problem Understanding the relationship between diversity and generalization in classification ensembles.
method Investigated diversity measurement, its relationship with generalization error, and pruning methods.
result Generalization error is reduced effectively only when diversity is increased in specific ranges, not in others.
The motivation of this work is to improve the performance of standard stacking approaches or ensembles, which are composed of simple, heterogeneous base models, through the integration of the generation and selection stages for regression problems. We propose two extensions to the standard stacking approach. In the fir…
Novel U-learning method for predicting continuous outcomes from high-dimensional data.
problem Challenges in making valid inferences on predictions from high-dimensional inputs.
method U-learning via combinatory multi-subsampling for ensemble predictions and confidence intervals.
result Valid inferences on predictions from Lasso and neural networks.
Stacked conformal prediction simplifies model validation.
problem Validating stacked predictive models efficiently.
method Meta-learner at the top of a stacked ensemble for approximate marginal validity.
result The method achieves approximate marginal validity without a separate calibration sample.
New theory improves understanding of ensemble learning systems.
problem Improving the effectiveness of ensemble learning systems.
method Generalized Fano's inequality to account for information loss in model combination.
result Reveals strengths and weaknesses of ensemble systems on accuracy and diversity.
Improved anomaly detection for incipient faults using ensemble learning.
problem Difficulty in detecting milder anomalies due to similarity to normal conditions.
method Utilize uncertainty information from ensemble learning to identify misclassified incipient anomalies.
result Ensemble learning improves performance on incipient anomaly detection.
A new method combines quantile regression, cross-conformalization, and out-of-bag predictions for valid prediction sets.
problem Valid prediction sets for classification and regression without distributional assumptions.
method Nested conformal prediction framework, combining quantile regression, cross-conformalization, and out-of-bag predictions.
result QOOB algorithm performs best or close to best on all simulated and real datasets.
The paper analyzes bootstrap ensemble classifiers in high-dimensional settings.
problem Performance of bootstrap ensemble classifiers in high-dimensional data.
method Random Matrix Theory applied to LSSVM ensemble.
result Strategies to optimize performance of LSSVM ensemble.
We describe and extract time-ordered multibody interactions from complex systems.
problem Complex systems with temporal and multibody dependencies.
method Decompose multivariate Markov chains into time-ordered multibody interactions. Algorithm to extract interactions from data. Measure complexity of interaction ensembles.
result Robust and efficient algorithm to infer time-ordered multibody interactions from data.
Improved gas demand forecasting using ensemble methods.
problem Short-term prediction of gas demand components.
method Nine base forecasters (Ridge Regression, GP, NN, ANN, Torus, LASSO, Elastic Net, RF, SVR) and four ensemble predictors (simple, weighted, subset, SVR aggregation) were evaluated.
result Ensemble predictors outperformed individual base forecasters and TSO predictions.
One of the most tedious tasks in the application of machine learning is model selection, i.e. hyperparameter selection. Fortunately, recent progress has been made in the automation of this process, through the use of sequential model-based optimization (SMBO) methods. This can be used to optimize a cross-validation per…
Proposes a neural network loss function for better uncertainty estimation.
problem Challenges in estimating predictive uncertainty of neural networks.
method Bayesian Validation Metric (BVM) framework with ensemble learning.
result Competitive and robust uncertainty estimation on in-distribution and out-of-distribution data.
This study improves sales forecasting for Intel Corporation in the semiconductor industry.
problem Accurate sales forecasting in the semiconductor industry for Intel Corporation.
method Innovative incorporation of various indicators into quantitative models, including multiple regressions, time series analysis, random forest, and boosting tree. Ensemble models selected based on validation errors and moving windows validation.
result Development of an ensemble model that captures distinct characteristics at lead time and lines of business levels, improving response to market fluctuations.
Paper proves global convergence of NCELM model.
problem Ensuring global convergence of NCELM model.
method Two-stage process: random base learners and NCL penalty term updates.
result Global convergence of NCELM proved using Banach theorem.
Adaptive ensemble improves flu forecasts with minimal data.
problem Accurate flu forecasts to help public health.
method Adaptive stacking of ensembles, changing model weights weekly.
result Adaptive ensemble outperforms static ensembles in flu forecasts.
This paper improves deep learning model consistency through ensemble methods.
problem Consistency and correct-consistency issues in deep learning models.
method Formal definition of consistency and correct-consistency, proving ensemble improvement, proposing dynamic snapshot ensemble method.
result Ensemble methods can improve correct-consistency of deep learning models.