arXiv research

Ensembles of random-feature models can't outperform a single large model.

Jointly tuning ensemble models improves performance and uncertainty calibration.

Ensembles of neural networks learn better by sharing information.

Enhances optimization and sampling methods using ensemble-based gradient inference.

ForestPrune optimizes tree ensemble pruning for compactness and speed.

We propose an algorithm to enhance certified robustness of a deep model ensemble by optimally weighting each base model. Unlike previous works on using ensembles to empirically improve robustness, our algorithm is based on optimizing a guaranteed robustness certificate of neural networks. Our proposed ensemble framewor…

Aggregating multiple learners through an ensemble of models aim to make better predictions by capturing the underlying distribution of the data more accurately. Different ensembling methods, such as bagging, boosting, and stacking/blending, have been studied and adopted extensively in research and practice. While baggi…

Quantum machine learning uses superposition to create a large ensemble of classifiers.

Transforms ensemble predictions to maintain interpretability.

Optimal survival trees ensemble reduces tree count and improves predictive performance.

Study ridge ensembles in proportional feature-to-sample size regime, proving risk equivalence and GCV consistency.

Tree ensemble kernels improve Bayesian optimization for mixed features and constraints.

New technique prevents Q-learning collapse by maximizing diversity among ensembles.

Optimal number of voters for a voting ensemble can be estimated from the distribution of classifier errors.

Multi-headed ensembles boost model performance with faster training.

ECV method optimizes ensemble parameters for randomized ensembles.

Unified framework for portfolio optimization using multiple hypotheses.

Optimal allocation between explainable and black box models for high performance and explainability.

We consider the problem of learning decision rules for prediction with feature budget constraint. In particular, we are interested in pruning an ensemble of decision trees to reduce expected feature cost while maintaining high prediction accuracy for any test example. We propose a novel 0-1 integer program formulation …

Ensemble learning is a methodology that integrates multiple DNN learners for improving prediction performance of individual learners. Diversity is greater when the errors of the ensemble prediction is more uniformly distributed. Greater diversity is highly correlated with the increase in ensemble accuracy. Another attr…

SWEEN improves certified robustness via weighted ensembling of smoothed classifiers.

EDRBO optimizes Bayesian optimization with continuous contexts using ensemble models and robust methods.

We develop ensemble Convolutional Neural Networks (CNNs) to classify the transportation mode of trip data collected as part of a large-scale smartphone travel survey in Montreal, Canada. Our proposed ensemble library is composed of a series of CNN models with different hyper-parameter values and CNN architectures. In o…

DSL uses supervised learning to optimize portfolios, improving stability and performance.

FIRE extracts interpretable rules from tree ensembles.

New method reduces ensemble size for linear bandits, achieving near optimal regret.

Convex regression is a promising area for bridging statistical estimation and deterministic convex optimization. New piecewise linear convex regression methods are fast and scalable, but can have instability when used to approximate constraints or objective functions for optimization. Ensemble methods, like bagging, sm…

Proposes optimizing neural network ensemble diversity in feature space.

Proposes a meta-learning method for robust portfolio optimization.

Optimizes feature shifts for tree ensemble reclassification.

New method combines ensembling and regularization for genomic disease prediction.

A new ensemble filter uses transport maps and MMD optimization for high-dimensional data assimilation.

Optimal ensemble construction improves prediction accuracy for multi-study tasks, especially in pandemic scenarios.

New hyperparameter ensembles boost neural network performance and uncertainty.

Tree ensemble method tackles multi-objective constrained optimization in energy systems.

POETS optimizes LLMs by combining policy ensembles and KL regularization.

One of the most tedious tasks in the application of machine learning is model selection, i.e. hyperparameter selection. Fortunately, recent progress has been made in the automation of this process, through the use of sequential model-based optimization (SMBO) methods. This can be used to optimize a cross-validation per…

The paper analyzes the risk of bagging regularized M-estimators under proportional asymptotics.

Existing information-theoretic frameworks based on maximum entropy network ensembles are not able to explain the emergence of heterogeneity in complex networks. Here, we fill this gap of knowledge by developing a classical framework for networks based on finding an optimal trade-off between the information content of a…

In this paper we propose to perform model ensembling in a multiclass or a multilabel learning setting using Wasserstein (W.) barycenters. Optimal transport metrics, such as the Wasserstein distance, allow incorporating semantic side information such as word embeddings. Using W. barycenters to find the consensus between…

Best-of-$\infty$ improves LLM performance by efficiently allocating inference-time computation.

The paper verifies the robustness of classifier ensembles against randomized attacks.

We address the problem of aggregating an ensemble of predictors with known loss bounds in a semi-supervised binary classification setting, to minimize prediction loss incurred on the unlabeled data. We find the minimax optimal predictions for a very general class of loss functions including all convex and many non-conv…

A new ensemble method improves kNN performance by extending the neighborhood rule.

Tree ensembles, such as random forests and boosted trees, are renowned for their high prediction performance. However, their interpretability is critically limited due to the enormous complexity. In this study, we present a method to make a complex tree ensemble interpretable by simplifying the model. Specifically, we …

Given a classifier ensemble and a set of examples to be classified, many examples may be confidently and accurately classified after only a subset of the base models in the ensemble are evaluated. This can reduce both mean latency and CPU while maintaining the high accuracy of the original ensemble. To achieve such gai…

Seasonal influenza infects between 10 and 50 million people in the United States every year, overburdening hospitals during weeks of peak incidence. Named by the CDC as an important tool to fight the damaging effects of these epidemics, accurate forecasts of influenza and influenza-like illness (ILI) forewarn public he…

Ensemble techniques are powerful approaches that combine several weak learners to build a stronger one. As a meta-learning framework, ensemble techniques can easily be applied to many machine learning methods. Inspired by ensemble techniques, in this paper we propose an ensemble loss functions applied to a simple regre…