arXiv research

Ensembles of random-feature models can't outperform a single large model.

Study ridge ensembles in proportional feature-to-sample size regime, proving risk equivalence and GCV consistency.

ECV method optimizes ensemble parameters for randomized ensembles.

This work investigates power laws in deep neural network ensembles and predicts their performance.

New method reduces ensemble size for linear bandits, achieving near optimal regret.

The paper analyzes the risk of bagging regularized M-estimators under proportional asymptotics.

JUCAL jointly calibrates aleatoric and epistemic uncertainties in classifier ensembles.

In this work a novel method to quantify spectral ergodicity for random matrices is presented. The new methodology combines approaches rooted in the metrics of Thirumalai-Mountain (TM) and Kullbach-Leibler (KL) divergence. The method is applied to a general study of deep and recurrent neural networks via the analysis of…

Paper analyzes ensemble Kalman updates for effective dimension and localization.

ForestPrune optimizes tree ensemble pruning for compactness and speed.

Ensembling smaller models can outperform larger models in terms of accuracy and efficiency.

Ensemble++ uses shared-factor ensembles to scale Thompson Sampling for linear and nonlinear bandits.

The paper develops a stationary-distribution theory for Random Forest ensemble size selection.

A new ensemble learning method called Residual Likelihood Forests improves performance and reduces model size.

The number of component classifiers chosen for an ensemble greatly impacts the prediction ability. In this paper, we use a geometric framework for a priori determining the ensemble size, which is applicable to most of existing batch and online ensemble classifiers. There are only a limited number of studies on the ense…

Improved regret bound for linear ensemble sampling.

Meta-learning predicts optimal ensemble size and methods for time series forecasting.

Enhanced ensemble filters use machine learning to improve accuracy in filtering models.

Analog methods improve forecast accuracy in complex models.

Adaptive XGBoost improves accuracy on evolving data streams by updating the ensemble dynamically.

AEA dynamically aggregates ensemble targets for actor-critic learning.

Ensembles of deep neural networks significantly improve generalization accuracy. However, training neural network ensembles requires a large amount of computational resources and time. State-of-the-art approaches either train all networks from scratch leading to prohibitive training cost that allows only very small ens…

Efficient method for vertex embedding and community detection.

Ensembles of neural networks improve training dynamics and performance.

Hypermodels improve exploration efficiency and accuracy.

The use of ensembles of neural networks (NNs) for the quantification of predictive uncertainty is widespread. However, the current justification is intuitive rather than analytical. This work proposes one minor modification to the normal ensembling methodology, which we prove allows the ensemble to perform Bayesian inf…

Heterogeneous ensembles built from the predictions of a wide variety and large number of diverse base predictors represent a potent approach to building predictive models for problems where the ideal base/individual predictor may not be obvious. Ensemble selection is an especially promising approach here, not only for …

Although the methods of bagging and random forests are some of the most widely used prediction methods, relatively little is known about their algorithmic convergence. In particular, there are not many theoretical guarantees for deciding when an ensemble is "large enough" --- so that its accuracy is close to that of an…

Memory split advantage: thinner networks outperform a single wide network.

Jointly tuning ensemble models improves performance and uncertainty calibration.

Ensemble method for fast portfolio valuation and risk management.

Optimal number of voters for a voting ensemble can be estimated from the distribution of classifier errors.

Ensemble models refer to methods that combine a typically large number of classifiers into a compound prediction. The output of an ensemble method is the result of fitting a base-learning algorithm to a given data set, and obtaining diverse answers by reweighting the observations or by resampling them using a given pro…

Study on how intraclass variability affects Temporal Ensembling accuracy.

Residual networks' depth is mathematically equivalent to expanding an implicit ensemble size.

In ensemble methods, the outputs of a collection of diverse classifiers are combined in the expectation that the global prediction be more accurate than the individual ones. Heterogeneous ensembles consist of predictors of different types, which are likely to have different biases. If these biases are complementary, th…

New method combines ensembling and regularization for genomic disease prediction.

Neural architecture search (NAS) is gaining more and more attention in recent years due to its flexibility and remarkable capability to reduce the burden of neural network design. To achieve better performance, however, the searching process usually costs massive computations that might not be affordable for researcher…

Ensembles of neural networks learn better by sharing information.

Ensembles, as a widely used and effective technique in the machine learning community, succeed within a key element -- "diversity." The relationship between diversity and generalization, unfortunately, is not entirely understood and remains an open research issue. To reveal the effect of diversity on the generalization…

Real world systems typically feature a variety of different dependency types and topologies that complicate model selection for probabilistic graphical models. We introduce the ensemble-of-forests model, a generalization of the ensemble-of-trees model. Our model enables structure learning of Markov random fields (MRF) …

A new method creates simpler, more interpretable decision trees from complex ensembles.

E-CIT framework reduces CITs' computational burden and improves causal discovery performance.

ES reduces high-probability regret in stochastic linear bandits.

New metrics improve quantum ensemble learning efficiency and power.

Ensembles, where multiple neural networks are trained individually and their predictions are averaged, have been shown to be widely successful for improving both the accuracy and predictive uncertainty of single neural networks. However, an ensemble's cost for both training and testing increases linearly with the numbe…

Deep neural networks improve ensemble weather forecasts.

The paper analyzes bootstrap ensemble classifiers in high-dimensional settings.