Ensemble learning is a methodology that integrates multiple DNN learners for improving prediction performance of individual learners. Diversity is greater when the errors of the ensemble prediction is more uniformly distributed. Greater diversity is highly correlated with the increase in ensemble accuracy. Another attr…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “ensemble diversity”
Research reveals how diversity impacts ensemble generalization in classification tasks.
problem Understanding the relationship between diversity and generalization in classification ensembles.
method Investigated diversity measurement, its relationship with generalization error, and pruning methods.
result Generalization error is reduced effectively only when diversity is increased in specific ranges, not in others.
High-capacity neural network ensembles often benefit more from high-capacity models than from increased diversity.
problem The performance of high-capacity neural network ensembles is often harmed by interventions that promote predictive diversity.
method A large-scale study of nearly 600 neural network classification ensembles, examining various interventions and architectures.
result Discouraging predictive diversity can be benign in large-network ensembles, and higher-capacity models often yield better performance than diverse architectures.
SharpBalance improves deep ensemble performance by balancing sharpness and diversity.
problem Improving deep ensemble performance in both in-distribution and out-of-distribution scenarios.
method Introducing SharpBalance, a novel training approach that balances sharpness and diversity within ensembles.
result SharpBalance effectively improves the sharpness-diversity trade-off and ensemble performance in ID and OOD scenarios.
New method improves ensemble diversity and generalization.
problem Ensemble diversity does not guarantee practical generalization.
method Introduced a new diversity metric and training method for extrapolating differently on local data patches.
result Improves generalization and diversity in practical settings, especially under data limits and covariate shift.
When constructing a classifier ensemble, diversity among the base classifiers is one of the important characteristics. Several studies have been made in the context of standard static data, in particular, when analyzing the relationship between a high ensemble predictive performance and the diversity of its components.…
Ensemble of diverse CNNs detects and mitigates adversarial attacks.
problem Detecting and defending against adversarial attacks.
method An ensemble of specialized CNNs with a voting mechanism.
result Significant reduction in adversarial attack risk rate.
Unified theory explains diversity in ensemble learning.
problem Explaining diversity in ensemble learning across various scenarios.
method Developed a framework revealing diversity as a hidden dimension in bias-variance decomposition.
result Proved exact bias-variance-diversity decompositions for multiple losses in regression and classification.
Diversity or complementarity of experts in ensemble pattern recognition and information processing systems is widely-observed by researchers to be crucial for achieving performance improvement upon fusion. Understanding this link between ensemble diversity and fusion performance is thus an important research question. …
Diverse Instances-Weighting Ensemble based on Region Drift Disagreement for Concept Drift Adaptationcs.LG
DiwE uses regional distribution changes to create diverse ensemble classifiers for concept drift.
problem Handling concept drift in evolving data streams.
method DiwE measures diversity based on regional distribution disagreement and uses it to weight instances and select classifiers.
result DiwE outperforms other algorithms on various synthetic and real-world data stream benchmarks.
Developing parsimonious ensembles using ensemble diversity within a reinforcement learning frameworkcs.LG
Heterogeneous ensembles built from the predictions of a wide variety and large number of diverse base predictors represent a potent approach to building predictive models for problems where the ideal base/individual predictor may not be obvious. Ensemble selection is an especially promising approach here, not only for …
SEERL uses ensemble methods to improve reinforcement learning efficiency.
problem High sample complexity and computational expense in reinforcement learning.
method Directed perturbation of model parameters to learn diverse policies, selection of an adequately diverse set of policies.
result Our approach outperforms state-of-the-art scores in Atari 2600 and Mujoco.
New method creates diverse neural ensembles for better uncertainty estimation and robustness.
problem Creating more robust neural networks for uncertainty estimation and dataset shift.
method Automatically constructing ensembles with varying architectures.
result Ensembles with varying architectures outperform deep ensembles in accuracy, uncertainty calibration, and robustness.
DASH improves ensemble generalizability by encouraging diverse, flat loss landscapes.
problem Improving generalization and robustness of deep ensembles.
method DASH promotes diversity and flatness in deep ensembles by encouraging base learners to move towards low-loss regions of minimal sharpness.
result DASH improves ensemble generalizability, as demonstrated by extensive empirical evidence.
Preventing Value Function Collapse in Ensemble {Q}-Learning by Maximizing Representation Diversitycs.LG
New technique prevents Q-learning collapse by maximizing diversity among ensembles.
problem Value function collapse in ensemble Q-learning.
method Maximizing representation diversity through regularization.
result Regularized approach significantly outperforms existing methods.
Though deep neural networks have achieved significant progress on various tasks, often enhanced by model ensemble, existing high-performance models can be vulnerable to adversarial attacks. Many efforts have been devoted to enhancing the robustness of individual networks and then constructing a straightforward ensemble…
Hydra distills ensemble models into a single model while preserving diversity and uncertainty.
problem Loss of ensemble diversity and uncertainty in distilled models.
method Single multi-headed neural network with shared body network.
result Hydra improves distillation performance and preserves ensemble diversity and uncertainty.
Diversified Ensemble of Independent Sub-Networks for Robust Self-Supervised Representation Learningstat.ML
Efficiently builds diverse sub-model ensembles for robust self-supervised learning.
problem Challenges in diversity and efficiency of deep ensembles for self-supervised representation learning.
method Ensemble of independent sub-networks with a new loss function for diversity.
result Significantly improves prediction reliability and model calibration.
SPQR improves Q-ensemble diversity in reinforcement learning.
problem Overestimation bias in Q-learning for complex tasks.
method Introduces SPQR for Q-ensemble independence regularization.
result SPQR outperforms baseline algorithms in online and offline RL benchmarks.
Paper presents a defense framework against adversarial examples.
problem Vulnerability of deep neural networks to adversarial examples.
method Cross-layer strategic ensemble defense with input and output transformations.
result Strategic ensemble defense achieves high defense success rates and robustness.
DVERGE diversifies adversarial vulnerabilities to enhance robust ensemble models.
problem Diverse adversarial vulnerabilities for robust ensemble models.
method Isolates and diversifies adversarial vulnerabilities through distillation and training.
result Achieves higher robustness against transfer attacks compared to previous methods.
Unified framework for portfolio optimization using multiple hypotheses.
problem Risk diversification in portfolio allocation.
method Structured ensemble learning approach with diversity control.
result Structured ensembles link predictor diversity to risk diversification.
The inaccuracy of neural network models on inputs that do not stem from the training data distribution is both problematic and at times unrecognized. Model uncertainty estimation can address this issue, where uncertainty estimates are often based on the variation in predictions produced by a diverse ensemble of models …
This paper improves ensemble learning for vision tasks by encouraging diversity in predictions.
problem Generating effective ensembles of neural networks for multi-modal data.
method Explicitly optimize a diversity inducing adversarial loss for learning stochastic latent variables.
result Significant improvements in classification accuracy and out-of-distribution detection compared to baselines.
Diverse projection ensembles improve distributional reinforcement learning.
problem Learning the distribution of returns in reinforcement learning.
method Combining multiple projection methods to improve model diversity and exploration.
result Diverse projection ensembles lead to significant performance improvements in exploration tasks.
Auto-Ensemble automates deep learning model ensembling with adaptive learning rate scheduling.
problem Difficulty in collecting diverse and accurate deep learning models through single training.
method Auto-Ensemble collects model checkpoints and uses adaptive learning rate scheduling to ensemble them.
result Ensembled models converge to various local optima, improving performance on few-shot learning.
Enhances model OOD detection by diversifying feature representation.
problem Limited diversity in model ensembles for OOD detection.
method Integrates diverse training tasks into model ensembles to generate distinct feature representations.
result Superior performance of Multi-Comprehension Ensemble in OOD detection.
Proposes a new method for ensembling neural subnetworks.
problem Computational expense and limited flexibility of traditional deep ensembles.
method Sequential Bayesian neural subnetwork ensembling.
result Outperforms traditional ensembles in various metrics.
HRF enhances tree diversity in random forests to improve performance.
problem Selection bias and lack of diversity in random forests.
method Introducing heterogeneity during tree construction by assigning lower weights to features used in previous trees.
result HRF outperforms other ensemble methods in accuracy across 52 datasets.
FoRDE uses input gradients to improve neural network ensembles.
problem Improving neural network ensembles for robustness and accuracy.
method Proposes FoRDE, an ensemble learning method based on ParVI, which repels function space by input gradients.
result FoRDE significantly outperforms DEs and other ensemble methods in accuracy and calibration.
This research introduces a new strategy in cluster ensemble selection by using Independency and Diversity metrics. In recent years, Diversity and Quality, which are two metrics in evaluation procedure, have been used for selecting basic clustering results in the cluster ensemble selection. Although quality can improve …
Repulsive deep ensembles improve diversity and Bayesian inference.
problem Challenges in maintaining diversity among ensemble members trained independently.
method Introducing a repulsive term in the update rule of deep ensembles.
result Training dynamics of repulsive ensembles follow a Wasserstein gradient flow of KL divergence with the true posterior.
The paper improves QD policy ensembles using distribution ratio estimators.
problem Training diverse and high-quality reinforcement learning agents.
method Using Stein variational gradient descent and distribution ratio estimators.
result The method generates diverse and high-quality reinforcement learning agents.
SAEP prunes sub-architectures to reduce search cost while maintaining performance.
problem Redundancy in ensemble sub-architectures leads to high computational cost.
method SAEP leverages diversity to prune sub-architectures, reducing ensemble size.
result SAEP reduces the number of sub-architectures without degrading performance.
The Wisdom of Crowds is a phenomenon described in social science that suggests four criteria applicable to groups of people. It is claimed that, if these criteria are satisfied, then the aggregate decisions made by a group will often be better than those of its individual members. Inspired by this concept, we present a…
Ensembles of deep neural networks significantly improve generalization accuracy. However, training neural network ensembles requires a large amount of computational resources and time. State-of-the-art approaches either train all networks from scratch leading to prohibitive training cost that allows only very small ens…
GNCL algorithm controls diversity in deep ensembles.
problem Managing bias and variance in deep ensembles.
method Generalized bias-variance decomposition for arbitrary loss functions, leading to GNCL algorithm.
result Explicit control over ensemble diversity and smooth interpolation between independent and joint training.
Stein variational neural network ensembles improve diversity and uncertainty estimation.
problem Lack of proper Bayesian justification and diversity guarantees in deep neural network ensembles.
method Particle-based inference methods, specifically Stein variational gradient descent (SVGD), operating in weight space, function space, and hybrid settings.
result SVGD methods improve diversity and uncertainty estimation, approaching the true Bayesian posterior more closely.
Consider a binary decision making process where a single machine learning classifier replaces a multitude of humans. We raise questions about the resulting loss of diversity in the decision making process. We study the potential benefits of using random classifier ensembles instead of a single classifier in the context…
Proposes a meta-learning method for robust portfolio optimization.
problem Optimizing a robust portfolio ensemble with diverse sub-portfolios.
method Uses a deep generative model with convolutional, LSTM, and dense layers to generate diverse sub-portfolios.
result The ensemble portfolio is robust and generalizes well, balancing performance and diversity.
New method uses SHapley Additive Explanations to identify anomaly detectors with complementary behaviors.
problem Challenges in unsupervised anomaly detection due to diverse data distributions and lack of labels.
method Characterize anomaly detectors using SHapley Additive Explanations to measure feature importance and similarity.
result Detectors with similar explanations produce correlated anomaly scores, while those with divergent explanations are complementary.
Packed-Ensembles improve uncertainty estimation in constrained hardware.
problem Hardware limitations restrict the size of ensembles and network capacity, degrading performance.
method Packed-Ensembles (PE) design and train lightweight structured ensembles by modulating encoding space and parallelizing into a single backbone.
result PE accurately preserves diversity and maintains performance on key metrics like accuracy, calibration, and out-of-distribution detection.
Deep ensembles improve model accuracy and robustness, but their theoretical underpinnings are not fully understood.
problem Understanding why deep ensembles work well in practice despite theoretical limitations.
method Investigating the loss landscape of neural networks and exploring the diversity of functions in function space.
result Random initializations explore diverse modes in function space, while ensembles along an optimization trajectory cluster within a single mode.
Ensemble models refer to methods that combine a typically large number of classifiers into a compound prediction. The output of an ensemble method is the result of fitting a base-learning algorithm to a given data set, and obtaining diverse answers by reweighting the observations or by resampling them using a given pro…
The combination of multiple classifiers using ensemble methods is increasingly important for making progress in a variety of difficult prediction problems. We present a comparative analysis of several ensemble methods through two case studies in genomics, namely the prediction of genetic interactions and protein functi…
BAE uses boosting to improve autoencoder ensembles for robust outlier detection.
problem Overfitting in autoencoders limits their effectiveness in unsupervised outlier detection.
method Boosting-based Autoencoder Ensemble (BAE) trains autoencoders sequentially with weighted sampling to reduce outliers and inject diversity.
result BAE outperforms state-of-the-art approaches in various outlier detection conditions.
SETrLUSI combines diverse knowledge from multiple domains for faster convergence.
problem Handling diverse knowledge from multiple domains in transfer learning.
method Stochastic Ensemble Multi-Source Transfer Learning Using Statistical Invariant (SETrLUSI).
result SETrLUSI accelerates convergence and outperforms related methods.
Deep Neural Networks are vulnerable to adversarial attacks even in settings where the attacker has no direct access to the model being attacked. Such attacks usually rely on the principle of transferability, whereby an attack crafted on a surrogate model tends to transfer to the target model. We show that an ensemble o…