Paper proposes ECOC for deep neural network ensembles to improve performance.
problem Designing an ensemble of deep networks is time-consuming and often not beneficial.
method ECOC framework applied to deep networks, with design strategies to balance accuracy and complexity.
result Proposed combinatory technique achieves highest classification performance.
Study improves Persian handwritten letter recognition using ECOC ensemble method.
problem Improving accuracy in identifying Persian handwritten letters.
method ECOC ensemble method with feature selection and Support Vector Machine (SVM).
result ECOC ensemble method outperforms other methods in identifying Persian handwritten letters.
A new ECOC algorithm enhances microarray data classification.
problem Classification of multi-class microarray datasets with small samples and overlaps.
method Enhance Class Separability related Data Complexity measures during encoding process.
result Our ECOC algorithm improves microarray data classification effectiveness and robustness.
Paper designs optimal ECOCs using IP for robust multiclass classification.
problem Designing robust ECOCs for multiclass classification.
method Integer Programming formulation to minimize codebooks with desirable error-correcting properties, leveraging graph-theoretic structure and edge clique covers.
result IP-generated codebooks achieve high nominal and robust adversarial accuracy.
Paper presents Label Mapping method to solve large-scale multi-class classification problems.
problem Large-scale multi-class classification problems in deep neural networks.
method Label Mapping (LM) method to decompose the problem into smaller sub-problems.
result LM significantly outperforms standard methods in terms of accuracy and model complexity.
The use of Reinforcement Learning in real-world scenarios is strongly limited by issues of scale. Most RL learning algorithms are unable to deal with problems composed of hundreds or sometimes even dozens of possible actions, and therefore cannot be applied to many real-world problems. We consider the RL problem in the…
Efficient graph-based decoding improves extreme classification accuracy.
problem Learning algorithms for extreme classification with large label sets.
method ECOC with loss-based decoding on graph-induced output codes.
result Efficient loss-based decoding on graph output codes improves classification accuracy.
APA improves brain decoding accuracy using fMRI.
problem Decoding human brain patterns from fMRI images.
method APA combines anatomical feature extraction and AdaBoost for binary and multi-class predictions.
result APA outperforms existing methods in decoding visual stimuli.
APA improves brain decoding accuracy for visual stimuli.
problem Decoding patterns in human brain using MVPA.
method Developed novel anatomical feature extraction and AdaBoost algorithm.
result Superior performance in decoding visual stimuli categories.
A new neural sequence prediction method using error-correcting codes improves accuracy and convergence.
problem Improving neural sequence prediction accuracy and speed.
method Error-correcting output codes, separable code maximization, latent variable mixture sampling.
result Consistent improvements on language modeling datasets and text generation tasks.
ECNN combines binary classifiers to protect neural networks from adversarial attacks.
problem Vulnerability of neural networks to adversarial inputs.
method Designs an error-correcting code matrix to maximize row and column distances, training end-to-end.
result ECNN effectively defends against adversarial attacks with good accuracy on normal examples.
Multi-headed ensembles boost model performance with faster training.
problem Limited computational resources hinder ensemble search performance.
method Extend NES to multi-headed ensembles, leveraging end-to-end training and one-shot NAS methods.
result Multi-headed ensemble search finds robust ensembles 3 times faster with comparable performance.
This paper explores how diverse neural network ensembles improve prediction accuracy and robustness against deception.
problem Improving prediction accuracy and robustness of neural networks against adversarial attacks.
method Examines and measures ensemble diversity, develops algorithms for creating and combining diverse ensembles.
result Greater diversity in neural network ensembles leads to higher accuracy and robustness against deception.
Transforms ensemble predictions to maintain interpretability.
problem Loss of interpretability in deep ensembles.
method Proposes transformation ensembles that aggregate predictions while preserving interpretability.
result Transformation ensembles yield better predictions than individual models and maintain interpretability.
Overparameterized ensembles don't offer generalization benefits over single large models.
problem Theoretical limitations of ensembles in overparameterized settings.
method Using ensembles of random feature (RF) regressors, the paper clarifies how modern ensembles differ from underparameterized counterparts.
result Infinite ensembles of overparameterized RF regressors become pointwise equivalent to single infinite-width RF regressors, and finite width ensembles converge to single models with the same parameter budget.
New method creates diverse neural ensembles for better uncertainty estimation and robustness.
problem Creating more robust neural networks for uncertainty estimation and dataset shift.
method Automatically constructing ensembles with varying architectures.
result Ensembles with varying architectures outperform deep ensembles in accuracy, uncertainty calibration, and robustness.
New method certifies joint adversarial robustness of model ensembles.
problem Ensuring robustness of model ensembles against adversarial attacks.
method Proposes a novel technique to certify joint robustness, building on prior work on single-model robustness certification.
result Demonstrates the effectiveness of certifying joint robustness of ensembles, improving understanding of ensemble defenses.
Embedded ensembles improve neural network performance efficiently.
problem Improving neural network performance with fewer resources.
method Analyzing the wide network limit of gradient descent dynamics using Neural-Tangent-Kernel.
result Embedded ensembles exhibit two regimes: independent and collective, affecting performance.
New algorithms improve ensemble diversity, leading to more accurate and smaller models.
problem Building accurate predictive models with diverse base predictors.
method Integrates ensemble diversity into a reinforcement learning framework for ensemble selection.
result Diversity-incorporating ensembles are more accurate and smaller in size.
A framework preserves uncertainty in ensemble distillation.
problem Preserving uncertainty decomposition in ensemble distillation.
method General framework for distilling both regression and classification ensembles, preserving natural uncertainty decomposition.
result Framework maintains decomposition of predictive uncertainty.
Study pitfalls of deep learning ensembles in uncertainty estimation.
problem Pitfalls in in-domain uncertainty estimation and ensembling in deep learning.
method Exploration of standards for uncertainty quantification and broad study of ensembling techniques.
result Many sophisticated ensembling techniques are equivalent to a simple ensemble of few networks.
New hyperparameter ensembles boost neural network performance and uncertainty.
problem Improving neural network robustness and uncertainty quantification.
method Designing ensembles over both weights and hyperparameters, stratified across random initializations.
result Hyper-deep and hyper-batch ensembles outperform deep and batch ensembles on various architectures.
InfiniteBoost builds an infinite ensemble using gradient descent.
problem Building high accuracy ensembles for various machine learning tasks.
method Combines gradient boosting and random forests properties using gradient descent to create an infinite ensemble.
result InfiniteBoost achieves high accuracy on regression, classification, and ranking tasks.
Proposes a method to reduce ensemble size while maintaining accuracy.
problem Complexity and computational burden of ensemble models in large-scale data.
method Optimizes margin distribution to reduce ensemble size while increasing diversity.
result Pruned ensemble uses only a fraction of original classifiers with improved or similar generalization performance.
ECV method optimizes ensemble parameters for randomized ensembles.
problem Efficient tuning of ensemble parameters in randomized ensembles.
method ECV (Extrapolated Cross-Validation) method for tuning ensemble and subsample sizes.
result ECV yields δ-optimal ensembles for squared prediction risk.
EnD 2 ^2 2 distills ensemble diversity into a single model.
problem Loss of ensemble diversity information in distillation.
method Prior Networks for distilling ensemble distribution into a single model.
result EnD 2 ^2 2 retains both ensemble performance and diversity information. The paper explores dynamic ensembles for multi-step forecasting.
problem Lack of research on dynamic ensembles for multi-step forecasting.
method Extensive experiments with 3568 time series and an ensemble of 30 multi-output models.
result Dynamic ensembles based on arbitrating and windowing perform best.
A new ensemble technique improves power forecasting for renewable energy.
problem Improving accuracy in power forecasting for renewable energy.
method Coopetitive soft-gating in a multi-scheme ensemble.
result The technique outperforms other models on publicly available data sets.
Bayesian deep ensembles improve prediction accuracy in various settings.
problem Improving prediction accuracy of deep ensembles in out-of-distribution settings.
method Introducing a randomised, untrainable function to each ensemble member, enabling a posterior predictive distribution interpretation.
result Bayesian deep ensembles make more conservative predictions and outperform standard ensembles in various tasks.
RCAM-based ensemble combines binary classifiers using similarity and vote scheme.
problem Improving binary classification accuracy through ensemble methods.
method RCAM-based ensemble combining classifiers using similarity and recurrent consult-vote scheme.
result RCAM-based ensemble outperforms individual classifiers and majority voting.
The paper simplifies complex tree ensembles for better interpretability.
problem Limited interpretability of tree ensembles like random forest and boosted trees.
method A post-processing method that approximates complex tree ensembles with a simpler, interpretable model using the EM algorithm.
result Complex tree ensembles can be approximated reasonably by simpler, interpretable models.
High-capacity neural network ensembles often benefit more from high-capacity models than from increased diversity.
problem The performance of high-capacity neural network ensembles is often harmed by interventions that promote predictive diversity.
method A large-scale study of nearly 600 neural network classification ensembles, examining various interventions and architectures.
result Discouraging predictive diversity can be benign in large-network ensembles, and higher-capacity models often yield better performance than diverse architectures.
MotherNets speed up deep ensemble learning with shared costs.
problem Training deep neural network ensembles is computationally expensive and time-consuming.
method MotherNets train a single or small set of MotherNets to share data movement and computation costs across ensembles, generating target ensembles by transferring function from MotherNets.
result MotherNets provide a new Pareto frontier for accuracy vs. training cost, scaling improvements as ensemble size increases.
Combines classifiers from different types to improve ensemble accuracy.
problem Improving ensemble accuracy by combining classifiers of different types.
method Builds heterogeneous ensembles by pooling classifiers from multiple homogeneous ensembles, using cross-validation or out-of-bag data for optimal composition.
result Optimal heterogeneous ensemble compositions can be determined using cross-validation or out-of-bag data.
Jointly tuning ensemble models improves performance and uncertainty calibration.
problem Improving both predictive performance and uncertainty calibration in deep ensembles.
method Investigated the impact of jointly tuning weight decay, temperature scaling, and early stopping.
result Jointly tuning ensemble models generally matches or improves performance, with significant variation across tasks.
New method reduces ensemble size for linear bandits, achieving near optimal regret.
problem Achieving near optimal regret in linear bandits with limited ensemble size.
method Ensemble sampling with a size of order d log T d \log T d log T for a d d d -dimensional stochastic linear bandit. result Regret is at most ( d log T ) 5 / 2 T (d \log T)^{5/2} \sqrt{T} ( d log T ) 5/2 T , improving over linear scaling with T T T . This paper reviews weighted clustering ensemble methods.
problem Improving clustering results from individual methods.
method Different types of weights and approaches to determining weight values.
result Unified framework for selecting appropriate weighting mechanisms.
Paper analyzes uncertainty metrics in ensemble learning for healthcare AI.
problem Selecting appropriate uncertainty metrics for ensemble learners in healthcare AI.
method Rigorous analysis of two uncertainty metrics: ensemble mean and variance.
result Ensemble mean is preferable to ensemble variance for decision making in healthcare AI.
Proposes a new method for deep ensembles that improves accuracy and calibration.
problem Improving accuracy and calibration of deep ensembles.
method Estimates confusion matrices of ensemble members and weighs them according to their inferred performance.
result Empirically shows superiority of soft Dawid Skene over ensemble averaging.
Self-paced ensemble learning improves audio classification models.
problem Improving performance of individual models in speech and audio classification.
method A self-paced ensemble learning scheme where models learn from each other over several iterations.
result SPEL significantly outperforms baseline ensemble models.
Hydra distills ensemble models into a single model while preserving diversity and uncertainty.
problem Loss of ensemble diversity and uncertainty in distilled models.
method Single multi-headed neural network with shared body network.
result Hydra improves distillation performance and preserves ensemble diversity and uncertainty.
We propose and evaluate alternative ensemble schemes for a new instance based learning classifier, the Randomised Sphere Cover (RSC) classifier. RSC fuses instances into spheres, then bases classification on distance to spheres rather than distance to instances. The randomised nature of RSC makes it ideal for use in en…
A fast method estimates stability of ensemble feature selectors.
problem Improving stability of ensemble feature selectors for better prediction.
method Simulator of a feature selector to estimate stability.
result Reduces computation time for estimating stability.
A new method balances accuracy and diversity in ensemble pruning.
problem Balancing accuracy and diversity in ensemble learning.
method Formalizing ensemble pruning as an objection maximization problem based on information entropy, proposing a distributed framework.
result Achieves less time-consuming execution with minimal accuracy degradation.
Ensemble++ uses shared-factor ensembles to scale Thompson Sampling for linear and nonlinear bandits.
problem Computational challenges in Thompson Sampling for large-scale or non-conjugate settings.
method Ensemble++ with shared-factor architecture and random linear combinations.
result Ensemble++ achieves comparable regret to exact Thompson Sampling with significantly smaller ensemble sizes.
Ensemble unsupervised anomaly detection using IRT for hidden ground truth.
problem Challenges in constructing an ensemble from unsupervised anomaly detection methods.
method Use Item Response Theory to compute latent traits and construct an ensemble that downplays noisy methods.
result Demonstrated effectiveness of IRT ensemble on extensive data repository.
New method improves ensemble quality by exploring the pre-train basin more effectively.
problem Limited diversity in ensembles trained from a single pre-trained checkpoint.
method Proposed StarSSE modification of Snapshot Ensembles for transfer learning.
result Stronger ensembles and uniform model soups achieved.
RobBoost optimizes a deep model ensemble for certified robustness.
problem Improving the robustness of deep models.
method Optimizes a neural network's robustness certificate through model selection and weighting.
result RobBoost forms a more robust ensemble with better certified robustness and clean accuracy.