Clustering ensemble, or consensus clustering, has emerged as a powerful tool for improving both the robustness and the stability of results from individual clustering methods. Weighted clustering ensemble arises naturally from clustering ensemble. One of the arguments for weighted clustering ensemble is that elements (…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “weighted ensembles”
This paper develops a new theory for ensemble learning beyond variance reduction.
problem Ensemble learning's effectiveness for stable estimators is not fully explained by variance reduction.
method Develops a general weighting theory for ensemble learning, formalizing ensembles as linear operators and introducing geometric and spectral constraints.
result Structured weights can outperform uniform averaging by reshaping approximation geometry and redistributing spectral complexity.
Optimizing Ensemble Weights and Hyperparameters of Machine Learning Models for Regression Problemsstat.ML
Aggregating multiple learners through an ensemble of models aim to make better predictions by capturing the underlying distribution of the data more accurately. Different ensembling methods, such as bagging, boosting, and stacking/blending, have been studied and adopted extensively in research and practice. While baggi…
Seasonal influenza infects between 10 and 50 million people in the United States every year, overburdening hospitals during weeks of peak incidence. Named by the CDC as an important tool to fight the damaging effects of these epidemics, accurate forecasts of influenza and influenza-like illness (ILI) forewarn public he…
WASH trains ensembles with shuffled weights to improve accuracy and reduce communication.
problem Training ensembles for weight averaging leads to models converging to different loss basins.
method WASH randomly shuffles a small percentage of weights during training to keep models within the same basin.
result WASH achieves state-of-the-art image classification accuracy with lower communication costs.
In this study, a novel sparsity-driven weighted ensemble classifier (SDWEC) that improves classification accuracy and minimizes the number of classifiers is proposed. Using pre-trained classifiers, an ensemble in which base classifiers votes according to assigned weights is formed. These assigned weights directly affec…
Bayesian Quadrature improves ensembling for neural networks with dispersed likelihood peaks.
problem Ensembling neural networks struggles with dispersed, narrow peaks in likelihood surfaces.
method Uses Bayesian Quadrature to construct weighted ensembles of architectures.
result Empirically outperforms state-of-the-art baselines in test likelihood, accuracy, and expected calibration error.
New hyperparameter ensembles boost neural network performance and uncertainty.
problem Improving neural network robustness and uncertainty quantification.
method Designing ensembles over both weights and hyperparameters, stratified across random initializations.
result Hyper-deep and hyper-batch ensembles outperform deep and batch ensembles on various architectures.
This paper studies recursive ensembles driven by Fibonacci updates, improving learning dynamics.
problem Improving learning dynamics in recursive ensemble learning.
method Develops second-order recursive architectures with Fibonacci-type update flows.
result Establishes global convergence conditions and generalization bounds for recursive ensembles.
This paper improves SSL methods using ensemble techniques with data-dependent weighted losses.
problem Improving SSL performance and robustness with large unlabeled data.
method Developed a framework for weighted cross-entropy losses in ensembling SSL methods without altering the backbone.
result Our method outperforms state-of-the-art SSL methods on ImageNet-1K, especially in few-shot learning.
Fibonacci Ensembles use Fibonacci weights to improve ensemble learning, inspired by natural growth patterns.
problem Improving ensemble learning methods to enhance model performance and interpretability.
method Introduces Fibonacci weights and a recursive ensemble dynamic to reduce variance and enrich representational depth.
result Fibonacci weighting can match or improve upon uniform averaging in ensemble learning experiments.
Diverse Instances-Weighting Ensemble based on Region Drift Disagreement for Concept Drift Adaptationcs.LG
DiwE uses regional distribution changes to create diverse ensemble classifiers for concept drift.
problem Handling concept drift in evolving data streams.
method DiwE measures diversity based on regional distribution disagreement and uses it to weight instances and select classifiers.
result DiwE outperforms other algorithms on various synthetic and real-world data stream benchmarks.
Best-of- improves LLM performance by efficiently allocating inference-time computation.
problem Achieving optimal performance in test-time LLM ensembling with infinite budget.
method Adaptive generation scheme and weighted ensembles of LLMs, formulated as mixed-integer linear program.
result Optimal ensemble weighting improves performance over individual models.
Partial Fusion of Neural Networks: Efficient Tradeoffs Between Ensembles and Weight Aggregationcs.LG
Partial fusion combines neural networks to balance accuracy and efficiency.
problem Balancing accuracy and computational cost in neural networks.
method Extending weight aggregation methods based on neuron-level similarity, using partial optimal transport to match similar neurons.
result Achieves a flexible tradeoff between computational cost and performance.
Proposes SWA for adversarial training to improve model robustness.
problem Overfitting in adversarial training due to insufficient sample complexity.
method Adversarial training with Stochastic Weight Average (SWA).
result Improves model robustness on CIFAR-10, CIFAR-100, and SVHN datasets.
A RL approach dynamically assigns and updates weights of ensemble models for better time series forecasting.
problem Static weight assignment for ensemble models fails to capture dynamic data changes.
method Reinforcement Learning (RL) to dynamically update weights of each model at different time instants.
result Dynamic weighted approach using RL learns weights better than static methods.
SWEEN improves certified robustness via weighted ensembling of smoothed classifiers.
problem Finding optimal base classifiers for randomized smoothing.
method Smoothed WEighted ENsembling (SWEEN) scheme.
result SWEEN achieves optimal certified robustness under mild assumptions.
Study improves Bayesian optimisation with ensemble transfer learning.
problem Improving sample efficiency in Bayesian optimisation of expensive functions.
method Empirical analysis of ensemble-based transfer learning methods and pipeline components.
result Two components (warm start initialisation and positive weight constraint) improve transfer learning Bayesian optimisation performance.
CDST improves ensemble prediction by adjusting model weights based on covariates.
problem Improving ensemble prediction accuracy in complex scenarios.
method Covariate-dependent stacking (CDST) with flexible model weights estimated via cross-validation.
result CDST consistently outperforms conventional model averaging methods in complex datasets.
We propose an algorithm to enhance certified robustness of a deep model ensemble by optimally weighting each base model. Unlike previous works on using ensembles to empirically improve robustness, our algorithm is based on optimizing a guaranteed robustness certificate of neural networks. Our proposed ensemble framewor…
Novel analysis improves weighted majority vote in multiclass classification.
problem Improving the performance of weighted majority vote in multiclass classification.
method Analyzes expected risk of weighted majority vote, considering prediction correlations and provides a bound for efficient minimization.
result Minimization of the new bound typically does not degrade the test error of the ensemble.
The recently introduced dropout training criterion for neural networks has been the subject of much attention due to its simplicity and remarkable effectiveness as a regularizer, as well as its interpretation as a training procedure for an exponentially large ensemble of networks that share parameters. In this work we …
Bayesian Neural Networks improve geophysical model ensembles with reduced uncertainty.
problem Improving geophysical model projections and uncertainty quantification.
method Developed a Bayesian Neural Network ensemble strategy for geophysical models.
result Bayesian Neural Network ensemble outperforms existing methods in ozone prediction.
A new algorithm improves credit scoring accuracy for imbalanced data.
problem Poor classification of minority class in credit scoring data sets.
method Weighted-Hybrid-Sampling-Boost (WHSBoost) algorithm with balanced data sampling.
result WHSBoost outperforms other methods in credit scoring accuracy.
Meta-learning predicts optimal ensemble size and methods for time series forecasting.
problem Finding the best ensemble of time series forecasting methods.
method Two-step approach using meta-learning to predict ensemble size and methods.
result Meta-learning outperformed benchmarks in forecasting errors for all data types and horizons.
New framework learns interpretable rule ensembles without sacrificing accuracy.
problem Trade-off between accuracy and interpretability in rule ensembles.
method Introduces local interpretability and a regularizer to promote it, using coordinate descent with local search.
result Learns rule ensembles with fewer rules to explain individual predictions, maintaining comparable accuracy.
We develop ensemble Convolutional Neural Networks (CNNs) to classify the transportation mode of trip data collected as part of a large-scale smartphone travel survey in Montreal, Canada. Our proposed ensemble library is composed of a series of CNN models with different hyper-parameter values and CNN architectures. In o…
A new ensemble of Gaussian processes improves active learning efficiency.
problem Efficiently labeling data in high-cost domains like medical imaging.
method Adaptive weighted ensemble of Gaussian processes (EGP) for active learning.
result EGP-based approaches outperform single GP-based active learning methods.
A Simple Zero-shot Prompt Weighting Technique to Improve Prompt Ensembling in Text-Image Modelscs.LG
Automates zero-shot classification by scoring and weighting prompts.
problem Improving zero-shot accuracy through prompt ensembling.
method Automatic prompt scoring and weighting method.
result Method outperforms existing techniques on various benchmarks.
With the increasing volume of data in the world, the best approach for learning from this data is to exploit an online learning algorithm. Online ensemble methods are online algorithms which take advantage of an ensemble of classifiers to predict labels of data. Prediction with expert advice is a well-studied problem i…
FoRDE uses input gradients to improve neural network ensembles.
problem Improving neural network ensembles for robustness and accuracy.
method Proposes FoRDE, an ensemble learning method based on ParVI, which repels function space by input gradients.
result FoRDE significantly outperforms DEs and other ensemble methods in accuracy and calibration.
WildWood improves Random Forest predictions using bootstrap out-of-bag samples.
problem Improving Random Forest predictions for supervised learning.
method Uses bootstrap out-of-bag samples to compute improved predictions by aggregating all possible subtrees with exponential weights.
result WildWood produces faster and more competitive predictions compared to other ensemble methods.
As data streams become more prevalent, the necessity for online algorithms that mine this transient and dynamic data becomes clearer. Multi-label data stream classification is a supervised learning problem where each instance in the data stream is classified into one or more pre-defined sets of labels. Many methods hav…
TREX explains tree ensembles by identifying key training examples.
problem Identifying which training examples most influence tree ensemble predictions.
method TREX builds a surrogate model using a kernel that captures tree ensemble structure, approximating the original model.
result TREX provides accurate and effective explanations for tree ensembles.
BatchEnsemble reduces ensemble costs by 3X in training and testing.
problem High costs for training and testing ensembles of neural networks.
method Defines each weight matrix as a Hadamard product of a shared matrix and a rank-one matrix per member.
result Achieves 3X speedup and 3X memory reduction in test time for ensembles of size 4.
Specialists outperform generalists in ensemble classification.
problem Determining the accuracy of an ensemble of classifiers when individual classifier accuracies are known.
method Proved upper and lower bounds on ensemble accuracy, constructed specialist and generalist classifiers.
result Upper and lower bounds on ensemble accuracy, practical implications for classifier construction.
Study on Gaussian ensemble of matrix products with mixed moments computed.
problem Understanding the statistical properties of matrix products of Gaussian matrices.
method Analysis of a multi-Wishart ensemble and enumeration of non-crossing pairings.
result Mixed moments of the product matrix are computed and found to be weighted by Fuss-Catalan numbers at large .
Vote-boosting is a sequential ensemble learning method in which the individual classifiers are built on different weighted versions of the training data. To build a new classifier, the weight of each training instance is determined in terms of the degree of disagreement among the current ensemble predictions for that i…
Dropout and similar stochastic neural network regularization methods are often interpreted as implicitly averaging over a large ensemble of models. We propose STE (stochastically trained ensemble) layers, which enhance the averaging properties of such methods by training an ensemble of weight matrices with stochastic r…
Enhances model OOD detection by diversifying feature representation.
problem Limited diversity in model ensembles for OOD detection.
method Integrates diverse training tasks into model ensembles to generate distinct feature representations.
result Superior performance of Multi-Comprehension Ensemble in OOD detection.
Ensemble learning is a mainstay in modern data science practice. Conventional ensemble algorithms assigns to base models a set of deterministic, constant model weights that (1) do not fully account for variations in base model accuracy across subgroups, nor (2) provide uncertainty estimates for the ensemble prediction,…
Gradient-free ensemble learns sector forecasts from diverse models.
problem Predicting sector returns in a volatile market.
method Dynamic model combination using out-of-sample R-squared.
result Ensemble outperforms individual models in sector rotation.
WRSE predicts dynamic survival distributions in ICU patients.
problem Dynamic assessment of ICU patient mortality risk.
method Non-parametric weighted-resolution ensemble model combining binary classifiers.
result Competitive results with state-of-the-art models, reducing training time.
Recent work on mode connectivity in the loss landscape of deep neural networks has demonstrated that the locus of (sub-)optimal weight vectors lies on continuous paths. In this work, we train a neural network that serves as a hypernetwork, mapping a latent vector into high-performance (low-loss) weight vectors, general…
DAMVI algorithm improves imbalanced binary classification by adjusting weights of examples and classifiers.
problem Imbalanced binary classification tasks where minority class is underrepresented.
method DAMVI algorithm increases positive example weights and optimizes classifier weights using PAC-Bayesian C-Bound.
result DAMVI outperforms state-of-the-art models on various imbalanced datasets.
New method models aptamer libraries as Boltzmann-weighted graph ensembles for better affinity predictions.
problem Anomalous candidates in SELEX datasets obscure true aptamer-ligand affinity.
method Boltzmann graph ensemble embeddings for thermodynamically parameterized exponential-family random graphs.
result Proposed embedding enables robust community detection and subgraph-level explanations for aptamer ligand affinity.
Improved neural networks by averaging late-stage weights.
problem Improving the performance of neural networks.
method Ensemble late-stage weights and average them.
result Augmenting standard models with late-phase weights improves generalization.
Deep Regression Ensemblesstat.ML
DRE combines DNN with random feature regression for efficient neural network design.
problem Designing and training deep neural networks (DNN) efficiently and effectively.
method DRE architecture with two-layer neural networks, randomly drawn input and output weights trained with linear ridge regression.
result DRE outperforms state-of-the-art DNN in many data sets with lower computational cost.