Auto-Ensemble automates deep learning model ensembling with adaptive learning rate scheduling.
problem Difficulty in collecting diverse and accurate deep learning models through single training.
method Auto-Ensemble collects model checkpoints and uses adaptive learning rate scheduling to ensemble them.
result Ensembled models converge to various local optima, improving performance on few-shot learning.
AEA dynamically aggregates ensemble targets for actor-critic learning.
problem Static ensemble aggregation methods struggle with overestimation bias and variance.
method Adaptive Ensemble Aggregation (AEA) dynamically constructs ensemble-based targets.
result AEA converges to optimal variance reduction and maximal Fisher information.
Seasonal influenza infects between 10 and 50 million people in the United States every year, overburdening hospitals during weeks of peak incidence. Named by the CDC as an important tool to fight the damaging effects of these epidemics, accurate forecasts of influenza and influenza-like illness (ILI) forewarn public he…
Adaptive XGBoost improves accuracy on evolving data streams by updating the ensemble dynamically.
problem Concept drift in evolving data streams.
method Adapts XGB to update the ensemble with new data, maintaining consistency with current concept.
result Improves classification accuracy on evolving data streams compared to other methods.
New method quantifies uncertainty in fine-tuned LLMs using LoRA ensembles.
problem Uncertainty in fine-tuned LLMs and how to trust their predictions.
method Posterior approximations using low-rank adaptation ensembles.
result Unexpected retention of acquired knowledge during fine-tuning in overfitting regime.
Three adaptive methods improve financial forecasting and portfolio management.
problem Improving financial forecasting and portfolio management in volatile markets.
method Dynamic Model Selection (DMS), Adaptive Ensemble (AE), Dynamic Asset Allocation (DAA).
result Adaptive methods outperform long-only benchmarks in US market returns.
Ensembling multiple predictions is a widely used technique for improving the accuracy of various machine learning tasks. One obvious drawback of ensembling is its higher execution cost during inference. In this paper, we first describe our insights on the relationship between the probability of prediction and the effec…
Best-of-∞ improves LLM performance by efficiently allocating inference-time computation.
problem Achieving optimal performance in test-time LLM ensembling with infinite budget.
method Adaptive generation scheme and weighted ensembles of LLMs, formulated as mixed-integer linear program.
result Optimal ensemble weighting improves performance over individual models.
EnMDAP aligns conditional distributions for multi-source domain adaptation using pseudolabels.
problem Training a target model with no labeled data in the absence of target data labels.
method EnMDAP uses label-wise moment matching and ensemble learning with multiple feature extractors.
result EnMDAP achieves state-of-the-art performance in multi-source domain adaptation tasks.
AGM improves model accuracy through adaptive generation and feature augmentation.
problem Improving model accuracy in machine learning competitions.
method Adaptive Generation Model (AGM) using stacked ensemble learning with horizontal and vertical model expansion and feature augmentation.
result AGM outperforms previous models in 7 data sets.
ABHT boosts regression by filtering regions with different smoothness.
problem Improving regression performance through local adaptivity.
method Gradient boosting with adaptive histogram transform.
result ABHT converges faster than PEHT in Hölder continuous spaces.
Differentially private ensemble classifiers adapt to data streams while protecting privacy.
problem Adapting to evolving data characteristics while protecting private information.
method Unbounded ensemble updates, model agnostic approach.
result Outperforms competitors on various privacy, drift, and distribution settings.
A new ensemble of Gaussian processes improves active learning efficiency.
problem Efficiently labeling data in high-cost domains like medical imaging.
method Adaptive weighted ensemble of Gaussian processes (EGP) for active learning.
result EGP-based approaches outperform single GP-based active learning methods.
Optimizes biomolecular simulations by ranking adaptive sampling policies.
problem Efficiently sampling biomolecular systems to capture complex dynamical behaviors.
method Metric-driven ranking of adaptive sampling policies to identify the optimal policy for each round.
result Different adaptive sampling policies lead to faster convergence and improved sampling performance.
Study reviews tree-based methods and introduces new ensemble strategies.
problem Improving the efficiency and performance of tree-based machine learning models.
method Review of tree-based methods, introduction of ISLE framework, ARM model combination strategy, and modified ISLEs.
result Performance evaluation of modified ISLEs on real data sets.
Paper tackles multi-source domain adaptation for regression.
problem Predicting HDL cholesterol levels using gut microbiome data.
method Two-step procedure: 1) Extend a flexible single-source DA algorithm for classification to regression. 2) Augment with ensemble learning for multi-source DA.
result Consistent improvement in HDL cholesterol level prediction performance over existing methods.
TabSurv adapts tabular neural networks for survival analysis.
problem Survival analysis on tabular data using deep learning methods.
method Adapts modern tabular architectures to survival analysis using Weibull distribution or non-parametric prediction. Optimizes SurvHL histogram loss function.
result TabSurv consistently outperforms classical and deep learning baselines on 10 real-world survival datasets.
Ant colonies and boosting algorithms both reduce bias and variance through adaptive mechanisms.
problem Understanding the mathematical principles behind ensemble learning and ant colony behavior.
method Developed a formal mapping between AdaBoost's adaptive reweighting and ant recruitment dynamics.
result Proved that the fundamental theorem of weak learnability has a direct analog in colony decision-making.
Recent researches have shown that deep forest ensemble achieves a considerable increase in classification accuracy compared with the general ensemble learning methods, especially when the training set is small. In this paper, we take advantage of deep forest ensemble and introduce the Dense Adaptive Cascade Forest (daF…
Paper explains how tree ensembles improve predictions by smoothing and regulating smoothness.
problem Understanding why tree ensembles perform well despite their complexity.
method Interpreting tree ensembles as adaptive and self-regularizing smoothers.
result Ensemble trees make more smooth predictions than individual trees and adjust smoothness based on input dissimilarity.
The non-stationary nature of electroencephalography (EEG) signals makes an EEG-based brain-computer interface (BCI) a dynamic system, thus improving its performance is a challenging task. In addition, it is well-known that due to non-stationarity based covariate shifts, the input data distributions of EEG-based BCI sys…
Hybrid engine analyzes news sentiment for markets in real-time.
problem Real-time market analysis of news sentiment.
method Three-way ensemble learning combining financial lexicon, adaptive TF-IDF clustering, and auto-calibrated weighting.
result Adaptive statistical clustering learner improves adaptability to market changes.
Gradient-free ensemble learns sector forecasts from diverse models.
problem Predicting sector returns in a volatile market.
method Dynamic model combination using out-of-sample R-squared.
result Ensemble outperforms individual models in sector rotation.
Ensemble learning is a mainstay in modern data science practice. Conventional ensemble algorithms assigns to base models a set of deterministic, constant model weights that (1) do not fully account for variations in base model accuracy across subgroups, nor (2) provide uncertainty estimates for the ensemble prediction,…
FASE-AL uses active learning to reduce labeling costs for data streams.
problem Reduction of labeling costs for data stream classification.
method Combines Fast Adaptive Stacking of Ensembles (FASE) with active learning.
result Achieves high accuracy with minimal labeled data.
New RL algorithms adapt to time limits, improving task performance.
problem Fixed RL behaviors cannot adapt to different time restrictions.
method Introduced two algorithms for time adaptive RL: Independent Gamma-Ensemble and n-Step Ensemble.
result Zero-shot adaptation between different time restrictions.
New NIDS uses hypergraphs for real-time detection of evolving port scans.
problem Challenges in offline NIDS for auto-generated port scans.
method Hypergraph-based metrics for real-time ML ensemble NIDS.
result High accuracy, precision, and recall in detecting port scans and attacks.
Data collection and annotation are time-consuming in machine learning, expecially for large scale problem. A common approach for this problem is to transfer knowledge from a related labeled domain to a target one. There are two popular ways to achieve this goal: adversarial learning and self training. In this article, …
Study cryptocurrency price dynamics using adaptive EMD and spectral analysis.
problem Analyze the time-varying volatility of cryptocurrency prices.
method Adaptive complementary ensemble empirical mode decomposition (ACE-EMD) and Hilbert spectral analysis.
result Reveal the properties of various timescales in cryptocurrency price dynamics.
SDF adapts Deep Forest for evolving data streams with active learning.
problem Adapting Deep Forest for evolving data streams.
method Streaming Deep Forest (SDF) with Augmented Variable Uncertainty (AVU) active learning.
result SDF with AVU outperforms other methods trained with all instances by 70% labeling budget.
Extracting actionable intelligence from distributed, heterogeneous, correlated and high-dimensional data sources requires run-time processing and learning both locally and globally. In the last decade, a large number of meta-learning techniques have been proposed in which local learners make online predictions based on…
New framework uses tree ensembles for contextual bandits.
problem Optimizing decisions in dynamic environments with contextual information.
method Adapts Upper Confidence Bound and Thompson Sampling to tree ensemble methods.
result Tree ensemble methods outperform traditional methods in regret minimization and runtime.
We show how an ensemble of Q∗-functions can be leveraged for more effective exploration in deep reinforcement learning. We build on well established algorithms from the bandit setting, and adapt them to the Q-learning setting. We propose an exploration strategy based on upper-confidence bounds (UCB). Our experimen…
IE-GP framework learns sequentially arriving data with adaptive kernels.
problem Sequential data adaptation in Gaussian process models.
method Incremental ensemble of Gaussian process learners with adaptive weights.
result IE-GP framework effectively learns and adapts to sequential data.
This study evaluates the robustness of transformation-based ensemble defense against evasion attacks.
problem Understanding the reasons behind the robustness improvement in transformation-based ensemble defense.
method Designing two adaptive attacks to evaluate transformation-based ensemble defense, conducting experiments to analyze robustness.
result The robustness improvement is mainly from irreversible transformations rather than the ensemble of models.
We investigate an algorithm named histogram transform ensembles (HTE) density estimator whose effectiveness is supported by both solid theoretical analysis and significant experimental performance. On the theoretical side, by decomposing the error term into approximation error and estimation error, we are able to condu…
The paper explores how prior functions and bootstrapping improve ensemble uncertainty estimation.
problem Improving uncertainty estimation in machine learning models.
method Investigates the benefits of prior functions and bootstrapping in ensemble models.
result Prior functions and bootstrapping enhance ensemble agents' uncertainty estimation across different inputs.
In this paper we examine the effect of applying ensemble learning to the performance of collaborative filtering methods. We present several systematic approaches for generating an ensemble of collaborative filtering models based on a single collaborative filtering algorithm (single-model or homogeneous ensemble). We pr…
We propose an online method for concept driftdetection based on dynamic classifier ensemble selection. Theproposed method generates a pool of ensembles by promotingdiversity among classifier members and chooses expert ensemblesaccording to global prequential accuracy values. Unlike currentdynamic ensemble selection app…
Proposes a new method to enhance neural learning by maximizing information gain.
problem Improving neural learning by selecting key variables to maximize information gain.
method Adaptive Ensemble Kalman Filter to quantify uncertainty and maximize information gain.
result The proposed method enables the neural network to learn more effectively from stochastic systems.
Paper improves training physics-informed neural networks with model ensembles.
problem Training physics-informed neural networks (PINNs) is difficult due to convergence to wrong solutions.
method Proposes training an ensemble of PINNs, using ensemble agreement to expand the solution interval.
result Algorithm stabilizes PINN training and yields competitive performance.
ReWTS ensemble improves time-series forecasting by adapting to changing dynamics.
problem Complex, multi-faceted, evolving data in process industries.
method Chunk-based, recency-weighted temporal segmentation of data for multi-step forecasting.
result Significantly outperforms conventional models in mean squared forecasting error.
When constructing a classifier ensemble, diversity among the base classifiers is one of the important characteristics. Several studies have been made in the context of standard static data, in particular, when analyzing the relationship between a high ensemble predictive performance and the diversity of its components.…
DriftMoE adapts to concept drifts in data streams efficiently.
problem Adapting to non-stationary data streams with concept drifts.
method Online Mixture-of-Experts (MoE) architecture with co-training framework.
result DriftMoE achieves competitive results with state-of-the-art stream learning adaptive ensembles.
The dynamic ensemble selection of classifiers is an effective approach for processing label-imbalanced data classifications. However, such a technique is prone to overfitting, owing to the lack of regularization methods and the dependence of the aforementioned technique on local geometry. In this study, focusing on bin…
FONT clusters patients across health systems with privacy and efficiency.
problem Challenges in multi-site cluster analysis due to data-sharing restrictions.
method Federated One-shot Ensemble Clustering (FONT) algorithm that requires only a single round of communication and exchanges only fitted model parameters and class labels.
result FONT improves consistency of patient clusters across sites compared to locally fitted clusters.
Ensembles of decision trees perform well on many problems, but are not interpretable. In contrast to existing approaches in interpretability that focus on explaining relationships between features and predictions, we propose an alternative approach to interpret tree ensemble classifiers by surfacing representative poin…
ESS improves MCMC efficiency for correlated & multimodal distributions.
problem Slice Sampling's sensitivity to initial length scale and difficulty with correlated distributions.
method Adaptive tuning and parallel walkers for efficient sampling.
result ESS improves efficiency by more than an order of magnitude on correlated distributions.