New method combines adaptive learning rate and flexible prediction for online loss aggregation.
problem Online aggregation of unbounded losses using shifting experts.
method Adapted AdaHedge algorithm with Fixed Share meta-algorithm for signed unbounded losses.
result Improved shifting regret and validity of regret bounds in adversarial setting.
Paper tackles covariate shift in offline IL using less proficient behavior data.
problem Mitigating covariate shift in Imitation Learning with limited data coverage.
method Model-based IL from Offline Data (MILO) framework.
result MILO can combat covariate shift even with sub-optimal behavior policy data.
Proposes a FoE prior for improving CNN performance in distribution shifts.
problem Improving CNN performance in image analysis tasks with distribution shifts.
method Uses a field-of-experts (FoE) prior to match feature distributions of test and training images.
result Outperforms previous TTA methods in lesion segmentation and most healthy tissue segmentation tasks.
RadGrad selects expert queries based on agent error and risk to reduce costs.
problem Mitigating covariate shift in sequential decision prediction problems.
method Selective querying based on prediction of agent error and risk.
result Improves upon existing safety-aware algorithms and matches DAgger's performance.
DSAC improves robustness of imitation learning.
problem Learning from few expert data and distribution shift.
method DSAC uses adversarial reward function.
result DSAC performs well on PyBullet environments.
Behavioral cloning fails due to ignoring causal structure, leading to worse performance.
problem Behavioral cloning's failure to account for causal structure causes worse performance.
method Investigates and proposes interventions to correct causal misidentification.
result Causal misidentification occurs in various domains and can be mitigated.
We consider a variation on the problem of prediction with expert advice, where new forecasters that were unknown until then may appear at each round. As often in prediction with expert advice, designing an algorithm that achieves near-optimal regret guarantees is straightforward, using aggregation of experts. However, …
Online L2D algorithm for multiclass classification with varying experts.
problem Handling streaming data, changing expert availability, and shifting expert distribution.
method First online L2D algorithm with O ( ( n + n e ) T 2 / 3 ) O((n+n_e)T^{2/3}) O (( n + n e ) T 2/3 ) and O ( ( n + n e ) T ) O((n+n_e)\sqrt{T}) O (( n + n e ) T ) regret guarantees. result Effective extension of standard L2D to settings with varying expert availability and reliability.
TFPS improves time series forecasting by learning pattern-specific experts.
problem Challenges in forecasting time series data with varying patterns across segments.
method Dual-domain encoder, subspace clustering, pattern-specific experts.
result Significantly improved forecasting accuracy, especially in long-term forecasting.
ROME improves algorithmic fairness by learning latent group structure robustly.
problem Latent subgroup disparities and distribution shifts in machine learning models.
method ROME uses an Expectation-Maximization algorithm for linear models and a neural Mixture-of-Experts for nonlinear settings.
result ROME significantly improves fairness compared to standard methods while maintaining average performance.
New method improves traffic forecasting models by adapting to spatial shifts.
problem Improving traffic forecasting models' ability to handle spatial shifts over years.
method Proposes a novel Mixture of Experts (MoE) framework for spatiotemporal models.
result Significant improvement in performance for handling spatial distribution shifts.
DeRegiME forecasts with regime structure, improving probabilistic predictions across various time series.
problem Probabilistic forecasting discards residual uncertainty, and distribution shifts are hard to capture.
method DeRegiME uses a sparse variational Gaussian process with a nonstationary regime-mixing kernel to separate latent uncertainty regimes.
result DeRegiME improves NLPD by 20.3% on average across benchmarks, with gains on CRPS and MSE.
MM-DREX adapts LLM experts for financial trading via dynamic routing.
problem Challenges of non-stationary financial markets and static expert designs.
method MM-DREX uses a VLM-powered dynamic router to allocate expert weights and designs heterogeneous trading experts.
result Significantly outperforms 15 baselines across key metrics.
The step of expert taxa recognition currently slows down the response time of many bioassessments. Shifting to quicker and cheaper state-of-the-art machine learning approaches is still met with expert scepticism towards the ability and logic of machines. In our study, we investigate both the differences in accuracy and…
DAC-SSM learns domain-agnostic states for better imitation learning.
problem Domain shifts hinder imitation learning in partially observable tasks.
method DAC-SSM uses adversarial training to remove domain-dependent information from states.
result DAC-SSM achieves comparable performance to experts in sparse reward tasks.
Causal Imitation Learning handles noisy measurements and distribution shifts.
problem Learning from noisy state observations and distributional shifts.
method Causal inference framework and adversarial RKHS learning.
result Improved robustness to distribution shifts compared to standard methods.
Detects harmful distribution shifts in deployed models without false alarms.
problem Detecting harmful distribution shifts in deployed models without false alarms.
method Sequential tools for testing if the difference between source and target distributions leads to a significant increase in a risk function.
result Demonstrated the efficacy of the proposed framework through extensive empirical studies.
TAME learns new tasks without knowing them, outperforming existing methods.
problem Learning new tasks without task identities.
method TAME detects task shifts using loss function deviations and employs online pruning.
result TAME outperforms existing methods on continual learning benchmarks.
Paper proposes methods to help autonomous vehicles adapt to unexpected driving scenarios.
problem Autonomous vehicles struggle with unexpected driving conditions.
method Robust imitative planning (RIP) and adaptive robust imitative planning (AdaRIP) methods to detect and adapt to distribution shifts.
result Methods outperform current state-of-the-art approaches in nuScenes prediction challenge.
This work improves neural network trustworthiness through uncertainty estimation.
problem Overconfident neural networks lead to poor performance under distribution shifts.
method Develops a general uncertainty framework for neural networks, including classification with rejection.
result Improves model trustworthiness and robustness in decision-making tasks.
New approach mitigates feedback divergence in imitation learning.
problem Divergence between held-out error and learner performance in imitation learning.
method Identifies covariate shift as the root cause and proposes a simulator-based solution.
result Naive behavioral cloning performs well in real-world decision making problems.
MARCD uses generative scenarios to improve portfolio decisions during regime shifts.
problem Improving portfolio decisions under regime shifts and drawdowns.
method MARCD employs a Gaussian HMM for regime inference, a diffusion generator for scenario production, and a CVaR allocator with tail-weighted and crisis-aware components.
result MARCD reduces maximum drawdowns by 34% compared to baseline methods over 2020-2025.
MELO predicts electricity loads by adapting to shifts without external indicators.
problem Adapting to non-stationary prediction challenges in online settings.
method MELO combines multiple forgetting factors and aggregation rules to adaptively predict.
result MELO reduces RMSE by 34.7% compared to base predictors and external covariates.
Squint bound improved by removing ln ln T \ln \ln T ln ln T term.
problem Improving the Squint bound by removing the ln ln T \ln \ln T ln ln T term. method Using the Krichevsky--Trofimov algorithm to change the prior.
result Removed the ln ln T \ln \ln T ln ln T term from the Squint bound. Adaptive Conformal Inference improves time series forecasting uncertainty.
problem Uncertainty quantification in time series models with dependency.
method AgACI, an adaptive method based on online expert aggregation.
result AgACI provides efficient prediction intervals for day-ahead electricity price forecasting.
New algorithms minimize regret in changing environments for piecewise Lipschitz functions.
problem Optimizing in unpredictable, time-varying conditions for piecewise Lipschitz functions.
method Introduced shifting regret as a metric, and provided bounds for O ( s d T log T + s T 1 − β ) O(\sqrt{sdT\log T}+sT^{1-β}) O ( s d T log T + s T 1 − β ) . result Improved bounds for β β β -dispersed functions, with empirical validation in online clustering. New agent learns from previous search spaces to improve NAS efficiency.
problem NAS requires restarting learning from scratch between different search spaces.
method Transformer-based agent for joint training and knowledge transfer.
result Efficient knowledge transfer between search spaces improves NAS performance.
Multi-expert L2D underfits more severely, requiring new methods.
problem Underfitting in multi-expert L2D settings.
method PiCCE (Pick the Confident and Correct Expert), a surrogate-based method.
result PiCCE effectively reduces multi-expert L2D to a single-expert-like problem, resolving underfitting.
Unified view of context-adaptive inference across statistics, meta-learning, and foundation models.
problem Adapting predictive systems to specific situations.
method Unified framework for context-adaptive inference, bridging statistics, meta-learning, and foundation models.
result Explicit and implicit approaches are mathematically equivalent under certain conditions.
TENP prunes experts and neurons in Mixture-of-Experts models for efficient deployment.
problem Efficient deployment of large language models constrained by static parameter footprint.
method Structured Trapezoidal ExpertNeuron Pruning (TENP) identifies and retains important experts and neurons.
result DeepSeek model achieves 10% better performance on code generation tasks with 40% expert sparsity.
A method to select important experts for Gaussian processes to balance computational efficiency and uncertainty quantification.
problem Balancing computational efficiency and uncertainty quantification in Gaussian processes for big data.
method Using graphical models to select important experts and aggregate their predictions while ensuring uncertainty quantification.
result Substantially reduces computational cost of aggregating dependent experts while ensuring calibrated uncertainty quantification.
We consider the problem of contextual bandits with stochastic experts, which is a variation of the traditional stochastic contextual bandit with experts problem. In our problem setting, we assume access to a class of stochastic experts, where each expert is a conditional distribution over the arms given a context. We p…
Improved time series forecasting with expert loss integration.
problem Enhancing time series forecasting accuracy and efficiency.
method Adaptive Mixture-of-Experts framework with expert-specific loss integration and online learning.
result Significantly improved forecasting accuracy and computational efficiency.
Deep learning improves reinforcement learning in video games.
problem Challenges in reinforcement learning with high-dimensional inputs.
method Used deep Q-network and batch normalization to improve reinforcement learning.
result Some agents learned to play T-rex Runner better than human experts.
HS-MoE selects sparse experts using adaptive priors and data-adaptive gating.
problem Sparse expert selection in mixture-of-experts architectures.
method Combines horseshoe prior with input-dependent gating for data-adaptive sparsity.
result Data-adaptive sparsity in expert usage.
NAMEx merges experts using Nash bargaining for improved performance.
problem Sparse Mixture of Experts merging strategies lack a principled weighting mechanism.
method Reinterpreting expert merging through game theory, introducing Nash Merging and complex momentum.
result NAMEx consistently outperforms competing methods across various tasks and system sizes.
Expert augmentation improves hybrid model generalization.
problem Limited generalization of hybrid models outside training distribution.
method Introducing expert augmentation to improve hybrid model performance.
result Expert augmentation improves generalization of hybrid models.
Algorithm learns data structure in real-time with outliers and change points.
problem Sequential online prediction in the presence of outliers and change points.
method INTEL algorithm using WGPs and POE model for real-time structure learning.
result Significantly better performance than benchmarks in real datasets.
New method calibrates Gaussian product experts for better predictions.
problem Erratic predictions and uncalibrated uncertainty in Gaussian product experts.
method Calibration via tempered softmax and Wasserstein barycenter for predictions.
result Improved predictions with better mean and uncertainty quantification.
Meta-algorithm optimizes nonstochastic bandits with infinitely many experts.
problem Maximizing reward by choosing actions sequentially from a set of experts.
method Proposed a variant of Exp4.P for infinitely many experts and a meta-algorithm.
result Proved high-probability upper bound of i l d e O ( i ∗ K + K T ) ilde{\mathcal{O}} \big( i^*K + \sqrt{KT} \big) i l d e O ( i ∗ K + K T ) on regret. RDLI integrates domain logic and context grounding to detect crypto anomalies under scarce labels.
problem Extreme label scarcity and evasion strategies in crypto networks.
method Relational Domain Logic Integration (RDLI) with Retrieval Grounded Context (RGC).
result RDLI outperforms GNN baselines by 28.9% in F1 score under 0.01% label scarcity.
New algorithm reduces expert prediction regret for two experts.
problem Efficient prediction with two experts under fixed time constraints.
method Optimal algorithm based on stochastic calculus techniques.
result Achieves optimal regret of sqrt(T/2π) + O(1) with O(1) per-turn processing time.
The article improves prediction by aggregating Kalman recursions online.
problem Improving expert aggregation in prediction models.
method Using exponential weights and state-space models to aggregate Kalman recursions.
result New algorithms outperform existing methods in Kalman recursion expert aggregation.
System uses conformal prediction to help experts make accurate decisions without understanding when to trust it.
problem Helping experts make accurate decisions in multiclass classification tasks.
method Develops an automated decision support system using conformal prediction to provide precise prediction sets and an efficient search method.
result System improves expert predictions by providing precise prediction sets and forcing experts to predict from these sets.
Improved regret bounds for bandits with fixed expert advice using information theory.
problem Optimizing regret in bandit problems with fixed expert distributions.
method Information-theoretic analysis and KL-divergence measures.
result First regret bounds for EXP4 that can get arbitrarily close to zero under certain conditions.
Bayesian models combine experts with a flexible gating mechanism for complex data.
problem Theoretical properties of Bayesian mixture-of-experts models with softmax gating remain unexplored.
method Investigated asymptotic behavior of posterior distribution for density estimation, parameter estimation, and model selection.
result Established posterior contraction rates for density estimation and parameter estimation, providing insights for practical model design.
Advice-efficient prediction with expert advice (in analogy to label-efficient prediction) is a variant of prediction with expert advice game, where on each round of the game we are allowed to ask for advice of a limited number M M M out of N N N experts. This setting is especially interesting when asking for advice of ever…
Reinforcement learning mimics expert behavior.
problem Learning from expert demonstrations in reinforcement learning.
method Reduction to reinforcement learning with a stationary reward.
result Expert reward can be recovered and imitation learning is bounded.