Unified strategy for efficient data compression and model estimation.
problem Limited interactive exploration and data interaction in linear model development and deployment.
method Conditionally sufficient statistics for optimal data compression and estimation of linear models.
result Linear models can be estimated from compressed data without loss of parameters or covariances.
DriftMoE adapts to concept drifts in data streams efficiently.
problem Adapting to non-stationary data streams with concept drifts.
method Online Mixture-of-Experts (MoE) architecture with co-training framework.
result DriftMoE achieves competitive results with state-of-the-art stream learning adaptive ensembles.
New state-only IL algorithm tackles MDP transition mismatch.
problem Transition dynamics mismatch between expert and imitator MDPs.
method Adversarial state-only IL with two subproblems solved iteratively.
result Effective performance improvement in transition dynamics mismatch scenarios.
New clustering method improves climate data analysis in Lesser Antilles.
problem Inducing undesirable effects in clustering algorithms using Euclidean distance.
method Replacing Euclidean distance with Expert Deviation (ED) based on symmetrized Kullback-Leibler divergence.
result KMS-ED produces more interpretable clusters with better discrimination of daily situations.
Automated machine learning aims to automate the whole process of machine learning, including model configuration. In this paper, we focus on automated hyperparameter optimization (HPO) based on sequential model-based optimization (SMBO). Though conventional SMBO algorithms work well when abundant HPO trials are availab…
Bayesian optimization reduces computational effort in aircraft design optimization.
problem High computational cost in industrial aircraft design optimization.
method Constrained Bayesian optimization (Super Efficient Global Optimization with Mixture of Experts)
result Significant computational efficiency improvements over existing Isight optimizers.
In this work we present Ludwig, a flexible, extensible and easy to use toolbox which allows users to train deep learning models and use them for obtaining predictions without writing code. Ludwig implements a novel approach to deep learning model building based on two main abstractions: data types and declarative confi…
Automated HPO design using Bayesian optimization and benchmarking.
problem Designing effective hyperparameter optimization algorithms is manual and lacks systematic understanding.
method Formalized space of HPO candidates, Bayesian optimization for search, ablation analysis.
result Simple configurations can perform well in HPO, especially with right parameters.
New method reduces labeler costs by aggregating predictions from local classifiers.
problem Reduce labeler costs in multiclass classification.
method Model K K K -class classification using smaller classifiers trained on subsets of tasks. result Near-optimal scheme for designing classifier configurations reduces labeler costs.
Designing rewards for Reinforcement Learning (RL) is challenging because it needs to convey the desired task, be efficient to optimize, and be easy to compute. The latter is particularly problematic when applying RL to robotics, where detecting whether the desired configuration is reached might require considerable sup…
AutoQuant addresses cryptocurrency backtesting fragility by modeling execution costs and improving strategy selection.
problem Fragile backtests of cryptocurrency perpetual futures ignoring microstructure frictions and execution costs.
method Execution-centric framework with Bayesian optimization, double screening, and strict T+1 semantics.
result Fee-only and zero-cost backtests overestimate returns, highlighting the importance of modeling execution costs.
Algorithm reduces audit costs by identifying best service configurations from biased textual evidence.
problem Designing service systems from textual evidence requires accurate selection despite biased automated scoring.
method Developed PP-LUCB algorithm combining LLM scores and selective audits to minimize costs.
result Correctly identified the best model in 40/40 trials with 90% cost reduction.
Auto-CASH uses Deep Q-Network to automatically select machine learning algorithms.
problem Automatic selection of machine learning algorithms and hyperparameters.
method Pre-trained model based on meta-learning using Deep Q-Network.
result Auto-CASH achieves better performance with shorter time compared to classical and state-of-the-art methods.
Early fault detection using instrumented sensor data is one of the promising application areas of machine learning in industrial facilities. However, it is difficult to improve the generalization performance of the trained fault-detection model because of the complex system configuration in the target diagnostic system…
Multi-expert L2D underfits more severely, requiring new methods.
problem Underfitting in multi-expert L2D settings.
method PiCCE (Pick the Confident and Correct Expert), a surrogate-based method.
result PiCCE effectively reduces multi-expert L2D to a single-expert-like problem, resolving underfitting.
Currently, the diagnosis of Autism Spectrum Disorder (ASD) is dependent upon a subjective, time-consuming evaluation of behavioral tests by an expert clinician. Non-invasive functional MRI (fMRI) characterizes brain connectivity and may be used to inform diagnoses and democratize medicine. However, successful construct…
TENP prunes experts and neurons in Mixture-of-Experts models for efficient deployment.
problem Efficient deployment of large language models constrained by static parameter footprint.
method Structured Trapezoidal ExpertNeuron Pruning (TENP) identifies and retains important experts and neurons.
result DeepSeek model achieves 10% better performance on code generation tasks with 40% expert sparsity.
A method to select important experts for Gaussian processes to balance computational efficiency and uncertainty quantification.
problem Balancing computational efficiency and uncertainty quantification in Gaussian processes for big data.
method Using graphical models to select important experts and aggregate their predictions while ensuring uncertainty quantification.
result Substantially reduces computational cost of aggregating dependent experts while ensuring calibrated uncertainty quantification.
With the present revival of interest in bistatic radar systems, research in that area has gained momentum. Given some of the strategic advantages for a bistatic configuration, and tech- nological advances in the past few years, large-scale implementation of the bistatic systems is a scope for the near future. If the bi…
NEST optimizes deep learning training by placing devices efficiently across networks and memory.
problem Inefficient device placement in distributed deep learning leads to high communication and memory overhead.
method NEST uses network-, compute-, and memory-aware dynamic programming to optimize device placement.
result NEST achieves up to 2.43 times higher throughput and better memory efficiency.
We consider the problem of contextual bandits with stochastic experts, which is a variation of the traditional stochastic contextual bandit with experts problem. In our problem setting, we assume access to a class of stochastic experts, where each expert is a conditional distribution over the arms given a context. We p…
Improved time series forecasting with expert loss integration.
problem Enhancing time series forecasting accuracy and efficiency.
method Adaptive Mixture-of-Experts framework with expert-specific loss integration and online learning.
result Significantly improved forecasting accuracy and computational efficiency.
The manual design of analog circuits is a tedious task of parameter tuning that requires hours of work by human experts. In this work, we make a significant step towards a fully automatic design method that is based on deep learning. The method selects the components and their configuration, as well as their numerical …
HS-MoE selects sparse experts using adaptive priors and data-adaptive gating.
problem Sparse expert selection in mixture-of-experts architectures.
method Combines horseshoe prior with input-dependent gating for data-adaptive sparsity.
result Data-adaptive sparsity in expert usage.
NAMEx merges experts using Nash bargaining for improved performance.
problem Sparse Mixture of Experts merging strategies lack a principled weighting mechanism.
method Reinterpreting expert merging through game theory, introducing Nash Merging and complex momentum.
result NAMEx consistently outperforms competing methods across various tasks and system sizes.
Expert augmentation improves hybrid model generalization.
problem Limited generalization of hybrid models outside training distribution.
method Introducing expert augmentation to improve hybrid model performance.
result Expert augmentation improves generalization of hybrid models.
New method calibrates Gaussian product experts for better predictions.
problem Erratic predictions and uncalibrated uncertainty in Gaussian product experts.
method Calibration via tempered softmax and Wasserstein barycenter for predictions.
result Improved predictions with better mean and uncertainty quantification.
Meta-algorithm optimizes nonstochastic bandits with infinitely many experts.
problem Maximizing reward by choosing actions sequentially from a set of experts.
method Proposed a variant of Exp4.P for infinitely many experts and a meta-algorithm.
result Proved high-probability upper bound of i l d e O ( i ∗ K + K T ) ilde{\mathcal{O}} \big( i^*K + \sqrt{KT} \big) i l d e O ( i ∗ K + K T ) on regret. Automated hyperparameter tuning aspires to facilitate the application of machine learning for non-experts. In the literature, different optimization approaches are applied for that purpose. This paper investigates the performance of Differential Evolution for tuning hyperparameters of supervised learning algorithms for…
New algorithm reduces expert prediction regret for two experts.
problem Efficient prediction with two experts under fixed time constraints.
method Optimal algorithm based on stochastic calculus techniques.
result Achieves optimal regret of sqrt(T/2π) + O(1) with O(1) per-turn processing time.
The article improves prediction by aggregating Kalman recursions online.
problem Improving expert aggregation in prediction models.
method Using exponential weights and state-space models to aggregate Kalman recursions.
result New algorithms outperform existing methods in Kalman recursion expert aggregation.
System uses conformal prediction to help experts make accurate decisions without understanding when to trust it.
problem Helping experts make accurate decisions in multiclass classification tasks.
method Develops an automated decision support system using conformal prediction to provide precise prediction sets and an efficient search method.
result System improves expert predictions by providing precise prediction sets and forcing experts to predict from these sets.
Computes fundamental groups of restricted configuration spaces.
problem Understanding the structure of configuration spaces after removing hypersurfaces.
method Fibration over unordered configuration spaces of n − 1 n-1 n − 1 points, computation of fundamental groups. result Fundamental groups of restricted configuration spaces computed in small dimensions.
Improved regret bounds for bandits with fixed expert advice using information theory.
problem Optimizing regret in bandit problems with fixed expert distributions.
method Information-theoretic analysis and KL-divergence measures.
result First regret bounds for EXP4 that can get arbitrarily close to zero under certain conditions.
HAL accelerates the generation of training sets for accurate interatomic potentials.
problem Generating accurate and transferable interatomic potentials is time-consuming and requires expert input.
method HAL framework using a physically motivated sampler with a biasing term to drive high uncertainty configurations.
result HAL-generated training databases for alloys and polymers predict macroscopic properties with high accuracy.
Bayesian models combine experts with a flexible gating mechanism for complex data.
problem Theoretical properties of Bayesian mixture-of-experts models with softmax gating remain unexplored.
method Investigated asymptotic behavior of posterior distribution for density estimation, parameter estimation, and model selection.
result Established posterior contraction rates for density estimation and parameter estimation, providing insights for practical model design.
Advice-efficient prediction with expert advice (in analogy to label-efficient prediction) is a variant of prediction with expert advice game, where on each round of the game we are allowed to ask for advice of a limited number M M M out of N N N experts. This setting is especially interesting when asking for advice of ever…
Reinforcement learning mimics expert behavior.
problem Learning from expert demonstrations in reinforcement learning.
method Reduction to reinforcement learning with a stationary reward.
result Expert reward can be recovered and imitation learning is bounded.
Adaptive Bayesian learning aggregates experts to improve performance.
problem Bayesian online learning's performance depends on inferential choices.
method Treat Bayesian update rules as experts and aggregate them based on sequential predictive losses.
result The aggregate competes with the best expert in hindsight at a low aggregation cost.
New method aggregates Gaussian experts by detecting conditional independence violations.
problem Aggregation of dependent Gaussian experts leads to sub-optimal solutions.
method Uses Gaussian graphical model to detect and correct conditional independence violations.
result Improves aggregation of Gaussian experts, outperforming SOTA DGP approaches.
Improved HMoE models using Laplace gating function enhance expert specialization and performance.
problem Improving performance of hierarchical mixture of experts models.
method Used Laplace gating function instead of Softmax in hierarchical mixture of experts models.
result Laplace gating function accelerates expert convergence and enhances specialization.
We present a new approach for learning compact and intuitive distributed representations with binary encoding. Rather than summing up expert votes as in products of experts, we employ for each variable the opinion of the most reliable expert. Data points are hence explained through a partitioning of the variables into …
ARGUE combines expert networks for anomaly detection.
problem Anomaly detection without labeled data.
method Gated mixture-of-experts architecture combining expert networks.
result Prior knowledge about normal data distribution is valuable.
Online L2D algorithm for multiclass classification with varying experts.
problem Handling streaming data, changing expert availability, and shifting expert distribution.
method First online L2D algorithm with O ( ( n + n e ) T 2 / 3 ) O((n+n_e)T^{2/3}) O (( n + n e ) T 2/3 ) and O ( ( n + n e ) T ) O((n+n_e)\sqrt{T}) O (( n + n e ) T ) regret guarantees. result Effective extension of standard L2D to settings with varying expert availability and reliability.
A new method for multi-expert learning-to-defer avoids optimization issues.
problem Optimization issues in multi-expert learning-to-defer systems.
method A decoupled surrogate with a softmax classifier head and independent sigmoid heads per expert.
result First multi-expert L2D guarantee with a constant not growing with the expert pool.
Sigmoid gating is more sample efficient than softmax in mixture of experts.
problem Softmax gating leads to unnecessary competition among experts, causing representation collapse.
method Theoretical analysis of a regression framework with mixture of experts, identifying identifiability conditions and convergence rates.
result Sigmoid gating requires fewer samples to achieve the same expert estimation error as softmax gating.
YRC-Bench benchmarks AI agents learning to collaborate with experts.
problem Learning to recognize when to consult an expert in novel situations.
method Validation strategy and proposer-validator decomposition.
result Insights for future AI safety research.
Budgeted deferral framework reduces expert query costs in machine learning.
problem High cost of querying all experts in deferral algorithms.
method Introduces budgeted deferral framework for selective expert querying.
result Empirical results show substantial reduction in training costs without sacrificing accuracy.