Optimizes biomolecular simulations by ranking adaptive sampling policies.
problem Efficiently sampling biomolecular systems to capture complex dynamical behaviors.
method Metric-driven ranking of adaptive sampling policies to identify the optimal policy for each round.
result Different adaptive sampling policies lead to faster convergence and improved sampling performance.
New model-based methods adapt pre-trained policies to unseen environments efficiently.
problem High sample complexity in reinforcement learning limits practical applications.
method Combines online learning and adaptive control to adapt policies in unseen environments.
result Proves policies can quickly recover trajectories from source to target environments.
We propose a new concept named adaptive submodularity ratio to study the greedy policy for sequential decision making. While the greedy policy is known to perform well for a wide variety of adaptive stochastic optimization problems in practice, its theoretical properties have been analyzed only for a limited class of p…
Paper tackles adaptive submodularity in sequential decision making.
problem Maximizing adaptive submodular functions with limited adaptive rounds.
method Proposes an efficient semi-adaptive policy with logarithmic adaptive rounds.
result Achieves an almost tight 1 − 1 / e − ε 1-1/e-ε 1 − 1/ e − ε approximation guarantee. Adaptive reduction scheme approximates optimal policy in regularized MDPs.
problem Finding near optimal policy in regularized MDPs with biased solutions.
method Adaptive reduction of regularization parameter λ to approximate optimal policy.
result Iteration complexity reduced for obtaining ε-optimal policy.
Accelerates policy optimization in RL with optimistic and adaptive updates.
problem Improving policy optimization methods in reinforcement learning.
method Integrates foresight into policy improvement step via optimistic and adaptive updates.
result Designs an optimistic policy gradient algorithm, adaptive via meta-gradient learning.
Adaptive policies for multi-agent RL improve adaptiveness in changing environments.
problem Non-stationarity in multi-agent reinforcement learning where other agents may alter their policies.
method Train multiple adaptive policies for each agent and a policy predictor to select the best policy at execution time.
result Agents trained with our method outperform state-of-the-art methods in all tested environments.
FLAP adapts policies quickly to new tasks using shared linear representations.
problem Adapting policies to new tasks efficiently and effectively.
method FLAP uses a shared linear representation and a separate adapter network for quick adaptation.
result FLAP achieves up to 8X faster adaptation and significantly better performance on out-of-distribution tasks.
Adaptive exploration scheme for evaluating multiple policies with different rewards.
problem Online multi-reward multi-policy evaluation.
method Adapted ( ε , δ ) (ε,δ) ( ε , δ ) -PAC perspective and MR-NaS exploration scheme to minimize sample complexity. result Demonstrated effectiveness of adaptive exploration in tabular domains.
Offline RL policies should adapt to unknown aspects of the environment.
problem Uncertainty in offline RL datasets leads to suboptimal policies.
method Adaptive policies that consider all transitions seen so far, solving an implicit POMDP.
result Optimal adaptive policies improve offline RL performance.
Adapting policy learning for data collected from evolving systems.
problem Challenges in learning optimal policies from adaptively collected data.
method Proposes an algorithm based on generalized augmented inverse propensity weighted (AIPW) estimators to control worst-case estimation variance.
result Achieves minimax rate optimal regret guarantees even with diminishing exploration.
AdaRL improves robust RL by adaptively adjusting policy complexity.
problem Handling epistemic uncertainty in environment dynamics.
method Bi-level optimization framework with adaptive rank adjustment.
result AdaRL outperforms existing methods on MuJoCo benchmarks.
Adapts model-based advice to stabilize black-box policies for nonlinear control.
problem Stabilizing machine-learned policies for nonlinear control with limited model information.
method Proposes an adaptive λ λ λ -confident policy to combine black-box and model-based advice. result Proves the stability of the adaptive λ λ λ -confident policy and its competitive ratio. MAME models a separate exploration policy for faster adaptation.
problem Efficient exploration strategies for quick task adaptation in meta-reinforcement learning.
method Explicitly models a separate exploration policy for task distribution, using self-supervised or supervised learning objectives for adaptation.
result Superior performance compared to prior works in meta-reinforcement learning.
Very recently proximal policy optimization (PPO) algorithms have been proposed as first-order optimization methods for effective reinforcement learning. While PPO is inspired by the same learning theory that justifies trust region policy optimization (TRPO), PPO substantially simplifies algorithm design and improves da…
Step-DAD improves BED by periodically updating a design policy during experiments.
problem Improving flexibility and robustness in Bayesian experimental design.
method Semi-amortized, policy-based approach that updates a design policy during data collection.
result Consistently superior decision-making and robustness compared to current BED methods.
Paper proposes a new DR estimator for adaptive experiments with improved performance.
problem Improving policy evaluation in adaptive experiments with dependent samples.
method Adaptive-fitting variant of sample-splitting for non-Donsker nuisance estimators.
result Proposed DR estimator shows better performance than other estimators with dependent samples.
We consider the problem of group testing with sum observations and noiseless answers, in which we aim to locate multiple objects by querying the number of objects in each of a sequence of chosen sets. We study a probabilistic setting with entropy loss, in which we assume a joint Bayesian prior density on the locations …
Meta-reinforcement learning improves fault-adaptive control efficiency.
problem Adaptive control under abrupt system faults with strict time constraints.
method Model-agnostic meta learning (MAML) with a fault library of prior policies.
result Improved sample efficiency and quick adaptation to new faults.
Credit assignment in Meta-reinforcement learning (Meta-RL) is still poorly understood. Existing methods either neglect credit assignment to pre-adaptation behavior or implement it naively. This leads to poor sample-efficiency during meta-training as well as ineffective task identification strategies. This paper provide…
ReCAP adapts to dynamic financial markets by segmenting and combining policy vectors.
problem Inefficient traditional PM approaches in non-stationary financial markets.
method Integrates continual learning into PM, segmenting regimes and adapting policies.
result Consistently outperforms baselines in real-world financial datasets.
Paper proposes a new method to optimize robot body structure and control policy.
problem Optimizing robot body structure and control policy in a coupled manner.
method Revisits co-design problem as a Stackelberg game, incorporating control adaptation dynamics.
result Stackelberg PPO outperforms standard PPO in stability and performance.
Study on learning strategies in adaptive Markov games with policy regret as metric.
problem Learning in dynamic Markov games with adaptive opponents is challenging.
method Introduced policy regret as a new learning metric and developed algorithms for consistent adaptive adversaries.
result Achieved T \sqrt{T} T policy regret against certain adaptive adversaries. LALR adapts learning rate for faster convergence in regression and neural nets.
problem Finding optimal learning rates for faster convergence in regression and neural networks.
method Lipschitz continuity theory applied to Mean Absolute Error and Quantile loss functions.
result Adaptive learning rate policy enables up to 20x faster convergence.
AdaRL adapts quickly to new environments with minimal data.
problem Quickly adapting to new environments in reinforcement learning.
method AdaRL uses a parsimonious graphical representation to encode changes across domains.
result AdaRL can efficiently adapt policies to target domains with few samples.
Adapts GRPO for off-policy RL, improving reward.
problem Improving training stability and efficiency in RL.
method Adapts GRPO to off-policy setting, uses clipped surrogate objectives.
result Off-policy GRPO outperforms on-policy GRPO in empirical tests.
Develops adaptive framework for estimating survival effects with censoring.
problem Estimating causal effects in survival data with censoring.
method Derives semiparametric efficiency bound, proposes efficiency-optimal allocation policy, and develops Adaptive Survival Estimator (ASE).
result ASE achieves asymptotic normality via martingale central limit theorem and demonstrates efficiency gains over uniform randomization.
Study optimizes experimental design for best treatment arm identification.
problem Identifying the best treatment arm given contextual information.
method Adaptive Sampling-Policy Learning (PLAS) strategy for minimax rate optimality.
result PLAS strategy achieves minimax rate optimality in expected simple regret.
Adaptive inference for M M M -estimators in bandit data with model misspecification.
problem Challenges in off-policy inference for adaptively collected bandit data with a misspecified model.
method A novel approach to define a projected solution over a stationary evaluation policy, stabilizing variance with flexible methods.
result Valid inference for M M M -estimators in adaptive settings, even with unstable treatment policies. The paper improves DR estimators for off-policy evaluation in contextual bandits.
problem Challenges in evaluating policies with adaptively collected data.
method Adaptive weighting to control variance in DR estimators.
result Improved t-statistic based on the estimator is asymptotically normal.
New methods for anytime-valid off-policy inference in contextual bandits.
problem Estimating properties of hypothetical policies in adaptive experiments.
method Modern martingale techniques for comprehensive OPE inference.
result Valid anytime inference for off-policy mean reward values and entire reward distributions.
Deep reinforcement learning finds optimal learning policies for adaptive systems.
problem Finding individualized learning plans for learners with unknown latent traits.
method Formulated as a Markov decision process, applied deep Q-learning with a transition model estimator.
result The algorithm efficiently discovers optimal learning policies with small data sets.
New guarantees for ERM with adaptively collected data.
problem Failure of ERM guarantees with adaptively collected data.
method Importance sampling weighted ERM algorithm with maximal inequality.
result First generalization guarantees and fast convergence rates for adaptively collected data.
In this paper we consider the basic version of Reinforcement Learning (RL) that involves computing optimal data driven (adaptive) policies for Markovian decision process with unknown transition probabilities. We provide a brief survey of the state of the art of the area and we compare the performance of the classic UCB…
We consider the core reinforcement-learning problem of on-policy value function approximation from a batch of trajectory data, and focus on various issues of Temporal Difference (TD) learning and Monte Carlo (MC) policy evaluation. The two methods are known to achieve complementary bias-variance trade-off properties, w…
AES improves policy gradient performance by adaptively selecting experience.
problem High variance in gradient estimators from past trajectories.
method AES learns an adaptive sampling distribution to minimise gradient variance.
result AES leads to significantly improved performance in continuous control tasks.
New algorithm learns optimal policy with multi-step lookahead information.
problem Learning optimal policy in reinforcement learning with multi-step lookahead information is NP-hard.
method Adaptive batching policies that process lookahead in state-dependent chunks.
result Order-optimal regret bounds up to a constant factor of lookahead horizon.
In real-world machine learning applications, there is a cost associated with sampling of different features. Budgeted learning can be used to select which feature-values to acquire from each instance in a dataset, such that the best model is induced under a given constraint. However, this approach is not possible in th…
Paper optimizes Bayesian optimization for complex functions with macro-actions.
problem Optimizing complex, highly uncertain functions efficiently.
method Generalized GP upper confidence bound with macro-actions for scalable lookahead.
result Asymptotically optimal anytime variant of epsilon-Macro-GPO policy.
Estimators computed from adaptively collected data do not behave like their non-adaptive brethren. Rather, the sequential dependence of the collection policy can lead to severe distributional biases that persist even in the infinite data limit. We develop a general method -- W \mathbf{W} W -decorrelation -- for transformi…
IntelligentPooling learns personalized mHealth policies from limited data.
problem Challenges in personalizing mHealth policies with limited user data and noise.
method Adaptive use of other users' data to learn personalized policies.
result Achieves 26% lower regret compared to state-of-the-art approaches.
We consider a multi-armed bandit problem in a setting where each arm produces a noisy reward realization which depends on an observable random covariate. As opposed to the traditional static multi-armed bandit problem, this setting allows for dynamically changing rewards that better describe applications where side inf…
iDAD uses neural networks to quickly adapt experiments without likelihoods.
problem Performing adaptive experiments in real-time with implicit models.
method iDAD learns a design policy network upfront to make quick design decisions.
result iDAD can make design decisions in milliseconds, unlike traditional BOED approaches.
Paper presents a method for efficient robot adaptation using fine-tuning.
problem Continuous adaptation of robot learning systems in real-world scenarios.
method Fine-tuning previously learned policies using off-policy reinforcement learning.
result Fine-tuning leads to substantial performance gains and adaptation to new conditions.
New method optimizes policies in non-stationary environments.
problem Optimizing policies in non-stationary, context-dependent environments.
method Two-phase approach: offline learning and online adaptation.
result Our method outperforms existing approaches in both synthetic and real-world datasets.
The paper sets bounds on how much regret is unavoidable in adaptive LQR with unknown B-matrix.
problem Understanding the limits of adaptive LQR with unknown B-matrix.
method Local asymptotic minimax regret lower bounds using van Trees' inequality and Bellman error representation.
result Logarithmic regret is impossible if the parametrization induces an uninformative optimal policy.
We study contextual bandit learning with an abstract policy class and continuous action space. We obtain two qualitatively different regret bounds: one competes with a smoothed version of the policy class under no continuity assumptions, while the other requires standard Lipschitz assumptions. Both bounds exhibit data-…
Reinforcement Learning methods are capable of solving complex problems, but resulting policies might perform poorly in environments that are even slightly different. In robotics especially, training and deployment conditions often vary and data collection is expensive, making retraining undesirable. Simulation training…