Greedy policy achieves good results for adaptive submodular problems.
problem Sequential decision making with adaptive stochastic optimization.
method Adaptive submodularity ratio to analyze greedy policy performance.
result Greedy policy achieves approximation guarantees for a broader class of problems.
Optimizes biomolecular simulations by ranking adaptive sampling policies.
problem Efficiently sampling biomolecular systems to capture complex dynamical behaviors.
method Metric-driven ranking of adaptive sampling policies to identify the optimal policy for each round.
result Different adaptive sampling policies lead to faster convergence and improved sampling performance.
New model-based methods adapt pre-trained policies to unseen environments efficiently.
problem High sample complexity in reinforcement learning limits practical applications.
method Combines online learning and adaptive control to adapt policies in unseen environments.
result Proves policies can quickly recover trajectories from source to target environments.
Paper tackles adaptive submodularity in sequential decision making.
problem Maximizing adaptive submodular functions with limited adaptive rounds.
method Proposes an efficient semi-adaptive policy with logarithmic adaptive rounds.
result Achieves an almost tight 1 − 1 / e − ε 1-1/e-ε 1 − 1/ e − ε approximation guarantee. Adaptive reduction scheme approximates optimal policy in regularized MDPs.
problem Finding near optimal policy in regularized MDPs with biased solutions.
method Adaptive reduction of regularization parameter λ to approximate optimal policy.
result Iteration complexity reduced for obtaining ε-optimal policy.
Accelerates policy optimization in RL with optimistic and adaptive updates.
problem Improving policy optimization methods in reinforcement learning.
method Integrates foresight into policy improvement step via optimistic and adaptive updates.
result Designs an optimistic policy gradient algorithm, adaptive via meta-gradient learning.
Adaptive policies for multi-agent RL improve adaptiveness in changing environments.
problem Non-stationarity in multi-agent reinforcement learning where other agents may alter their policies.
method Train multiple adaptive policies for each agent and a policy predictor to select the best policy at execution time.
result Agents trained with our method outperform state-of-the-art methods in all tested environments.
FLAP adapts policies quickly to new tasks using shared linear representations.
problem Adapting policies to new tasks efficiently and effectively.
method FLAP uses a shared linear representation and a separate adapter network for quick adaptation.
result FLAP achieves up to 8X faster adaptation and significantly better performance on out-of-distribution tasks.
Adaptive exploration scheme for evaluating multiple policies with different rewards.
problem Online multi-reward multi-policy evaluation.
method Adapted ( ε , δ ) (ε,δ) ( ε , δ ) -PAC perspective and MR-NaS exploration scheme to minimize sample complexity. result Demonstrated effectiveness of adaptive exploration in tabular domains.
Offline RL policies should adapt to unknown aspects of the environment.
problem Uncertainty in offline RL datasets leads to suboptimal policies.
method Adaptive policies that consider all transitions seen so far, solving an implicit POMDP.
result Optimal adaptive policies improve offline RL performance.
Adapting policy learning for data collected from evolving systems.
problem Challenges in learning optimal policies from adaptively collected data.
method Proposes an algorithm based on generalized augmented inverse propensity weighted (AIPW) estimators to control worst-case estimation variance.
result Achieves minimax rate optimal regret guarantees even with diminishing exploration.
AdaRL improves robust RL by adaptively adjusting policy complexity.
problem Handling epistemic uncertainty in environment dynamics.
method Bi-level optimization framework with adaptive rank adjustment.
result AdaRL outperforms existing methods on MuJoCo benchmarks.
Paper compares UCB policy to new adaptive RL methods.
problem Optimal adaptive policies for Markovian decision processes with unknown transition probabilities.
method Compared UCB policy with MDP-Deterministic Minimum Empirical Divergence and Posterior sampling methods.
result MDP-DMED outperforms UCB in the tested RL scenarios.
Adapts model-based advice to stabilize black-box policies for nonlinear control.
problem Stabilizing machine-learned policies for nonlinear control with limited model information.
method Proposes an adaptive λ λ λ -confident policy to combine black-box and model-based advice. result Proves the stability of the adaptive λ λ λ -confident policy and its competitive ratio. Adaptive TD learning reduces bias in policy evaluation by switching between TD and MC methods.
problem Achieving accurate policy evaluation with Temporal Difference (TD) learning in the presence of state-specific uncertainty.
method Adaptive switching between TD and Monte Carlo (MC) methods, using learned confidence intervals to detect and mitigate bias.
result The proposed adaptive algorithm outperforms existing methods in policy evaluation tasks.
MAME models a separate exploration policy for faster adaptation.
problem Efficient exploration strategies for quick task adaptation in meta-reinforcement learning.
method Explicitly models a separate exploration policy for task distribution, using self-supervised or supervised learning objectives for adaptation.
result Superior performance compared to prior works in meta-reinforcement learning.
Very recently proximal policy optimization (PPO) algorithms have been proposed as first-order optimization methods for effective reinforcement learning. While PPO is inspired by the same learning theory that justifies trust region policy optimization (TRPO), PPO substantially simplifies algorithm design and improves da…
Step-DAD improves BED by periodically updating a design policy during experiments.
problem Improving flexibility and robustness in Bayesian experimental design.
method Semi-amortized, policy-based approach that updates a design policy during data collection.
result Consistently superior decision-making and robustness compared to current BED methods.
ProMP improves Meta-RL by better handling credit assignment.
problem Poor credit assignment in Meta-RL leads to suboptimal performance.
method Develops a novel meta-learning algorithm controlling pre-adaptation and adapted policies.
result Significantly improves sample-efficiency and asymptotic performance.
Paper proposes a new DR estimator for adaptive experiments with improved performance.
problem Improving policy evaluation in adaptive experiments with dependent samples.
method Adaptive-fitting variant of sample-splitting for non-Donsker nuisance estimators.
result Proposed DR estimator shows better performance than other estimators with dependent samples.
Method adapts policies for new environments efficiently.
problem Difficulties in transferring reinforcement learning policies to new environments.
method Variational Policy Embedding (VPE) learns latent variables and a master policy.
result Policies can quickly adapt to new environments in latent space.
We consider the problem of group testing with sum observations and noiseless answers, in which we aim to locate multiple objects by querying the number of objects in each of a sequence of chosen sets. We study a probabilistic setting with entropy loss, in which we assume a joint Bayesian prior density on the locations …
Meta-reinforcement learning improves fault-adaptive control efficiency.
problem Adaptive control under abrupt system faults with strict time constraints.
method Model-agnostic meta learning (MAML) with a fault library of prior policies.
result Improved sample efficiency and quick adaptation to new faults.
ReCAP adapts to dynamic financial markets by segmenting and combining policy vectors.
problem Inefficient traditional PM approaches in non-stationary financial markets.
method Integrates continual learning into PM, segmenting regimes and adapting policies.
result Consistently outperforms baselines in real-world financial datasets.
Paper proposes a new method to optimize robot body structure and control policy.
problem Optimizing robot body structure and control policy in a coupled manner.
method Revisits co-design problem as a Stackelberg game, incorporating control adaptation dynamics.
result Stackelberg PPO outperforms standard PPO in stability and performance.
Study on learning strategies in adaptive Markov games with policy regret as metric.
problem Learning in dynamic Markov games with adaptive opponents is challenging.
method Introduced policy regret as a new learning metric and developed algorithms for consistent adaptive adversaries.
result Achieved T \sqrt{T} T policy regret against certain adaptive adversaries. LALR adapts learning rate for faster convergence in regression and neural nets.
problem Finding optimal learning rates for faster convergence in regression and neural networks.
method Lipschitz continuity theory applied to Mean Absolute Error and Quantile loss functions.
result Adaptive learning rate policy enables up to 20x faster convergence.
AdaRL adapts quickly to new environments with minimal data.
problem Quickly adapting to new environments in reinforcement learning.
method AdaRL uses a parsimonious graphical representation to encode changes across domains.
result AdaRL can efficiently adapt policies to target domains with few samples.
Adapts GRPO for off-policy RL, improving reward.
problem Improving training stability and efficiency in RL.
method Adapts GRPO to off-policy setting, uses clipped surrogate objectives.
result Off-policy GRPO outperforms on-policy GRPO in empirical tests.
Develops adaptive framework for estimating survival effects with censoring.
problem Estimating causal effects in survival data with censoring.
method Derives semiparametric efficiency bound, proposes efficiency-optimal allocation policy, and develops Adaptive Survival Estimator (ASE).
result ASE achieves asymptotic normality via martingale central limit theorem and demonstrates efficiency gains over uniform randomization.
Study optimizes experimental design for best treatment arm identification.
problem Identifying the best treatment arm given contextual information.
method Adaptive Sampling-Policy Learning (PLAS) strategy for minimax rate optimality.
result PLAS strategy achieves minimax rate optimality in expected simple regret.
Adaptive inference for M M M -estimators in bandit data with model misspecification.
problem Challenges in off-policy inference for adaptively collected bandit data with a misspecified model.
method A novel approach to define a projected solution over a stationary evaluation policy, stabilizing variance with flexible methods.
result Valid inference for M M M -estimators in adaptive settings, even with unstable treatment policies. The paper improves DR estimators for off-policy evaluation in contextual bandits.
problem Challenges in evaluating policies with adaptively collected data.
method Adaptive weighting to control variance in DR estimators.
result Improved t-statistic based on the estimator is asymptotically normal.
New methods for anytime-valid off-policy inference in contextual bandits.
problem Estimating properties of hypothetical policies in adaptive experiments.
method Modern martingale techniques for comprehensive OPE inference.
result Valid anytime inference for off-policy mean reward values and entire reward distributions.
Deep reinforcement learning finds optimal learning policies for adaptive systems.
problem Finding individualized learning plans for learners with unknown latent traits.
method Formulated as a Markov decision process, applied deep Q-learning with a transition model estimator.
result The algorithm efficiently discovers optimal learning policies with small data sets.
HiPPO adapts skills and higher-level policies together for better transfer in hierarchical RL.
problem Sub-optimality in skill transfer when lower-level skills are fixed.
method HiPPO: a novel hierarchical policy gradient method that trains all levels of the hierarchy jointly.
result Improved robustness of skills to environment changes through training time-abstractions.
New guarantees for ERM with adaptively collected data.
problem Failure of ERM guarantees with adaptively collected data.
method Importance sampling weighted ERM algorithm with maximal inequality.
result First generalization guarantees and fast convergence rates for adaptively collected data.
AES improves policy gradient performance by adaptively selecting experience.
problem High variance in gradient estimators from past trajectories.
method AES learns an adaptive sampling distribution to minimise gradient variance.
result AES leads to significantly improved performance in continuous control tasks.
New algorithm learns optimal policy with multi-step lookahead information.
problem Learning optimal policy in reinforcement learning with multi-step lookahead information is NP-hard.
method Adaptive batching policies that process lookahead in state-dependent chunks.
result Order-optimal regret bounds up to a constant factor of lookahead horizon.
Paper optimizes Bayesian optimization for complex functions with macro-actions.
problem Optimizing complex, highly uncertain functions efficiently.
method Generalized GP upper confidence bound with macro-actions for scalable lookahead.
result Asymptotically optimal anytime variant of epsilon-Macro-GPO policy.
IntelligentPooling learns personalized mHealth policies from limited data.
problem Challenges in personalizing mHealth policies with limited user data and noise.
method Adaptive use of other users' data to learn personalized policies.
result Achieves 26% lower regret compared to state-of-the-art approaches.
Estimators computed from adaptively collected data do not behave like their non-adaptive brethren. Rather, the sequential dependence of the collection policy can lead to severe distributional biases that persist even in the infinite data limit. We develop a general method -- W \mathbf{W} W -decorrelation -- for transformi…
MB-MPO meta-learns a policy to adapt quickly to model ensembles, improving robustness and performance.
problem Challenges in learning accurate dynamics models for model-based reinforcement learning.
method Model-Based Meta-Policy-Optimization (MB-MPO) using an ensemble of learned dynamic models.
result MB-MPO achieves asymptotic performance similar to model-free methods with less experience and robustness to model imperfections.
We consider a multi-armed bandit problem in a setting where each arm produces a noisy reward realization which depends on an observable random covariate. As opposed to the traditional static multi-armed bandit problem, this setting allows for dynamically changing rewards that better describe applications where side inf…
Paper presents a method for efficient robot adaptation using fine-tuning.
problem Continuous adaptation of robot learning systems in real-world scenarios.
method Fine-tuning previously learned policies using off-policy reinforcement learning.
result Fine-tuning leads to substantial performance gains and adaptation to new conditions.
iDAD uses neural networks to quickly adapt experiments without likelihoods.
problem Performing adaptive experiments in real-time with implicit models.
method iDAD learns a design policy network upfront to make quick design decisions.
result iDAD can make design decisions in milliseconds, unlike traditional BOED approaches.
New method optimizes policies in non-stationary environments.
problem Optimizing policies in non-stationary, context-dependent environments.
method Two-phase approach: offline learning and online adaptation.
result Our method outperforms existing approaches in both synthetic and real-world datasets.
The paper sets bounds on how much regret is unavoidable in adaptive LQR with unknown B-matrix.
problem Understanding the limits of adaptive LQR with unknown B-matrix.
method Local asymptotic minimax regret lower bounds using van Trees' inequality and Bellman error representation.
result Logarithmic regret is impossible if the parametrization induces an uninformative optimal policy.