New algorithm estimates treatment effects for more efficient contextual bandits.
problem Contextual bandits struggle with action-independent reward redundancies.
method Reduces contextual bandits to heterogeneous treatment effect estimation.
result Heterogeneous treatment effect estimation leads to more efficient model estimation.
Algorithm improves transfer learning by inferring successor maps.
problem Machine learning challenges in multi-task scenarios.
method Combining factorized representations and nonparametric memory-based approaches.
result Improves transfer capabilities and outperforms other algorithms.
Paper proposes Imitative Models combining IL and planning for flexible goal achievement.
problem Difficult to direct IL to arbitrary goals and specify reward functions for goal-directed planning.
method Imitative Models are probabilistic predictive models that plan interpretable expert-like trajectories.
result Imitative Models outperform IL and planning approaches in a dynamic autonomous driving task.
Unified framework for human-like decision making in various sequential tasks.
problem Real-life decision-making involves diverse strategies leading to similar outcomes.
method Two-stream reward processing mechanism for flexible and unified models.
result Framework unified MAB, CB, and RL with comparable performance.
Develops a flexible batched experimentation framework for limited adaptivity.
problem Challenges of continual reallocation in bandit algorithms with delayed feedback.
method Computational framework leveraging Gaussian sequential experiment and dynamic programming.
result Improves statistical power over standard methods, even compared to Bayesian bandit algorithms.
Flexible algorithms for maximizing rewards in structured bandits.
problem Reward maximization in structured stochastic multi-armed bandit problems.
method Asymptotically optimal algorithms using iterative saddle-point solvers.
result Achieves optimal performance with minimal computational burden.
The paradox of the energy transition is that the low marginal costs of new renewable energy sources (RES) drag electricity prices down and discourage investments in flexible productions that are needed to compensate for the lack of dispatchability of the new RES. The energy transition thus discourages the investments t…
Enhances reward specification in RL with a novel language-based approach.
problem Reward specification in RL can lead to unintended, potentially harmful behaviours.
method Developed a novel class of language-based Reward Machines using RML's built-in memory.
result Can specify non-regular, non-Markovian reward functions for complex tasks.
A new method for exploration in reinforcement learning improves performance on Atari games.
problem Improving exploration in reinforcement learning algorithms, especially for complex games.
method Random Network Distillation (RND) bonus combined with flexible reward combination.
result Significant progress on hard exploration Atari games, including Montezuma's Revenge.
Paper establishes DRL for high-dimensional rewards.
problem Intractable reinforcement learning with high-dimensional rewards.
method Theoretical foundations and a novel DRL algorithm.
result Bellman operator contraction in high-dimensional spaces.
New method tackles high-dimensional contextual bandits with flexible kernel models.
problem Maximizing rewards in decision-making scenarios with many features.
method Introduces stochastic assumptions and no-regret learning for Gaussian kernels.
result Achieves no-regret learning even with feature dimensions growing with samples.
A new RL framework learns flexible point process models from event data.
problem Learning flexible models for complex, ordered event data in continuous time.
method Inspired by reinforcement learning, a policy is parameterized as a recurrent neural network to mimic the observed event distribution.
result The proposed RL framework efficiently learns flexible point process models and performs well in both synthetic and real data.
Bayesian nonparametric models improve multi-armed bandit performance with uncertain rewards.
problem Reward model uncertainty in multi-armed bandits.
method Bayesian nonparametric Gaussian mixture models for flexible reward density estimation.
result Achieves successful regret performance with asymptotic regret bound.
New RL algorithm tackles non-stationary environments with flexible policy updates.
problem Non-stationary reinforcement learning with time-varying rewards and transition probabilities.
method Model-free policy-based algorithm NS-NAC with restart-based exploration and dynamic learning rates.
result Dynamic regret of i l d e O ( ∣ S ∣ 1 / 2 ∣ A ∣ 1 / 2 Δ T 1 / 6 T 5 / 6 ) ilde{\mathscr O}(|S|^{1/2}|A|^{1/2}Δ_T^{1/6}T^{5/6}) i l d e O ( ∣ S ∣ 1/2 ∣ A ∣ 1/2 Δ T 1/6 T 5/6 ) for both algorithms. A two-stream reinforcement learning model improves decision-making across human and neuropsychiatric studies.
problem Improving reinforcement learning models to better simulate human decision-making and neuropsychiatric conditions.
method Proposes a two-stream reinforcement learning model that processes positive and negative rewards and incorporates reward-processing biases.
result The two-stream model outperforms standard Q-learning and SARSA methods on various tasks and datasets.
RoME optimizes mobile health interventions by modeling user and time-specific effects.
problem Challenges in optimizing mobile health interventions due to participant heterogeneity, nonstationarity, and nonlinear relationships.
method RoME uses a Robust Mixed-Effects contextual bandit algorithm with random effects, network cohesion penalties, and debiased machine learning.
result RoME achieves robust regret bounds even with complex baseline rewards, demonstrating superior performance in simulations and studies.
SDQL uses modular deep Q networks to efficiently learn multi-stage optimal control tasks.
problem Training complex deep reinforcement learning models for multi-stage control tasks is inefficient and unstable.
method Stacked Deep Q Learning (SDQL) with modular Q networks and backward training.
result SDQL efficiently learns optimal control policies for multi-stage tasks with high-dimensional state and action spaces.
A new model-free algorithm achieves near-optimal regret for infinite-horizon MDPs.
problem Model-free reinforcement learning for infinite-horizon average-reward MDPs.
method Exploration Enhanced Q-learning (EE-QL) for weakly communicating MDPs.
result Achieves O ( T ) O(\sqrt{T}) O ( T ) regret bound for general weakly communicating MDPs. New method uses SMC for Bayesian MABs with time-varying, nonlinear rewards.
problem Learning optimal actions in dynamic, non-stationary environments.
method Sequential Monte Carlo (SMC) for estimating statistics of unknown reward distributions.
result SMC-based Bayesian MAB policies achieve good regret performance in non-stationary, nonlinear reward scenarios.
Active inference enhances RL by balancing exploration and exploitation.
problem Traditional RL's balance between exploration and exploitation is often suboptimal.
method Developed a new decision-making objective based on active inference.
result The new algorithm successfully balances exploration and exploitation on various RL benchmarks.
EBIL simplifies IL by estimating expert energy as reward, achieving effective performance.
problem Recovering optimal policy from expert demonstrations without reward signals.
method EBIL uses a two-stage solution: first estimating expert energy as reward, then learning policy.
result EBIL achieves effective performance and interpretable reward signals.
A new algorithm balances global reward and group constraints in federated multi-armed bandits.
problem Maximizing global reward while protecting client privacy in federated learning.
method Combinatorial contextual bandit with group constraints, using a two-output Gaussian process.
result TCGP-UCB incurs low regret, balancing super arm reward and group reward constraints.
New approach learns human planning algorithms for reward inference.
problem Learning reward functions from human demonstrations with biases.
method Data-driven approach to learn planning algorithms directly from demonstrations.
result Mixed results: better reward inference but at cost of differentiability.
This paper presents a novel nonmyopic adaptive Gaussian process planning (GPP) framework endowed with a general class of Lipschitz continuous reward functions that can unify some active learning/sensing and Bayesian optimization criteria and offer practitioners some flexibility to specify their desired choices for defi…
The paper proposes biased advantage estimates for reinforcement learning, improving efficiency and performance.
problem Estimating advantages in reinforcement learning algorithms.
method A family of estimates based on order statistics over the path ensemble.
result Biased estimates can significantly benefit reinforcement learning, especially in environments with sparse rewards or critical actions.
A RL framework selects features to balance bias and accuracy dynamically.
problem Bias in automated feature selection when predictors are correlated.
method Multi-component reward function with policy gradient for dynamic regularization and bias mitigation.
result Model balances fairness and accuracy during training.
ACE improves GFlowNet exploration efficiency by balancing complementary search strategies.
problem Efficient exploration of diverse high-probability regions in GFlowNets.
method Adaptive Complementary Exploration (ACE) trains a separate GFlowNet to search underexplored regions.
result Significantly improves approximation accuracy and diverse state discovery.
New L 1 L_1 L 1 -Coverage objective simplifies exploration in reinforcement learning.
problem Challenges in exploration for high-dimensional domains.
method Introduces L 1 L_1 L 1 -Coverage objective to enable efficient exploration and planning. result First computationally efficient algorithms for online reinforcement learning with low coverability.
New algorithm for multi-agent reinforcement learning with attention mechanism.
problem Challenges in training decentralized policies in multi-agent settings.
method Actor-attention-critic algorithm with centrally computed critics and attention mechanism.
result More effective and scalable learning in complex multi-agent environments.
The paper analyzes RLVR's training dynamics, proving convergence depends on aligning update direction with Gradient Gap.
problem Understanding why RLVR works and its limitations.
method Analysis of RLVR's training process at trajectory and token levels, introducing Gradient Gap.
result Convergence depends on aligning update direction with Gradient Gap, with a sharp step-size threshold.
Proposes a framework for energy-efficient AIGC workload scheduling in cloud data centers.
problem Challenges of scheduling AIGC workloads for energy efficiency and quality control.
method Joint energy management and coordinated AIGC workload scheduling framework with diffusion model-aided reward shaping.
result Effective learning of scheduling policies under sparse environmental feedback.
EGFs use ergodicity to simplify generative flows for easier training and imitation learning.
problem Challenges in training generative flows, especially in continuous settings and for imitation learning.
method EGFs leverage ergodicity to build simple flows with universality guarantees and tractable FM loss. They introduce a KL-weakFM loss for IL training without a separate reward model.
result EGFs simplify generative flow training and enable effective imitation learning.
The paper tackles combinatorial pure exploration with various feedback structures and proposes efficient algorithms.
problem Identifying the optimal action in a combinatorial space with limited feedback and nonlinear rewards.
method Designs polynomial-time adaptive algorithms for CPE-BL and CPE-PL, providing sample complexity analyses.
result The proposed algorithms achieve sample complexity close to lower bounds and outperform existing methods.
Uses RL to discover causal DAGs from data.
problem Discovering causal structure among variables.
method Reinforcement Learning to search for DAGs.
result Improved search ability and flexible scoring.
Thompson Sampling reduces regret in non-episodic restless bandits.
problem Learning optimal policies in time-varying reward systems.
method Variant of Thompson Sampling applied to infinite horizon restless bandits.
result Sub-linear regret bound of O ( T log T ) O(\sqrt{T}\log T) O ( T log T ) proved for non-episodic case. Generalized dual discriminator GANs improve upon traditional GANs by using two discriminators and a flexible loss function.
problem Mode collapse in GANs.
method Introducing dual discriminator α α α -GANs and extending the approach to arbitrary functions. result The approach reduces the optimization problem to a linear combination of an f f f -divergence and a reverse f f f -divergence. RLMH improves adaptive MCMC by optimizing contrastive divergence reward.
problem Tuning MCMC samplers is challenging and time-consuming.
method Formulated Metropolis-Hastings as a Markov decision process and used RL to adaptively tune it.
result A novel reward function based on contrastive divergence outperforms existing ones.
Paper proposes a deep RL approach for traffic signal control balancing efficiency and equity.
problem Inefficient and inflexible traffic signal controllers.
method Deep reinforcement learning with a novel reward function combining efficiency and equity.
result The proposed algorithm achieves state-of-the-art performance on various traffic scenarios.
New algorithm DUSA minimizes regret in structured bandits with structural information.
problem Optimal decision-making under uncertainty with structural reward information.
method DUSA algorithm exploiting convex structural information.
result Regret matches information-theoretic lower bound up to a constant factor.
RL framework optimizes trading costs in noisy markets.
problem Optimal execution and placement in noisy markets.
method Dual-window Denoise PPO RL network, imitation learning, comprehensive market features, flexible action formulation.
result RL agents outperformed TWAP strategy in execution cost.
MVPI framework optimizes risk in reinforcement learning, improving performance in robot simulations.
problem Optimizing risk in reinforcement learning control problems.
method Mean-Variance Policy Iteration (MVPI) framework for risk-averse control in MDPs.
result Risk-averse TD3 outperforms previous methods in robot simulation tasks.
Proposes a new semi-parametric framework for batched bandits with covariates.
problem Sequential decision-making with batched feedback and contextual information.
method Batched single-Index Dynamic binning and Successive arm elimination (BIDS) using single-index regression.
result Achieves minimax-optimal rates for nonparametric batched bandits.
New approach to meta-learning with variational Bayes for unlabeled data.
problem Improving machine learning systems' adaptability to small data.
method Generative meta-learning using AEVB and mean-field variational Bayes.
result Relevant VB updates do not depend on generative neural networks for certain models.
A new framework for offline RL improves policy flexibility and regularity.
problem Lack of environmental interactions in offline RL leads to poor policy performance.
method Proposes a behavior-regularized implicit policy framework with modified policy-matching methods.
result The framework improves policy effectiveness and robustness beyond static datasets.
New algorithms identify best arm with less pulls, adapting to arm covariances.
problem Best arm identification under dependent and correlated arm distributions.
method Adaptive algorithms estimating arm covariances to minimize pulls.
result Substantial improvement in best arm identification over standard setting.
This paper introduces a package for reinforcement learning in R.
problem Lack of a package for reinforcement learning in R.
method Developed a new package called ReinforcementLearning for R.
result Demonstrated the use of the ReinforcementLearning package in various problems.
Animals (especially humans) have an amazing ability to learn new tasks quickly, and switch between them flexibly. How brains support this ability is largely unknown, both neuroscientifically and algorithmically. One reasonable supposition is that modules drawing on an underlying general-purpose sensory representation a…
HRL improves open-domain dialog models by optimizing long-term conversational goals.
problem Challenges in open-domain dialog generation, including repetitive outputs, difficulty tracking conversational goals, and inappropriate text.
method Proposes VHRL, a hierarchical reinforcement learning approach using policy gradients to tune utterance-level embeddings of a variational sequence model.
result Significant improvements in human evaluation and automatic metrics over state-of-the-art dialog models.