arXiv research

A new multi-objective RL framework improves intrinsic exploration performance.

We propose a framework based on distributional reinforcement learning and recent attempts to combine Bayesian parameter updates with deep reinforcement learning. We show that our proposed framework conceptually unifies multiple previous methods in exploration. We also derive a practical algorithm that achieves efficien…

Interactive user interfaces need to continuously evolve based on the interactions that a user has (or does not have) with the system. This may require constant exploration of various options that the system may have for the user and obtaining signals of user preferences on those. However, such an exploration, especiall…

Unified framework for randomized exploration in cooperative MARL.

Maximizes Rényi entropy for efficient exploration in reward-free RL.

The outcome of the explorative data analysis (EDA) phase is vital for successful data analysis. EDA is more effective when the user interacts with the system used to carry out the exploration. In the recently proposed paradigm of iterative data mining the user controls the exploration by inputting knowledge in the form…

New framework uses unsupervised learning for efficient exploration in RL.

Paper optimizes industrial refrigeration using adaptive exploration.

DiCE uses diverse agents to explore and learn, avoiding local minima.

The area of constrained clustering has been extensively explored by researchers and used by practitioners. Constrained clustering formulations exist for popular algorithms such as k-means, mixture models, and spectral clustering but have several limitations. A fundamental strength of deep learning is its flexibility, a…

The paper explores sampling problems and shows minimal exploration is needed.

New $L_1$-Coverage objective simplifies exploration in reinforcement learning.

AutoBayes automates Bayesian graph exploration for robust machine learning.

This work introduces CAET, an algorithm for cost-aware pairwise pure exploration.

We describe a novel extension of soft actor-critics for hierarchical Deep Q-Networks (HDQN) architectures using mutual information metric. The proposed extension provides a suitable framework for encouraging explorations in such hierarchical networks. A natural utilization of this framework is an adversarial setting, w…

Framework uses dropout to efficiently explore Rashomon set for multiplicity estimation.

A new MARL framework for community-based cooperation with transfer and active exploration.

Improved exploration in RL with latent state marginalization.

A framework for auto-tuning hyper-parameters in contextual bandit algorithms.

Study of repeated principal-agent bandit game with self-interested and exploratory learning agents.

Curriculum learning in reinforcement learning is used to shape exploration by presenting the agent with increasingly complex tasks. The idea of curriculum learning has been largely applied in both animal training and pedagogy. In reinforcement learning, all previous task sequencing methods have shaped exploration with …

HyperAgent improves RL exploration in large-scale problems.

Efficient explorative data analysis systems must take into account both what a user knows and wants to know. This paper proposes a principled framework for interactive visual exploration of relations in data, through views most informative given the user's current knowledge and objectives. The user can input pre-existi…

Study uses RL to optimize investment with financial constraints, showing exploration benefits.

Optimal hedging framework with variational preferences under convex risk measures.

Exploration is widely regarded as one of the most challenging aspects of reinforcement learning (RL), with many naive approaches succumbing to exponential sample complexity. To isolate the challenges of exploration, we propose a new "reward-free RL" framework. In the exploration phase, the agent first collects trajecto…

Exploration in multi-task reinforcement learning is critical in training agents to deduce the underlying MDP. Many of the existing exploration frameworks such as $E^3$, $R_{max}$, Thompson sampling assume a single stationary MDP and are not suitable for system identification in the multi-task setting. We present a nove…

Control of non-episodic, finite-horizon dynamical systems with uncertain dynamics poses a tough and elementary case of the exploration-exploitation trade-off. Bayesian reinforcement learning, reasoning about the effect of actions and future observations, offers a principled solution, but is intractable. We review, then…

A decentralized routing framework for lunar exploration robots.

Study how untrained policies explore in RL environments.

Proposes an amortized variational framework for Deep Q Networks.

Algorithm achieves optimal pricing with minimal exploration for dynamic markets.

Safe exploration in RF-RL doesn't increase sample complexity.

SAMBA improves safe reinforcement learning with active exploration metrics.

We propose a novel framework to identify sub-goals useful for exploration in sequential decision making tasks under partial observability. We utilize the variational intrinsic control framework (Gregor et.al., 2016) which maximizes empowerment -- the ability to reliably reach a diverse set of states and show how to ide…

Deep reinforcement learning (DRL) is a booming area of artificial intelligence. Many practical applications of DRL naturally involve more than one collaborative learners, making it important to study DRL in a multi-agent context. Previous research showed that effective learning in complex multi-agent systems demands fo…

Bayesian optimization learns DM preferences for multi-outcome experiments.

In autonomous vehicle (AV) control, allowing mistakes can be quite dangerous and costly in the real world. For this reason we investigate methods of training an AV without allowing the agent to explore and instead having a human explorer collect the data. Supervised learning has been explored for AV control, but it enc…

This paper gives a review and synthesis of methods of evaluating dimensionality reduction techniques. Particular attention is paid to rank-order neighborhood evaluation metrics. A framework is created for exploring dimensionality reduction quality through visualization. An associated toolkit is implemented in R. The to…

Research in reinforcement learning has produced algorithms for optimal decision making under uncertainty that fall within two main types. The first employs a Bayesian framework, where optimality improves with increased computational time. This is because the resulting planning task takes the form of a dynamic programmi…

We develop a coherent framework for integrative simultaneous analysis of the exploration-exploitation and model order selection trade-offs. We improve over our preceding results on the same subject (Seldin et al., 2011) by combining PAC-Bayesian analysis with Bernstein-type inequality for martingales. Such a combinatio…

Unified framework controls false discovery rate in bandit multiple testing.

In Interactive Machine Learning (IML), we iteratively make decisions and obtain noisy observations of an unknown function. While IML methods, e.g., Bayesian optimization and active learning, have been successful in applications, on real-world systems they must provably avoid unsafe decisions. To this end, safe IML algo…

AlphaSAGE mines diverse alphas via GFlowNets, overcoming RL issues.

Task-agnostic RL tackles exploration in MDPs with multiple tasks.

ACS is an interactive framework for model-free selection with guaranteed error control.

New technique improves imitation learning by preventing local minima and exploring states.

Non-negative Matrix Factorization (NMF) is a popular tool for data exploration. Bayesian NMF promises to also characterize uncertainty in the factorization. Unfortunately, current inference approaches such as MCMC mix slowly and tend to get stuck on single modes. We introduce a novel approach using rapidly-exploring ra…