arXiv research

Optimally explores dynamical systems with varying properties using context inference.

Algorithm achieves optimal pricing with minimal exploration for dynamic markets.

Paper explores asset pricing dynamics in Bachelier model.

Improved RL algorithm stabilizes unknown linear systems with polynomial regret.

FLEX optimizes exploration for nonlinear systems with minimal data.

ASE safely explores unknown MDPs with unknown dynamics, improving sample efficiency.

Proposes r2SGLD for efficient constrained exploration in non-convex learning.

Efficient exploration remains a challenging problem in reinforcement learning, especially for those tasks where rewards from environments are sparse. A commonly used approach for exploring such environments is to introduce some "intrinsic" reward. In this work, we focus on model uncertainty estimation as an intrinsic r…

Study task-guided exploration in linear dynamical systems, improving sample complexity.

BYOL-Explore learns to explore visually-rich environments by predicting world dynamics.

We introduce Parameterized Exploration (PE), a simple family of methods for model-based tuning of the exploration schedule in sequential decision problems. Unlike common heuristics for exploration, our method accounts for the time horizon of the decision problem as well as the agent's current state of knowledge of the …

Paper develops a dynamic Bayesian approach for active learning that optimizes exploration-exploitation balance.

New algorithm tackles nonstationary linear bandits with latent dynamics.

New method learns adaptive exploration strategies for dynamic tasks.

Greedy policy maximizes information in unknown linear systems.

AER dynamically adjusts entropy regularization for better LLM reinforcement learning.

All reinforcement learning algorithms must handle the trade-off between exploration and exploitation. Many state-of-the-art deep reinforcement learning methods use noise in the action selection, such as Gaussian noise in policy gradient methods or $ε$-greedy in Q-learning. While these methods are appealing due to their…

LNUCB-TA improves MAB performance by dynamically adjusting exploration rates and recognizing spatiotemporal patterns.

Batch Thompson Sampling reduces exploration-exploitation trade-off in online decision making.

RECODE uses clustering and embedding to track state visitation counts in RL.

Efficient exploration is a long-standing problem in sensorimotor learning. Major advances have been demonstrated in noise-free, non-stochastic domains such as video games and simulation. However, most of these formulations either get stuck in environments with stochastic dynamics or are too inefficient to be scalable t…

Study local exploration on dynamic graphs with time-varying edges.

Optimal control in latent factor models uses Tsallis entropy for exploration.

To maximize its success, an AGI typically needs to explore its initially unknown world. Is there an optimal way of doing so? Here we derive an affirmative answer for a broad class of environments.

We present an adversarial active exploration for inverse dynamics model learning, a simple yet effective learning scheme that incentivizes exploration in an environment without any human intervention. Our framework consists of a deep reinforcement learning (DRL) agent and an inverse dynamics model contesting with each …

Safe learning of stochastic dynamics with safety constraints.

Study explores algorithmic collusion in repeated games using various learning dynamics.

Interpolates between SPG and NeuRD with Capped Implicit Exploration.

Curriculum learning speeds up agent learning in Minecraft, a complex visual domain.

Paper explores foundation models for dynamical systems using synthetic data.

Using reinforcement learning to learn control policies is a challenge when the task is complex with potentially long horizons. Ensuring adequate but safe exploration is also crucial for controlling physical systems. In this paper, we use temporal logic to facilitate specification and learning of complex tasks. We combi…

Transformer learns to infer partial MDPs for efficient in-context adaptation and exploration.

New algorithms reduce dynamic regret in non-stationary RL environments.

Exploration in environments with continuous control and sparse rewards remains a key challenge in reinforcement learning (RL). Recently, surprise has been used as an intrinsic reward that encourages systematic and efficient exploration. We introduce a new definition of surprise and its RL implementation named Variation…

D2D converts CLDs into SDMs to explore leverage points under uncertainty.

Control of non-episodic, finite-horizon dynamical systems with uncertain dynamics poses a tough and elementary case of the exploration-exploitation trade-off. Bayesian reinforcement learning, reasoning about the effect of actions and future observations, offers a principled solution, but is intractable. We review, then…

The paper explores the geometry of holomorphic flows and orbits.

The paper uses LSM to solve complex monetary utility functions.

SPEDER extracts state-action abstraction from dynamics for reinforcement learning.

DiMS sampler explores neural network loss minima via dissipative dynamics.

Develops a method to plan exploration that learns strong policies with fewer samples.

In this paper, we explore the effectiveness of dynamic analysis techniques for identifying malware, using Hidden Markov Models (HMMs) and Profile Hidden Markov Models (PHMMs), both trained on sequences of API calls. We contrast our results to static analysis using HMMs trained on sequences of opcodes, and show that dyn…

Interprets how intrinsic motivation shapes behavior in RL agents.

Multivariate time series (MTS) forecasting is widely used in various domains, such as meteorology and traffic. Due to limitations on data collection, transmission, and storage, real-world MTS data usually contains missing values, making it infeasible to apply existing MTS forecasting models such as linear regression an…

Study on cryptocurrency market dynamics and correlations over time.

A given set of data-points in some feature space may be associated with a Schrodinger equation whose potential is determined by the data. This is known to lead to good clustering solutions. Here we extend this approach into a full-fledged dynamical scheme using a time-dependent Schrodinger equation. Moreover, we approx…

Free lunch from noise reveals linear spectral features for RL.

STCA discovers dynamic functional brain networks using spatial-temporal convolution and attention.