arXiv research

Bayesian method estimates dynamics from near-optimal trajectories.

New algorithm clusters trajectories from multiple Markov chains with near-optimal error.

Estimates missing mass in Markovian sequences with linear runtime and near-optimal risk.

Standard reinforcement learning methods aim to master one way of solving a task whereas there may exist multiple near-optimal policies. Being able to identify this collection of near-optimal policies can allow a domain expert to efficiently explore the space of reasonable solutions. Unfortunately, existing approaches t…

Many prediction domains, such as ad placement, recommendation, trajectory prediction, and document summarization, require predicting a set or list of options. Such lists are often evaluated using submodular reward functions that measure both quality and diversity. We propose a simple, efficient, and provably near-optim…

New algorithm infers reward function from agent's learning trajectories.

Recent years have seen a flurry of activities in designing provably efficient nonconvex procedures for solving statistical estimation problems. Due to the highly nonconvex nature of the empirical loss, state-of-the-art procedures often require proper regularization (e.g. trimming, regularized cost, projection) in order…

Reinforcement learning offers the promise of automating the acquisition of complex behavioral skills. However, compared to commonly used and well-understood supervised learning methods, reinforcement learning algorithms can be brittle, difficult to use and tune, and sensitive to seemingly innocuous implementation decis…

Transformers learn to make decisions in new contexts from offline data.

Exploration is widely regarded as one of the most challenging aspects of reinforcement learning (RL), with many naive approaches succumbing to exponential sample complexity. To isolate the challenges of exploration, we propose a new "reward-free RL" framework. In the exploration phase, the agent first collects trajecto…

Study infers interaction kernels from multiple particle trajectories.

Algorithm learns linear systems from partial observations with near-optimal rate.

New algorithm reduces sample complexity for planning in MDPs.

Adaptive OMD reduces variance in learning optimal strategies for imperfect information games.

Task-agnostic RL tackles exploration in MDPs with multiple tasks.

$Q$-learning with function approximation is one of the most popular methods in reinforcement learning. Though the idea of using function approximation was proposed at least 60 years ago, even in the simplest setup, i.e, approximating $Q$-functions with linear functions, it is still an open problem on how to design a pr…

New methods tackle robust reinforcement learning in sparse, corrupted data.

We study discrete time dynamical systems governed by the state equation $h_{t+1}=φ(Ah_t+Bu_t)$. Here $A,B$ are weight matrices, $φ$ is an activation function, and $u_t$ is the input data. This relation is the backbone of recurrent neural networks (e.g. LSTMs) which have broad applications in sequential learning tasks. …

Paper tackles non-stationary kernelized bandits with near-optimal algorithm.

Large learning rates improve neural network generalization, study shows.

A critical flaw of existing inverse reinforcement learning (IRL) methods is their inability to significantly outperform the demonstrator. This is because IRL typically seeks a reward function that makes the demonstrator appear near-optimal, rather than inferring the underlying intentions of the demonstrator that may ha…

Paper tackles RLHF with DCPPO method, proving near-optimal suboptimality.

We study the worst-case adaptive optimization problem with budget constraint that is useful for modeling various practical applications in artificial intelligence and machine learning. We investigate the near-optimality of greedy algorithms for this problem with both modular and non-modular cost functions. In both case…

Efficient streaming algorithms for robust statistics with near-optimal memory.

The paper offers simple, near-optimal algorithms for multi-group learning.

GP-MRO discovers robust mixed strategies for unknown objectives.

New algorithm reduces RL complexity with low switching costs.

We study the problem of regret minimization for distributed bandits learning, in which $M$ agents work collaboratively to minimize their total regret under the coordination of a central server. Our goal is to design communication protocols with near-optimal regret and little communication cost, which is measured by the…

Paper tackles clustering with ordinal comparisons, achieving near-optimal results.

Federated Q-learning achieves linear speedup with heterogeneity, improving sample complexity.

New algorithm detects changes quickly without knowing parameters, near optimally.

Efficiently find near-optimal medical treatments with less trial and error.

New DP algorithms achieve near-optimal regret bounds for online learning problems.

Algorithm extsc{Pedel} learns near-optimal policies efficiently on specific problems.

Improved algorithm for adaptive dueling bandits with near-optimal regret bound.

We study the problem of discriminative sub-trajectory mining. Given two groups of trajectories, the goal of this problem is to extract moving patterns in the form of sub-trajectories which are more similar to sub-trajectories of one group and less similar to those of the other. We propose a new method called Statistica…

A single policy suffices for near-optimal parallel exploration in RL.

Short proof for ideal polygons with near optimal orthogeodesic decomposition.

A new UCB algorithm for heavy-tailed bandits with near-optimal regret.

New algorithm reduces bias in trained models, near-optimal performance proven.

A new method finds diverse near-optimal portfolios using quality-diversity.

Paper uses deep imitation learning to predict aircraft trajectories accurately.

Near-optimal algorithms for predicting across multiple loss functions efficiently.

New algorithms minimize risk in MNL bandits, achieving near-optimal performance.

Ensembles of random-feature models can't outperform a single large model.

AE-LSVI identifies near-optimal policies in complex systems with minimal data.

Pattern ensembling fills in missing or inaccurate trajectory data.

New RL algorithm reduces deployment cost for linear function approximations.