arXiv research

Two entropy measures quantify suboptimal portfolio performance.

Novel IRL method identifies suboptimal medical decisions in ICU data.

New suboptimal algorithm for best subset selection in high-dimensional data.

The maximum a posteriori (MAP) configuration of binary variable models with submodular graph-structured energy functions can be found efficiently and exactly by graph cuts. Max-product belief propagation (MP) has been shown to be suboptimal on this class of energy functions by a canonical counterexample where MP conver…

The Lasso is suboptimal in sparse linear regression due to design matrix constraints.

Paper explores limits of imitation learning in MDPs, setting new suboptimality bounds.

New bounds on adaptivity cost in stochastic optimization.

Practical model building processes are often time-consuming because many different models must be trained and validated. In this paper, we introduce a novel algorithm that can be used for computing the lower and the upper bounds of model validation errors without actually training the model itself. A key idea behind ou…

Enhances reinforcement learning from sparse data.

New algorithm reduces suboptimality in imitation learning to nearly optimal levels.

Least Squares Estimators are suboptimal for 5D convex functions.

New study shows exponential lower bound for RL even with constant suboptimality gap.

Training a source model optimally for its own task is suboptimal for downstream transfer.

A new measure helps compute suboptimality in entropy-regularized methods.

Understanding the loss surface of neural networks is essential for the design of models with predictable performance and their success in applications. Experimental results suggest that sufficiently deep and wide neural networks are not negatively impacted by suboptimal local minima. Despite recent progress, the reason…

Posterior inference in directed graphical models is commonly done using a probabilistic encoder (a.k.a inference model) conditioned on the input. Often this inference model is trained jointly with the probabilistic decoder (a.k.a generator model). If probabilistic encoder encounters complexities during training (e.g. s…

IDT learns human preferences from uncertain decisions, even when humans are suboptimal.

Paper explores PG for MCR, finding suboptimal policies but providing bounds.

Negative momentum accelerates convergence in minimax games but at a suboptimal rate.

Inferring intent from observed behavior has been studied extensively within the frameworks of Bayesian inverse planning and inverse reinforcement learning. These methods infer a goal or reward function that best explains the actions of the observed agent, typically a human demonstrator. Another agent can use this infer…

In this paper, we propose and study opportunistic bandits - a new variant of bandits where the regret of pulling a suboptimal arm varies under different environmental conditions, such as network load or produce price. When the load/price is low, so is the cost/regret of pulling a suboptimal arm (e.g., trying a suboptim…

Interpolating estimators in nonparametric regression become suboptimal under adversarial attacks.

New algorithms tackle robust RL with linear models, revealing unique challenges.

Using a fast numerical technique, we investigate a large database of investor suboptimal non-exercise of short maturity American call options on dividend-paying stocks listed on the Dow Jones. The correct modelling of the discrete dividend is essential for a correct calculation of the early exercise boundary as confirm…

Myopic investors make suboptimal choices that benefit others, leading to market inefficiencies.

In many platforms, user arrivals exhibit a self-reinforcing behavior: future user arrivals are likely to have preferences similar to users who were satisfied in the past. In other words, arrivals exhibit positive externalities. We study multiarmed bandit (MAB) problems with positive externalities. We show that the self…

New algorithm reduces MDP regret by accounting for state suboptimality gaps and variance.

New method solves complex optimization problems faster.

Layer-wise preconditioning methods improve neural network optimization and feature learning.

We show that any smooth bi-Lipschitz $h$ can be represented exactly as a composition $h_m \circ ... \circ h_1$ of functions $h_1,...,h_m$ that are close to the identity in the sense that each $\left(h_i-\mathrm{Id}\right)$ is Lipschitz, and the Lipschitz constant decreases inversely with the number $m$ of functions com…

In this paper, we study the portfolio optimization problem with general utility functions and when the return and volatility of underlying asset are slowly varying. An asymptotic optimal strategy is provided within a specific class of admissible controls under this problem setup. Specifically, we first establish a rigo…

Let $\cF$ be a set of $M$ classification procedures with values in $[-1,1]$. Given a loss function, we want to construct a procedure which mimics at the best possible rate the best procedure in $\cF$. This fastest rate is called optimal rate of aggregation. Considering a continuous scale of loss functions with various …

Thompson Sampling fails to perform well in high dimensions.

Paper shows ERM's suboptimality due to bias, not variance.

MAPS and MAPS-SE learn from multiple suboptimal experts to improve policies efficiently.

Boltzmann exploration is a classic strategy for sequential decision-making under uncertainty, and is one of the most standard tools in Reinforcement Learning (RL). Despite its widespread use, there is virtually no theoretical understanding about the limitations or the actual benefits of this exploration scheme. Does it…

Paper studies offline RL with linear approx, focusing on inherent Bellman error.

The exploration mechanism used by a Deep Reinforcement Learning (RL) agent plays a key role in determining its sample efficiency. Thus, improving over random exploration is crucial to solve long-horizon tasks with sparse rewards. We propose to leverage an ensemble of partial solutions as teachers that guide the agent's…

There has been substantial research on sub-linear time approximate algorithms for Maximum Inner Product Search (MIPS). To achieve fast query time, state-of-the-art techniques require significant preprocessing, which can be a burden when the number of subsequent queries is not sufficiently large to amortize the cost. Fu…

New algorithm estimates edge density of random graphs robustly, achieving optimal breakdown point.

A new scaling law predicts optimal batch size for training models.

Curriculum learning has been successfully used in reinforcement learning to accelerate the learning process, through knowledge transfer between tasks of increasing complexity. Critical tasks, in which suboptimal exploratory actions must be minimized, can benefit from curriculum learning, and its ability to shape explor…

Two algorithms improve Federated RL in diverse environments.

We develop a robust RL algorithm for off-dynamics environments with improved suboptimality bounds and computational efficiency.

Improved multilevel scheme for value-at-risk computation.

A new algorithm reduces suboptimal arm selection in correlated bandits.

Imitation learning (IL) consists of a set of tools that leverage expert demonstrations to quickly learn policies. However, if the expert is suboptimal, IL can yield policies with inferior performance compared to reinforcement learning (RL). In this paper, we aim to provide an algorithm that combines the best aspects of…

This paper provides an analysis of the tradeoff between asymptotic bias (suboptimality with unlimited data) and overfitting (additional suboptimality due to limited data) in the context of reinforcement learning with partial observability. Our theoretical analysis formally characterizes that while potentially increasin…