arXiv research

Papers learn from data to make decisions without interacting, improving on previous methods.

Polynomial-time RL algorithm for constant actions under linear Bellman completeness.

New algorithm learns optimal policies without explicit rewards.

New algorithms accelerate value function convergence in MDPs.

Study tight offline learning bounds for linear MDPs using variance information.

New algorithm reduces reinforcement learning regret to sqrt(T) without strong dynamics assumptions.

In this paper we consider a problem of searching a space of predictive models for a given training data set. We propose an iterative procedure for deriving a sequence of improving models and a corresponding sequence of sets of non-linear features on the original input space. After a finite number of iterations N, the n…

FVI method calculates bicausal OT with neural networks, outperforming other methods.

VA-OPE improves OPE by incorporating variance information, achieving tighter error bounds.

New algorithms solve robust MDPs efficiently, significantly faster than existing methods.

Paper studies offline RL with linear approx, focusing on inherent Bellman error.

Develops fast approximations for conditional Shapley values in linear and polynomial models.

New method for optimistic planning in MDPs using regularization.

New algorithm reduces regret in linear mixture SSPs without cost bounds.

The paper sharpens the analysis of sketch-and-project methods using randomized singular value decomposition.

In this paper we study several classes of stochastic optimization algorithms enriched with heavy ball momentum. Among the methods studied are: stochastic gradient descent, stochastic Newton, stochastic proximal point and stochastic dual subspace ascent. This is the first time momentum variants of several of these metho…

Improved convergence rates for saddle-point optimization algorithms.

Subset selection in multiple linear regression aims to choose a subset of candidate explanatory variables that tradeoff fitting error (explanatory power) and model complexity (number of variables selected). We build mathematical programming models for regression subset selection based on mean square and absolute errors…

Following the lines of a celebrated result by R. Bott (Comm. Pure Appl. Math. 9, 1956) we study the Morse index of the iterated of a closed geodesic in stationary Lorentzian manifolds, or, more generally, of a closed Lorentzian geodesic that admits a timelike periodic Jacobi field. Given one such closed geodesic $γ$, w…

Value iteration is a fixed point iteration technique utilized to obtain the optimal value function and policy in a discounted reward Markov Decision Process (MDP). Here, a contraction operator is constructed and applied repeatedly to arrive at the optimal solution. Value iteration is a first order method and therefore …

SCOPE iteratively optimizes sparsity-constrained problems without tuning hyperparameters.

KL regularization helps RL algorithms by implicitly averaging q-values.

Alternative proof and extension of curvature estimates for minimal immersions.

Graph neural network executes value iteration for flexible environments.

Improved API to achieve optimal error bound and query complexity in local planning.

A new method learns robust policies from offline data with latent structures.

Optimal unimodal fitting for linear loss functions in a sequential, efficient manner.

Extends linear MDP to handle nonlinear rewards.

SIM-Shapley improves SV approximation efficiency and stability.

ReLU networks trained with MILPs match deep learning accuracy.

We consider solving the $\ell_1$-regularized least-squares ($\ell_1$-LS) problem in the context of sparse recovery, for applications such as compressed sensing. The standard proximal gradient method, also known as iterative soft-thresholding when applied to this problem, has low computational cost per iteration but a r…

High dimensional sparse learning has imposed a great computational challenge to large scale data analysis. In this paper, we are interested in a broad class of sparse learning approaches formulated as linear programs parametrized by a {\em regularization factor}, and solve them by the parametric simplex method (PSM). O…

Robust reinforcement learning aims to produce policies that have strong guarantees even in the face of environments/transition models whose parameters have strong uncertainty. Existing work uses value-based methods and the usual primitive action setting. In this paper, we propose robust methods for learning temporally …

XLVINs improve data efficiency in implicit planning by leveraging latent space.

We present the Parallel, Forward-Backward with Pruning (PFBP) algorithm for feature selection (FS) in Big Data settings (high dimensionality and/or sample size). To tackle the challenges of Big Data FS PFBP partitions the data matrix both in terms of rows (samples, training examples) as well as columns (features). By e…

Often in real-world datasets, especially in high dimensional data, some feature values are missing. Since most data analysis and statistical methods do not handle gracefully missing values, the first step in the analysis requires the imputation of missing values. Indeed, there has been a long standing interest in metho…

We consider the networked multi-agent reinforcement learning (MARL) problem in a fully decentralized setting, where agents learn to coordinate to achieve the joint success. This problem is widely encountered in many areas including traffic control, distributed control, and smart grids. We assume that the reward functio…

Paper analyzes iterates in high-dimensional linear models and proposes estimators for their generalization error.

We study reinforcement learning under model misspecification, where we do not have access to the true environment but only to a reasonably close approximation to it. We address this problem by extending the framework of robust MDPs to the model-free Reinforcement Learning setting, where we do not have access to the mod…

Last SGD iterate bounds for overparameterized linear regression.

New method speeds up distributed linear regression.

Approximate dynamic programming algorithms, such as approximate value iteration, have been successfully applied to many complex reinforcement learning tasks, and a better approximate dynamic programming algorithm is expected to further extend the applicability of reinforcement learning to various tasks. In this paper w…

We give improved algorithms for the $\ell_{p}$-regression problem, $\min_{x} \|x\|_{p}$ such that $A x=b,$ for all $p \in (1,2) \cup (2,\infty).$ Our algorithms obtain a high accuracy solution in $\tilde{O}_{p}(m^{\frac{|p-2|}{2p + |p-2|}}) \le \tilde{O}_{p}(m^{\frac{1}{3}})$ iterations, where each iteration requires s…

Modern Reinforcement Learning (RL) is commonly applied to practical problems with an enormous number of states, where function approximation must be deployed to approximate either the value function or the policy. The introduction of function approximation raises a fundamental set of challenges involving computational …

New algorithms compute Volterra signature efficiently for time series analysis.

Study optimal and instance-dependent guarantees for solving linear equations with Markovian data.

Financial portfolios are often optimized for maximum profit while subject to a constraint formulated in terms of the Conditional Value-at-Risk (CVaR). This amounts to solving a linear problem. However, in its original formulation this linear problem has a very large number of linear constraints, too many to be enforced…

This paper presents a way of solving Markov Decision Processes that combines state abstraction and temporal abstraction. Specifically, we combine state aggregation with the options framework and demonstrate that they work well together and indeed it is only after one combines the two that the full benefit of each is re…