arXiv research

New algorithm finds near-optimal policies efficiently in zero-sum games.

Proposes efficient stochastic algorithms for optimizing NDCG with provable convergence guarantees.

While policy-based reinforcement learning (RL) achieves tremendous successes in practice, it is significantly less understood in theory, especially compared with value-based RL. In particular, it remains elusive how to design a provably efficient policy optimization algorithm that incorporates exploration. To bridge su…

Topic models provide a useful method for dimensionality reduction and exploratory data analysis in large text corpora. Most approaches to topic model inference have been based on a maximum likelihood objective. Efficient algorithms exist that approximate this objective, but they have no provable guarantees. Recently, a…

Extended analysis of Q-learning's efficiency, matching optimal regret.

This thesis improves practical reinforcement learning methods with robustness, scalability, and efficiency.

Efficient RL algorithm for multinomial logistic MDPs with provable guarantees.

We study the design of computationally efficient algorithms with provable guarantees, that are robust to adversarial (test time) perturbations. While there has been an proliferation of recent work on this topic due to its connections to test time robustness of deep networks, there is limited theoretical understanding o…

Efficient poisoning attack converges to any target classifier with provable convergence.

We study Imitation Learning (IL) from Observations alone (ILFO) in large-scale MDPs. While most IL algorithms rely on an expert to directly provide actions to the learner, in this setting the expert only supplies sequences of observations. We design a new model-free algorithm for ILFO, Forward Adversarial Imitation Lea…

Proposes a method for obtaining interval bounds in off-policy evaluation.

We develop efficient algorithms to estimate the stability of Ordinary Least Squares regression results.

Linear dynamical systems are a fundamental and powerful parametric model class. However, identifying the parameters of a linear dynamical system is a venerable task, permitting provably efficient solutions only in special cases. This work shows that the eigenspectrum of unknown linear dynamics can be identified without…

Paper analyzes distributional reinforcement learning with value function approximation, introducing Bellman unbiasedness and a new algorithm.

We present a provable, sampling-based approach for generating compact Convolutional Neural Networks (CNNs) by identifying and removing redundant filters from an over-parameterized network. Our algorithm uses a small batch of input data points to assign a saliency score to each filter and constructs an importance sampli…

Modern Reinforcement Learning (RL) is commonly applied to practical problems with an enormous number of states, where function approximation must be deployed to approximate either the value function or the policy. The introduction of function approximation raises a fundamental set of challenges involving computational …

We present an algorithm, HOMER, for exploration and reinforcement learning in rich observation environments that are summarizable by an unknown latent state space. The algorithm interleaves representation learning to identify a new notion of kinematic state abstraction with strategic exploration to reach new states usi…

Generalized Linear Models (GLMs) and Single Index Models (SIMs) provide powerful generalizations of linear regression, where the target variable is assumed to be a (possibly unknown) 1-dimensional function of a linear predictor. In general, these problems entail non-convex estimation procedures, and, in practice, itera…

Paper introduces SPADE method to protect classifiers from OOD and adversarial samples.

Efficient method for constrained optimization under partial observations with provable convergence.

New framework uses unsupervised learning for efficient exploration in RL.

Proposes Optimistic Pessimistically Initialised Q-Learning (OPIQ) for better exploration in RL.

Proves efficient learning of hierarchical structure in meta-reinforcement learning.

POLICE enforces linear constraints on deep neural networks efficiently.

Improves neural network verification by merging abstract domains and Lagrangian methods.

$Q$-learning with function approximation is one of the most popular methods in reinforcement learning. Though the idea of using function approximation was proposed at least 60 years ago, even in the simplest setup, i.e, approximating $Q$-functions with linear functions, it is still an open problem on how to design a pr…

Novel algorithm accelerates PnP methods for image deblurring and super-resolution.

Matrix completion, where we wish to recover a low rank matrix by observing a few entries from it, is a widely studied problem in both theory and practice with wide applications. Most of the provable algorithms so far on this problem have been restricted to the offline setting where they provide an estimate of the unkno…

We introduce a pruning algorithm that provably sparsifies the parameters of a trained model in a way that approximately preserves the model's predictive accuracy. Our algorithm uses a small batch of input points to construct a data-informed importance sampling distribution over the network's parameters, and adaptively …

HySRL improves RL sample efficiency with shifted-dynamics data.

Suppose an agent is in a (possibly unknown) Markov Decision Process in the absence of a reward signal, what might we hope that an agent can efficiently learn to do? This work studies a broad class of objectives that are defined solely as functions of the state-visitation frequencies that are induced by how the agent be…

New algorithm reduces neural net error in contextual bandits.

The paper provides fast, sample-efficient algorithms for learning common features in multi-task linear regression.

We consider distributed on-device learning with limited communication and security requirements. We propose a new robust distributed optimization algorithm with efficient communication and attack tolerance. The proposed algorithm has provable convergence and robustness under non-IID settings. Empirical results show tha…

High sensitivity of neural networks against malicious perturbations on inputs causes security concerns. To take a steady step towards robust classifiers, we aim to create neural network models provably defended from perturbations. Prior certification work requires strong assumptions on network structures and massive co…

We develop necessary and sufficient conditions and a novel provably consistent and efficient algorithm for discovering topics (latent factors) from observations (documents) that are realized from a probabilistic mixture of shared latent factors that have certain properties. Our focus is on the class of topic models in …

New framework explains how larger pre-trained models reduce downstream learning sample complexity.

ASE safely explores unknown MDPs with unknown dynamics, improving sample efficiency.

The problem of adversarial robustness has been studied extensively for neural networks. However, for boosted decision trees and decision stumps there are almost no results, even though they are widely used in practice (e.g. XGBoost) due to their accuracy, interpretability, and efficiency. We show in this paper that for…

We prove a lower bound for feature dimension in linear MDPs and propose a novel dynamics aggregation framework.

Efficient RL in partially observable risk-sensitive environments with hindsight observations.

Layer-wise preconditioning methods improve neural network optimization and feature learning.

New method for robust prediction valid under non-exchangeable data.

Independent component analysis (ICA) is the problem of efficiently recovering a matrix $A \in \mathbb{R}^{n\times n}$ from i.i.d. observations of $X=AS$ where $S \in \mathbb{R}^n$ is a random vector with mutually independent coordinates. This problem has been intensively studied, but all existing efficient algorithms w…

This paper investigates multi-task reinforcement learning in non-Markovian decision making, showing benefits in sample efficiency.

OKRidge solves sparse ridge regression problems for nonlinear systems.

Machine-learning models for security-critical applications such as bot, malware, or spam detection, operate in constrained discrete domains. These applications would benefit from having provable guarantees against adversarial examples. The existing literature on provable adversarial robustness of models, however, exclu…

New method speeds up learning of complex dynamical systems.