arXiv research

New algorithms for private GLM estimation with minimax lower bounds.

Study minimax linear regression under quantile risk, improving existing bounds and providing new results.

Paper finds a lower bound for estimating low-rank matrices in logistic regression.

Develops high-probability minimax quantile bounds for statistical problems.

New research sets the minimax lower bound for KSD estimation at sqrt(n).

We prove non-asymptotic lower bounds on the expectation of the maximum of $d$ independent Gaussian variables and the expectation of the maximum of $d$ independent symmetric random walks. Both lower bounds recover the optimal leading constant in the limit. A simple application of the lower bound for random walks is an (…

Exact minimax risk derived for linear prediction with sample covariance analysis.

Score attack method provides a lower bound on privacy-constrained minimax risk.

We study the problem of switching-constrained online convex optimization (OCO), where the player has a limited number of opportunities to change her action. While the discrete analog of this online learning task has been studied extensively, previous work in the continuous setting has neither established the minimax ra…

GP-UCB performs suboptimally under certain conditions, as shown by a new regret lower bound.

Study finds optimal regret bound for multi-armed bandit problem with expert advice.

Study on statistical estimation over Gaussian MAC, comparing analog and digital schemes.

Dictionary learning is the problem of estimating the collection of atomic elements that provide a sparse representation of measured/collected signals or data. This paper finds fundamental limits on the sample complexity of estimating dictionaries for tensor data by proving a lower bound on the minimax risk. This lower …

Study minimax regret in sequential probability assignment with and without side information.

The study establishes minimax bounds for estimating operators from noisy samples.

Develops locally private methods for nonparametric contextual bandits.

Paper explores fair classification with bounded disparity using finite datasets.

New tensor model reduces GLM estimation error and sample complexity.

We consider a sequential learning problem with Gaussian payoffs and side information: after selecting an action $i$, the learner receives information about the payoff of every action $j$ in the form of Gaussian observations whose mean is the same as the mean payoff, but the variance depends on the pair $(i,j)$ (and may…

Transductive learning considers a training set of $m$ labeled samples and a test set of $u$ unlabeled samples, with the goal of best labeling that particular test set. Conversely, inductive learning considers a training set of $m$ labeled samples drawn iid from $P(X,Y)$, with the goal of best labeling any future sample…

Detecting a planted submatrix in random matrices with non-asymptotic methods.

The cost-sensitive classification problem plays a crucial role in mission-critical machine learning applications, and differs with traditional classification by taking the misclassification costs into consideration. Although being studied extensively in the literature, the fundamental limits of this problem are still n…

We study the linear contextual bandit problem with finite action sets. When the problem dimension is $d$, the time horizon is $T$, and there are $n \leq 2^{d/2}$ candidate actions per time period, we (1) show that the minimax expected regret is $Ω(\sqrt{dT (\log T) (\log n)})$ for every algorithm, and (2) introduce a V…

An important class of distance metrics proposed for training generative adversarial networks (GANs) is the integral probability metric (IPM), in which the neural net distance captures the practical GAN training via two neural networks. This paper investigates the minimax estimation problem of the neural net distance ba…

The paper optimizes distribution estimation with high probability in Kullback-Leibler divergence.

This paper improves the convergence rates of bilevel optimization algorithms.

In this paper, we consider low rank matrix estimation using either matrix-version Dantzig Selector $\hat{A}_λ^d$ or matrix-version LASSO estimator $\hat{A}_λ^L$. We consider sub-Gaussian measurements, $i.e.$, the measurements $X_1,\ldots,X_n\in\mathbb{R}^{m\times m}$ have $i.i.d.$ sub-Gaussian entries. Suppose $\textrm…

We consider the problem of learning a dictionary matrix from a number of observed signals, which are assumed to be generated via a linear model with a common underlying dictionary. In particular, we derive lower bounds on the minimum achievable worst case mean squared error (MSE), regardless of computational complexity…

We consider the problem of accurately estimating the reliability of workers based on noisy labels they provide, which is a fundamental question in crowdsourcing. We propose a novel lower bound on the minimax estimation error which applies to any estimation procedure. We further propose Triangular Estimation (TE), an al…

New methods solve complex optimization problems without strong convexity assumptions.

In adaptive data analysis, the user makes a sequence of queries on the data, where at each step the choice of query may depend on the results in previous steps. The releases are often randomized in order to reduce overfitting for such adaptively chosen queries. In this paper, we propose a minimax framework for adaptive…

Study on efficient estimation of Gaussian mean with limited communication.

We consider the problem of dictionary learning under the assumption that the observed signals can be represented as sparse linear combinations of the columns of a single large dictionary matrix. In particular, we analyze the minimax risk of the dictionary learning problem which governs the mean squared error (MSE) perf…

Deep neural networks are optimal for dependent data using PAC-Bayes bounds.

GNA optimally identifies the best arm with small gaps.

MOTS improves Thompson sampling to match minimax bounds for bandit problems.

We consider in this paper the problem of noisy 1-bit matrix completion under a general non-uniform sampling distribution using the max-norm as a convex relaxation for the rank. A max-norm constrained maximum likelihood estimate is introduced and studied. The rate of convergence for the estimate is obtained. Information…

The paper achieves nearly optimal regret bounds for contextual multinomial logit bandits.

Neural networks minimize error with shallow ReLU models for function estimation.

Privacy-preserving data analysis is a rising challenge in contemporary statistics, as the privacy guarantees of statistical methods are often achieved at the expense of accuracy. In this paper, we investigate the tradeoff between statistical accuracy and privacy in mean estimation and linear regression, under both the …

Unified framework for lower bounds in interactive decision making.

Paper improves algorithms for convex-concave minimax optimization problems.

New combinatorial dimension VCL refines learning curve theory.

Study non-asymptotic estimation bounds for LTI models with Gaussian noise.

This paper studies the problem of inferring a global preference based on the partial rankings provided by many users over different subsets of items according to the Plackett-Luce model. A question of particular interest is how to optimally assign items to users for ranking and how many item assignments are needed to a…

Proposes online debiasing estimators for adaptive linear regression.

An algorithm learns from multiple models to match an oracle's risk.

If F is a family of mod 2 flat k-cycles in the unit n-ball, we lower bound the maximal volume of any cycle in F in terms of the homology class of F in the space of all cycles. We give examples to show that these lower bounds are fairly sharp.