arXiv research

Develops accelerated fixed-point methods with delayed oracles for scientific computing.

Study on optimal rates for sequential probability assignment using smoothed analysis.

Develops new oracle inequalities for Gaussian ranking estimators.

Crowdsourcing is an effective tool for human-powered computation on many tasks challenging for computers. In this paper, we provide finite-sample exponential bounds on the error rate (in probability and in expectation) of hyperplane binary labeling rules under the Dawid-Skene crowdsourcing model. The bounds can be appl…

Through the direct study of the analysis estimator we derive oracle inequalities with fast and slow rates by adapting the arguments involving projections by Dalalyan, Hebiri and Lederer (2017). We then extend the theory to the square root analysis estimator. Finally, we focus on (square root) total variation regularize…

The paper studies the benefits of curriculum learning in linear regression tasks.

New research shows fixed-budget best-arm identification cannot match static oracle performance.

New algorithm reduces contextual bandits to efficient regression.

Oracle inequality for sparse neural nets adapts to unknown structure.

This paper investigates tradeoffs among optimization errors, statistical rates of convergence and the effect of heavy-tailed errors for high-dimensional robust regression with nonconvex regularization. When the additive errors in linear models have only bounded second moment, we show that iteratively reweighted $\ell_1…

The aim of this paper is to provide some theoretical understanding of quasi-Bayesian aggregation methods non-negative matrix factorization. We derive an oracle inequality for an aggregated estimator. This result holds for a very general class of prior distributions and shows how the prior affects the rate of convergenc…

New oracles improve stochastic optimization with noisy or biased measurements.

The paper analyzes the efficiency of gradient estimation methods in noisy function evaluations.

Novel oracle-type inequality for logistic loss in DNNs achieves sharp convergence rates.

Crowdsourcing has become an effective and popular tool for human-powered computation to label large datasets. Since the workers can be unreliable, it is common in crowdsourcing to assign multiple workers to one task, and to aggregate the labels in order to obtain results of high quality. In this paper, we provide finit…

The effect of errors in variables in quantization is investigated. We prove general exact and non-exact oracle inequalities with fast rates for an empirical minimization based on a noisy sample $Z_i=X_i+ε_i,i=1,\ldots,n$, where $X_i$ are i.i.d. with density $f$ and $ε_i$ are i.i.d. with density $η$. These rates depend …

New method improves CATE model selection with optimal regret rates.

Paper tackles dynamic pricing in a geometrically decaying environment, achieving better occupancy with lower rates.

We present a unified framework for low-rank matrix estimation with nonconvex penalties. We first prove that the proposed estimator attains a faster statistical rate than the traditional low-rank matrix estimator with nuclear norm penalty. Moreover, we rigorously show that under a certain condition on the magnitude of t…

Paper proposes estimators for sparse PCA with oracle property.

Minimizing a function over an intersection of convex sets is an important task in optimization that is often much more challenging than minimizing it over each individual constraint set. While traditional methods such as Frank-Wolfe (FW) or proximal gradient descent assume access to a linear or quadratic oracle on the …

Improved Frank-Wolfe algorithm for constrained convex optimization with nearest extreme point oracle.

We provide tight upper and lower bounds on the complexity of minimizing the average of $m$ convex functions using gradient and prox oracles of the component functions. We show a significant gap between the complexity of deterministic vs randomized optimization. For smooth functions, we show that accelerated gradient de…

Nonparametric empirical Bayes denoising on Riemannian manifolds

The paper improves count data regression models for overdispersed data.

We give nearly matching upper and lower bounds on the oracle complexity of finding $ε$-stationary points ($\| \nabla F(x) \| \leqε$) in stochastic convex optimization. We jointly analyze the oracle complexity in both the local stochastic oracle model and the global oracle (or, statistical learning) model. This allows u…

A discrete diffusion model learns denoising, scoring, and bridging in different coordinates.

New algorithm optimizes convex functions with noisy evaluations in one dimension.

We consider the problem of learning high-dimensional Gaussian graphical models. The graphical lasso is one of the most popular methods for estimating Gaussian graphical models. However, it does not achieve the oracle rate of convergence. In this paper, we propose the graphical nonconvex optimization for optimal estimat…

Improved GANs estimate convergence rate for density estimation.

Study on costs of manipulating AMM-based price oracles.

Paper introduces SGD for nonparametric additive models with optimal risk.

Paper proposes a new method to optimize deep neural networks with sparse regularization.

In this work we introduce a conditional accelerated lazy stochastic gradient descent algorithm with optimal number of calls to a stochastic first-order oracle and convergence rate $O\left(\frac{1}{\varepsilon^2}\right)$ improving over the projection-free, Online Frank-Wolfe based stochastic gradient descent of Hazan an…

Proposes RDIV for IV estimation avoiding limitations of existing methods.

Minimax PAC bounds for learning in exogenous contextual MDPs

A new method for faster optimization of noisy functions.

New algorithm achieves small-loss bounds in online learning with improved rates.

This paper investigates the supervised learning problem with observations drawn from certain general stationary stochastic processes. Here by \emph{general}, we mean that many stationary stochastic processes can be included. We show that when the stochastic processes satisfy a generalized Bernstein-type inequality, a u…

This paper establishes non-asymptotic oracle inequalities for the prediction error and estimation accuracy of the LASSO in stationary vector autoregressive models. These inequalities are used to establish consistency of the LASSO even when the number of parameters is of a much larger order of magnitude than the sample …

The paper analyzes sparse high-dimensional linear regression with random design and unknown error variance, providing adaptiveness and concentration rates.

New algorithms ensure reproducibility and optimal convergence in convex optimization.

Two important goals of high-dimensional modeling are prediction and variable selection. In this article, we consider regularization with combined $L_1$ and concave penalties, and study the sampling properties of the global optimum of the suggested method in ultra-high dimensional settings. The $L_1$-penalty provides th…

High-dimensional data analysis has motivated a spectrum of regularization methods for variable selection and sparse modeling, with two popular classes of convex ones and concave ones. A long debate has been on whether one class dominates the other, an important question both in theory and to practitioners. In this pape…

This paper analyzes and compares different Automated Market Maker mechanisms.

The paper analyzes prediction error in nonstationary settings using weighted risk minimization.

SGD's performance improves with critical batch size, minimizing SFO complexity.

Local LMO optimizes constrained problems using local linear minimization.