arXiv research

The paper analyzes convergence in SGD with momentum and proposes a diagnostic test.

New theorem for generalized group sparsity improves consistency and convergence rates.

We study statistical risk minimization problems under a privacy model in which the data is kept confidential even from the learner. In this local privacy framework, we establish sharp upper and lower bounds on the convergence rates of statistical estimation procedures. As a consequence, we exhibit a precise tradeoff be…

Differential privacy is a cryptographically-motivated definition of privacy which has gained significant attention over the past few years. Differentially private solutions enforce privacy by adding random noise to a function computed over the data, and the challenge in designing such algorithms is to control the added…

The study provides statistical guarantees for Bayesian variational boosting.

We extend the Fourier cosine method to discrete probability distributions, achieving faster convergence rates.

Study problem-dependent rates in statistical learning theory, achieving optimal generalization error bounds.

Paper uses optimal transport-based statistics for change point detection.

Triangular flows ensure statistical consistency and fast rates in generative modeling.

This paper strengthens the central limit theorem for order statistics using relative entropy.

The rectified flow method is analyzed for its statistical properties.

Enhanced ROOT-SGD optimizes stochastic optimization with diminishing stepsizes.

Bayesian method improves neural net convergence for character recognition.

Study improves distributional regression evaluation with CRPS, finding optimal rates of convergence.

SGDM accelerates faster than SGD with large batch sizes and permits broader learning rates.

We provide theoretical analysis of the statistical and computational properties of penalized $M$-estimators that can be formulated as the solution to a possibly nonconvex optimization problem. Many important estimators fall in this category, including least squares regression with nonconvex regularization, generalized …

Random forests remain among the most popular off-the-shelf supervised learning algorithms. Despite their well-documented empirical success, however, until recently, few theoretical results were available to describe their performance and behavior. In this work we push beyond recent work on consistency and asymptotic no…

The paper studies kernel smoothing and mean shift for directional data, deriving convergence rates and mode estimation.

The paper improves OT map estimation rates without strict assumptions.

Many statistical $M$-estimators are based on convex optimization problems formed by the combination of a data-dependent loss function with a norm-based regularizer. We analyze the convergence rates of projected gradient and composite gradient methods for solving such problems, working within a high-dimensional framewor…

New neural networks converge faster, approximating smooth functions better.

Error bound conditions (EBC) are properties that characterize the growth of an objective function when a point is moved away from the optimal set. They have recently received increasing attention in the field of optimization for developing optimization algorithms with fast convergence. However, the studies of EBC in st…

Improved GANs estimate convergence rate for density estimation.

Paper establishes convergence rates for learning elliptic pseudo-differential operators.

Gradient descent achieves exact linear convergence rate for symmetric matrix completion.

In this paper, we investigate the statistical convergence rate of a Bayesian low-rank tensor estimator. Our problem setting is the regression problem where a tensor structure underlying the data is estimated. This problem setting occurs in many practical applications, such as collaborative filtering, multi-task learnin…

Despite the great empirical success of deep reinforcement learning, its theoretical foundation is less well understood. In this work, we make the first attempt to theoretically understand the deep Q-network (DQN) algorithm (Mnih et al., 2015) from both algorithmic and statistical perspectives. In specific, we focus on …

New concentration inequality for U-statistics of Markov chains.

Rank-statistic method approximates $f$-divergences without density-ratio estimation.

New algorithms improve convergence rates for non-log-concave sampling and log-partition estimation.

Paper analyzes deep neural networks with dependent data, establishing convergence rates and error bounds.

Proposes GBBHE for efficient large-scale regression.

This work analyzes the statistical properties of neural ODEs for distribution learning.

Study rates of convergence for approximate solutions to linear ill-posed problems in Hilbert scales.

We study the problem of estimating a nonparametric probability density under a large family of losses called Besov IPMs, which include, for example, $\mathcal{L}^p$ distances, total variation distance, and generalizations of both Wasserstein and Kolmogorov-Smirnov distances. For a wide variety of settings, we provide b…

Paper derives convergence rates and confidence intervals for LSA with Markovian noise.

We provide a general theory of the expectation-maximization (EM) algorithm for inferring high dimensional latent variable models. In particular, we make two contributions: (i) For parameter estimation, we propose a novel high dimensional EM algorithm which naturally incorporates sparsity structure into parameter estima…

This paper presents a class of new algorithms for distributed statistical estimation that exploit divide-and-conquer approach. We show that one of the key benefits of the divide-and-conquer strategy is robustness, an important characteristic for large distributed systems. We establish connections between performance of…

Many problems in machine learning and statistics involve nested expectations and thus do not permit conventional Monte Carlo (MC) estimation. For such problems, one must nest estimators, such that terms in an outer estimator themselves involve calculation of a separate, nested, estimation. We investigate the statistica…

Paper proposes an optimal framework for tensor estimation across various applications.

CD converges linearly for MCP/SCAD penalized least squares.

Study shows convergence rates for Cheeger cuts on data clouds.

Study guarantees convergence of mean shift mode estimation.

The paper develops a new algorithm for RBMs using dynamical mean-field theory.

In this paper, we examine a geometrical projection algorithm for statistical inference. The algorithm is based on Pythagorean relation and it is derivative-free as well as representation-free that is useful in nonparametric cases. We derive a bound of learning rate to guarantee local convergence. In special cases of m-…

Computational topology has recently known an important development toward data analysis, giving birth to the field of topological data analysis. Topological persistence, or persistent homology, appears as a fundamental tool in this field. In this paper, we study topological persistence in general metric spaces, with a …

Empirical Bayes rates via variational approximations and prior decomposition.

We propose a nonconvex estimator for joint multivariate regression and precision matrix estimation in the high dimensional regime, under sparsity constraints. A gradient descent algorithm with hard thresholding is developed to solve the nonconvex estimator, and it attains a linear rate of convergence to the true regres…