arXiv research

Proves equivalence of two types of Ricci curvature bounds.

The standard interpretation of importance-weighted autoencoders is that they maximize a tighter lower bound on the marginal likelihood than the standard evidence lower bound. We give an alternate interpretation of this procedure: that it optimizes the standard variational lower bound, but using a more complex distribut…

Optimal SQ bounds for learning binary product distributions and Ising models.

The paper establishes bounds on scalar curvature on asymptotically flat manifolds.

New lower bounds for sampling from log-concave distributions in higher dimensions.

Paper studies signal detection in noisy environments with limited communication.

New lower bounds for private covariance estimation of Gaussian distributions are proven.

New study on regret lower bounds for multi-agent multi-armed bandit problems.

Exact bounds derived for neural network outputs with noisy inputs.

We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L2 regularization: We introduce the margin-adapted dimension, which is a simple function of the second order statistics of the data distribution, and show distribution-specific upper and lower bounds on…

The paper sets lower bounds for sampling non-log-concave distributions using Fisher information.

New combinatorial dimension VCL refines learning curve theory.

In this work, we consider the sample complexity required for testing the monotonicity of distributions over partial orders. A distribution $p$ over a poset is monotone if, for any pair of domain elements $x$ and $y$ such that $x \preceq y$, $p(x) \leq p(y)$. To understand the sample complexity of this problem, we intro…

Mean-field variational inference is a method for approximate Bayesian posterior inference. It approximates a full posterior distribution with a factorized set of distributions by maximizing a lower bound on the marginal likelihood. This requires the ability to integrate a sum of terms in the log joint likelihood using …

The paper extends statistical estimation techniques under differential privacy.

Paper finds how many neurons are needed to approximate histogram distributions.

Lower bounds on MALA and HMC for well-conditioned distributions.

In this paper, we derive a useful lower bound for the Kullback-Leibler divergence (KL-divergence) based on the Hammersley-Chapman-Robbins bound (HCRB). The HCRB states that the variance of an estimator is bounded from below by the Chi-square divergence and the expectation value of the estimator. By using the relation b…

The study establishes SQ lower bounds for learning halfspaces and ReLUs under Gaussian marginals.

Study compares synthetic and distributional Ricci curvature bounds.

Whereas subriemannian geometry usually deals with smooth horizontal distributions, partially hyperbolic dynamical systems provide many examples of subriemannian geometries defined by non-smooth (namely, Hölder continuous) distributions. These distributions are of great significance for the behavior of the parent dynami…

Exact minimax risk derived for linear prediction with sample covariance analysis.

New convergence bounds for shuffling-based SGD methods in distributed learning.

Lower bounds show density estimation requires linear samples or query time.

We show how to compute lower bounds for the supremum Bayes error if the class-conditional distributions must satisfy moment constraints, where the supremum is with respect to the unknown class-conditional distributions. Our approach makes use of Curto and Fialkow's solutions for the truncated moment problem. The lower …

Paper establishes universal lower bounds and optimal rates for clustering sub-exponential mixture models.

Lower bounds on private estimation of Gaussian covariance matrices.

The Poisson model is frequently employed to describe count data, but in a Bayesian context it leads to an analytically intractable posterior probability distribution. In this work, we analyze a variational Gaussian approximation to the posterior distribution arising from the Poisson model with a Gaussian prior. This is…

We will study metric measure spaces $(X,d,m)$ beyond the scope of spaces with synthetic lower Ricci bounds. In particular, we introduce distribution-valued lower Ricci bounds BE$_1(κ,\infty)$ $\bullet$ for which we prove the equivalence with sharp gradient estimates, $\bullet$ the class of which will be preserved under…

The paper proves scalar curvature lower bounds along Ricci flow on compact manifolds.

A new method for multi-objective Bayesian optimization using entropy search and variational lower bound maximization.

Study minimax linear regression under quantile risk, improving existing bounds and providing new results.

New study shows non-interactive privacy model requires exponentially more data.

The study optimizes polynomial regression for learning under Gaussian distributions.

New SQ lower bound shows complexity nearly matches known upper bound for smoothed agnostic learning.

We present a simple noise-robust margin-based active learning algorithm to find homogeneous (passing the origin) linear separators and analyze its error convergence when labels are corrupted by noise. We show that when the imposed noise satisfies the Tsybakov low noise condition (Mammen, Tsybakov, and others 1999; Tsyb…

Paper finds a lower bound for estimating low-rank matrices in logistic regression.

We study distributed optimization algorithms for minimizing the average of convex functions. The applications include empirical risk minimization problems in statistical machine learning where the datasets are large and have to be stored on different machines. We design a distributed stochastic variance reduced gradien…

We study distributed estimation methods under communication constraints in a distributed version of the nonparametric random design regression model. We derive minimax lower bounds and exhibit methods that attain those bounds. Moreover, we show that adaptive estimation is possible in this setting.

New SQ lower bounds for NGCA without requiring chi-squared condition.

Sum-of-Squares lower bound shows NGCA requires more samples than known algorithms.

New bounds on learning from multiple distributions for VC classes.

Recently, there has been an increasing interest in designing distributed convex optimization algorithms under the setting where the data matrix is partitioned on features. Algorithms under this setting sometimes have many advantages over those under the setting where data is partitioned on samples, especially when the …

Improved bounds for estimating discrete distributions in KL divergence.

The paper develops a theory for identifying the best arm in non-parametric multi-armed bandits with a fixed budget.

In order to study the phenomenon in detail that income distribution follows Pareto law, we analyze the database of high income companies in Japan. We find a quantitative relation between the average capital of the companies and the Pareto index. The larger the average capital becomes, the smaller the Pareto index becom…

The paper improves lower bounds on deep neural network trajectories.

The study examines conditions for achieving a simple lower bound in estimating mean from samples.